My threads are not indexing in Google - what could I be doing wrong?

R

rosstqz

New member
Hi guys

I'm pulling my hair out over this one! I can't get my threads indexed by google - my forum is indexed, but only the top-level domain (www.glutenbee.com.au/forum). Sadly, that's not where the content is.

I've had my forum up for a solid 3 moths now and have multiple threads with a few hundred posts. I've installed this plugin for my sitemap: http://xenforo.com/community/resources/sitemap-for-xenforo.67/

My webmaster had tried a bunch of things and none have worked. I've manually submitted the site to google (through Webmaster tools) a number of times but it just won't budge.

Have you got any ideas or tricks? The content is good and useful and relevant; I have enable the right settings in Xenforo control panel....I just don't get it.

Many thanks in advance!!!!
Ross
 
You need three things for that:

  1. Backlinks to your site.
  2. More links
  3. And more links

And I am talking about real links. Not that crap spammy links to you member section.
 
Hmm..

You have this:

http://glutenbee.com.au/robots.txt

Which shows nothing but WordPress links @ http://glutenbee.com.au/sitemap.xml.gz

http://glutenbee.com.au/forum/robots.txt << Doesn't exist.

Did you submit a new sitemap through Google Webmaster Central? (http://www.google.com/webmasters/) [Edit: Sorry, missed it in the OP]

I'd say that Google is mostly watching your WordPress map, which may be slowing down your Forum indexing.
Also, did you have vBulletin installed before?

When I look here: https://www.google.com/#q=site:glutenbee.com.au -- I see a bunch of broken links to old forum pages, which look like vBulletin.

If you didn't redirect old links to the XenForo install, you could check here or here (vb3). If you saved your previous ID's -- this would be helpful to fix broken links to old threads.

Others can throw in words of advice, this is just what I found in a few minutes of checking.

EDIT: I just found your forum link at the very bottom of the sitemap that's visible:

Code: 
<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="http://glutenbee.com.au/wp-content/plugins/google-sitemap-generator/sitemap.xsl"?><!-- generator="wordpress/3.8.3" -->
<!-- sitemap-generator-url="http://www.arnebrachhold.de" sitemap-generator-version="3.4" -->
<!-- generated-on="April 23, 2014 11:10 am" -->
<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">   <url>
     <loc>http://glutenbee.com.au/</loc>
     <lastmod>2014-04-23T09:01:15+00:00</lastmod>
     <changefreq>daily</changefreq>
     <priority>1.0</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/04/celiac-disease-foundation-usa-7-day-gluten-free-meal-plan/</loc>
     <lastmod>2014-04-23T09:01:15+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/04/drugs-development-combat-coeliac-disease/</loc>
     <lastmod>2014-04-10T10:41:15+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/02/ross-journey/</loc>
     <lastmod>2014-03-21T08:49:52+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/02/gluten-free-eating-malaysia/</loc>
     <lastmod>2014-03-21T08:49:39+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/02/mystory/</loc>
     <lastmod>2014-03-21T08:49:26+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.5</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/03/gluten-punishment-abc-checkout/</loc>
     <lastmod>2014-03-21T08:49:13+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/03/celicdiseasefoundation-symptomchecklist/</loc>
     <lastmod>2014-03-21T08:49:02+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/03/new-coeliac-disease-treatment-trials-early-stage/</loc>
     <lastmod>2014-03-21T08:48:41+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/03/low-gluten-labelling-mean-gluten-free-eating/</loc>
     <lastmod>2014-03-21T08:48:29+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/03/gluten-free-food-growing-industry/</loc>
     <lastmod>2014-03-16T10:34:09+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/03/coeliac-awareness-week-2014/</loc>
     <lastmod>2014-03-16T09:21:08+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/about-us/</loc>
     <lastmod>2014-03-11T08:00:10+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/home-3/</loc>
     <lastmod>2014-03-10T03:58:27+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/03/new_coeliac_treatment_trial/</loc>
     <lastmod>2014-03-09T07:18:43+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/home/</loc>
     <lastmod>2014-03-03T03:13:52+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/02/good-bad-gluten-free/</loc>
     <lastmod>2014-03-03T02:45:06+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/02/products-like/</loc>
     <lastmod>2014-03-03T02:44:59+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/gluten/</loc>
     <lastmod>2014-03-02T08:41:29+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/key-research-links-coeliac-gluten-ataxia/</loc>
     <lastmod>2014-03-02T08:37:15+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/gluten-free-labelling-requirements/</loc>
     <lastmod>2014-03-02T08:34:45+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/gluten-ataxia/</loc>
     <lastmod>2014-03-02T08:33:50+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/coeliac-disease/</loc>
     <lastmod>2014-03-02T08:31:44+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/2014/02/research/</loc>
     <lastmod>2014-03-02T07:52:12+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.2</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/gluten-free-safe-foods-list/</loc>
     <lastmod>2014-03-01T17:11:27+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/gluten-free-grains-and-flours/</loc>
     <lastmod>2014-03-01T17:01:40+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/contact-us/</loc>
     <lastmod>2014-03-01T16:46:24+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/adopting-gluten-free-diet/</loc>
     <lastmod>2014-03-01T16:32:21+00:00</lastmod>
     <changefreq>weekly</changefreq>
     <priority>0.6</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/category/gf_featured_recipes/</loc>
     <changefreq>weekly</changefreq>
     <priority>0.3</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/category/gluten-free-travel/</loc>
     <changefreq>weekly</changefreq>
     <priority>0.3</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/category/ourstory/</loc>
     <changefreq>weekly</changefreq>
     <priority>0.3</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/category/products/</loc>
     <changefreq>weekly</changefreq>
     <priority>0.3</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/category/research/</loc>
     <changefreq>weekly</changefreq>
     <priority>0.3</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/category/restaurant_reviews/</loc>
     <changefreq>weekly</changefreq>
     <priority>0.3</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/category/whats_the_buzz/</loc>
     <changefreq>weekly</changefreq>
     <priority>0.3</priority>
   </url>
   <url>
     <loc>http://glutenbee.com.au/forum/</loc>
     <changefreq>daily</changefreq>
     <priority>0.9</priority>
   </url>
</urlset>
 
There is an auto generated robots.txt file, you need to add a htaccess rewrite from the .txt file to the .php file.

Liam
 
Hi guys - you are freaking awesome! Thanks very much for your help!! I am extremely grateful for you all taking the time to lay it out. You ROCK!!!

Come onnnnn!!!! I'm excited now; it's good to know it's something you can fix and not just some hidden part of Google's algorithm that doesn't like you.
 
You must log in or register to reply here.

Similar threads

El Porcharo
  • Question Question
Google Indexing
Replies
14
Views
1K
MySiteGuy
MySiteGuy
Num7
Google Indexation issue, not sure what to do
Replies
5
Views
1K
Alvin63
A
P
Problem with SEO, indexing and getting google to crawl pages
Replies
13
Views
2K
Miri
Miri
benFF
vB 3.8 > xF 2.2 migration : 8 months later
2
Replies
28
Views
3K
benFF
benFF
Nulumia
Add-on A Look At The Upcoming Public Release of SEO & Index Tools
2 3
Replies
46
Views
10K
Forsaken
Forsaken
Top Bottom