1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

My threads are not indexing in Google - what could I be doing wrong?

Discussion in 'Forum Management' started by rosstqz, Apr 28, 2014.

  1. rosstqz

    rosstqz New Member

    Hi guys

    I'm pulling my hair out over this one! I can't get my threads indexed by google - my forum is indexed, but only the top-level domain (www.glutenbee.com.au/forum). Sadly, that's not where the content is.

    I've had my forum up for a solid 3 moths now and have multiple threads with a few hundred posts. I've installed this plugin for my sitemap: http://xenforo.com/community/resources/sitemap-for-xenforo.67/

    My webmaster had tried a bunch of things and none have worked. I've manually submitted the site to google (through Webmaster tools) a number of times but it just won't budge.

    Have you got any ideas or tricks? The content is good and useful and relevant; I have enable the right settings in Xenforo control panel....I just don't get it.

    Many thanks in advance!!!!
    Ross
     
  2. DRaver

    DRaver Active Member

    You need three things for that:

    1. Backlinks to your site.
    2. More links
    3. And more links

    And I am talking about real links. Not that crap spammy links to you member section.
     
  3. ENF

    ENF Well-Known Member

    Hmm..

    You have this:

    http://glutenbee.com.au/robots.txt

    Which shows nothing but WordPress links @ http://glutenbee.com.au/sitemap.xml.gz

    http://glutenbee.com.au/forum/robots.txt << Doesn't exist.

    Did you submit a new sitemap through Google Webmaster Central? (http://www.google.com/webmasters/) [Edit: Sorry, missed it in the OP]

    I'd say that Google is mostly watching your WordPress map, which may be slowing down your Forum indexing.
    Also, did you have vBulletin installed before?

    When I look here: https://www.google.com/#q=site:glutenbee.com.au -- I see a bunch of broken links to old forum pages, which look like vBulletin.

    If you didn't redirect old links to the XenForo install, you could check here or here (vb3). If you saved your previous ID's -- this would be helpful to fix broken links to old threads.

    Others can throw in words of advice, this is just what I found in a few minutes of checking.

    EDIT: I just found your forum link at the very bottom of the sitemap that's visible:

    Code:
    <?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="http://glutenbee.com.au/wp-content/plugins/google-sitemap-generator/sitemap.xsl"?><!-- generator="wordpress/3.8.3" -->
    <!-- sitemap-generator-url="http://www.arnebrachhold.de" sitemap-generator-version="3.4" -->
    <!-- generated-on="April 23, 2014 11:10 am" -->
    <urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">   <url>
         <loc>http://glutenbee.com.au/</loc>
         <lastmod>2014-04-23T09:01:15+00:00</lastmod>
         <changefreq>daily</changefreq>
         <priority>1.0</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/04/celiac-disease-foundation-usa-7-day-gluten-free-meal-plan/</loc>
         <lastmod>2014-04-23T09:01:15+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/04/drugs-development-combat-coeliac-disease/</loc>
         <lastmod>2014-04-10T10:41:15+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/02/ross-journey/</loc>
         <lastmod>2014-03-21T08:49:52+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/02/gluten-free-eating-malaysia/</loc>
         <lastmod>2014-03-21T08:49:39+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/02/mystory/</loc>
         <lastmod>2014-03-21T08:49:26+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.5</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/03/gluten-punishment-abc-checkout/</loc>
         <lastmod>2014-03-21T08:49:13+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/03/celicdiseasefoundation-symptomchecklist/</loc>
         <lastmod>2014-03-21T08:49:02+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/03/new-coeliac-disease-treatment-trials-early-stage/</loc>
         <lastmod>2014-03-21T08:48:41+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/03/low-gluten-labelling-mean-gluten-free-eating/</loc>
         <lastmod>2014-03-21T08:48:29+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/03/gluten-free-food-growing-industry/</loc>
         <lastmod>2014-03-16T10:34:09+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/03/coeliac-awareness-week-2014/</loc>
         <lastmod>2014-03-16T09:21:08+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/about-us/</loc>
         <lastmod>2014-03-11T08:00:10+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/home-3/</loc>
         <lastmod>2014-03-10T03:58:27+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/03/new_coeliac_treatment_trial/</loc>
         <lastmod>2014-03-09T07:18:43+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/home/</loc>
         <lastmod>2014-03-03T03:13:52+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/02/good-bad-gluten-free/</loc>
         <lastmod>2014-03-03T02:45:06+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/02/products-like/</loc>
         <lastmod>2014-03-03T02:44:59+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/gluten/</loc>
         <lastmod>2014-03-02T08:41:29+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/key-research-links-coeliac-gluten-ataxia/</loc>
         <lastmod>2014-03-02T08:37:15+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/gluten-free-labelling-requirements/</loc>
         <lastmod>2014-03-02T08:34:45+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/gluten-ataxia/</loc>
         <lastmod>2014-03-02T08:33:50+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/coeliac-disease/</loc>
         <lastmod>2014-03-02T08:31:44+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/2014/02/research/</loc>
         <lastmod>2014-03-02T07:52:12+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.2</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/gluten-free-safe-foods-list/</loc>
         <lastmod>2014-03-01T17:11:27+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/gluten-free-grains-and-flours/</loc>
         <lastmod>2014-03-01T17:01:40+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/contact-us/</loc>
         <lastmod>2014-03-01T16:46:24+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/adopting-gluten-free-diet/</loc>
         <lastmod>2014-03-01T16:32:21+00:00</lastmod>
         <changefreq>weekly</changefreq>
         <priority>0.6</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/category/gf_featured_recipes/</loc>
         <changefreq>weekly</changefreq>
         <priority>0.3</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/category/gluten-free-travel/</loc>
         <changefreq>weekly</changefreq>
         <priority>0.3</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/category/ourstory/</loc>
         <changefreq>weekly</changefreq>
         <priority>0.3</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/category/products/</loc>
         <changefreq>weekly</changefreq>
         <priority>0.3</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/category/research/</loc>
         <changefreq>weekly</changefreq>
         <priority>0.3</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/category/restaurant_reviews/</loc>
         <changefreq>weekly</changefreq>
         <priority>0.3</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/category/whats_the_buzz/</loc>
         <changefreq>weekly</changefreq>
         <priority>0.3</priority>
       </url>
       <url>
         <loc>http://glutenbee.com.au/forum/</loc>
         <changefreq>daily</changefreq>
         <priority>0.9</priority>
       </url>
    </urlset>
    
     
    rosstqz likes this.
  4. Liam W

    Liam W Well-Known Member

    There is an auto generated robots.txt file, you need to add a htaccess rewrite from the .txt file to the .php file.

    Liam
     
    rosstqz likes this.
  5. ENF

    ENF Well-Known Member

    One more thing, you may want to turn off directory browsing. :eek:

    directory_browse.PNG
     
    rosstqz and Steve F like this.
  6. ENF

    ENF Well-Known Member

    rosstqz likes this.
  7. Liam W

    Liam W Well-Known Member

    rosstqz likes this.
  8. rosstqz

    rosstqz New Member

    Hi guys - you are freaking awesome! Thanks very much for your help!! I am extremely grateful for you all taking the time to lay it out. You ROCK!!!

    Come onnnnn!!!! I'm excited now; it's good to know it's something you can fix and not just some hidden part of Google's algorithm that doesn't like you.
     

Share This Page