Traffic Got A Hit Last Week; Down ~40% . Am I Doing Anything Wrong?

TheBigK said:
1.
So these URLs were http://www.crazyengineers.com/tag/<tag> , and I had over 10k of them that I removed. I decided only to keep the best ones. How do I redirect only the ones that were removed? I'm not experienced in setting up redirects. Plus the current set of tags list well in Google, so I'd keep them intact. Can we automatically make the 404s to redirect to the homepage?

2. I see that MagnusB suggestions are also good, consider disallowing these pages..

K: Not sure how can I disallow the 404s. I'll have to look for a wordpress plugin to do that.

3. It took about one week.. In this week I've edited the h1 tags to be bigger (xx-large size), and also added a plugin which generates search keywords in the thread, those keywords are the actual keywords that were searched to find that thread..

K: Interesting that you got it back in just one week. I think we've the font size properly set (we're using XF default). I'm considering addition of XenTags plugin to link our content. But the task of adding keywords to our existing contents is going to be HUGE.

4. Also check in the webmasters tools, check your keyword impressions vs actual clicks if they dropped.
Good luck :)

K: This is how it is -

View attachment 34304


The traffic decline (as indicated by Analytics) started on 5th of September, after attaining a peak on 4th. It was unusual traffic decline.
Click to expand...
My full robots.txt;
Code: 
User-agent: *
 
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /conversations/
Disallow: /data/
Disallow: /forums/-/
Disallow: /forums/tweets/
Disallow: /goto/
Disallow: /help/
Disallow: /internal_data/
Disallow: /js/
Disallow: /library/
Disallow: /login/
Disallow: /lost-password/
Disallow: /misc/contact/
Disallow: /members/
Disallow: /online/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /posts/
Disallow: /search/
Disallow: /styles/
Allow: /

The /forums/-/ just disallows the "Mark forums as read" link, though by default it should not be displayed to Google. I also disallow a few account specific URLs, like /conversations/ and /account/, though Google should never see them, but it is better to be safe than sorry. I also disallow indexing my member list and profiles, but this is my choice, I don't know if it has any effect on anything. I also disallow all forms and search pages, they are very uninteresting for Google. How well these affect SEO, I don't know, except for those that reduces double entries (like /posts/ and /goto/). I don't remember the intention of everything in this one, I think I mostly copied it from a post I found somewhere here.

As for automatically adding keywords to pages, look at AzuCloud: http://xenforo.com/community/resources/azucloud-search-keyword-cloud.113/

As for h1 font size, I don't think that affects seo, unless you are using a very small font size.
 
TheBigK said:
1.
So these URLs were http://www.crazyengineers.com/tag/<tag> , and I had over 10k of them that I removed. I decided only to keep the best ones. How do I redirect only the ones that were removed? I'm not experienced in setting up redirects. Plus the current set of tags list well in Google, so I'd keep them intact. Can we automatically make the 404s to redirect to the homepage?

2. I see that MagnusB suggestions are also good, consider disallowing these pages..

K: Not sure how can I disallow the 404s. I'll have to look for a wordpress plugin to do that.

3. It took about one week.. In this week I've edited the h1 tags to be bigger (xx-large size), and also added a plugin which generates search keywords in the thread, those keywords are the actual keywords that were searched to find that thread..

K: Interesting that you got it back in just one week. I think we've the font size properly set (we're using XF default). I'm considering addition of XenTags plugin to link our content. But the task of adding keywords to our existing contents is going to be HUGE.

4. Also check in the webmasters tools, check your keyword impressions vs actual clicks if they dropped.
Good luck :)

K: This is how it is -

View attachment 34304


The traffic decline (as indicated by Analytics) started on 5th of September, after attaining a peak on 4th. It was unusual traffic decline.
Click to expand...
1. This code can be used in htaccess to redirect all 404 pages to the homepage
Code: 
 ErrorDocument 404 /community/
3. I don't mean I added tags to threads.. I will give you an example.. If I had a thread talking about how to create email account and a visitor found this thread by searching the keyword "email accounts" in Google, the plugin will automatically add "email accounts" to the page (actually I have a block called keywords in the page).
4. The impressions number isn't important, the important is actual clicks number.. just check the actual clicks number if it dropped. and then check which keyword drop has caused this..
 
Your robots file is good. But I don't understand why do you use this :cautious: .. anyway, It won't affect anything..
Code: 
Disallow: /forums/-/
 
Really appreciate it. I'm wondering why Mashable keeps 404 page and not redirect it to homepage. Just wondering if it's a good SEO practice even it looks like a good thing to do, logically.
 
TheBigK said:
I"m already on it. Would you recommend adding a permanent redirect to homepage in the 404.php file in my wordpress?
Click to expand...
If you are worry about that, then you can add a condition to only redirect these 404 pages if they were caused by tags :) just search WordPress resources, and you will find such conditions..
 
Khaleal said:
Your robots file is good. But I don't understand why do you use this :cautious: .. anyway, It won't affect anything..
Code: 
Disallow: /forums/-/
Click to expand...
If you hover over "Mark forums as read" you will see it, however, it seems it is different on forum by forum basis, so that rule is basically worthless (besides, that is a logged in member only function, so it should never cause a double URL). IIRC, Google supports wild cards in robots.txt, so /forums/*/-/ might cover the rest of them, if you really want to block it.

TheBigK said:
Is it a good practice to permanently redirect 404 page to site's homepage? How does XF handle it?
Click to expand...
xF handles not found pages with a 404. You can see googles recommendations regarding 404 or redirects here: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=181708
 
Khaleal said:
If no 301 redirects were implemented then the following would happen:
http://www.crazyengineers.com/tag/abcdefg
http://www.crazyengineers.com/tag/abcdefghij
would show the same content resulting in duplicate content pages..
By redirecting these pages to the homepage, you are telling Google that these pages have been replaced..
Click to expand...
Actually, those pages are not duplicate content, they are both missing or non existent. Google recommends you return a 404 (hard 404, not soft) instead of redirecting them. From what I gather, Google does not hurt you for 404 pages, but you can loose some link juice if some of them are linked on external pages.
 
I might as well shut up now, cause I am confused. I tried to figure out whether or not 404 are harmful or not, and I can't seem to find an answer. There is one statement made by a google guy from 2010 saying that 404 are not harmful, but that is a long time ago. I can't really find a straight answer to it.
 
Khaleal said:
Sorry to hear that. I would suggest you the following
1. first of all you should 301 redirect all 404 pages in the forum to the forum homepage, this would resolve the 404 errors issue.
Click to expand...

No, all 404 pages should return a proper 404 error code in the document header. A good 404 page will provide links to get the visitor either back to the forums, or have a search box built in to it so they can search.

Google will remove these links when it gets the 404 error code.

Also don't expect your traffic to return overnight, continue to build your site for the user and not search engines and you won't have anything to worry about.
 
David_ said:
No, all 404 pages should return a proper 404 error code in the document header. A good 404 page will provide links to get the visitor either back to the forums, or have a search box built in to it so they can search.

Google will remove these links when it gets the 404 error code.

Also don't expect your traffic to return overnight, continue to build your site for the user and not search engines and you won't have anything to worry about.
Click to expand...
Then the best choice for him is to replace his 404.php file content with his index.php or search.php content, that would be user friendly..
 
Yeah, I've been building the site for the users for a long time now and have been wondering what went wrong. I need to be absolutely sure that the site has nothing 'broken' that needs my attention.
 
TheBigK said:
Yeah, I've been building the site for the users for a long time now and have been wondering what went wrong. I need to be absolutely sure that the site has nothing 'broken' that needs my attention.
Click to expand...

As long as you have a google webmasters account and no errors are showing up in there chances are good it's nothing you've done.

Google makes changes to the their search results and how it ranks stuff constantly. Sometimes it's not the amount of traffic you're getting but the quality that matters. Are you seeing an increase in conversions, lower bounce rates, etc etc.
 
Khaleal said:
Then the best choice for him is to replace his 404.php file content with his index.php or search.php content, that would be user friendly..
Click to expand...

No the best choice is to have a 404 page and return the correct headers to the bots.

http://productforums.google.com/forum/#!topic/webmasters/g4_jplpldhc
http://www.webmasterworld.com/google/4471757.htm

And from the most credible source of all Google Webmaster Blog: http://googlewebmastercentral.blogspot.com/2011/05/do-404s-hurt-my-site.html

404s are a perfectly normal part of the web; the Internet is always changing, new content is born, old content dies, and when it dies it (ideally) returns a 404 HTTP response code. Search engines are aware of this; we have 404 errors on our own sites, as you can see above, and we find them all over the web. In fact, we actually prefer that, when you get rid of a page on your site, you make sure that it returns a proper 404 or 410 response code (rather than a “soft 404”). Keep in mind that in order for our crawler to see the HTTP response code of a URL, it has to be able to crawl that URL—if the URL is blocked by your robots.txt file we won’t be able to crawl it and see its response code. The fact that some URLs on your site no longer exist / return 404s does not affect how your site’s other URLs (the ones that return 200 (Successful)) perform in our search results.
Click to expand...


If you redirect all 404 errors to your home page, you're creating a ton of soft 404 errors.Which is bad.

 
TheBigK said:
I do use Google Webmaster tools. It threw following messages at me-

1. Large increase in the number of 404 (most of them are the removed tags, of course).
2. Sudden decrease traffic for top URL. Surprisingly, google still shows the URL on #1 position for the keywords. I've received the same message for the same URL in the past, but the overall forum traffic never went down.
Click to expand...
You're not alone. I got the same thing, and the mainpages were all xenForo-powered.

CODForums is fine, MetalGearFans is fine, MVC3Forum is fine... but EverythingHalo, and CarlosX360 Community are hit with "Increase in not found errors" and "Increase in authorization permission errors" and the worst part, both are stock XF, nothing other than Jaxel's xenUtilies.

However, CODForums is the one that I'm worried about, Google Analytics has been showing a drop of average visitors, and organic traffic. So, I tried to bring it back up and put Twitter, Google+, and Facebook to work. It sounds like google's trying to say "#@^& you" to us. :(
 
Back
Top Bottom