Unhappiness with paths and routes and imperfect SEO options

[TacticsUK]

Its covered here: One Month After Switching From vBulletin To XenForo...
I block Alexa from my servers. It seems Shawn does something similar.

No Google stats there at all. The Alexa stats are gathered from users who have the Alexa toolbar installed, you can't block that from your servers as far as I am aware, the toolbar sends back stats to Alexa, it isn't from robots or spiders. Shawn is saying they are mitigating users of little value, how that could make your traffic drop from being what looks like being in the top 250 on Alexa down to less than a thousand I don't know.

So how do you block Alexa from your servers?

 

[Alpha1]

I block Alexa bots by user agent and alexa toolbars by using bad behavior: http://bad-behavior.ioerror.us/
Bots are low quality users that only burden the servers and add no value. Its better to block them. It saves a ton of bandwidth costs and server load.

 

[TacticsUK]

I can't see anything on the Bad Behaviour site that mentions anything about blocking people who have the Alexa toolbar installed on their browser. Having the Alexa toolbar installed and it reporting usage data back to Alexa is no different to Chrome reporting usage data back to Google.

Actually have just checked some logs and it does look like the Alexa Toolbar may well show up in the User Agent e.g.:

"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; AlexaToolbar/amzni-3.0; GTB7.5; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 1.1.4322; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E; .NET CLR 3.0.30729; AlexaToolbar/amzni-3.0)"

But that said out of the 662,367 entries in today's log file only 357 lines contain "Alexa".

root@banana:/var/www/logs/***.co.uk# grep "" ***.co.uk-banana-access_log | wc -l
662367
root@banana:/var/www/logs/***.co.uk# grep "Alexa" ***.co.uk-banana-access_log | wc -l
357

I host a large recruitment website, currently handling about 1.2M human PVs per month (as measured by Google Analytics) and yes, robot and spider traffic is a heck of a lot more than this. For example I actively block all traffic from Amazon due to the fact that all I see from their network is automated robots scraping content. But I have never seen a way of blocking someone who has the Alexa toolbar installed and indeed, why would you? For example I saw that my Mother-in-Law had it on her laptop last year, it is installed by "normal" people (not including my MiL in that group of course ) and making your site inaccessible to them would seem like madness to me. Ban Alexa, Baidu, ahrefs, majestic12 and whatever if you think they are hogging resource but to ban end users for having the Alexa toolbar would be a bit mental as I don't think you can class someone who has it installed as being a "bad" user, a little naive maybe but not bad.

But I guess if DigitalPoint are blocking users with the Alexa toolbar then surely their site position would have dropped off the scale rather than just reducing slightly? Will have more of a look at Bad Behaviour when I get home as got to leave the office now.

Cheers for the discussion, have learnt a lot so far!

 

[Digital Doctor]

Alexa is so 1999.
Last time I remember caring about Alexa, I was using astalavista.box.sk from time to time.

 

[Alpha1]

Alexa can be considered spyware. This 'free and handy' toolbar is installed by people who are unaware that they are installing tracking software. Bad Behavior blocks can be setup to block malicious browser addons. Alexa falls in this category IMO. Users with Alexa installed generally have a load of other problematic browser plugins installed. Such plugins can cause site issues. Bad Behavior does not simply block the user, but also tells the user that they have malicious software installed.

Alexa has bots with different user agents and Alexa also has various toolbar users tracking. So once you start blocking bots or toolbars, then you would see a drop of traffic even if you do not block all of them. Alexa may change things around once in a while to get around such things.

 

[TacticsUK]

@Digital Doctor - I preferred Hotbot arond that time, or Lycos. I do still use Alexa to show trends in competitor sites and it is usually pretty close. This is when comparing it to Hitwise data that a competitor puts out in press releases so show how big they are so using that as a bench-mark is what enables me to suggest that Alexa, in our industry at least, isn't far off the mark!

@Alfa1 - could the same not be said of Chrome? Nah, because that's Google and they are much less evil than Amazon Off to have a closer look at Bad Behaviour now.

 

[Alpha1]

Nowadays we can say the same about a lot of services and software. You are correct about that. But at least Chrome gives something of value in return and its too widespread to even consider if you would want to block it.
Anyway, we are straying waaay off topic here.

 

[CTXMedia]

What happens to the "SEO" value when the thread is moved (completely changing the URL structure)?

 

[CTXMedia]

Oh, and just for interests sake - since moving my "big board" to XF the Google ranking has increased and, for the first time ever, I've taken the #1 spot for my primary key phrase.

[caveat: The previous #1 spot holder also changed their site software and stuffed it so full of ads that the are now hemorrhaging members / visitors - so that could have contributed?]

 

[Digital Doctor]

Hotbot Lycos

Classic ! I remember Lycos and Hotbot !

 

[Sherrie]

Thank you for this thread as this was exactly what I was going to ask! I almost upgraded my 3.8 to xenforo last year but at the time the tone here was really dismal with the lawsuit and I eventually decided on buying vb 5 and installing vb4. I am currently contemplating moving again because the support on vb seems to have changed a lot and the software seems so unreliable. My forum members loved vb4 believe it or not, that actually went really well but my SEO changed so badly once I moved my articles from simple HTML/CSS pages to the CMS. I have been wracking my brain out for so long and I finally realised my biggest problem is the change in directory structure. I have watched new sites with few organic links change their directly structure, not just nicely but totally stuff keywords 3 and 4 directories deep to get to a page and fly past me.

I have looked at the URL structure on vb5 and it will go something like /forum/forum section/thread name

This is what I would like to see when going to a thread instead of the directory structure changing to say /forum/ threads/
Same thing with the resource manager, when I go to an article I would like to see the category name in the URL structure.

From what I can see the resource manager is just as good for me as having a CMS it's just the URL structure that is causing me to hesitate aswell as the fact I will be moving away from a layout they are happy with but that will happen if I upgrade to vb5 anyway except I guess I don't need to purchase vb5

 

[Digital Doctor]

Welcome to Xenforo, first time poster Sherrie.

Thank you for this thread as this was exactly what I was going to ask! I almost upgraded my 3.8 to xenforo last year but at the time the tone here was really dismal with the lawsuit and I eventually decided on buying vb 5 and installing vb4. I am currently contemplating moving again because the support on vb seems to have changed a lot and the software seems so unreliable. My forum members loved vb4 believe it or not, that actually went really well but my SEO changed so badly once I moved my articles from simple HTML/CSS pages to the CMS. I have been wracking my brain out for so long and I finally realised my biggest problem is the change in directory structure. I have watched new sites with few organic links change their directly structure, not just nicely but totally stuff keywords 3 and 4 directories deep to get to a page and fly past me.

I have looked at the URL structure on vb5 and it will go something like /forum/forum section/thread name

This is what I would like to see when going to a thread instead of the directory structure changing to say /forum/ threads/
Same thing with the resource manager, when I go to an article I would like to see the category name in the URL structure.

From what I can see the resource manager is just as good for me as having a CMS it's just the URL structure that is causing me to hesitate aswell as the fact I will be moving away from a layout they are happy with but that will happen if I upgrade to vb5 anyway except I guess I don't need to purchase vb5

 

[Sherrie]

Thanks

 

[digitalpoint]

We don't block people using the Alexa toolbar. We do mitigate low quality users, of which a higher percent tend to have the Alexa toolbar installed.