XF 1.5 Convert URLs to Page Titles issue w/ one website

[Optic]

There is one website in particular that just doesn't seem to work with this feature: MyAnimeList

Example URL:
http://myanimelist.net/anime/28999/Charlotte

I realise it may be down to something on their end but given that a lot of our members link to this particular website from time to time it would be great if we could somehow get it working for a more consistent experience.

Any help would greatly be appreciated - also did a test thread here.

 

[melbo]

http://myanimelist.net/

 

[melbo]

It might be that they have a url in their title?
Charlotte - MyAnimeList.net

 

[Mike]

It's an issue on their end. Specifically, it seems to be giving the fetcher different content (an Incapsula thing).

 

[Mr. Goodie2Shoes]

Would that be because of the user agent "Zend_Http_Client"?

 

[Mike]

Possible. It could also be because we make HTTP/1.0 requests due to other limitations.