Third party Amazon URL conversion not working

[suineg]

I am running 1.5.9 and Amazon links are not converting into the "pretty titles" that other websites do.

I am unsure if this is new or it has never worked. We recently disabled s9e from taking Amazon and making it media tagged embeds so we just noticed it. Following is a list of different URL's we have tried.

http://www.amazon.com/dp/B004EEOACC

http://a.co/2qix6ML

http://amzn.to/1mijugk

https://www.amazon.com/Another-********-Night-Suck-City/dp/0393051390/

Our site is fixing other URL's properly as far as I can tell like this recent one: http://money.cnn.com/2016/09/06/technology/weapons-of-math-destruction/index.html

I read an old post about Amazon having their title on their page deeper than 20kb and that there was a fix pushed to look for title up to 50kb. I edited the AutoLink.php file to look up to 100kb based on that post and there was no change. I didn't restart php-fm and I assume I didn't need to but let me know if I should try that.

Attached is what I have it set to in the ACP.

 

[Mike]

There are probably 3 different things at play here:

1. As of 1.5.10/the patch, we don't follow redirects for security reasons. Most of those URLs will redirect.
2. Taking that out of the equation, Amazon blocks the request based on user agent.
3. Bypassing that, for one of the test URLs, the <title> tag was over 300KB into the response.

The fetch limit is mostly to try to ensure the request returns quickly/defensive programming. Amazon is inlining their CSS and JS to reduce the number of requests which is pushing the title tag way down. I'm unsure whether we want to change this. We would need to resolve #2 first; that's fairly trivial though (unless they block it again, in which case I wouldn't want to take further steps).

 

[suineg]

No problem, all of those are great reasons why it isn't working. I didn't see a current report on it and felt you guys should know just in case.

I can imagine that Amazon is always messing with what they feel is the ultimate optimization of their site. Playing catch up nonstop is probably exhausting and a waste of resources.

 

[Mike]

It actually looks like Amazon has changed things again (unless one of my more recent changes has incidentally fixed this), so it appears to be working now (at least with the current development version). I'm going to call this a service issue in general.

 

[rfc0001]

Looks like they changed things once again so not working again