MG 2.1 No title or description when embedding YouTube videos

[Martok]

This used to work just fine but I've noticed today that when I embed a YouTube video into my Media Gallery, it's only pulling the thumbnail of the video but not the title or description. I've tested this with several YouTube videos (plus add-ons disabled and in a default style) and got the same thing with each one. However, all of these work fine if I try it in the Media Gallery here.

I did test Daily Motion too and that only pulled the thumbnail but that did the same here, so that may need looking at. Also Twitch videos aren't working as they should on this site or mine - live streams will embed but only pull the thumbnail, no title or description, and you can't embed videos from the site e.g. URLs with the format like this https://www.twitch.tv/videos/479307011. I can add these as a separate bug report if you wish?

I'm guessing my YouTube issues are down to a server setting but I'm not sure what that would be, so a pointer in the right direction would be appreciated.

 

[Chris D]

This used to work just fine but I've noticed today that when I embed a YouTube video into my Media Gallery, it's only pulling the thumbnail of the video but not the title or description. I've tested this with several YouTube videos (plus add-ons disabled and in a default style) and got the same thing with each one. However, all of these work fine if I try it in the Media Gallery here.

Essentially the metadata is grabbed the same way as we grab the metadata for any URL, including the unfurl feature. So the best way to test it is probably to run it through the "Test URL unfurling" page under Tools in the Admin CP and see what output that gives. It should look something like this:



Unfortunately when we've seen this before, it has been some sort of CAPTCHA being loaded instead, some anti-spam/anti-DDoS measure or your site's IP address is simply blocked for some reason. As an example, I think massive swathes of OVH server IP addresses are banned from remotely accessing YouTube in this way.

Sadly, if this is the problem, there is literally nothing that can be done as far as I know.

The thumbnail will work regardless because we can infer the thumbnail URL from the ID of the video in the URL.

I did test Daily Motion too and that only pulled the thumbnail but that did the same here, so that may need looking at.

Given what I've just said, that's interesting and I can confirm that. Which, in actual fact, means I likely need to contradict what I just said

Essentially the metadata is grabbed the same way as we grab the metadata for any URL, including the unfurl feature.

It actually appears as though we don't do it the same way in XFMG. The code in XFMG predates the new functionality in XF 2.1 and it seems as though we haven't moved the XFMG code to that newer system. That's a bug so I'll fix that for the next release.

Also Twitch videos aren't working as they should on this site or mine - live streams will embed but only pull the thumbnail, no title or description, and you can't embed videos from the site e.g. URLs with the format like this https://www.twitch.tv/videos/479307011. I can add these as a separate bug report if you wish?

Yes, please post that as a bug report (it's an XF bug, not gallery specific).

 

[Martok]

Unfortunately when we've seen this before, it has been some sort of CAPTCHA being loaded instead, some anti-spam/anti-DDoS measure or your site's IP address is simply blocked for some reason. As an example, I think massive swathes of OVH server IP addresses are banned from remotely accessing YouTube in this way.

I've just tested the unfurling with a YouTube URL and got this:



Is this effectively what you have alluded to above?

Error 429 appears to be too many requests in a given amount of time (ie rate limiting) though if it is, I'm not sure why I'd be getting this - my site is quite small and very quiet with not a lot of videos recently added to the Media Gallery or indeed threads. Things seem to have been working back in July (when a YouTube video was last added to the gallery) but not now.

 

[Chris D]

Yeah this is roughly what I expected.

It might not even be your site specifically, but it could be a range of IPs they've blocked.

I suspect YouTube has a bit of a problem and occasionally tries to fight those sites that allow you to download YouTube videos. They may be blocking entire IP ranges, or specific hosts.

The only practical step I can recommend to circumvent this would be to implement a HTTP proxy. This would be a seperate server (or online service if such a thing exists) that your HTTP requests are proxied through.

We support this within the software. The primary purpose being to mask your server's real IP address if you attempt to hide it via proxying through CloudFlare, or similar (a site running through CloudFlare would leak its real IP address when performing HTTP requests unless it uses a proxy).

We have a separate VPS running Tinyproxy and all of our HTTP requests are proxied through that server's IP address which we define in our config.php.

Perhaps a bit overkill, but the benefit is that with it being a kind of "throwaway" server if you were to face a similar problem again, you could just get a new IP address for it if the server provider allows that.