s9e Media Sites 2.15.6

[OUTL4W]

Is there any easy fix for the twitter embeds not working in the xF1.5 version? I'm soon upgrading to 2.x but just wondering if there was a temporary fix?

 

[JoshyPHP]

update. just realized that the oembed folder on my b2 bucket is now full of thousands of json files. are these also being used on every youtube embed? or they are just created by xenforo by default?

It's created by XenForo and it's not used otherwise. A file is created when the data is fetched from the oEmbed (YouTube's) server, I assume they get pruned automatically but I don't know how/when it happens.

10,000 different YouTube videos is definitely a lot. With enough bots crawling your old content, you would end up constantly refreshing the oEmbed data if your TTL isn't set to 0 (= keep indefinitely.)

I'm attaching a quick patch, version 2.6.2-RC which limits the number of new fetch to 1 per page. (technically one per thread_view template but that's functionally the same)

Is there any easy fix for the twitter embeds not working in the xF1.5 version? I'm soon upgrading to 2.x but just wondering if there was a temporary fix?

You'd have to manually edit MediaBBCodes.php and replace ?autoplay=false with ?autoplay=false&parent=example.org where example.org is the full hostname for the page that displays the embed. If you have other issues with the XF1 add-on please do post in the corresponding thread. https://xenforo.com/community/resources/s9e-media-bbcodes-pack.2476/

 

[Chromaniac]

updated. page load seems to be better now. i have set the cache to unlimited so that's done. second issue i reported in the previous post still exists. on videos with long titles, the title goes out of the view. and! it expands the click area of the cover image. which means that play image is misaligned. and you lose access to content area below the video embed. in my case the links to edit/delete and so on. here is a screenshot showcasing the cursor at the end of the cover image . also just realized that with this update, the page with multiple videos would load slow multiple times instead of just once heh. but should be fine!

parent=example.org

i do wonder though. the embed code provided by twitter itself does not ask for your domain name. so how does that work? i assume this is connected to that CSRF thing?

 

[JoshyPHP]

on videos with long titles, the title goes out of the view.

That's weird, it looks like it's missing some of the CSS. Try disabling/reenabling both click-to-load option, see if it fixes it?

i do wonder though. the embed code provided by twitter itself does not ask for your domain name. so how does that work? i assume this is connected to that CSRF thing?

I read that as Twitch. I think @OUTL4W was talking about Twitch, whose latest update requires the full hostname of every parent frame. Twitter should work fine afaik.

 

[Chromaniac]

no luck. disabled cloudflare to check if it was a caching problem. tried in incognito as well. rebuild the addon.

 

[JoshyPHP]

Can you send me a link to a page that shows that video? I'd like to compare to my local board.

 

[Xon]

@Xon Do you mean that in theory or is that something you witnessed after enabling it in the add-on? If that's something that happened on your board, what was the approximate number of current users?

Witnessed.

The board had something like ~12000 users in the last 15 minutes (so about 200-500 per second) with +30 php-fpm workers. nginx would get very unhappy when all php workers where busy

It looks like fetching the oembed was being very slow and blocking. Disabling the oembed option in your add-on made things work as fast as expected.

:edit: This might have been an interaction with how internal_data is handled by the plugable filesystem stuff XenForo has and the custom code on that site, but I'ld need todo more investigating.

I'm actually unsure why XF bothers storing it on disk. The entire record needs to be fetched from the DB, and the oembed data should be fairly small...

 

[Xon]

@JoshyPHP after digging into how your add-on interacts with oembed framework I believe I've found the issue.

The problem is the call to XF\Service\Oembed::fetchNewOembed, which always fetches data from the source instead of hitting the cache. XF\Service\Oembed::getOembed looks to be what you want to call

 

[JoshyPHP]

It calls fetchNewOembed after checking the database so unless the Oembed service deletes rows in the database but leaves cache files on disk, it should generally be alright if I read it correctly. It does however allow for the possibility of a race condition in v2.6.1 where the database could be updated between fetches. In v2.6.2 there's only one fetch per page.

Edit: on second thought, getOembed does check for concurrency across processes whereas fetchNewOembed doesn't so that part could make a big difference under heavy traffic.

 

[JoshyPHP]

I'm actually unsure why XF bothers storing it on disk. The entire record needs to be fetched from the DB, and the oembed data should be fairly small...

I retraced how the XF:Oembed service works. I was operating on the assumption that the files on disk were only used temporarily and that the database was the real repository. In actuality, the oEmbed data is always stored in files and the database is only used to store logs. Since I only use data from the logs, that means the oEmbedCacheTTL setting doesn't really matter for my use case and only the oEmbedLogLength setting should be set to a high number, or 0 to keep logs indefinitely. XenForo's hourly cleanup is in charge of pruning the cache and logs.

Currently, my plan to make that feature usable under heavy traffic is:

• Limit the number of fetches to 1 per page.
• Limit the number of concurrent fetches across processes. XenForo's default is 10.
• Recommend to keep the default value for oEmbedCacheTTL (7 days) to reduce the number of files on disk over time and set a large value for oEmbedLogLength. The default is 90 days, it could be extended to a year or even be kept indefinitely.

 

[Chromaniac]

so if the database entry exists already, xenforo would not recreate the json file on disk?

 

[JoshyPHP]

The XenForo service would, but this add-on doesn't use the service if the logs are still in the database so it shouldn't. In your case, you'd still get 10,000 files on disk until oEmbedCacheTTL is reached and they're pruned normally.

 

[JoshyPHP]

JoshyPHP updated s9e Media Sites with a new update entry:

Fixes and improvements to the oEmbed-enabled click-to-load feature

The add-on's CSS has been updated to correctly handle long titles.
The oEmbed logic has been updated to limit the number of concurrent fetches to prevent resource exhaustion. The number of fetches is limited to 1 per page, with a maximum of 3 concurrent fetches across processes.

This is a recommended update if you use the click-to-load feature with oEmbed enabled.

Read the rest of this update entry...

 

[Chromaniac]

So, oEmbed media cache lifetime could be a smaller number of our preference. oEmbed media log length should be as high as possible (0 preferred). oEmbed media cache refresh is irrelevant. Right?

 

[JoshyPHP]

As far as this add-on is concerned, yes. oEmbed may be used by other stuff though, so be careful if you want to set the media cache lifetime to a low(-er than default) value.

 

[Chromaniac]

right. that folder has been empty for me all this time so maybe it should be fine. i plan to keep an eye on the oembed folder for a few days and see how it is managed by xenforo with a low setting. formatting issue is fixed. thanks for this update!

 

[Chromaniac]

just wanted to check if config.php file has any option to change the location of the oembed_cache folder! manual does not mention any setting for this amongst other folders that should logically be kept on internal storage! (sitemap comes to mind).

 

[JoshyPHP]

It doesn't look like it does, but I don't know.

 

[Xon]

This medium embed is borderline unreadable, any hints on how to hit it with a css styling stick? Or this just on Medium ****ing with their layout again;

https://medium.com/@WookieMonsterTV/my-interaction-with-angryjoe-d715aedb0e02

I'm not sure what it should really look like, but that isn't it.

 

[Chromaniac]

he did say that medium seems to have given up on their embed code on previous page.

s9e Media Sites

I don't know how important the Lighthouse score is to site owners It should be top priority to every admin since site speed influences Google ranking as well as user bounce rate. It's critical. I have an unrelated question about the Amazon embed. Can the title be truncated to avoid...

xenforo.com