XF 2.1 Unfurling URLs, Emoji enhancements and video uploads

Here we go! We're four HYS threads in already and you might be wondering just how many there are left. Well, I can't tell you But what I can tell you is - we're not even half way through yet! In case you've not yet seen the previous entries, you can check them out here.

As ever, to ensure you're kept up to date, we strongly recommend caressing that "Watch forum" link and make sure you enable email notifications if you haven't done so already

 

[Chris D]

If XenForo 2.1 will be supporting video playback as it is native to the browser and OS that you are using, will it also support audio uploads?

Theoretically, yes, but we don’t currently.

 

[Mr Lucky]

If XenForo 2.1 will be supporting video playback as it is native to the browser and OS that you are using, will it also support audio uploads?

For us this would open up a can of worms re: copyright.

Whereas most video uploads would be stuff people video on their phones, but in many cases with audio they would just upload their favourite tracks (ripped off CDs etc.) and I can see this being a bit of a nightmare regarding IP. (This is why I do not have audio uploads on Media gallery)

Granted that would be a forum management issue though in enforcing members to only upload tracks they have the rights to, not xenforo's responsibility of course.

But it would help if there could be a tick box in the uploader to confirm you have the right.

 

[Martok]

Theoretically, yes, but we don’t currently.

Ah OK. It's not something I'd particularly need but I was browsing some threads and saw requests for audio support in posts as well as in XFMG so thought I'd check to see if this feature added audio support.

There's a suggestion already here for anyone who wants to support it:

https://xenforo.com/community/threads/attachment-mp3-player.354/

 

[Martok]

For us this would open up a can of worms re: copyright.

Whereas most video uploads would be stuff people video on their phones, but in many cases with audio they would just upload their favourite tracks (ripped off CDs etc.) and I can see this being a bit of a nightmare regarding IP. (This is why I do not have audio uploads on Media gallery)

Audio files can already be uploaded if a site allows them so the copyright issue is there already. I am asking is if they can be embedded and played from the post (for sites that allow audio uploads).

Copyright issues don't just apply to audio files anyway, they can apply to video, audio, images, documents etc. It is something that needs to be in terms and rules for forums and moderated and for admins that don't want copyright issues then the solution is to disallow attachments altogether.

 

[dethfire]

Video uploads would be interesting if attachments could be stored and accessed from a cloud service?

 

[delicatebobster]

Video uploads would be interesting if attachments could be stored and accessed from a cloud service?

Im doing this now on xf1.5
Videos are uploaded to my server and then sent to a amazon s3 server after

 

[dethfire]

Im doing this now on xf1.5
Videos are uploaded to my server and then sent to a amazon s3 server after

custom addon?

 

[Kevin]

Not sure if they host them on a CDN, but individual icons can be obtained as svg or png from their public repository. They are the official android emojis by the way.

Noto is available as a font, SVG, or pngs. https://github.com/googlei18n/noto-emoji I've been updating my site to use their SVGs instead of the EmojiOne images. Backed by Google, they are updated with the current unicodes pretty fast.

Just an FYI for those that are familiar with https://emojipedia.org/ -- The official "Google" emoji shown (eg: https://emojipedia.org/google/) are, like Lukas mentions, the "Noto" emoji project.

Personally I like some of the official Apple emoji best (I know, I know... me an ardent Android & Windows user saying something like that! ) but I think Google is the only major vendor, besides Twitter, who makes their official emoji sets available for anybody to use.

 

[delicatebobster]

custom addon?

yep it was coded in less than 14 days not sure why xenforo cant have this in there core...it seems to me if you ask xenforo everything takes years to code...they are too slow at adding new features....its 2018 and xenforo has no real video upload solutions, no mobile app, no built in image optimization, no hash-tagging, no ecommerce solution... makes it real hard to run my business on xenforo

 

[Kevin]

What on earth is URL unfurling?

I'll just show you. It's easier!

View attachment 185717

When a URL is inserted into content and that URL exists on its own line within the content, we will "unfurl" it to a richer preview which includes the page title, metadata logo, description and favicon. Such rich previews add more context for users as to what the link contains. URL unfurling can be used anywhere that accepts BB code currently.

The functionality is enabled by default, but you can switch it off if you want to in the "Messages" section of the Admin CP.

If you have a minute to spare would you mind satisfying a curiosity item and posting an example of how it would look when linking to an Amazon product page? (eg: https://www.amazon.com/gp/product/B07G4MNFS1)

For those of us that use addons to append our tag to Amazon links I'm kind of curious if this kind of URL "unfurling" would help stimulate clicks instead of just a text link.

 

[Chris D]

If you have a minute to spare would you mind satisfying a curiosity item and posting an example of how it would look when linking to an Amazon product page? (eg: https://www.amazon.com/gp/product/B07G4MNFS1)

For those of us that use addons to append our tag to Amazon links I'm kind of curious if this kind of URL "unfurling" would help stimulate clicks instead of just a text link.

Unfortunately Amazon links just won’t unfurl. It’s the same currently with the existing urlToPageTitle functionality.

The first 100KB (the limit is so grabbing the data is as quick as possible) doesn’t contain any of the metadata we need, not even the title.

I just did a test with the Facebook debugger and it seems there’s nearly 600KB of script and style before the useful tags are available.

I just bumped up the limit on my dev machine but then it also appears to hit the time limit we also have to make sure the request doesn’t take too long.

So realistically it just totally rules it out.

It would make sense for Amazon to have the metadata in their page before their scripts and style but I guess for whatever reason they don’t want to do that

 

[Kevin]

Unfortunately Amazon links just won’t unfurl. It’s the same currently with the existing urlToPageTitle functionality.

The first 100KB (the limit is so grabbing the data is as quick as possible) doesn’t contain any of the metadata we need, not even the title.

I just did a test with the Facebook debugger and it seems there’s nearly 600KB of script and style before the useful tags are available.

I just bumped up the limit on my dev machine but then it also appears to hit the time limit we also have to make sure the request doesn’t take too long.

So realistically it just totally rules it out.

It would make sense for Amazon to have the metadata in their page before their scripts and style but I guess for whatever reason they don’t want to do that

Thanks for taking the time to both test it out and to give the details as to what the issue you ran into is. Your answer also helps to understand a bit better exactly what the expectation will be when using this feature.

You're right, you'd think that Amazon would try to make it as easy as possible to link to them and so something like metadata should be a given for their product pages but, alas, they like to throw us a curveball.

 

[Ozzy47]

but I guess for whatever reason they don’t want to do that

They do it that way just to irritate @Kevin

 

[Jake B.]

yep it was coded in less than 14 days not sure why xenforo cant have this in there core...it seems to me if you ask xenforo everything takes years to code...they are too slow at adding new features....its 2018 and xenforo has no real video upload solutions, no mobile app, no built in image optimization, no hash-tagging, no ecommerce solution... makes it real hard to run my business on xenforo

This technically would already be possible since they use Flysystem for file management in xf2. https://xenforo.com/community/threa...ations-for-cloud-storage.135150/#post-1180295

Just a matter of loading in the aws SDK

 

[Chris D]

Thanks for taking the time to both test it out and to give the details as to what the issue you ran into is. Your answer also helps to understand a bit better exactly what the expectation will be when using this feature.

You're right, you'd think that Amazon would try to make it as easy as possible to link to them and so something like metadata should be a given for their product pages but, alas, they like to throw us a curveball.

Just wanted to expand upon something we discovered earlier after some further testing. Amazon are actively preventing sites from being able to crawl metadata from their page. They might be doing this based on a whitelist of approved user agents.

If you use the Facebook debugger, you can see the metadata in their source. It's still stupidly far into the source, and too far in to be feasible for us to be crawling unless we want to accept the increased load times when they're fist crawled.

However, if you use the XF HTTP client to crawl the metadata and you essentially remove all of the limits (so we download all 1.6MB of a typical product page... yes... nearly 2MB...) then there is no metadata there. All you get is the title and description, and that just comes from <title> and <meta name="description"> tags. There's no og:title, og:image etc. like there is when Facebook crawls. There’s not even a favicon. They are actively not presenting that to most users.

Really annoying. I guess we could spoof the Facebook UA (though that's clearly a bit dicey ). It might not even be UA based, could well be whitelisted by IP address.

 

[delicatebobster]

I know you said you're still on 1.5, but if you ever decide to make the plunge to XF2, there's always these solutions:
[DBTech] DragonByte User Tagging
[DBTech] DragonByte eCommerce

If I was to hazard a guess - bearing in mind I do not work for XenForo so I have no way to know how they decide which 1st party add-ons to produce - the reason why they haven't created a 1st party eCommerce tool is that it has limited use. I would wager that the percentage of forums that have no use for an eCommerce tool is vastly larger than the percentage of forums that do have a use for an eCommerce tool.

On the flip-side, the percentage of forums that would have some use for a Media Gallery and/or Resource Manager / download repository (without payment) is probably larger than the percentage of forums that have no use for such tools.
Every forum could probably benefit from having a gallery (gaming forums: game screenshots. car forum: ...well, cars. etc).

I totally understand how frustrating it can be to have to go to 500 different sites to buy 500 different products rather than just being able to have everything you need as a 1st party solution, but I think it's important to look at a potential lack of ROI as part of a reason why they might not have created such a 1st party tool yet.

I think this could be an interesting discussion, so if it's something you want to discuss further we should probably take it to another subforum so as to not derail this thread too much


Fillip

In 2018 these items should already be included in the xenforo core

 

[Kevin]

Really annoying. I guess we could spoof the Facebook UA (though that's clearly a bit dicey ). It might not even be UA based, could well be whitelisted by IP address.

Nice catch! Poking around it looks like they're using the UA. Besides using FB's Debugger (link for anybody curious) the same result can be observed in Chrome's Developer Tools by using GoogleBot as the UA.

Again, if anybody is curious (if you're not curious just ignore this part)....

• Load the page source in Chrome with "view-source:https://www.amazon.com/Sony-WH1000XM3-Canceling-Headphones-WH-1000XM3/dp/B07G4MNFS1" and then do a CTRL+F and look for <meta property="og: and you won't find anything.
• Now go into your dev tools (from the menu or CTRL+SHIFT+J on Win) then from the 3-dots drop-down menu choose "More Tools" and then "Network Conditions".
• The "User Agent" will be pre-checked on "Select Automatically". Uncheck that and from the dropdown box choose "Googlebot".
• Now reload the page and do your CTRL+F again and search for the same thing.
  • Surprise, you now see the OG tags for URL, Title, Description and Image! The linked image is pretty interesting because besides including a thumbnail of the product it is dynamically also including the number of product reviews and customer rating.

 

[Jeffin]

HYS has become by favourite forum section on Xenforo. Thank you @Chris D for all your contribution!

 

[Chris D]

HYS has become by favourite forum section on Xenforo. Thank you @Chris D for all your contribution!

Team effort of course

 

[RobinHood]

View attachment 185717

Does this automatically fetch and embed when composing a reply? Or only only after post submission?

I really like this, but it could get annoying if pasting lots of links that you simply want to display as text links.

Is there the ability to cancel or remove it? Like so:

There are no fancy HTML5 video players, there's no complex dependencies, and there's no performance taxing transcoding process. Everything above is totally native to the browser and OS that you're using.

Looking forward to this immensely, but I do also hope this gets developed to more advanced standards in future releases. What with 4k, 60fps and Slo Mo video being standard on most decent smartphones these days, file sizes even for short videos can be huge.

It's one of the big reasons IMO for a dedicated app, so that the video compression can be done client side on the phone before uploading to create a better user experience for the user (and less taxing on the forum server), however transcoding after the fact would still be incredibly beneficial for both admins and other users streaming the freshly uploaded video. If the freshly uploaded video is a very large uncompressed raw file from the phone, then many internet connections may not be able to stream it smoothly.

Loving the emoji search box too with fuzzy search, great addition!