Review resources before approving them (XF Community)

Status
Not open for further replies.
WordPress is a much, much bigger community than XenForo. We have < 20 active XenForo add-on developers that publish somewhat regularly here, and 20 is perhaps still being kind, it's probably around 15.

Sites quote things like:


It's a market with competition, lots of demand, plenty of developers and hence manpower and requirement for staffing and review.

Let's not compare against something on a very different scale. Strict regulation being applied is *not* going to attract more developers into this community and that should be what customers want above anything else. I'd start with clear documentation and a clear list of guidelines that resources should meet (again, lightly enforced as it is now) and see where that gets us. That should be the first priority for XenForo before certifications or any of the other suggestions in this thread.

Again, there isn't really a big issue with the RM.

I know this. I was simply pointing out how they operated the review process since there seemed to have been some confusion.
 
Just so we are clear how how WordPress, manages the reviews, the fact of the matter is. They don't, not directly at least. there a a core team of volunteers that handle it. WordPress simple provides the means and the space where it is managed from. but it is a team of unpaid supporters that does all the work.

Wow, according to this it seems in 2014 when there was 30K plugins available there was only two people doing the majority of the plugin reviews? They must have a pretty great workflow and system in place to be able handle all those submissions.

https://pippinsplugins.com/tips-getting-plugin-approved-wordpress-org/
 
It's a good job WP have that review and approval system in place, otherwise it would potentially result in insecure code, vulnerabilities, exploits, sites and servers being hacked, etc.

At least now people can install third party add-ons safe in the knowledge that there are no issues with them, and if there were to be, they could hold WP accountable.

Oh wait ...
 
Can you imaging what would happen if they did not review the code on Wordpress. They have now issues with security even when they review.
 
At least now people can install third party add-ons safe in the knowledge that there are no issues with them, and if there were to be, they could hold WP accountable.

Oh wait ...
Yes, it will be entirely bulletproof and move responsibility from developers to xenforo. lolol. :Do_Oo_O
 
30 plugins reviewed in an evening? Surely those would be equivalent to some template mods here...

Imagine having to go through a add-on with 1000 lines or so along with updates. There would need to be some sort a diff comparison so no time is wasted on files already approved.
 
Some interesting comments by Woltlab staff on TAZ:
dtdesign said:
Our Plugin-Store has a few advantages over XenForo.com's approach:
  • Every submitted files is manually checked by a staff member, this includes checks for common security issues (such as XSS) and other potential problems
  • Call-home functions or any other comparable functions are strictly forbidden and such submissions will be immediately rejected
  • All payments are processed by us, at no time will you make payments on 3rd party websites
  • All files are hosted by us, no matter if the vendor's own page has issues, the downloads will remain available
dtdesign said:
There is an automatic system in place but its sole purpose is to perform an overall validation of each submitted package. This includes checks for file existence, unnecessary files (Mac OS X's ._* or .thumbsdb on Windows) as well as PHP and XML syntax checks. To sum it up, it only checks if the package looks like it could be installed successfully and does not clutter the software with garbage files.

Once a file has passed this check it is queued up for review which is entirely a manual process performed by our staff members.
Ozzy47 said:
Interesting. How long does that take, let's say for a medium sized mod? What is checked, every line of code, or just common stuff?
dtdesign said:
I find it difficult to spit out any number for this, because the amount of code does not really affect the time required to perform a full check, rather than its complexity. While this might sound odd for most people, it actually makes sense: If your code contains a lot of generic business logic such as calculations, we can quickly skim that part.

On the other hand we carefully examine all those parts where data is written or updated in the database (thanks to real prepared statements SQL injections are a non-issue) to see if it does something unexpected, e.g. updating a database table while missing conditions and suddenly overwriting everything. Another important section is the display of user-provided data in templates, XSS is still something people are not aware of in 2015. Thankfully all variables that are used for output in templates are already sanitized by our template engine unless the developer explicitly opt-outs single variables ("Hello {$user}" will be automatically sanitized, "Hello {@$user}" will prevent this).

So while we do not check every single line, our experience allows us to quickly identify critical components and focus on these. Our goal is to ensure that plugins adapt the general UI/UX and contain no critical bugs that could harm the software. And last but not least, any sort of callhome function are a big no-go and result in an immediate rejection.
Ozzy47 said:
How much does this cost the person releasing the mod?
dtdesign said:
These checks are non-optional for all packages uploaded to our Plugin-Store, this includes both free and paid mods, and because of this free of charge. Being able to provide a trusted platform with consistent and secure plugins is far more worth than the extra time we have to invest for this.
https://theadminzone.com/threads/*******-at-it-again.136863/page-8#post-1016030
 
First, we make it clear that resources are not reviewed nor affiliated with us via a notice which displays on the resource:

View attachment 161138
Yes, the disclaimer states that resources are not reviewed by or affiliated with XenForo Ltd... but not that they might be dangerous. It doesn't mention the possible consequences of installing unreviewed code or how to determine if a download is safe. In my opinion it should, and members should not be able to download resources hosted here or link to resources hosted elsewhere until they indicate acknowledgement of this. It might not occur to some XenForo users that XenForo Ltd. would have dangerous downloads available on XenForo.com.
 
Last edited:
To be honest, I wouldn't expect Wordpress or XF or any other platform that has plugins to be able to thoroughly scan or examine every single add on and completely rely on that system wholly anyway. I think some of the steps we've seen other platform taking here are great, and the developer standards and guides are wonderful for WP to try and keep the quality high, but I would never expect them to take the full burden of responsibility for user submitted (plugin) content either.

Rogue or dodgy ones will slip through, and admins will still install plugins that are hosted and purchasable off site for whatever reason. Many premium plugins aren't hosted in or visible in the WP directory. As with any app or plugin in any app store, whether it's the Apple App store, Play store, Wordpress.org or XF.com, you should always be asking yourself is that piece of software safe to install on my device or platform? The two most important signals a user has after any automated or manual repo submission checks there are (if any), is that of community reviews and install numbers.

On the WP platform, once it's installed, even if it's not in the directory you can still see a decent amount of info about that plugin.

Take the WP Rocket example from before. It's a premium, paid plugin, not available when searching in the directory so I had to discover it via blogs and recommendations. I visited their site, did my own research, it seemed to come highly recommended and so I bought and installed it.

Now that it's installed it still integrates incredibly tightly however the WP system is setup. I can still see reviews on the plugin from within my ACP, I can see how many active installs there are for the plugin in order to gauge it's popularity. And even though my licence has expired I can still see the change log for the latest updates, when the latest update was published and other info. (I'm not 100% sure where the review data is coming from as it doesn't link off to the normal plugin repo review section it as with the hosted plugins, I'm not sure if this is because my licence is expired or not, I don't have any other paid plugins to test this with)

I'm not sure if Wordpress has ever scanned any of the WP Rocket code, as it's an unhosted premium plugin that users have to purchase direct from the developer and manually install at first. After that though, updates are pushed through the ACP, I can still access reviews and if there's any issues that other users pick up on I can pick up on that from the review. Say we get another ******* situation and 20 admins suddenly rate the add on as 1*, then my attention would be drawn to it in the review section when I check my ACP and update my plugins every couple of weeks.

Some kind of submission checks are all well and good, but in the end there needs to be some kind of community feedback system for all plugins where admins who have the plugins installed can get some kind of signal on their own site when logging in, if something has been flagged as dodgy, dangerous or just poorly coded so they can make an informed decision as to whether they want to continue using it or not.
 
Just curious, all these suggestions about reviewing code, yet no one has stepped up and attempted to do it on their own. I'm sure an admin site or some type of (dot) org could surely attempt it. I've seen the excuse that "blah blah blah ... they are there to share, that just how in the world are they supposed to vet the code, they couldn't possibly have the time, who's qualified, etc, etc, etc, etc..." I think I have figure it out though, everyone wants the responsibility to fall on XenForo. The same XenForo team most of you bash and moan about when the releases of their software aren't out fast enough.

Yet now, you want them to either review code submitted to them or apparently hire someone to do it so you all can feel the warm and fuzzy cause they are hosted here? And those that aren't what, they now can't be shown here because *gasp* Team XF hasn't went through the code? Give me a break. Seriously.

There is about 7+ pages of absolute nothingness here, with a few common sense replies sprinkled in. Those that replied with a bit of sense, thank you for showing others that they are perfectly capable of reading, and understanding the notice that states:

Resources listed here are not affiliated with and have not been reviewed by XenForo Ltd. If you have any questions regarding a resource, please contact the author. XenForo Ltd. is not involved in any resource-related transactions.

Screenshot_12.webp

I'm not certain how that can be any more telling, any more informative. Read. They cannot hold your hands anymore.

If XF reviews the code, it passes their check, and *gasp, a security issue shows up... then what? Tell me, please. Are we supposed to sue XF now? I mean, they are some of the most talented developers I've seen, and if I recall, they have had to issue a security update themselves. So, they review, it passes, you get hacked from that add-on somewhere down the line, what exactly is the solution from that point? That's what I would really love to know.

In the end, aren't we all responsible for our OWN sites? Aren't we supposed to vet these addons, vet the code, read and review, trial on a test site, etc? I realize most don't, simply for time.. yet, again, that same time you want used here by the developers of this software. What's next, they have a CSS expert look over your site upon submission to see what can be fixed, what errors are there? This is about the most jacked up snowflake discussion I have read in quite some time.

Again, if someone has an answer to exactly what everyone with the torches and pitchforks will do if an add-on passes this newly created add-on vetting, and ends up being hacked or site blows up for whatever reason... what exactly the next step is.... happens to interest me. Class action lawsuit, what? Please inform.

Careful what you all wish for.
 
Yes, the disclaimer states that resources are not reviewed by or affiliated with XenForo Ltd... but not that they might be dangerous. It doesn't mention the possible consequences of installing unreviewed code or how to determine if a download is safe. In my opinion it should, and members should not be able to download resources hosted here or link to resources hosted elsewhere until they indicate acknowledgement of this. It might not occur to some XenForo users that XenForo Ltd. would have dangerous downloads available on XenForo.com.
Seems like you're just picking arguments.

Seems fairly reasonable to assume that if there's a disclaimer stating that the resource has not been reviewed, then a risk of results exists. The same as when you buy foods that aren't reviewed by the FDA: "This statement has not been evaluated by the Food and Drug Administration. This product is not intended to diagnose, treat, cure, or prevent any disease". You are made to realise that this means that the contents are not reviewed. Substances in the past have been banned containing illegal substances, in some cases causing death (for example Jack3d) but they still were not required to have the disclaimer: "This statement has not been evaluated by the Food and Drug Administration. This product is not intended to diagnose, treat, cure, or prevent any disease. This product may be dangerous. The FDA is not liable for any death caused by consumption of this product.". The same applies for unregulated financial companies and whatnot. The lack of regulation is shown, the risks are meant to be assumed by the reader.

You're the only one to claim that this risk is not understood.
 
Just to note, WBB has a total of 640 plugins, 219 styles, 27 development and 23 applications. In comparison, XenForo has over 2000 add-ons for 1.x alone and 154 for 2.x already, plus a lot of other resources besides:

1510138133427.webp

Reviewing this amount of resources would be some serious undertaking for just 3 developers when we also expect them to deliver XF2 right now and then expect 2.1 very soon with lots of added features. Yes, they can take on more full time staff to do this but that would mean a price increase for XenForo to cover the increased staff costs, something that I know some people wouldn't be happy about.
 
If Xenforo would review resources then likely a large number of addons would never have been approved. For example: themehouse recently marked a very large number as unsupported or deleted them because those did not meet their own quality standards.

There are also a lot of malfunctioning add-ons in that total of 2000. The issue there is that such add-ons do occupy a slot that may otherwise have been filled by another developer.

@Martok why do you think that additional costs would require a raise in price? Has this been stated by xenforo somewhere or are you just assuming this?
Software is not a tangible good with a solid production price . Instead it has a variable production price per license which is dependent upon total licenses sold, which is dependent upon optimal price mark.
 
  • Like
Reactions: RDR
I think it's important to note that reviewing code is going to be nowhere near the bit that takes the most time. Instead it will be the inevitable back and forth after you reject an add-on. It might take no more than 5 minutes to ascertain that an add-on doesn't meet the expected standard. But it will take a lot longer to explain why it doesn't meet the expected standard and what actions are required to meet that standard.

Please don't see this as a precedent but over the last 24 hours I actually reviewed the code of an add-on. This was actually more of a coincidence more than anything. It wasn't a conscious decision that I wanted to review it, it was just something I spotted. There were some issues with how the extension of our code was taking place that would likely cause problems down the line. It took maybe 3 minutes for me to ascertain why it didn't meet the standards and why. However it took maybe 15-30 minutes for me to convey in appropriate terms what needed to be changed. The initial concerns were resolved, but after reviewing the changes again I had some other changes to recommend and that, again, took another 15-30 minutes and it's possible that there could be further queries off the back of that.

Which brings me on to this:
If Xenforo would review resources then likely a large number of addons would never have been approved. For example: themehouse recently marked a very large number as unsupported or deleted them because those did not meet their own quality standards.
I disagree. That large number of add-ons would have been approved. Eventually. To get there, it would likely take many hours of discussion and further reviews. You might think that perhaps it would just be a case of rejecting something and giving a high level reason, but that explanation likely won't be detailed enough for the developer to know what actions are required. Even if we're strict on that, there'll always be some level of subjectiveness to our reasons which I'm sure will result in some lengthy conversations probably exceeding the paltry few minutes it would take to review the code in the first place.
 
Status
Not open for further replies.
Top Bottom