Review resources before approving them (XF Community)

[Freelancer]

As for everything else that has been discussed, I just wanted to make sure that people know that we're watching the feedback in this thread very closely, and although we don't have any direct answers right now, I'm certain this will be a point of consideration and discussion in the future. I think the previous several pages have demonstrated that it is a complex issue with no clear solution, so bear with us.

That’s much appreciated. Any thought on this is welcomed. And the users that criticize some parts do only do so because they want to bear with XF and improve things.

 

[Martok]

Not sure for now, but when I submitted my plugin in 2015, I got the bot's report of 191 errors/22 warnings with a long list of every single file with problem plus the pieces of code and suggested changes a little bit after submitting. If that was done by a staff manually, hats off!

There is some very basic automated checking but beyond that it is manual checking. Details here:

https://docs.moodle.org/dev/Plugin_validation

 

[Dadparvar]

There is some very basic automated checking

Great. We reached to a common point. At least a basic auto check. This can include checking for raw queries, some sort of callbacks, and other things that you may know better than me.
(although this topic is not limited to just this)

 

[Martok]

At least a basic auto check. This can include checking for raw queries, some sort of callbacks, and other things that you may know better than me.

What you are beginning to ask for is more complex than the automated checks on Moodle plugins. Most of those checks are very simple things like plugin name, single zip file, language file present. From what I can tell from that validation page, there's only 5 checks of actual code:

• Language file must declare either $string['modulename'] (for activity modules) or $string['filtername'] (for Filters) or $string['pluginname'] (for all other plugin types). See exceptions below.

• $module->version defined in version.php
• $module->requires defined in version.php

• file lib.php contains “function xxxx_add_instance”
• file lib.php contains “function xxxx_update_instance”

It would be enough of a job to create an automated checker just to check basic things like this. To add more complex checking would be more difficult to do and, in many cases, impossible. Ultimately any code checking is going to need to be dealt with by a human being (which is what happens on Moodle even after the automated checking). That involves time, human resources and ultimately money, which would mean a price rise for XenForo (and possibly add-ons if developers are charged for this service). Whilst some of us would be happy with this, not everyone would be.

 

[The Sandman]

I just wanted to make sure that people know that we're watching the feedback in this thread very closely, and although we don't have any direct answers right now, I'm certain this will be a point of consideration and discussion in the future. I think the previous several pages have demonstrated that it is a complex issue with no clear solution, so bear with us.

I think there is currently no clear solution because of the underlying conflict of allowing unvetted, troublesome code to be either located on or promoted from XenForo.com (and thereby being tacitly supported and approved by XenForo) . Either the code should be checked in some fashion or it should be located and promoted elsewhere. Otherwise, it will continue to cause problems that can reasonably be blamed on XenForo.

 

[Chris D]

I literally couldn't disagree more.

First, we make it clear that resources are not reviewed nor affiliated with us via a notice which displays on the resource:



Second, there's no clear solution because, as I alluded to earlier, it is a complicated issue which certainly doesn't have the black or white solution which you imply.

 

[The Sandman]

First, we make it clear that resources are not reviewed nor affiliated with us via a notice which displays on the resource

Yes, the disclaimer is there to deny responsibility. From a legal standpoint it may or may not have effect depending on the circumstances. In and of itself the disclaimer does not mean that XenForo customers will not blame XenForo to some extent for troublesome code hosted and/or promoted here.

Second, there's no clear solution because, as I alluded to earlier, it is a complicated issue which certainly doesn't have the black or white solution which you imply.

I guess semantics come into play here - are you saying it's a complicated issue to decide whether or not to review add-on code as the OP suggests? Or that it's complicated to decide how to review the code once the decision has been made to do so? The first question is black and white, the second clearly more complicated.

 

[we_are_borg]

There’s still no answer to the question I posed here earlier:

What about paid resources that aren’t hosted on this site?

That’s easy they either submit there addon for review or there not welcome to promote their add-on on XF.

 

[empire]

Maybe instead of checking each add-on, there could be some process for individual developers (not teams) to earn some kind of Xenforo certification. If the developer had passed some process that showed they understood how to code add-ons that meet certain standards, then I'd be more likely to trust them.

The problem of course is that it still wouldn't prevent developers from abandoning add-ons or being lazy about keeping high standards on custom development.

 

[Bionic Rooster]

Maybe instead of checking each add-on, there could be some process for individual developers (not teams) to earn some kind of Xenforo certification. If the developer had passed some process that showed they understood how to code add-ons that meet certain standards, then I'd be more likely to trust them.

That would put the onus on XF which they clearly don't want and I don't blame them. To take a hand in certifying anything/anyone beyond their control is begging for trouble.

 

[Slavik]

Yes, the disclaimer is there to deny responsibility. From a legal standpoint it may or may not have effect depending on the circumstances. In and of itself the disclaimer does not mean that XenForo customers will not blame XenForo to some extent for troublesome code hosted and/or promoted here.

What an bizarre point of view, I dont think we have had anyone blame XenForo in this manner ever? Its very obvious what it is stating. You have to really try hard to take it in any other way.

I guess semantics come into play here - are you saying it's a complicated issue to decide whether or not to review add-on code as the OP suggests? Or that it's complicated to decide how to review the code once the decision has been made to do so? The first question is black and white, the second clearly more complicated.

We could implement an external (or internal) code audit on an hourly basis very easily. But who would pick up the cost? When something was suggested similar last year via some sort of certification, there was a rather vocal outcry that people would feel disadvantaged or unwilling to pay for it.

There is an alternative I talked to a few devs and providers about a while back, about a blind peer reviewed certification system. It seemed to have quite a lot of support on the basics.

 

[The Sandman]

What an bizarre point of view, I dont think we have had anyone blame XenForo in this manner ever? Its very obvious what it is stating. You have to really try hard to take it in any other way.

Again, semantics - to blame is to assign responsibility for. If people didn't think XenForo had some responsibility for the code hosted/promoted on XenForo.com why would they be asking that XenForo review the code? I don't see people asking for code reviews of add-ons being hosted or promoted elsewhere. Having the add-ons here implies some degree of approval by XenForo, as evidenced by adding warning labels to some add-ons and removal of some add-ons by XenForo.

 

[Chris D]

You're implying that we are responsible for code hosted here. We're not, and that's clearly stated. People could argue that if they wish, but if someone else writes the code, they decide to use the code and ignore the disclaimers while doing so, there's not really much that's going to stand up there. If someone puts the wrong fuel in their car, is the car manufacturer responsible when the engine breaks?

You're analysing it wrong and looking for an argument that isn't there. No one else has implied that we are responsible for the code being distributed here. People are, however, asking that we take responsibility for it. The lack of responsibility that we have is the implied problem here.

It doesn't seem right for us to argue that point particularly much further though. I've already stated that it could be something that we give consideration to in the future, until then, there's really not much more to say.

 

[Slavik]

There are no semantics, only the ones you are inventing. The statement is the statement. Nothing more. Nothing less.

People are not asking for code review because they are assigning responsibility onto XenForo for allowing them to be listed, they are asking for it as a method of improving code standards moving forwards. If XenForo was involved or not, the case for code review would remain the same.

 

[Member_68162]

WoltLab only checks plugins that are hosted on their server (their website). How would XenForo check every paid add-on that has to be downloaded elsewhere on the internet? Would add-on developers no longer be able to advertise their add-ons on xenforo.com unless they were inspected first? How would updates be checked if the add-ons are hosted elsewhere? There wouldn't be anything to stop a developer from changing his or her code after it was approved if their add-on wasn't hosted here.

 

[AndrewSimm]

Perhaps the start is moderating how developers classify their add-ons. There is one particular developer who while has some add-ons, also has a lot of template modifications classified as add-ons. It's shame that a Chat mod that supports multiple rooms, PMs and sanctioning gets classified the same as an addon that removes the register button. Also better standards on what is displayed on the description page and lack of screenshots. Again a start is simply moderating resources the same as you do the board. If you don't blame on doing that then introduce a feature that allows me to favorite developers or ignore them.

 

[Jake B.]

If you don't blame on doing that then introduce a feature that allows me to favorite developers or ignore them.

The normal ignore feature will hide resources by that user as well as their posts/threads. I think it'd be nice if the "follow" feature in XenForo had a bit more functionality though to give you the option to be alerted when they created a new thread/post/resource/other content type. Or even if ignore gave you the option to select the content types you ignore so you could ignore media and resources by a user but not their threads and posts

 

[Alpha1]

It does lead to empty pages, but that is another topic.

 

[Robust]

That’s easy they either submit there addon for review or there not welcome to promote their add-on on XF.

Now you’re just over simplifying the issue.

You’d require all add-on sellers to post a file here for every update as well, and you have no way to confirm that’s the same file being given to customers without wasting even more time.

“Not welcome on XF” - some are arguing there’s a shortage of developers and quality add-ons as it is. Not sure you want to be banning multiple developers because they won’t submit files for review (I doubt any of them will have an issue with it, I wouldn’t, but the above point still stands).

Edit: Literally the only way to get this to work properly is to make add-on developers upload their content and manage it here. I’d love that from a UI standpoint and am sure it would assist sales, but that’s increasing burden significantly on XF and more importantly some developers (ThemeHouse springs to mind, ourselves as well) have invested in custom panels to deliver a better experience. The panel on nanocode.io actually complies with various laws too, namely VAT MOSS and issues compliant invoices. There’s way too much this site would need to do for legal compliance to make that worth it, and even then some developers would be angry they’ve invested time in their panels that they can no longer use.

So you’re stuck to the problem where add-ons are hosted externally and we go back to problem 1.

Honestly, certification is the only real solution I see here.

 

[we_are_borg]

Now you’re just over simplifying the issue.

But the issue is simple.

You’d require all add-on sellers to post a file here for every update as well, and you have no way to confirm that’s the same file being given to customers without wasting even more time.

Ever heard from a checksum, its also the trust between customer and developer. Do you think people would trust someone that say its checked by XF see this checksum and file and it did not add up.

“Not welcome on XF” - some are arguing there’s a shortage of developers and quality add-ons as it is. Not sure you want to be banning multiple developers because they won’t submit files for review (I doubt any of them will have an issue with it, I wouldn’t, but the above point still stands).

What if you had a developer that did not make quality add-on do you then still want his/here add-ons. Your only saying that they cant promote there add-ons here not that they are bad developers. I would rather see that add-ons have a standard check then non at all it does not have to be a indepth check.

Edit: Literally the only way to get this to work properly is to make add-on developers upload their content and manage it here. I’d love that from a UI standpoint and am sure it would assist sales, but that’s increasing burden significantly on XF and more importantly some developers (ThemeHouse springs to mind, ourselves as well) have invested in custom panels to deliver a better experience. The panel on nanocode.io actually complies with various laws too, namely VAT MOSS and issues compliant invoices. There’s way too much this site would need to do for legal compliance to make that worth it, and even then some developers would be angry they’ve invested time in their panels that they can no longer use.

So you’re stuck to the problem where add-ons are hosted externally and we go back to problem 1.

Honestly, certification is the only real solution I see here.

No thats not needed after the check and if the developer you get a checksum so if they want to host it on their own site they can. When you have the file as customer you can go to the add-on page here on XF press the button check checksum you upload and it either says agree or failed. You can also do the checksum on your machine and compare. So for the add-on developer it stays the same there is only a step in between. You can even do the following you know about WHQM drivers of Microsoft well a file checked is XFC. You can then say this file is XFC as on XF website and this file is not checked (yet) its RC for now later you can say its now XFC and this is the checksum as on XF website. So from developer standpoint you only have a extra step and on their site they can say this is unchecked version and will be the next checked. Also as developer you can do at least three things first check add-on and see if you have issues but not publish. Second check add-on and publish and get checksum, third upload add-on as beta and still get checksum but its shown as either beta or rc. This system is to gain trust between developer and customer so that they see the developer did everything to pass the test and not be a burden, so it needs to be good for developer and customer.