XenFor with node.js (for grins)

[ForestForTrees]

Heya, folks,

For those not familiar, Node.js allows you to write applications on the server in JavaScript. It's been a hot app recetnly, and it's beyond the scope of this thread to explain Node.js in depth. However, because of the non-blocking way that it handles I/O, it's great for communicating with Javascript in the browser, giving JS in the browser a good way to communicate with the server to share data (AJAX).

Suppose there were an option in the XenForo Admin interface to enable Node.js integration. What new things might then become feasible in future versions of XenForo or in addons?

I've put this thread in "off topic" and put "(for grins)" in the title because I don't see it happening. But I'm curious about what might become possible, and sometimes it is valuable to think about what we would do with power that isn't currently on the roadmap.

I would imagine a separate node.js application that would have access to the same SQL database that XenForo uses, but if there are other ideas, I'm all ears. It only takes a couple of lines on the command line to install node.

Thoughts? Any cool features that might become feasible? I've got a couple in mind, but want to know what the hive mind comes up with.

 

[ForestForTrees]

Tough audience....

One application would be that it would be easy to code a chat room or shoutbox.

I was thinking it would be nice for when new posts are added to a thread or new replies to a profile post. Basically, the site could live-update more like other newer sites do. This came up in the XF 2.0 thread, but IIRC someone nixed it because people wouldn't have access to the proper server.

 

[Jeremy]

Live updating would only work via threads being served by node and not PHP do live application listeners could be added that receive new post alerts asynchronously. Mode is also compiled versus interpreted in node. Honestly, I cannot think of a single core application that would improve greatly via a nice component without essentially switching to pure node.

 

[Anthony Parsons]

I tried node.js in another CMS... and to be honest, it was terrible. Inflexible in essence for the wide majority of mobile devices used today, clunky and horrible, to say the least. I wouldn't recommend it at all.

 

[ForestForTrees]

Live updating would only work via threads being served by node and not PHP do live application listeners could be added that receive new post alerts asynchronously. Mode is also compiled versus interpreted in node. Honestly, I cannot think of a single core application that would improve greatly via a nice component without essentially switching to pure node.

Saying that live updating wouldn't work unless the threads were served by node is a pretty strong statement and sounds fishy to me. Node, PHP, and the browser-based javascript are all Turing complete and can communicate with each other, so you should be able to do whatever you want. It might require a big rewrite of the PHP code, but this thread is only for grins, so I'm not concerned about that.

I tried node.js in another CMS... and to be honest, it was terrible. Inflexible in essence for the wide majority of mobile devices used today, clunky and horrible, to say the least. I wouldn't recommend it at all.

Do you mean that the CMS you used was inflexible, clunky and horrible for mobile devices or that node was? It doesn't seem fair to judge a programming language based on one CMS written in it. For example, I'm sure that there are plenty of terrible projects written in PHP, but I don't think it's a bad thing for XenForo to be written in PHP.

 

[Jeremy]

I worked professionally as a node.js and PHP developer working on Fortune 500 sites such as Costco.com for years, I learned the limitations of the languages extensively and wrote large, asynchronous applications in node and the same application type in PHP.

It's not fishy what I said, it's the truth. PHP is a preprocessor and once it runs through the code, it's done, finished, no longer connected to the server. The page is built and served to the client browser.

JavaScript, by its nature, is an event driven language that works in an asynchronous manner. An application written in JavaScript can be served and maintain a connection to a server while awaiting a notification before performing an action.

 

[ForestForTrees]

That's true, but what stops a web page that has been constructed by PHP from making a connection with node to exchange further data, like what Wikipedia/Mediawiki does with their new visual editor? The core wiki pages are rendered in PHP, but when you edit using the visual editor these pages access a node.js web service called Parsoid. A live updating service for a forum would of course be different, but the principle of a web page using javascript to speak to node is the same.

Of course, node would need access to the same SQL database that PHP was using. However, this seems no less realistic than expecting people to install node and a node package when Xenforo Ltd, for excellent reasons, doesn't even want to raise the PHP version too high. Again, this thread is "for grins."

The architecture doesn't seem like it would be that hard. Basically, the javascript on a thread_view page would initiate a connection with the node server and would tell the server its thread id and the last time it was updated. If the thread has been updated since that timestamp, the thread_view page sends its state information to the node server, which queries the SQL database to send a JSON object to the thread_view page letting the page know how it needs to update itself. It's just basic one-directional synchronization.

I don't mean to suggest that it would be easy, but the hardest parts are inevitable with live updating. That can't be blamed on node. I bet that what I've described it would significantly cut down on server load for live updating on large sites. Node's creation was inspired by progress bars, so why couldn't it be used for live updating?

Plus, live updating would be awesome.

 

[ForestForTrees]

Here's another idea. I never use the news feed because I can't be bothered to click on it:
https://xenforo.com/community/account/news-feed
(Does anyone else use this?)

However, if there were an option to turn on a live-updated version of the news feed in the right of every page of the forum, for me at least, that functionality would change from being unused to a significant engagement-increaser. I follow @Mike, and while I'd never bother looking at my news feed to see if he's posted anything, if I saw some movement on the right hand side of my screen from a live update, I'd totally check it out.

This would be a bit harder because there are security concerns. You'd have to be clever to come up with a scheme where the webserver (PHP) would include a security token in the page that the in-browser javascript could use to authenticate with, perhaps based on existing session info, cookies, IP#s, or just a long random string. It might take some cleverness, but I bet it could be done.

 

[Jeremy]

Long polling servers like what you want is extremely server intensive and would need to be implemented via client side scripting...

 

[Xon]

Additionally something like CloudFlare only supports requests which last ~100 seconds or so and only permit websockets on their Enterprise plans.

 

[Anthony Parsons]

or that node was

Node.js powered the entire thing... and it is terrible. There's good reason why javascript enhances PHP and not the other way round.

 

[Daniel Hood]

Long polling servers like what you want is extremely server intensive and would need to be implemented via client side scripting...

But he wants to happen with node and websockets and computer magic instead which is so much less server intensive.

 

[Rob]

I think more and more web apps will make the move to client side. Xenforo will one day, for sure, its a inevitable consequence of progress. This is a good thing as servers will only need to enforce referential integrity and business logic leaving everything else to the client. It is going to happen, look at gmail.... mostly client side.

 

[ForestForTrees]

I agree, Rob.

I'll respond to others later, but Anthony, do you have experience developing software for node or are you just speaking from your experience with that CMS? Do you understand what I mean when I say that you shouldn't judge an entire platform based on just one app?

If people want to check out an application where node shines, consider etherpad lite. I've used it in a couple of my projects and think it's great. It's like Google docs in that it allows collaborative editing of documents by any devices connected to the internet. The difference is that it is open source and much simpler because it focuses just on the basics of text rather than attempting to be a full word processor.

Since it's nice and simple, I find it great for embedding in web pages. What makes node perfect for it is the steady flow of relatively small packets of data required to keep several text editors in sync.

People seemed to like some videos I made in another thread, so I made one illustrating Node. Of all the videos I've made I think it's the most interesting. I apologize in advance for the noise I made during the iphone demo at around 6:33 - you might want to turn your speakers down for that:

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

​

 

[ForestForTrees]

Long polling servers like what you want is extremely server intensive and would need to be implemented via client side scripting...

Is there a significant downside to having the connection initiated by the client?

Is there another recommended design for live updating?

If I'm wrong about my impression that Node would be better than a web server for this type of thing, then I'd like to understand why. Certainly, my experience with Etherpad-Lite, which I described in the video above, has left me quite impressed with the product.

 

[Rob]

.... and then there is ember

 

[ForestForTrees]

.... and then there is ember

That sounds potentially interesting. I gather that Discourse uses it. What in particular interests you about Ember?

I like the idea of having a thread that is just for fun where people can discuss other technologies that realistically are highly unlikely to be included into XF, just for entertainment value and to stretch our minds.

Any other technologies that might be interesting to add into XF? ... Perhaps it would be worthwhile to consider ember first, though.

 

[Rob]

ember.js seems beneficial on many levels. Reusable components, precompiled templates, very light on the server, very fast render on the client, local storage, clean short code. I read this and see many uses and benefit. There are other cool frameworks but the more I read about ember the more I see many similarities with xenforo code, which I guess means it could transition easy. There are other frameworks that might be worth a nosey too

 

[motowebmaster]

Your video is interesting, and makes sense, but don't believe my communities would be ready for an etherpad integration.

Most of my users are older motorcyclists, who will spend a few minutes carefully considering what they type. If they thought their uncommitted words were being broadcast, they be less likely to visit. There are a few occasions daily where they will even go back and remove what they posted afterward.

For the record, my personal domain is a Ghost Blog on node.js/nginx. I'll concede that node applications come from a different universe, but am having fun.

 

[Rob]

@ForestForTrees - Look at CanJS - another pretty exciting framework