How to archive a XenForo forum to make it read-only

How to archive a XenForo forum to make it read-only

Sim

Well-known member
Sim submitted a new resource:

How to archive a XenForo forum to make it read-only - Detailed instructions on how to archive and optimise a site that is no longer active

I took over administration of the Somersoft forums in 2002 and ran the site on behalf of the owners until 2015 when we decided to archive the site and make it read-only. The community moved to a new site which I built on XF 1.x - PropertyChat.

Originally launched on vBulletin v2.2, Somersoft was still running vB v3.8 when it was archived in 2015. The archived site was migrated to XF 1.x in 2016...

Read more about this resource...
 
  • Like
Reactions: frm

frm

Well-known member
Wouldn't it be better to just scrape and archive the public view of the site after UI modifications?

I'm unsure if this would allow you to retain a license for another purpose, but would look into that.

If you're gonna permanently archive something, it doesn't seem necessary to run a licence to keep it "active" with static pages served with URL rewrites.

Haven't looked into licensing issues of this but seems a waste of a license that could be used for a 2nd forum.
 

Sim

Well-known member
Wouldn't it be better to just scrape and archive the public view of the site after UI modifications?

I'm unsure if this would allow you to retain a license for another purpose, but would look into that.

If you're gonna permanently archive something, it doesn't seem necessary to run a licence to keep it "active" with static pages served with URL rewrites.

Haven't looked into licensing issues of this but seems a waste of a license that could be used for a 2nd forum.

I haven't explicitly covered the option of creating a static version of the site in the tutorial - but I did explain indirectly why you might not want to do this: we have had to edit the content on numerous occasions after archiving the site, due to privacy or legal reasons. Retaining the ability to log in and modify the content quickly and easily is important to us, which is why we have not considered going down the static path.

We also have 1.3 million posts across 100,000 threads. The huge number of static pages you'd need to generate would make the exercise rather non-trivial and you'd also potentially break existing some links within the content.

Given that licenses for XenForo are extremely cheap I don't see the point in trying to reinvent the wheel by serving a static site just to avoid paying for a license when you've already got a perfectly good CMS already there called XenForo which does everything you need.

Put it this way: I currently host Somersoft (plus several other smaller XenForo sites) on a 4GB Linode which costs me US$25 per month + a separate ElasticSearch server which costs $7 per month. Annual cost is around US$384 for the hosting. A XenForo license costs $40 per year to maintain - which is a very small additional cost.
 
  • Like
Reactions: frm

Chromaniac

Well-known member
XenForo is USD 55 per year now right? :D Anyhow. Good guide. I wonder if it would also be useful to put the site on CloudFlare and enable full site caching using Page Rules. Though one would need to disable it temporary (or go Pro) for allowing selected users to login to make required changes. This should help reduce the load on the server hosting the archive site. Would also likely prevent hacking attempts.
 

frm

Well-known member
Thanks for the tip, @Sim. This would make a lot more sense on a sizeable forum than on something small.
 

HannahP

Well-known member
Wouldn't it be better to just scrape and archive the public view of the site after UI modifications?
If you know of a program or service that will do that, please let me know. I haven't found one yet that will create a stand-alone, mirror-image copy of a Xenforo forum (or, I would imagine any other similar forum or site). They always create a copy that's broken in one way or another, and certainly isn't "stand-alone."
 

frm

Well-known member
If you know of a program or service that will do that, please let me know. I haven't found one yet that will create a stand-alone, mirror-image copy of a Xenforo forum (or, I would imagine any other similar forum or site). They always create a copy that's broken in one way or another, and certainly isn't "stand-alone."
Before I go into that, I would have to bring @Chris D, @Mike, or @Brogan into the conversation to see if it's against the license or not (and if it is, the license should be updated to reflect). This is because it's 1 install per license, but when you make a site static you are still using XenForo code to an extent (the HTML, CSS, and JS). While the board may not be active and you don't have a license for the site to run anymore, I'm unsure whether or not the copyrighted works of how it is displayed would be in violation as that is copyrighted as well. If it is, you could essentially strip the posts one by one and add them to your own formatting, but keep the same URL format, making this task much more daunting (which would be easier doing the recommended in the last paragraph here).

Afterwhich, it would take some coding to spider your site to get the links to everything (say /threads/how-to-archive-a-xenforo-forum-to-make-it-read-only.180888/ saved as threads-180888.html but the static HTML file served when the URL is accessed. That way you wouldn't need a 301/302 redirect and everything stays intact.

The custom coding shouldn't be much if you don't know how to already do it if the display of XenForo's HTML, CSS, and JS is allowed without a license. Otherwise, it will be a lot more complex and you may just want to try and import XF into a free forum while turning off posting/registrations at that point.
 

Brogan

XenForo moderator
Staff member
If the site is present on the net and can be accessed publicly then it requires a license, regardless whether it is "static" or not.
 

frm

Well-known member
If the site is present on the net and can be accessed publicly then it requires a license, regardless whether it is "static" or not.
Even if this thread is crawled after the board is turned off with no replies, saved as threads-180888.html and URL rewrites serve that static file with the typical request while XenForo is no longer present on the system. If done that way, there is absolutely no way to install it again to update minor things... one would need to edit file by file.

What you're saying is that the HTML, CSS, and JS are copyright of XF and that would apply to the default XF theme. Would that be the case if you used a third-party theme? In the case of Font Awesome, that's licensed to XF for use there, and an easy remedy is to remove Font Awesome support.

There ought to be a way to archive a board in compliance with the license, otherwise, XF would have a case against Archive.org and Google for caching pages of content that doesn't belong to them either.
 

Brogan

XenForo moderator
Staff member
No, that's not what I'm saying.

If it's an XF installation then it requires a license.

If it's just static html pages then clearly an XF license isn't required.
 
  • Like
Reactions: frm

HannahP

Well-known member
I've been paying for a license that doesn't expire for more than 10 years. That's not the question or an issue.

My question is whether anyone knew of software or a service that would make a copy.

If not (and I suspect not, since I sure tried to find a good one, but couldn't), that's fine. :)
 

frm

Well-known member
If it's just static html pages then clearly an XF license isn't required.
This should be added to the terms so that everyone is aware they can abandon their XF installation for a new project while keeping the old board with content "alive" that could potentially rake in a few bucks from speedy/cached static pages (as I thought it was grey area due to the HTML/CSS/JS still being copyrighted).

While it does say installation, it should be specified because you could merely turn the board off to posting and it appear static, but it really isn't.

I'm sure you can track it with licensing though (belongs to a board, not on this license) where you could flick off the new live site for abuse in that regard.
 

frm

Well-known member
I've been paying for a license that doesn't expire for more than 10 years. That's not the question or an issue.

My question is whether anyone knew of software or a service that would make a copy.

If not (and I suspect not, since I sure tried to find a good one, but couldn't), that's fine. :)
You could probably find software to spider and save the content, but the issue is getting the URLs just right to where you wouldn't need Google to crawl it again and lose juice because the title would be removed from the URL to something automated like domain_com-000001.html, domain_com-000002.html... While the software may copy it and change links on the saved page, it would lose some ranking by linking to 000002.html instead of the original /threads/thread-title.id/ post.

That's where custom coding would come into play. It would download the page's HTML/files as is (could probably write a fairly small script in bash and use lynx/curl even...) and compile another file with URL rewrites in Apache or NGINX format to make the original request display the 000001.html file without changing the links at all.

I don't imagine paying more than $30-50 for this as I don't foresee it being many lines of code at all (a simple loop and save) and if it's a lot of pages that rank and can perpetually earn you money, I wouldn't see why you would give up not investing in that as it could earn itself back in a year with AdSense on it (throw that on the live site before you make copies as it'd be harder to replace ads on all the pages without extra $30~ help to do a find/replace).
 

Brogan

XenForo moderator
Staff member
This should be added to the terms
If the XF files are not being used then it's not an XF licensing issue.

What anyone does with their content is up to them and has nothing to do with XF.
Whether they export it to other forum software, or convert it to static html pages.

While it does say installation, it should be specified because you could merely turn the board off to posting and it appear static, but it really isn't.
Again, no.

That would require a license as it would be using the XF license files.
 

frm

Well-known member
Whether they export it to other forum software, or convert it to static html pages.
That's why the terms of a license should be clarified, in my opinion. XF still has a copyright on the HTML/CSS/JS that is rendered on the static pages. Also, if the pages are static and one didn't pay for branding removal, could branding be stripped from those static pages?

But, I'm glad that it was almost clarified here for me to proceed if one board "fails", I can keep it open with static pages to collect whatever revenue I can from its content.
 

Brogan

XenForo moderator
Staff member
No clarification is required.

If the XF license files are in use on a publicly accessible URL (whether posting is enabled or not) then a license is required - as is already stated in the license agreement.
 
  • Like
Reactions: frm

frm

Well-known member
No clarification is required.

If the XF license files are in use on a publicly accessible URL (whether posting is enabled or not) then a license is required - as is already stated in the license agreement.
Don't come at me with a bunch of C&Ds if I have static pages up with branding stripped from the HTML because I put in just enough effort for them to be AdSense revenue sites then! :p
 
Top