7 days to upgrade Spacebattles

[jmurrayhead]

Here we go again... Why do you like to call names, you like to create a tense atmosphere for nothing? Some people are interested to get updates, others are not. You don't see me going after people's posts and attack them with words, while telling them what to do. Don't you think this is really immature and not productive at all? Or you think that "please" word you added at the end changes anything? These posts will probably get deleted, making the mods shrug again. Plus, you just made me waste 5min of my time for nothing.

Add to ignore...I just did

Edit: I'm not referring to Floren.

 

[Floren]

What the hell was that? I never called you anything but a bad speculator? Was it that offensive? I'm sorry you have no backbone.
By all means continue to try and act like the good guy when you are doing what was said not to do (ask for updates and roadmaps), just indirectly to get away with it.

Let me help you understand:

I don't know why though since it means nothing? Much like most the things you post.

You just told me that all my posts are useless in your eyes.

You are one of the worst speculators I have seen in a long time.

I'm sorry you have no backbone.

Nice insults, you don't even know me and have no idea who I am since I rarely post on these forums.

Just give it a break, please.

By all means continue to try and act like the good guy when you are doing what was said not to do (ask for updates and roadmaps), just indirectly to get away with it.

You just told me what to do, once with a fake "please" at the end.

Let me know if I missed other insults.
Let me remind you what is this discussion about:
Find a way to improve the importer speed and provide feedback to devs, as at the current rate it takes too long.

We now know that with the 1.1.3 version we still encounter import slowdowns. You should stop interpreting everything in a twisted way, I really don't understand from where you get that energy to fight. One thing is sure, I will not continue on this path and reply to any similar posts of yours. As a side note (since you liked his post), jmurrayhead added you to the Ignore list, not myself.

 

[bushidou]

What the hell was that? I never called you anything but a bad speculator? Was it that offensive? I'm sorry you have no backbone.
By all means continue to try and act like the good guy when you are doing what was said not to do (ask for updates and roadmaps), just indirectly to get away with it.

This guy.

Robbo, you are not the first to notice, a whole load of us on irc have noticed the same, you should come chill with us. i dot know if you saw my previously removed post but you are not alone. this thread was made from the start to gleen information whether the mods want to accept it or not.

just look at that pathetic tit for tat post "oh look you liked a post about ignoring you".

people like floren are why communities around developers turn hostile, people who bring nothing yet act like thay deserve answers more than anyone else. as i said before I hope he puts his money where his mouth is and moves onto another website so we can be done with his crap here.

 

[jmurrayhead]

Robbo, you are not the first to notice, a whole load of us on irc have noticed the same, you should come chill with us. i dot know if you saw my previously removed post but you are not alone. this thread was made from the start to gleen information whether the mods want to accept it or not.

just look at that pathetic tit for tat post "oh look you liked a post about ignoring you".

people like floren are why communities around developers turn hostile, people who bring nothing yet act like thay deserve answers more than anyone else. as i said before I hope he puts his money where his mouth is and moves onto another website so we can be done with his crap here.

Actually, it's people like you that have to butt in on legit customer concerns and take threads off topic. I'll respect your right to disagree, but not your childish comments that derail threads that don't concern you. Don't like threads like this? Get out. Don't like the people posting? Ignore them. Simple as that. XenForo isn't a helpless woman that needs protection from a big strong man. It's a commercial software that other businesses may base their products around.

 

[Floris]

Amazing how much bull Floren is getting.
Someone emailed me today if XenFans would be re-opened, I linked to a handful of threads like these.
He's getting more crap thrown at him than I have given him in the past.

 

[Mikey]

The problem here is that realistically, people don't have 64GB ramdisks, the converter simply isn't optimized for big boards. While I'm no longer performing xenforo services, from my experience converting / upgrading 1/2/3million+ post forums, it's a nightmare. The cache building process takes the longest, and while I don't know what Kier/Mike can do about it, it's not realistic to expect that non-technical people can do it in 6 hours just because you've managed to do it on a testcase. I agree with Floren's initial post, +1 for a special converter for big boards.

 

[mrGTB]

The problem here is that realistically, people don't have 64GB ramdisks

You not joking, I only have a 16GB one here.

 

[guiltar]

It seems like xenforo site needs topic starter's rights addon. Author will be able to delete all useless trash from his thread.
Also testing XF on highload brings it extra quality provided all reports are considered.
Next step would be building highload sites completely on XF instead of using it as just a forum.

 

[mrGTB]

Ha! I can just imagine the fun that would bring if I'm able to delete others posts from my threads, even mods replying!

 

[Deebs]

The problem here is that realistically, people don't have 64GB ramdisks, the converter simply isn't optimized for big boards. While I'm no longer performing xenforo services, from my experience converting / upgrading 1/2/3million+ post forums, it's a nightmare. The cache building process takes the longest, and while I don't know what Kier can do, it's not realistic to expect that non-technical people can do it in 6 hours just because you've managed to do it on a testcase. I agree with Floren's initial post, +1 for a special converter for big boards.

I found my conversion the exact opposite in fact. It took me less than 5 hours or so to convert 3.8 million posts using the CLI converter and that included the rebuilding of the search index and all caches. No ramdisks were used in this conversion. This conversion time also included addon installation and configuration, style tweaks and fixing things I broke

Nightmare? Absolutely not. It was slick, most of the time was spent waiting. The thing to remember is to run the CLI converter from a screen process in case of network failure.

 

[Mikey]

I found my conversion the exact opposite in fact. It took me less than 5 hours or so to convert 3.8 million posts using the CLI converter and that included the rebuilding of the search index and all caches. No ramdisks were used in this conversion. This conversion time also included addon installation and configuration, style tweaks and fixing things I broke

Nightmare? Absolutely not. It was slick, most of the time was spent waiting. The thing to remember is to run the CLI converter from a screen process in case of network failure.

the setup you used though, is it the average shared hosting? or a beastly dedicated server?

This is what I mean, people can have a moderately large forum (lets say what, 750k posts as an example) and still be on high end shared hosting, or low end managed vps.. and don't know this stuff, they dunno how to optimize mysql, they dunno how to setup thier server, etc.

and this is not a dig at you, good for you to have that knowledge of running a server, optimizing this stuff, but just because it took x hours for you and was a dream, and you used the CLI, doesn't mean that non-technical (please note this part ) people will find the experience the same, is all I was saying...

 

[Naatan]

the setup you used though, is it the average shared hosting? or a beastly dedicated server?

This is what I mean, people can have a moderately large forum (lets say what, 750k posts as an example) and still be on high end shared hosting, or low end managed vps.. and don't know this stuff, they dunno how to optimize mysql, they dunno how to setup thier server, etc.

and this is not a dig at you, good for you to have that knowledge of running a server, optimizing this stuff, but just because it took x hours for you and was a dream, and you used the CLI, doesn't mean that non-technical (please note this part ) people will find the experience the same, is all I was saying...

Non-technical people really shouldn't be moving a database with millions of entries across servers though..

Regardless of how long it takes though, I think based on what's been said that SpaceBattles was an exception to the rule - on average 1 or 2 days should suffice (least that's what I gather from reading the posts here).

 

[Mikey]

Non-technical people really shouldn't be moving a database with millions of entries across servers though..

But they will try anyway, that's how people are.

Edit: And good for them for actually being willing to take an active hand in the management of their community! not paying someone but jumping in knee deep to do it themselves. Learning feet first. It's how the best developers & server admins start.

 

[Floris]

You're both right deebs/mikey, one reason why I have done more imports localhost is because the fact is that a looooot of bigger sites are running on accounts I wouldn't even run my own sites on. Or they're simply under weird restrictions, or under managed, etc. It's great the vBulletin and XenForo can run on a big shared hosting solution for $10/month, but converts usually take ages, if not only because of constant timeouts, no shell access, poor restore features, etc. And those are sits with 250k to 750k posts. A lot of 500k to 1m posts sites are not on dedicated solutions but managed vps accounts. And only the really active 750k and higher ones seem to run on dedicated solutions that have tech guys to keep things running smoothly.

A lot of sites have 500k posts, but only 25 online members, and usually they dont even notice any lag. Converts take ages, hence why I prefer to just download the db dump, run my shell script on it, do a convert that's appropriate and return the site their database to restore (or restore on their behalf). Sometimes the restore took longer then the whole convert process.

When the site matched the hosting, and the hosting didn't suck, the converts went nice and smooth, regardless of size.

 

[Deebs]

the setup you used though, is it the average shared hosting? or a beastly dedicated server?

This is what I mean, people can have a moderately large forum (lets say what, 750k posts as an example) and still be on high end shared hosting, or low end managed vps.. and don't know this stuff, they dunno how to optimize mysql, they dunno how to setup thier server, etc.

and this is not a dig at you, good for you to have that knowledge of running a server, optimizing this stuff, but just because it took x hours for you and was a dream, and you used the CLI, doesn't mean that non-technical (please note this part ) people will find the experience the same, is all I was saying...

I wouldn't say it was a beastly setup, and to be fair, big boards of many millions of posts will most probably have better hardware than I did when I made the conversion.

The webserver was actually a virtual server with 4 gig of RAM, the SQL box was a dedicated server connected to a 100mb switch port. My beef with your post is that you are saying that there is no available conversion tool for big forum users, that is simply rubbish. I helped beta test the CLI and found bugs and made recommendations (which were taken on board) and I can say that MOST if not all big board forum admins are technical and can use the tools available else they would be suffering even before migrating to XF.

Remember, XF is a PHP script. When you are dealing with large forums you generally have technically savvy personnel involved to help, those same people can convert to XF using the supplied tools easy.

 

[Deebs]

Non-technical people really shouldn't be moving a database with millions of entries across servers though..

Regardless of how long it takes though, I think based on what's been said that SpaceBattles was an exception to the rule - on average 1 or 2 days should suffice (least that's what I gather from reading the posts here).

I believe the issue was deeper than the tools available.

 

[mrGTB]

No idea about this one, but when you have a huge database lets say. Could not use something like WinRAR and create multi-span ZIP's (2mb) in size for example of the database itself, so you can upload it in segments, rather than one large huge file?

 

[Naatan]

No idea about this one, but when you have a huge database lets say. Could not use something like WinRAR and create multi-span ZIP's (2mb) in size for example of the database itself, so you can upload it in segments, rather than one large huge file?

Depends on the structure of the database, to a degree it would definitely be possible. Depending on your server hardware it might not make a huge difference though.

Makes me wonder if MySQL has some sort of multi-threading functionality build-in for importing SQL data, technically it should definitely be possible but it'd have to load the entire SQL file into RAM just to be able to segment the data, which I don't believe it does.

 

[Floris]

To be fair, Mikey has helped from almost day one of XenForo to convert from the top 15 forum products to XenForo, don't forget that only quite recently the CLI was introduced. Our experience isn't just doing a handful of boards. We constantly found bugs, talked to Kier in private about it, and we even helped Mike beta test the phpBB converter just before release. We've shared resources, customized ImpEx for the best results; talked to Jerry who wrote it to find out undocumented tricks and tips for best result. Etc. We wrote documentation and wiki pages based on our experience, and done our best to help people move over. We run into enough situations, sized databases, active boards that can't afford downtime and enough variation of hosting solutions.

The CLI tool isn't the solution for big boards. It's a solution for those bigger boards that run vBulletin and where it doesn't go wrong.

 

[Mikey]

Jack is a forum administrator running phpBB. Jack's site is a medium sized hobby site, with 750k posts (not quite classed as a big board). Jack has heard about XenForo and would like to convert. Jack saves up and pays the $140 for the XenForo licence out of his own pocket. Jack has no more money to hire help, so he tries to go it alone. Jack doesn't know about XenForo's CLI importer, and even if he did it wouldn't work for phpBB. The convert takes over a day for him to do, during which time his members are constantly bugging him "When is fightclubfans.net back, Jack?", and by the end of the process it he's wishing he'd never started. What a great way for Jack to be welcomed into the world of XenForo. I am Jacks advocate, is all.

The question is though, could a non technical person run the CLI importer?


My point is made, anyway. I'm not going to argue over it, there is no point, I've said all I wanna say.

*Jack is fictional.