XF 1.4 Struggling to import styles and install add-ons?

[Mindzipper]

There has to be something wrong here. Whenever I do anything that requires the server to do an import it takes a very long time.

for instance, I did a test, and installed 3 styles. it took over 15 minutes just to import the XML on all of them.

I installed Sportsbook as well, it too an hour to import that xml file.

this can't be normal? the server is very lean with only one website running on it. there are plenty of available resources, and when i monitor the DB server, there are very long delays between times transactions happen. almost 90% of that 15 minutes is spent with the db user account sleeping

 

[Mindzipper]

sure

user=mysql
bind-address=0.0.0.0
connect_timeout=10
wait_timeout=15
local-infile=0
log_warnings=0
log-bin=/var/lib/mysql/data/mysqld-bin.log
log-bin-index=/var/lib/mysql/data/mysqld-bin.index

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

# Recommended in standard MySQL setup
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

# DEFAULT CHARACTER SET AND COLLATION
character-set-server=utf8
collation-server=utf8_general_ci

# SSL
ssl-ca=/etc/mysql.d/ca-cert.pem
ssl-cert=/etc/mysql.d/mysql-server.pem
#ssl-key=/etc/mysql.d/server-key.pem

# LOCATIONS
datadir=/var/lib/mysql/data
tmpdir=/tmp/.mysql
socket=/var/lib/mysql/mysql.sock

# PASSWORDS
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=0

# CONNECTIONS & THREADS
max_connections=60
max_delayed_threads=120

 

[Mindzipper]

i did some testing and the problem is DEFINITELY in the db server. i did a test install at an ec2 site and the install took about 3 minutes. did it again at this location? it's been 30 minutes and it's still rebuilding phrases.

so if you have some idaes, i'm getting a bit desperate. i had to shut down the main forum during this process,m and I can't bring it back up until i get migrated. it's been 7 hours now and importing those threads, even if I DO get it working, is going to take a week at the speeds i'm geting

 

[Paul B]

I'm no server expert, which is why I suggested:

It may be worth posting your config here, so some of our resident server experts can chime in: https://xenforo.com/community/forums/server-configuration-and-hosting.48/

 

[Mindzipper]

oh lol sorry, misread.

EDIT: I posted a thread there. thank you

 

[Mike]

It's worth pointing out that the error you received is usually down to a connection timeout: https://dev.mysql.com/doc/refman/5.0/en/gone-away.html Or it may indicate that the server died (or the thread reading died) while executing the query. I feel like I may have actually seen something that seemed to imply that with MariaDB once before, but I don't recall specifics.

At least for debugging purposes and to eliminate variables, I would look at using the same server for all parts of the import process:

• Running the web server and PHP/connecting to the DBs
• Running the source database and any files (attachments, etc)
• Running the destination database and any files

Since all communication will be over localhost and using the same file system, that should eliminate any networking variables.

That said, I'll respond to your other thread with config elements.

 

[Mindzipper]

ok thanks, and unfortunately I don't have the ability to do that. We don't run db's on the front end server. I know what you're saying though

I'm tempted to install mysql/mariadb on the server just to do this, but man that's not really the right way to approach it.

frustrating.

EDIT: and I did see someone mention it too, they said they'd used all the RAM and it killed it

 

[Mike]

I was actually proposing the other way around (installing a web server/PHP on the DB server). Or you can just spin up a temporary server to do the import and move the data back when you're finished.

I'm mostly just trying to eliminate variables to determine the underlying cause. You've seen with a different server it's much faster, so the key is to remove possibilities by changing a small amount at a time to be able to identify the root cause.

 

[Mindzipper]

actually i'm going to take a different direction.

I want to start this out by saying, the problems i've had are not to be blamed on Xenforo at all. They are primarily because we are new to transactional databases, and we are dealing with a forum that's been running for over seven years. that's a long time.

I've decided a new path. I have a server sitting doing nothing that used to run this forum. but it runs both mysql and web on the same box. 6 months ago we migrated to a new path with dedicated db servers and dedicated front end servers. So what I'm going to do, is dump the sql to the old server. push all the files over there, change my hosts file to think the old box is the live one, and i'm going to do this migration there. which has much more RAM and is configured with a more familiar scenario.

I think this is my best bet. but since i've gotten 2.5 hours sleep in the last two days, I'm going to rest first

 

[Mindzipper]

Well, what a difference it makes when your server is configured correctly lol

56 minutes, importing 733,000 posts. nice