XF 1.5 Database size grows by x25 after import?

[dotpro]

Hi,

I did a test run of small forum migration from vb 3.8 to xenforo and photopost 8.5 (107k pics, over 4500 categories) and database size went from 90MB (for vb) and 40MB (for photopost) to 2350MB.

I read some threads about size increasing due to search index... but with just 18000 posts, what made the size grow by 25 times? The 107k pics in 4500 categories?

 

[Chris D]

There will be lots of differences between the platforms as to how data is stored and presented.

Off the top of my head the biggest tables will likely be compiled templates, phrases, IP address storage, posts, search index, template history, permissions and post edit history.

Essentially some of the stuff stored in the DB is duplicated across various caches, and those caches are pretty essential in terms of performance. So it may be the case that we're storing more which reduces the amount of queries and preparation of the data required.

 

[Mike]

Additionally, InnoDB generally has larger storage requirements than MyISAM.

That said, it does sound like a big jump, so I'd be curious which specific tables are large (and how you're determining that).

 

[dotpro]

So, I did the migration again and it came out to same size. Looking at the table sizes, it appears that XMG has about 1.8GB of that. 106k pictures in 4600 categories.

other thing which was odd was that import of tags took longer than import of the pictures... pics took about 9 hours, and tags took around 16 hours.

 

[Chris D]

What are the table sizes for all of the xengallery tables specifically?

That seems like a lot.

Is this after running the various rebuilds or before?

How many tags do you have? How many rows in the xf_tag and xf_tag_content table?

 

[dotpro]

It is after running the rebuilds.

xf_tag = 3480 rows.
xf_tag_content = 645150

It may just be the number of tags, many pictures are assigned useless tags which should not be there. Would emptying these tables make sense and start fresh as far as tags are concerned?

 

[Chris D]

The media tables after rebuilds will contain a lot of EXIF data. There's a chance we actually store too much of this; we may need to address this in the future.

Your strategy with tags is up to you. The tags step of the importer is optional. If you start again I wouldn't just empty those tables. Instead you should do another import and just not run that step.