Search storage

[Darfuria]

How does XF store search data? Frequent searches, cached searches, etc.?

phpBB has the search_results, search_wordlist and search_wordmatch tables, and these tables can get pretty full on a big forum. My shared webhost has a database size limit of 256MB - and the search tables cause me to constantly exceed this and have to purge the data.

Will this be a similar issue when using XF?

 

[Mike]

Really, a database size limit like that is unreasonable. I couldn't make a recommendation of a host with an unreasonable size limit, as the definition of how that's calculated could vary quite a lot. (Is it the data files themselves?) Ideally, you wouldn't want a limit, but if we're talking in the GBs range, you'd be unlikely to hit it without a fairly large amount of posts.

 

[Darfuria]

Ah - I have no dispute with the host, they provide a fantastic shared hosting service, and really the forum should be on a dedicated server, but the client won't pay for one.

The database size would be fine if it wasn't for the search data that phpBB stores in the search_wordmatch (and similar) tables, so I'm just wondering if XF similarly stores search data like that - or whether the search is handled at database level by MySQL doing a fulltext search on the posts table.

 

[Mike]

It's a full text search, but it's not on the post table itself. Generally speaking, we will trade size for performance.

Unless the whole web space is limited to 256MB, then a 256MB DB limit is quite Draconian, IMO.

 

[Darfuria]

Just to bump this, I have had to disable search on the main forum I am running XenForo on due to the size of the search table. Is there anything I can do to compress and reduce the amount of space this takes up (besides pruning old posts)? I'm happy with the web host I'm with, so don't want to go through the inconvenience of switching hosts for this if I can avoid it.

 

[Mike]

You'd have to remove records from it to shrink it.

 

[Darfuria]

Alas, no other solution?

 

[Deebs]

Not yet... but watch this space, search is undergoing a change ..... How many posts and how big is the search table?

 

[Darfuria]

Not yet... but watch this space, search is undergoing a change ..... How many posts and how big is the search table?

230,000 posts... I don't know how big the table is, because search is disabled... I would guess at least 100MB because it pushes my database over the aforementioned 256MB limit, and it is currently 140MB.

 

[Deebs]

Um, seriously, you need to get a different host, 256mb of storage for a MySQL database backing a forum is NOT a lot of space. You should be able to use PHPMyAdmin to have a look at the size of the table, or have you completely removed the table?

 

[Darfuria]

Um, seriously, you need to get a different host, 256mb of storage for a MySQL database backing a forum is NOT a lot of space. You should be able to use PHPMyAdmin to have a look at the size of the table, or have you completely removed the table?

I like my host. Regardless of their 256MB storage for a single MySQL database. They provide a high-quality service that I'm happy with.

The table is empty, which is why I cannot provide you with a size.

 

[Deebs]

I am not saying that your host is good or bad, just that 256MB of storage is way too low. Can you not get an increase from them?

Edit:

As an example, a XF forum with 117 posts is consuming 117MB here, the size of the average post is extremely large so makes the indexing top heavy.

 

[Darfuria]

Is it possible to prevent the search table from being populated? I mean, search is disabled, so there's no reason why it should be populated... I tire of MySQL being restricted because I exceed a database size limit (which is due to be revised at the end of this month, thankfully).

 

[Markos]

Darfuria, i suggest you to contact your host for resolve your problem. This limit is too restricted...i think you can obtain a decent increase of space.

 

[Darfuria]

Sorry to necro-bump my own thread. Out of pure curiosity, is a full text search not run on the posts table purely for performance reasons, or are there other reasons as well?

 

[Slavik]

Pretty much. Full text searches on a large database (particularly ones if the data is not short and simple) would be a massive performance hit to any forum with a decent post count. If you are still having the same problems, then put simply you either need to move host or request a larger database allocation.

I don't think the xf team is going to massively change how their search works for 1 user (no offence).

 

[Digital Doctor]

How does XF store search data

Can you replace xenforo native search with a google search of your site ?