1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Sphinx Search Engine

Discussion in 'Resource and Add-on Discussions' started by mlx, Oct 14, 2010.

  1. mlx

    mlx Well-Known Member

    Sphinx Search Engine v0.0.2 for XenForo v1.0.0 (gold/final/stable)

    Please consider this a very first proof of concept beta release.

    XenForo is new to me. So if I'm doing something terribly wrong here please let me know ;)

    The following Installation and Upgrade instructions assume that you already know how to setup Sphinx Search Engine, including how to setup cronjobs to reindex your search index and update your delta search index inbetween. See below for a more detailed howto for CentOS5 (including how to setup Sphinx itself).

    I've been using the latest Sphinx 1.10-beta release. An example sphinx.conf is included.

    I believe it should work with Sphinx 0.9.9 as well. I didn't have time to test this though.

    Quick Installation
    1. Upload all files from the "upload" folder to your XenForo directory
    2. Go to your XenForo Admin Control Panel and click Install Add-on
    3. Select addon_xt_sphinx.xml and click Install Add-on
    4. add the source and index definitions from the included sphinx.conf to your sphinx.conf
    5. reindex your search index
    6. enjoy your Sphinx Search Engine
    7. please report any bugs and suggestions
    Quick Upgrade
    1. Upload all files from the "upload" folder to your XenForo directory
    2. Go to your XenForo Admin Control Panel and click List Add-ons
    3. Look for Sphinx Search Engine 0.0.1 and click Controls -> Upgrade
    4. Select addon_xt_sphinx.xml and click Upgrade Add-on
    5. enjoy your Sphinx Search Engine
    6. please note you no longer need to manually edit
    7. please report any bugs and suggestions
    Detailed HowTo (for CentOS5)
    (off the top of my head)

    Download and compile Sphinx
    • Code:
           # wget http://sphinxsearch.com/files/sphinx-1.10-beta.tar.gz
           # tar xzvf sphinx-1.10-beta.tar.gz
           # cd sphinx-1.10-beta
           # ./configure --prefix=/usr/local/sphinxbeta
           # make
           # make install
           # useradd -M searchd
           # chown searchd.searchd /usr/local/sphinxbeta/var/data
           # chown searchd.searchd /usr/local/sphinxbeta/var/log
    Setup your Sphinx config
    • Code:
            # cd /usr/local/sphinxbeta/etc
            # vi sphinx.conf
    • press
    • paste the content from examples/sphinx.conf
    • press
    Setup PHP scripts
    • Upload all files from the "upload" folder to your XenForo directory
    • Go to your XenForo Admin Control Panel and click Install Add-on
    • Select addon_xt_sphinx.xml and click Install Add-on
    Setup startup script
    • Code:
             # cd /etc/init.d
             # vi searchd
    • press
    • paste the content from examples/searchd_initd
    • press
    • Code:
             # chmod +x searchd
             # mkdir /var/run/sphinxbeta
             # chown searchd.searchd /var/run/sphinxbeta
             # /bin/su -c "/usr/local/sphinxbeta/bin/indexer --all" searchd
             # service searchd start
    Setup cron scripts
    • Code:
             # cd /etc/cron.daily
             # vi sphinx_reindex
    • press
    • paste the content from examples/sphinx_reindex
    • press
    • Code:
             # chmod +x sphinx_reindex
             # ./sphinx_reindex
             # cd /etc
             # mkdir cron.1minute
             # vi crontab
    • press
    • add this
    • press
    • Code:
             # cd cron.1minute
             # vi sphinx_delta
    • press
    • paste the content from examples/sphinx_delta
    • press
    • Code:
             # chmod +x sphinx_delta
             # ./sphinx_delta
    Make sure searchd is up on reboot
    • Code:
             # chkconfig --add searchd
             # chkconfig searchd on
    P.S. personally I'm reindexing every 24 hours at 4a.m. and run the delta each and every minute. This setup works fine for us, but we only have about 3 million posts to reindex anyway. So for really big forums a delta update every 5 minutes might be better (that would be */5 * * * * root run-parts /etc/cron.5minutes in crontab, in the howto above). And you need to decide when and if to run the reindex. That really depends on your setup and hardware I guess.

    Attached Files:

    EQnoble, Jarod, KingPin and 22 others like this.
  2. Shadab

    Shadab Well-Known Member

    Just a quick note; nothing serious:
    The variable $db is undefined in SphinxSearch_Install::uninstallSphinx().

    Nice work, btw. I wish my forum(s) required Sphinx search. :p
    Anthony Parsons likes this.
  3. mlx

    mlx Well-Known Member

    D'oh, you are so right.

    Edit: I updated the ZIP file. I didn't update the version number for this tiny little bug. I guess the worst thing it does is not dropping the 4 sphinx tables when you decide to uninstall.
  4. Andy.N

    Andy.N Well-Known Member

    Looking forward to your beginner guide.
    I hope your first step would help make the sphinx integration into default xenforo
    Weiyan likes this.
  5. ExpertPixels.com

    ExpertPixels.com Well-Known Member

    "sphinx integration" yes it is tricky so a user-friendly / beginner guide would be great, i screwed up a vb4 installation attempting this lol so i will wait for a full guide also. Thanks
  6. RvG

    RvG Active Member

    a brand new software with sphinx installed? :cool:
  7. Blandt

    Blandt Well-Known Member

    Some are porting old cranky vB 3's with gazillions of posts ;)
  8. Anthony Parsons

    Anthony Parsons Well-Known Member

    Yer... that would be awesome to have a massive forum with over a million posts. Nice work...
  9. jonsidneyb

    jonsidneyb Well-Known Member

    I only have half a million posts but I like that this is out here.
  10. Weiyan

    Weiyan Well-Known Member

    Have brothers and sisters tested this mod in Chiinese envirornment? Pls. share your experience.
  11. mlx

    mlx Well-Known Member

  12. jonsidneyb

    jonsidneyb Well-Known Member

    At what size does Sphinx start becoming a meaningful addition?
  13. Andy.N

    Andy.N Well-Known Member

    shouldn't this be useful at any size? so if you have a small site, will this hurt?
  14. Forsaken

    Forsaken Well-Known Member

    Smaller sites usually are on shared hosting.
  15. Blandt

    Blandt Well-Known Member

    I think what Andy means small as in less than a million post :)

    In Sphinx terms a million post is a small tiny site :D
  16. Andy.N

    Andy.N Well-Known Member

    Ok, shared hosting issue aside, what kind of site that sphinx would be ideal for?
    I'm always of the thought that extra power, better code, optimized search engine can only help in the long run.
  17. jonsidneyb

    jonsidneyb Well-Known Member

    I am at about 500,000 now so well below a million. Well, half that many.
  18. Andy.N

    Andy.N Well-Known Member

    i love to see this built into Xenforo at some point.
  19. Forsaken

    Forsaken Well-Known Member

  20. mlx

    mlx Well-Known Member

    On our test import on a test server XenForo MySQL FT search queries took up to 10 seconds. Compared to <1s with Sphinx. So I guess that's when Sphinx starts making sense. When your MySQL FT search slows your site down. If it's responsive enough you might just want to stick to it.

Share This Page