1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.1 Varnish / LSWC for XenForo?

Discussion in 'XenForo Questions and Support' started by Alfa1, Mar 21, 2013.

  1. Alfa1

    Alfa1 Well-Known Member

    Is there a Varnish addon or a LiteSpeed web cache implementation for XenForo?

    I currently use LSWC with the vb varnish xml and its has done wonders for my big board. Is there something similar for XenForo?
     
  2. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

  3. CyberAP

    CyberAP Well-Known Member

    So did anyone figure it out how to configure Varnish rewrites to be compatible with XenForo? It's still a big issue.
     
  4. Arkshine

    Arkshine Active Member

    I have varnish, and don't remember adding specific code to get it working, by default it works already fin, or I don't understand your question CyberAP.
     
  5. CyberAP

    CyberAP Well-Known Member

    Do you use varnish + nginx or apache?
     
  6. Arkshine

    Arkshine Active Member

    Varnish + nginx
     
  7. CyberAP

    CyberAP Well-Known Member

    Can you share your nginx config with me please?

    Right now I have this for /xf folder:

    Code:
    location ~* (^(?!(?:(?!(php|inc)).)*/uploads/).*?(php)) {
            try_files $uri = 404;
            fastcgi_split_path_info ^(.+.php)(.*)$;
            fastcgi_pass unix:/var/run/php5-fpm.sock;
            fastcgi_index index.php;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            include fastcgi_params;
            fastcgi_intercept_errors on;
            fastcgi_ignore_client_abort off;
            fastcgi_connect_timeout 60;
            fastcgi_send_timeout 180;
            fastcgi_read_timeout 180;
            fastcgi_buffer_size 128k;
            fastcgi_buffers 4 256k;
            fastcgi_busy_buffers_size 256k;
            fastcgi_temp_file_write_size 256k;
        }
    
        location /xf {
                try_files $uri $uri/ /index.php?$uri&$args;
        }
    BTW, I am using php-fpm through socket for php serving.
     
    AlexT likes this.
  8. Arkshine

    Arkshine Active Member

    I'm using Varnish as frontend, is it what you want too ?
     
  9. CyberAP

    CyberAP Well-Known Member

    Yep.
     
  10. Alfa1

    Alfa1 Well-Known Member

    Same here.
     
  11. CyberAP

    CyberAP Well-Known Member

    So I've been playing with configuration and one thing I can't understand: who should listen to 80 port and who to 8080? Right now I have nginx on 8080 and Varnish on 80. But it's not working. Maybe I am not getting something...
     
  12. CyberAP

    CyberAP Well-Known Member

    That was my default.vcl varnish config that messed up everything. Now it works (at least fine), but full friendly URLs still not working. Don't know what to do with it :(
     
  13. Arkshine

    Arkshine Active Member

    I'm under Debian (Wheezy) ; so files may not be exactly at the same place.
    Also I don't remember exactly what you need for varnish, so I may give unnecessary code.

    Varnish will be on port 80.
    Nginx will be on port 8080.


    Varnish configuration
    • /etc/default/varnish
    To tell varnish to listen port 80.
    Adjust the max cache memory.
    Code:
    ## Alternative 2, Configuration with VCL
    #
    # Listen on port 6081, administration on localhost:6082, and forward to
    # one content server selected by the vcl file, based on the request.  Use a 1GB
    # fixed-size cache file.
    #
    DAEMON_OPTS="-a :80 \
                -T localhost:6082 \
                -f /etc/varnish/default.vcl \
                -S /etc/varnish/secret \
                -s malloc,128m"
    • /etc/varnish/default.vcl
    By default, it should be enough to start after looking a bit.
    You can do many things to filter/improve cache, I can give generic piece of codes.
    I don't say you have to use them, just examples what you can do.

    In vlc_recv(), something you can put after the main checks :
    Code:
        # Normalize gzip compression
        if( req.http.Accept-Encoding )
        {
            if( req.url ~ "(?i)(\.|-)(jpe?g|png|gif|gz|tgz|bz2|tbz|mp3|ogg)(.*)?$" )
            {
                remove req.http.Accept-Encoding;
            }
            elseif( req.http.Accept-Encoding ~ "gzip" )
            {
                set req.http.Accept-Encoding = "gzip";
            }
            elsif( req.http.Accept-Encoding ~ "deflate" )
            {
                set req.http.Accept-Encoding = "deflate";
            }
            else
            {
                remove req.http.Accept-Encoding;
            }
        }
    
        # Use anonymous, cached pages if all backends are down.
        # Remove cookies from things that should be static, if any are set
        if( req.url ~ "(?i)(\.|-)(jpg|jpeg|gif|png|ico|gz|svg|svgz|mp4|ogg|ogv|css|js|ttf|ttc|otf|eot|woff|ico)(?:\.[0-9]+|(\?[a-zA-Z0-9\=\.\-_]+)?)$" )
        {
            remove req.http.Set-Cookie;
            remove req.http.Cookie;
            set req.url = regsub( req.url, "^/.*$", "" );
            return ( lookup );
        }
    
        # Remove Google Analytics.
        if( req.http.Cookie )
        {
            # Some generic URL manipulation, useful for all templates that follow
            # First remove the Google Analytics added parameters, useless for our backend
            if( req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=" )
            {
                set req.url = regsuball(req.url, "&(utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "" );
                set req.url = regsuball(req.url, "\?(utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "?" );
                set req.url = regsub(req.url, "\?&", "?" );
                set req.url = regsub(req.url, "\?$", "" );
            }
     
            if( req.url ~ "\?$" )
            {
                set req.url = regsub(req.url, "\?$", "" );
            }
            set req.http.Cookie = regsuball( req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "" );
         
            # Some generic cookie manipulation, useful for all templates that follow
            # Remove the "has_js" cookie
            set req.http.Cookie = regsuball( req.http.Cookie, "has_js=[^;]+(; )?", "");
            # Remove any Google Analytics based cookies
            set req.http.Cookie = regsuball( req.http.Cookie, "__utm.=[^;]+(; )?", "" );
            set req.http.Cookie = regsuball( req.http.Cookie, "utmctr=[^;]+(; )?", "" );
            set req.http.Cookie = regsuball( req.http.Cookie, "utmcmd.=[^;]+(; )?", "" );
            set req.http.Cookie = regsuball( req.http.Cookie, "utmccn.=[^;]+(; )?", "" );
        }
     
        if( req.http.Cookie == "" )
        {
            remove req.http.Cookie;
        }
    In vcl_fetch(), more tips. You can force max-age here for example.
    Code:
        # Adjusting Varnish's caching - hold on to all cachable objects fo 24 hours.
        # Objects declared explicibly as uncacheable are held for 60 seconds, which
        # helps in the event of a sudden ridiculous rush of traffic.
        if( beresp.ttl < 24h )
        {
            if( beresp.http.Cache-Control ~ "(private|no-cache|no-store)" )
            {
                set beresp.ttl = 60s;
            }
            else
            {
                set beresp.ttl = 24h;
            }
        }
    
    if( req.url ~ "(?i)(\.|-)(jpg|jpeg|gif|png|ico|gz|svg|svgz|mp4|ogg|ogv|css|js|ttf|ttc|otf|eot|woff|ico)(?:\.[0-9]+|(\?[a-zA-Z0-9\=\.\-_]+)?)$" ) 
        {
            set beresp.http.X-Varnish-Action = "FETCH (deliver - static sontent)";
         
            set beresp.ttl = 30d;
            set beresp.http.Cache-Control = "public, max-age=31556926";
            unset beresp.http.Set-Cookie;
            unset beresp.http.Cookie;
            set req.url = regsub( req.url, "\?.*$", "" );
         
            return ( deliver );
        }
        elseif( beresp.http.Set-Cookie )
        {
            return( hit_for_pass );
        }
        elseif( beresp.http.Cache-Control ~ "(private|no-cache|no-store)" )
        {
            return( hit_for_pass );
        }
        elseif( req.http.Authorization && !beresp.http.Cache-Control ~ "public" )
        {
            return( hit_for_pass );
        }

    To make redirections, it's kind of weird, here an example where you would want to redirect http://www.mysite.com to http://mysite.com

    In vcl_recv() : at the top
    Code:
        if( req.http.host == "www.mysite.com" )
        {
            error 301;
        }
    In vcl_error() :
    Code:
        if( obj.status == 301 && req.http.host == "www.mysite.com" )
        {
            set obj.http.Location = "http://mysite.com" + req.url;
            set obj.status = 301;
            return( deliver );
        }
    In vlc_error, you can handle error, here actually the full function for me :
    Code:
    sub vcl_error
    {
        if( obj.status == 301 && req.http.host == "www.mysite.com" )
        {
            set obj.http.Location = "http://mysite.com" + req.url;
            set obj.status = 301;
            return( deliver );
        }
        elseif( obj.status == 700 )
        {
            # Include a general error message handler for debugging purposes
            include "/etc/varnish/conf.d/_error.vcl";
        }
        elseif( obj.status == 701 )
        {
            # Redirect error handler
            set obj.http.Location = "http://" + obj.response + req.url;
         
            # Change this to 302 if you want temporary redirects
            set obj.status = 301;
         
            return (deliver);
        }
        elseif( obj.status == 503 && req.restarts < 5 )
        {
            set obj.http.X-Restarts = req.restarts;
            return ( restart );
        }
    
        return ( deliver );
    } 

    Nginx configuration
    • etc/nginx/nginx.conf
    Purpose is to retrieve real user's ip.
    Inside http{} :
    Code:
        set_real_ip_from    127.0.0.1;
        real_ip_header        X-Forwarded-For; 
    In log_format, you can add if you want : "$http_x_forwarded_for"
    • etc/nginx/default (or whatever it's called)
    Purpose is to tell nginx to listen port 8080.
    Here how it starts for me. The first block is only to redirect www.mysite.com to site.com.
    Code:
    server
    {
        listen          8080;
        server_name    www.mysite.com;
    
        rewrite        ^http://cs-amx.fr$request_uri? permanent;
    }
    server
    {
        listen      8080;
        server_name mysite.com;
     
        root        /path/to/your/wwww;
        index      index.html index.htm index.php;
     
        ...
    }
    After that, it should be enough to work. :)
    Don't forget to restart varnish/nginx.

    I strongly recommend you search on google example of config for varnish. You will find many tips and to understand what you can do and how. My file is a complete mess so I don't want to post it, but the point is you can really configuring finely the cache behavior. If you have high traffic, it's definitely important to take a look.

    EDIT: did not understand you were talking about URL rewriting :ROFLMAO: ; if not working, I would say it's more nginx config to check.



     
    Jarod, Coop1979, Ketola and 1 other person like this.
  14. CyberAP

    CyberAP Well-Known Member

    With a help of @Yoskaldyr figured out what was wrong with my config, it was try_files, I didn't set a path to index.php, so here is correct config (notice /xf/ path before index.php):

    Code:
    location /xf/ {
            index  index.php;
            try_files    $uri $uri/ /xf/index.php?$uri&$args;
    }
    Also I removed Varnish until I figure out how to configure it properly, right now it's cache is too aggressive for me.
     
  15. Arkshine

    Arkshine Active Member

    I feel like I've lost my time written my previous message :ROFLMAO: ; but what you say is wise. I remember having some trouble to adjust configuration at the start. So, yes, you should have first a solid nginx config ; and once all is working fine, if you feel you need to improve forum loading/reactivity, you could try to install it again. Varnish would be really worth if you have a lot of cacheable objects. If not, using nginx cache would be enough.
     
    CyberAP likes this.
  16. dbembibre

    dbembibre Active Member


    Dont worry you dont lost your time writing this, this post help me to adjust my incorrect nginx config too :D
     
  17. StormRider

    StormRider Member

    Have anybody tried to use nginx-varnish-nginx sandwich with the first nginx as SSL terminator?

    Thanks to Arkshine, Varnish works perfect on HTTP layer. Varnish is configured on port 6081 and backend nginx on port 8080 and frontend nginx on port 5443. When opening http://mysite:6081/ everything looks fine and inner links are like http://mysite:6081/... However, when opening https://mysite:5443, I see all pages without styles and scripts, all the inner links are like http://mysite/... (not https and without port).

    So, I wonder, if something is wrong with my nginx config or that is Varnish. The part of nginx config for SSL proxy:
    Code:
    server {
        listen  5443 default_server ssl;
        server_name  mysite;
    
        location / {
            proxy_pass  http://127.0.0.1:6081;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_redirect     off;
        }
    }
    
     
    Jarod likes this.

Share This Page