1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Implemented Storage of parsed posts

Discussion in 'Closed Suggestions' started by digitalpoint, Oct 21, 2011.

  1. digitalpoint

    digitalpoint Well-Known Member

    XenForo's post parser seems pretty fast (not going to argue that), but was starting to think that storage of parsed posts would be pretty nice when you start to factor in people doing non-standard things with posts. For example I made a GeSHi Syntax Highlighter for XenForo, which obviously is going to slow down post parsing a bit.
    Alfa1, tyteen4a03, Softlineck and 5 others like this.
  2. Andy.N

    Andy.N Well-Known Member

    Anything that can increase page speed, reduce overhead is a good thing in my book.
    Softlineck likes this.
  3. Rigel Kentaurus

    Rigel Kentaurus Well-Known Member

    I second this. I wanted to port over my "MySmilies" hack that I did for some other platform. That lets the user add customized smilies (only for them) in a similar way that you can have custom smilies in Live Messenger

    This was going alright, until I realized there is no parsed posts cache, which means I would be doing a bunch of substitutions every single time on the post itself. User smilies are not controlled in quantity (a user can have 50 if they want), and making the post go through that stress made me look away for now.
    dieketzer likes this.
  4. digitalpoint

    digitalpoint Well-Known Member

    So I sat down tonight and started to make an addon that does it. It's relatively rudimentary (like it's not currently taking into account if you have attachments in a post that are only viewable by some users and not others)... but for the most part it seems to work.

    A little test on a 4 post thread...

    Page generation time without the caching addon: 0.2999 seconds (10 queries)
    Page generation time for first thread view (generating cache): 0.4116 seconds (14 queries... one extra per post being stored in post_parsed table)
    Page generation time for subsequent thread views (pulling parsed posts from cache): 0.0897 seconds (10 queries)

    So the net effect is that the first time a post is viewed, it takes slightly longer to generate the HTML for that post (since it's also storing it), but then all subsequent views of that post are MUCH faster.

    When the DataWriter triggers an update of the post record (like a post edit), we are flushing that post from the cache so it will be rebuilt the next time it's viewed. I probably should change it though to not be so generic (changing something the number of likes a post received isn't going to change what the HTML of the post itself looks like).
    Jarod, Insy, Fuhrmann and 8 others like this.
  5. Andy.N

    Andy.N Well-Known Member

    Subsequent views for that same viewer or for all viewers? So in effect, this is not similar to browser cache?
  6. digitalpoint

    digitalpoint Well-Known Member

    Subsequent views for all viewers... The cache is not user specific. Wouldn't be worth processing and storage overhead to do it per user. :)
  7. digitalpoint

    digitalpoint Well-Known Member

    Now if only I can figure out how to tell which style ID is active from the XenForo_ViewPublic_Thread_View class... XenForo_Visitor::getInstance()->language_id seems to work for language, but XenForo_Visitor::getInstance()->style_id is the style the user selected (I think), not necessarily the active style (if the user selected no style it will reflect a styleid of 0... probably messed up on nodes where you override the style as well I would assume). Will go back to digging for it later. :)
    high1976 and Fuhrmann like this.
  8. Van Damm

    Van Damm Member

    Hi! Did you have any progress implementing this since that time?
  9. digitalpoint

    digitalpoint Well-Known Member

    The system as a whole, yes... For use on a site with multiple styles/languages, no.
  10. guiltar

    guiltar Well-Known Member

    Just notice that parsed post may depend on many factors, not only language and style.
    For example, there is hide addon which makes [hide=5]text[/hide] visible only for users with more than 5 posts.
    Maybe it's reasonable to cache parsed posts only for guests (default language, default style, etc..) because they are usually 90% of visitors.
  11. digitalpoint

    digitalpoint Well-Known Member

    You would also have to factor in attachment visibility. For *me*, the system I built works fine because I have one language, one style, and nothing that alters posts per user... For something similar to be built-in, you would of course need to address those issues.
  12. guiltar

    guiltar Well-Known Member

    Ok understand you. I probably also need to cache parsed posts because of unlimited number of them per page in blog entries (threaded comments).
    One more suggestion: generate cache when post is being changed or inserted (in DataWriter's postSave).
  13. Van Damm

    Van Damm Member

    Could you please share this caching system? It would be of great use for me.
  14. TnT

    TnT Member

    On which place do you cache the post? BBCode is rendered in XenForo_ViewPublic_Thread_View. Do you overwrite this method to stop rendering cached posts? My Problem at the moment is, that the BBCode renderer checks for a view when attachment rendering at Xenforo_Bbcode_Formatter_Base::renderTagAttach

            if (!$this->_view)
    '<a href="' XenForo_Link::buildPublicLink('attachments', array('attachment_id' => $id)) . '">View attachment ' $id '</a>';
    So i have problems to render the post outside a view for caching
  15. AndyB

    AndyB Well-Known Member

    I'm so glad Xenforo doesn't store parsed posts because I have a complete mobile front end that writes directly the xF tables and not having to deal with parsed posts makes my life much easier. Besides xF is extremely fast even without a parsed posts.
  16. AeroCryX

    AeroCryX New Member

    +1 To the suggestion

Share This Page