Using file-based caching could in theory speed certain things up, but in general cases (particularly relating to a shared server where the configuration may not be ideal), it may end up being worse. I probably would never use a file-based cache system for XF. I'd only enable a cache system that was memory-based (such as Memcache or maybe APC, but I have some concerns with how APC expires entries).