Well, I'm thinking of setting up WordPress in the root; but not disturb my existing URL structure with XenForo. I'm guessing there'd be a way to make WP and XF co-exist in the same directory - sharing an index.php and some common htacess rules.
If both applications want to rewrite URLs to them, you're going to need to list every single possible URL format for one of them to ensure it is rewritten to there as expected. I suspect you're going to have other problems then as well (with the index.php conflict for example).
In your example, I'd move XF to a sub-directory and then add rewrites for the common URLs (forums/threads/members) to the sub-directory.
I'm mostly bothered about the thread URLs. Others don't matter; because threads are where all our precious backlinks are. Having moved the forum from subdirectory to the root in the past; I really don't want to touch the URLs again. But looks like I'll eventually have to do that; if there's no solution.
I did try installing both systems in same folder on my localhost; and they worked fine except that the WordPress had issue with seo-friendly URLs. I don't remember what I really did; but I do remember changing the index file reference inside: library/XenForo/Link.php.
I think with a few tweaks; I should be able to achieve what I'm trying to do.