You can have different applications under the same domain name. For example, your wordpress can be installed on
www.yoursite.com and your forum under a subdirectory such as
www.yoursite.com/community. That is the most straightforward way of doing it.
However, in my case it was a little bit more complicated. XenForo has always been installed on the root of my domain. When I decided I wanted to install Wordpress, I had two options: 1. Moving my forum from the root of the site to a subdirectory, which means I had to change my whole URL structure and possibly hurt my ranking with Google. 2. Installing Wordpress on a subdirectory.
I went for the number two option and by using a webserver rewrite trick, I managed to serve the homepage of my site with Wordpress without moving XenForo from the root of the site.
This means I have the following URLs:
www.mysite.com > wordpress (new homepage managed by wordpress)
www.mysite.com/forums > xenforo (this url is the new forum home)
www.mysite.com/threads/thread-title > xenforo thread (this URL didn't change nor any of the other XenForo URLs except from the forumhome)
www.mysite.com/content/news-title > wordpress post (wordpress content under a subfolder)
I think it works pretty cool.