Are you using shared hosting or a VPS/dedicated server? Either way, you'd need to assign the subdomain to the document root, and then change the domain on the software. It's not really a rewrite, unless you want to forward everything from /forum/ to forum.example.com/
I'm on dedicated server and yes, I would like to redirect everything from /forum/ to forum.example.com because there have a lot of posts and threads already. Unfortunately, the FAQ does not cover this type of redirect. Can you please help me out?