If you're running Nginx, you're going to need an alternative to the rewrite rule as .htaccess doesn't do anything. (This is a good example of where running Nginx requires additional technical knowledge).
I have not tried this, but the following may work:
Code:
location ~ /forum/(viewforum|viewtopic)\.php$ {
try_files /xf/phpbb.php?$uri&$args =404;
}