I changed the base href in the templates, and it does change all the links to images, css, javascript, since those links are relative URLs.
But all links to actual pages are absolute URLs, and use the www. in it.
As an alternative, is it possible to just set all links to use relative URLs?