I test with IE7, but nothing more. If it looks horrible, I do debugging and perform a minimal fix. IE was a real problem, it is much better now, especially with version 9, version 8 still have some quirks, but much less than it used to. My main problem with IE is the lack of proper developer tools, which are available by default in Chrome and Firebug in Firefox (I know there is a default one, but Firebug is so much better).
Be warned though, I have noticed some cases of firefox behaving differently between systems. I don't remember the exact issue, but I think it was a couple of years ago while using Arch Linux (and they use a "fork" of Firefox, if I am not mistaken, from the nightly build branch).