Realistically how difficult is it going unmanaged?
The most appropriate answer is; It depends!
If you are a full time linux system admin in real life then it's easy. If you are not, then things can get complicated quickly. Are you a quick learner, do you have the time? What's your time worth? Can you really think of all the issues that need to be considered (i.e. DOS attack prevention, brute force discouragement, bind monitoring, tweaking of apache, php, mysql to consume less memory)? Do those things matter when you just start out? Probably not, on a small forum most those things are pretty irrelevant. If you are just starting out and you have no idea how well your forum will do then a decent shared plan (don't do HostGator, they DB performance blows) will be fine. Pick a host that has a good VPS plan though so that you don't need to switch hosts when you need to upgrade to VPS as your forum grows.
To me the most important aspect of managed is that it saves me time. There are only 24 hours in a day, what am I going to spend that time on?
Let's say you go unmanaged, and let's just for the sake of this argument assume that once everything is set up (which you will burn countless hours on, dozens for sure to learn how it all works and what you need to look out for and how to optimize it etc.) you spend 1 hour per week on maintaining your unmanaged system.
That's 4 hours per month spent to save $20 per month, meaning you are working for $5/hour. Alternatively you can go managed, have those 4 hours a month available, work one hour out of the 4 to make $20 and still come out three hours ahead which you can spend on whatever you like.
Unless a host provides 24/7 phone support I would not buy managed services from them simply because I wouldn't trust them to not be some reseller making some easy money by providing "managed" services with guaranteed 24 hour response times on tickets.