I think you need to set expectations here.
5 random visitors on your site is very low when you're talking about converting a random visitor to a registered user.
I get maybe a dozen (non-spam) registrations per day, which sounds great, right? Except that you need to consider I see 4,000 new visitors every day. That means 0.3% of my daily new visitors convert to members. I don't know if that's normal in the industry, just using it as an example.
If you were to get the same conversion rate, you'd have to get 300+ visitors to convert one to a new user.
That doesn't mean you couldn't score a much higher conversion rate (a huge chunk of my traffic is random click throughs on Google which don't tend to convert at a high rate), only that you need to be mindful of the fact that the significant majority of people who click on your site from Google or a link in a forum are not interested in participating, and as such, are not going to register an account.