As far as I understand, you get an Alert for the first reply to a thread you're watching or have started. If you're not getting an Alert for replies to a thread you started, this may be As Designed although personally I think you should get an alert for it.
Also, you only get 1 alert at a time per thread. So if you get an alert for a thread and never check it, you'll never get another, however, once you check it, a new reply will send another one. Only if you're watching the thread though.
There appears to be an intermittent bug in sending that one alert as well. I haven't tracked it down yet, but if you don't get that alert you don't get any until you read it again. (Makes sense: the one opportunity is missed.)