Err, not sure I understand this. If you got a reply alert, it's because there's a new reply in that thread -- the mod action doesn't create any alerts. Alternatively, you could've had the alert before the post was moved to a new thread.
I've moved it a couple of times now and the alert seems to "get" the thread name change and use the latest thread name.
I'm assuming the ajax re-gets the alert data before displaying it?
I moved it to "What's my IP?" and that's the thread name that appeared in the alert. I moved it again to "What's my IP address?" and when I next hover over alerts it changes the alert text and now uses the new thread title.