Just tested this with Jethro, and I had the delete link on comments on a comment I made on his profile.
He suggested it was right, because it was I was the original conversation starter, I would get to delete comments I didn't like.... sounds reasonable, but I am not sure it is actually designed that way.