I understand that... but that suggests the other participant understands how the forum software works and also recognizes that he is the only one left in the conversation. And it would prevent the other participant from replying and expecting a response from the creator who is no longer in the conversation. Quite frankly, when I opened up a conversation and replied, I paid no attention to the fact the other person had left the conversation... as it still shows that person as a participant, even though they left the conversation.
A delete button would certainly eliminate any confusion. A Google search testifies to the fact there is obviously a lot of confusion about it.