I tried adding the path to the node to my robots.txt file and that did not help. I realize the path to all threads is /threads. Can I use a wildcard with robot.txt inline?
What I mean is all the threads begin as:
Can I add www.mysite.com/threads/tweet-from-* to the robots.txt file?