I have an RSS feed which automatically posts in a forum. This forum is used exclusively for this feed. I want to block this forum from being crawled. What is the best method? I tried adding the path to the node to my robots.txt file and that did not help. I realize the path to all threads is /threads. Can I use a wildcard with robot.txt inline? What I mean is all the threads begin as: www.mysite.com/threads/tweet-from- Can I add www.mysite.com/threads/tweet-from-* to the robots.txt file?