I forgot where the setting is to block all search engines and spiders . Let me know where it is . Also if i just make a robot.txt to disallow all is that the best way to not duplicate content.
Yes, that will stop all robots that respects robots.txt. Your best way to stop it from being indexed though is to prevent the link from being found, or protect the directory in some manner.
Yes, then they wouldn't get access to the test folder without the password, and probably end up with a 403 error or something like that. I have that, plus disallow / and never put the link to my test site anywhere a crawler can see it.
But as said above, only spiders that follow it will obey. Many don't and will ignore what's in your robots.txt file. If you want to use htaccess to password protect a folder, this site is very good for doing it easy. http://tools.dynamicdrive.com/password/