There are a handful of other ways to prevent content appearing in
search results, such as adding "NOINDEX" to your robots meta tag,
using .htaccess to password protect directories, and using Google
Webmaster Tools to remove content that has already been crawled.
Google engineer Matt Cutts walks through the caveats of each URL