robots.txt for WordPress - short & sweet
Sunday July 9th, 2006
On my first Wordpress site I went and added all of the files I didn't want search engines indexing to my robots.txt file thinking that it was necessary to keep them out of the SERPS. But Wordpress makes it easier than I was trying to make it by starting almost every file in the root, and all subfolders, with 'wp-'. With a 2 line robots.txt file you can block most everything that needs to be blocked.
CODE:
-
User-agent: *
-
Disallow: /wp-
Alexander on October 26th, 2006 at 11:03 am
Wow!
I was thinking about exactly this way, just was looking for a ready-made file to copy-paste :)
Unfortunately this isn’t the place to copy the whole thing directly, but anyway, nice to see like-minded folks