# This says to apply these settings to ALL search engine spiders/crawlers # These settings will keep spiders from indexing your unwanted pages # This assumes that your robots.txt is in your web site's ROOT directory # ie: http://www.ggrego.com/index.html # User-agent: Googlebot-Image # Disallow: / User-agent: * Disallow: /admin/ Disallow: /cgi-bin/ Disallow: /cache/ Disallow: /db/ Disallow: /cp/ Disallow: /flash/ Disallow: /gallery/ Disallow: /hidden/ Disallow: /pub/ Disallow: /weblaizer/ Sitemap: http://www.ggrego.com/sitemap.xml