# Disallow all crawlers access to certain pages. User-agent: * Disallow: /buttons/ Disallow: /city/ Disallow: /dealer/ Disallow: /gif/ Disallow: /htmlblock/ Disallow: /include/ Disallow: /jpg/ Disallow: /oldstuff/ Disallow: /periodical/ Disallow: /state/