# /robots.txt file for http://webcrawler.com/ # mail jhepler@wfubmc.edu for assistance User-agent: * Disallow: /admin Disallow: /admintools Disallow: /abstracts