« Huhu you say? | Main | When nothing else will do »

Sunday, February 23, 2003

User agents and stuff

Someday I'm going to put this all together so I can not only understand it, but actually do something valuable with it. But for now, here are some links re various user agents, banning IPs, and such. Most of these, but not all, were found at WebMasterWorld on the Search Engine Spider Identification category. The links, in no particular order:

• Search Engine IP Addresses
• SpiderHunter.com
• Search Engine Hunter
• List of User Agents
• Database of Web Robots
• BotSpot
• Robots, Spiders and Other UA's
• Fleiner on how to keep out bad bots
• Robot Cop
• Open Directory: .htaccess
• Comprehensive guide to .htaccess
• Ralf D. Kloth & List of Bad Bots & How to build a Bot Trap

That should be enough for now.

Posted by Marie at February 23, 2003 2:02 PM