Rewrite conditions being ignored

[mobilebadboy]

I'm trying to block various spider/spam/harvester bots, and although pretty much every site gives the same code, it doesn't seem to work for me. After adding the rules I still see them coming back. And I've even spoofed my UA to do some testing, and I'm allowed to visit the site every time. Here's what I have:




RewriteCond %{HTTP_USER_AGENT} ^almaden [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^ASPSeek [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^DTS\ Agent [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^iaea.org [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^InternetSeer [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Microsoft\ URL\ Control [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^NPbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^QuepasaCreep [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^TurnitinBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [NC]
RewriteRule ^/.* - [F]

[chromate]

I don't know anything about Rewrite but you could easily use PHP to test for user agents like that.

[mobilebadboy]

Yeah. If it was just one site and only a couple of bots I'd normally just do it that way. But since I plan to use this on all of my sites, and since all of my sites aren't set up the same (some sites use 1 common header, some don't) it'd be a whole lot easier to just edit 9 or 10 htaccess files. Plus I plan to expand the lists over time.

And some of my sites (like visual intensity) already have a couple of hundred lines of detect/control code in the header. :p I'm trying to limit that more and more.

[mobilebadboy]

Well, I ran across diveintomark.org and got it working. I noticed in his examples he was putting things into quotes. Every other site gave examples as posted above. I put everything into quotes and they all worked. Thanks Mark (http://diveintomark.org)!