Home »
FUDforum »
How To »
Spiders and Bots
| Re: Spiders and Bots [message #165988 is a reply to message #165961] |
Fri, 02 September 2011 20:34   |
The Witcher
Messages: 675
Registered: May 2009
Location: USA
Karma:
|
Senior Member |
|
|
I guess you missed my point: I wasn't sure what a user agent strung is, and from what I found I was uncertain of which portion of it to use! Nor was I certain of how to identify spiders and bots beyond individual searches or online lists.
As for "JPNIC" I could not identify them as a bot or spider or explain all the reply or page not found Fud errors they were generating and I didn't just want to ban a range of IP's without knowing since the forums subject has an international user base.
But I think this HERE explains it.
As for "Spiders and Bots"
I defined Google as a test on a backup. Today Google is the newest registered user, on logging out I see:
Warning: preg_match() [function.preg-match]: Unknown modifier '5' in /home/user/domain.name/index.php on line 309
309 if (preg_match('/^'. $spider['useragent'] .'/i', $_SERVER['HTTP_USER_AGENT'])) {
if (empty($spider['bot_ip'])) {
Quote:Defined spider:
Google: GoogLe:Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) : xx.xxx.xx.xxx
I Deleted the '/5.0" from spider definition and checked again:
Warning: preg_match() [function.preg-match]: Unknown modifier '2' in /home/user/domain.name/index.php on line 309
I Deleated: "(compatible; Googlebot/2.1; +http://www.google.com/bot.html)" from spider definition.
No further warnings yet! So it appears that you just need the user name and browser type.
"I'm a Witcher, I solve human problems; not always using a sword!"
|
|
|
|
Current Time: Sun Nov 24 13:09:28 GMT 2024
Total time taken to generate the page: 0.03837 seconds
Calendar
Search
Help
Members
Register
Login
Home
]