FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum » FUDforum Suggestions » recognise google (/other) rather than having zillions of anon users
Show: Today's Messages :: Polls :: Message Navigator
Return to the default flat view Create a new topic Submit Reply
Re: recognise google (/other) rather than having zillions of anon users [message #26867 is a reply to message #26850] Thu, 18 August 2005 11:51 Go to previous messageGo to previous message
djzort is currently offline  djzort   Australia
Messages: 30
Registered: May 2005
Karma:
Member
Ilia wrote on Thu, 18 August 2005 00:00

Most people want their forum to be indexed by search engines, so a default robots.txt would be a bad idea. People who want to avoid indexing usually know enough to use robots.txt already.



just detect google probing then
[Message index]
 
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Previous Topic: registered users readable forums/categories
Next Topic: QuickReply
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Sun Nov 10 11:01:39 GMT 2024

Total time taken to generate the page: 0.20821 seconds