FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum » How To » Robots.txt for search engines (How to get search engines to index only messages)
Show: Today's Messages :: Polls :: Message Navigator
Switch to threaded view of this topic Create a new topic Submit Reply
Robots.txt for search engines [message #35819] Wed, 14 February 2007 04:30 Go to next message
jcarter is currently offline  jcarter   United States
Messages: 25
Registered: August 2006
Karma: 0
Junior Member
I would like to see if anyone has a robots.txt for the current version of FUDforum. There are some older ones here, but it doesn't look like they would work with the current version which uses URL arguments rather than subfolders for all the features.

It seems like they are indexing everything many times - reply messages, etc.
Re: Robots.txt for search engines [message #35823 is a reply to message #35819] Wed, 14 February 2007 14:09 Go to previous message
jcarter is currently offline  jcarter   United States
Messages: 25
Registered: August 2006
Karma: 0
Junior Member
Here's what I have so far - I don't know yet how it will affect google.

User-agent: *

Disallow: /FUDforum2/index.php?t=tree
Disallow: /FUDforum2/index.php?t=post
Disallow: /FUDforum2/index.php?t=mnav
Disallow: /FUDforum2/index.php?t=selmsg
Disallow: /FUDforum2/index.php?t=search
Disallow: /FUDforum2/index.php?t=help_index
Disallow: /FUDforum2/index.php?t=report
Disallow: /FUDforum2/index.php?t=buddy_list
Disallow: /FUDforum2/index.php?t=ignore_list
Disallow: /FUDforum2/pdf.php
Disallow: /FUDforum2/rdf.php


  Switch to threaded view of this topic Create a new topic Submit Reply
Previous Topic: how to add a link before the members button at the top
Next Topic: just a question
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Sun Nov 10 05:12:52 GMT 2024

Total time taken to generate the page: 0.02981 seconds