robots.txt

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • AndyUK
    Blueparker
    • Apr 2009
    • 95

    #1

    robots.txt

    is there a way to add anything to the robots.txt file?

    I want to add disallow lines to all but the live template.

    Disallow: /?template=mytemplate1
    Disallow: /?template=mytemplate3

    as the page

    www.mysite.com/?template=mytemplate1

    is still indexed despite the fact i am now on mytemplate2

    i have got round it by using a canonical element in the default page, but as mum said "a tidy house is a happy house" and i have ocd!



  • Dave
    Senior Blueparker
    • Dec 2007
    • 3981

    #2
    Hi Andy,

    Is it not just the case that the search engines have not updated their cached version of your website yet? If so it's only a matter of time before it picks the changes up!

    With regards to the robots.txt this was posted by Rich in an earlier post
    "You don't have access to this file, as you can't access the true root directory (the "root" directory in your File Manager is actually your domain name forward-slash "user")."
    Regards,

    Dave

    Comment

    • AndyUK
      Blueparker
      • Apr 2009
      • 95

      #3
      We get crawled and updated a few times a day, but there are still pages that some how include old tempaltes, and also when you look at google webmaster tools at the crawl errors there are prob 20 in there to the old templates.

      canonical helps, but it's a pain to have to inlcude on every page

      Comment

      Working...