1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Sitemap help

Discussion in 'Server Configuration and Hosting' started by gavpeds, Oct 22, 2014.

  1. gavpeds

    gavpeds Active Member

    Not sure if this is the right area to post but just want your feedback on the errors i am getting.

    wmt.png

    I dont understand why i am getting 19,000 warnings on three urls, any help on this would be great.

    My robots text is this.
    Code:
    User-agent: Mediapartners-Google
    Disallow:
    
    User-agent: Adsbot-Google
    Disallow:
    
    User-agent: Googlebot-Mobile
    Disallow:
    
    User-agent: AhrefsBot
    Disallow: /
    
    User-agent: Baidu
    Disallow: /
    
    User-agent: Baiduspider
    Disallow: /
    
    User-agent: Baiduspider-video
    Disallow: /
    
    User-agent: Baiduspider-image
    Disallow: /
    
    User-agent: Cliqzbot
    Disallow: /
    
    User-agent: Diffbot
    Disallow: /
    
    User-agent: DotBot
    Disallow: /
    
    User-agent: EasouSpider
    Disallow: /
    
    User-agent: Exabot
    Disallow: /
    
    User-agent: linkdexbot
    Disallow: /
    
    User-agent: linkdexbot-mobile
    Disallow: /
    
    User-agent: magpie-crawler
    Disallow: /
    
    User-agent: meanpathbot
    Disallow: /
    
    User-agent: MJ12bot
    Disallow: /
    
    User-agent: NaverBot
    Disallow: /
    
    User-agent: omgilibot
    Disallow: /
    
    User-agent: proximic
    Disallow: /
    
    User-agent: Rogerbot
    Disallow: /
    
    User-agent: SiteBot
    Disallow: /
    
    User-agent: sogou
    Disallow: /
    
    User-agent: sogou spider
    Disallow: /
    
    User-agent: Sogou web spider
    Disallow: /
    
    User-agent: spbot
    Disallow: /
    
    User-agent: trendictionbot
    Disallow: /
    
    User-agent: Twiceler
    Disallow: /
    
    User-agent: URLAppendBot
    Disallow: /
    
    User-agent: Yandex
    Disallow: /
    
    User-agent: YoudaoBot
    Disallow: /
    
    User-agent: Yeti
    Disallow: /
    
    User-agent: BoardReader
    Disallow: /
    
    User-Agent: *
    Disallow: /?page=
    Disallow: /find-new/
    Disallow: /account/
    Disallow: /attachments/
    Disallow: /goto/
    Disallow: /posts/
    Disallow: /login/
    Disallow: /admin.php
    Disallow: /conversations/
    Allow: /
    
    Sitemap: http://windowcleaningforums.co.uk/sitemap.php
     
  2. gavpeds

    gavpeds Active Member

    Anyone?
     

Share This Page