Google AdSense Training
SEO Training ChennaiSEO Training in Chennai by Perambur Kumar

SEO Tips: #4 Robots.txt

SEO Tips: #4 Robots.txt

  1. First make sure your file access by Search Engines
  2. Hide Duplicate Content
  3. Put your Sitemap in Robots.txt at the end of the coding.
  4. Make sure you don’t Block Google from Crawling.
  5. URL Structure: http://www.yourdomainname.com/robots.txt

Related Topics on Robots.txt

  1. Learn More About Robots.Txt Files.
  2. Create a Robots.Txt File
  3. Test Your Robots.Txt with the Robots.Txt Tester
  4. Submit Your Robots.Txt Updated File to Google Webmaster Tools
  5. Robots.Txt File – FAQ

Please leave your comments below, share this post with your friends.

Give Like if you like it. Thank you

SEO Tips: #4 Robots.txt

Please copy the below code and paste it in Google Webmaster Tools – Robots.txt tester page.

I will be back tomorrow with another set of SEO Tips. Bye

Sample of Robots.txt file

User-agent: Mediapartners-Google*
Allow: /
User-agent: Googlebot-Image
Allow: /
User-agent: Adsbot-Google
Allow: /
User-agent: Googlebot-Mobile
Allow: /
User-agent: Twitterbot
Allow: /
User-agent: *
Disallow: /search
Disallow: /cgi-bin/
Allow: /
User-agent: *
Disallow: /*.html
Allow: /*.html$
Sitemap: http://www.yourdomainname.com/sitemap.xml
3 Comments

Add a Comment

Your email address will not be published. Required fields are marked *

CommentLuv badge