Devtantra Blog

Tutorials about Android, Angular, ReactJS, PHP, MySQL and Web Development

Tag: robot.txt

Work with robot.txt

Your site content is very important for your Site . The site crawl take most important part of your site , robot.txt tells search engine how they should visit your site .If you want to block specific search engine & also directories. So by the help of robot.txt you can do the job easily .The […]