Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.
robots. txt is a file that can be placed in the root folder of your website to help search engines index your site more appropriately. Search engines such as Google use website crawlers, or robots that review all the content on your website.
