Protecting your Files from Google

• A robots.txt file restricts access to your site by search engine
robots that crawl the web. These bots are automated, and before
they access pages of a site, they check to see if a robots.txt file
exists that prevents them from accessing certain pages.



• You need a robots.txt file only if your site includes content that
you don't want search engines to catch. If you want search
engines to index everything in your site, you don't need a
robots.txt file (not even an empty one).


Example of a Robot txt file is below

 

If  you do not want to  display your  site  in  Google  search results then add this robot.txt file in your site and empty robot.txt file also can prevent the crawling and displaying your site in the Google search results.

If  you want  to  your site to be displayed in search results of Google then remove this robot.txt file from your site.

referred from articles.