################################################## # Sample robots.txt file # # Courtesy Cyboretum.com hosting # # # # Basic convention: # # Specify user agent, followed by what # # the specified user agent can or can not do # ################################################## # # Typically you address any and all user agents User-agent: * # list folders robots are not allowed to index # Directories: # # Disallow: /test/ Disallow: /ar/ Disallow: /artic/ Disallow: /bak/ Disallow: /cbms/ Disallow: /dev/ Disallow: /fc/ Disallow: /fp/ Disallow: /r/ Disallow: /rs/ Disallow: /share/ Disallow: /Library/ Disallow: /Templates/ # list specific files robots are not allowed to index # Files: # # Disallow: /english/index-new.php # # ##################################################