CIW - Certified Internet Web Professional Practice Test
CIW CIW Site Development Associate 3
What is the purpose of the robots.txt file on a website?
Select your answer
A
Detect and block malicious bots
B
Instruct search engine crawlers which pages to crawl or avoid
C
Define CSS styles for automated browsers
D
Set server response time limits
Hint