"This ingenious gallop in the mind of Steven Landsburg allows you to Imagine like an economist without incurring a Keynesian headache or an enormous pupil bank loan.txt to disallow almost everything or precise endpoints (hackers can still lookup robots.txt for endpoints) which stops Google bots from crawling delicate endpoints such as admin panels.