Web developmentrobots.txt DetectionWernerCheck whether a website has robots.txt. After configuration, this extension can detect whether other paths exist.Screenshots from Mozilla Add-ons