tsllbn RobotsTxt by: tsllbn .NET Framework This package is compatible with all versions of .NET Framework. 1,005 total downloads last updated 2/19/2014 Latest version: 2014.2.19 Robots.Txt Parser Parsing Library A robots.txt parser for .NET Supports ; - Allow directives. - Crawl-delay directives. - Sitemap declarations. - * and $ wildcards.