This is an integration environment. Data may not be preserved.
Skip To Content
Toggle navigation
Packages
Upload
Statistics
Documentation
Downloads
Blog
Sign in
Advanced search filters
Frameworks
Include compatible frameworks
Framework Filter Mode
ALL
ANY
.NET
net8.0
net7.0
net6.0
net5.0
.NET Core
netcoreapp3.1
netcoreapp3.0
netcoreapp2.2
netcoreapp2.1
netcoreapp2.0
netcoreapp1.1
netcoreapp1.0
.NET Standard
netstandard2.1
netstandard2.0
netstandard1.6
netstandard1.5
netstandard1.4
netstandard1.3
netstandard1.2
netstandard1.1
netstandard1.0
.NET Framework
net481
net48
net472
net471
net47
net462
net461
net46
net452
net451
net45
net40
net35
net30
net20
Package type
All types
Dependency
.NET tool
Template
Options
Include prerelease
12 packages returned for Tags:"Spider"
Sort by
Relevance
Downloads
Recently updated
HtmlAgilityPack
by:
darthobiwan
.NET Framework 2.0
693,435 total downloads
last updated
9/16/2014
Latest version:
1.4.9
HTML
parse
crawler
spider
linq
This is an agile HTML parser that builds a read/write DOM and supports plain XPATH or XSLT (you actually don't HAVE to understand XPATH nor XSLT to use it, don't worry...). It is a .NET code library that allows...
More information
Abot
by:
sjdirect
.NET Framework 4.0
13,568 total downloads
last updated
4/4/2015
Latest version:
1.5.0.93
crawler
robot
spider
Abot is an open source C# web crawler built for speed and flexibility. It takes care of the low level plumbing (multithreading, http requests, scheduling, link parsing, etc..). You just register for events to...
More information
SkyScraper
by:
JonCanning
.NET Framework 4.5
914 total downloads
last updated
6/14/2013
Latest version:
1.0.45
spider
crawler
scraper
bot
Web scraper / crawler / spider. Supports robots protocol and user agent.
MisterHex.
WebCrawling
by:
misterhex
.NET Framework 4.5
407 total downloads
last updated
4/2/2013
Latest version:
1.0.0.5
crawler
rx
crawling
spider
Simple web crawler that return IObservable using Reactive Extension(Rx) and async await.
NCrawler.
HtmlProcessor
by:
codevision
.NET Framework 4.5
218 total downloads
last updated
1/1/2015
Latest version:
3.0.0
NCrawler
crawler
spider
robot
HTML processing of crawled data for NCrawler
NCrawler
by:
codevision
.NET Framework 4.5
376 total downloads
last updated
1/2/2015
Latest version:
3.0.1
NCrawler
crawler
spider
robot
Crawler and scrapping framework which is written in C#
NCrawler.
FileStorageServices
by:
codevision
.NET Framework 4.5
135 total downloads
last updated
1/1/2015
Latest version:
3.0.0
NCrawler
crawler
spider
robot
Storage in file system for the NCrawler
NCrawler.
EntityFramework
by:
codevision
.NET Framework 4.5
236 total downloads
last updated
1/2/2015
Latest version:
3.0.2
NCrawler
crawler
spider
robot
Provides storing crawler data using EF for NCrawler
NCrawler.
EsentServices
by:
codevision
.NET Framework 4.5
124 total downloads
last updated
1/1/2015
Latest version:
3.0.0
NCrawler
crawler
spider
robot
This is ESENT storage providers for the NCrawler
Spidy
by:
Taha
.NET Framework 4.0
141 total downloads
last updated
9/24/2012
Latest version:
1.0.0
FSharp
Web
Crawler
Spider
Webbot
Bot
FSharp Web Crawler
CrawlerLib.
Engine.
TestHelper
by:
Crawler-Lib
.NET Framework
863 total downloads
last updated
3/7/2015
Latest version:
2.3.5544.21265
crawler-lib
test
helper
nunit
mstest
unit-test
crawler
spider
bot
scraper
More tags
The Crawler-Lib Engine Test Helper simplifies the test of tasks. It can be used to develop unit tests and integration tests for tasks.
CrawlerLib.
Engine
by:
Crawler-Lib
.NET Framework
1,386 total downloads
last updated
3/7/2015
Latest version:
2.3.5544.21265
crawler-lib
task
processor
workflow
action
async
await
crawler
spider
bot
More tags
The Crawler-Lib Engine is a general purpose workflow enabled task processor. It has evolved from a web crawler over data mining and information retrieval. It is throughput optimized and can perform thousands of...
More information