url-harvester. A web crawling tool that collects URLs from a given domain and stores them in Parquet format.

github.com/mihneamanolache/url-harvester

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

No recent activity.