LxL

scrapydweb. Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. Docs 文档 :point_right:

files. Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

scrapyd-cluster-on-heroku. Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO :point_right:

logparser. A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

scrapyd. [PR #326] Native support for basic auth :lock: `pip install -U git+https://github.com/scrapy/scrapyd.git`, then add `username = yourusername` and `password = yourpassword` in the scrapyd.conf file. DOCS :point_right:

scrapyd-cluster-on-heroku-scrapyd-app. How to set up Scrapyd cluster on Heroku

awesome-scrapy. A curated list of awesome packages, articles, and other cool resources from the Scrapy community.

awesome-python-cn. Python资源大全中文版，包括：Web框架、网络爬虫、模板引擎、数据库、数据可视化、图片处理等，由伯乐在线持续更新。

awesome-crawler. A collection of awesome web crawler,spider in different languages

notes. Keep on reading

awesome-web-scraping. List of libraries, tools and APIs for web scraping and data processing.

Python-Algorithms. All Algorithms implemented in Python

awesome-flask. A curated list of awesome Flask resources and plugins

awesome-python. A curated list of awesome Python frameworks, libraries, software and resources

awesome-python-applications. 💿 Free software that works great, and also happens to be open-source Python.

scrapyd-cluster-on-heroku-scrapydweb-app-git. How to set up Scrapyd cluster on Heroku

scrapyd-CircleCI. Python

scrapy-redis. Redis-based components for Scrapy.

public-test. public-test