Playscrape. A tool that visits websites and saves what it finds.

github.com/jeresig/playscrape

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

November 2024
  • Upgrade Yarn.
October 2024
  • Rename to shouldRevisit
  • Fix up some mirrored file logic.
June 2024
  • Add in image download testing.
  • Add shouldVisit, handle skipped visit, fix imageDir, fix hanging spin…
May 2024
  • Support retries.
  • Output more logging info, fix up timeout.
  • Update drizzle-kit.
  • Fix a number of bugs, switch to using PG instead of SQLite.
  • Improve exported data to include downloads and other fields.
  • Add in source to scrapes/records.
  • Remove debug code.
  • Fix a number of issues with the browser scraping logic.
  • Fix up some extract test bugs.
  • Re-build schema from scratch.
  • Undo last drizzle migration.
  • Fix up Drizzle output.
  • Merge branch 'main' of github.com:jeresig/playscrape
  • Upgrade to Yarn v4.
  • Release —v0.2.1
  • Version Packages (#17)
  • Release —v0.2.0
April 2024
  • Release —v0.1.1
January 2024
  • Release —v0.1.0
December 2023
  • Release —v0.0.13
  • Release —v0.0.12
  • Release —v0.0.11
  • Release —v0.0.10
  • Release —v0.0.9
November 2023
  • Release —v0.0.8