Rare find

HuggingFace-Datasets-Text-Quality-Analysis. Retrieves parquet files from Hugging Face, identifies and quantifies junky data, duplication, contamination, and biased content in dataset using pandas

github.com/onesuper/HuggingFace-Datasets-Text-Quality-Analysis

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

No recent activity.