Martin Durant

dask-tutorial-scipy-2018. Materials for "Parallelizing Scientific Python with Dask"

rfsspec. Rust python FS

fastparquet. python implementation of the parquet columnar file format.

async-zarr. Hack wrapper around zarr to make data access async

dask-gke. kubernetes setup to bootstrap distributed on google container engine

daskberg. dask client for iceberg (super-alpha)

splunk_connector. Splunk to dataframes via REST access

libhdfs3-downstream. a native c/c++ hdfs client (downstream fork from apache-hawq)

pydata_global_2024. Material for "akimbo: vectorized processing of nested/ragged dataframe columns"

gcsfs. Pythonic file-system interface for Google Cloud Storage

misc. Python tidbits that may be useful

intake. A plugin system for loading your data and making data catalogs.

dask. Versatile parallel programming with task scheduling

intake-release-blog. Materials related the release notice of the Intake project

lance. Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more integrations coming..

thriftlike. Playing thrift with rust

fastfeather. Feather to pandas loader without pyarrow

libhdfs3-feedstock. A conda-smithy repository for libhdfs3.

blog_fsspec. Article to appear on anaconda.com