Pachyderm

Joe Doliner and Joey Zwicker founded Pachyderm in 2014 and joined Y Combinator W15 to bring software-version-control ideas to datasets and data pipelines. Pachyderm combined a Git-like data repository/versioning layer (Pachyderm File System/PFS) with containerized, Kubernetes-native pipelines (PPS), automatic incremental processing, lineage and immutable provenance so a team could reproduce exactly which code and data generated an output. The open-source product targeted computer vision, life sciences, geospatial, defense and other workloads where large unstructured datasets change over time. Enterprise editions added access controls, support, deployment/administration features and collaboration/governance. The company raised about $28m total: seed/Series A capital from YC, Benchmark, Foundation Capital and others, then a $16m Microsoft M12-led round in August 2020; HPE Pathfinder invested strategically in 2022. Hewlett Packard Enterprise announced its acquisition on January 12, 2023 and closed that month for undisclosed terms, integrating the technology with HPE's Machine Learning Development Environment/System and broader HPC/AI portfolio. Roughly 50-60 employees joined/formed part of the transaction. The independent Pachyderm commercial organization no longer operates as a standalone venture, though its open-source repositories/releases and HPE-integrated capabilities have persisted to varying degrees. Pricing from the independent period was open-source Community at no license fee and custom Enterprise/cloud contracts; current HPE pricing is solution/quote based. Risks include Kubernetes/operational complexity, overlap with cloud-native orchestrators and lakehouse/versioning tools, uncertain standalone roadmap after acquisition, and governance/security duties around high-value training data.

Founding story

Doliner and Zwicker believed data science lacked the reproducibility guarantees software engineers got from Git and immutable builds, so they created a data-aware pipeline system that versioned inputs, code execution and outputs together.

Business model

Historical open-source core plus enterprise support/software subscription; technology now monetized within HPE AI/HPC systems and services.

Before acquisition: custom annual enterprise licenses/support and managed/cloud deployment around a free open-source core. After acquisition: bundled/custom HPE AI solution revenue.

Traction

YC lists team size 60 and acquired status; contemporary reports cited roughly 50-55 employees and $28m funding. Customer, ARR and current HPE deployment counts were never disclosed consistently.

Latest developments

Pachyderm capabilities have been folded into HPE's machine-learning development and AI-at-scale platform alongside Determined AI and HPE supercomputing; HPE positions reproducible pipelines as part of end-to-end production AI.