Clipper is a container registry that does file-level deduplication across layers and across images. By ditching tarballs, we get up to 10x faster pulls and 7x faster builds. Clipper is drop-in compatible with your existing container workflows: ull with the Clipper client, get standard OCI images on disk. Same workflow, dramatically less data stored and transferred. Compatible with Docker, Kubernetes, Podman, containerd, AppImage/sif, and others.
No reviews yetBe the first to leave a review for Clipper
Maker
📌
Working in robotics, I oftentimes suffered from a problem: updating one dependency in a Docker image resulted in shipping gigabytes of other dependency data that my robot already had, trapped in other layers. In trying to fix this, I stumbled upon many other performance improvements that has lead me to one conclusion: the current container ecosystem just isn't built for large images. I've been told "keep your images under 1GB" which just isn't feasible for anything using ML. There's a bit of a blindspot in the Docker community around this area, and I want to fill that need.
Report
How does the deduplication actually hold up in practice when images share a lot of small config files but differ in the actual binary layers?
Report
Maker
@sabanl23423 There's less of an advantage, here. There's still speed benefits when pulling, but small files are currently bundled up together to reduce the number of HTTP fetches.
Report
plugged it into our k8s cluster and the pull times on our larger images dropped noticeably without changing anything in our pipelines. really nice that it just slots in next to docker
How does the deduplication actually hold up in practice when images share a lot of small config files but differ in the actual binary layers?
@sabanl23423 There's less of an advantage, here. There's still speed benefits when pulling, but small files are currently bundled up together to reduce the number of HTTP fetches.
plugged it into our k8s cluster and the pull times on our larger images dropped noticeably without changing anything in our pipelines. really nice that it just slots in next to docker