opti-pipe reads your Spark/dbt pipeline config alongside real run metrics and tells you what’s misconfigured — with the reasoning shown, not just a number to change. Rule-based and fully auditable, nothing auto-applies. dbt has a real integration (upload your own run_results.json); Spark runs on sample data for now.
No reviews yetBe the first to leave a review for opti-pipe
Maker
📌
Hey PH 👋 Built this because I kept seeing the same pattern on every data team I worked with: someone senior ‘just knows’ a job’s OOM-ing because memory’s sized for last quarter’s data volume, or that shuffle partitions are way overcounted — and that knowledge lives in one person’s head, not anywhere checkable.
opti-pipe makes that pattern-matching explicit: 7 threshold rules, each fully auditable
You can see exactly why a recommendation fired. Nothing ML/black-box, and nothing auto-applies — you approve or you don’t.
Try the live demo (no signup): opti-pipe.onrender.com. Would genuinely love feedback, especially ‘this rule is wrong’ — that’s the fastest way I improve it.”
Let me know once it’s live and I can help draft replies to early comments if you want.