Launching today

Data-Bridge
Data Bridge – turn any database into webhooks
4 followers
Data Bridge – turn any database into webhooks
4 followers
Build webhook automations from your data — create hooks/jobs that stream rows from any database (Postgres, MySQL, SQLite, Mongo, Redis) to any HTTP endpoint. Durable, templated, resumable. - ozmanghani/data-bridge


Hey PH, I'm Osman 👋
bit of backstory on why I made this. I was working as the lead software
developer at a company, building out their HRMIS, the system that runs all the
HR workflows for 500+ employees across a few countries. most of it ran itself,
but one piece kept biting us: getting attendance off the biometric devices into
the system. those devices drop data sometimes (network, power, random sync
issues), and in an HR system a missing punch isn't a small bug. it's someone's
payroll.
so to fix it I wrote a service that sits next to the device, watches for new
records, and pushes them up the second they appear, with retries and
resume-on-crash so nothing slips through. it solved our problem, and then it
kind of dawned on me that the attendance part wasn't really the hard part. the
actual problem was generic: catch a change in a database and get it somewhere
live without dropping a row. so I pulled that idea out, cleaned it up, and open
sourced it as Data Bridge.
the main thing it does is live, event-driven hooks. you connect a database and
it taps straight into the change log (Postgres logical replication, MySQL
binlog, Mongo change streams, Redis keyspace events). every insert/update/delete
fires the moment it happens and gets POSTed to whatever endpoint you want, with
the operation attached. no polling, no cron jobs.
delivery is the part I cared about most since that's what burned us originally.
runs are durable so a crash resumes exactly where it stopped, redelivery is
idempotent, and there's a live timeline where you can click any event and see
the exact request, the response, timing, and copy it as a curl command. you set
it all up visually too, pick the table and columns, shape the json, done.
there's also a one-shot replay mode and a polling mode for backfills or DBs that
can't do CDC, but honestly the live sync is the reason I built it.
it's free and MIT licensed. would really appreciate any feedback, especially on
the CDC side and how to make the delivery guarantees tighter.
what's next: letting a database sync straight into another database, not just to
an HTTP endpoint. so you could pipe one db's data into another (Postgres →
Postgres, Mongo → Postgres, etc), or keep multiple databases continuously in
sync with each other so a change in one shows up live in the rest. is that
something you'd use? curious which source/target pairs people want most 👇
i'll be around in the comments all day 🙏