Launching today
Faktory
Reliable background jobs for any language, over one protocol
10 followers
Reliable background jobs for any language, over one protocol
10 followers
Faktory is an open-source background job server by Mike Perham, the creator of Sidekiq. Producers push jobs as JSON and workers fetch them over a small protocol, so any language can use it: official Ruby and Go clients, community ones for Python, Node, PHP, Rust, Elixir, Java,NET and more. A fetched job is reserved, not removed: if a worker crashes, it comes back. Failures retry with backoff, then land in a Dead set you can re-run. Web UI included.
Hunting this one for its design, not because it's new: it's been around since 2017 and v1.10.0 shipped in August. I'm an engineer who's read through the docs and the protocol spec more than once, and I still think it's one of the cleanest job servers there is.
The core idea: Faktory is a server, not a library. Producers PUSH JSON jobs, workers FETCH them, over a small text protocol (FWP, port 7419) that anything can speak. Ruby and Go clients are official; the community covers Python, Node, PHP, Rust, Elixir, Java, .NET and more. One queue can sit between very different codebases without a shared client library.
The design choice I admire most is that a fetched job isn't a removed job. It's reserved (30 minutes by default) until the worker ACKs or FAILs it, so a crashed worker just lets the reservation lapse and the job comes back. Failures retry 25 times by default, with a growing, jittered backoff over roughly three weeks, then go to a dead set you can inspect and re-run from the UI once you've shipped the fix. That's at-least-once delivery, so write idempotent handlers, but the failure behavior is written down and predictable, and that's most of what you want from a queue.
Two more things I like: the Web UI ships with the server (queues, busy workers, retries, dead jobs), and the open-source/Enterprise line is easy to read. The AGPL core covers the queue, retries and UI; cron, unique and expiring jobs, batches and throttling are the paid Enterprise edition, from $299/mo, with no licence needed in development.
All credit to Mike Perham and contribsys, the same person who wrote Sidekiq — he's here as the maker, so questions about the roadmap or Enterprise are best asked of him. I'm not affiliated with Faktory or contribsys, just a fan of the design. Curious what people here use for crashed-worker semantics today.
@zhukmax thank you for the kind words Max!
the reservation model makes the failure case easy to reason about
a worker disappearing should not mean the job disappears too keeping the retry and dead job views in the same ui helps