Honker: The Rust Extension That Turns SQLite Into a Message Queue and Pub/Sub System
What if your humble SQLite database could do the job of Redis, RabbitMQ, and Kafka — all from a single `.db` file?
That's the promise of **Honker**, a new Rust-based SQLite extension by Russell Romney that brings PostgreSQL-style NOTIFY/LISTEN semantics to SQLite. Instead of standing up separate infrastructure for message queues or event streams, Honker packages everything into the database you already have.
The extension provides three core primitives: ephemeral pub/sub via `notify()`, durable Kafka-style streams with per-consumer offset tracking, and at-least-once work queues with automatic retries and dead-letter tables.
What makes Honker technically clever is its wake mechanism. Rather than expensive application-level polling, it monitors SQLite's `PRAGMA data_version` — a monotonic counter that increments on every commit. A dedicated thread checks this value every millisecond, achieving single-digit millisecond cross-process notifications with negligible overhead.
The real killer feature is the **transactional outbox pattern**: you can enqueue jobs inside the same database transaction as your business writes. Both commit or rollback together — no dual-write problems, no eventual consistency headaches.
Honker ships with bindings for seven languages (Python, Node.js, Go, Ruby, Elixir, Bun, and Rust), a built-in crontab-style scheduler with leader election, and optional task result storage.
The trade-off? It's single-machine only — SQLite's locking model doesn't survive NFS. But for small teams, side projects, or any application where operational simplicity matters more than horizontal scale, Honker eliminates an entire category of infrastructure.
📄 Source
Simon Willison