IronTide
A from-scratch Rust BitTorrent engine, published to crates.io and daily-driven as my torrent client.
BitTorrent looks simple on paper and is brutal in code. Hundreds of peers, untrusted bytes on every wire, an unreliable network, and mutable shared state. I picked it as a first serious systems project because there is nowhere to hide: every layer has to actually work, on real torrents against real swarms, or nothing downloads.
Libtorrent-class parity: a library other applications can embed, with configurable transport, pluggable storage, and deterministic behaviour under load. It had to work on real torrents against real swarms, and read clearly enough that someone else could follow what is happening. That target is met. The engine is published on crates.io, split cleanly from its client, and running as my daily torrent client.
Async actors over shared state
A BitTorrent client has dozens of things happening at once: peers connecting and dropping, pieces being verified, trackers being polled, the UI updating. Locks everywhere is the obvious approach and a debugging nightmare. Instead each peer is its own tokio task, each piece-picker its own task, each tracker session its own task, all communicating over typed channels. tokio's select! is the load-bearing primitive. Structured concurrency is not optional here; without it the system becomes impossible to reason about within days.
The wire protocol, byte by byte
Hand-rolling the wire protocol meant a length-prefixed message framer, a bencode parser that does not allocate per key, peer handshake with extension negotiation (BEP-10), uTP for UDP transport, Mainline DHT for trackerless discovery, and PEX for peer exchange. uTP taught me that TCP-on-UDP congestion control is harder than it looks: LEDBAT, sequence numbers, selective ACKs. DHT is where you learn what it means to talk to thousands of strangers' implementations of the same spec, all behaving slightly differently.
Deterministic concurrency simulation
Concurrent tokio code is non-deterministic by default: the scheduler picks tasks however it wants, so a test passes ninety-nine times and fails on the build server. In tests I swap tokio for a deterministic runtime with the same APIs, where every poll is scheduled by a seeded PRNG. A failing test prints its seed and you replay it byte-for-byte to watch the bug again. FoundationDB and TigerBeetle use this approach; getting a working version running here changed how I think about testing async systems.
Backpressure and the cost of zero-copy
Bytes come off a TCP socket, get verified against a piece hash, and have to land at the right offset of the right file. Naively that is three or four allocations per chunk. With io_uring and refcounted buffer slices you can do it in one, but zero-copy only holds if every actor in the pipeline honours it, and any buffering decision becomes a backpressure decision. When the disk is slow the network task has to feel it, by yielding upstream until the channel drains rather than dropping bytes. Getting this right is what keeps memory flat instead of climbing.
Pluggable backends, real abstractions
Storage is a trait. Network transport is a trait. The piece picker is a trait. That sounds like over-engineering until the first integration test, when you need to swap the disk for an in-memory implementation and the network for a deterministic one. The abstractions exist because testing demanded them, not because I planned for hypothetical future backends.
The workspace as compilation boundaries
The engine is split into many crates, and each crate is a compilation boundary, an API surface, and a rebuild horizon. When peer-protocol changes, disk-io does not rebuild. When the UI changes, the engine does not rebuild. That matters when a full build takes minutes, but a change to one crate rebuilds in seconds. It also forces the layering question on you: what does this crate depend on, and what is it allowed to know about. You enforce those boundaries in the dependency graph instead of in your head.
Concurrent-session systems
A torrent engine, an API gateway, and a message broker look nothing alike from the outside. Underneath, all three are many concurrent sessions handling untrusted input and streaming bytes through a pipeline with backpressure and per-actor observability. Building one teaches you how to build the others.
Why Rust for this
For a one-person project with this much concurrency, I needed the compiler to catch what I could not hold in my head. Refactoring a hot path and having the borrow checker tell me immediately what broke is the difference between shipping and stalling.
How the learning happened
The first month was bencode and the wire format and not understanding why my parser kept allocating. The second was the actor architecture and learning that Send/Sync bounds are not theoretical. The third was DHT and discovering that half the peers on the network do not implement the spec correctly and you handle their bugs anyway. The fourth was the deterministic runtime, and realising the bug I had chased for two weeks was a race that only appeared under one scheduler order.
See the rest of the work.