diff --git a/public/blog/update-00/bbb.png b/public/blog/update-00/bbb.png new file mode 100644 index 0000000..1a550c8 Binary files /dev/null and b/public/blog/update-00/bbb.png differ diff --git a/public/blog/update-00/globe.png b/public/blog/update-00/globe.png new file mode 100644 index 0000000..54c2d4f Binary files /dev/null and b/public/blog/update-00/globe.png differ diff --git a/src/pages/blog/update-00.mdx b/src/pages/blog/update-00.mdx new file mode 100644 index 0000000..8d63bc2 --- /dev/null +++ b/src/pages/blog/update-00.mdx @@ -0,0 +1,242 @@ +--- +layout: "@/layouts/global.astro" +title: Update 00 +author: kixelated +description: Update 00 notes +cover: "/blog/update-00/bbb.png" +date: 2026-06-25 +--- + +# Update 00 + +sup nerds. +We now doing low-energy progress reports. + +Here's a summary of stuff that changed in the last 4? months. +IDK my brain is so fried from talking to Claude all day. + + +## Hop-based Routing +Now you can [connect relays to each other](https://doc.moq.dev/bin/relay/) however you desire. +They'll gossip available broadcasts to each other and proxy any subscriptions automatically via the shortest path. + +For example, I can host a relay in Denver and configure it with: +```bash +moq-relay --cluster-connect chicago.moq.dev --cluster-connect seattle.moq.dev ... +``` + +Imagine relays are also configured to connect to their nearest neighbors. +A subscription from a user in Europe, subscribing to a broadcast in Denver, might travel:\ +`Frankfurt -> London -> New York -> Chicago -> Denver` + +At each hop, we deduplicate subscriptions. +Which means at most only one copy of a track will ever be transmitted between two relays. + +
+ ![bbb](/blog/update-00/globe.png) +
**SPOILER**: an upcoming marketing image
+
+ +For context, the previous architecture was a mesh. +Every relay would connect to every other relay, which would have resulted in:\ +`Frankfurt -> Denver` (oof) + +Oh yeah and +Your local relay (snug behind a firewall) can proxy all broadcasts to/from a public CDN: +```bash +moq-relay --cluster-connect https://cdn.moq.dev ... +``` + +It'll work with any MoQ CDN that implements enough of the [`moq-transport`](https://datatracker.ietf.org/doc/draft-ietf-moq-transport/) spec (*cough* not Cloudflare yet *cough*). +But you'll need `moq-lite-03+` to establish multiple connections, otherwise a cycle will ruin your day. + +## Standards +Speaking of standards, the IETF keeps marching forward, so I have to "support" every new draft version lul. + +**Supported:** +- [moq-transport](https://datatracker.ietf.org/doc/draft-ietf-moq-transport/) drafts 14-18. +- [qmux](https://datatracker.ietf.org/doc/draft-nandakumar-moq-qmux-moqt/) drafts 00-01 +- [msf](https://datatracker.ietf.org/doc/draft-ietf-moq-msf/) drafts 00-01 +- [loc](https://datatracker.ietf.org/doc/draft-ietf-moq-loc/) draft 01 + +It takes forever to litigate every single addition/change in the IETF. +I'm trying my best [to publish extensions](https://www.ietf.org/archive/id/draft-lcurley-moq-probe-00.html), but it's such a drain on my time. + +So instead the moq.dev libraries **prefer to negotiate:** +- moq-lite drafts 01-04 +- hang drafts 00-01 + +moq-lite-05 is coming SOON with more stuff. + + +## Muxing +My philosophy is that we should **not** transmit arbitrary media formats over MoQ. +It's possible to [transmit TS over MoQ](https://www.ietf.org/archive/id/draft-gregoire-moq-msfts-00.html) verbatim, but then every player/subscriber needs to decode Transport Streams (gross). + +Instead, [`moq-mux`](https://doc.moq.dev/lib/rs/crate/moq-mux.html) is our transmuxing layer. +We can import/export other formats without re-encoding, preserving the intent but not the original structure. + +[Here's the codecs we support](https://github.com/moq-dev/moq/tree/main/rs/moq-mux/src/codec): +- H.264/H.265 +- AV1 +- VP8/VP9 +- Opus +- AAC + +[And the containers we support](https://github.com/moq-dev/moq/tree/main/rs/moq-mux/src/container): +- fMP4/CMAF +- FLV/RTMP +- Matroska/WebM +- HLS +- LOC +- TS + +Going back to Transport Streams (gross) as an example. +When you import a TS stream with `moq-mux`, we'll split A/V into MoQ tracks, but we'll also preserve "unknown" tracks with a special "ts" section. +If you export the same stream to TS with `moq-mux`, we'll re-assemble the TS stream and reinsert the unknown tracks. + +This is a **REALLY BIG DEAL** for broadcast TV. +Your precious SCTE35 markers are preserved. +ur welcome + +## CLI +Note that `moq-mux` is available as both a library and a CLI via [`moq-cli`](https://doc.moq.dev/bin/cli.html). + +```bash +# Publish: pipe in a MPEG-TS file +ffmpeg -re -i input.ts -f mpegts - | \ + moq-cli publish --url https://cdn.moq.dev/anon --broadcast my-stream ts + +# Subscribe: pull MPEG-TS back out and play it +moq-cli subscribe --url https://cdn.moq.dev/anon --broadcast my-stream --format ts | \ + ffplay - +``` + +You can download all of the moq binaries (`moq-relay`, `moq-token`, etc) from your favorite package manager: +- Homebrew +- .deb/.rpm +- Docker +- Nix +- ??? + +Cool I guess? + + +## JSON +JSON isn't known for its efficiency. + +[moq-json](https://github.com/moq-dev/moq/tree/main/rs/moq-json) (Rust) and [`@moq/json`](https://github.com/moq-dev/moq/tree/main/js/json) (Typescript) publish arbitrary JSON over MoQ tracks. +The first frame in each group is a full snapshot, then subsequent frames are [JSON Merge Patch](https://datatracker.ietf.org/doc/html/rfc7396) deltas. +We periodically make new groups so late joiners can catch up without having to download the entire stream. + +We also stack [moq-flate](https://github.com/moq-dev/moq/tree/main/rs/moq-flate) (Rust) and [`@moq/flate`](https://github.com/moq-dev/moq/tree/main/js/flate) (Typescript) to add DEFLATE compression on top. +Normally you would compress each frame individually, like a gzip file, but it turns out we can do significantly better by using `Z_SYNC_FLUSH`. +Each subsequent frame *within the same group* can reuse the previous frames as a compression window, instead of starting from scratch. + +Claude ran a benchmark over a synthetic stream of JSON sensor snapshots. +The type of output you'd expect from a drone. + +| Configuration | Bitrate saved | +| ---------------------------------------- | ------------- | +| DEFLATE per frame | 39% | +| JSON Merge Patch | 58% | +| JSON Merge Patch + DEFLATE per frame | 72% | +| DEFLATE per group | 89% | +| **JSON Merge Patch + DEFLATE per group** | **91%** | + +I was surprised to see how important `Z_SYNC_FLUSH` is given it's not supported by the [browser implementation](https://developer.mozilla.org/en-US/docs/Web/API/Compression_Streams_API) and [is bugged](https://github.com/101arrowz/fflate/pull/286) in the most popular Javascript implementation. +Really the main benefit of JSON Merge Patch is to save some CPU cycles. + +## Backends +We've used Quinn for the longest time, but now you can use other QUIC backends! +Why? IDK different features and performance. + +QUIC Backends (WebTransport + raw QUIC): +- quinn +- noq +- quiche (Cloudflare) +- iroh (p2p) + +You can also use non-QUIC backends with Qmux! +It emulates a QUIC connection, but over a reliable transport. +We do suffer more from head-of-line blocking but sometimes it be like that. + +Non-QUIC Backends (Qmux): +- WebSocket +- TLS +- TCP +- Unix domain sockets + +For example, I'm using unix domain sockets (or TCP) to run localhost workers without paying the TLS/UDP overhead tax. +It's still Media over QUIC even if QUIC is not used... + + +## UI Polish +Claude made the web UI look better. +tfw an AI has better taste than me. + +[Clone the repo](https://doc.moq.dev/setup/) and run `just dev` to see: + +
+ ![bbb](/blog/update-00/bbb.png) +
he a sniffer
+
+ +You can use the UI on your own site via the [web components](https://doc.moq.dev/bin/web.html): +```html + + + + + + + +``` + +There's also one for `` which is equally slick. + +## Polyglot (Alpha) +We full [monorepo](https://github.com/moq-dev/moq) now. + +`moq-ffi` uses Uniffi to generate async bindings for various languages. +It's all the same Rust code under the hood, but IDK now you can use it on iOS/Android/whatever. + +[MoQKit](https://fishjam.swmansion.com/blog/moq-boy-escaped-the-browser) uses it. +neat + +Supported: +- [Swift](https://doc.moq.dev/lib/swift/) +- [Kotlin](https://doc.moq.dev/lib/kt/) +- [Python](https://doc.moq.dev/lib/py/) +- [Go](https://doc.moq.dev/lib/go/) (wip) + +I'm still working on more ergonomic bindings for each language, hence the (Alpha) tag. + +Oh and don't worry, we still have [C support](https://doc.moq.dev/lib/c/) via [`libmoq`](https://doc.moq.dev/lib/rs/crate/libmoq.html). +`moq-ffi` is for the async languages who don't want to deal with callbacks and APIs designed for dinosaurs. + + +## Production Stuff +I'm almost done spinning up a PAID MoQ CDN. +It's like a week or two away, **GET HYPED**. + +As part of that, I actually have to make sure everything is production-ready, which means focusing on the boring stuff like: +- `mTLS`: relays can authorize via client certificates +- `TLS`: cert reloading and custom root CAs + +And more of the static configuration is now possible via a REST API. +- `--auth-api`: authorize a user +- `--cluster-connect-api`: discover neighboring relay nodes + +And finally my favorite one, but it deserves its own blog post later: +- `.stats`: deliver MoQ metrics over MoQ + +## Conclusion +I hope you enjoyed this low-energy update. +Super low. + +Written by [@kixelated](https://github.com/kixelated). +![@kixelated](/blog/avatar.png)