Framer¶

ww_framer packs variable-length messages into fixed-size frames and unpacks them back on the other side. It is no_std, allocation-free and is meant to sit between the serialized bytes produced by ShrinkWrap and a physical link (USB, CAN, UART, UDP, etc.).

A frame is whatever the underlying medium can carry as one unit: a 512B USB bulk packet, a 8B CAN frame, a DMA buffer for UART, and so on. A message is what the application cares about: a request, a reply, a stream item. The two rarely have the same size, which is exactly the problem this crate solves:

• several small messages are packed into one frame, so that the link is not wasted on padding;
• a large message is split across as many frames as needed and re-assembled on the receiver;
• a frame does not have to be full before it is sent — flush early to trade bandwidth for latency.

Model¶

 application               ww_framer                   medium
 ──────────────            ──────────────────          ─────────────────
 message A ─┐                                          ┌─ frame 1 ─┐
 message B ─┼── Tx::write ─► [ hA A hB B hC C… ] ── ►  │ hA A hB B │
 message C ─┘                                          └───────────┘
                                                       ┌─ frame 2 ─┐
                                                       │ hC C  …   │
                                                       └───────────┘

 medium                    ww_framer                   application
 frame 1 ──► FramedRx::stage ─► reassemble ─► message() ──► A, B
 frame 2 ──► FramedRx::stage ─► reassemble ─► message() ──► C

Each message inside a frame is preceded by a small head that carries three things:

Optionally a checksum over the message and a tail may follow the payload. Both are traits and default to no-ops (NopChecksum, NopTail), because frame-based media like USB and CAN already check frame integrity. See Checksum and tail for the ready-made CRCs and an example with both enabled.

Once per frame, before the first head, a prefix may be written: a frame counter and a session epoch for media that lose, duplicate or reorder frames (UDP). It is a fourth generic parameter, NopPrefix (nothing) by default, see Frame prefix.

The pieces are generic parameters:

use ww_framer::{Tx, FramedRx};
use ww_framer::framed::U2Head;
use ww_framer::traits::{NopChecksum, NopTail};

type LinkTx<'a> = Tx<'a, U2Head, NopChecksum, NopTail>; // prefix: NopPrefix
type LinkRx<'a> = FramedRx<'a, U2Head, NopChecksum, NopTail>;

Writing and reading¶

Tx::write appends a message to the frame being assembled, flush hands the frame out. Three messages written into a 16-byte frame:

let mut buf = [0u8; 16];
let mut tx = LinkTx::new(&mut buf);

tx.write(0, &[0x01, 0x02, 0x03, 0x04]);        // Ok(true)  — fits
tx.write(1, &[0x11, 0x12, 0x13, 0x14, 0x15]);  // Ok(true)  — fits
tx.write(2, &[0x21, 0x22]);                    // Ok(true)  — fits

let len = tx.flush();                          // 14
let frame = &tx.buf()[..len];

write returns Ok(true) when the whole message has been consumed, so a producer loop can simply keep writing until it gets Ok(false) (frame full, flush it) or runs out of messages (flush what is there).

The receiver yields them one at a time:

let mut rx_buf = [0u8; 64];
let mut rx = LinkRx::new(&mut rx_buf);

rx.stage(frame)?;
loop {
    rx.reassemble();
    let Some((user_kind, message)) = rx.message() else { break };
    // (0, &[01 02 03 04]) then (1, &[11 12 13 14 15]) then (2, &[21 22])
}

Full messages are returned in place from the staged frame — no copy is made.

A message that does not fit into what is left of the frame is split: write fills the frame, returns Ok(false), and is called again with the same message after the frame is sent. A 20-byte message over 8-byte frames:

let mut buf = [0u8; 8];
let mut tx = LinkTx::new(&mut buf);
let msg: [u8; 20] = core::array::from_fn(|i| i as u8 + 1);

loop {
    let done = tx.write(0, &msg).unwrap();
    let len = tx.flush();
    send(&tx.buf()[..len]);
    if done { break; }
}

On the receiving side stage + reassemble must be called per frame, because frame boundaries carry meaning; the message is delivered once its last piece has arrived. How the pieces look in each frame, and what the receiver does with lost or reordered frames, is on framer wire format.

Sending a partial frame early (latency)¶

Tx never sends anything by itself: it only fills the assembly buffer. flush() returns how many bytes are currently in it and rewinds to the start. That means the caller decides when a frame goes out:

• Throughput-oriented: keep calling write until it returns Ok(false), then flush. Frames are as full as possible, per-frame overhead is minimal.
• Latency-oriented: flush as soon as there is nothing more to send right now, even if the frame is mostly empty. The message is sent out immediately instead of waiting for enough traffic to fill the frame.

let mut buf = [0u8; 64];                      // USB full-speed bulk packet
let mut tx = LinkTx::new(&mut buf);

tx.write(0, &[0xAA, 0xBB, 0xCC, 0xDD]);       // Ok(true)
let len = tx.flush();                         // 5, not 64
usb.write(&tx.buf()[..len]).await;            // short packet goes out now

 assembly buffer (64 B)
 ┌────┬────┬────┬────┬────┬─────────────────────────────────────── ─ ─ ─ ┐
 │ 04 │ AA │ BB │ CC │ DD │             59 bytes unused                  │
 └────┴────┴────┴────┴────┴─────────────────────────────────────── ─ ─ ─ ┘
  ◄──── sent: 5 bytes ────►◄───────────────── not sent ──────────────────►

A typical embedded loop is a hybrid: drain the outgoing queue into the frame, and flush when the queue is empty or the frame is full — whichever comes first. This gives full frames under load and minimal latency when idle.

loop {
    let mut pending: Option<(u8, &[u8])> = queue.peek();
    while let Some((uk, msg)) = pending {
        match tx.write(uk, msg) {
            Ok(true)  => { queue.pop(); pending = queue.peek(); } // fully written, next
            Ok(false) => break,                                   // frame full, ship it
            Err(())   => { queue.pop(); pending = queue.peek(); } // too big, drop
        }
    }
    let len = tx.flush();
    if len > 0 {
        link.send(&tx.buf()[..len]).await;
    } else {
        queue.wait_non_empty().await;
    }
}

Latency and priority¶

The framer only fills frames; when one goes out is decided above it, by the WireWeaver host event loop (wire_weaver_client) and the device server (ww_device), with one rule on both ends:

• Default traffic (requests at the default priority, their replies, stream and property updates): a frame goes out when it is full or when the link's accumulation window is over, whichever comes first. The window is the device's LinkConfig::accumulation_time (sent to the host in DeviceInfo): 1 ms with UsbTimings::fs_higher_speed(), 250 µs on USB HS. A request or reply alone waits for the window too: it is what lets requests made close together share one packet, and it keeps packetization the same from run to run.
• Never later than the window: the host waits for it on a timer accurate to tens of microseconds (tokio's own counts whole milliseconds and made a 1 ms window cost about 2), so a window of W costs at most W (CLT-53).
• Urgent requests go out at once. Every request carries a 4-bit priority (wire_weaver::Priority, 0..=15, 4 by default); 8 and up (Priority::URGENT) is urgent: the host flushes the frame as soon as the request is in it, taking what was already waiting along, and the device flushes its reply the same way, also a deferred reply sent later through Server::sink() (SRV-42).

let v = client.read_temperature().urgent().read().await?;           // sent and answered at once
let r = client.i2c_read(0x48, 2).priority(Priority::HIGHEST).call().await?;
let client = Board::config(|c| c.default_priority(Priority::URGENT)).connect().await?; // every request

A multi request goes at the highest priority of its items. Through the hub the priority stays as the client sent it, and the hub's link to the device flushes an urgent request at once as well.

Handlers see the priority as cx.priority(), and a deferred method's cx.reply_to() keeps it, so <method>_send_return(out, reply_to, ..) answers urgent calls at once without anything else to do. Firmware that wants to send what waits right now calls server.sink().sink.flush() (ww_device Sink::flush).

The level inside each half orders requests against each other; serving a queue by priority (the host's tx queue, the device's, the hub's across clients) is planned (CLT-52, SRV-44, SRV-45).

Checksum and tail¶

After the payload of a Full or End message, the framer writes the Checksum and then the Tail, both byte-aligned. Start and Continue carry neither — the checksum covers the whole re-assembled message and is verified once, when the End arrives.

 ┌──────┬─────────────────┬──────────┬──────┐
 │ head │     payload     │ checksum │ tail │      Full / End
 └──────┴─────────────────┴──────────┴──────┘
 ┌──────┬───────────────────────────────────┐
 │ head │  payload … till the end of frame  │      Start / Continue
 └──────┴───────────────────────────────────┘

Both must be fixed length: read has to consume exactly LEN_BYTES_* bytes. The receiver uses these constants to decide upfront whether a whole message is present in a frame, and Tx uses them to decide whether an End fits.

The checksum length is configured separately for Full and split messages. A medium that checks every frame and keeps them in order within a session (USB) needs none at all; one that loses frames but checks them (CAN) can take a small one on split messages only, as a lost frame in between would not be detected by the medium alone; one whose frames may arrive corrupted (UDP: its 16-bit checksum is weak and optional over IPv4) takes a CRC on every message. Byte-level examples with a checksum and a tail are on framer wire format.

A checksum covers more than the payload: Checksum::write / read get a header too, the frame's prefix followed by the head of the piece the checksum belongs to (the Full, or the End of a split message), so that a flipped bit in a user kind, a length or the prefix fails it as well.

Ready-made CRCs: ww_framer::crc¶

Instead of hand-rolling a checksum, use CrcChecksum<A, FULL>. A selects both the width (u8 / u16 / u32 / u64, written little endian) and the algorithm; the provided ones use the crc crate with a lookup table computed at compile time. FULL (default false) controls whether Full messages are checksummed too:

use ww_framer::crc::{CrcChecksum, Crc16Usb, Crc32IsoHdlc};

// 2-byte CRC-16/USB on split messages only — e.g. for CAN, which may lose a frame between pieces
type LinkTx<'a> = Tx<'a, U2Head, CrcChecksum<Crc16Usb>, NopTail>;

// 4-byte CRC-32 on every message — for media without its own frame CRC
type StreamTx<'a> = Tx<'a, U2Head, CrcChecksum<Crc32IsoHdlc, true>, EndMarker>;

CrcAlgorithm::checksum(header, message) is a plain function, so any other algorithm from the crc catalog is a few lines, and the CRC can also come from elsewhere: a hardware CRC unit, or code shared with the rest of the firmware (ww_link's UDP configuration takes its CRC-32 from ww_block::crc32_update, whose table size the firmware picks once for both):

use ww_framer::crc::{CrcAlgorithm, crc_of};

struct Crc8Autosar;
impl CrcAlgorithm for Crc8Autosar {
    type Width = u8;
    fn checksum(header: &[u8], message: &[u8]) -> u8 {
        const CRC: crc::Crc<u8> = crc::Crc::<u8>::new(&crc::CRC_8_AUTOSAR);
        crc_of(&CRC, header, message)
    }
}
type LinkChecksum = CrcChecksum<Crc8Autosar>;

Hand-rolled example¶

The traits are small enough to implement directly. CRC-8 (poly 0x07) for split messages only, and a 0x7E end marker after every message:

use shrink_wrap::{BufReader, BufWriter};
use ww_framer::traits::{Checksum, RdError, Tail, WrError};

struct Crc8;

impl Checksum for Crc8 {
    const LEN_BYTES_FULL: usize = 0;   // frame CRC is enough for Full messages
    const LEN_BYTES_SPLIT: usize = 1;

    fn write(header: &[u8], message: &[u8], is_split: bool, wr: &mut BufWriter<'_>) -> Result<(), WrError> {
        if is_split {
            wr.write_u8(crc8(header, message))?;
        }
        Ok(())
    }

    fn read(header: &[u8], message: &[u8], is_split: bool, rd: &mut BufReader<'_>) -> Result<(), RdError> {
        if is_split && rd.read_u8()? != crc8(header, message) {
            return Err(RdError::ChecksumMismatch);
        }
        Ok(())
    }
}

struct EndMarker;

impl Tail for EndMarker {
    const LEN_BYTES: usize = 1;

    fn write(wr: &mut BufWriter<'_>) -> Result<(), WrError> {
        wr.write_u8(0x7E)?;
        Ok(())
    }

    fn read(rd: &mut BufReader<'_>) -> Result<(), RdError> {
        if rd.read_u8()? == 0x7E { Ok(()) } else { Err(RdError::BadTail) }
    }
}

type LinkTx<'a> = Tx<'a, U2Head, Crc8, EndMarker>;
type LinkRx<'a> = FramedRx<'a, U2Head, Crc8, EndMarker>;

message passed to Checksum::write / read is always the complete message, even for End, so the CRC does not need to be incremental; crc8(header, message) is the CRC-8 of header followed by message.

Frame prefix¶

A frame prefix is written by Tx once at the start of every frame and read by FramedRx once, before the first head; heads, checksums and tails stay as they are (a checksum covers the prefix, see above). Tx keeps a u32 frame counter (from 0, up by one per non-empty flush()) and a session epoch (up by one per new_session()) and hands them to the prefix, which decides what to write. The trait is ww_framer::prefix::FramePrefix:

use ww_framer::prefix::{FrameReport, SeqPrefix};

type UdpTx<'a> = Tx<'a, U2Head, CrcChecksum<Crc32IsoHdlc, true>, NopTail, SeqPrefix>;
type UdpRx<'a> = FramedRx<'a, U2Head, CrcChecksum<Crc32IsoHdlc, true>, NopTail, SeqPrefix>;

match rx.stage(frame)? {
    FrameReport::Gap { lost } => warn!("{lost} frames lost"),
    report if report.is_dropped() => return Ok(()), // duplicate, late, stale, too short: not staged
    _ => {}
}

stage (and commit_staged) check the prefix and say what they found in a FrameReport. SeqPrefix's receiver rebuilds the full counter from its low bits, as the nearest value to the expected one:

So with a prefix, messages never come out twice or out of order, and a split message never gets a piece from another session. prefix_stats() counts each case (PrefixStats), resync() takes the next frame as it is (a new peer), and Tx::new_session() drops a half-written message and starts the next frame in a new epoch, so that the receiver drops whatever of the old session is still on the way. Process every frame (reassemble until message() is None) before staging the next one, as with split messages. The bytes are on framer wire format.

Limits of 4 + 2 bits: losing 8 or more frames in a row costs two more (the receiver needs two frames to see it), or up to 8 more when the frames before were lost too; a frame delayed across two session changes is taken as new; a corrupted prefix can move the receiver to a wrong position before the CRC rejects its frame, after which order is not guaranteed for a few frames (integrity still is: every message passes its CRC).

API summary¶

Tx<H, C, T, P = NopPrefix>¶

FramedRx<H, C, T, P = NopPrefix>¶

Intended receive loop:

rx.stage(frame)?;
loop {
    rx.reassemble();
    let Some((user_kind, message)) = rx.message() else { break };
    handle(user_kind, message);
}

Both also have reset() (drop whatever is half-written or half-assembled, e.g. on a reconnect) and into_parts() / from_parts() to keep the state apart from the buffer. With the std feature, TxOwned and FramedRxOwned own a Vec buffer and have the same methods (TxOwned::new(frame_size), FramedRxOwned::new(assembly_size)); the host transports use them.

Traits¶

Everything above is generic over these traits (in ww_framer::traits and ww_framer::prefix), so a different medium can plug in its own encoding without touching the packing/splitting logic:

See Checksum and tail for an example implementation of the last two.

Cargo features¶

Without either feature, the maximum message size is 1023 bytes and the head is at most 2 bytes (3 with an extended user kind), which is what most microcontroller links need.

Testing and Fuzzing¶

The framer carries every message over USB, RTT and UDP (WebSocket has framing of its own), so it has to be right. Unit tests cover the head forms, splitting, the prefix cases and the receiver failure modes listed on the wire format page. Two fuzz targets (from fuzz/): framer-tx-rx round-trips random messages over random frame sizes (just fuzz-framer), framer-lossy runs SeqPrefix with a CRC-32 on every message over a channel that loses, duplicates, delays and corrupts frames, with new sessions on the sender: the receiver must deliver only sent messages, and in order unless a frame was corrupted (just fuzz-framer-lossy).