Showcase¶

A tour of shrink_wrap tricks, from plain bit-packing to self-referential types, illustrated with real code from wire_weaver itself and from the ww_stdlib crates (ww_version, ww_date_time, ww_client_server, ww_self). Every byte sequence shown below was produced by actually running the snippet - none of it is hand-waved.

Read ShrinkWrap, derive and size assumptions first if you haven't; this page assumes you know what Unsized/final_structure/self_describing/sized mean and how #[derive_shrink_wrap(..)] decides between borrowed and owned types.

Sub-byte packing¶

bool is 1-bit aligned, nib (the Nibble type) is 4-bit aligned, and u1..u64/i2..i64 (U1, U2, ..., I2, ...) are 1-bit aligned regardless of width - they just cost exactly as many bits as you ask for, packed back-to-back with no padding in between:

let mut buf = [0u8; 8];
let mut wr = BufWriter::new(&mut buf);
wr.write_bool(true).unwrap();          // 1 bit
wr.write(&U2::new(3).unwrap()).unwrap(); // 2 bits
wr.write(&U1::new(1).unwrap()).unwrap(); // 1 bit
let bytes = wr.finish().unwrap();
assert_eq!(bytes, &[0b1111_0000]); // 1 + 11 + 1 = 4 bits used, rest padded with zeros

A whole tuple of 8 bools packs into a single byte, no framing at all:

let bytes = (true, false, true, false, true, true, false, false)
    .to_ww_bytes(&mut buf).unwrap();
assert_eq!(bytes, &[0b1010_1100]);

This is what the ElementSize::Sized { size_bits } case (see size assumptions) is built on: a sized struct/enum's fields are just laid out bit after bit, so putting a bool field or a small enum next to another Sized field is free - they share the same byte instead of each getting its own.

Bit-sized fields and variant payloads: a 7- or 10-bit I2C address¶

The same U1..U64 / I2..I64 types work as struct fields, enum variant payloads and #[ww_trait] method arguments, returns and properties. Next to a 1-bit discriminant they share its byte:

#[derive_shrink_wrap(borrowed, derive(Debug, PartialEq, Eq, Copy, Clone), ww_repr = u1, self_describing)]
enum I2cAddress {
    Seven(U7),
    Ten(U10),
}

let seven = I2cAddress::Seven(U7::new(0x50).unwrap());
assert_eq!(seven.to_ww_bytes(&mut buf).unwrap(), &[0x50]);       // 0 1010000: 8 bits
let ten = I2cAddress::Ten(U10::new(0x3ff).unwrap());
assert_eq!(ten.to_ww_bytes(&mut buf).unwrap(), &[0xff, 0xe0]);   // 1 1111111111 00000: 11 bits

With Seven(u8) instead, the byte-aligned u8 pads the discriminant to a byte and the same address takes [0x00, 0x50]. The variants are 7 and 10 bits, so the enum is self_describing, not sized (a sized enum needs variants of one size, see size assumptions). Same-sized payloads can be sized and then nest without any size information, e.g. a 32-bit frame (shrink_wrap/tests/sub_byte.rs):

#[derive_shrink_wrap(borrowed, derive(..), ww_repr = u1, sized)]
enum Offset { Up(U7), Signed(I7) }                                  // 1 + 7 bits

#[derive_shrink_wrap(borrowed, derive(..), ww_repr = u2, sized)]
enum Reg {                                                          // 2 + 14 bits
    Wide { addr: U10, write: bool, len: U3 },
    Narrow { addr: U12, bank: I2 },
    Idle(U14),
}

#[derive_shrink_wrap(borrowed, derive(..), sized)]
struct Frame { kind: U3, offset: Offset, reg: Reg, tail: I5 }       // 3 + 8 + 16 + 5 bits

let frame = Frame {
    kind: U3::new(0b101).unwrap(),
    offset: Offset::Signed(I7::minus_one()),
    reg: Reg::Wide { addr: U10::new(0x2aa).unwrap(), write: true, len: U3::new(5).unwrap() },
    tail: I5::new(-16).unwrap(),
};
// 101 1 1111111 00 1010101010 1 101 10000
assert_eq!(frame.to_ww_bytes(&mut buf).unwrap(), &[0xbf, 0xe5, 0x55, 0xb0]);

U7::new(x) returns None when x does not fit, U7::try_from(x) an Error::SubtypeOutOfRange, and u8::from(addr) / addr.value() give the number back. The dynamic client (ww_self::ValueOwned) reads and writes these as the smallest standard integer (U7 as a u8, I10 as an i16) and rejects values that don't fit.

UNib32: variable-length numbers by the nibble¶

UNib32 encodes a u32 as 1-or-more nibbles: 1 continuation bit + 3 value bits per nibble.

assert_eq!(UNib32(0).to_ww_bytes(&mut buf).unwrap(), &[0x00]);   // 1 nibble
assert_eq!(UNib32(7).to_ww_bytes(&mut buf).unwrap(), &[0x70]);   // 1 nibble, max for that size
assert_eq!(UNib32(8).to_ww_bytes(&mut buf).unwrap(), &[0x81]);   // 2 nibbles: needs a continuation
assert_eq!(UNib32(0o777).to_ww_bytes(&mut buf).unwrap(), &[0xff, 0x70]); // 3 nibbles

Small numbers (the overwhelming majority in practice - lengths, indices, enum discriminants) cost a single nibble. It's self_describing: a reader always knows where it ends from the continuation bits alone, no length prefix needed. This is why ww_version::Version uses UNib32 for major/minor/patch instead of u8/u32 - most real-world version numbers are small, so Version::new(0, 1, 2) serializes to just 2 bytes:

let version = Version::new(0, 1, 2);
assert_eq!(version.to_ww_bytes(&mut buf).unwrap(), hex!("01 20"));

UNib32 is also what a struct/enum's own reverse-length marker is built from when it's Unsized - see the FIFO-of-lengths section below.

For numbers that are usually large, UVlq32 is the byte-based alternative: standard big endian VLQ, 1 continuation bit + 7 value bits per byte, most significant group first, byte-aligned. It is also self_describing and takes 1 to 5 bytes instead of 1 to 11 nibbles:

assert_eq!(UVlq32(0).to_ww_bytes(&mut buf).unwrap(), &[0x00]);         // 1 byte
assert_eq!(UVlq32(127).to_ww_bytes(&mut buf).unwrap(), &[0x7f]);       // 1 byte, max for that size
assert_eq!(UVlq32(128).to_ww_bytes(&mut buf).unwrap(), &[0x81, 0x00]); // 2 bytes
assert_eq!(UVlq32(u32::MAX).to_ww_bytes(&mut buf).unwrap(), &[0x8f, 0xff, 0xff, 0xff, 0x7f]); // 5 bytes

UVlq64 is the same encoding widened to u64, for values that can outgrow 32 bits (byte counters, unix milliseconds): still self_describing, 1 to 10 bytes (10 groups of 7 bits cover 70 bits, with room to spare for u64::MAX):

assert_eq!(UVlq64(0).to_ww_bytes(&mut buf).unwrap(), &[0x00]);         // 1 byte
assert_eq!(UVlq64(127).to_ww_bytes(&mut buf).unwrap(), &[0x7f]);       // 1 byte, max for that size
assert_eq!(UVlq64(128).to_ww_bytes(&mut buf).unwrap(), &[0x81, 0x00]); // 2 bytes
assert_eq!(UVlq64(u32::MAX as u64).to_ww_bytes(&mut buf).unwrap(), &[0x8f, 0xff, 0xff, 0xff, 0x7f]); // 5 bytes, same as UVlq32
assert_eq!(UVlq64(u64::MAX).to_ww_bytes(&mut buf).unwrap(), &[0x81, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x7f]); // 10 bytes

There is no UVlq64Backfill; UVlq32Backfill is the same number, but always written as the full 5 bytes, padded with empty 0x80 groups in front. That reserves room for a value that is only known after the message is serialized, like a request sequence number assigned right before sending. Once it is known, UVlq32Backfill::backfill() writes it right-justified into those 5 bytes and returns the slice starting at its shortest encoding, so it costs no more than a plain UVlq32:

#[derive_shrink_wrap(borrowed)]
struct Message<'i> {
    seq: UVlq32Backfill, // must be the first field, so that bytes in front of it can be dropped
    payload: &'i str,
}

let len = Message { seq: UVlq32Backfill(0), payload: "ab" }.to_ww_bytes(&mut buf).unwrap().len();
// buf[..len] is 80 80 80 80 00 61 62 02
let bytes = UVlq32Backfill::backfill(&mut buf[..len], 300).unwrap();
assert_eq!(bytes, hex!("82 2c 61 62 02"));

The skipped leading bytes are filled with the padded form, so the whole buffer would also read back as 300.

#[derive_shrink_wrap(..)] rejects UVlq32Backfill anywhere other than the first field of a struct: after another field (a bool or an Option flag would be written in front of it), nested in Option, Vec, tuples or arrays, and in enums, whose discriminant always comes first. The macro recognizes the type by name and only sees one type at a time, so it can't check that the struct itself is serialized first and is not, for example, a field of another struct.

Bounding a value from the inside with TailSize¶

A value read from a buffer longer than itself - a header at the start of a file, a record at a fixed offset - has no back for its FIFO of lengths to live at. TailSize<N> (see the wire format page) is a field the writer fills in with the number of bytes from the end of the field to the end of the value, as a UVlq32 right-justified in N bytes, and the reader bounds the rest of the value with:

#[derive_shrink_wrap(borrowed, derive(Debug, PartialEq))]
struct Header<'i> {
    magic: u16,
    size: TailSize<2>, // ignored on write, filled in; up to 16 KB with 2 bytes
    name: &'i str,
    count: u32,
}

let header = Header { magic: 0xABCD, size: TailSize(0), name: "ab", count: 7 };
let mut region = [0xFFu8; 64]; // the file region: longer than the value, garbage after it
let len = header.to_ww_bytes(&mut region).unwrap().len();
assert_eq!(&region[..len], hex!("CD AB 80 07 61 62 07 00 00 00 02"));
//                                 ^^^^^ 7 bytes follow the slot: "ab", count, "ab"'s reversed length

// the whole region: the reader stops where the slot says, the 0xFF bytes are never looked at
let back = Header::from_ww_bytes(&region).unwrap();
assert_eq!(back, Header { size: TailSize(7), ..header });
// one byte short: an error, not garbage
assert!(Header::from_ww_bytes(&region[..len - 1]).is_err());

A field added after count later is skipped by this reader the same way, since it only ever reads the 7 bytes. Nested values each carry their own slot; the sizes compose the way the FIFO does:

#[derive_shrink_wrap(borrowed, derive(Debug, PartialEq))]
struct Inner<'i> { a: u8, size: TailSize<1>, s: &'i str }

#[derive_shrink_wrap(borrowed, derive(Debug, PartialEq))]
struct Outer<'i> { size: TailSize, inner: Inner<'i>, after: u8 }

let outer = Outer { size: TailSize(0), inner: Inner { a: 1, size: TailSize(0), s: "xyz" }, after: 9 };
assert_eq!(outer.to_ww_bytes(&mut buf).unwrap(), hex!("80 80 80 80 08 01 04 78 79 7A 03 09 06"));
//  outer slot, 5 bytes, 8 follow ^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^ inner: a, slot (4 follow), "xyz", its length
//                                         after ^^ ^^ inner's size (6) in the outer's FIFO, inside the outer's 8

This is what makes Unsized records at fixed file offsets work: serialize each into a buffer of the slot's capacity (a record that outgrows it fails to serialize instead of overflowing), read each with from_ww_bytes on its region, and evolve the record type with #[default] fields like any other. The fields before the slot must be Sized or SelfDescribing, and there is one slot per struct or enum variant, in Unsized types only; the macro checks all of that at compile time. A reader that doesn't care about the value reads the fields up to the slot and split()s past it.

Option<T> and Result<T, E>: a flag, not a tag¶

Both are SelfDescribing: one bool flag followed by the payload (for Result, true picks Ok, false picks Err - both variants exist on the wire, unlike a length-prefixed encoding that would need to know the size of whichever branch to skip):

impl<T: SerializeShrinkWrap> SerializeShrinkWrap for Option<T> {
    const ELEMENT_SIZE: ElementSize = ElementSize::SelfDescribing;
    fn ser_shrink_wrap(&self, wr: &mut BufWriter) -> Result<(), Error> {
        match self {
            Some(val) => { wr.write_bool(true)?; wr.write(val) }
            None => wr.write_bool(false),
        }
    }
}

No size marker is stored for the Option/Result itself - the flag bit is the delimiter. That's why stacking several Options in a struct costs almost nothing extra: each is just one more bit plus its payload's bits, all packed contiguously:

// from docs/types.md - v1 struct, then evolved with an inserted Option<U6> field
wr.write_bool(true)?;                        // 1 bit
wr.write(&Some(U6::new(5).unwrap()))?;       // 1 (flag) + 6 (value) bits
wr.write_u8(0xAA)?;                          // byte-aligned, starts a fresh byte
// -> [0xC5, 0xAA], same total size as before the field was added

Picking an enum's discriminant width - and when Option<bool> beats it¶

ww_repr = <width> sets exactly how many bits an enum's discriminant costs (see derive reference). Pick the width for the variant count you'll ever need - a 3- or 4-variant enum needs ww_repr = u2:

#[derive_shrink_wrap(borrowed, derive(Debug, PartialEq), ww_repr = u2, sized)]
enum Speed3 { Slow, Medium, Fast }

assert_eq!(Speed3::Slow.to_ww_bytes(&mut buf).unwrap(), &[0b00_000000]);
assert_eq!(Speed3::Fast.to_ww_bytes(&mut buf).unwrap(), &[0b10_000000]);

A sized enum like this always costs exactly its declared width, no matter which variant. But if your "enum" is really just "a common default state, plus one bit of extra information when it isn't the default", Option<bool> can be smaller on average, because it's SelfDescribing rather than fixed-width:

assert_eq!(None::<bool>.to_ww_bytes(&mut buf).unwrap(),       &[0b0_0000000]); // 1 bit used
assert_eq!(Some(false).to_ww_bytes(&mut buf).unwrap(),        &[0b10_000000]); // 2 bits used
assert_eq!(Some(true).to_ww_bytes(&mut buf).unwrap(),         &[0b11_000000]); // 2 bits used

Same worst case (2 bits, tying the u2 enum), but the common/default case (None) costs only 1 bit instead of 2. ww_date_time::Timezone uses exactly this shape for the same reason - UTC is the overwhelmingly common case:

#[derive_shrink_wrap(borrowed, derive(Copy, Clone, Debug, ..), self_describing, ww_repr = u1)]
pub enum Timezone {
    UTC,              // costs 1 bit
    Other(OtherTimezone), // costs 1 bit + OtherTimezone's own bits
}

Timezone has a fixed size discriminant (ww_repr = u1, so UTC and Other(..) both cost at least 1 bit), and it is self_describing rather than sized, because its two variants are not of one size - but notice it's shaped exactly like Option<OtherTimezone> would be, and was written as an explicit enum instead so that Other's payload (OtherTimezone, itself ww_repr = u3, self_describing) could be added as a real, named variant rather than a bare tuple. Same bit-level trick, better ergonomics for a type with more than two states down the line.

Relocating presence flags with #[flag]¶

By default, an Option<T>/Result<T, E> field gets its presence flag written immediately in front of it. #[flag] lets you park that flag bit somewhere else entirely - most usefully, right next to other small fields so they all land in the same byte instead of the flag re-opening a new bit run right before its Option. ww_version::Version does exactly this, grouping both presence flags together right after the three UNib32 numbers, ahead of the two Option<&str> fields they belong to:

pub struct Version<'i> {
    pub major: UNib32,
    pub minor: UNib32,
    pub patch: UNib32,
    #[flag]
    build: bool,          // presence flag for the `build` field below, relocated here
    pub pre: Option<&'i str>,
    pub build: Option<&'i str>,
}

Flags are tracked in LIFO order - #[flag] fields are pushed in declaration order and popped by the Options that consume them, so #[flag] build (pushed first) must be consumed after pre's auto-generated flag (pushed second, popped first) - the macro checks this ordering at compile time and rejects anything that isn't valid LIFO. The payoff: major/minor/patch/build-flag/pre-flag all land in the same 14 bits, so a version with no pre-release or build metadata (the common case) still fits in 2 bytes, and a fully-populated one packs its two strings back to back with no gap for their flags:

let version = Version::full(0, 1, 2, Some("pre"), Some("build"));
// major/minor/patch nibbles + 2 flag bits, then "pre" bytes, then "build" bytes,
// then (per the FIFO-of-lengths rule below) build's reverse length, then pre's.
assert_eq!(version.to_ww_bytes(&mut buf).unwrap(), hex!("01 2C 707265 6275696C64 5 3"));

The FIFO-of-lengths: why strings can come before their length¶

An Unsized value's length isn't written where the value starts - it's appended to the back of the buffer as a reversed UNib32, so the writer never needs to know a value's size before it has finished writing it (see the high-level overview). Multiple lengths stack up in reverse order, forming a FIFO read from the back:

let bytes = ("abc", "de").to_ww_bytes(&mut buf).unwrap();
assert_eq!(bytes, hex!("61 62 63 64 65 23"));
//                      a  b  c  d  e  ^^ reversed nibble lengths: 2 ("de"), then 3 ("abc")

The last byte, 0x23, holds both strings' lengths as two reversed nibbles - "de" (written second, so its length is closest to where the FIFO starts) then "abc". Reading walks the same FIFO from the back to know exactly where each string ends, without ever touching a byte before it needs to.

Zero-copy vectors, strings and pre-serialized payloads¶

RefVec<'i, T> is the no-alloc, zero-copy stand-in for Vec<T> - it stores a reference into the original buffer (or a native &[T] slice) and only decodes elements as you .iter() over them:

let v = RefVec::new_bytes(&[0xAA, 0xBB, 0xCC]); // RefVec<'_, u8>, zero-copy byte array

When owned(feature = "std") is requested, the exact same field becomes a Vec<T> on the owned side - see the type mapping table.

TailBytes<'i> is a more specialized trick used by ww_client_server: a byte slice that is not length-prefixed at all, because it's guaranteed to be the last thing in an already-Unsized message (the framer/transport already knows the whole message's length, so re-encoding the tail's length would be pure waste):

pub struct Request<'i> {
    pub seq: u16,
    pub path_kind: PathKind<'i>,
    pub kind: RequestKind<'i>, // eventually bottoms out in `Call { args: TailBytes<'i> }`
}

Method arguments and property values are serialized once into their own buffer, then handed to the outer message as TailBytes - one buffer, no copying to reframe it, no length to write since it's already implied.

#[derive_shrink_wrap(..)] rejects TailBytes / TailBytesOwned anywhere other than the last field of a struct or enum variant: before another field, which it would swallow, and nested in Option, Vec, tuples or arrays. As with UVlq32Backfill, the macro only sees one type at a time: a sized or final_structure type ending in TailBytes must itself be last, up to the first Unsized type on the way up, and that is not checked.

Self-referential types with RefBox¶

Rust needs a compile-time-known size for any type; a struct or enum can't directly contain itself. RefBox<'i, T> breaks the cycle the same way Box<T> does on the heap, but for no_std/no-alloc: it stores either a plain &'i T reference or an unparsed BufReader slice, and only actually deserializes T when you call .read(). Serializing a RefBox is nearly free (it forwards straight to T's own serialization, no boxing overhead on the wire), and deserializing one is a plain buffer-position save - reading the boxed value is deferred until you ask for it:

#[derive_shrink_wrap(owned(feature = "std"), derive(Debug, PartialEq))]
struct Linked<'i> {
    a: u8,
    next: Option<RefBox<'i, Linked<'i>>>,
}

let linked = Linked {
    a: 1,
    next: Some(RefBox::new(&Linked { a: 2, next: None })),
};
assert_eq!(linked.to_ww_bytes(&mut buf).unwrap(), hex!("01 80 02 00 02"));
//                                                       a1 fl a2 fl len
// a=1, Option-flag=1, a=2, Option-flag=0 (nested Linked has no `next`), reverse length of the boxed Linked

RefBox composes with Option/Result/enum variants exactly like any other field, which is how you build actual recursive data. ww_self (the introspection/AST crate) uses this to describe types themselves recursively - Type::Vec(RefBox<'i, Type<'i>>) means "a Vec whose element type is this boxed Type", Type::Option(RefBox<..>) likewise for Option<T>, and so on all the way down. Stripped to the essentials, the same idea looks like this:

#[derive_shrink_wrap(borrowed, derive(Debug, PartialEq, Clone), ww_repr = nib)]
enum Ty<'i> {
    U8,
    U32,
    Str,
    Vec(RefBox<'i, Ty<'i>>),      // Vec<T>, T described recursively
    Option(RefBox<'i, Ty<'i>>),   // Option<T>
    Tuple(RefVec<'i, Ty<'i>>),    // (T0, T1, ...) - a real collection this time, no RefBox needed
}

// describes the Rust type `Vec<Option<u8>>`
let inner = Ty::U8;
let opt = Ty::Option(RefBox::new(&inner));
let vec_of_opt = Ty::Vec(RefBox::new(&opt));
let bytes = vec_of_opt.to_ww_bytes(&mut buf).unwrap(); // [0x30, 0x40, 0x00, 0x01, 0x03]

let des = Ty::from_ww_bytes(bytes).unwrap();
// des == Vec(RefBox(..)); nothing past the outer variant is decoded yet
if let Ty::Vec(elem_ty) = &des {
    assert_eq!(elem_ty.read().unwrap(), opt); // .read() decodes lazily, one level at a time
}

Note RefVec<'i, Ty<'i>> needs no RefBox even though Ty is self-referential - a RefVec never holds T inline, only a reference to where its elements live in the buffer, so there's no Rust-level sizing problem to begin with. RefBox is only needed for a single recursive field (Option<T>, a boxed enum payload, etc.), never for a collection of them.

Patching a discriminant after you've already started writing its payload¶

Sometimes you don't know which variant you're building until you've already started writing it - a response builder that reserves space for a result flag or an event kind before it knows whether the call will succeed, for example. discriminants (see the derive reference) generates a fieldless {Name}Discriminants enum for exactly this, and BufWriter::save_state/restore_state let you go back and patch bits you already reserved:

// ww_client_server's EventKindBuilder, trimmed
pub struct EventKindBuilder { discriminant: BufWriterState }

impl EventKindBuilder {
    pub fn new(wr: &mut BufWriter<'_>) -> Result<Self, Error> {
        let discriminant = wr.save_state();   // remember where the discriminant nibble goes
        wr.write_nib_masked(0)?;              // write a placeholder
        Ok(Self { discriminant })
    }

    pub fn finish_with_kind(self, kind: EventKindDiscriminants, wr: &mut BufWriter<'_>) {
        let after = wr.save_state();          // remember where we ended up
        wr.restore_state(self.discriminant);  // rewind to the placeholder
        _ = wr.write_nib_masked(kind.discriminant()); // patch it in
        wr.restore_state(after);              // resume from where we left off
    }
}

Between new() and finish_with_kind(), the caller writes the variant's actual payload with no idea yet which EventKindDiscriminants it'll turn out to be - ErrorBuilder in the same module does the same trick one level deeper, wrapping the whole payload in an UnsizedBuilder so its reverse-length still comes out correct even though the discriminant in front of it was written last. ww_client_server's tests confirm the builder's output is byte-for-byte identical to building the equivalent value up front and calling .to_ww_bytes() on it directly - the builder is purely a streaming-friendly way to produce the exact same bytes.

Compressed sequences of numbers: Series¶

A Series<T> of plain numbers is one column: the item count, a 4-bit header naming the column's codec, the values bit by bit against the previous one, padding to the byte. Five u8 readings, as a Vec<u8> and as a Series<u8>:

let v: Vec<u8> = vec![100, 100, 101, 99, 200];
assert_eq!(v.to_ww_bytes(&mut buf).unwrap(), hex!("64 64 65 63 c8 05"));
assert_eq!(Series::new(&[100u8, 100, 101, 99, 200]).to_ww_bytes(&mut buf).unwrap(), hex!("54 64 40 a0 7e 50"));

The Vec ends in its count nibble 5; the Series starts with it, then the header 0100 (predictor delta, coder prefix: the smallest here). Then 01100100 (the first value, 8 bits), 0 (100 again), 10 0000010 (+1: zigzag 2 in the 7-bit bucket of the delta code), 10 0000011 (-2: zigzag 3), 11 11001010 (+101: zigzag 202 does not fit 7 bits, so the full 8 bits after 11); 29 bits padded to 4 bytes. Here the saving is small; it grows with every repeated or slowly moving value, which costs 1 or 9 bits instead of 8 to 64:

// timestamps every 10 s with a little jitter
let ts = [1_700_000_000u32, 1_700_000_010, 1_700_000_020, 1_700_000_030, 1_700_000_041, 1_700_000_050];
let delta = CodecPolicy::fixed(NumericCodec::new(Predictor::Delta, Coder::Prefix), FloatCodec::Xor);
assert_eq!(Series::with_policy(&ts, delta).to_ww_bytes(&mut buf).unwrap().len(), 11); // 8 + 32 + 5 x 9 bits
assert_eq!(Series::new(&ts).to_ww_bytes(&mut buf).unwrap(), hex!("68 65 53 f1 00 8a 10 28 18"));

The smallest is delta of delta: 6 (the count), 1000 (predictor delta of delta, coder prefix), 65 53 f1 00 is 1_700_000_000, most significant bit first; then 10 0 0010100 (the first difference, 10, as a change from 0), 0, 0 (two more steps of 10), 10 0000010 (+1: a step of 11), 10 0000011 (-2: back to 9), 69 bits in 9 bytes. A regular clock costs one bit per tick (with the constant coder, nothing at all).

let temps = [21.5f32, 21.5, 21.5, 21.75, 21.75, 22.0];
assert_eq!(Series::new(&temps).to_ww_bytes(&mut buf).unwrap(), hex!("61 41 ac 00 00 37 02 d6 3f"));

6, then 0001 (the XOR code); 41 ac 00 00 is 21.5, then 0, 0 (repeats); 21.75 XORs to 0x0002_0000, one significant bit after 14 leading zeros: 11 01110 00000 1 (14, length 1 written as 0, the bit); 0 (repeat); 22.0 XORs with 21.75 to 0x001e_0000, four bits after 11 leading zeros, which do not fit the previous window (14 leading, 17 trailing), so again an explicit 11 01011 00011 1111. 72 bits in 9 bytes. Had the next value XORed to, say, 0x0004_0000, it would fit that window and cost 10 plus 4 bits. In a real block one Series per signal holds a minute or an hour of samples, and the block is bounded by a TailSize slot (see bounding from the inside), so it can sit at any offset of a file.

Compressed columns: Series<T>¶

Series<T> (layout) writes a list column by column instead of item by item. IvSample { v: Volt, i: Ampere }, Volt/Ampere each a ww_si-shaped { prefix: Prefix, value: NumericValue } (series, final_structure), 4 readings with v around 3300 mV and i around 152 000 µA, both in their usual range:

let items: Vec<IvSample> = [3300u16, 3301, 3301, 3299]
    .into_iter()
    .zip([152_000i32, 152_002, 151_999, 152_001])
    .map(|(v, i)| IvSample {
        v: Volt { prefix: Prefix::Milli, value: NumericValue::U16(v) },
        i: Ampere { prefix: Prefix::Micro, value: NumericValue::I32(i) },
    })
    .collect();
assert_eq!(
    Series::new(&items).to_ww_bytes_owned().unwrap(),
    hex!("42 11 14 33 90 51 91 8a 00 04 a3 80 1c b0")
);

14 bytes against 33 for the smallest plain form (Vec<IvSample> as final_structure, no evolution): 4 (n = 4), 2 (IvSample's 2 fields), v's prefix run (Milli, no change, end: 0001 0), v's value tag run (U16, end), v's U16 column (header 01 01: delta + packed, first value 3300 at full width, then a packed block of 3 residuals: +1, 0, -2), then the same shape for i's prefix, tag and I32 column (delta + packed: 152000, then +2, -3, +2). The header and first-value bytes are what batches of 4 pay for; at 100 items the same IvSample shrinks to 104 bytes (7.7x smaller) because they're paid once per batch, not once per item - see sizes against Vec<T>.

The same 4 items, but i jumps from the µA range to the mA range after item 2 (Milli U16 storage once there):

let items: Vec<IvSample> = [
    (3300u16, Prefix::Micro, NumericValue::I32(480)),
    (3300, Prefix::Micro, NumericValue::I32(482)),
    (3301, Prefix::Milli, NumericValue::U16(120)),
    (3300, Prefix::Milli, NumericValue::U16(121)),
]
.into_iter()
.map(|(v, i_prefix, i_value)| IvSample {
    v: Volt { prefix: Prefix::Milli, value: NumericValue::U16(v) },
    i: Ampere { prefix: i_prefix, value: i_value },
})
.collect();
assert_eq!(
    Series::new(&items).to_ww_bytes_owned().unwrap(),
    hex!("42 11 14 33 90 44 94 44 d1 20 5e 2f 22 57 81 e2 00")
);

17 bytes against 29 plain. i's prefix run now has a change (Micro, gap 1, Milli, end) and so does its value tag run (I32, gap 1, U16, end); because the prefix is an enum field declared earlier in Ampere, i's numeric column restarts where it changes: the I32 column holds only 480, 482 (one delta), the U16 column starts fresh at 120 (not a delta of -362 against 482) and holds just 120, 121. Columns never carry a delta across a boundary the reader can't make sense of.

Sparse updates with OptionAnyVec¶

A trait level with 32 f32 properties, of which 3 changed since the last update: slots 3, 17 and 30 with 1.0, 2.5 and -3.0 (12 bytes of values whatever the encoding). The OptionAnyVec header in each mode, and the alternatives:

let mut wr = OptionAnyVecWriter::new(&mut buf, [3u32, 17, 30]).unwrap();
wr.write(3, &1.0f32).unwrap();
wr.write(17, &2.5f32).unwrap();
wr.write(30, &-3.0f32).unwrap();
assert_eq!(
    wr.finish_and_take().unwrap(),
    hex!("33 3d 1c 10 00 00 80 3f 00 00 20 40 00 00 40 c0")
);

The event sizes are without the link's framing, which comes once per message on top, so separate events lose more than shown. One level deeper (paths [2, slot] and [2]) it is 26 against 20 bytes.

With byte-aligned values EitherAnyVec takes as much here, but it pays a flag byte per 8 slots whatever is present, pads every group to a byte and can't be read by a side with another slot count. Where that shows:

• Other sets of 32: slots 0, 1, 2 take 13 bytes (03: all, n = 3), slots 29, 30, 31 take 15 (24 10 70: two-level), a single slot 3 takes 6 (14 10: dense, len 4, bits 0001); EitherAnyVec takes 16, 16 and 8.
• Few of many: slots 3, 70, 120 of 128 take 16 bytes (33 3a 81 96: the index list), dense would be 30, EitherAnyVec 28.
• Bit-sized values: 3 bools at slots 3, 17, 30 take 4 bytes (33 3d 1c 1e: the three 1 bits share the last header byte), EitherAnyVec 7.
• All present: 32 f32s take 130 bytes, 2 of them header (08 40: all, n = 32).

Putting it together: real wire sizes¶

A few real types from ww_stdlib, each combining several of the tricks above:

• ww_version::Version - UNib32 for major/minor/patch, final_structure (flattened into its parent, no reverse-length of its own), a relocated #[flag] for build. Result: 2 bytes for a 0.1.2 with no pre-release/build metadata, growing gracefully only when those fields are actually used.
• ww_date_time::DateTime - self_describing, a Year newtype storing years as UNib32 shifted by 2025 (so "now" costs a handful of bits, ancient or far-future dates cost more), and a Timezone that's UTC in 1 bit for the common case. A full UTC timestamp with second-level precision packs into 4 bytes:

  let dt = DateTime::from_ymd_hms_utc_opt(2025, 5, 30, 16, 20, 0, 0).unwrap();
  assert_eq!(dt.to_ww_bytes(&mut buf).unwrap(), hex!("05 F3 96 C0"));
  

• ww_client_server::PathKind - ww_repr = nib picking between three addressing modes (Absolute, GlobalCompact, GlobalFull), each shaped so the cheapest, most common case (a short absolute path) costs a single nibble discriminant plus the path itself, while the most expensive case (routing to an arbitrary crates.io trait by name) only costs more when you actually use it.

None of these types hand-roll any bit-twiddling - they're all plain #[derive_shrink_wrap(..)] structs/enums built from the pieces on this page.