The setup

At Astronics Test Systems I built the Flutter front end for a next-generation test system for land mobile radios. Firmware processes streamed measurements up over MQTT, including spectrum analyzer traces, and the UI charted them live at 30 Hz.

We serialized the payloads with FlatBuffers, picked over Protobuf because it has no unpacking step: accessors read fields directly out of the received bytes. That matters when you hand trace data to a chart thirty times a second.

The waterfall

While chasing dropped frames, I recorded a session with the Flutter DevTools profiler in VS Code and read through the waterfall. In the middle of every trace read was an allocation I couldn't account for. The trace bytes, already in memory inside the received message, were being copied into a fresh list before the chart could use them.

The copy came from the generated accessors. Our schema declares trace data as a [ubyte]vector, and FlatBuffers' Dart library surfaces that as a plain List<int> rather than a Uint8List. A List<int> is fine to iterate, but the chart and the math downstream want typed data, so the read path ended in Uint8List.fromList(): a full copy of every trace, thirty times a second, of bytes that were already in the right layout inside the buffer.

The open issue

I started on a fix in the library and, partway through, checked whether anyone had already reported the same behavior. Issue #8183 turned out to describe the same problem in an image workload: a [ubyte] vector, a List<int> accessor, and a forced copy to get typed data back. As the reporter put it, “the whole purpose of trying to use Flatbuffers instead of Protobuf was to avoid copying the byte array.” My case was packed signal data rather than an image, and I said as much in the thread.

A commenter had sketched a patch: have the lazy read path return a Uint8List.viewover the buffer instead of the library's _FbUint8Listwrapper. The right idea, but the diff itself wasn't correct. The signed bytes ended up written as Int32 instead of Int8, and it deleted the library's _FbInt8List wrapper rather than fixing the latent bug the thread had already spotted: Int8ListReader was using the unsigned wrapper while _FbInt8List sat unused. I pointed this out in the last comment on the issue and opened PR #8289 with an implementation that writes the bytes correctly and puts _FbInt8List to use for the signed reader.

The patch

The PR teaches the library's Dart implementation to return dart:typed_data equivalents where one exists: Uint8List, Int8List, Uint16List, and the other integer types, each a view backed by the message's own bytes. Reading a spectrum trace no longer allocates.

Byte order is the catch. FlatBuffers is little-endian on the wire, while typed-data views read multi-byte values in host order, so a view is only correct on little-endian hosts. On big-endian systems the library falls back to the existing element-by-element readers.

What review caught

My first version also converted the float readers. Review turned up a real problem there: a Float64 view needs 8-byte alignment, the buffer layout only guaranteed 4, and some buffers failed with a RangeError at runtime. I reverted the float changes and kept the scope to the integer types, which cover the byte-vector cases that motivated the work. The PR is reviewed and awaiting merge as I write this.

What I took from it

  • The profiler was the only thing that could have caught this. The schema, the docs, and the generated code all looked right. The copy only showed up in a recording of our actual read path.
  • A workaround in our app code would have taken an afternoon. The copy affects every Dart user of the library, though, so the fix belonged upstream.
  • Reverting the float readers shrank the PR to the cases that motivated it and kept an alignment bug out of a widely used library. The float types can follow once alignment is handled properly.

References