1.7 KiB
| layout | title | description | category | tags |
|---|---|---|---|---|
| post | new post |
Designing the Test
My use case is as follows: ingest binary market data from IEX and turn it into a format understandable by each library being tested. Then we'll write a simple program to find total trade volume per ticker, and the highest and lowest bid/ask price per ticker as well.
Note: Market data is the use case here simply because IEX makes the data freely available; no code or analysis in this blog is related to my past or present work.
Now, the basic criteria used to evaluate each library:
-
The library must have cross-language support, and treat Rust as a first-class citizen.
-
The schema must be able to evolve and add new fields. The information I'm gathering now is fairly simple, but would evolve in the future.
-
Performance is a priority; material performance differences (time to de/serialize, memory usage) matter.
Under those three criteria, we're excluding a lot of systems that may make sense in other contexts:
-
Bincode has great Rust support and a simple wire format (message structure) but isn't usable from other languages and doesn't deal well with schema evolution.
-
Protocol Buffers have great cross-language support, but material performance issues compared to other systems like FlatBuffers.
-
JSON/Msgpack are schema-less; while the wire format is simple, having code generated from a schema is too nice to pass up.
While each of these have a niche they perform well in, they're not suited for the system under consideration.