mirror of
https://github.com/bspeice/speice.io
synced 2024-11-05 01:28:09 -05:00
45 lines
2.1 KiB
Markdown
45 lines
2.1 KiB
Markdown
|
---
|
||
|
|
layout: post
|
||
|
|
title: "Binary Format Shootout - Prologue: Nom"
|
||
|
|
description: "Making sense of binary streams"
|
||
|
|
category:
|
||
|
|
tags: [rust, binary-shootout]
|
||
|
|
---
|
||
|
|
|
||
|
|
I've been interested in using a binary protocol library for personal projects recently,
|
||
|
|
and found myself with a strong case of decision paralysis. Do I use
|
||
|
|
[Cap'n Proto](https://capnproto.org/), which has supported Rust the longest?
|
||
|
|
[Flatbuffers](https://google.github.io/flatbuffers) recently added support,
|
||
|
|
or I could take a look at [SBE](https://github.com/real-logic/simple-binary-encoding).
|
||
|
|
Or what about building something myself? A lot of these seem unnecessarily
|
||
|
|
complicated, when my personal use case is just providing views on top of
|
||
|
|
buffers with a relatively simple structure.
|
||
|
|
|
||
|
|
Even in my personal projects, I want the choices to be the best possible;
|
||
|
|
I hate the feeling of looking back at anything I've built and saying "I regret
|
||
|
|
that decision and I could have done better." So after agonizing over the choice
|
||
|
|
of protocol library for too long, I decided it would be worth building a test
|
||
|
|
to get a feel for each. It would give me a way to build a proof-of-concept
|
||
|
|
and become familiar with how each library worked, what the performance
|
||
|
|
characteristics were of each, and evaluate whether it was worth putting
|
||
|
|
in the effort of building yet another binary protocol library myself.
|
||
|
|
|
||
|
|
To that end, this is the summation of research into the binary protocol
|
||
|
|
systems that currently support Rust. The goal isn't to recommend "the best,"
|
||
|
|
but to understand each well enough to make an informed decision.
|
||
|
|
|
||
|
|
My use case is as follows: ingest binary market data from
|
||
|
|
[IEX](https://iextrading.com/trading/market-data/) and turn it into
|
||
|
|
a format understandable by each library being tested. We'll later
|
||
|
|
write a simple program to analyze the data.
|
||
|
|
|
||
|
|
<span style="font-size: .8em">Note: Market data is the use case here
|
||
|
|
simply because IEX makes the data freely available; no code or analysis
|
||
|
|
in this blog is related to my past or present work.</span>
|
||
|
|
|
||
|
|
But before we can run any analysis, we need to read in the files
|
||
|
|
supplied by IEX. To do that, we'll use a library in Rust
|
||
|
|
called [`nom`](https://docs.rs/nom/5.0.1/nom/).
|
||
|
|
|
||
|
|
# Ingesting Market Data
|