Skip to content

Commit

Permalink
*: replace rust-protobuf with prost
Browse files Browse the repository at this point in the history
It's time to switch from rust-protobuf to Prost. There are a few factors
in this decision:

  * The maintainer of rust-protobuf has announced that the project is
    essentially unmaintained for the time being [0].

  * Prost is the more popular Protobuf library across the ecosystem. In
    particular, the Tonic GRPC library uses Prost, and the Tokio tracing
    console uses Prost. These are both extremely important for
    Materialize Platform, which we expect to be built on top of Tonic
    and making heavy use of distributed tracing.

There have been been two major blockers to adopting Prost:

  1. Prost lacked dynamic message support. That was miraculously fixed
     last month with the release of a fantastic prost-reflect crate [1].

  2. Prost embeds `protoc` in its Git repository (!). See
     tokio-rs/prost#575. That is a disaster for supply chain security.
     This commit patches in a Materialize-specific fork of Prost that
     compiles `protoc` from source rather than using upstream's embedded
     binary blobs.

This commit excludes one crucial piece: converting the dynamic Protobuf
decoding in the `interchange` and `testdrive` crates from rust-protobuf
to prost-reflect. I'll do that migration in a future commit to avoid
inflating the size of this PR unnecessarily.

[0]: stepancheg/rust-protobuf#583 (comment)
[1]: https://www.reddit.com/r/rust/comments/rsg6gx/announcing_prostreflect_a_crate_for_protobuf/
  • Loading branch information
benesch committed Jan 19, 2022
1 parent e32c1c4 commit 1463421
Show file tree
Hide file tree
Showing 43 changed files with 609 additions and 454 deletions.
219 changes: 215 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -82,3 +82,6 @@ debug = 1
headers = { git = "https://github.com/MaterializeInc/headers.git" }
# Until https://github.com/jorgecarleitao/parquet-format-rs/pull/2 is merged and released
parquet-format-async-temp = { git = "https://github.com/MaterializeInc/parquet-format-rs", branch = "main" }
# Waiting on https://github.com/tokio-rs/prost/pull/576.
prost = { git = "https://github.com/MaterializeInc/prost.git" }
prost-types = { git = "https://github.com/MaterializeInc/prost.git" }
Loading

0 comments on commit 1463421

Please sign in to comment.