-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add a roadmap for the next 6 months #1784
Changes from 3 commits
347f246
d06659f
60c4fce
ddc29ae
51f4cb8
80b7746
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,64 @@ | ||||||
# go-libp2p roadmap Q4’22/Q1’23 | ||||||
|
||||||
## 📊 Comprehensive Metrics | ||||||
|
||||||
Why: For far too long, go-libp2p has been a black box. This has hurt us many times, by allowing trivial bugs to go undetected for a long time ([example](https://github.com/ipfs/kubo/pull/8750)). Having metrics will allow us to track the impact of performance improvements we make over time. | ||||||
|
||||||
Goal: Export a wider set of metrics across go-libp2p components and enable node operators to monitor their nodes in production. Optionally provide a sample Grafana dashboard similar to the resource manager dashboard. | ||||||
|
||||||
How: This will look similar to how we already expose resource manager metrics. Metrics can be added incrementally for libp2p’s components. First milestone is having metrics for the swarm. | ||||||
|
||||||
Tracking issue: [https://github.com/libp2p/go-libp2p/issues/1356](https://github.com/libp2p/go-libp2p/issues/1356) | ||||||
|
||||||
## 📺 Universal Browser Connectivity | ||||||
|
||||||
Why: A huge part of “the Web” is happening inside the browser. As a universal p2p networking stack, libp2p needs to be able to offer solutions for browser users. | ||||||
|
||||||
Goal: go-libp2p ships with up-to-date WebTransport and (libp2p-) WebRTC implementations, enabled by default. This allows connections between browsers and public nodes, browsers and non-public nodes as well as two browsers. | ||||||
|
||||||
1. WebTransport: as the protocol is still under development by IETF and W3C, our implementation needs to follow. To stay up to date, we will have to move as soon as Chrome ships support for a new draft version | ||||||
2. WebRTC: while browser to public node is getting close to finalized, there’ll be a push to make the other combinations work as well | ||||||
|
||||||
## ⚡️Handshakes at the Speed of Light | ||||||
|
||||||
Why: Historically, libp2p has been very wasteful when it comes to round trips spent during connection establishment. This is slowing down our users, especially their TTFB (time to first byte) metrics. | ||||||
|
||||||
Goal: go-libp2p optimizes its handshake latency up to the point where only increasing the speed of light would lead to further speedups. In particular, this means (in chronological order of a handshake): | ||||||
|
||||||
1. cutting off the 1 RTT wasted on security protocol negotiation by including the security protocol in the multiaddr: [https://github.com/libp2p/specs/pull/353](https://github.com/libp2p/specs/pull/353) | ||||||
2. cutting off the 1 RTT wasted on muxer negotiation: [https://github.com/libp2p/specs/issues/426](https://github.com/libp2p/specs/issues/426) | ||||||
3. using 0.5-RTT data (for TLS) / a Noise Extension to ship the list of Identify protocols, cutting of 1 RTT that many protocols spend waiting on `IdentifyWait` | ||||||
|
||||||
## 🧠 Smart Dialing | ||||||
|
||||||
Why: Having a large list of transports to pick from is great. Having an advanced stack that can dial all of them is even greater. But dialing all of them at the same time wastes our, the network’s and the peer’s resources. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It also wastes our CPU cycles to start that many handshakes. |
||||||
|
||||||
Goal: When given a list of multiaddrs of a peer, go-libp2p is smart enough to pick the address that results in the most performant connection (for example, preferring QUIC over TCP), while also picking the address such that maximizes the likelihood of a successful handshake. | ||||||
|
||||||
How: | ||||||
|
||||||
1. implement some kind of “Happy-Eyeballs” style prioritization among all supported transports | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
2. estimation of the expected RTT of a connection based on two nodes’ IP addresses, so that Happy Eyeballs Timeouts can be set dynamically | ||||||
3. detection of blackholes, especially relevant to detect UDP (QUIC) blackholing | ||||||
|
||||||
## 🧪 Future-proof Testing | ||||||
|
||||||
Why: Having lots of transports is great, and shaving off RTTs is awesome. We need to stay backwards-compatible with legacy go-libp2p implementations, and less advanced libp2p stacks. | ||||||
|
||||||
Goal: We have cross-version and cross-implementation Testground coverage that makes sure that we are able to establish a libp2p connection between two nodes, in the expected number of RTTs. This makes sure that the optimizations don’t break compatibility, actually have the performance impact we expect them to have, and serves as a regression test in the future. | ||||||
marten-seemann marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We talk about ensuring compability between versions and implementations, but I assume there is an extra dimension we need to add around transports (and likely encryption and muxer). Basically want to show <go-libp2p, vX, QUIC> can talk with <go-libp2p, vY, QUIC> |
||||||
|
||||||
## ⏱ Future-proof Benchmarking | ||||||
|
||||||
Why: For libp2p to be competitive, it needs to delivers comparable performance to widely used protocols on the internet, namely HTTP/2 and HTTP/3. | ||||||
|
||||||
Goal: We have a test suite that runs libp2p transfers between nodes located at different locations all over the world, proving that libp2p is able to achieve performance on par with HTTP. The test suite is run on a continuous basis and results are published to a public performance dashboard. | ||||||
|
||||||
## 📢 Judicious Address Advertisements | ||||||
|
||||||
Why: A node that advertises lots of addresses hurts itself. Other nodes will have to try dialing a lot of addresses before they find one that actually works, dramatically increasing handshake latencies. | ||||||
|
||||||
Goal: Nodes only advertise addresses that they are actually reachable at. | ||||||
|
||||||
How: Unfortunately, the AutoNAT protocol can’t be used to probe the reachability of any particular address (especially due to a bug in the go-libp2p implementation deployed years ago). Most likely, we need a second version of the AutoNAT protocol. | ||||||
|
||||||
Related discussion: [https://github.com/libp2p/go-libp2p/issues/1480](https://github.com/libp2p/go-libp2p/issues/1480) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great write up!
A general thought is that as libp2p is getting more widely deployed maybe we also need to think about beef up more security features. May or may not fit in to the next 6 months time frame.
For example behavior based security. The peer is usually not backed by a root of trust, so the traditional identity and chain of trust based approach does not work in this scenario. We may need to consider new approaches to enforce security.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@julian88110 Do you have anything specific in mind?
Would that belong into the go-libp2p roadmap, or would that first and foremost be an item for the general libp2p roadmap (https://github.com/libp2p/specs/blob/master/ROADMAP.md)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes this should belong to the general roadmap for libp2p. I have some thoughts, may take some time to write it up after the muxer thing is done.