NAT traversal tracking issue #312

mxinden · 2021-04-08T16:00:49Z

This GitHub issue tracks the status of NAT traversal capabilities across libp2p implementations and platforms.

See Hole Punching document for greater picture.

Please comment below to suggest additions or corrections.

Projects

The following projects combine various protocols from the table below to achieve NAT traversal:

Project Flare - project proposal, Golang tracking issue and high level architecture
- Transport Protocols
  - TCP
  - QUIC
- TURN-like Protocols
  - circuit relay v2
- Signaling Protocols
  - Direct Connection Upgrade through Relay
- STUN-like Protocols
  - AutoNAT
- Other Protocols:
  - multistream-select v1 simulataneous open
  - AutoRelay
WebRTC star
- Transport Protocols
  - WebRTC star flavour
- Signaling Protocols
  - WebRTC star signaling protocol
HOPR Connect
- Transport Protocols
  - WebRTC HOPR connect flavour
- STUN-like Protols
  - STUN
- TURN-like Protocols
  - circuit relay inspired HOPR connect flavoured
- Signaling Protocols
  - HOPR signaling messages
Project proposal: browser nodes can connect to any node out of the box
- Transport Protocols
  - Websockets for signaling communication
  - WebRTC for data communication
- TURN-like Protocols
  - circuit relay v2
- STUN-like Protols
  - STUN
- Signaling Protocols
  - ?

Protocols

None of the Protocols below enable NAT traversal by themselves. Instead combinations of these protocols do.

	specification	go-libp2p	js-libp2p/browser	js-libp2p/NodeJS	rust-libp2p
Transport Protocols
TCP		go-tcp-transport + port-reuse		js-libp2p-tcp	rust libp2p-tcp + port-reuse option
QUIC		go-libp2p-quic-transport			libp2p/rust-libp2p#1334
Websockets		go-ws-transports	js-libp2p-websockets	js-libp2p-websockets	rust libp2p-websockets
WebRTC
* WebRTC spec compliant	#220
* WebRTC star flavoured		via go-libp2p-webrtc-star	via js-libp2p-webrtc-star	via js-libp2p-webrtc-star
* WebRTC HOPR connect flavoured				via hopr-connect



TURN-like Protocols
circuit relay v1	v1	go-libp2p-circuit	via js-libp2p	via js-libp2p	rust libp2p-relay
circuit relay v2	v2	go-libp2p-circuit			rust libp2p-relay
circuit relay inspired HOPR connect relay				via hopr-connect



Signaling Protocols
WebRTC specific or generalized signaling protocol	#159
Direct Connection Upgrade through Relay	DCUtR spec	libp2p/go-libp2p#1057			libp2p-dcutr
WebRTC star signaling		client via go-libp2p-webrtc-star	client via js-libp2p-webrtc-star	client + server via js-libp2p-webrtc-star
HOPR signaling messages				via hopr-connect




STUN-like Protocols
STUN				via hopr-connect
AutoNAT	AutoNAT spec	https://github.com/libp2p/go-libp2p-autonat			rust libp2p-autonat



Other Protocols
multistream-select v1 sim open	connections/simopen.md	libp2p/go-libp2p#712			libp2p/rust-libp2p#2066
AutoRelay	#181	go-libp2p/autorelay		libp2p/js-libp2p#699
UPnP		via go-libp2p-nat

Keywords for search engines: hole punching, TCP, QUIC, WebRTC, UPnP, ICE, STUN, TURN, meta

robertkiel · 2021-04-09T08:13:05Z

Hi @mxinden,

thanks for the overview.

Remarks concerning hopr-connect:

it uses a circuit-relay inspired protocol to exchange signaling messages (otherwise it is impossible to get through NATs)
direct connection upgrade is handled within hopr-connect
it could make sense to rename Signaling Protocols into TURN-like protocols

I think the description misses one of the main issues when reasoning about NAT traversal: there is a bunch of smaller protocols such as STUN, TURN, UPnP, TCP, and QUIC. Each of them is able to solve their main purpose for example STUN allows the discovery of public IPv4 addresses.

But the main issue is that a combination of these protocols is required to successfully bypass NATs, and this makes handling unsuccessful attempts and automatic fallbacks necessary. Otherwise you end up with a transport module that always fails on certain internet setups. Unfortunately, this is the most tricky part.

mxinden · 2021-04-09T17:25:22Z

Updated the description:

Split Signaling Protocols into TURN-like Protocols and Signaling Protocols.
Update Projects section to match Protocols sections.
Added HOPR connect Signaling and TURN-like protocol.

I think the description misses one of the main issues when reasoning about NAT traversal: there is a bunch of smaller protocols such as STUN, TURN, UPnP, TCP, and QUIC. Each of them is able to solve their main purpose for example STUN allows the discovery of public IPv4 addresses.

I am not quite sure I follow. Each of these are listed in the table. An individual protocol in itself can not achieve NAT traversal by itself. Instead Projects, like HOPR connect, are combining various protocols to overcome a NAT.

But the main issue is that a combination of these protocols is required to successfully bypass NATs, and this makes handling unsuccessful attempts and automatic fallbacks necessary. Otherwise you end up with a transport module that always fails on certain internet setups. Unfortunately, this is the most tricky part.

Correct. This should be handled by the individual libp2p implementations. Today, if none of the direct connection approaches succeed, the fallback would be the circuit relay v1 protocol.

mxinden · 2021-04-20T09:55:07Z

Changelog

Added Project Flare(decentralised Hole Punching) Phase1 Meta Issue go-libp2p#1039 to project Flare as another reading resource.
Added @vasco-santos's proposal ([WIP] propose: browser nodes can connect to any node out of the box protocol/web3-dev-team#70) as one of the projects involved in NAT traversal.

mxinden · 2021-04-27T10:47:40Z

Changelog

Referenced protocols/relay: Implement circuit relay v2 protocol rust-libp2p#2059 as the rust-libp2p circuit relay v2 implementation.

mxinden · 2021-05-05T14:58:15Z

Changelog

Reference p2p-circuit v2 spec #325 - circuit relay v2 specification draft.
Referenced Autonat rust-libp2p#1672 for AutoNAT implementation.
Reference misc/multistream-select: Implement simultaneous open extension rust-libp2p#2066 - an multistream-select 1.0 simultaneous open extension implementation.

mxinden · 2021-05-11T13:32:45Z

Changelog

With multistream-select 1.0: simultaneous open protocol extension #196 merged, link to connections/simopen.md directly.

mxinden · 2021-05-25T13:17:53Z

Changelog

Reference protocols: Implement Direct Connection Upgrade through Relay (dcutr) rust-libp2p#2076 - a Direct Connection Upgrade through Relay implementation for rust-libp2p.

mxinden · 2021-06-29T19:11:19Z

Changelog

Refernce Circuit Relay v2 specification directly with p2p-circuit v2 spec #325 being merged.

0xjjpa · 2021-07-09T08:56:04Z

Thanks for the update @mxinden!

@andreykiryushkin worth for you to take a dive on this now that you are exploring hopr-connect

mxinden · 2021-07-15T14:55:41Z

Below sequence diagram depicts the process of Hole Punching with Project Flare. I have yet to find the right place for this to live long-term. Still I am posting it here in case others are looking for a high level overview already today.

To reproduce via plantuml.com

@startuml
participant Initiator
participant Relay
participant Responder
collections Other_Peers

mainframe Goal: The **Initiator** peer establishes a direct connection to the non-dialable **Responder** peer.

== Responder determining whether it is dialable\n\nAutoNAT Protocol ==

Responder -> Other_Peers: Dial Request

alt Dialable. No need for Hole Punching. Don't continue.
    Other_Peers -> Responder: Dial Response "Ok"
else Not dialable. Hole Punching needed. Continue.
    Other_Peers -> Responder: Dial Response "Error"
end

== Responder finding closest public Relay nodes\n\n E.g. via Kademlia Protocol ==

Responder -> Other_Peers: Find nodes closest to Responder Peer ID
Other_Peers -> Responder: List of nodes closest to Responder Peer ID

== Responder listening for incoming connections via closest Relays\n\nCircuit Relay v2 Protocol ==

group For each closest Relay
  Responder -> Relay: Establish connection
  Responder -> Relay: Request reservation
  Relay -> Responder: Accept reservation
end

== Initiator establish relayed connection to Responder\n\nCircuit Relay v2 Protocol ==

Initiator -> Relay: Establish Connection
Initiator -> Relay: Request connection to Responder
Relay -> Responder: Request connection from Initiator
Responder -> Relay: Accept connection request
Relay -> Initiator: Accept connection request

skinparam sequenceMessageAlign center

Initiator <-> Responder: **Relayed Connection established**


== Initiator and Responder coordinate simultaneous Hole Punch\n\nDirect Connection Upgrade through Relay (DCUtR) Protocol ==

loop until direct connection established
  Initiator -> Responder: Sync message
  hnote over Initiator: Measure round-trip time (RTT)
  Responder -> Initiator: Sync message 
  Initiator -> Responder: Connect message

  hnote over Initiator, Responder: Simultaneously establish connection\nInitiator after 1/2 RTT, Responder when receiving Connect.
  Initiator <-> Responder: **Direct Connection established**
end
@enduml

mxinden · 2021-08-25T15:44:51Z

Changelog

DCUtR specification is merged (relay/DCUtR: Add Direct Connection Upgrade through Relay protocol #173), thus referencing the specification directly.
Circuit Relay v2 implementation on go-libp2p is merged (implement circuit v2 go-libp2p-circuit#136).
Track AutoRelay protocol (Add spec for AutoRelay #181).

cheako · 2021-11-10T15:20:27Z

https://discuss.libp2p.io/t/nat-traversal-with-libp2p/295
https://discuss.libp2p.io/t/how-to-construct-relay-peer/1112

cheako · 2021-11-10T15:21:55Z

https://gitlab.torproject.org/tpo/core/torspec/-/issues/64

cheako · 2021-11-11T06:29:48Z

There is also a transport that "talks" to a running instance of tor and configures it: https://github.com/berty/go-libp2p-tor-transport

mxinden · 2022-01-14T19:10:35Z

Changelog

AutoNAT specification merged via add a spec for autonat #362.
Rust AutoNAT implementation merged via protocols/: Add basic AutoNAT implementation rust-libp2p#2262.
Rust Circuit Relay implementation merged via protocols/relay: Implement circuit relay v2 protocol rust-libp2p#2059.

mxinden · 2022-02-09T16:18:26Z

Changelog

Rust DCUtR implementation merged via protocols/: Implement Direct Connection Upgrade through Relay (DCUtR) rust-libp2p#2438.

Thus all protocols for basic hole punching in rust-libp2p are in master now 🎉 If you want to punch holes yourself, check out libp2p/rust-libp2p#2460.

0xjjpa · 2022-02-10T13:29:18Z

Hey @mxinden, seems like a lot of work has been done lately around the rust implementation. Would it be safe to say that teams looking to implement hole-punching from scratch using libp2p API would be better off doing it via Rust?

vyzo · 2022-02-10T13:31:31Z

go-libp2p is further ahead and supports QUIC which has better penetration succeess rate, so no this isnt an accurate statement.

mxinden · 2022-02-10T15:44:19Z

a lot of work has been done lately around the rust implementation.

Yes, a lot of progress in regards to hole punching.

Would it be safe to say that teams looking to implement hole-punching from scratch using libp2p API would be better off doing it via Rust?

rust-libp2p does not yet have support for QUIC (see libp2p/rust-libp2p#2289) which bears higher hole punchings success rates given that it is based on UDP. For now, rust-libp2p only supports TCP hole punching. As mentioned above by @vyzo, go-libp2p is further ahead with QUIC support and a more battle tested hole punching stack.

TheRook · 2023-02-07T08:14:53Z

IPFS developed Delegate routers which can help connect nodes behind NAT as well as browser-based clients:
https://github.com/ipfs/js-ipfs/blob/master/docs/DELEGATE_ROUTERS.md

Something like delegates could be ported to the libp2p core and a side-channel like DNS, or another bootstrap can be used to identify delegate nodes which can facilitate connectivity. IPFS Delegates do not support the full range of libp2p features like pub/sub.

mxinden mentioned this issue Apr 8, 2021

Decentralized NAT traversal using nodes in the network #307

Open

mxinden mentioned this issue Apr 20, 2021

NAT traversal libp2p/rust-libp2p#2052

Open

14 tasks

mxinden mentioned this issue May 25, 2021

protocols: Implement Direct Connection Upgrade through Relay (dcutr) libp2p/rust-libp2p#2076

Closed

mxinden mentioned this issue Jun 11, 2021

Feat/comms behaviour protocol iotaledger/stronghold.rs#210

Merged

8 tasks

mxinden mentioned this issue Jun 21, 2021

rust-libp2p for browser-to-browser communication libp2p/rust-libp2p#2108

Closed

iduartgomez mentioned this issue Sep 1, 2021

NAT traversal freenet/freenet-core#2

Closed

10 tasks

mxinden mentioned this issue Dec 29, 2021

libp2p : Proper way to connect 2 peers behind their respective NAT each other libp2p/rust-libp2p#2351

Closed

cheako mentioned this issue Jan 6, 2022

Onion routing considerations for libp2p #200

Open

gitaaron mentioned this issue Mar 9, 2022

research fula-client overcoming NAT to reach fula-api functionland/docs#28

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NAT traversal tracking issue #312

NAT traversal tracking issue #312

mxinden commented Apr 8, 2021 •

edited

Loading

robertkiel commented Apr 9, 2021

mxinden commented Apr 9, 2021

mxinden commented Apr 20, 2021

mxinden commented Apr 27, 2021

mxinden commented May 5, 2021

mxinden commented May 11, 2021

mxinden commented May 25, 2021

mxinden commented Jun 29, 2021

0xjjpa commented Jul 9, 2021

mxinden commented Jul 15, 2021

mxinden commented Aug 25, 2021 •

edited

Loading

cheako commented Nov 10, 2021

cheako commented Nov 10, 2021

cheako commented Nov 11, 2021

mxinden commented Jan 14, 2022

mxinden commented Feb 9, 2022

0xjjpa commented Feb 10, 2022

vyzo commented Feb 10, 2022

mxinden commented Feb 10, 2022

TheRook commented Feb 7, 2023

NAT traversal tracking issue #312

NAT traversal tracking issue #312

Comments

mxinden commented Apr 8, 2021 • edited Loading

Projects

Protocols

robertkiel commented Apr 9, 2021

mxinden commented Apr 9, 2021

mxinden commented Apr 20, 2021

Changelog

mxinden commented Apr 27, 2021

mxinden commented May 5, 2021

mxinden commented May 11, 2021

mxinden commented May 25, 2021

mxinden commented Jun 29, 2021

0xjjpa commented Jul 9, 2021

mxinden commented Jul 15, 2021

mxinden commented Aug 25, 2021 • edited Loading

cheako commented Nov 10, 2021

cheako commented Nov 10, 2021

cheako commented Nov 11, 2021

mxinden commented Jan 14, 2022

mxinden commented Feb 9, 2022

0xjjpa commented Feb 10, 2022

vyzo commented Feb 10, 2022

mxinden commented Feb 10, 2022

TheRook commented Feb 7, 2023

mxinden commented Apr 8, 2021 •

edited

Loading

mxinden commented Aug 25, 2021 •

edited

Loading