Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stabilize the "IP" feature #76098

Closed
wants to merge 23 commits into from
Closed

Stabilize the "IP" feature #76098

wants to merge 23 commits into from

Conversation

caass
Copy link
Contributor

@caass caass commented Aug 30, 2020

Latest Status Update: #76098 (comment)


Continuation of #66584, since it seems to be dead. This is my first time contributing and I did my best to read through the rustc-dev-guide, but I'll be honest I kind of skipped around a little bit. If there's anything I need to do that hasn't been done yet in terms of process, please let me know and I'll do it 😄

The old PR seemed to me (could be wrong) to be almost done, but there were two issues that were left unresolved. I left a comment on the PR, but I'll copy/paste it here for convenience:

If I understand correctly, the blockers currently are:

  1. There should be some checking for IPv4 addresses mapped it IPv6 space (comment)

From the RFC:

Two types of IPv6 addresses are defined that carry an IPv4 address in the low-order 32 bits of the address. These are the "IPv4-Compatible IPv6 address" and the "IPv4-mapped IPv6 address".

[...]

The "IPv4-Compatible IPv6 address" was defined to assist in the IPv6 transition. The format of the "IPv4-Compatible IPv6 address" is as follows:

80 bits 16 bits 32 bits
0000..............................0000 0000 IPv4 address

[...]

A second type of IPv6 address that holds an embedded IPv4 address is defined. This address type is used to represent the addresses of IPv4 nodes as IPv6 addresses. The format of the "IPv4-mapped IPv6 address" is as follows:

80 bits 16 bits 32 bits
0000..............................0000 FFFF IPv4 address

If we already have capability to check IPv4 addresses, I feel like it's almost trivial to implement these rules for IPv6 as well if we just need to pad it with some extra bytes? Famous last words, I know...

  1. There's some concern over the need for both is_unicast_link_local_strict and is_unicast_link_local (comment)

From the commit that introduced the change:

RFC 4291 is a little unclear about what is a unicast link local address.
According to section 2.4, the entire fe80::/10 range is reserved for
these addresses, but section 2.5.3 defines a stricter format for such
addresses.

After a discussion is has been decided to add a different method for
each definition, so this commit:

  • renames is_unicast_link_local() into is_unicast_link_local_strict()
  • relaxed the check in is_unicast_link_local()

From section 2.4:

Address type Binary prefix IPv6 notation Section
Unspecified 00...0 (128 bits) ::/128 2.5.2
Loopback 00...1 (128 bits) ::1/128 2.5.3
Multicast 11111111 FF00::/8 2.7
Link-Local unicast 1111111010 FE80::/10 2.5.6
Global Unicast (everything else)

Section section 2.5.3 seems...unrelated? It has to do with the loopback address. Section 2.5.6 however, is where the problem lies:

Link-Local addresses are for use on a single link. Link-Local addresses have the following format:

10 bits 16 bits 64 bits
1111111010 0 interface ID

So the difference in the spec is this:

Spec CIDR Start End Available
2.4 FE80::/10 fe80:0:0:0:0:0:0:0 febf:ffff:ffff:ffff:ffff:ffff:ffff:ffff 2^118
2.5.6 FE80::/64 fe80:0:0:0:0:0:0:0 fe80:0:0:0:ffff:ffff:ffff:ffff 2^64

This does seem like an issue, so I did some more digging to figure out what's going on. I found this draft, which appears to have died. It does however point us in the right direction:

The RFC "IPv6 Address Archi" illustrates the format of the link-local addresses. From the illustration it MAY be understood that the length of the link-local prefix is 10 bits of value 1111111010 and 54 0 bits.

IANA lists the "IPv6 prefix", and "Address Block", to be "fe80::/10" on its website. It is possible that in the future the IETF could decide to use the bits 11-53.

The RFC 2464 "IPv6-over-Ethernet" states that the prefix for link-local addresses is "fe80::/64".

RFC 6874, "Representing IPv6 Zone Identifiers in Address Literals and Uniform Resource Identifiers" specifies the link-local addresses to be under prefix "fe80::/10".

I'm not an expert in reading IETF material -- far from it -- but I am vaguely aware of the passage of time. Most documents linked here that specify /64 are more recent than those specifying /10, unless I've missed some -- which is totally possible. Additionally, there are countless secondhand sources (blogs, stackoverflow, etc) online all talking about how /64 is what's commonly used.

Again, not an expert. I posted all this info here so someone more versed than me can evaluate and decide what the best course of action is. Without further input, I'd personally go ahead with just the one is_unicast_link_local that validates against /64, since it's a more strict subset of /10, and if someone runs into trouble with running custom link-local addresses outside of /64 space I kind of feel like that is a problem outside the scope of the Rust programming language, and is something they should talk to their network administrator about.

Edit: Per further discussion here and here, I've decided to simply remove the _strict check to bring Rust's functionality more in line with other similar features in other languages (e.g. Go, C, Python) as well as the Linux kernel.

At the moment, this PR is just fast-forwarding the existing work done by @little-dude to be up-to-date with rust-lang/rust. I wanted to get some confirmation that this hasn't been implemented yet in some other way / i'm not duplicating work / this feature is still needed before I get started on the code. Thanks in advance for your time! ❤️

@rust-highfive
Copy link
Collaborator

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @cramertj (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 30, 2020
@caass
Copy link
Contributor Author

caass commented Aug 30, 2020

r? @KodrAus

hi! tagging you because you were pretty active on the old pr 😅

@rust-highfive rust-highfive assigned KodrAus and unassigned cramertj Aug 30, 2020
library/std/src/net/ip.rs Outdated Show resolved Hide resolved
library/std/src/net/ip.rs Outdated Show resolved Hide resolved
@caass caass marked this pull request as draft August 30, 2020 16:08
@KodrAus
Copy link
Contributor

KodrAus commented Aug 31, 2020

Thanks for taking the time to dig through the state of things nd put this together @caass!

I’ll spend some time digging through this and try follow along the rationale.

Also cc @nanpuyue who might be interested in this.

@KodrAus
Copy link
Contributor

KodrAus commented Aug 31, 2020

Since the FCP on the previous PR didn’t complete I think we’ll need to start a fresh one once this is ready to go.

@little-dude
Copy link
Contributor

Since the FCP on the previous PR didn’t complete I think we’ll need to start a fresh one once this is ready to go.

Yeah I apologize for the time reviewers lost on this =/ And thank you @caass for picking this up.

@caass
Copy link
Contributor Author

caass commented Aug 31, 2020

Yeah I apologize for the time reviewers lost on this =/

@little-dude without your work I wouldn't have even known where to start -- not all is lost!

@bors
Copy link
Contributor

bors commented Aug 31, 2020

☔ The latest upstream changes (presumably #75979) made this pull request unmergeable. Please resolve the merge conflicts.

@caass caass marked this pull request as ready for review September 4, 2020 03:42
@KodrAus
Copy link
Contributor

KodrAus commented Sep 4, 2020

@little-dude Oh that's not at all a comment on you! You've done awesome work bringing this together.

Copy link
Contributor

@KodrAus KodrAus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking the time to put this together @caass!

I've left some suggestions for adding the right #[stable] attributes we need to actually stabilize this API. It's not visible from the diff, but at the top of ip.rs there's also this attribute:

#![unstable(
    feature = "ip",
    reason = "extra functionality has not been \
                                      scrutinized to the level that it should \
                                      be to be stable",
    issue = "27709"
)]

We'll need to remove this along with adding the #[stable] attributes.

library/std/src/net/ip.rs Outdated Show resolved Hide resolved
library/std/src/net/ip.rs Show resolved Hide resolved
library/std/src/net/ip.rs Outdated Show resolved Hide resolved
library/std/src/net/ip.rs Outdated Show resolved Hide resolved
library/std/src/net/ip.rs Outdated Show resolved Hide resolved
library/std/src/net/ip.rs Outdated Show resolved Hide resolved
library/std/src/net/ip.rs Outdated Show resolved Hide resolved
library/std/src/net/ip.rs Outdated Show resolved Hide resolved
library/std/src/net/ip.rs Outdated Show resolved Hide resolved
library/std/src/net/ip.rs Outdated Show resolved Hide resolved
@KodrAus
Copy link
Contributor

KodrAus commented Sep 4, 2020

I'd personally go ahead with just the one is_unicast_link_local that validates against /64, since it's a more strict subset of /10, and if someone runs into trouble with running custom link-local addresses outside of /64 space I kind of feel like that is a problem outside the scope of the Rust programming language, and is something they should talk to their network administrator about.

This sounds like a good path forward to me! Thanks for digging through and making sense of all this material ❤️

@caass
Copy link
Contributor Author

caass commented Sep 4, 2020

Ok, so I went through and added checks for secret hidden IPv4 addresses to all (i think -- i may have missed some) of the IPv6 checks. I also renamed is_unicast_link_local_strict to is_unicast_link_local and removed the old is_unicast_link_local.

The last thing to do I think is to add test cases for the edge cases of e.g. ::ffff:7f00:1 as an IPv6-mapped 127.0.0.1. I'm not sure about the best way to do this, as there's some macro_rules! stuff and I've never really messed with that before. Additionally, there's some bit-shifting going on which always gives me a headache.

Any sort of walkthrough on how lines 213-261 work would be very, very much appreciated.

@bors
Copy link
Contributor

bors commented Sep 7, 2020

☔ The latest upstream changes (presumably #76422) made this pull request unmergeable. Please resolve the merge conflicts.

Copy link
Contributor

@KodrAus KodrAus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @caass!

I think the merge conflicts will be coming from this change: a2e077e They've just moved inline to the ip module.

It looks like we've got a failing UI test in here:

failures:

---- [ui] ui/consts/std/net/ipv6.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/consts/std/net/ipv6/a"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
thread 'main' panicked at 'assertion failed: IS_LOOPBACK', /checkout/src/test/ui/consts/std/net/ipv6.rs:21:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

------------------------------------------

It might be easier to figure out what test this is after a rebase.

#[rustc_const_unstable(feature = "const_ipv6", issue = "76205")]
#[stable(since = "1.7.0", feature = "ip_17")]
pub const fn is_unspecified(&self) -> bool {
u128::from_be_bytes(self.octets()) == u128::from_be_bytes(Ipv6Addr::UNSPECIFIED.octets())
if let Some(v4_addr) = self.to_ipv4() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is a behavioral change, because an unspecified address is also just 0.0.0.0.

#[rustc_const_unstable(feature = "const_ipv6", issue = "76205")]
#[stable(since = "1.7.0", feature = "ip_17")]
pub const fn is_loopback(&self) -> bool {
u128::from_be_bytes(self.octets()) == u128::from_be_bytes(Ipv6Addr::LOCALHOST.octets())
if let Some(v4_addr) = self.to_ipv4() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documenting that this is a behavioral breaking change for:

  • ::127.0.0.1 / ::ffff:127.0.0.1

#[rustc_const_unstable(feature = "const_ipv6", issue = "76205")]
#[stable(since = "1.7.0", feature = "ip_17")]
pub const fn is_multicast(&self) -> bool {
(self.segments()[0] & 0xff00) == 0xff00
if let Some(v4_addr) = self.to_ipv4() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documenting that this is a behavioral breaking change for:

  • ::224.254.0.0 / ::ffff:224.254.0.0
  • ::236.168.10.65 / ::ffff:236.168.10.65

@jonas-schievink jonas-schievink added relnotes Marks issues that should be documented in the release notes of the next release. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. labels Sep 13, 2020
@JohnCSimon JohnCSimon added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 29, 2020
@JohnCSimon
Copy link
Member

Ping from triage
@caass - Can you please address the build failures and merge conflicts, thank you.

@caass
Copy link
Contributor Author

caass commented Sep 29, 2020

@caass - Can you please address the build failures and merge conflicts, thank you.

@JohnCSimon hi! I have been sitting here hoping someone with more knowledge of macros than I will be able to write/fix the test cases before i keep pushing on this PR, but I suppose eventually I'll have to learn them myself 😅 I'll resolve the merge conflicts, but the tests are broken until I (or someone else 🤞) rewrite them to reflect the behavioral changes mentioned by @KodrAus :)

@little-dude
Copy link
Contributor

@caass I can probably help with that.

@caass
Copy link
Contributor Author

caass commented Sep 29, 2020

@caass I can probably help with that.

!!! @little-dude i would love that! i'll merge later today and write some comments talking about what exactly needs to be done :)

@Dylan-DPC-zz
Copy link

r? @dtolnay

@caass any updates on the failing tests? thanks

@rust-highfive rust-highfive assigned dtolnay and unassigned KodrAus Apr 1, 2021
@lygstate
Copy link
Contributor

Would the platform-independent functional to be moved to core?

@crlf0710
Copy link
Member

crlf0710 commented May 1, 2021

@caass Ping from triage, CI is still red. Would you mind fixing the tests? Thanks.

@crlf0710 crlf0710 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 1, 2021
@JohnCSimon JohnCSimon added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 17, 2021
@JohnCSimon
Copy link
Member

Ping from triage:
@caass - no activity since February 25th, I'm closing this as inactive. Feel free to reopen when you have the chance to continue. Thanks!

@JohnCSimon JohnCSimon closed this May 17, 2021
JohnTitor added a commit to JohnTitor/rust that referenced this pull request May 26, 2021
Move stability attribute for items under the `ip` feature

The `#[unstable]` attribute for items under the `ip` feature is currently located on the `std::net::ip` module itself. This is unusual, and less readable. This has sidetracked discussion about these items numerous times (rust-lang#60145 (comment), rust-lang#76098 (comment), rust-lang#76098 (comment), rust-lang#75019 (comment), rust-lang#75019 (comment)) and lead to incorrect assumptions about which items are actually stable (rust-lang#60145 (comment), rust-lang#76098 (comment)).

This PR moves the attribute from the module to the items themselves.
JohnTitor added a commit to JohnTitor/rust that referenced this pull request May 26, 2021
Move stability attribute for items under the `ip` feature

The `#[unstable]` attribute for items under the `ip` feature is currently located on the `std::net::ip` module itself. This is unusual, and less readable. This has sidetracked discussion about these items numerous times (rust-lang#60145 (comment), rust-lang#76098 (comment), rust-lang#76098 (comment), rust-lang#75019 (comment), rust-lang#75019 (comment)) and lead to incorrect assumptions about which items are actually stable (rust-lang#60145 (comment), rust-lang#76098 (comment)).

This PR moves the attribute from the module to the items themselves.
bors added a commit to rust-lang-ci/rust that referenced this pull request May 31, 2021
…r=joshtriplett

Remove `Ipv6Addr::is_unicast_link_local_strict`

Removes the unstable method `Ipv6Addr::is_unicast_link_local_strict` and keeps the behaviour of `Ipv6Addr::is_unicast_link_local`, see also rust-lang#85604 where I have tried to summarize related discussion so far.

My intent is for `is_unicast_link_local`, `is_unicast_site_local` and `is_unicast_global` to have the semantics of checking if an address has Link-Local, Site-Local or Global scope, see also rust-lang#85696 which changes the behaviour of `is_unicast_global` and renames these methods to `has_unicast_XXX_scope` to reflect this.

For checking Link-Local scope we currently have two methods: `is_unicast_link_local` and `is_unicast_link_local_strict`. This is because of what appears to be conflicting definitions in [IETF RFC 4291](https://datatracker.ietf.org/doc/html/rfc4291).

From [IETF RFC 4291 section 2.4](https://datatracker.ietf.org/doc/html/rfc4291#section-2.4): "Link-Local unicast" (`FE80::/10`)
```text
Address type         Binary prefix        IPv6 notation   Section
------------         -------------        -------------   -------
Unspecified          00...0  (128 bits)   ::/128          2.5.2
Loopback             00...1  (128 bits)   ::1/128         2.5.3
Multicast            11111111             FF00::/8        2.7
Link-Local unicast   1111111010           FE80::/10       2.5.6
Global Unicast       (everything else)
```

From [IETF RFC 4291 section 2.5.6](https://datatracker.ietf.org/doc/html/rfc4291#section-2.5.6): "Link-Local IPv6 Unicast Addresses" (`FE80::/64`)
```text
| 10 bits  |         54 bits         |          64 bits           |
+----------+-------------------------+----------------------------+
|1111111010|           0             |       interface ID         |
+----------+-------------------------+----------------------------+
```

With `is_unicast_link_local` checking `FE80::/10` and `is_unicast_link_local_strict` checking `FE80::/64`.

There is also [IETF RFC 5156 section 2.4](https://datatracker.ietf.org/doc/html/rfc5156#section-2.4) which defines "Link-Scoped Unicast" as `FE80::/10`.

It has been pointed out that implementations in other languages and the linux kernel all use `FE80::/10` (rust-lang#76098 (comment), rust-lang#76098 (comment)).

Given all of this I believe the correct interpretation to be the following: All addresses in `FE80::/10` are defined as having Link-Local scope, however currently only the block `FE80::/64` has been allocated for "Link-Local IPv6 Unicast Addresses". This might change in the future however; more addresses in `FE80::/10` could be allocated and those will have Link-Local scope. I therefore believe the current behaviour of `is_unicast_link_local` to be correct (if interpreting it to have the semantics of `has_unicast_link_local_scope`) and `is_unicast_link_local_strict` to be unnecessary, confusing and even a potential source of future bugs:

Currently there is no real difference in checking `FE80::/10` or `FE80::/64`, since any address in practice will be `FE80::/64`. However if an application uses `is_unicast_link_local_strict` to implement link-local (so non-global) behaviour, it will be incorrect in the future if addresses outside of `FE80::/64` are allocated.

r? `@joshtriplett` as reviewer of all the related PRs
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Oct 15, 2023
Stabilize `{IpAddr, Ipv6Addr}::to_canonical`

Make `IpAddr::to_canonical` and `IpV6Addr::to_canonical` stable (+const), as well as const stabilize `Ipv6Addr::to_ipv4_mapped`.

Newly stable API:

```rust
impl IpAddr {
    // Newly stable under `ip_to_canonical`
    const fn to_canonical(&self) -> IpAddr;
}

impl Ipv6Addr {
    // Newly stable under `ip_to_canonical`
    const fn to_canonical(&self) -> IpAddr;

    // Already stable, this makes it const stable under
    // `const_ipv6_to_ipv4_mapped`
    const fn to_ipv4_mapped(&self) -> Option<Ipv4Addr>
}
```

These stabilize a subset of the following tracking issues:

- rust-lang#27709
- rust-lang#76205

Stabilization of all methods under the `ip` gate was attempted once at rust-lang#66584 then again at rust-lang#76098. These were not successful because there are still unknowns about `is_documentation` `is_benchmarking` and similar; `to_canonical` is much more straightforward.

I have looked and could not find any known issues with `to_canonical`. These were added in 2021 in rust-lang#87708

cc implementor `@the8472`

r? libs-api
`@rustbot` label +T-libs-api +needs-fcp
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Oct 16, 2023
Stabilize `{IpAddr, Ipv6Addr}::to_canonical`

Make `IpAddr::to_canonical` and `IpV6Addr::to_canonical` stable (+const), as well as const stabilize `Ipv6Addr::to_ipv4_mapped`.

Newly stable API:

```rust
impl IpAddr {
    // Newly stable under `ip_to_canonical`
    const fn to_canonical(&self) -> IpAddr;
}

impl Ipv6Addr {
    // Newly stable under `ip_to_canonical`
    const fn to_canonical(&self) -> IpAddr;

    // Already stable, this makes it const stable under
    // `const_ipv6_to_ipv4_mapped`
    const fn to_ipv4_mapped(&self) -> Option<Ipv4Addr>
}
```

These stabilize a subset of the following tracking issues:

- rust-lang#27709
- rust-lang#76205

Stabilization of all methods under the `ip` gate was attempted once at rust-lang#66584 then again at rust-lang#76098. These were not successful because there are still unknowns about `is_documentation` `is_benchmarking` and similar; `to_canonical` is much more straightforward.

I have looked and could not find any known issues with `to_canonical`. These were added in 2021 in rust-lang#87708

cc implementor ``@the8472``

r? libs-api
``@rustbot`` label +T-libs-api +needs-fcp
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Oct 16, 2023
Rollup merge of rust-lang#115955 - tgross35:ip-to-canonical, r=dtolnay

Stabilize `{IpAddr, Ipv6Addr}::to_canonical`

Make `IpAddr::to_canonical` and `IpV6Addr::to_canonical` stable (+const), as well as const stabilize `Ipv6Addr::to_ipv4_mapped`.

Newly stable API:

```rust
impl IpAddr {
    // Newly stable under `ip_to_canonical`
    const fn to_canonical(&self) -> IpAddr;
}

impl Ipv6Addr {
    // Newly stable under `ip_to_canonical`
    const fn to_canonical(&self) -> IpAddr;

    // Already stable, this makes it const stable under
    // `const_ipv6_to_ipv4_mapped`
    const fn to_ipv4_mapped(&self) -> Option<Ipv4Addr>
}
```

These stabilize a subset of the following tracking issues:

- rust-lang#27709
- rust-lang#76205

Stabilization of all methods under the `ip` gate was attempted once at rust-lang#66584 then again at rust-lang#76098. These were not successful because there are still unknowns about `is_documentation` `is_benchmarking` and similar; `to_canonical` is much more straightforward.

I have looked and could not find any known issues with `to_canonical`. These were added in 2021 in rust-lang#87708

cc implementor ``@the8472``

r? libs-api
``@rustbot`` label +T-libs-api +needs-fcp
jstasiak added a commit to jstasiak/rust-libs-team that referenced this pull request Dec 15, 2023
There have been no substantial changes in implementation of various
Rust's IpAddr helper methods (is_loopback() etc.) that I can think of.

The PR[1] that the "Rust (New)" implementation was based on has been
closed in May 2021, I think it's safe to say it's no longer relevant
and doesn't help anyone.

[1] rust-lang/rust#76098
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
relnotes Marks issues that should be documented in the release notes of the next release. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.