Associate an allocator to boxes

[glandium]

This turns Box<T> into Box<T, A: Alloc = Global>. This is a
minimalist change to achieve this, not touching anything that could have
backwards incompatible consequences like requiring type annotations in
places where they currently aren't required,
per #50822 (comment)

[rust-highfive]

r? @bluss

(rust_highfive has picked a reviewer for you, use r? to override)

[glandium]

Note this has trivial conflicts with PR #50880, so whichever goes in first will need the other to be rebased on top of it.

[glandium]

The various UI test changes raise the interesting question whether the compiler should "hide" default generic params in error messages.

[glandium]

A few comments now that this is passing tests:

• Since it's now burried in the test reports noise, I'll link to this previous comment, which still applies: Associate an allocator to boxes #50882 (comment)
• I didn't intend for into_raw_and_alloc to be public, but I couldn't find a way to make tests work with it being pub(crate) because the boxed module comes from std in that case.
• into_raw_and_alloc kind of sucks as a name, so please feel free to bikeshed. I was wondering if into_raw should return a (*mut T, A) when A is not Global and *mut T if it is, but I'm not sure it's worth the extra complexity.

[bors]

☔ The latest upstream changes (presumably #50984) made this pull request unmergeable. Please resolve the merge conflicts.

[TimNN]

Triage ping, @bluss / @rust-lang/libs: This PR needs you review, and, I assume, a crater run & FCP.

[alexcrichton]

We've long wanted to do something like this but we've been hesitant to do so historically due to various compiler bugs (which indeed aren't well tracked!). From the looks of the diff here it seems that error messages related to Box may be getting worse? Since I don't think we get a lot of benefit from landing this as-is (as opposed to having a crates.io crate that does this) that to me at least is fairly serious

[glandium]

There is a crates.io crate that does this, but it can't replace Box. It's only by implementing things in liballoc that we end up figuring out what's good and what's bad about it. See #50882 (comment).

[bors]

☔ The latest upstream changes (presumably #50880) made this pull request unmergeable. Please resolve the merge conflicts.

[alexcrichton]

Sure yeah makes sense, but as a PR to rust is this expected to be merged? I'd personally still be wary of the error message segregation

[alexcrichton]

Re, degredation*

[glandium]

As a PR to rust, it allows tests to run. A crater run could be useful, for instance. There are some implementation details that still need to be figured out. For example, the current patch doesn't remove the use of the box keyword, so only Box has fn new, while Box<T, A: Alloc + Default> theoretically could have it.

[pietroalbini]

Ping from triage! How should we move forward with this?

[bors]

☔ The latest upstream changes (presumably #51263) made this pull request unmergeable. Please resolve the merge conflicts.

[Ericson2314]

I'm just still disappointed that unstable methods in an unstable create (std's Box has no alloc param, even on nightly) needs any "serious discussion" at all. The Placer API, for example, affected core APIs, sat unstable for a while, and then was reverted, without harming anyone, right?

I'm reminded of the vision best described by @gnzlbg in rust-lang/rfcs#2480 (comment) on making std a facade over rust-lang-nursery, as it relates to enabling more decentralized library experimentation while still giving the core libs team absolute control over what gets stabilized. Hopefully that gets us the best of both worlds by still keeping experiments out of rust-lang/rust while achieving that goal, but until that happens it would be nice to be more liberal about libraries in this repo provided everything is unstable.

[glandium]

I'll note that it is possible to experiment with crates today, to some extent, and I've done so with the allocator_api crate, that implements Box<T, A> (could implement more, but I didn't need it, contributions welcome).

BUT "to some extent" is key. Because the liballoc exposes types rather than traits, one can't use an hypothetical my_crate::Vec<T, A> in place of a std::vec::Vec<T>. BTW, this is going to be something to figure out long term, even for std::vec::Vec<T, A>, and even for std::box::Box<T, A>, even if this PR was merged.

That being said, this PR is a relatively small step to start experimenting with these things, and, like @Ericson2314, I don't see why this would need to wait.

Well, except for the fact that, at the moment, I'm concerned that it breaks some compiler assumptions and that that would be the reason for the test failure.

[Ericson2314]

Yeah, it's precisely because this experiment will take so long that that it's important to unlock the parallelism of not hogging tons of lib team time.

[alexcrichton]

Yes the libs team understands the value in experimentation and doesn't want to unnecessarily block efforts, but this PR is touching the Box type, which is a change we cannot take lightly. As a result, we need to postpone this until after the 2018 edition to ensure that other features ship.

[TimNN]

Ping from triage! If I understand correctly, this PR is blocked until after the 2018 edition ships, which is still quite some time to go. As such I'm closing this PR as blocked, following our triage procedure. Please add a comment or re-open if I missunderstood.

[phil-opp]

@glandium Do you have plans to reopen this PR now that the 2018 edition has shipped? This would be really great to have.

[Ericson2314]

Yes please! Happy to assist in any way once it is, too.

[glandium]

As long as #52694 is a problem (and it is because travis uses a system llvm that is affected), I can't make progress on this.

[Ericson2314]

Can we cfg that dealloc call and do a static assert with old LLVM that the Alloc parameter is always the global allocator?

[glandium]

Can we?

[glandium]

So...

I tried to gave a quick try to the idea, but I hit two snags:

• This happens:

error[E0378]: the trait `DispatchFromDyn` may only be implemented for structs containing the field being coerced, `PhantomData` fields, and nothing else
   --> src/liballoc/boxed.rs:794:1
    |
794 | impl<T: ?Sized + Unsize<U>, U: ?Sized, A: Alloc> DispatchFromDyn<Box<U, A>> for Box<T, A> {}
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: extra field `1` of type `A` is not allowed

which I don't know what to do about.

• I picked an older rust tree, and tested making box_free Global-only, to possibly put that behind some cfg. Unfortunately, it seems type checking is not happening (I was hoping it would).

With:

pub(crate) unsafe fn box_free<T: ?Sized>(ptr: Unique<T>, mut a: Global)

I can still compile code that uses Box::new_in(value, SomeAllocator), which calls SomeAllocator.alloc for allocations as expected, but Global.dealloc for deallocations, which is not going to do any good at runtime.

[glandium]

error[E0378]: the trait DispatchFromDynmay only be implemented for structs containing the field being coerced,PhantomData fields, and nothing else

That apparently comes from #54383. @mikeyhew, @nikomatsakis, @eddyb, any idea how to get around this?

[Ericson2314]

Thanks does trying! And sorry I didn't respond earlier. I think we can make an A: Is<Global> bound to allow the parameter in all cases, but constrain it to be useless with old LLVM. Maybe that helps with the second point?

Also did you push this experiment yet? I'd love to try it.

[mikeyhew]

@glandium

That apparently comes from #54383. @mikeyhew, @nikomatsakis, @eddyb, any idea how to get around this?

In this PR, you update the definition of Box:

pub struct Box<T: ?Sized, A: Alloc = Global>(Unique<T>, A);

The problem is A isn't necessarily zero-sized. Right now, dynamic dispatch is only implemented for receiver types that have the same ABI as a *const T pointer (Scalar when T is Sized, ScalarPair when T is dyn Trait). If A wasn't zero-sized, this would change the ABI of Box<T, A>, and that's why DispatchFromDyn can only be implemented for newtyped pointers. PhantomData fields are allowed as an exception because the compiler knows that they are zero-sized and do not affect the ABI.

You need to edit this code here: https://github.com/rust-lang/rust/blob/master/src/librustc_typeck/coherence/builtin.rs#L226. There are a couple options:

• A quick and dirty solution: change the if statement to ignore std::alloc::Global in addition to PhantomData fields, and then implement DispatchFromDyn<Box<U, Global>> for Box<T, Global>. This would mean that Box<T, A> where A is not the global allocator would not be object-safe
• Alternatively, you could introduces a new unsafe trait ZeroSized, add an A: ZeroSized bound to Box<T, A>, and update the if statement to ignore anything that implements ZeroSized. The trait could be safe if there were checks to make sure Self is actually zero-sized and doesn't have #[repr(C)]. (And whatever else is required to avoid affecting the ABI. @eddyb can comment on that.)

The real fix will be to support dynamic dispatch for method receivers with arbitrary ABIs, but that's pretty far off. For now it's probably best to use the ZeroSized trait.

[glandium]

Also did you push this experiment yet? I'd love to try it.

I haven't fully rebased, because I was only trying to see if box_free could be made to type-error. Considering the DispatchFromDyn part requires compiler changes anyways, that means there's going to be a need to #[cfg] away the whole thing anyways, for stage0, so it's not going to be more burdensome to add another condition for old-LLVM. The one problem that I have, though, is that currently, only the librustc_llvm crate knows anything about the llvm version...

[Ericson2314]

Ah, what a mess! The one good thing is hopefully Sized<size=0> is approaching possible so a ZeroSized isn't totally a temporary hack.

[eddyb]

At this point it might be easier to support extra fields in DispatchFromDyn, than use some new traits to limit it to size_of::<T>() == 0.

I believe such support could work if built on top of the chalk-oriented "bounded/universe" types, e.g. computing the layout of exists T: Sized. Rc<T, FooAlloc> should be possible, the same way it can work for struct Foo<T>(Rc<T, FooAlloc>), but the compiler could do it without needing a nominal T parameter from anywhere.

This could also be used to represent the type of the data pointer field of DSTs, without overloading the meaning of a *mut dyn Trait TyLayout with a Scalar(Ptr) ABI.

cc @nikomatsakis

[glandium]

So the good news is that #52694 is now "magically" gone. I'm currently in the process of reviving this patch in a different form, working around the DispatchFromDyn issue entirely. I'll open a new PR when I'm done, hopefully in a few hours.

[glandium]

Opened #58457.