Only include metadata for non-dynamic libraries in rustc-dev #105609

bjorn3 · 2022-12-12T12:54:48Z

The actual object code should be linked from librustc_driver.so, which is still included in rustc-dev. This saves on download time and disk usage.

Fixes #103538

rustbot · 2022-12-12T12:54:54Z

r? @jyn514

(rustbot has picked a reviewer for you, use r? to override)

bjorn3 · 2022-12-12T12:54:58Z

@bors try

bors · 2022-12-12T12:55:07Z

⌛ Trying commit 1103d021d3c253ab0885497995fb4d9c06d3eb96 with merge 143447c08dc7c029d9b9f4c5905aabb7aa218e68...

bors · 2022-12-12T15:35:32Z

💔 Test failed - checks-actions

bjorn3 · 2022-12-12T15:43:10Z

@bors try

bors · 2022-12-12T15:43:18Z

⌛ Trying commit 6662d1eb4992f351ab434dc52eb9be245d366bbf with merge 68167fc36cc9c280dd8942dc97107334abaee4aa...

bors · 2022-12-12T18:32:06Z

💔 Test failed - checks-actions

bjorn3 · 2022-12-12T19:37:41Z

@bors try

bors · 2022-12-12T19:37:50Z

⌛ Trying commit 61bc7a640cc5b2f0bbe55274910419c3cdb37ab0 with merge 6d5b237d87d74959bd7608dfc370ef9feda36b2c...

bors · 2022-12-12T22:30:24Z

☀️ Try build successful - checks-actions
Build commit: 6d5b237d87d74959bd7608dfc370ef9feda36b2c (6d5b237d87d74959bd7608dfc370ef9feda36b2c)

bjorn3 · 2022-12-13T10:43:17Z

Works as intended. It cuts both download and disk size in half.

Before

detecting the channel of the `37d7de337903a558dbeb1e82c844fe915ab8ff25` toolchain...
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/37d7de337903a558dbeb1e82c844fe915ab8ff25/rustc-nightly-x86_64-unknown-linux-gnu.tar.xz>...
67.75 MB / 67.75 MB [==================================================================================================================================================] 100.00 % 13.27 MB/s 
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/37d7de337903a558dbeb1e82c844fe915ab8ff25/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz>...
220.29 MB / 220.29 MB [================================================================================================================================================] 100.00 % 12.68 MB/s 
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/37d7de337903a558dbeb1e82c844fe915ab8ff25/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz>...
29.74 MB / 29.74 MB [==================================================================================================================================================] 100.00 % 12.33 MB/s 
toolchain `37d7de337903a558dbeb1e82c844fe915ab8ff25` is successfully installed!

$ du -sh ~/.rustup/toolchains/37d7de337903a558dbeb1e82c844fe915ab8ff25/lib/rustlib/x86_64-unknown-linux-gnu/lib/
1.6G    /home/gh-bjorn3/.rustup/toolchains/37d7de337903a558dbeb1e82c844fe915ab8ff25/lib/rustlib/x86_64-unknown-linux-gnu/lib/

After

detecting the channel of the `6d5b237d87d74959bd7608dfc370ef9feda36b2c` toolchain...
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/6d5b237d87d74959bd7608dfc370ef9feda36b2c/rustc-nightly-x86_64-unknown-linux-gnu.tar.xz>...
67.71 MB / 67.71 MB [==================================================================================================================================================] 100.00 % 11.51 MB/s 
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/6d5b237d87d74959bd7608dfc370ef9feda36b2c/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz>...
106.88 MB / 106.88 MB [=================================================================================================================================================] 100.00 % 6.31 MB/s 
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/6d5b237d87d74959bd7608dfc370ef9feda36b2c/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz>...
29.73 MB / 29.73 MB [===================================================================================================================================================] 100.00 % 9.17 MB/s 
toolchain `6d5b237d87d74959bd7608dfc370ef9feda36b2c` is successfully installed!

$ du -sh ~/.rustup/toolchains/6d5b237d87d74959bd7608dfc370ef9feda36b2c/lib/rustlib/x86_64-unknown-linux-gnu/lib/
729M    /home/gh-bjorn3/.rustup/toolchains/6d5b237d87d74959bd7608dfc370ef9feda36b2c/lib/rustlib/x86_64-unknown-linux-gnu/lib/

rustbot · 2022-12-13T11:31:15Z

rustc_error_messages was changed

cc @davidtwco, @compiler-errors, @JohnTitor, @estebank, @TaKO8Ki

Some changes occurred in src/tools/rustfmt

cc @rust-lang/rustfmt

bjorn3 · 2022-12-13T11:31:26Z

@rustbot ready

rustbot · 2022-12-13T11:58:11Z

The Miri subtree was changed

cc @rust-lang/miri

The actual object code should be linked from librustc_driver.so, which is still included in rustc-dev. This saves on download time and disk usage.

bjorn3 · 2022-12-31T17:21:12Z

Done. There is a CI failure that doesn't make any sense though.

jyn514 · 2023-01-02T23:09:15Z

@bors r+

bors · 2023-01-02T23:09:17Z

📌 Commit 8c000a6 has been approved by jyn514

It is now in the queue for this repository.

bors · 2023-01-03T08:05:58Z

⌛ Testing commit 8c000a6 with merge 3b1c8a9...

bors · 2023-01-03T11:05:59Z

☀️ Test successful - checks-actions
Approved by: jyn514
Pushing 3b1c8a9 to master...

rust-timer · 2023-01-03T12:24:46Z

Finished benchmarking commit (3b1c8a9): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.3%	[4.3%, 4.3%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.1%	[-1.1%, -1.1%]	1
All ❌✅ (primary)	-	-	0

Cycles

This benchmark run did not return any relevant results for this metric.

dtolnay · 2023-01-04T03:47:16Z

For anyone landing here with an error like this:

error: crate `rustc_ast_pretty` required to be available in rlib format, but was not found in this form
  |
  = help: try adding `extern crate rustc_driver;` at the top level of this crate

If you add extern crate rustc_driver and then get an error like this:

error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "/tmp/rustcJ6ZtcA/symbols.o" "target/debug/deps/foo-71b8be8abb28cf36.y793e40jtmbzz9q.rcgu.o" "-Wl,--as-needed" "-L" "target/debug/deps" "-L" ".rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "-L" ".rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bdynamic" "-lrustc_driver-eaaa85fc498d717c" "-L" ".rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-lstd-62c4894b82797b30" "-Wl,-Bstatic" ".rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-66b9c3ae5ff29c13.rlib" "-Wl,-Bdynamic" "-lLLVM-15-rust-1.68.0-nightly" "-ldl" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" ".rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro,-znow" "-nodefaultlibs"
  = note: /usr/bin/ld: cannot find -lLLVM-15-rust-1.68.0-nightly: No such file or directory
          collect2: error: ld returned 1 exit status

then you need to do rustup component add llvm-tools.

Only include metadata for non-dynamic libraries in rustc-dev The actual object code should be linked from librustc_driver.so, which is still included in rustc-dev. This saves on download time and disk usage. Fixes rust-lang#103538

Without this patch rustfmt installs successfully but upon running it it throws the following error $ rustfmt dyld[68147]: Library not loaded: @rpath/librustc_driver-c577224f5690b349.dylib Referenced from: <no uuid> /nix/store/w46inc6cad6xg2kw09v9n629iqb7aqrw-rustfmt-1.69.0/bin/rustfmt Reason: tried: '/nix/store/ih18sy1kcbl5kdgq2l8l1lm70ai6aybj-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/librustc_driver-c577224f5690b349.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/nix/store/ih18sy1kcbl5kdgq2l8l1lm70ai6aybj-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/librustc_driver-c577224f5690b349.dylib' (no such file), '/nix/store/ih18sy1kcbl5kdgq2l8l1lm70ai6aybj-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/librustc_driver-c577224f5690b349.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/nix/store/ih18sy1kcbl5kdgq2l8l1lm70ai6aybj-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/librustc_driver-c577224f5690b349.dylib' (no such file), '/usr/local/lib/librustc_driver-c577224f5690b349.dylib' (no such file), '/usr/lib/librustc_driver-c577224f5690b349.dylib' (no such file, not in dyld cache) Abort trap: 6 Reading this [rust-issue] it seems that we have to link against rustc_driver. The [clippy.nix] expression already does something similar Of the 4 executables found in the result of building rustfmt only rustfmt and git-rustfmt needed to be linked. The other worked without linking to rustc_driver. [rust-issue]: rust-lang/rust#105609 [clippy.nix]: https://github.com/NixOS/nixpkgs/blob/c8cf570dae9b41f30395b71f9b432418b4ff0ebc/pkgs/development/compilers/rust/clippy.nix#L27-L36

Reading this [rust-issue] it seems that we have to link against rustc_driver. The [clippy.nix] expression already does something similar Of the 4 executables found in the result of building rustfmt only rustfmt and git-rustfmt needed to be linked. The other worked without linking to rustc_driver. [rust-issue]: rust-lang/rust#105609 [clippy.nix]: https://github.com/NixOS/nixpkgs/blob/c8cf570dae9b41f30395b71f9b432418b4ff0ebc/pkgs/development/compilers/rust/clippy.nix#L27-L36

bjorn3 added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Dec 12, 2022

rustbot assigned jyn514 Dec 12, 2022

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 12, 2022

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 12, 2022

This comment has been minimized.

Sign in to view

rustbot added A-translation Area: Translation infrastructure, and migrating existing diagnostics to SessionDiagnostic T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Dec 13, 2022

bjorn3 marked this pull request as ready for review December 13, 2022 11:31

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Dec 13, 2022

bjorn3 mentioned this pull request Dec 13, 2022

rustc-dev component recently became a lot bigger #103538

Closed

bjorn3 added 3 commits December 31, 2022 17:20

Only include metadata for non-dynamic libraries in rustc-dev

5c84f76

The actual object code should be linked from librustc_driver.so, which is still included in rustc-dev. This saves on download time and disk usage.

Add help for the error message when missing rustc_driver

7837058

Add missing extern crate rustc_driver

8c000a6

bjorn3 force-pushed the shrink_rustc_dev branch from c098a8b to 8c000a6 Compare December 31, 2022 17:20

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 2, 2023

bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 3, 2023

bors merged commit 3b1c8a9 into rust-lang:master Jan 3, 2023

rustbot added this to the 1.68.0 milestone Jan 3, 2023

bjorn3 deleted the shrink_rustc_dev branch January 3, 2023 11:08

This was referenced Jan 3, 2023

Revert "bootstrap: Get rid of tail_args in stream_cargo" #106387

Merged

Make x.py clippy download and use beta clippy #106394

Closed

ehuss mentioned this pull request Jan 4, 2023

test: revive nightly plugin tests to work rust-lang/cargo#11534

Merged

bjorn3 mentioned this pull request Jan 25, 2023

Bump bootstrap compiler to 1.68 #107297

Merged

jyn514 mentioned this pull request Feb 2, 2023

Update rustfmt path rust-lang/rustc-dev-guide#1574

Merged

nilehmann mentioned this pull request Feb 9, 2023

Update toolchain flux-rs/flux#361

Merged

PuercoPop mentioned this pull request May 10, 2023

rustfmt: fix on darwin NixOS/nixpkgs#231078

Merged

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Only include metadata for non-dynamic libraries in rustc-dev #105609

Only include metadata for non-dynamic libraries in rustc-dev #105609

bjorn3 commented Dec 12, 2022

rustbot commented Dec 12, 2022

bjorn3 commented Dec 12, 2022

bors commented Dec 12, 2022

bors commented Dec 12, 2022

bjorn3 commented Dec 12, 2022

bors commented Dec 12, 2022

bors commented Dec 12, 2022

bjorn3 commented Dec 12, 2022

bors commented Dec 12, 2022

bors commented Dec 12, 2022

This comment has been minimized.

bjorn3 commented Dec 13, 2022

This comment has been minimized.

rustbot commented Dec 13, 2022

bjorn3 commented Dec 13, 2022

rustbot commented Dec 13, 2022

bjorn3 commented Dec 31, 2022

jyn514 commented Jan 2, 2023

bors commented Jan 2, 2023

bors commented Jan 3, 2023

bors commented Jan 3, 2023

rust-timer commented Jan 3, 2023

dtolnay commented Jan 4, 2023

Only include metadata for non-dynamic libraries in rustc-dev #105609

Only include metadata for non-dynamic libraries in rustc-dev #105609

Conversation

bjorn3 commented Dec 12, 2022

rustbot commented Dec 12, 2022

bjorn3 commented Dec 12, 2022

bors commented Dec 12, 2022

bors commented Dec 12, 2022

bjorn3 commented Dec 12, 2022

bors commented Dec 12, 2022

bors commented Dec 12, 2022

bjorn3 commented Dec 12, 2022

bors commented Dec 12, 2022

bors commented Dec 12, 2022

This comment has been minimized.

bjorn3 commented Dec 13, 2022

Before

After

This comment has been minimized.

rustbot commented Dec 13, 2022

bjorn3 commented Dec 13, 2022

rustbot commented Dec 13, 2022

bjorn3 commented Dec 31, 2022

jyn514 commented Jan 2, 2023

bors commented Jan 2, 2023

bors commented Jan 3, 2023

bors commented Jan 3, 2023

rust-timer commented Jan 3, 2023

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

dtolnay commented Jan 4, 2023