Download pre-compiled bootstrap from CI

[dvtkrlbs]

because of a silly rebase mistake i accidently nuked my branch which in turn closed the pr #112598. I recovered the branch using the github repo events and creating a new branch using their API.

r? @ozkanonur

Currently with no changes on the bootstrap directory while doing a pre-push hook. I am getting

extracting /Users/dvtkrlbs/DEV/rust/build/cache/bootstrap-nightly-aarch64-apple-darwin.tar.xz
[Errno 2] No such file or directory: '/Users/dvtkrlbs/DEV/rust/build/aarch64-apple-darwin/stage0/bootstrap-nightly-aarch64-apple-darwin.'

Other than this issue (it triggers a download from ci even if it is not available or trying to extract the wrong file)

[rustbot]

⚠️ Warning ⚠️

• These commits modify submodules.

[rustbot]

Some changes occurred in src/tools/cargo

cc @ehuss

[dvtkrlbs]

Sorry for the wrong cc. I accidentally forgot to reset the stdarch submodule. Fixed now

[dvtkrlbs]

For the last issue for modification change detection I am open to suggestion. I don't really comprehend my current take on the issue and it is really hard to test since there is no easy way to mock git rev-parse.

[rust-log-analyzer]

The job x86_64-gnu-tools failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

GITHUB_ACTION=__run_7
GITHUB_ACTIONS=true
GITHUB_ACTION_REF=
GITHUB_ACTION_REPOSITORY=
GITHUB_ACTOR=dvtkrlbs
GITHUB_API_URL=https://api.github.com
GITHUB_BASE_REF=master
GITHUB_ENV=/home/runner/work/_temp/_runner_file_commands/set_env_18f4a110-0031-4e19-83be-6bb8f9e68121
GITHUB_EVENT_NAME=pull_request
---
GITHUB_SERVER_URL=https://github.com
GITHUB_SHA=af2509094a98e32e9a01fc008bb04db619deda33
GITHUB_STATE=/home/runner/work/_temp/_runner_file_commands/save_state_18f4a110-0031-4e19-83be-6bb8f9e68121
GITHUB_STEP_SUMMARY=/home/runner/work/_temp/_runner_file_commands/step_summary_18f4a110-0031-4e19-83be-6bb8f9e68121
GITHUB_TRIGGERING_ACTOR=dvtkrlbs
GITHUB_WORKFLOW_REF=rust-lang/rust/.github/workflows/ci.yml@refs/pull/115076/merge
GITHUB_WORKFLOW_SHA=af2509094a98e32e9a01fc008bb04db619deda33
GITHUB_WORKSPACE=/home/runner/work/rust/rust
GOROOT_1_18_X64=/opt/hostedtoolcache/go/1.18.10/x64
---
[Errno 2] No such file or directory: '/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bootstrap-nightly-x86_64-unknown-linux-gnu.'
Traceback (most recent call last):
  File "/checkout/src/bootstrap/bootstrap.py", line 1221, in <module>
    main()
  File "/checkout/src/bootstrap/bootstrap.py", line 1205, in main
    bootstrap(args)
  File "/checkout/src/bootstrap/bootstrap.py", line 1180, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/checkout/src/bootstrap/bootstrap.py", line 181, in run
    ret = subprocess.Popen(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/checkout/obj/build/bootstrap/debug/bootstrap'
Command failed. Attempt 2/5:
extracting /checkout/obj/build/cache/bootstrap-nightly-x86_64-unknown-linux-gnu.tar.xz
[Errno 2] No such file or directory: '/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bootstrap-nightly-x86_64-unknown-linux-gnu.'
Traceback (most recent call last):
Traceback (most recent call last):
  File "/checkout/src/bootstrap/bootstrap.py", line 1221, in <module>
    main()
  File "/checkout/src/bootstrap/bootstrap.py", line 1205, in main
    bootstrap(args)
  File "/checkout/src/bootstrap/bootstrap.py", line 1180, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/checkout/src/bootstrap/bootstrap.py", line 181, in run
    ret = subprocess.Popen(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/checkout/obj/build/bootstrap/debug/bootstrap'
Command failed. Attempt 3/5:
extracting /checkout/obj/build/cache/bootstrap-nightly-x86_64-unknown-linux-gnu.tar.xz
[Errno 2] No such file or directory: '/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bootstrap-nightly-x86_64-unknown-linux-gnu.'
Traceback (most recent call last):
Traceback (most recent call last):
  File "/checkout/src/bootstrap/bootstrap.py", line 1221, in <module>
    main()
  File "/checkout/src/bootstrap/bootstrap.py", line 1205, in main
    bootstrap(args)
  File "/checkout/src/bootstrap/bootstrap.py", line 1180, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/checkout/src/bootstrap/bootstrap.py", line 181, in run
    ret = subprocess.Popen(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/checkout/obj/build/bootstrap/debug/bootstrap'
Command failed. Attempt 4/5:
extracting /checkout/obj/build/cache/bootstrap-nightly-x86_64-unknown-linux-gnu.tar.xz
[Errno 2] No such file or directory: '/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bootstrap-nightly-x86_64-unknown-linux-gnu.'
Traceback (most recent call last):
Traceback (most recent call last):
  File "/checkout/src/bootstrap/bootstrap.py", line 1221, in <module>
    main()
  File "/checkout/src/bootstrap/bootstrap.py", line 1205, in main
    bootstrap(args)
  File "/checkout/src/bootstrap/bootstrap.py", line 1180, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/checkout/src/bootstrap/bootstrap.py", line 181, in run
    ret = subprocess.Popen(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/checkout/obj/build/bootstrap/debug/bootstrap'
Command failed. Attempt 5/5:
extracting /checkout/obj/build/cache/bootstrap-nightly-x86_64-unknown-linux-gnu.tar.xz
[Errno 2] No such file or directory: '/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bootstrap-nightly-x86_64-unknown-linux-gnu.'
Traceback (most recent call last):
Traceback (most recent call last):
  File "/checkout/src/bootstrap/bootstrap.py", line 1221, in <module>
    main()
  File "/checkout/src/bootstrap/bootstrap.py", line 1205, in main
    bootstrap(args)
  File "/checkout/src/bootstrap/bootstrap.py", line 1180, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/checkout/src/bootstrap/bootstrap.py", line 181, in run
    ret = subprocess.Popen(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/checkout/obj/build/bootstrap/debug/bootstrap'
The command has failed after 5 attempts.
  local time: Mon Aug 21 20:48:08 UTC 2023
  network time: Mon, 21 Aug 2023 20:48:08 GMT
##[error]Process completed with exit code 1.

[dvtkrlbs]

There also seems to be changes on the component download url format. I am not sure how the bootstrap artifacts fit into those

[onur-ozkan]

I am quite busy and don't have time for mentoring tasks these days. Overally, the flow should include the steps I mentioned on the previous PR #112598 (comment)

As for the catching the modification changes, you can find several references in the codebase(I see I even left a suggestion related to that on #112598 (comment)).

@rustbot author

[bors]

☔ The latest upstream changes (presumably #116260) made this pull request unmergeable. Please resolve the merge conflicts.

[dvtkrlbs]

Sorry for the lack of progress on this. I've been quite busy for a while and couldn't find any time to work on this I am closing this PR in case someone wants to continue feel free to base from my changes.