bootstrap: add more tracing to compiler/std/llvm flows #137080

jieyouxu · 2025-02-15T15:11:02Z

Add more tracing to compiler/std/llvm flows.
Two drive-by nits:
1. Take TargetSelection by-value for builder.is_builder_target(). Noticed while adding tracing; follow-up to improve host/cross target checking #136767.
2. Coalesce enzyme build logic into one branch.
Document COMPILER{,_FOR} tracing targets for bootstrap: Remove the distinction between compiler and compiler_for #96176.
No functional changes.

Testing

You can play with the tracing locally with:

$ BOOTSTRAP_TRACING=bootstrap=debug ./x build library
$ BOOTSTRAP_TRACING=bootstrap=trace ./x build library
$ BOOTSTRAP_TRACING=bootstrap=trace,COMPILER=trace,COMPILER_FOR=trace ./x build library

Previews

$ BOOTSTRAP_TRACING=bootstrap=debug ./x build library

$ BOOTSTRAP_TRACING=bootstrap=trace,COMPILER=trace,COMPILER_FOR=trace ./x build library

r? @onur-ozkan (or reroll)

rustbot · 2025-02-15T15:11:13Z

The rustc-dev-guide subtree was changed. If this PR only touches the dev guide consider submitting a PR directly to rust-lang/rustc-dev-guide otherwise thank you for updating the dev guide with your changes.

cc @BoxyUwU, @jieyouxu, @Kobzol

This PR changes how LLVM is built. Consider updating src/bootstrap/download-ci-llvm-stamp.

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

jieyouxu · 2025-02-15T15:21:19Z

src/bootstrap/src/core/build_steps/compile.rs

        // Build enzyme
-        let enzyme_install = if builder.config.llvm_enzyme {
-            Some(builder.ensure(llvm::Enzyme { target: build_compiler.host }))
-        } else {
-            None
-        };
-
-        if let Some(enzyme_install) = enzyme_install {
+        if builder.config.llvm_enzyme {
+            debug!("`llvm_enzyme` requested");
+            let enzyme_install = builder.ensure(llvm::Enzyme { target: build_compiler.host });


I collapsed this into one single block, I had to do a double-take on this.

jieyouxu · 2025-02-15T15:22:00Z

src/bootstrap/src/lib.rs

-    fn is_builder_target(&self, target: &TargetSelection) -> bool {
-        &self.config.build == target
+    fn is_builder_target(&self, target: TargetSelection) -> bool {
+        self.config.build == target
    }


We try very hard in bootstrap to keep TargetSelection Copyable, noticed while adding some tracing steps.

I would rather say that we have to live with the fact that TargeSelection is Copyable 😅

jieyouxu · 2025-02-15T15:28:29Z

Ugh, it's because trace! expands to empty w/out tracing feature.

[DO NOT MERGE] `./x test rust-analyzer` I somehow made `./x test rust-analyzer` work on my machine[^machine], **but at what cost?** Not intended for merge but only as a reference. If we do want to land this, I'll need to tidy this up. Notes: - Unrelated tracing bits are extracted to rust-lang#137080. - I abused a bunch of cargo features `in-rust-tree`. It probably doesn't need to be, and simply `--cfg` might work. I was trying to get the main rust-analyzer tests to build *at all*. Anything building is already a miracle. - I had to slap a bunch of the following to all the r-a crates to get the tests to build at all. I don't 100% understand why, but otherwise I get a whole ton of ``expected `rustc_lexer` to be available in rlib format`` build failures. ```rs #![cfg_attr(feature = "in-rust-tree", feature(rustc_private))] #[cfg(all(feature = "in-rust-tree", test))] extern crate rustc_driver as _; ``` - Skipped one config test that was fixed on r-a master but not synced here in r-l/r yet. [^machine]: `x86_64-unknown-linux-gnu`, haven't bothered trying this on msvc yet. try-job: aarch64-gnu try-job: x86_64-apple-1 try-job: aarch64-apple try-job: i686-mingw-1 try-job: x86_64-mingw-1 try-job: i686-msvc-1 try-job: x86_64-msvc-1

onur-ozkan

Seems good to me other than one nit

src/bootstrap/src/core/build_steps/compile.rs

onur-ozkan · 2025-02-16T07:56:08Z

@rustbot author

jieyouxu · 2025-02-16T10:49:36Z

Removed "recursively", no other changes since last review.
@rustbot ready

src/bootstrap/src/core/build_steps/compile.rs

Shourya742

tACk 05ba1a4

onur-ozkan · 2025-02-16T12:00:08Z

@bors r+

bors · 2025-02-16T12:00:10Z

📌 Commit 05ba1a4 has been approved by onur-ozkan

It is now in the queue for this repository.

bootstrap: add more tracing to compiler/std/llvm flows - Add more tracing to compiler/std/llvm flows. - Two drive-by nits: 1. Take `TargetSelection` by-value for `builder.is_builder_target()`. Noticed while adding tracing; follow-up to rust-lang#136767. 2. Coalesce enzyme build logic into one branch. - Document `COMPILER{,_FOR}` tracing targets for rust-lang#96176. - No functional changes. ### Testing You can play with the tracing locally with: ``` $ BOOTSTRAP_TRACING=bootstrap=debug ./x build library $ BOOTSTRAP_TRACING=bootstrap=trace ./x build library $ BOOTSTRAP_TRACING=bootstrap=trace,COMPILER=trace,COMPILER_FOR=trace ./x build library ``` ### Previews ``` $ BOOTSTRAP_TRACING=bootstrap=debug ./x build library ``` ![Screenshot 2025-02-15 230824](https://github.com/user-attachments/assets/c3b02b62-d52e-4c03-a00a-da0d95618989) ``` $ BOOTSTRAP_TRACING=bootstrap=trace,COMPILER=trace,COMPILER_FOR=trace ./x build library ``` ![Screenshot 2025-02-15 233859](https://github.com/user-attachments/assets/842e4ece-4c26-4191-acbb-5f93e42de4dc) r? `@onur-ozkan` (or reroll)

…iaskrgr Rollup of 7 pull requests Successful merges: - rust-lang#136466 (Start removing `rustc_middle::hir::map::Map`) - rust-lang#136671 (Overhaul `rustc_middle::limits`) - rust-lang#136817 (Pattern Migration 2024: clean up and comment) - rust-lang#136844 (Use `const_error!` when possible) - rust-lang#137080 (bootstrap: add more tracing to compiler/std/llvm flows) - rust-lang#137101 (`invalid_from_utf8[_unchecked]`: also lint inherent methods) - rust-lang#137140 (Fix const items not being allowed to be called `r#move` or `r#static`) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#137080 - jieyouxu:more-tracing, r=onur-ozkan bootstrap: add more tracing to compiler/std/llvm flows - Add more tracing to compiler/std/llvm flows. - Two drive-by nits: 1. Take `TargetSelection` by-value for `builder.is_builder_target()`. Noticed while adding tracing; follow-up to rust-lang#136767. 2. Coalesce enzyme build logic into one branch. - Document `COMPILER{,_FOR}` tracing targets for rust-lang#96176. - No functional changes. ### Testing You can play with the tracing locally with: ``` $ BOOTSTRAP_TRACING=bootstrap=debug ./x build library $ BOOTSTRAP_TRACING=bootstrap=trace ./x build library $ BOOTSTRAP_TRACING=bootstrap=trace,COMPILER=trace,COMPILER_FOR=trace ./x build library ``` ### Previews ``` $ BOOTSTRAP_TRACING=bootstrap=debug ./x build library ``` ![Screenshot 2025-02-15 230824](https://github.com/user-attachments/assets/c3b02b62-d52e-4c03-a00a-da0d95618989) ``` $ BOOTSTRAP_TRACING=bootstrap=trace,COMPILER=trace,COMPILER_FOR=trace ./x build library ``` ![Screenshot 2025-02-15 233859](https://github.com/user-attachments/assets/842e4ece-4c26-4191-acbb-5f93e42de4dc) r? ``@onur-ozkan`` (or reroll)

rustbot assigned onur-ozkan Feb 15, 2025

jieyouxu commented Feb 15, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

jieyouxu force-pushed the more-tracing branch 2 times, most recently from a8e6b84 to 087d8eb Compare February 15, 2025 15:32

jieyouxu mentioned this pull request Feb 15, 2025

[DO NOT MERGE] ./x test rust-analyzer #136779

Draft

onur-ozkan approved these changes Feb 16, 2025

View reviewed changes

src/bootstrap/src/core/build_steps/compile.rs Outdated Show resolved Hide resolved

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 16, 2025

jieyouxu added 3 commits February 16, 2025 18:47

bootstrap: add more tracing to compiler/std/llvm flows

b621a48

bootstrap: take target by value in is_builder_target

7b11816

rustc-dev-guide: document COMPILER and COMPILER_FOR tracing targets

05ba1a4

jieyouxu force-pushed the more-tracing branch from 087d8eb to 05ba1a4 Compare February 16, 2025 10:48

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 16, 2025

Shourya742 reviewed Feb 16, 2025

View reviewed changes

src/bootstrap/src/core/build_steps/compile.rs Show resolved Hide resolved

Shourya742 approved these changes Feb 16, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 16, 2025

GuillaumeGomez mentioned this pull request Feb 16, 2025

Rollup of 10 pull requests #137130

Closed

matthiaskrgr mentioned this pull request Feb 17, 2025

Rollup of 7 pull requests #137163

Merged

bors merged commit 5dd48bc into rust-lang:master Feb 17, 2025
6 checks passed

rustbot added this to the 1.87.0 milestone Feb 17, 2025

jieyouxu deleted the more-tracing branch February 17, 2025 11:26

jieyouxu mentioned this pull request Feb 19, 2025

Making compiletest more maintainable: reworking directive handling rust-lang/rust-project-goals#259

Open

5 tasks

lqd mentioned this pull request Feb 27, 2025

Raw comments rust-lang/rust-project-goals#298

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bootstrap: add more tracing to compiler/std/llvm flows #137080

bootstrap: add more tracing to compiler/std/llvm flows #137080

jieyouxu commented Feb 15, 2025 •

edited

Loading

rustbot commented Feb 15, 2025

jieyouxu Feb 15, 2025

jieyouxu Feb 15, 2025

Kobzol Feb 15, 2025

This comment has been minimized.

jieyouxu commented Feb 15, 2025

onur-ozkan left a comment

onur-ozkan commented Feb 16, 2025

jieyouxu commented Feb 16, 2025

Shourya742 left a comment

onur-ozkan commented Feb 16, 2025

bors commented Feb 16, 2025

bootstrap: add more tracing to compiler/std/llvm flows #137080

bootstrap: add more tracing to compiler/std/llvm flows #137080

Conversation

jieyouxu commented Feb 15, 2025 • edited Loading

Testing

Previews

rustbot commented Feb 15, 2025

jieyouxu Feb 15, 2025

Choose a reason for hiding this comment

jieyouxu Feb 15, 2025

Choose a reason for hiding this comment

Kobzol Feb 15, 2025

Choose a reason for hiding this comment

This comment has been minimized.

jieyouxu commented Feb 15, 2025

onur-ozkan left a comment

Choose a reason for hiding this comment

onur-ozkan commented Feb 16, 2025

jieyouxu commented Feb 16, 2025

Shourya742 left a comment

Choose a reason for hiding this comment

onur-ozkan commented Feb 16, 2025

bors commented Feb 16, 2025

jieyouxu commented Feb 15, 2025 •

edited

Loading