Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow running cargo pgo bolt optimize without a profile #14

Merged
merged 1 commit into from
Aug 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,19 +115,20 @@ BOLT is not supported directly by `rustc`, so the instrumentation and optimizati
directly applied to binaries built by `rustc`. Instead, `cargo-pgo` creates additional binaries that
you have to use for gathering profiles and executing the optimized code.

1) **Generate the profiles**

First, you need to generate the BOLT profiles. To do that, execute the following command:
```bash
$ cargo pgo bolt build
```
The instrumented binary will be located at `<target-dir>/<target-triple>/release/<binary-name>-bolt-instrumented`.
Execute it on several workloads to gather as much data as possible.
### Generating the profiles
First, you need to generate the BOLT profiles. To do that, execute the following command:
```bash
$ cargo pgo bolt build
```
The instrumented binary will be located at `<target-dir>/<target-triple>/release/<binary-name>-bolt-instrumented`.
Execute it on several workloads to gather as much data as possible.

2) **Build an optimized binary**
Note that for BOLT, the profile gathering step is optional. You can also simply run the optimization
step (see below) without any profiles, although it will probably not have a large effect.

Once you have generated some profiles, you can execute `cargo pgo bolt optimize` to build an
optimized version of your binary. The optimized binary will be named `<binary-name>-bolt-optimized`.
### Building an optimized binary
Once you have generated some profiles, you can execute `cargo pgo bolt optimize` to build an
optimized version of your binary. The optimized binary will be named `<binary-name>-bolt-optimized`.

## BOLT + PGO
Yes, BOLT and PGO can even be combined :) To do that, you should first generate PGO profiles and
Expand Down
95 changes: 55 additions & 40 deletions src/bolt/optimize.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,22 +51,28 @@ pub fn bolt_optimize(ctx: CargoContext, args: BoltOptimizeArgs) -> anyhow::Resul
);

let profile_dir = get_binary_profile_dir(&bolt_dir, &artifact);
let target_file = profile_dir.join("merged.profdata");
if !merge_profiles(&bolt_env, &profile_dir, &target_file)? {
log::warn!(
"No profiles found for target {}. It will NOT be optimized!",
artifact.target.name.blue()
);
} else {
let optimized_path =
optimize_binary(&bolt_env, &args.bolt_args, executable, &target_file)?;
log::info!(
"{} {} successfully optimized with BOLT. You can find it at {}.",
capitalize(get_artifact_kind(&artifact)).yellow(),
artifact.target.name.blue(),
cli_format_path(&optimized_path.display())
);
}
let optimized_path = match merge_profiles(&bolt_env, &profile_dir)? {
Some(profile_file) => optimize_binary(
&bolt_env,
&args.bolt_args,
executable,
Some(&profile_file),
)?,
None => {
log::warn!(
"No profiles found for target {}. \
The optimization will probably not be very effective.",
artifact.target.name.blue()
);
optimize_binary(&bolt_env, &args.bolt_args, executable, None)?
}
};
log::info!(
"{} {} successfully optimized with BOLT. You can find it at {}.",
capitalize(get_artifact_kind(&artifact)).yellow(),
artifact.target.name.blue(),
cli_format_path(&optimized_path.display())
);
}
}
Message::BuildFinished(res) => {
Expand All @@ -88,13 +94,20 @@ fn optimize_binary(
bolt_env: &BoltEnv,
bolt_args: &BoltArgs,
binary: &Utf8PathBuf,
profile: &Path,
profile: Option<&Path>,
) -> anyhow::Result<PathBuf> {
log::debug!(
"Optimizing {} with BOLT profile {}.",
binary.as_str(),
profile.display()
);
match profile {
Some(profile) => {
log::debug!(
"Optimizing {} with BOLT profile {}.",
binary.as_str(),
profile.display()
);
}
None => {
log::debug!("Optimizing {} without a BOLT profile.", binary.as_str());
}
}

let basename = binary
.as_path()
Expand All @@ -106,16 +119,19 @@ fn optimize_binary(
.expect("Cannot get parent of compiled binary")
.join(format!("{}-bolt-optimized", basename));

let mut args = vec![
binary.to_string(),
"-data".to_string(),
profile
.to_str()
.expect("Could not convert profile path")
.to_string(),
"-o".to_string(),
target_path.to_string(),
];
let mut args = vec![binary.to_string()];

if let Some(profile) = profile {
args.extend([
"-data".to_string(),
profile
.to_str()
.expect("Could not convert profile path")
.to_string(),
]);
}

args.extend(["-o".to_string(), target_path.to_string()]);

match bolt_args.bolt_args {
Some(ref bolt_args) => add_bolt_args(&mut args, bolt_args)?,
Expand Down Expand Up @@ -150,23 +166,22 @@ fn optimize_binary(
Ok(target_path.into_std_path_buf())
}

fn merge_profiles(
bolt_env: &BoltEnv,
profile_dir: &Path,
target_profile: &Path,
) -> anyhow::Result<bool> {
/// Merges BOLT profiles from `profile_dir` and returns a path to the merged profile file,
/// if it was successfully generated.
fn merge_profiles(bolt_env: &BoltEnv, profile_dir: &Path) -> anyhow::Result<Option<PathBuf>> {
let mut command = Command::new(&bolt_env.merge_fdata);

let profile_files = gather_fdata_files(profile_dir);
if profile_files.is_empty() {
return Ok(false);
return Ok(None);
}

for file in profile_files {
command.arg(file);
}

let output_file = File::create(target_profile)?;
let target_profile = profile_dir.join("merged.profdata");
let output_file = File::create(&target_profile)?;
let output_stdio = Stdio::from(output_file);
command.stdout(output_stdio);

Expand All @@ -176,7 +191,7 @@ fn merge_profiles(
"Merged BOLT profile(s) to {}.",
cli_format_path(target_profile.display())
);
Ok(true)
Ok(Some(target_profile))
} else {
Err(anyhow!(
"Failed to merge BOLT profile(s): {}.",
Expand Down
1 change: 1 addition & 0 deletions tests/integration/bolt.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ fn test_bolt_instrument_run_instrumented_binary() -> anyhow::Result<()> {
fn test_bolt_optimize_no_profile() -> anyhow::Result<()> {
let project = init_cargo_project()?;
project.run(&["bolt", "optimize"])?.assert_ok();
assert!(project.bolt_optimized_binary().is_file());

Ok(())
}
Expand Down