Skip to content

Commit

Permalink
Use Resolver in pip sync (#2696)
Browse files Browse the repository at this point in the history
## Summary

This PR removes the custom `DistFinder` that we use in `pip sync`. This
originally existed because `VersionMap` wasn't lazy, and so we saved a
lot of time in `DistFinder` by reading distribution data lazily. But
now, AFAICT, there's really no benefit. Maintaining `DistFinder` means
we effectively have to maintain two resolvers, and end up fixing bugs in
`DistFinder` that don't exist in the `Resolver` (like #2688.

Closes #2694.

Closes #2443.

## Test Plan

I ran this benchmark a bunch. It's basically a wash. Sometimes one is
faster than the other.

```
❯ python -m scripts.bench \
        --uv-path ./target/release/main \
        --uv-path ./target/release/uv \
        scripts/requirements/compiled/trio.txt --min-runs 50 --benchmark install-warm --warmup 25
Benchmark 1: ./target/release/main (install-warm)
  Time (mean ± σ):      54.0 ms ±  10.6 ms    [User: 8.7 ms, System: 98.1 ms]
  Range (min … max):    45.5 ms …  94.3 ms    50 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

Benchmark 2: ./target/release/uv (install-warm)
  Time (mean ± σ):      50.7 ms ±   9.2 ms    [User: 8.7 ms, System: 98.6 ms]
  Range (min … max):    44.0 ms …  98.6 ms    50 runs

  Warning: The first benchmarking run for this command was significantly slower than the rest (77.6 ms). This could be caused by (filesystem) caches that were not filled until after the first run. You should consider using the '--warmup' option to fill those caches before the actual benchmark. Alternatively, use the '--prepare' option to clear the caches before each timing run.

Summary
  './target/release/uv (install-warm)' ran
    1.06 ± 0.29 times faster than './target/release/main (install-warm)'
```
  • Loading branch information
charliermarsh authored Mar 27, 2024
1 parent d41ab0e commit f8fa887
Show file tree
Hide file tree
Showing 9 changed files with 64 additions and 570 deletions.
4 changes: 2 additions & 2 deletions crates/install-wheel-rs/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,9 @@ pub enum Error {
"The .dist-info directory {0} does not consist of the normalized package name and version"
)]
MissingDistInfoSegments(String),
#[error("The .dist-info directory {0} does not start with the normalized package name: {0}")]
#[error("The .dist-info directory {0} does not start with the normalized package name: {1}")]
MissingDistInfoPackageName(String, String),
#[error("The .dist-info directory {0} does not start with the normalized version: {0}")]
#[error("The .dist-info directory {0} does not start with the normalized version: {1}")]
MissingDistInfoVersion(String, String),
#[error("The .dist-info directory name contains invalid characters")]
InvalidDistInfoPrefix,
Expand Down
195 changes: 0 additions & 195 deletions crates/uv-dev/src/install_many.rs

This file was deleted.

6 changes: 0 additions & 6 deletions crates/uv-dev/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ use resolve_many::ResolveManyArgs;
use crate::build::{build, BuildArgs};
use crate::clear_compile::ClearCompileArgs;
use crate::compile::CompileArgs;
use crate::install_many::InstallManyArgs;
use crate::render_benchmarks::RenderBenchmarksArgs;
use crate::resolve_cli::ResolveCliArgs;
use crate::wheel_metadata::WheelMetadataArgs;
Expand All @@ -45,7 +44,6 @@ static GLOBAL: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc;
mod build;
mod clear_compile;
mod compile;
mod install_many;
mod render_benchmarks;
mod resolve_cli;
mod resolve_many;
Expand All @@ -66,7 +64,6 @@ enum Cli {
/// cargo run --bin uv-dev -- resolve-many scripts/popular_packages/pypi_10k_most_dependents.txt
/// ```
ResolveMany(ResolveManyArgs),
InstallMany(InstallManyArgs),
/// Resolve requirements passed on the CLI
Resolve(ResolveCliArgs),
WheelMetadata(WheelMetadataArgs),
Expand All @@ -88,9 +85,6 @@ async fn run() -> Result<()> {
Cli::ResolveMany(args) => {
resolve_many::resolve_many(args).await?;
}
Cli::InstallMany(args) => {
install_many::install_many(args).await?;
}
Cli::Resolve(args) => {
resolve_cli::resolve_cli(args).await?;
}
Expand Down
Loading

0 comments on commit f8fa887

Please sign in to comment.