Skip to content

Commit

Permalink
Allow customizing the project environment path with `UV_PROJECT_ENVIR…
Browse files Browse the repository at this point in the history
…ONMENT` (#6834)

Allows configuration of the (currently hard-coded) path to the virtual
environment in projects using the `UV_PROJECT_ENVIRONMENT` environment
variable.

If empty, we'll ignore it. If a relative path, it will be resolved
relative to the workspace root. If an absolute path, we'll use that.

This feature targets use in Docker images and CI. The variable is
intended to be set once in an isolated system and used for all uv
operations.

We do not expose a CLI option or configuration file setting — we may
pursue those later but I see them as lower priority. I think a
system-level environment variable addresses the most pressing use-cases
here.

This doesn't special-case the system environment. Which means that you
can use this to write to the system Python environment. I would
generally strongly recommend against doing so. The insightful comment
from @edmorley at
#5229 (comment)
provides some context on why. More generally, `uv sync` will remove
packages from the environment by default. This means that if the system
environment contains any packages relevant to the operation of the
system (that are not dependencies of your project), `uv sync` will break
it. I'd only use this in Docker or CI, if anywhere. Virtual environments
have lots of benefits, and it's only [one line to "activate"
them](https://docs.astral.sh/uv/guides/integration/docker/#using-the-environment).

If you are considering using this feature to use Docker bind mounts for
developing in containers, I would highly recommend reading our [Docker
container development
documentation](https://docs.astral.sh/uv/guides/integration/docker/#developing-in-a-container)
first. If the solutions there do not work for you, please open an issue
describing your use-case and why.

We do not read `VIRTUAL_ENV` and do not have plans to at this time.
Reading `VIRTUAL_ENV` is high-risk, because users can easily leave an
environment active and use the uv project interface today. Reading
`VIRTUAL_ENV` would be a breaking change. Additionally, uv is
intentionally moving away from the concept of "active environments" and
I don't think syncing to an "active" environment is the right behavior
while managing projects. I plan to add a warning if `VIRTUAL_ENV` is
set, to avoid confusion in this area (see
#6864).

This does not directly enable centrally managed virtual environments. If
you set `UV_PROJECT_ENVIRONMENT` to an absolute path and use it across
multiple projects, they will clobber each other's environments. However,
you could use this with something like `direnv` to achieve "centrally
managed" environments. I intend to build a prototype of this eventually.
See #1495 for more details on this use-case.

Lots of discussion about this feature in:

- astral-sh/rye#371
- astral-sh/rye#1222
- astral-sh/rye#1211
- #5229
- #6669
- #6612

Follow-ups:

- #6835 
- #6864
- Document this in the project concept documentation (can probably
re-use some of this post)

Closes #6669
Closes #5229
Closes #6612
  • Loading branch information
zanieb committed Sep 3, 2024
1 parent bc7b6f1 commit 1234b6d
Show file tree
Hide file tree
Showing 4 changed files with 370 additions and 19 deletions.
25 changes: 24 additions & 1 deletion crates/uv-workspace/src/workspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -435,8 +435,31 @@ impl Workspace {
}

/// The path to the workspace virtual environment.
///
/// Uses `.venv` in the install path directory by default.
///
/// If `UV_PROJECT_ENVIRONMENT` is set, it will take precedence. If a relative path is provided,
/// it is resolved relative to the install path.
pub fn venv(&self) -> PathBuf {
self.install_path.join(".venv")
/// Resolve the `UV_PROJECT_ENVIRONMENT` value, if any.
fn from_environment_variable(workspace: &Workspace) -> Option<PathBuf> {
let value = std::env::var_os("UV_PROJECT_ENVIRONMENT")?;

if value.is_empty() {
return None;
};

let path = PathBuf::from(value);
if path.is_absolute() {
return Some(path);
};

// Resolve the path relative to the install path.
Some(workspace.install_path.join(path))
}

// TODO(zanieb): Warn if `VIRTUAL_ENV` is set and does not match
from_environment_variable(self).unwrap_or_else(|| self.install_path.join(".venv"))
}

/// The members of the workspace.
Expand Down
52 changes: 35 additions & 17 deletions crates/uv/tests/common/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -165,31 +165,49 @@ impl TestContext {
self
}

/// Create a new test context with multiple Python versions.
///
/// Does not create a virtual environment by default, but the first Python version
/// can be used to create a virtual environment with [`TestContext::create_venv`].
/// Add extra standard filtering for a given path.
#[must_use]
pub fn with_filtered_path(mut self, path: &Path, name: &str) -> Self {
// Note this is sloppy, ideally we wouldn't push to the front of the `Vec` but we need
// this to come in front of other filters or we can transform the path (e.g., with `[TMP]`)
// before we reach this filter.
for pattern in Self::path_patterns(path)
.into_iter()
.map(|pattern| (pattern, format!("[{name}]/")))
{
self.filters.insert(0, pattern);
}
self
}
/// Discover the path to the XDG state directory. We use this, rather than the OS-specific
/// temporary directory, because on macOS (and Windows on GitHub Actions), they involve
/// symlinks. (On macOS, the temporary directory is, like `/var/...`, which resolves to
/// `/private/var/...`.)
///
/// See [`TestContext::new`] if only a single version is desired.
pub fn new_with_versions(python_versions: &[&str]) -> Self {
// Discover the path to the XDG state directory. We use this, rather than the OS-specific
// temporary directory, because on macOS (and Windows on GitHub Actions), they involve
// symlinks. (On macOS, the temporary directory is, like `/var/...`, which resolves to
// `/private/var/...`.)
//
// It turns out that, at least on macOS, if we pass a symlink as `current_dir`, it gets
// _immediately_ resolved (such that if you call `current_dir` in the running `Command`, it
// returns resolved symlink). This is problematic, as we _don't_ want to resolve symlinks
// for user-provided paths.
let bucket = env::var("UV_INTERNAL__TEST_DIR")
/// It turns out that, at least on macOS, if we pass a symlink as `current_dir`, it gets
/// _immediately_ resolved (such that if you call `current_dir` in the running `Command`, it
/// returns resolved symlink). This is problematic, as we _don't_ want to resolve symlinks
/// for user-provided paths.
pub fn test_bucket_dir() -> PathBuf {
env::var("UV_INTERNAL__TEST_DIR")
.map(PathBuf::from)
.unwrap_or_else(|_| {
etcetera::base_strategy::choose_base_strategy()
.expect("Failed to find base strategy")
.data_dir()
.join("uv")
.join("tests")
});
})
}

/// Create a new test context with multiple Python versions.
///
/// Does not create a virtual environment by default, but the first Python version
/// can be used to create a virtual environment with [`TestContext::create_venv`].
///
/// See [`TestContext::new`] if only a single version is desired.
pub fn new_with_versions(python_versions: &[&str]) -> Self {
let bucket = Self::test_bucket_dir();
fs_err::create_dir_all(&bucket).expect("Failed to create test bucket");

let root = tempfile::TempDir::new_in(bucket).expect("Failed to create test root directory");
Expand Down
266 changes: 265 additions & 1 deletion crates/uv/tests/sync.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@

use anyhow::Result;
use assert_cmd::prelude::*;
use assert_fs::prelude::*;
use assert_fs::{fixture::ChildPath, prelude::*};
use insta::assert_snapshot;

use common::{uv_snapshot, TestContext};
use predicates::prelude::predicate;
use tempfile::tempdir_in;

mod common;

Expand Down Expand Up @@ -1483,3 +1485,265 @@ fn convert_to_package() -> Result<()> {

Ok(())
}

#[test]
fn sync_custom_environment_path() -> Result<()> {
let mut context = TestContext::new("3.12");

let pyproject_toml = context.temp_dir.child("pyproject.toml");
pyproject_toml.write_str(
r#"
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["iniconfig"]
"#,
)?;

// Running `uv sync` should create `.venv` by default
uv_snapshot!(context.filters(), context.sync(), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Resolved 2 packages in [TIME]
Prepared 1 package in [TIME]
Installed 1 package in [TIME]
+ iniconfig==2.0.0
"###);

context
.temp_dir
.child(".venv")
.assert(predicate::path::is_dir());

// Running `uv sync` should create `foo` in the project directory when customized
uv_snapshot!(context.filters(), context.sync().env("UV_PROJECT_ENVIRONMENT", "foo"), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Using Python 3.12.[X] interpreter at: [PYTHON-3.12]
Creating virtualenv at: foo
Resolved 2 packages in [TIME]
Prepared 1 package in [TIME]
Installed 1 package in [TIME]
+ iniconfig==2.0.0
"###);

context
.temp_dir
.child("foo")
.assert(predicate::path::is_dir());

// We don't delete `.venv`, though we arguably could
context
.temp_dir
.child(".venv")
.assert(predicate::path::is_dir());

// An absolute path can be provided
uv_snapshot!(context.filters(), context.sync().env("UV_PROJECT_ENVIRONMENT", "foobar/.venv"), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Using Python 3.12.[X] interpreter at: [PYTHON-3.12]
Creating virtualenv at: foobar/.venv
Resolved 2 packages in [TIME]
Prepared 1 package in [TIME]
Installed 1 package in [TIME]
+ iniconfig==2.0.0
"###);

context
.temp_dir
.child("foobar")
.assert(predicate::path::is_dir());

context
.temp_dir
.child("foobar")
.child(".venv")
.assert(predicate::path::is_dir());

// An absolute path can be provided
uv_snapshot!(context.filters(), context.sync().env("UV_PROJECT_ENVIRONMENT", context.temp_dir.join("bar")), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Using Python 3.12.[X] interpreter at: [PYTHON-3.12]
Creating virtualenv at: bar
Resolved 2 packages in [TIME]
Prepared 1 package in [TIME]
Installed 1 package in [TIME]
+ iniconfig==2.0.0
"###);

context
.temp_dir
.child("bar")
.assert(predicate::path::is_dir());

// And, it can be outside the project
let tempdir = tempdir_in(TestContext::test_bucket_dir())?;
context = context.with_filtered_path(tempdir.path(), "OTHER_TEMPDIR");
uv_snapshot!(context.filters(), context.sync().env("UV_PROJECT_ENVIRONMENT", tempdir.path().join(".venv")), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Using Python 3.12.[X] interpreter at: [PYTHON-3.12]
Creating virtualenv at: [OTHER_TEMPDIR]/.venv
Resolved 2 packages in [TIME]
Prepared 1 package in [TIME]
Installed 1 package in [TIME]
+ iniconfig==2.0.0
"###);

ChildPath::new(tempdir.path())
.child(".venv")
.assert(predicate::path::is_dir());

Ok(())
}

#[test]
fn sync_workspace_custom_environment_path() -> Result<()> {
let context = TestContext::new("3.12");

let pyproject_toml = context.temp_dir.child("pyproject.toml");
pyproject_toml.write_str(
r#"
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["iniconfig"]
"#,
)?;

// Create a workspace member
context.init().arg("child").assert().success();

// Running `uv sync` should create `.venv` in the workspace root
uv_snapshot!(context.filters(), context.sync(), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Resolved 3 packages in [TIME]
Prepared 1 package in [TIME]
Installed 1 package in [TIME]
+ iniconfig==2.0.0
"###);

context
.temp_dir
.child(".venv")
.assert(predicate::path::is_dir());

// Similarly, `uv sync` from the child project uses `.venv` in the workspace root
uv_snapshot!(context.filters(), context.sync().current_dir(context.temp_dir.join("child")), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Resolved 3 packages in [TIME]
Uninstalled 1 package in [TIME]
- iniconfig==2.0.0
"###);

context
.temp_dir
.child(".venv")
.assert(predicate::path::is_dir());

context
.temp_dir
.child("child")
.child(".venv")
.assert(predicate::path::missing());

// Running `uv sync` should create `foo` in the workspace root when customized
uv_snapshot!(context.filters(), context.sync().env("UV_PROJECT_ENVIRONMENT", "foo"), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Using Python 3.12.[X] interpreter at: [PYTHON-3.12]
Creating virtualenv at: foo
Resolved 3 packages in [TIME]
Prepared 1 package in [TIME]
Installed 1 package in [TIME]
+ iniconfig==2.0.0
"###);

context
.temp_dir
.child("foo")
.assert(predicate::path::is_dir());

// We don't delete `.venv`, though we arguably could
context
.temp_dir
.child(".venv")
.assert(predicate::path::is_dir());

// Similarly, `uv sync` from the child project uses `foo` relative to the workspace root
uv_snapshot!(context.filters(), context.sync().env("UV_PROJECT_ENVIRONMENT", "foo").current_dir(context.temp_dir.join("child")), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Resolved 3 packages in [TIME]
Uninstalled 1 package in [TIME]
- iniconfig==2.0.0
"###);

context
.temp_dir
.child("foo")
.assert(predicate::path::is_dir());

context
.temp_dir
.child("child")
.child("foo")
.assert(predicate::path::missing());

// And, `uv sync --package child` uses `foo` relative to the workspace root
uv_snapshot!(context.filters(), context.sync().arg("--package").arg("child").env("UV_PROJECT_ENVIRONMENT", "foo"), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Resolved 3 packages in [TIME]
Audited in [TIME]
"###);

context
.temp_dir
.child("foo")
.assert(predicate::path::is_dir());

context
.temp_dir
.child("child")
.child("foo")
.assert(predicate::path::missing());

Ok(())
}
Loading

1 comment on commit 1234b6d

@akx
Copy link
Contributor

@akx akx commented on 1234b6d Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rare 1234 commit ID! 😄

Please sign in to comment.