mirror of
https://github.com/pytorch/pytorch.git
synced 2026-11-18 18:14:04 +08:00
Apply one precedence rule to all forwarded build env vars (#201666)
Go through the following checklist - [x] Passes lint (`spin fixlint`) - [x] Added/updated tests - [x] Updated documentation (if applicable) - [ ] Included benchmark results (for PRs impacting perf) --- ## Linked issue or supporting maintainer Part of #201664 ## BC-breaking? No. One behavior change for existing build directories: a `BLAS`, `MKL_THREADING` or `CUDNN_*` value exported in the shell now re-applies on every build and overwrites a ccmake edit of that entry, as `USE_*` and `BUILD_*` already did since #188242. ## Summary (human written only) This PR homogenizes the treatment of configure options for the Pytorch build. It is part of #201664, implementing most of the necessary changes that are not covered in other PRs or dependent on future changes. ## Agent details (if applicable) Drafted with Claude Code; diff and commit message reviewed by hand. The commit message follows. cmake/EnvVarForwarding.cmake forwarded its variables with two different semantics. Names matched by prefix (BUILD_*, USE_*, CMAKE_*) have overridden the cache on every configure since #188242, so changing USE_CUDA in the environment takes effect on the next build. The names in the passthrough and alias lists (BLAS, TORCH_CUDA_ARCH_LIST, CUDNN_*, MKL_THREADING, ...) still used the original seed-only loop, guarded by NOT DEFINED, so they were written once and then ignored: changing BLAS in the environment did nothing to an existing build directory, and -D beat the environment for these names while losing to it for the prefixed ones. This change routes the passthrough names and all aliases through the same _envfwd_apply the prefix names use, so one rule holds for every knob: environment variables win over -D / cmake.define, those win over pyproject.toml, and all three win over what the build directory remembers. For developers this means a BLAS, MKL_THREADING or CUDNN_* value exported in the shell now re-applies on every build, overwriting a ccmake edit of that entry, exactly as USE_* and BUILD_* already did. The _ENV_ALIASES and _LOW_PRIORITY_ALIASES lists collapse into one, with a single semantics: an alias applies only when its canonical name is not itself in the environment, and where several aliases name the same variable the first listed one wins. Among the aliases, the only observable difference is that a canonical name now beats its alias when both are set (CUDNN_LIBRARY over CUDNN_LIB_DIR); no CI job sets both. As a consequence of going through _envfwd_apply, the passthrough and alias names are now also recorded in env_forwarded.txt, so the native-AOT stage-2 drift check covers them, as its docstring already claimed. The alternative, restoring flags-over-environment for the prefixed names instead, was rejected: scikit-build-core resolves its own and every declared config-setting as environment > -C > default, and a project cannot invert that, so flags-over-environment could not have been delivered uniformly. #200651 binds torch.blas and torch.cuda-arch-list to these names and documents that the bound environment variable takes precedence over the setting; that sentence is only true under this rule, so that PR rebases on this one. The README "Adjust Build Options" and CONTRIBUTING "subsequent builds" paragraphs stated the old seed-only behavior and now state the rule; the EnvVarForwarding header carries it too. Test Plan tools/test/test_env_var_forwarding.py configures a NONE-language CMake project that includes the module and asserts the four tiers for a prefix name and for BLAS (environment over -D, environment over cache, cache kept when the environment is silent, -D over cache), plus canonical-over-alias, alias-over-cache, first-listed-alias-wins and the drift-file recording. Against the unmodified module four of the eight tests fail, exactly the BLAS and alias cases. ``` python -m pytest tools/test/test_env_var_forwarding.py -q lintrunner -a cmake/EnvVarForwarding.cmake tools/test/test_env_var_forwarding.py tools/native_aot/build_stage2.py README.md CONTRIBUTING.md ``` Every commit that touched the module since its introduction was traced to its PR and checked against the new semantics; each CMake-side write to a forwarded variable is either a non-FORCE default or a normal-variable shadow, both of which behave the same under either semantics. Authored with an AI assistant; the diff was reviewed by hand. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Pull Request resolved: https://github.com/pytorch/pytorch/pull/201666 Approved by: https://github.com/atalman
This commit is contained in:
committed by
PyTorch MergeBot
parent
02a1dc067a
commit
8a9a9e737d
+7
-4
@@ -997,10 +997,13 @@ BUILD_CONFIG USE_CUDA=1 pip install --no-build-isolation -v -e .
|
||||
BUILD_CONFIG USE_CUDA=1 USE_DISTRIBUTED=1 pip install --no-build-isolation -v -e .
|
||||
```
|
||||
|
||||
For subsequent builds (i.e., when `build/CMakeCache.txt` exists), the build
|
||||
options passed for the first time will persist; please run `ccmake build/`, run
|
||||
`cmake-gui build/`, or directly edit `build/CMakeCache.txt` to adapt build
|
||||
options.
|
||||
For subsequent builds (i.e., when `build/CMakeCache.txt` exists), an option
|
||||
that is no longer in the environment keeps the value the cache remembers from
|
||||
the first build, while a variable that is in the environment is re-applied on
|
||||
every build. Environment variables win over `-C` flags, `-C` flags win over
|
||||
`pyproject.toml`, and all three win over the cache. To change a cached option
|
||||
without setting its variable, run `ccmake build/`, run `cmake-gui build/`, or
|
||||
directly edit `build/CMakeCache.txt`.
|
||||
|
||||
### Profile build time
|
||||
|
||||
|
||||
@@ -396,10 +396,11 @@ MACOSX_DEPLOYMENT_TARGET=11.0 spin develop
|
||||
|
||||
After a first `spin develop`, you can inspect or adjust the CMake cache interactively with
|
||||
`ccmake build` (or `cmake-gui build`) and rebuild with `spin develop`; your edits persist, because
|
||||
reconfiguration does not use `--fresh` and environment variables only seed cache entries that are
|
||||
not already set. The corollary: once a variable is in the cache, changing its environment variable
|
||||
no longer affects it -- edit the cache directly (or delete `build/CMakeCache.txt`) to change it.
|
||||
The build type and compiler are the exception; the build always re-applies them.
|
||||
reconfiguration does not use `--fresh`. Environment variables win over `-C` flags, `-C` flags win
|
||||
over `pyproject.toml`, and all three win over what the build directory remembers from last time:
|
||||
a forwarded environment variable is re-applied on every build, so a cache edit only survives for
|
||||
variables the environment and the flags leave alone. To change one setting for a single command,
|
||||
put the variable in front of the command.
|
||||
|
||||
### Docker Image
|
||||
|
||||
|
||||
@@ -3,9 +3,16 @@
|
||||
# Forwarding rule: an environment variable reaches CMake (as a cache variable of
|
||||
# the same name) if it (a) starts with BUILD_, USE_, or CMAKE_, (b) ends with
|
||||
# EXITCODE or EXITCODE__TRYRUN_OUTPUT, or (c) appears in the _ENV_ALIASES /
|
||||
# _ENV_PASSTHROUGH / _LOW_PRIORITY_ALIASES lists below. Anything else is not
|
||||
# _ENV_PASSTHROUGH lists below. Anything else is not
|
||||
# forwarded -- set it as a CMake option with -D / cmake.define instead.
|
||||
#
|
||||
# Precedence: a forwarded variable overrides the cache on every configure, so the
|
||||
# environment wins over -D / cmake.define, which win over pyproject.toml, and all
|
||||
# three win over what the build directory remembers from the last configure. To
|
||||
# change one setting for a single command, put the variable in front of the
|
||||
# command. An alias applies only when its canonical name is not in the
|
||||
# environment itself.
|
||||
#
|
||||
# ============================================================================
|
||||
# Build environment variable reference. Vars below that match the forwarding
|
||||
# rule are passed through by this module; the rest are handled where noted.
|
||||
@@ -128,12 +135,6 @@
|
||||
# USE_NINJA select the generator via CMAKE_GENERATOR=Ninja instead
|
||||
# (ninja is the default when available).
|
||||
|
||||
# Additional env vars that are forwarded with a different CMake variable name.
|
||||
set(_ENV_ALIASES
|
||||
"CUDNN_LIB_DIR=CUDNN_LIBRARY"
|
||||
"USE_CUDA_STATIC_LINK=CAFFE2_STATIC_LINK_CUDA"
|
||||
)
|
||||
|
||||
# Additional env vars forwarded with the same name.
|
||||
set(_ENV_PASSTHROUGH
|
||||
UBSAN_FLAGS
|
||||
@@ -172,8 +173,11 @@ set(_ENV_PASSTHROUGH
|
||||
PYTHON_SIX_SOURCE_DIR
|
||||
)
|
||||
|
||||
# Low-priority aliases: if the canonical var is not set, use the alias.
|
||||
set(_LOW_PRIORITY_ALIASES
|
||||
# Env vars forwarded under a different CMake name ("env name=cmake name"). Where
|
||||
# several aliases name the same variable, the first listed one wins.
|
||||
set(_ENV_ALIASES
|
||||
"CUDNN_LIB_DIR=CUDNN_LIBRARY"
|
||||
"USE_CUDA_STATIC_LINK=CAFFE2_STATIC_LINK_CUDA"
|
||||
"CUDA_HOST_COMPILER=CMAKE_CUDA_HOST_COMPILER"
|
||||
"CUDAHOSTCXX=CUDA_HOST_COMPILER"
|
||||
"CMAKE_CUDA_HOST_COMPILER=CUDA_HOST_COMPILER"
|
||||
@@ -181,23 +185,6 @@ set(_LOW_PRIORITY_ALIASES
|
||||
"CUDACXX=CUDA_NVCC_EXECUTABLE"
|
||||
)
|
||||
|
||||
# Forward aliased env vars (env name -> different cmake name)
|
||||
foreach(_alias IN LISTS _ENV_ALIASES)
|
||||
string(REPLACE "=" ";" _parts "${_alias}")
|
||||
list(GET _parts 0 _env_name)
|
||||
list(GET _parts 1 _cmake_name)
|
||||
if(DEFINED ENV{${_env_name}} AND NOT DEFINED ${_cmake_name})
|
||||
set(${_cmake_name} "$ENV{${_env_name}}" CACHE STRING "From env ${_env_name}" FORCE)
|
||||
endif()
|
||||
endforeach()
|
||||
|
||||
# Forward passthrough env vars (same name)
|
||||
foreach(_var IN LISTS _ENV_PASSTHROUGH)
|
||||
if(DEFINED ENV{${_var}} AND NOT DEFINED ${_var})
|
||||
set(${_var} "$ENV{${_var}}" CACHE STRING "From env ${_var}" FORCE)
|
||||
endif()
|
||||
endforeach()
|
||||
|
||||
# Forward all BUILD_*, USE_*, CMAKE_* environment variables (plus names ending
|
||||
# in EXITCODE / EXITCODE__TRYRUN_OUTPUT) into the CMake cache, mirroring the -D
|
||||
# flags setup.py used to pass.
|
||||
@@ -288,14 +275,25 @@ if(NOT _envfwd_rc EQUAL 0)
|
||||
endif()
|
||||
cmake_language(EVAL CODE "${_envfwd_code}")
|
||||
|
||||
# Low-priority aliases
|
||||
foreach(_alias IN LISTS _LOW_PRIORITY_ALIASES)
|
||||
# Passthrough names, same semantics as the prefix names above.
|
||||
foreach(_var IN LISTS _ENV_PASSTHROUGH)
|
||||
if(DEFINED ENV{${_var}})
|
||||
_envfwd_apply("${_var}" "$ENV{${_var}}")
|
||||
endif()
|
||||
endforeach()
|
||||
|
||||
# Aliases apply with the same semantics, but only when the canonical name is not
|
||||
# in the environment itself (the prefix and passthrough passes above already
|
||||
# applied it), and the first listed alias of a name wins over later ones.
|
||||
set(_envfwd_aliased "")
|
||||
foreach(_alias IN LISTS _ENV_ALIASES)
|
||||
string(REPLACE "=" ";" _parts "${_alias}")
|
||||
list(GET _parts 0 _env_name)
|
||||
list(GET _parts 1 _cmake_name)
|
||||
if(DEFINED ENV{${_env_name}} AND NOT DEFINED ${_cmake_name})
|
||||
_envfwd_note("${_cmake_name}")
|
||||
set(${_cmake_name} "$ENV{${_env_name}}" CACHE STRING "From env alias ${_env_name}" FORCE)
|
||||
if(DEFINED ENV{${_env_name}} AND NOT DEFINED ENV{${_cmake_name}}
|
||||
AND NOT "${_cmake_name}" IN_LIST _envfwd_aliased)
|
||||
list(APPEND _envfwd_aliased "${_cmake_name}")
|
||||
_envfwd_apply("${_cmake_name}" "$ENV{${_env_name}}")
|
||||
endif()
|
||||
endforeach()
|
||||
|
||||
|
||||
@@ -153,13 +153,12 @@ def _report_probe_failure(expr: str, stderr: str, returncode: int) -> None:
|
||||
def _arch_list() -> str:
|
||||
"""The arch list THIS build targets, as the CONFIGURE resolved it.
|
||||
|
||||
Read from the file cmake/Codegen.cmake records, because that is the only place
|
||||
the resolved value exists: EnvVarForwarding forwards the environment into the
|
||||
cache only when the variable is undefined (so -D wins), and Dependencies.cmake
|
||||
then shadows the cache with the environment for the rest of the configure. Either
|
||||
source read alone is therefore wrong for one of the two kinds of build. Reading
|
||||
the record also means a developer who changes the environment without
|
||||
reconfiguring gets the list the build COMPILED for, not the one they now intend.
|
||||
Read from the file cmake/Codegen.cmake records, because that is what the
|
||||
configure resolved: EnvVarForwarding FORCEs a set environment variable into the
|
||||
cache (the environment wins over -D), and an unset one leaves the cache value,
|
||||
from -D or an earlier configure, in effect. Reading the record means a developer
|
||||
who changes the environment without reconfiguring gets the list the build
|
||||
COMPILED for, not the one they now intend.
|
||||
|
||||
Recorded-but-empty is not the same as no record: it means the configure resolved
|
||||
no arch list, and the on-device path takes over. Absent means nothing configured
|
||||
@@ -573,10 +572,10 @@ def _refuse_settings_drift(before: dict[str, str] | None) -> None:
|
||||
"""Refuse a reconfigure that changed a setting the environment forwards.
|
||||
|
||||
EnvVarForwarding FORCEs every BUILD_*/USE_*/CMAKE_* environment variable (and the
|
||||
names in its alias lists, TORCH_CUDA_ARCH_LIST among them) into the cache, so a
|
||||
stage-2 run in another environment than the build silently relinks torch_cuda
|
||||
against other settings. Those variables are the whole surface: nothing else here
|
||||
can change the configuration.
|
||||
names in its passthrough and alias lists, TORCH_CUDA_ARCH_LIST among them) into
|
||||
the cache, so a stage-2 run in another environment than the build silently
|
||||
relinks torch_cuda against other settings. Those variables are the whole surface:
|
||||
nothing else here can change the configuration.
|
||||
"""
|
||||
after = _forwarded_settings()
|
||||
if after is None:
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
"""Precedence of cmake/EnvVarForwarding.cmake, measured with a real configure.
|
||||
|
||||
The rule under test: a forwarded environment variable overrides the cache on
|
||||
every configure (environment > -D > cache), for prefix names, passthrough names
|
||||
and aliases alike, with an alias yielding to its canonical name.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
MODULE = REPO / "cmake" / "EnvVarForwarding.cmake"
|
||||
|
||||
PREFIX_NAME = "USE_ENVFWD_PROBE"
|
||||
PASSTHROUGH_NAME = "BLAS"
|
||||
PROBED = [
|
||||
PREFIX_NAME,
|
||||
PASSTHROUGH_NAME,
|
||||
"CUDA_NVCC_EXECUTABLE",
|
||||
"CUDA_HOST_COMPILER",
|
||||
"CMAKE_CUDA_HOST_COMPILER",
|
||||
]
|
||||
# Names the tests set; scrubbed from the ambient environment so a developer's
|
||||
# shell cannot leak into a configure.
|
||||
SCRUBBED = PROBED + ["CUDACXX", "CUDAHOSTCXX", "CMAKE_CUDA_COMPILER"]
|
||||
|
||||
CMAKELISTS = """
|
||||
cmake_minimum_required(VERSION 3.27)
|
||||
project(envfwd NONE)
|
||||
set(Python_EXECUTABLE "{python}")
|
||||
include("{module}")
|
||||
file(WRITE "${{CMAKE_BINARY_DIR}}/resolved.txt" {lines})
|
||||
"""
|
||||
|
||||
|
||||
@unittest.skipUnless(shutil.which("cmake"), "cmake not found")
|
||||
class TestEnvVarForwardingPrecedence(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
tmp = tempfile.TemporaryDirectory(prefix="envfwd-")
|
||||
self.addCleanup(tmp.cleanup)
|
||||
self.tmp = Path(tmp.name)
|
||||
self.src = self.tmp / "src"
|
||||
self.build = self.tmp / "build"
|
||||
self.src.mkdir()
|
||||
lines = " ".join(f'"{n}=${{{n}}}\\n"' for n in PROBED)
|
||||
(self.src / "CMakeLists.txt").write_text(
|
||||
CMAKELISTS.format(
|
||||
python=Path(sys.executable).as_posix(),
|
||||
module=MODULE.as_posix(),
|
||||
lines=lines,
|
||||
)
|
||||
)
|
||||
|
||||
def configure(
|
||||
self, env: dict[str, str] | None = None, defines: dict[str, str] | None = None
|
||||
) -> dict[str, str]:
|
||||
full_env = {k: v for k, v in os.environ.items() if k not in SCRUBBED}
|
||||
full_env.update(env or {})
|
||||
cmd = ["cmake", "-S", str(self.src), "-B", str(self.build)]
|
||||
cmd += [f"-D{k}={v}" for k, v in (defines or {}).items()]
|
||||
subprocess.run(cmd, env=full_env, check=True, capture_output=True, text=True)
|
||||
resolved = (self.build / "resolved.txt").read_text().splitlines()
|
||||
return dict(line.split("=", 1) for line in resolved)
|
||||
|
||||
def forwarded_names(self) -> list[str]:
|
||||
lines = (self.build / "env_forwarded.txt").read_text().splitlines()
|
||||
return [line.split("=", 1)[0] for line in lines]
|
||||
|
||||
def test_environment_wins_over_flag(self) -> None:
|
||||
for name in (PREFIX_NAME, PASSTHROUGH_NAME):
|
||||
with self.subTest(name=name):
|
||||
got = self.configure(env={name: "env"}, defines={name: "flag"})
|
||||
self.assertEqual(got[name], "env")
|
||||
|
||||
def test_environment_reapplies_over_cache(self) -> None:
|
||||
for name in (PREFIX_NAME, PASSTHROUGH_NAME):
|
||||
with self.subTest(name=name):
|
||||
self.configure(env={name: "first"})
|
||||
got = self.configure(env={name: "second"})
|
||||
self.assertEqual(got[name], "second")
|
||||
|
||||
def test_cache_keeps_value_when_environment_is_silent(self) -> None:
|
||||
for name in (PREFIX_NAME, PASSTHROUGH_NAME):
|
||||
with self.subTest(name=name):
|
||||
self.configure(env={name: "first"})
|
||||
got = self.configure()
|
||||
self.assertEqual(got[name], "first")
|
||||
|
||||
def test_flag_updates_cache_when_environment_is_silent(self) -> None:
|
||||
for name in (PREFIX_NAME, PASSTHROUGH_NAME):
|
||||
with self.subTest(name=name):
|
||||
self.configure(env={name: "first"})
|
||||
got = self.configure(defines={name: "flag"})
|
||||
self.assertEqual(got[name], "flag")
|
||||
|
||||
def test_alias_yields_to_canonical_name(self) -> None:
|
||||
got = self.configure(env={"CUDACXX": "alias", "CUDA_NVCC_EXECUTABLE": "canon"})
|
||||
self.assertEqual(got["CUDA_NVCC_EXECUTABLE"], "canon")
|
||||
|
||||
def test_alias_reapplies_over_cache(self) -> None:
|
||||
self.configure(env={"CUDACXX": "first"})
|
||||
got = self.configure(env={"CUDACXX": "second"})
|
||||
self.assertEqual(got["CUDA_NVCC_EXECUTABLE"], "second")
|
||||
|
||||
def test_first_listed_alias_wins(self) -> None:
|
||||
# CUDAHOSTCXX precedes CMAKE_CUDA_HOST_COMPILER in _ENV_ALIASES for
|
||||
# CUDA_HOST_COMPILER; the latter is still forwarded under its own name.
|
||||
env = {"CUDAHOSTCXX": "one", "CMAKE_CUDA_HOST_COMPILER": "two"}
|
||||
got = self.configure(env=env)
|
||||
self.assertEqual(got["CUDA_HOST_COMPILER"], "one")
|
||||
self.assertEqual(got["CMAKE_CUDA_HOST_COMPILER"], "two")
|
||||
|
||||
def test_forwarded_names_are_recorded(self) -> None:
|
||||
self.configure(env={PREFIX_NAME: "x", PASSTHROUGH_NAME: "y", "CUDACXX": "z"})
|
||||
names = self.forwarded_names()
|
||||
self.assertIn(PREFIX_NAME, names)
|
||||
self.assertIn(PASSTHROUGH_NAME, names)
|
||||
self.assertIn("CUDA_NVCC_EXECUTABLE", names)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user