Files
pytorch/codex_setup.sh
T
Klaus Zimmermann 94365ecbd4 Migrate build system from setuptools to scikit-build-core (#180247)
Switch pyproject.toml's build backend from setuptools to
scikit-build-core (>= 1.0), which invokes CMake directly instead of
having setup.py orchestrate it. This PR makes the backend swap plus the
1.0-native configuration it enables; the now-unused setuptools code is
removed separately in #180248, so this diff stays focused and is
revertible on its own.

Where to look, in order of importance:
- pyproject.toml: the new [build-system], the scikit-build-core config
  (the [tool.scikit-build.env] table, cmake defines, wheel layout,
  editable mode), and the dynamic-metadata hooks. This is the heart of
  the change.
- tools/metadata/{version,dependencies}.py: the providers backing those
  hooks -- wheel version (from generate_torch_version) and dependencies
  (base + CUDA/ROCm extras).
- CMakeLists.txt: project name/version now come from SKBUILD_* with
  fallbacks so a standalone `cmake` invocation still works.
- torch/__init__.py, torch/_utils_internal.py: import-time shims that
  keep editable installs working under the new backend. scikit-build-core's
  redirect finder resolves the extension module itself (`import torch._C`);
  the shims only cover the places PyTorch locates native artifacts by hand,
  off `__file__`, which the import system never sees: the `ctypes.CDLL`
  preload of libtorch_global_deps, the Windows `os.add_dll_directory` for
  `_C`'s dependent DLLs, and the `bin/` data-file path. Redirect mode is the
  first layout to split the Python sources (checkout) from the compiled
  artifacts (install tree); in wheel and `setup.py develop` installs they
  sat together, so those `dirname(__file__)` lookups happened to be right.
- CONTRIBUTING.md: documents the editable-install flow, including
  scikit-build-core's optional rebuild-on-import (SKBUILD_EDITABLE_REBUILD),
  which stays off by default.
- tools/linter/adapters/codespell_linter.py: works around a codespell bug (it
  crashes reading a pyproject.toml with an array-of-tables under [tool], which
  our new [[tool.dynamic-metadata]] introduces) by pointing codespell at the
  dictionary directly instead of `--toml pyproject.toml`. Reported upstream as
  codespell-project/codespell#3969.
The remainder is mechanical: CI scripts swap `python setup.py clean` for
`python -m spin clean` and probe for pyproject.toml instead of setup.py,
and the build-requirement files gain scikit-build-core.

The configuration uses the scikit-build-core 1.0 feature set, which keeps
several PyTorch-specific workarounds out of the tree:
- MAX_JOBS -> CMAKE_BUILD_PARALLEL_LEVEL is aliased in the
  [tool.scikit-build.env] table (added upstream in scikit-build-core#1377
  for exactly this), so no custom PEP 517 wrapper backend is needed -- the
  backend is plain scikit_build_core.build.
- Dynamic version/dependencies use the stable [[tool.dynamic-metadata]]
  table (no experimental flag); the dependencies provider marks deps
  Dynamic in the sdist via a dynamic_wheel hook.
- Build type comes from the environment: 1.0 honors CMAKE_BUILD_TYPE
  natively, so only the DEBUG / REL_WITH_DEB_INFO env-var overrides remain.
- Per-image Linux compiler handling is CC/CXX = { env = ... } in the env
  table, which opts out of scikit-build-core's sysconfig-compiler injection
  (documented behavior since #1377) and lets CMake detect the compiler from
  PATH -- what green setup.py builds always did -- replacing two per-image
  cmake.define overrides.
- The Windows Ninja generator is cmake.define.CMAKE_GENERATOR in a win32
  override (1.0's generator-resolution chain honors it), so pyproject is the
  single source of truth and a plain editable Windows build gets Ninja too;
  the CI scripts no longer export CMAKE_GENERATOR.

The build-system floor and requirements-build.txt are scikit-build-core
>= 1.0; minimum-version pins the schema to that floor; and the CI pin
(.ci/docker/requirements-ci.txt) plus the Windows build-dep installers are
== 1.0.0. scikit-build-core 1.0.0 released 2026-07-06.

## Test plan

Wheel content-equivalent to the setuptools build (#180243), fresh cp312
CPU artifacts:
- [x] **Wheel size**: 193,274,760 B vs #180243 193,263,155 B (+0.006 %).
- [x] **Manifest diff**: adds only 6 license files and removes the
  setuptools-specific `top_level.txt`; no layout spillage.
  ```
  diff <(unzip -Z1 243.whl | sort) <(unzip -Z1 247.whl | sort)
  ```
- [x] **METADATA / version.py** identical bar `git_version` and the extra
  License-File entries; exactly one `_C.cpython-312.so`; aotriton payload
  intact (10,612 `aotriton.images/` kernels across 6 arches).
- [x] **Build time**: no regression; manywheel builds faster (the gain is
  packaging, not compilation).

Clean-build + editable import against the released scikit-build-core 1.0.0:
- [x] Linux (qgpu3, CUDA op), macOS (qmac, MPS), Windows (win11 VM): fresh
  `pip install -e . --no-build-isolation` builds and `import torch`
  resolves the CMake-built native artifacts from the install location. The
  Windows run also confirmed the Ninja generator comes purely from
  pyproject (no CMAKE_GENERATOR env var; build.ninja present, no .sln).
- [x] lintrunner -a clean (two pre-existing torch.bcomplex32 PYREFLY errors
  in torch/storage.py + torch/testing/_creation.py are on trunk, untouched).

Re-validated on this PR's CI (carries ciflow/trunk): the dropped per-image
Linux compiler overrides, exercised by the anaconda (conda-gcc11,
compiler_compat) and aarch64 image builds -- the two cases those overrides
originally fixed.

Based on a patch by Henry Schreiner (scikit-build-core maintainer); the env
table (scikit-build-core#1377) was added upstream specifically to let
PyTorch delete its wrapper backend.

Co-Generated with [Claude Code](https://claude.com/claude-code)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/180247
Approved by: https://github.com/atalman, https://github.com/albanD
2026-07-21 15:37:13 +00:00

13 lines
414 B
Bash
Executable File

set -ex
uv venv
source .venv/bin/activate
uv pip install -r requirements.txt
uv pip install numpy
lintrunner init
git rev-parse HEAD > /tmp/orig_work.txt
# tools/nightly.py checks out the nightly commit (detached HEAD) and installs
# the matching pre-built nightly binaries into the repo, avoiding a
# from-source build.
python tools/nightly.py checkout -p .venv
echo "source $PWD/.venv/bin/activate" >> ~/.bashrc