mirror of
https://github.com/pytorch/pytorch.git
synced 2026-11-20 08:39:27 +08:00
94365ecbd4
Switch pyproject.toml's build backend from setuptools to scikit-build-core (>= 1.0), which invokes CMake directly instead of having setup.py orchestrate it. This PR makes the backend swap plus the 1.0-native configuration it enables; the now-unused setuptools code is removed separately in #180248, so this diff stays focused and is revertible on its own. Where to look, in order of importance: - pyproject.toml: the new [build-system], the scikit-build-core config (the [tool.scikit-build.env] table, cmake defines, wheel layout, editable mode), and the dynamic-metadata hooks. This is the heart of the change. - tools/metadata/{version,dependencies}.py: the providers backing those hooks -- wheel version (from generate_torch_version) and dependencies (base + CUDA/ROCm extras). - CMakeLists.txt: project name/version now come from SKBUILD_* with fallbacks so a standalone `cmake` invocation still works. - torch/__init__.py, torch/_utils_internal.py: import-time shims that keep editable installs working under the new backend. scikit-build-core's redirect finder resolves the extension module itself (`import torch._C`); the shims only cover the places PyTorch locates native artifacts by hand, off `__file__`, which the import system never sees: the `ctypes.CDLL` preload of libtorch_global_deps, the Windows `os.add_dll_directory` for `_C`'s dependent DLLs, and the `bin/` data-file path. Redirect mode is the first layout to split the Python sources (checkout) from the compiled artifacts (install tree); in wheel and `setup.py develop` installs they sat together, so those `dirname(__file__)` lookups happened to be right. - CONTRIBUTING.md: documents the editable-install flow, including scikit-build-core's optional rebuild-on-import (SKBUILD_EDITABLE_REBUILD), which stays off by default. - tools/linter/adapters/codespell_linter.py: works around a codespell bug (it crashes reading a pyproject.toml with an array-of-tables under [tool], which our new [[tool.dynamic-metadata]] introduces) by pointing codespell at the dictionary directly instead of `--toml pyproject.toml`. Reported upstream as codespell-project/codespell#3969. The remainder is mechanical: CI scripts swap `python setup.py clean` for `python -m spin clean` and probe for pyproject.toml instead of setup.py, and the build-requirement files gain scikit-build-core. The configuration uses the scikit-build-core 1.0 feature set, which keeps several PyTorch-specific workarounds out of the tree: - MAX_JOBS -> CMAKE_BUILD_PARALLEL_LEVEL is aliased in the [tool.scikit-build.env] table (added upstream in scikit-build-core#1377 for exactly this), so no custom PEP 517 wrapper backend is needed -- the backend is plain scikit_build_core.build. - Dynamic version/dependencies use the stable [[tool.dynamic-metadata]] table (no experimental flag); the dependencies provider marks deps Dynamic in the sdist via a dynamic_wheel hook. - Build type comes from the environment: 1.0 honors CMAKE_BUILD_TYPE natively, so only the DEBUG / REL_WITH_DEB_INFO env-var overrides remain. - Per-image Linux compiler handling is CC/CXX = { env = ... } in the env table, which opts out of scikit-build-core's sysconfig-compiler injection (documented behavior since #1377) and lets CMake detect the compiler from PATH -- what green setup.py builds always did -- replacing two per-image cmake.define overrides. - The Windows Ninja generator is cmake.define.CMAKE_GENERATOR in a win32 override (1.0's generator-resolution chain honors it), so pyproject is the single source of truth and a plain editable Windows build gets Ninja too; the CI scripts no longer export CMAKE_GENERATOR. The build-system floor and requirements-build.txt are scikit-build-core >= 1.0; minimum-version pins the schema to that floor; and the CI pin (.ci/docker/requirements-ci.txt) plus the Windows build-dep installers are == 1.0.0. scikit-build-core 1.0.0 released 2026-07-06. ## Test plan Wheel content-equivalent to the setuptools build (#180243), fresh cp312 CPU artifacts: - [x] **Wheel size**: 193,274,760 B vs #180243 193,263,155 B (+0.006 %). - [x] **Manifest diff**: adds only 6 license files and removes the setuptools-specific `top_level.txt`; no layout spillage. ``` diff <(unzip -Z1 243.whl | sort) <(unzip -Z1 247.whl | sort) ``` - [x] **METADATA / version.py** identical bar `git_version` and the extra License-File entries; exactly one `_C.cpython-312.so`; aotriton payload intact (10,612 `aotriton.images/` kernels across 6 arches). - [x] **Build time**: no regression; manywheel builds faster (the gain is packaging, not compilation). Clean-build + editable import against the released scikit-build-core 1.0.0: - [x] Linux (qgpu3, CUDA op), macOS (qmac, MPS), Windows (win11 VM): fresh `pip install -e . --no-build-isolation` builds and `import torch` resolves the CMake-built native artifacts from the install location. The Windows run also confirmed the Ninja generator comes purely from pyproject (no CMAKE_GENERATOR env var; build.ninja present, no .sln). - [x] lintrunner -a clean (two pre-existing torch.bcomplex32 PYREFLY errors in torch/storage.py + torch/testing/_creation.py are on trunk, untouched). Re-validated on this PR's CI (carries ciflow/trunk): the dropped per-image Linux compiler overrides, exercised by the anaconda (conda-gcc11, compiler_compat) and aarch64 image builds -- the two cases those overrides originally fixed. Based on a patch by Henry Schreiner (scikit-build-core maintainer); the env table (scikit-build-core#1377) was added upstream specifically to let PyTorch delete its wrapper backend. Co-Generated with [Claude Code](https://claude.com/claude-code) Pull Request resolved: https://github.com/pytorch/pytorch/pull/180247 Approved by: https://github.com/atalman, https://github.com/albanD
13 lines
414 B
Bash
Executable File
13 lines
414 B
Bash
Executable File
set -ex
|
|
uv venv
|
|
source .venv/bin/activate
|
|
uv pip install -r requirements.txt
|
|
uv pip install numpy
|
|
lintrunner init
|
|
git rev-parse HEAD > /tmp/orig_work.txt
|
|
# tools/nightly.py checks out the nightly commit (detached HEAD) and installs
|
|
# the matching pre-built nightly binaries into the repo, avoiding a
|
|
# from-source build.
|
|
python tools/nightly.py checkout -p .venv
|
|
echo "source $PWD/.venv/bin/activate" >> ~/.bashrc
|