mirror of
https://github.com/pytorch/pytorch.git
synced 2026-11-20 08:39:27 +08:00
[native_aot][7/7] build_stage2: relink, patch the wheel, and the build wiring (#194243)
Human note: Finally putting everything together. Relink libtorch_cuda, correctly fixup the wheel, and make sure everything (paths, manifests, etc.) are all aligned and correct. Claude note: The driver on top of the verdict, and the CI and dev flows that call it. With this commit a standard CUDA build ships embedded AOT kernels. A post-install step, because the kernel builders package-import torch and scikit-build-core has no post-build hook inside the PEP 517 backend. It reconfigures before a targeted relink of torch_cuda: the generated file registers itself in CMAKE_CONFIGURE_DEPENDS only from the reconfigure that first READS it, and the build.ninja on disk predates this generation -- without that the relink silently omitted the kernels (417 MiB where 423 ships). --wheel patches the relinked library and its RECORD entry back into the built wheel, so test jobs get the kernels rather than a library replaced only on disk; released wheels get the same before repair_wheel.py, so it goes through patchelf like any other. It asserts that torch imports, because a kernel-free release wheel must fail rather than skip -- and the ASan and TSan images cannot `import torch` plainly, which is why the CI call is CUDA-guarded. It assumes the torch it finds installed is the one this tree just built, and VERIFIES that rather than repairing it: the reconfigure has to report embedding before anything is relinked, the relinked library has to report its own kernels before this exits, and a failure past that point says to reinstall. A run that DECLINES overwrites a native_aot.cmake an earlier one left behind, because the include is unconditional and the release matrix shares one build tree across eight interpreters -- so the wheel for an interpreter with no DSL runtime was shipping the previous interpreter's objects. caffe2/CMakeLists.txt gets the one include line, EnvVarForwarding.cmake caches TORCH_NATIVE_AOT so the opt-out survives a reconfigure, and CONTRIBUTING.md states the skip reasons. Test Plan: ``` pytest tools/test/test_native_aot_tools.py ``` Run at this commit's own tree, in a checkout with no built torch -- which is what lint.yml's "Test tools" job has. This commit's tests cover patch_wheel's RECORD and its ZIP64 metadata -- a copied member must not inherit the source's stale offsets, which pip and unzip accept and `uv` refuses -- each of main()'s refusals from the verdict down to a relink that produced no library, the argv and env contract with both children including the one artifacts directory they share, and the CI shells and CMake read as text. PR CI cannot reach the relink at this commit -- no declaration exists yet, so should_run() reports "nothing declares kernels" in every build job -- so it was exercised by hand on a clean tree: ``` rm -rf build && USE_CUDNN=1 USE_CUDA=1 USE_MSLK=1 MAX_JOBS=32 spin develop ``` Stage 2 exported all four declarations (54 objects + 4 generated sources, 4.0 MiB), reconfigured, relinked, copied 324 MiB over the installed torch and passed its embedded-kernel probe. On that build: test/python_native, one file per process, 331 passed; `test_ops.py -k "cuda and (topk or scatter_add or index_add)"` 115 passed; test_sort_and_select 207; test_scatter_gather_ops 247; test_indexing 380. Authored with an AI assistant (Claude Code). Pull Request resolved: https://github.com/pytorch/pytorch/pull/194243 Approved by: https://github.com/albanD ghstack dependencies: #190896, #194238, #194239, #194240, #194241, #194242
This commit is contained in:
committed by
PyTorch MergeBot
parent
48a6a67022
commit
adfebb61bc
@@ -27,6 +27,36 @@ case "${GPU_ARCH_TYPE:-BLANK}" in
|
||||
|
||||
cd "${PYTORCH_ROOT}"
|
||||
python3 "${SCRIPTPATH}/build_wheel.py" "${RAW_WHEEL_DIR}"
|
||||
|
||||
# native-AOT stage 2, BEFORE the repair so the relinked libtorch_cuda goes
|
||||
# through patchelf like the rest of the wheel. The kernel builders import
|
||||
# torch, so the raw wheel is installed first.
|
||||
#
|
||||
# CUDA arch types only: this arm also serves cpu/xpu/rocm, where stage 2 can
|
||||
# only skip.
|
||||
if [[ "${GPU_ARCH_TYPE}" == cuda* ]]; then
|
||||
# nullglob so an empty directory counts as zero, not the pattern.
|
||||
naot_wheels=()
|
||||
shopt -s nullglob
|
||||
naot_wheels=("${RAW_WHEEL_DIR}"/*.whl)
|
||||
shopt -u nullglob
|
||||
if [[ ${#naot_wheels[@]} -ne 1 ]]; then
|
||||
echo "native-AOT: expected one raw wheel in ${RAW_WHEEL_DIR}, found ${#naot_wheels[@]}" >&2
|
||||
exit 1
|
||||
fi
|
||||
RAW_WHEEL="${naot_wheels[0]}"
|
||||
# --no-deps --no-index: Requires-Dist would pull GBs of CUDA wheels.
|
||||
python3 -m pip install --progress-bar off --no-deps --no-index "${RAW_WHEEL}"
|
||||
# Sourced for install_cutlass_dsl, so the DSL pin lives in one place.
|
||||
source "${PYTORCH_ROOT}/.ci/pytorch/common_utils.sh"
|
||||
# Stage 2 owns the verdict (see .ci/pytorch/build.sh for why a separate
|
||||
# probe here was wrong).
|
||||
if [[ "$(python3 tools/native_aot/build_stage2.py --print-verdict)" == "RUN" ]]; then
|
||||
install_cutlass_dsl
|
||||
fi
|
||||
python3 tools/native_aot/build_stage2.py --wheel "${RAW_WHEEL}"
|
||||
fi # GPU_ARCH_TYPE == cuda*
|
||||
|
||||
python3 "${SCRIPTPATH}/repair_wheel.py" "${RAW_WHEEL_DIR}" "${PYTORCH_FINAL_PACKAGE_DIR}"
|
||||
;;
|
||||
cpu-s390x)
|
||||
|
||||
@@ -283,6 +283,32 @@ if [[ "$BUILD_ENVIRONMENT" != *libtorch* ]]; then
|
||||
fi
|
||||
pip_install_whl "$(echo dist/*.whl)"
|
||||
|
||||
# native-AOT stage 2: export DSL kernels, relink torch_cuda with them embedded, and
|
||||
# patch the library back into the wheel test jobs get (tools/native_aot/build_stage2.py).
|
||||
#
|
||||
# CUDA-only, as in .ci/manywheel/build.sh: --wheel makes stage 2 refuse a torch that
|
||||
# does not import, and in the ASan and TSan images `import torch` cannot work.
|
||||
if [[ "$BUILD_ENVIRONMENT" == *cuda* ]]; then
|
||||
# Installed HERE, not in .ci/docker/requirements-ci.txt, which every image
|
||||
# shares: that would put ~190 MB of CUDA-only tooling into the CPU/ROCm/XPU images.
|
||||
#
|
||||
# ONE owner of the decision: stage 2 prints the verdict, we install only on RUN.
|
||||
if [[ "$(python tools/native_aot/build_stage2.py --print-verdict)" == "RUN" ]]; then
|
||||
install_cutlass_dsl
|
||||
fi
|
||||
# One wheel expected; a stale second would be glued into one argument. nullglob
|
||||
# so an empty dist/ counts as zero.
|
||||
naot_wheels=()
|
||||
shopt -s nullglob
|
||||
naot_wheels=(dist/*.whl)
|
||||
shopt -u nullglob
|
||||
if [[ ${#naot_wheels[@]} -ne 1 ]]; then
|
||||
echo "native-AOT: expected exactly one wheel in dist/, found ${#naot_wheels[@]}" >&2
|
||||
exit 1
|
||||
fi
|
||||
python tools/native_aot/build_stage2.py --wheel "${naot_wheels[0]}"
|
||||
fi # BUILD_ENVIRONMENT == *cuda*
|
||||
|
||||
# Smoke-test tools/build_with_debinfo.py against the real build tree: it must
|
||||
# still emit a debug-rebuild plan with a -g compile and the libtorch_python
|
||||
# relink. This guards against build-system changes (e.g. a new
|
||||
|
||||
@@ -572,6 +572,14 @@ def _pip_install_cmd(editable):
|
||||
return cmd + [".", "-v", "--no-build-isolation"]
|
||||
|
||||
|
||||
def _native_aot_stage2():
|
||||
# Post-install: the kernel builders import the installed torch, and
|
||||
# scikit-build-core has no post-build hook inside the PEP 517 backend. Skips when
|
||||
# AOT does not apply, but a machine with an exportable GPU needs the DSL wheels
|
||||
# installed, or TORCH_NATIVE_AOT=0.
|
||||
spin.util.run([sys.executable, "tools/native_aot/build_stage2.py"])
|
||||
|
||||
|
||||
@click.command()
|
||||
def develop():
|
||||
"""Build PyTorch (editable install).
|
||||
@@ -582,6 +590,7 @@ def develop():
|
||||
CMakeLists.txt and the supported env vars in cmake/EnvVarForwarding.cmake.
|
||||
"""
|
||||
spin.util.run(_pip_install_cmd(editable=True))
|
||||
_native_aot_stage2()
|
||||
|
||||
|
||||
# Alias so `spin editable` also works.
|
||||
@@ -599,6 +608,7 @@ def install():
|
||||
CMakeLists.txt and the supported env vars in cmake/EnvVarForwarding.cmake.
|
||||
"""
|
||||
spin.util.run(_pip_install_cmd(editable=False))
|
||||
_native_aot_stage2()
|
||||
|
||||
|
||||
PYREFLY_LINTER_SCRIPT = CWD / "tools" / "linter" / "adapters" / "pyrefly_linter.py"
|
||||
|
||||
@@ -819,6 +819,7 @@ On the initial build, you can also speed things up by disabling the features you
|
||||
- `USE_PYTORCH_QNNPACK=0` will disable PyTorch's internal QNNPACK quantized kernels.
|
||||
- `USE_CPU_VECTORIZATION=0` will disable building vectorized CPU kernel variants (AVX2, AVX512, VSX, ZVECTOR, SVE). Only the scalar DEFAULT kernels are built. Fine for correctness/dispatch work; not for CPU benchmarking.
|
||||
- `USE_COLORIZE_OUTPUT=1` will colorize compiler output for easier reading.
|
||||
- `TORCH_NATIVE_AOT=0` will disable the native-AOT stage-2 step (exporting the DSL kernels and embedding them into `libtorch_cuda`; see `tools/native_aot/build_stage2.py`, whose module docstring lists these in the order they are checked). Stage 2 already skips itself when the platform is not Linux, when the built torch does not import or was built without CUDA, when no toolchain targets this backend, when CUDA is older than 13 or cannot be determined, when the interpreter has no published DSL wheel and none is installed, when `BUILD_SHARED_LIBS=OFF` leaves a static `torch_cuda` that cannot take the version script, when nothing declares kernels, and when no supported arch is targeted -- note that with `TORCH_CUDA_ARCH_LIST` unset it exports for whatever GPU is present, so a machine with a supported GPU does not hit that last one. Once it decides it *will* export, a missing DSL wheel is a hard error rather than a skip, so this is the switch to use when you want a build without the DSL toolchain installed.
|
||||
|
||||
The full list of build environment variables, what each one does, and how it reaches CMake is
|
||||
documented at the top of [`cmake/EnvVarForwarding.cmake`](./cmake/EnvVarForwarding.cmake).
|
||||
|
||||
@@ -1038,6 +1038,11 @@ elseif(USE_CUDA)
|
||||
torch_compile_options(torch_cuda) # see cmake/public/utils.cmake
|
||||
target_compile_definitions(torch_cuda PRIVATE USE_CUDA)
|
||||
|
||||
# native-AOT: embed the AOT-exported DSL kernels into torch_cuda. ONE include and no
|
||||
# logic here: tools/native_aot/gen_aot_lib.py decides everything and emits this file.
|
||||
# OPTIONAL, so a tree that embeds nothing does nothing.
|
||||
include("${CMAKE_BINARY_DIR}/native_aot/native_aot.cmake" OPTIONAL)
|
||||
|
||||
if(USE_CUFILE)
|
||||
target_link_libraries(torch_cuda PRIVATE torch::cufile)
|
||||
target_compile_definitions(torch_cuda PRIVATE USE_CUFILE)
|
||||
|
||||
@@ -26,6 +26,11 @@
|
||||
# is set)
|
||||
# USE_CUDA=0, BUILD_TEST=0, ... feature toggles, next section
|
||||
# TORCH_CUDA_ARCH_LIST CUDA arches to build for, e.g. "8.0;9.0"
|
||||
# TORCH_NATIVE_AOT=0 skip the native-AOT stage-2 step (export DSL
|
||||
# kernels, relink them into libtorch_cuda). Honored by
|
||||
# both tools/native_aot/build_stage2.py and the
|
||||
# generated build/native_aot/native_aot.cmake. Not
|
||||
# forwarded: both sides read the environment directly
|
||||
#
|
||||
# Feature toggles (USE_*/BUILD_*, forwarded by prefix):
|
||||
# USE_CUDA=0 disables CUDA build
|
||||
|
||||
@@ -3,10 +3,16 @@ generate the stub sources, and relink torch_cuda with them embedded.
|
||||
|
||||
A post-install step because the kernel builders package-import torch, and
|
||||
scikit-build-core has no post-build hook inside the PEP 517 backend (the wheel
|
||||
is assembled before torch is importable).
|
||||
is assembled before torch is importable):
|
||||
|
||||
* CI: .ci/pytorch/build.sh, after installing the built wheel, with --wheel so
|
||||
the relinked library is patched back into it before test jobs get it
|
||||
* dev: chained by `spin develop` / `spin install`; after a raw
|
||||
`pip install -e .`, run `python tools/native_aot/build_stage2.py`
|
||||
|
||||
Skips -- leaving a normal artifacts-free build -- when AOT kernels are not
|
||||
applicable. Keep this list in sync with should_run(), which reports each one:
|
||||
applicable. Keep this list in sync with should_run(), which reports each one,
|
||||
and with CONTRIBUTING.md, which states them for users:
|
||||
|
||||
* TORCH_NATIVE_AOT=0 (the environment, else the CMake cache)
|
||||
* not Linux: everything downstream is ELF
|
||||
@@ -28,13 +34,18 @@ Two modes, split by the skip list.
|
||||
|
||||
TORCH_NATIVE_AOT=0 opts out, and every entry point honours it; it is expected to
|
||||
stay constant across a build's phases.
|
||||
|
||||
Assumes the torch it finds installed is the one this tree just built, which every
|
||||
caller above arranges, and verifies that rather than repairing it.
|
||||
"""
|
||||
|
||||
import glob
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import sysconfig
|
||||
import zipfile
|
||||
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
@@ -44,6 +55,9 @@ BUILD_DIR = os.path.join(REPO, "build")
|
||||
# directory: a contract with cmake/Codegen.cmake, so the path is spelled once.
|
||||
# Joined onto BUILD_DIR at call time, like _cmake_cache_value, so a test can move it.
|
||||
ARCH_LIST_RECORD = os.path.join("native_aot", "arch_list.txt")
|
||||
# Where caffe2/CMakeLists.txt include()s native_aot.cmake from, as
|
||||
# ${CMAKE_BINARY_DIR}/native_aot: a change on either side has to move both.
|
||||
NATIVE_AOT_ARTIFACTS_DIR = os.path.join(BUILD_DIR, "native_aot")
|
||||
# See export.py: as a script sys.path[0] is this directory, so the repo root
|
||||
# has to go on the path for `tools.native_aot` to import from any cwd. Appended,
|
||||
# not inserted, so the source torch/ tree never shadows the installed wheel.
|
||||
@@ -521,3 +535,465 @@ def _backend() -> str:
|
||||
""" "cuda" or "rocm", as torch reports it. backends.cuda.is_built() is True on
|
||||
ROCm too, so torch.version.hip is the discriminator."""
|
||||
return "rocm" if _torch_probe("torch.version.hip is not None") else "cuda"
|
||||
|
||||
|
||||
def _cache_entries() -> dict[str, tuple[str, str]]:
|
||||
"""This build's cache as name -> (TYPE=value, the doc line above it).
|
||||
|
||||
Last assignment wins, as in _cmake_cache_value. The doc line names the source of an
|
||||
entry EnvVarForwarding wrote ("From environment", "From env <NAME>").
|
||||
"""
|
||||
entries: dict[str, tuple[str, str]] = {}
|
||||
doc = ""
|
||||
try:
|
||||
with open(os.path.join(BUILD_DIR, "CMakeCache.txt"), errors="replace") as f:
|
||||
for line in f:
|
||||
line = line.rstrip("\n")
|
||||
if line.startswith("//"):
|
||||
doc = line[2:]
|
||||
continue
|
||||
key, sep, value = line.partition("=")
|
||||
if sep and not line.startswith("#") and ":" in key:
|
||||
name, _, kind = key.partition(":")
|
||||
entries[name] = (f"{kind}={value}", doc)
|
||||
doc = ""
|
||||
except OSError:
|
||||
return {}
|
||||
return entries
|
||||
|
||||
|
||||
# The doc lines EnvVarForwarding writes above an entry it sourced from the environment.
|
||||
_ENV_SOURCED = ("From environment", "From env ")
|
||||
|
||||
|
||||
def _refuse_cache_drift(before: dict[str, tuple[str, str]]) -> None:
|
||||
"""Refuse a reconfigure that changed this build's configuration.
|
||||
|
||||
EnvVarForwarding writes every BUILD_*/USE_*/CMAKE_* environment variable (and the
|
||||
names in its alias lists, TORCH_CUDA_ARCH_LIST among them) into the cache when it
|
||||
differs from the cached value, so a stage-2 run in another environment than the
|
||||
build silently relinks torch_cuda against other settings. It CREATES the entry when
|
||||
the build had none, so an added env-sourced one is drift too; additions CMake makes
|
||||
itself are not.
|
||||
"""
|
||||
drift = []
|
||||
for name, (value, doc) in _cache_entries().items():
|
||||
if name in before:
|
||||
if before[name][0] != value:
|
||||
drift.append((name, before[name][0], value, doc))
|
||||
elif doc.startswith(_ENV_SOURCED):
|
||||
drift.append((name, "absent", value, doc))
|
||||
if not drift:
|
||||
return
|
||||
changed = "\n".join(
|
||||
f" {name}: {old} -> {new}" + (f" ({doc})" if doc else "")
|
||||
for name, old, new, doc in drift
|
||||
)
|
||||
raise RuntimeError(
|
||||
f"native-AOT stage 2: reconfiguring {BUILD_DIR} changed this build's "
|
||||
f"configuration, so torch_cuda would be relinked against different settings "
|
||||
f"than the rest of this install:\n{changed}\n"
|
||||
f"Run stage 2 in the same environment as the build (the CI shells and "
|
||||
f"`spin develop` do), or re-run the build with the settings you want."
|
||||
)
|
||||
|
||||
|
||||
def _lib_snapshot() -> dict[str, tuple[int, int]]:
|
||||
"""(mtime, size) for every file in the build tree's lib/ except torch_cuda's.
|
||||
|
||||
Stage 2 installs libtorch_cuda.so and nothing else, so these are exactly the
|
||||
libraries a relink may rebuild but this script will not ship.
|
||||
"""
|
||||
lib = os.path.join(BUILD_DIR, "lib")
|
||||
out = {}
|
||||
with os.scandir(lib) as entries:
|
||||
for e in entries:
|
||||
if e.name == "libtorch_cuda.so" or not e.is_file(follow_symlinks=False):
|
||||
continue
|
||||
st = e.stat(follow_symlinks=False)
|
||||
out[e.name] = (st.st_mtime_ns, st.st_size)
|
||||
return out
|
||||
|
||||
|
||||
def _refuse_unshipped_rebuilds(before: dict[str, tuple[int, int]]) -> None:
|
||||
"""Refuse a relink that rebuilt a library stage 2 does not install.
|
||||
|
||||
`--target torch_cuda` builds its DEPENDENCIES too, so a source edited since the
|
||||
last full build lands in libtorch_cpu.so (or c10) as well -- and only
|
||||
libtorch_cuda.so is copied over the installed torch, leaving the two mismatched.
|
||||
Every supported caller builds everything immediately before this, so a difference
|
||||
here means stage 2 was run by hand against an edited tree.
|
||||
"""
|
||||
after = _lib_snapshot()
|
||||
rebuilt = sorted(n for n, v in after.items() if before.get(n, v) != v)
|
||||
if not rebuilt:
|
||||
return
|
||||
raise RuntimeError(
|
||||
f"native-AOT stage 2: relinking torch_cuda also rebuilt "
|
||||
f"{', '.join(rebuilt[:4])}{' and others' if len(rebuilt) > 4 else ''}, which "
|
||||
f"stage 2 does not install -- the tree has sources newer than the last full "
|
||||
f"build, and installing libtorch_cuda.so alone would leave the two "
|
||||
f"mismatched. Re-run the build (`spin develop`, or `pip install -e .` "
|
||||
f"followed by this script), which installs them together."
|
||||
)
|
||||
|
||||
|
||||
def _cmake_for_this_build() -> str:
|
||||
"""The cmake that configured this build tree, else cmake from PATH.
|
||||
|
||||
Recorded as CMAKE_COMMAND, and not necessarily on PATH: a build uses whatever cmake
|
||||
its environment had, often a pip wheel's rather than /usr/local/bin/cmake.
|
||||
"""
|
||||
cached = _cmake_cache_value("CMAKE_COMMAND")
|
||||
if cached and os.path.exists(cached):
|
||||
return cached
|
||||
if cached:
|
||||
_report(f"{cached} configured this build but is gone; using cmake from PATH")
|
||||
return "cmake"
|
||||
|
||||
|
||||
def _run_child(cmd: list[str], what: str, **kw: object) -> None:
|
||||
"""Run one of stage 2's own steps, naming the step and a signal death on failure,
|
||||
neither of which check_call reports. The child's own output is inherited."""
|
||||
code = subprocess.call(cmd, **kw) # type: ignore[arg-type]
|
||||
if code == 0:
|
||||
return
|
||||
raise RuntimeError(
|
||||
f"native-AOT stage 2: {what} "
|
||||
+ (f"was killed by signal {-code}" if code < 0 else f"exited {code}")
|
||||
+ " (its output is above). Command: "
|
||||
+ " ".join(cmd)
|
||||
)
|
||||
|
||||
|
||||
def _installed_lib_dir() -> str:
|
||||
"""The lib/ directory of the INSTALLED torch package.
|
||||
|
||||
Anchored on the compiled _C extension, not torch.__file__: an editable install
|
||||
serves Python from the source tree while the artifacts live in site-packages."""
|
||||
code = (
|
||||
"import importlib.util, os\n"
|
||||
"spec = importlib.util.find_spec('torch._C')\n"
|
||||
"if spec is not None and spec.origin:\n"
|
||||
" print('NAOT_VALUE:' + os.path.dirname(spec.origin), flush=True)\n"
|
||||
)
|
||||
# find_spec imports the PARENT package, so this is a full `import torch`: bounded
|
||||
# and guarded through _run_probe like every other one here.
|
||||
probe = _run_probe(code, "find_spec('torch._C')")
|
||||
if probe is not None:
|
||||
# Marked rather than taking stdout whole: this is joined into a path, so any
|
||||
# other line the child writes (a warning, a site hook) becomes part of it.
|
||||
for line in probe.stdout.splitlines():
|
||||
if line.startswith("NAOT_VALUE:"):
|
||||
return os.path.join(line[len("NAOT_VALUE:") :], "lib")
|
||||
_report_probe_failure("find_spec('torch._C')", probe.stderr, probe.returncode)
|
||||
why = f"exited {probe.returncode} having printed no path" + (
|
||||
" (its stderr is above)" if probe.stderr.strip() else " and nothing at all"
|
||||
)
|
||||
else:
|
||||
why = "could not be run at all (reported above)"
|
||||
raise RuntimeError(
|
||||
f"native-AOT stage 2: cannot locate the installed torch._C, so there is no "
|
||||
f"torch/lib to copy the relinked library into. The probe {why}. Install the "
|
||||
f"wheel from this build first."
|
||||
)
|
||||
|
||||
|
||||
def _copied_member(info: zipfile.ZipInfo) -> zipfile.ZipInfo:
|
||||
"""A ZipInfo for a member copied into a new archive.
|
||||
|
||||
Field by field, not the source's ZipInfo: `info.extra` carries its ZIP64 header
|
||||
offset, so above zipfile's ZIP64_LIMIT (2 GiB) a copy inherits stale offsets."""
|
||||
out = zipfile.ZipInfo(info.filename, info.date_time)
|
||||
out.compress_type = info.compress_type
|
||||
out.external_attr = info.external_attr
|
||||
out.internal_attr = info.internal_attr
|
||||
out.create_system = info.create_system
|
||||
out.comment = info.comment
|
||||
# file_size decides whether open(..., "w") writes a ZIP64 header, so left at 0 it
|
||||
# raises on exactly the members above the limit.
|
||||
out.file_size = info.file_size
|
||||
return out
|
||||
|
||||
|
||||
def _write_hashed(dst: zipfile.ZipFile, info: zipfile.ZipInfo, path: str) -> str:
|
||||
"""Stream `path` into `dst` as `info`; return its RECORD `sha256=...,size` fields.
|
||||
|
||||
Hashed from the bytes actually archived, in the same pass: a second read would
|
||||
double a ~400 MiB read."""
|
||||
import base64
|
||||
import hashlib
|
||||
|
||||
info.file_size = os.path.getsize(path) # see _copied_member: sizes the header
|
||||
h = hashlib.sha256()
|
||||
size = 0
|
||||
with open(path, "rb") as f, dst.open(info, "w") as d:
|
||||
for chunk in iter(lambda: f.read(1 << 20), b""):
|
||||
h.update(chunk)
|
||||
size += len(chunk)
|
||||
d.write(chunk)
|
||||
digest = base64.urlsafe_b64encode(h.digest()).rstrip(b"=").decode()
|
||||
return f"sha256={digest},{size}"
|
||||
|
||||
|
||||
def patch_wheel(wheel_path: str, lib_path: str) -> None:
|
||||
"""Replace torch/lib/libtorch_cuda.so inside an already-built wheel and fix its
|
||||
RECORD entry.
|
||||
|
||||
zipfile, not the zip CLI (absent from the manywheel images) or `wheel pack`, which
|
||||
emits invalid ZIP64 above 4GB (pytorch#189748).
|
||||
"""
|
||||
|
||||
lib_rel = "torch/lib/libtorch_cuda.so"
|
||||
with zipfile.ZipFile(wheel_path) as zf:
|
||||
names = zf.namelist()
|
||||
records = [n for n in names if n.endswith(".dist-info/RECORD")]
|
||||
if lib_rel not in names or len(records) != 1:
|
||||
raise RuntimeError(f"{wheel_path}: not a torch wheel ({lib_rel}/RECORD)")
|
||||
record_rel = records[0]
|
||||
record_lines = zf.read(record_rel).decode().splitlines()
|
||||
|
||||
# Located before a byte is written, so a wheel this cannot patch costs nothing.
|
||||
for lib_line, line in enumerate(record_lines):
|
||||
if line.startswith(lib_rel + ","):
|
||||
break
|
||||
else:
|
||||
raise RuntimeError(f"{wheel_path}: RECORD has no entry for {lib_rel}")
|
||||
|
||||
# Rebuilt beside the original and renamed over it, so an interrupted rewrite
|
||||
# cannot replace a valid wheel.
|
||||
tmp_whl = f"{wheel_path}.naot.{os.getpid()}.tmp"
|
||||
try:
|
||||
with (
|
||||
zipfile.ZipFile(wheel_path) as src,
|
||||
zipfile.ZipFile(tmp_whl, "w", allowZip64=True) as dst,
|
||||
):
|
||||
entry = ""
|
||||
for info in src.infolist():
|
||||
# The RECORD last, once the digest is known; every other member
|
||||
# keeps its position, so .dist-info stays where archivers expect it.
|
||||
if info.filename == record_rel:
|
||||
continue
|
||||
out = _copied_member(info)
|
||||
if info.filename == lib_rel:
|
||||
entry = _write_hashed(dst, out, lib_path)
|
||||
continue
|
||||
# Streamed, not read()+writestr(): several members run to hundreds
|
||||
# of MiB and the read form holds each twice.
|
||||
with src.open(info) as s, dst.open(out, "w") as d:
|
||||
shutil.copyfileobj(s, d)
|
||||
record_lines[lib_line] = f"{lib_rel},{entry}"
|
||||
# The RECORD keeps its original compression: writestr() with a plain
|
||||
# name would take dst's default, which is none, and store it uncompressed.
|
||||
dst.writestr(
|
||||
_copied_member(src.getinfo(record_rel)),
|
||||
"\n".join(record_lines) + "\n",
|
||||
)
|
||||
shutil.move(tmp_whl, wheel_path)
|
||||
finally:
|
||||
if os.path.exists(tmp_whl):
|
||||
os.remove(tmp_whl)
|
||||
|
||||
|
||||
def _invalidate_stale_include() -> None:
|
||||
"""Stop this build tree from embedding what a PREVIOUS run wired up.
|
||||
|
||||
caffe2/CMakeLists.txt include()s the generated file unconditionally, and
|
||||
.ci/manywheel/build_all.sh shares one build/ across eight interpreters. OVERWRITTEN
|
||||
rather than deleted: CMake registers an include()d file as a configure dependency
|
||||
only if it existed at configure time."""
|
||||
from tools.native_aot.gen_aot_lib import CMAKE_INCLUDE, write_nothing_to_embed
|
||||
|
||||
art = NATIVE_AOT_ARTIFACTS_DIR
|
||||
if not os.path.exists(os.path.join(art, CMAKE_INCLUDE)):
|
||||
return
|
||||
write_nothing_to_embed(art)
|
||||
_report(f"a previous run left kernels wired up in {art}; disabled them")
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"--wheel",
|
||||
default=None,
|
||||
help="also embed the relinked libtorch_cuda into this built wheel "
|
||||
"(CI: the dist/*.whl handed to test jobs must carry the kernels)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--print-verdict",
|
||||
action="store_true",
|
||||
help="print RUN or SKIP and exit; the shell callers install the DSL "
|
||||
"runtimes only when this says RUN, so one place owns the decision (their "
|
||||
"own probe disagreed with this one on GPU-less CUDA builders)",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
if args.print_verdict:
|
||||
print("RUN" if should_run() else "SKIP", flush=True)
|
||||
return 0
|
||||
# The explicit opt-out first, and only that: it needs no torch, so
|
||||
# TORCH_NATIVE_AOT=0 is a kill switch even on the binary-build path.
|
||||
if _opted_out():
|
||||
return 0 # _opted_out() reports the value and where it came from
|
||||
# --wheel means torch was installed on the line above, so "not importable" is a
|
||||
# broken build, not "not applicable"; hence ahead of the gates that need torch.
|
||||
if args.wheel and not _torch_probe("True"):
|
||||
raise RuntimeError(
|
||||
"native-AOT stage 2: --wheel was given, so torch was just installed, "
|
||||
"but it does not import (stderr above). Refusing to patch a wheel "
|
||||
"without kernels. Fix the install, or drop --wheel for a build where "
|
||||
"`import torch` cannot work (the ASan and TSan images need LD_PRELOAD "
|
||||
"or a different interpreter). TORCH_NATIVE_AOT=0 also exempts it."
|
||||
)
|
||||
# ...and only then that it exists, so a typo costs a second rather than a full
|
||||
# export, reconfigure, relink and copy.
|
||||
if args.wheel and not os.path.exists(args.wheel):
|
||||
raise RuntimeError(f"native-AOT stage 2: --wheel {args.wheel} does not exist")
|
||||
if not should_run():
|
||||
_invalidate_stale_include()
|
||||
return 0
|
||||
require_runtimes()
|
||||
py = sys.executable
|
||||
art = NATIVE_AOT_ARTIFACTS_DIR
|
||||
_report("exporting kernels")
|
||||
export = [py, os.path.join(HERE, "export.py"), "--out-dir", art]
|
||||
# Read ONCE for both children below, so they cannot be told different arch lists,
|
||||
# and into the export child's environment even if this build only cached it.
|
||||
arch_list = _arch_list()
|
||||
env = dict(os.environ)
|
||||
if arch_list:
|
||||
env["TORCH_CUDA_ARCH_LIST"] = arch_list
|
||||
_run_child(export, "exporting kernels", cwd=REPO, env=env)
|
||||
_report("generating stub sources")
|
||||
gen = [py, os.path.join(HERE, "gen_aot_lib.py"), "--artifacts-dir", art]
|
||||
# The archive the generator names in the CMake it emits.
|
||||
if archive := _dsl_runtime_archive():
|
||||
gen += ["--dsl-runtime", archive]
|
||||
# Name the arches THIS build targets, so a tree left by a build with a different
|
||||
# TORCH_CUDA_ARCH_LIST is ignored. Omitted for an on-device export.
|
||||
if arch_list:
|
||||
from tools.native_aot import export as export_mod
|
||||
|
||||
# Both: --archs filters the trees, --arch-list is the raw value recorded
|
||||
# in the emitted CMake. Only this caller knows they are one request.
|
||||
gen += ["--archs", *export_mod.archs_from_cuda_arch_list(arch_list)]
|
||||
gen += ["--arch-list", arch_list]
|
||||
_run_child(gen, "generating stub sources", cwd=REPO)
|
||||
# Nothing generated is legitimate: no declaration ships kernels for this arch.
|
||||
# Stop rather than relink unchanged and then assert kernels are in it.
|
||||
sources = glob.glob(os.path.join(art, "*", "aot_*.cpp"))
|
||||
if not sources:
|
||||
_report("no declaration ships kernels for this build; nothing embedded")
|
||||
return 0
|
||||
# The count, and the size delta after the relink, rather than parsing the generated
|
||||
# CMake: these bytes scale with declarations x precompile points x arches.
|
||||
_report(f"embedding kernels from {len(sources)} generated source(s)")
|
||||
# Reconfigure explicitly: the generated file registers itself in
|
||||
# CMAKE_CONFIGURE_DEPENDS only from the reconfigure that first reads it.
|
||||
# Keyed on the CACHE, not the directory: `cmake -B` on a missing directory exits 0
|
||||
# and configures FROM SCRATCH, without scikit-build-core's -D flags.
|
||||
if not os.path.exists(os.path.join(BUILD_DIR, "CMakeCache.txt")):
|
||||
raise RuntimeError(
|
||||
f"native-AOT stage 2: {BUILD_DIR} holds no CMakeCache.txt, so it is not "
|
||||
f"the build this torch came from. It must match pyproject.toml's "
|
||||
f"build-dir; re-run the build from the repo root, or set "
|
||||
f"TORCH_NATIVE_AOT=0 to skip stage 2."
|
||||
)
|
||||
_report("reconfiguring to pick up the generated CMake")
|
||||
# The cmake that configured this tree, and its cache as it stands, so the
|
||||
# reconfigure can be held to both (see _refuse_cache_drift).
|
||||
cmake_exe = _cmake_for_this_build()
|
||||
cache_before = _cache_entries()
|
||||
# Captured because CMake prints its failure context on stdout, and the STATUS line
|
||||
# the generated file emits is the only pre-relink evidence that it will embed.
|
||||
# --log-level, because that marker is a message(STATUS) and a cached
|
||||
# CMAKE_MESSAGE_LOG_LEVEL=WARNING (EnvVarForwarding FORCEs it) would hide it.
|
||||
configure = subprocess.run(
|
||||
[cmake_exe, "--log-level=STATUS", "-S", REPO, "-B", BUILD_DIR],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
if configure.returncode != 0:
|
||||
print(configure.stdout.rstrip(), file=sys.stderr, flush=True)
|
||||
print(configure.stderr.rstrip(), file=sys.stderr, flush=True)
|
||||
raise RuntimeError(
|
||||
f"native-AOT stage 2: reconfiguring {BUILD_DIR} failed (exit "
|
||||
f"{configure.returncode}, output above)."
|
||||
)
|
||||
# Ahead of the relink, so a drifting configure cannot reach the installed torch.
|
||||
_refuse_cache_drift(cache_before)
|
||||
from tools.native_aot.gen_aot_lib import EMBED_STATUS
|
||||
|
||||
if EMBED_STATUS not in configure.stdout:
|
||||
raise RuntimeError(
|
||||
f"native-AOT stage 2: generation wrote "
|
||||
f"{os.path.join(art, 'native_aot.cmake')}, but the reconfigure did not "
|
||||
f"report embedding it. The build has declined the kernels this run just "
|
||||
f"exported -- most often TORCH_NATIVE_AOT is falsy in "
|
||||
f"{BUILD_DIR}/CMakeCache.txt (a -D from an earlier configure) while stage "
|
||||
f"2 was told to run. Reconfigure with -DTORCH_NATIVE_AOT=1, or set "
|
||||
f"TORCH_NATIVE_AOT=0 to skip stage 2 entirely."
|
||||
)
|
||||
# Relink JUST torch_cuda: `--target install` walks the whole manifest (~15 min).
|
||||
# The copy below matches it only because the CMake sets BUILD_WITH_INSTALL_RPATH.
|
||||
_report("relinking torch_cuda with embedded kernels")
|
||||
# pyproject.toml aliases MAX_JOBS only inside scikit-build-core's subprocess.
|
||||
relink = [cmake_exe, "--build", ".", "--target", "torch_cuda"]
|
||||
jobs = os.getenv("MAX_JOBS") or os.getenv("CMAKE_BUILD_PARALLEL_LEVEL")
|
||||
if jobs and jobs.isdigit():
|
||||
relink += ["--parallel", jobs]
|
||||
build_lib = os.path.join(BUILD_DIR, "lib", "libtorch_cuda.so")
|
||||
# Taken across the relink, for the size delta reported below.
|
||||
before = os.path.getsize(build_lib) if os.path.exists(build_lib) else 0
|
||||
siblings = _lib_snapshot()
|
||||
_run_child(relink, "relinking torch_cuda", cwd=BUILD_DIR)
|
||||
# Before the copy, so a mismatched pair never reaches the installed torch.
|
||||
_refuse_unshipped_rebuilds(siblings)
|
||||
|
||||
if not os.path.exists(build_lib):
|
||||
raise RuntimeError(f"expected relinked library at {build_lib}")
|
||||
installed = os.path.join(_installed_lib_dir(), "libtorch_cuda.so")
|
||||
if not os.path.exists(installed):
|
||||
# Refuse to create it: _installed_lib_dir found *a* torch, so a layout that
|
||||
# never held the library means we are pointed at the wrong environment.
|
||||
raise RuntimeError(
|
||||
f"native-AOT stage 2: {installed} does not exist, so the torch on "
|
||||
f"sys.path is not the one this tree built. Install the wheel from "
|
||||
f"this build first, or set TORCH_NATIVE_AOT=0."
|
||||
)
|
||||
# Temp file + rename: copying in place truncates a library others may be mapping,
|
||||
# and one os.replace means `installed` never stops existing.
|
||||
staged = f"{installed}.naot.{os.getpid()}.tmp"
|
||||
try:
|
||||
shutil.copy2(build_lib, staged)
|
||||
os.replace(staged, installed)
|
||||
finally:
|
||||
if os.path.exists(staged):
|
||||
os.remove(staged)
|
||||
grew = (os.path.getsize(build_lib) - before) >> 20
|
||||
_report(
|
||||
f"{os.path.getsize(build_lib) >> 20} MiB relinked into {installed} "
|
||||
f"({grew:+d} MiB of embedded kernels)"
|
||||
)
|
||||
# Size is not evidence: this script's artifacts dir must agree with the one
|
||||
# caffe2/CMakeLists.txt include()s, or the relink embeds nothing and still exits 0.
|
||||
if not _torch_probe("torch._native._native_aot_embedded()"):
|
||||
raise RuntimeError(
|
||||
"native-AOT stage 2: relinked libtorch_cuda reports no embedded "
|
||||
"kernels (torch._native._native_aot_embedded() is False). The "
|
||||
f"artifacts in {art} were not linked in -- check that this build's "
|
||||
f"CMAKE_BINARY_DIR is {BUILD_DIR} (it must match pyproject.toml's "
|
||||
f"build-dir), and that {art}/native_aot.cmake exists and is the file "
|
||||
f"caffe2/CMakeLists.txt includes. {installed} now holds that library, "
|
||||
f"so reinstall the wheel from this build to get back to a known state."
|
||||
)
|
||||
if args.wheel:
|
||||
_report(f"embedding into {args.wheel}")
|
||||
patch_wheel(args.wheel, build_lib)
|
||||
_report("done")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
||||
@@ -648,7 +648,17 @@ def _cmake_str(value: str) -> str:
|
||||
|
||||
def _write_atomic(path: str, text: str) -> str:
|
||||
"""Write via a temp file + rename, so a crash cannot leave a partial file
|
||||
that CMake would read as authoritative."""
|
||||
that CMake would read as authoritative.
|
||||
|
||||
Identical content is left alone: these files are build inputs, and restamping them
|
||||
recompiles the generated sources and relinks torch_cuda on every run, which dirties
|
||||
every target that consumes it."""
|
||||
try:
|
||||
with open(path) as f:
|
||||
if f.read() == text:
|
||||
return path
|
||||
except OSError:
|
||||
pass
|
||||
# The pid in the name, not just the target's: nothing locks the artifacts dir, so
|
||||
# two generations in flight (a hand stage-2 run beside `spin develop`) shared one
|
||||
# staging file -- the second truncated it under the first, and the first published
|
||||
@@ -960,7 +970,15 @@ def main(argv: list[str] | None = None) -> None:
|
||||
# write_nothing_to_embed states, and what the other two invalidation sites do.
|
||||
# The nothing-to-embed form names no sources, so it invalidates as completely as
|
||||
# unlinking without dropping CMake's configure dependency.
|
||||
if os.path.exists(os.path.join(args.artifacts_dir, CMAKE_INCLUDE)):
|
||||
include = os.path.join(args.artifacts_dir, CMAKE_INCLUDE)
|
||||
original = None
|
||||
if os.path.exists(include):
|
||||
# Kept so the timestamp can be restored below: this write and the final one
|
||||
# both differ from what is on disk, so the pair restamps the include even when
|
||||
# generation changes nothing, and it is a configure dependency.
|
||||
st = os.stat(include)
|
||||
with open(include) as f:
|
||||
original = (f.read(), st.st_atime_ns, st.st_mtime_ns)
|
||||
write_nothing_to_embed(args.artifacts_dir)
|
||||
|
||||
# decl_id -> its arch dirs, so one declaration generates once however many
|
||||
@@ -1219,9 +1237,18 @@ def main(argv: list[str] | None = None) -> None:
|
||||
print(f"wrote {out} ({n} kernels)")
|
||||
ver = write_version_script(args.artifacts_dir, all_prefixes)
|
||||
print(f"wrote {ver}")
|
||||
print(
|
||||
f"wrote {write_cmake_include(args.artifacts_dir, [p[1] for p in pending], link_objects, ver, args.dsl_runtime, args.arch_list)}"
|
||||
written = write_cmake_include(
|
||||
args.artifacts_dir,
|
||||
[p[1] for p in pending],
|
||||
link_objects,
|
||||
ver,
|
||||
args.dsl_runtime,
|
||||
args.arch_list,
|
||||
)
|
||||
print(f"wrote {written}")
|
||||
with open(written) as f:
|
||||
if original is not None and f.read() == original[0]:
|
||||
os.utime(written, ns=original[1:])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user