### Summary
Replaces the `OpaqueGenerator` Python wrapper from #179658 by making
`torch._C.Generator` itself a proper opaque reference type. This eliminates the
wrapper class entirely.
C++ changes to `THPGenerator`:
- Set `OpaqueBaseMeta` as the metaclass (same pattern as `ProcessGroup`)
- Add weakref support (`tp_weaklistoffset` + `weakreflist` field)
- Add `tp_richcompare` (identity-based `__eq__`/`__ne__`)
- Add `tp_hash` (identity-based)
- Add `__fx_repr__` method for FX codegen
Also fixes `THPModule_addDocStr` to use `PyType_Check(obj)` instead of
`Py_TYPE(obj) == &PyType_Type`, since Generator's metaclass is now
`OpaqueBaseMeta` rather than `type`.
With these changes, `torch._C.Generator` is registered directly via
`register_opaque_type(torch._C.Generator, typ="value")` and flows through
`make_fx`, FX codegen, and the repro infrastructure without any wrapper.
Authored with Claude.
### Test plan
- `python test/dynamo/test_after_aot.py` (all 6 tests)
- `python test/inductor/test_torchinductor.py -k test_randn_generator_cuda`
- `python test/test_opaque_obj_v2.py TestOpaqueGenerator` (make_fx + correctness)
- ` python test/test_torch.py -k test_pickle_generator` (generator can be pickled)
- `python test/dynamo/test_dynamic_shapes.py -k test_user_getattr1_dynamic_shapes`
- `python agent_space/repro.py` (end-to-end repro)
- `python test/test_fx.py -k test_impure_random` (Generator is a C type, it has __init__ without __code__)
- `python test/dynamo/test_misc.py -k test_torch_generator_initial_seed`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/179661
Approved by: https://github.com/angelayi
### Summary
Replaces the `OpaqueGenerator` Python wrapper from #179658 by making
`torch._C.Generator` itself a proper opaque reference type. This eliminates the
wrapper class entirely.
C++ changes to `THPGenerator`:
- Set `OpaqueBaseMeta` as the metaclass (same pattern as `ProcessGroup`)
- Add weakref support (`tp_weaklistoffset` + `weakreflist` field)
- Add `tp_richcompare` (identity-based `__eq__`/`__ne__`)
- Add `tp_hash` (identity-based)
- Add `__fx_repr__` method for FX codegen
Also fixes `THPModule_addDocStr` to use `PyType_Check(obj)` instead of
`Py_TYPE(obj) == &PyType_Type`, since Generator's metaclass is now
`OpaqueBaseMeta` rather than `type`.
With these changes, `torch._C.Generator` is registered directly via
`register_opaque_type(torch._C.Generator, typ="value")` and flows through
`make_fx`, FX codegen, and the repro infrastructure without any wrapper.
Authored with Claude.
### Test plan
- `python test/dynamo/test_after_aot.py` (all 6 tests)
- `python test/inductor/test_torchinductor.py -k test_randn_generator_cuda`
- `python test/test_opaque_obj_v2.py TestOpaqueGenerator` (make_fx + correctness)
- ` python test/test_torch.py -k test_pickle_generator` (generator can be pickled)
- `python test/dynamo/test_dynamic_shapes.py -k test_user_getattr1_dynamic_shapes`
- `python agent_space/repro.py` (end-to-end repro)
- `python test/test_fx.py -k test_impure_random` (Generator is a C type, it has __init__ without __code__)
- `python test/dynamo/test_misc.py -k test_torch_generator_initial_seed`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/179661
Approved by: https://github.com/angelayi
### Summary
Replaces the `OpaqueGenerator` Python wrapper from #179658 by making
`torch._C.Generator` itself a proper opaque reference type. This eliminates the
wrapper class entirely.
C++ changes to `THPGenerator`:
- Set `OpaqueBaseMeta` as the metaclass (same pattern as `ProcessGroup`)
- Add weakref support (`tp_weaklistoffset` + `weakreflist` field)
- Add `tp_richcompare` (identity-based `__eq__`/`__ne__`)
- Add `tp_hash` (identity-based)
- Add `__fx_repr__` method for FX codegen
Also fixes `THPModule_addDocStr` to use `PyType_Check(obj)` instead of
`Py_TYPE(obj) == &PyType_Type`, since Generator's metaclass is now
`OpaqueBaseMeta` rather than `type`.
With these changes, `torch._C.Generator` is registered directly via
`register_opaque_type(torch._C.Generator, typ="value")` and flows through
`make_fx`, FX codegen, and the repro infrastructure without any wrapper.
Authored with Claude.
### Test plan
- `python test/dynamo/test_after_aot.py` (all 6 tests)
- `python test/inductor/test_torchinductor.py -k test_randn_generator_cuda`
- `python test/test_opaque_obj_v2.py TestOpaqueGenerator` (make_fx + correctness)
- ` python test/test_torch.py -k test_pickle_generator` (generator can be pickled)
- `python test/dynamo/test_dynamic_shapes.py -k test_user_getattr1_dynamic_shapes`
- `python agent_space/repro.py` (end-to-end repro)
- `python test/test_fx.py -k test_impure_random` (Generator is a C type, it has __init__ without __code__)
- `python test/dynamo/test_misc.py -k test_torch_generator_initial_seed`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/179661
Approved by: https://github.com/angelayi
This change was separated from #170609, and adds a context flag to select the backend kernel for depthwise convolution. This allows easy benchmarking of cuDNN performance relative to fallback performance for heuristic tuning (and could also be used for debugging), while maintaining default behavior for standard use.
There are three options: `["auto", "cudnn", "native"]`. The `"auto"` option chooses a kernel based on the existing heuristic function. The `"cudnn"` and `"native"` flags skip the heuristic, instead always dispatching to a cuDNN kernel or a fallback kernel, respectively.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/176500
Approved by: https://github.com/malfet, https://github.com/eqy
This adds Python functions to get, set, and reset cuBLAS or cuBLASLt workspace sizes. Workspaces are updated lazily the next time they are accessed if the new workspace size is greater than the current size.
@ngimel, is this what you were hoping for? I believe the current BLAS backend can already be queried with `torch.backends.cuda.preferred_blas_library`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/177912
Approved by: https://github.com/ngimel, https://github.com/malfet, https://github.com/eqy
### Summary
Replaces the `OpaqueGenerator` Python wrapper from #179658 by making
`torch._C.Generator` itself a proper opaque reference type. This eliminates the
wrapper class entirely.
C++ changes to `THPGenerator`:
- Set `OpaqueBaseMeta` as the metaclass (same pattern as `ProcessGroup`)
- Add weakref support (`tp_weaklistoffset` + `weakreflist` field)
- Add `tp_richcompare` (identity-based `__eq__`/`__ne__`)
- Add `tp_hash` (identity-based)
- Add `__fx_repr__` method for FX codegen
Also fixes `THPModule_addDocStr` to use `PyType_Check(obj)` instead of
`Py_TYPE(obj) == &PyType_Type`, since Generator's metaclass is now
`OpaqueBaseMeta` rather than `type`.
With these changes, `torch._C.Generator` is registered directly via
`register_opaque_type(torch._C.Generator, typ="value")` and flows through
`make_fx`, FX codegen, and the repro infrastructure without any wrapper.
Authored with Claude.
### Test plan
- `python test/dynamo/test_after_aot.py` (all 6 tests)
- `python test/inductor/test_torchinductor.py -k test_randn_generator_cuda`
- `python test/test_opaque_obj_v2.py TestOpaqueGenerator` (make_fx + correctness)
- ` python test/test_torch.py -k test_pickle_generator` (generator can be pickled)
- `python test/dynamo/test_dynamic_shapes.py -k test_user_getattr1_dynamic_shapes`
- `python agent_space/repro.py` (end-to-end repro)
- `python test/test_fx.py -k test_impure_random` (Generator is a C type, it has __init__ without __code__)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/179661
Approved by: https://github.com/angelayi
ghstack dependencies: #179657
TorchVitals was an undocumented feature gated behind the TORCH_VITAL
environment variable that allowed setting and reading key-value "vital signs" (e.g. whether CUDA or DataLoader was used). It was never documented and the test itself had a "FIXME: document or deprecate" comment.
Originally added in PR #51047 (Jan 2021), with a Python API in #53238 and a CUDA vital in #58059. There have been no feature changes since June 2021 — all subsequent commits were mechanical cleanups (clang-tidy,
thread-safe getenv, etc.).
Update: `torch.set_vital` was used from TorchData (see https://github.com/meta-pytorch/data/pull/1537 ) so will keep the dummy API around
Authored with Claude.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/178479
Approved by: https://github.com/msaroufim, https://github.com/albanD, https://github.com/mlazos
This PR adds [hipDNN](https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipdnn) convolution support to PyTorch on ROCm, following the newer dispatch mechanism which no longer requires per-backend frontend op registration.
New ROCm kernel providers will be able to register plugins and ship faster kernels through hipDNN without touching PyTorch code. The graph-based API of hipDNN also supports fused kernels. The eventual goal is to replace all MIOpen-specific PyTorch code with hipDNN, wherein MIOpen will be just one of multiple first-class kernel providers.
# Note
This PR is being opened parallel to #177534 in anticipation that batch norm's dispatch migration issues might stall the review process for hipdnn as a whole.
# Summary
- Integrates the lightweight hipDNN frontend graph API as an alternative convolution backend to MIOpen on ROCm, covering forward, backward-data, and backward-weight for 2D/3D convolutions including transposed variants.
- Adds cmake `USE_HIPDNN` detection, `torch.backends.hipdnn` Python module with `enabled`/`flags()` context manager, and `compiledWithHipDNN()` runtime hook.
- Implements a generic thread-local `ParamsLRUCache` to amortize hipDNN `graph->build()` cost across repeated convolution calls with identical parameters.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/178515
Approved by: https://github.com/jeffdaily
Co-authored-by: Dmitry Nikolaev <dmitry.nikolaev@amd.com>
Summary:
CONTEXT: D97799113 was reverted by D98681402. This relands the original change.
Port GitHub PR https://github.com/pytorch/pytorch/issues/172246 to internal. This adds composable_kernel (CK) backend
support for scaled-dot-product-attention varlen path on ROCm:
- Fix philox seed/offset access in varlen backward to guard on is_dropout
- Fix softmax_lse shape and stride for CK varlen forward path
- Add `is_ck_sdpa_available()` API to `torch.backends.cuda`
- Update varlen_attn fake to use correct logsumexp shape per backend
- Add CK backend parametrization to varlen attention tests
Co-authored-by: Chinmay Kuchinad ChinmayDattanand.Kuchinad@amd.com
Co-authored-by: Jeff Daily jeff.daily@amd.com
Test Plan: buck test on relevant test targets
Differential Revision: D98693429
Pulled By: haoyuz
Pull Request resolved: https://github.com/pytorch/pytorch/pull/178729
Approved by: https://github.com/echen4096
The old implementation used a static std::unordered_set protected only by the GIL. Under free-threaded Python (3.14t) this is a data race.
Replace it with a Python dict stored in per-module state. The hot path (already-seen event) calls PyDict_Contains which is lock-free in 3.14t. The cold path (first occurrence) uses PyDict_SetDefaultRef which uses CPython's per-dict fine-grained locking to atomically check-and-insert, avoiding TOCTOU races between threads.
Authored with Claude.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/178867
Approved by: https://github.com/Skylion007
TorchVitals was an undocumented feature gated behind the TORCH_VITAL
environment variable that allowed setting and reading key-value "vital
signs" (e.g. whether CUDA or DataLoader was used). It was never
documented and the test itself had a "FIXME: document or deprecate"
comment.
Originally added in PR #51047 (Jan 2021), with a Python API in #53238
and a CUDA vital in #58059. There have been no feature changes since
June 2021 — all subsequent commits were mechanical cleanups (clang-tidy,
thread-safe getenv, etc.).
A search on cs.github.com and general web search confirmed no external
projects depend on torch.set_vital, torch.read_vitals, or
torch.vitals_enabled — usage was limited to pytorch/pytorch itself.
Authored with Claude.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/178479
Approved by: https://github.com/msaroufim, https://github.com/albanD
## Summary
Enables variable-length (varlen) attention support for the Composable Kernel (CK) SDPA backend on ROCm.
## Changes
### Forward pass (`mha_varlen_fwd_ck.hip`)
- Fixed LSE tensor allocation: changed from 3D `{batch_size, num_heads, max_seqlen_q}` to 2D `{num_heads, total_q}` to match CK group mode expectation
- Fixed `nhead_stride_lse`: use `stride(0)` instead of `stride(1)` for 2D layout
- Fixed `batch_stride_lse`: set to `0` (no batch dimension in group mode LSE)
- Fixed `min_seqlen_q`: changed from `-1` to `1` (valid minimum for kernel dispatch)
### Backward pass (`mha_varlen_bwd_ck.hip`)
- Fixed philox seed/offset access: guarded with `if (is_dropout)` to avoid dtype mismatch when dropout is disabled
### Test infrastructure (`test/test_varlen_attention.py`)
- Added `sdpa_backend` parametrization to test both `aotriton` and `ck` backends on ROCm
- Backend selection: `["aotriton", "ck"]` when CK is available, `["aotriton"]` otherwise
- Uses `preferred_rocm_fa_library()` to switch backends per test
### Platform detection
- Added `PLATFORM_SUPPORTS_CK_SDPA` in `torch/testing/_internal/common_cuda.py`
- Added `_is_ck_sdpa_available()` in `torch/csrc/Module.cpp` for runtime CK availability check
### Fake tensor implementation (`torch/nn/attention/varlen.py`)
- Updated `_varlen_attn_fake` to use standard `[num_heads, total_q]` logsumexp format (matches aotriton/CUDA)
## Dependencies: CK submodule bump
- **CK PR**: (https://github.com/ROCm/rocm-libraries/pull/4292)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/172246
Approved by: https://github.com/jeffdaily
This PR enables ROCm/HIP support for PyTorch's StaticCudaLauncher, which provides static compilation and launching of Triton kernels. The implementation has been tested on AMD MI300 and MI200 hardware.
**Changes**
**Python (torch/_inductor/runtime/)**
- static_cuda_launcher.py: Added ROCm detection, .hsaco binary support, and ROCm-specific scratch parameter handling
- triton_heuristics.py: Updated device type checks to support both cuda and hip
**C++ (torch/csrc/)**
- Module.cpp: Enabled StaticCudaLauncher for ROCm builds
- inductor/static_cuda_launcher.cpp: Added HIP API equivalents for all CUDA driver calls
- inductor/static_cuda_launcher.h: Updated header guard
**Tests (test/inductor/)**
- test_static_cuda_launcher.py: Removed @skipIfRocm decorators and updated binary file handling
**Enabled Unit Tests**
All tests in test/inductor/test_static_cuda_launcher.py now pass on ROCm:
1. Fixes#168654.
2. Fixes#168655.
3. Fixes#168656.
4. Fixes#168657.
5. Fixes#168658.
6. Fixes#168659.
7. Fixes#168660.
8. Fixes#168661.
9. Fixes#168662.
10. Fixes#168663.
11. Fixes#168664.
12. Fixes#168666
13. Fixes#168667
14. Fixes#168668
15. Fixes#168669
16. Fixes#168670
17. Fixes#168671
18. Fixes#168575
Pull Request resolved: https://github.com/pytorch/pytorch/pull/166492
Approved by: https://github.com/jeffdaily
This PR refactor the cuda static launcher to triton static launcher to be able to be reused by XPU triton backend as follow:
- Rename class `StaticallyLaunchedCudaKernel` -> class `StaticallyLaunchedTritonKernel`, and then keep `StaticallyLaunchedCudaKernel` inherit the `StaticallyLaunchedTritonKernel`. Then there will be `StaticallyLaunchedXpuKernel`.
- Rename static_cuda_launcher.py -> static_triton_launcher.py
- Rename torch/csrc/inductor/static_cuda_launcher.cpp -> torch/csrc/inductor/static_launcher/cuda.cpp
- Rename test/_inductor/test_static_cuda_launcher.py -> test/_inductor/test_static_triton_launcher.py
Pull Request resolved: https://github.com/pytorch/pytorch/pull/169121
Approved by: https://github.com/EikanWang, https://github.com/desertfire
### Summary:
Added support for flash attention v3 to SDPA
Only supports fp8 forward pass for now
Created new `_fa3.py` mirroring `_fa4.py` registration
modified `torch/nn/attention/__init__.py` and `_registry.py` to add FA3 support
Added C++ hook to expose FA3 activation (to allow for fp8 dtype)
### Design:
- The design follows the same basic structure as the [FA4 implementation](https://github.com/pytorch/pytorch/pull/167348).
- I added a `_fa3.py` which mirrors the same registration framework as `_fa4.py` (wiring up the new fa3 impl with aten ops `_flash_attention_forward`, `_flash_attention_backward`, `_scaled_dot_product_flash_attention`, `_scaled_dot_product_flash_attention_backward`)
- I additionally added a C++ hook to expose the FA3 activation to `sdp_utils.cpp`, and added a new function to check for low precision dtypes in flash attention (since before fp8 was not allowed)
- Note that only fp8 forward pass is supported for now. We can add fp16/bf16 support later, but fp16/bf16 performance with FA4 is better anyway. The backwards pass currently will throw an error: "FA3 does not support backward pass. Either: 1. Use torch.no_grad() for inference. 2. Unregister FA3 before training: `restore_flash_attention_impl`"
### Test Plan:
Install FA3 library: https://github.com/Dao-AILab/flash-attention/tree/main, follow directions for "FlashAttention-3 beta release", tldr; clone the repo, cd into hopper (where the FA3 implementation lives), python setup.py install
Following the same steps as FA4 to keep it consistent:
```python
activate_flash_attention_impl("FA3")
with sdpa_kernel(SDPBackend.FLASH_ATTENTION):
_ = F.scaled_dot_product_attention(q, k, v)
```
It also works with torch.compile
```python
activate_flash_attention_impl("FA3")
def sdpa_fn(q, k, v):
with sdpa_kernel(SDPBackend.FLASH_ATTENTION):
return F.scaled_dot_product_attention(q, k, v)
compiled_fn = torch.compile(sdpa_fn, fullgraph=True)
_ = compiled_fn(q, k, v)
```
### Some quick runtime results:
I ran some very quick tests with the pytorch profiler to test the runtime of fp8 using FA3 versus bf16 using FA4. Mostly just to catch any easy mistakes slowing it down. These are the results when running on (8, 16, 1024, 128) shape tensors:
FA3 eager:
[sdpa_profiler_fa3_eager.txt](https://github.com/user-attachments/files/24511657/sdpa_profiler_fa3_eager.txt)
FA3 compile:
[sdpa_profiler_fa3_compile.txt](https://github.com/user-attachments/files/24511664/sdpa_profiler_fa3_compile.txt)
FA4 eager:
[sdpa_profiler_fa4_eager.txt](https://github.com/user-attachments/files/24511667/sdpa_profiler_fa4_eager.txt)
FA4 compile:
[sdpa_profiler_fa4_compile.txt](https://github.com/user-attachments/files/24511669/sdpa_profiler_fa4_compile.txt)
### Some quick accuracy results:
I ran a very quick SQNR test on (8, 16, 1024, 128) shape tensors 100 times. I got an average SQNR of 25.55.
This is still a draft, so I'll be doing a quick test with the memory next. Then, I'll do some tests on the diffusers library stable diffusion models.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/172040
Approved by: https://github.com/drisspg
This PR enables ROCm/HIP support for PyTorch's StaticCudaLauncher, which provides static compilation and launching of Triton kernels. The implementation has been tested on AMD MI300 and MI200 hardware.
**Changes**
**Python (torch/_inductor/runtime/)**
- static_cuda_launcher.py: Added ROCm detection, .hsaco binary support, and ROCm-specific scratch parameter handling
- triton_heuristics.py: Updated device type checks to support both cuda and hip
**C++ (torch/csrc/)**
- Module.cpp: Enabled StaticCudaLauncher for ROCm builds
- inductor/static_cuda_launcher.cpp: Added HIP API equivalents for all CUDA driver calls
- inductor/static_cuda_launcher.h: Updated header guard
**Tests (test/inductor/)**
- test_static_cuda_launcher.py: Removed @skipIfRocm decorators and updated binary file handling
**Enabled Unit Tests**
All tests in test/inductor/test_static_cuda_launcher.py now pass on ROCm:
1. Fixes#168654.
2. Fixes#168655.
3. Fixes#168656.
4. Fixes#168657.
5. Fixes#168658.
6. Fixes#168659.
7. Fixes#168660.
8. Fixes#168661.
9. Fixes#168662.
10. Fixes#168663.
11. Fixes#168664.
12. Fixes#168666
13. Fixes#168667
14. Fixes#168668
15. Fixes#168669
16. Fixes#168670
17. Fixes#168671
18. Fixes#168575
Pull Request resolved: https://github.com/pytorch/pytorch/pull/166492
Approved by: https://github.com/jeffdaily, https://github.com/huydhn, https://github.com/atalman
This PR refactor the cuda static launcher to triton static launcher to be able to be reused by XPU triton backend as follow:
- Rename class `StaticallyLaunchedCudaKernel` -> class `StaticallyLaunchedTritonKernel`, and then keep `StaticallyLaunchedCudaKernel` inherit the `StaticallyLaunchedTritonKernel`. Then there will be `StaticallyLaunchedXpuKernel`.
- Rename static_cuda_launcher.py -> static_triton_launcher.py
- Rename torch/csrc/inductor/static_cuda_launcher.cpp -> torch/csrc/inductor/static_launcher/cuda.cpp
- Rename test/_inductor/test_static_cuda_launcher.py -> test/_inductor/test_static_triton_launcher.py
Pull Request resolved: https://github.com/pytorch/pytorch/pull/169121
Approved by: https://github.com/EikanWang, https://github.com/desertfire
## Addressed Issue
Issue #162845
## Summary of Changes
This PR introduces a unified `DLPackExchangeAPI` struct as described in proposal [175](https://github.com/dmlc/dlpack/issues/175). This new convention replaces the previous mechanism of separate function pointers, and aligns with the latest DLPack standard as shown in PR [174](https://github.com/dmlc/dlpack/pull/174).
Specifically, the new `DLPackExchangeAPI` struct is exposed as `torch.Tensor.__c_dlpack_exchange_api__`, which stores and exposes the following function pointers:
* `managed_tensor_allocator`
* `managed_tensor_from_py_object_no_sync`
* `managed_tensor_to_py_object_no_sync`
* `dltensor_from_py_object_no_sync`
* `current_work_stream`
Within the new `DLPackExchangeAPI` struct, the new `current_work_stream` function pointer allows more robust and integrated querying of the current device stream (e.g., CUDA stream) during DLPack tensor exchanges. All the conversion from/to DLPack has been updated to `_no_sync`, meaning you should use `current_work_stream` to explicitly handle stream synchronization. It also includes a non-owning DLTensor conversion `dltensor_from_py_object_no_sync` to avoid unnecessary reference counting.
Following this change, the `dlpack.h` has been updated to the latest DLPack.
Unit tests are added using `torch.utils.cpp_extension.load_inline` to avoid GIL release issues
when calling `THPVariable_Wrap`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/165483
Approved by: https://github.com/tqchen, https://github.com/albanD
This PR enables ROCm/HIP support for PyTorch's StaticCudaLauncher, which provides static compilation and launching of Triton kernels. The implementation has been tested on AMD MI300 and MI200 hardware.
**Changes**
**Python (torch/_inductor/runtime/)**
- static_cuda_launcher.py: Added ROCm detection, .hsaco binary support, and ROCm-specific scratch parameter handling
- triton_heuristics.py: Updated device type checks to support both cuda and hip
**C++ (torch/csrc/)**
- Module.cpp: Enabled StaticCudaLauncher for ROCm builds
- inductor/static_cuda_launcher.cpp: Added HIP API equivalents for all CUDA driver calls
- inductor/static_cuda_launcher.h: Updated header guard
**Tests (test/inductor/)**
- test_static_cuda_launcher.py: Removed @skipIfRocm decorators and updated binary file handling
**Enabled Unit Tests**
All tests in test/inductor/test_static_cuda_launcher.py now pass on ROCm:
1. test_basic
2. test_unsigned_integers
3. test_signed_integers
4. test_basic_1arg
5. test_constexpr
6. test_implied_constant
7. test_kernel_no_args
8. test_high_shared_mem
9. test_too_high_shared_mem
10. test_kernel_empty_tensor
11. test_kernel_many_args
12. test_basic_compile
13. test_incompatible_code
14. test_static_launch_user_defined_triton_kernels
15. test_empty_tensor
16. test_any
17. test_disable_static_cuda_launcher
In addition to this, the following tests from test/inductor/test_codecache.py also pass:
1. test_remote_cache_load_function_device_cuda_float32_dynamic_False_bundle_triton_False_use_static_cuda_launcher_False
2. test_remote_cache_load_function_device_cuda_float32_dynamic_False_bundle_triton_True_use_static_cuda_launcher_False
3. test_remote_cache_load_function_device_cuda_float32_dynamic_False_bundle_triton_True_use_static_cuda_launcher_True
4. test_remote_cache_load_function_device_cuda_bfloat16_dynamic_False_bundle_triton_False_use_static_cuda_launcher_False
5. test_remote_cache_load_function_device_cuda_bfloat16_dynamic_False_bundle_triton_True_use_static_cuda_launcher_False
6. test_remote_cache_load_function_device_cuda_bfloat16_dynamic_False_bundle_triton_True_use_static_cuda_launcher_True
The following tests are skipped since triton bundling is necessary for StaticCudaLauncher:
1. test_remote_cache_load_function_device_cuda_float32_dynamic_False_bundle_triton_False_use_static_cuda_launcher_True
2. test_remote_cache_load_function_device_cuda_bfloat16_dynamic_False_bundle_triton_False_use_static_cuda_launcher_True
Pull Request resolved: https://github.com/pytorch/pytorch/pull/166492
Approved by: https://github.com/jeffdaily