Resolves#171827
`enable_symm_mem_for_group` is for getting access to the store of a group.
But the store can be also retrieved by `ProcessGroup.getStore()` in C++.
Thus makes little sense to require users to call `enable_symm_mem_for_group`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/172163
Approved by: https://github.com/Skylion007
# Motivation
This PR aims to introduce a new API `torch.xpu.memory_snapshot` to provide detailed information for each memory segment managed by the allocator, including its size, owning pool, associated stream, call stack traces, and other relevant attributes.
# Additional Context
We use the UT to verify the consistency of memory information between `torch.xpu.memory_snapshot` and `torch.xpu.memory_stats`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/169442
Approved by: https://github.com/EikanWang, https://github.com/albanD, https://github.com/eellison
ghstack dependencies: #169296
part of series of changes to size hinting:
1. rename hint_int to size_hint (to be consistent with sizevars similar to all other functions that we unified their names)
2. remove size_hint allow_none and add fallback argument instead same as size_hint.
3. add explicit check for common wrong usage of sizevars.size_hint with symint inputs to throw clear error message for better user experience. and found two bugs :
a) a bug in estimate_flops and fix it !
b) a bug in tune_kernel and fix it.
C) bug in count_flops_fx.
4. support calling size_hint from traced code.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/171944
Approved by: https://github.com/bobrenjc93
ghstack dependencies: #171843
This pull request introduces the new `ExportableModule` abstract interface for ONNX exportable modules, integrates it into the ONNX export workflow, and provides comprehensive unit tests for its functionality. The changes improve the extensibility and usability of ONNX export by allowing models to define example arguments, dynamic shapes, and input/output names. The most important changes are grouped below by theme.
Example:
```py
class Model(torch.nn.Module):
def forward(self, x):
return x * 2
class MyExportableModule(torch.onnx.ExportableModule):
def __init__(self):
super().__init__()
self.model = Model()
def forward(self, x):
return self.model(x)
def example_arguments(self):
return (torch.randn(2, 3, 224, 224),), None
def input_names(self):
return ("input",)
def output_names(self):
return ("output",)
def dynamic_shapes(self):
return ({0: "batch_size"},)
exportable_module = MyExportableModule()
onnx_program = exportable_module.to_onnx()
# The model can also be supplied directly to torch.onnx.export
onnx_program = torch.onnx.export(exportable_module)
```
Fix https://github.com/pytorch/pytorch/issues/165354
Pull Request resolved: https://github.com/pytorch/pytorch/pull/170810
Approved by: https://github.com/titaiwangms
Fix outdated documentation links in torch.compile docs.
While reading the torch.compile documentation, I noticed several links pointing to
deprecated pytorch.org URLs. This PR updates them to the correct pages under
docs.pytorch.org to avoid broken references.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/171696
Approved by: https://github.com/Lucaskabela
## Summary
This PR adds a new accelerator documentation chapter, `docs/source/accelerator/profiler.md`, and wires it into the accelerator guide index. The doc explains PyTorch’s profiler integration paths at a high level (legacy autograd vs Kineto) and then focuses on how `PrivateUse1` accelerators can integrate via the legacy autograd path. It uses OpenReg as the illustrative example in the Implementation section, without coupling Design/Background to any specific backend.
## Motivation
Third‑party accelerators integrating via `PrivateUse1` need a concise, device‑agnostic reference for profiler integration. Today, details are scattered across C++ stubs, Python control code, and tests. This documentation centralizes:
- Where an accelerator plugs into the profiler surface
- What hooks are required (record/elapsed/synchronize/onEachDevice/annotations)
- How to validate behavior with multi‑stream/device, shapes, and chrome trace export
## What’s changed
- Added `docs/source/accelerator/profiler.md`
- Background: links to `torch/csrc/profiler/README.md` and outlines the two profiler paths
- Design: backend‑agnostic control plane (Python, ProfilerStubs, backend runtime)
- Implementation: introduces OpenReg as the example for `PrivateUse1` legacy path
- Usage patterns: snippets for multi‑stream, training loops, and trace export
- Validation and limitations: summarizes tested scenarios and current constraints
- Updated `docs/source/accelerator/index.md` to include `profiler` in the toctree
Pull Request resolved: https://github.com/pytorch/pytorch/pull/170276
Approved by: https://github.com/fffrog
Creating a new guard filter for AOT precompile that only preserve the following 3 types of guards:
1. TENSOR_MATCH
2. SHAPE_ENV
3. GLOBAL_STATE
These are the guards that matter empirically in AOT use cases like large scale training.
Other types of dynamo guards are dropped from this policy because:
1. Serialization on those guards often require complex pickling behavior that brings additional implementation cost.
2. Those guards (global variable id matches, class matches, etc) usually doesn't change a lot in training.
3. More guards will introduce more overhead during runtime. By default we'd rather keep guards checking minimal at runtime if possible.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/170082
Approved by: https://github.com/bobrenjc93
ghstack dependencies: #170081
Fixes#168317
Adds support for ignoring arbitrary Python callables during TorchDynamo tracing.
Introduces the new `ignore_logging_functions` config and updates DebuggingVariable
so functions in this set are treated as full no-ops without reordering, execution,
or graph breaks. The existing behavior for other logging configurations remains unchanged.
Tested locally by adding functions to `ignore_logging_functions` and verifying
that Dynamo skips them without graph breaks and produces the correct compiled output.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/168913
Approved by: https://github.com/williamwen42
Summary:
The implementation adds 9 RNG functions (get/set_rng_state, manual_seed, seed, initial_seed, and their _all variants) that integrate with the existing C++ generator infrastructure. The lazy initialization pattern ensures RNG operations queue correctly before device initialization and execute during _lazy_init(). We follow the existing CUDA pattern. In addition, we integrated with the general torch.manual_seed and torch.seed functions following the exisiting custom accelerator pattern.
Added comprehensive test coverage for all API calls as well as for the torch integration.
Test Plan:
buck2 test //mtia/host_runtime/torch_mtia/tests:test_torch_mtia_api
https://www.internalfb.com/intern/testinfra/testrun/2533275122809554
buck2 test fbcode//mtia/host_runtime/torch_mtia/tests:test_mtia_rng_generators
https://www.internalfb.com/intern/testinfra/testrun/9288674362815734
Reviewed By: StellarrZ
Differential Revision: D88014851
Pull Request resolved: https://github.com/pytorch/pytorch/pull/169982
Approved by: https://github.com/albanD
# Overview
This PR adds a new float32 precision API
`torch.backends.cuda.math_sdp.fp32_precision` to configure fp32 precision
behavior of `SDPBackend.MATH`
# Rationale
The `test/test_transformers.py` testing suite calculates the numerical
tolerance by comparing output tensors from the same precision ("reference")
and higher precision ("golden"), both calculated by `SDPBackend.MATH`.
However, the golden output is calculated with TF32 rather than FP32, which in
fact is less accurate than the FA/ME backend if they used IEEE rather than
TF32 for their accumulation.
The loss of precison causes false negatives in SDPA tests like
`TestSDPACudaOnlyCUDA.test_flash_attention_vs_math_ref_grads_batch_size_8_seq_len_q_143_seq_len_k_4_head_dim_203_is_causal_False_dropout_p_0_22_float16_scale_l1_enable_gqa_True_n_heads1_cuda_float16`
, at least on ROCM platform. The false negative disappears after forcing
`higher_precision_dtype = torch.float64`
# Major Changes
To restore the precision of golden output, a new API
`torch.backends.cuda.math_sdp.fp32_precision` is introduced, which allows
configuration of "matmul" precision during `SDPBackend.MATH`, and a new
decorator `@math_sdp_precision("ieee")` is added to all tests that use
`check_out_and_grad`. At last, an assert is added to the inner most function
`_check_equal` as a sanity check to ensure `math_sdp` has the right precison
configured for `torch.float32` golden tensors.
# Known Issues
The backward phase honors the configuration when calling `backward()`, regardless
the configuration when creating the graph.
This is copy of PR https://github.com/pytorch/pytorch/pull/167157 due to pytorchbot limited access to the ROCm fork.
As reference, pr for checking test state: https://github.com/pytorch/pytorch/pull/169676
Pull Request resolved: https://github.com/pytorch/pytorch/pull/169694
Approved by: https://github.com/jeffdaily
Co-authored-by: Xinya Zhang <Xinya.Zhang@amd.com>
# Motivation
This PR aims to document `PYTORCH_ALLOC_CONF` in the doc and comments and reminder the user that the previous environment variable `PYTORCH_CUDA_ALLOC_CONF` is its alias.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/167659
Approved by: https://github.com/albanD
## Summary
This PR refactors parts of the OpenReg device guard implementation, tightens up device/event/stream handling, and adds focused documentation to help developers understand and reuse the guard API for the PrivateUse1 backend.
## Motivation
- The OpenReg guard implementation previously mixed device, stream and event responsibilities in a single, less-structured area. That made it harder to include focused code snippets in the docs and to reason about semantics (e.g., which methods are device-only vs stream or event related).
- The documentation lacked a single, discoverable Guard chapter and literalinclude-friendly snippet markers that docs can include precisely.
- Small API/implementation correctness improvements (e.g., ensure runtime checks are applied) reduce subtle runtime errors when switching devices, recording events, or synchronizing streams.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/166288
Approved by: https://github.com/fffrog