mirror of
https://github.com/pytorch/pytorch.git
synced 2026-11-20 09:04:25 +08:00
19da3da3d3
The fix-cpp-build branch is a major restructuring that: - Drops Exhale (which auto-generated API docs but caused OOM in CI) in favor of hand-written RST pages using Breathe directives - Reorganizes notes into a hierarchical API reference with proper categories - Adds new content: XPU docs, stable ABI docs, DeviceGuard/Stream docs - Always runs Doxygen instead of gating on an env var Pull Request resolved: https://github.com/pytorch/pytorch/pull/174096 Approved by: https://github.com/janeyx99, https://github.com/seemethere, https://github.com/albanD
97 lines
1.9 KiB
Markdown
97 lines
1.9 KiB
Markdown
---
|
|
myst:
|
|
html_meta:
|
|
description: CUDA device and stream guards in PyTorch C++ — CUDAGuard, CUDAStreamGuard, and CUDAMultiStreamGuard.
|
|
keywords: PyTorch, C++, CUDA, CUDAGuard, CUDAStreamGuard, device guard, multi-GPU
|
|
---
|
|
|
|
# CUDA Guards
|
|
|
|
CUDA guards are RAII wrappers that set a CUDA device or stream as the current
|
|
context and automatically restore the previous context when the guard goes
|
|
out of scope.
|
|
|
|
## CUDAGuard
|
|
|
|
```{doxygenstruct} c10::cuda::CUDAGuard
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
**Example:**
|
|
|
|
```cpp
|
|
#include <c10/cuda/CUDAGuard.h>
|
|
|
|
{
|
|
c10::cuda::CUDAGuard guard(1); // Switch to device 1
|
|
// All CUDA operations here run on device 1
|
|
auto tensor = torch::zeros({2, 2}, torch::device(torch::kCUDA));
|
|
}
|
|
// Previous device is restored
|
|
```
|
|
|
|
## CUDAStreamGuard
|
|
|
|
```{doxygenstruct} c10::cuda::CUDAStreamGuard
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
**Example:**
|
|
|
|
```cpp
|
|
#include <c10/cuda/CUDAGuard.h>
|
|
|
|
auto stream = c10::cuda::getStreamFromPool();
|
|
{
|
|
c10::cuda::CUDAStreamGuard guard(stream);
|
|
// Operations here use the specified stream
|
|
}
|
|
// Previous stream is restored
|
|
```
|
|
|
|
## OptionalCUDAGuard
|
|
|
|
```{doxygenstruct} c10::cuda::OptionalCUDAGuard
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
**Example:**
|
|
|
|
```cpp
|
|
c10::cuda::OptionalCUDAGuard guard;
|
|
if (use_cuda) {
|
|
guard.set_device(0);
|
|
}
|
|
// Guard only switches device if set_device was called
|
|
```
|
|
|
|
## OptionalCUDAStreamGuard
|
|
|
|
```{doxygenstruct} c10::cuda::OptionalCUDAStreamGuard
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
## CUDAMultiStreamGuard
|
|
|
|
```{doxygenstruct} c10::cuda::CUDAMultiStreamGuard
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
**Example:**
|
|
|
|
```cpp
|
|
at::cuda::CUDAStream stream0 = at::cuda::getStreamFromPool(false, 0);
|
|
at::cuda::CUDAStream stream1 = at::cuda::getStreamFromPool(false, 1);
|
|
|
|
{
|
|
at::cuda::CUDAMultiStreamGuard multi_guard({stream0, stream1});
|
|
// stream0 is current on device 0, stream1 on device 1
|
|
}
|
|
// Both streams restored
|
|
```
|