Files
pytorch/docs/cpp/source/api/cuda/utilities.md
T
Svetlana Karslioglu 19da3da3d3 Restructure C++ docs (#174096)
The fix-cpp-build branch is a major restructuring that:

- Drops Exhale (which auto-generated API docs but caused OOM in CI) in favor of hand-written RST pages using Breathe directives
- Reorganizes notes into a hierarchical API reference with proper categories
- Adds new content: XPU docs, stable ABI docs, DeviceGuard/Stream docs
- Always runs Doxygen instead of gating on an env var

Pull Request resolved: https://github.com/pytorch/pytorch/pull/174096
Approved by: https://github.com/janeyx99, https://github.com/seemethere, https://github.com/albanD
2026-04-07 17:24:38 +00:00

227 lines
5.0 KiB
Markdown

---
myst:
html_meta:
description: CUDA utility functions in PyTorch C++ — device properties, cuBLAS/cuDNN handles, and stream management.
keywords: PyTorch, C++, CUDA, device_count, cuBLAS, cuDNN, cuSPARSE, TensorDescriptor
---
# CUDA Utility Functions
PyTorch provides utility functions for querying and managing CUDA devices,
streams, and library handles.
## Device Management
```{doxygenfunction} torch::cuda::device_count
```
```{cpp:function} int c10::cuda::current_device()
Returns the index of the current CUDA device.
```
**Example:**
```cpp
#include <c10/cuda/CUDAFunctions.h>
// Check available devices
int num_devices = c10::cuda::device_count();
// Get current device
int current = c10::cuda::current_device();
```
## Device Properties
```{doxygenfunction} at::cuda::getCurrentDeviceProperties
```
```{doxygenfunction} at::cuda::getDeviceProperties
```
```{doxygenfunction} at::cuda::canDeviceAccessPeer
```
```{doxygenfunction} at::cuda::warp_size
```
**Example:**
```cpp
#include <ATen/cuda/CUDAContext.h>
// Query properties of the current device
cudaDeviceProp* props = at::cuda::getCurrentDeviceProperties();
std::cout << "Device: " << props->name << std::endl;
std::cout << "Compute capability: " << props->major << "." << props->minor << std::endl;
// Query a specific device
cudaDeviceProp* dev1_props = at::cuda::getDeviceProperties(1);
// Check peer access
bool can_access = at::cuda::canDeviceAccessPeer(0, 1);
```
## Library Handles
These functions return handles for CUDA math libraries on the current device
and stream. They are primarily useful when writing custom CUDA kernels that
call cuBLAS, cuSPARSE, or cuSOLVER directly.
```{doxygenfunction} at::cuda::getCurrentCUDABlasHandle
```
```{doxygenfunction} at::cuda::getCurrentCUDABlasLtHandle
```
```{doxygenfunction} at::cuda::getCurrentCUDASparseHandle
```
```{doxygenfunction} at::cuda::getCurrentCUDASolverDnHandle
```
**Example:**
```cpp
#include <ATen/cuda/CUDAContext.h>
// Get cuBLAS handle for current device/stream
cublasHandle_t handle = at::cuda::getCurrentCUDABlasHandle();
// Get cuSPARSE handle
cusparseHandle_t sparse_handle = at::cuda::getCurrentCUDASparseHandle();
```
## cuDNN Descriptors
When writing custom kernels that call cuDNN directly, PyTorch provides RAII
wrapper classes for cuDNN descriptors. These are defined in
`ATen/cudnn/Descriptors.h`.
### Descriptor (Base Class)
```{doxygenclass} at::native::Descriptor
:members:
:undoc-members:
```
A generic RAII wrapper for cuDNN descriptor types. Descriptors default
construct to a `nullptr` and are initialized on first use via `mut_desc()`.
Use `desc()` for read-only access.
### TensorDescriptor
```{doxygenclass} at::native::TensorDescriptor
:members:
:undoc-members:
```
Wraps `cudnnTensorDescriptor_t`. Supports padding lower-dimensional tensors
to meet cuDNN broadcasting requirements (see `pad` parameter).
**Example:**
```cpp
#include <ATen/cudnn/Descriptors.h>
at::Tensor input = torch::randn({32, 3, 224, 224}, torch::kCUDA);
at::native::TensorDescriptor desc(input);
cudnnTensorDescriptor_t raw = desc.desc();
```
### FilterDescriptor
```{doxygenclass} at::native::FilterDescriptor
:members:
:undoc-members:
```
Wraps `cudnnFilterDescriptor_t` for convolution filter weights.
### ConvolutionDescriptor
```{doxygenstruct} at::native::ConvolutionDescriptor
:members:
:undoc-members:
```
Wraps `cudnnConvolutionDescriptor_t`. Configures padding, stride, dilation,
groups, and math type (TF32, tensor ops) for convolution operations.
### RNNDataDescriptor
```{doxygenclass} at::native::RNNDataDescriptor
:members:
:undoc-members:
```
Wraps `cudnnRNNDataDescriptor_t` for variable-length sequence data.
### DropoutDescriptor
```{doxygenstruct} at::native::DropoutDescriptor
:members:
:undoc-members:
```
Wraps `cudnnDropoutDescriptor_t`. Manages RNG state for cuDNN dropout.
### ActivationDescriptor
```{doxygenstruct} at::native::ActivationDescriptor
:members:
:undoc-members:
```
Wraps `cudnnActivationDescriptor_t`.
### SpatialTransformerDescriptor
```{doxygenstruct} at::native::SpatialTransformerDescriptor
:members:
:undoc-members:
```
### CTCLossDescriptor
```{doxygenstruct} at::native::CTCLossDescriptor
:members:
:undoc-members:
```
## Stream Management
```{doxygenfunction} c10::cuda::getDefaultCUDAStream
```
```{doxygenfunction} c10::cuda::getCurrentCUDAStream
```
```{doxygenfunction} c10::cuda::setCurrentCUDAStream
```
```{doxygenfunction} c10::cuda::getStreamFromPool(const bool isHighPriority, DeviceIndex device)
```
```{doxygenfunction} c10::cuda::getStreamFromExternal
```
**Example:**
```cpp
#include <c10/cuda/CUDAStream.h>
// Create and set custom stream
auto stream = c10::cuda::getStreamFromPool();
c10::cuda::setCurrentCUDAStream(stream);
// Get default stream
auto default_stream = c10::cuda::getDefaultCUDAStream();
// Wrap an externally created CUDA stream
cudaStream_t ext_stream;
cudaStreamCreate(&ext_stream);
auto wrapped = c10::cuda::getStreamFromExternal(ext_stream, /*device_index=*/0);
```