Files
pytorch/docs/cpp/source/api/nn/utilities.md
T
Svetlana Karslioglu 19da3da3d3 Restructure C++ docs (#174096)
The fix-cpp-build branch is a major restructuring that:

- Drops Exhale (which auto-generated API docs but caused OOM in CI) in favor of hand-written RST pages using Breathe directives
- Reorganizes notes into a hierarchical API reference with proper categories
- Adds new content: XPU docs, stable ABI docs, DeviceGuard/Stream docs
- Always runs Doxygen instead of gating on an env var

Pull Request resolved: https://github.com/pytorch/pytorch/pull/174096
Approved by: https://github.com/janeyx99, https://github.com/seemethere, https://github.com/albanD
2026-04-07 17:24:38 +00:00

5.0 KiB

myst
myst
html_meta
description keywords
Neural network utilities in PyTorch C++ — parameter initialization, module cloning, padding layers, and vision utilities. PyTorch, C++, nn utilities, init, Cloneable, AnyModule, padding, PixelShuffle, Upsample

Utilities

Additional utilities for building neural networks: parameter initialization, module cloning, type-erased containers, padding layers, and vision utilities.

Parameter Initialization

The torch::nn::init namespace provides functions for initializing module parameters:

#include <torch/nn/init.h>

// Xavier/Glorot initialization
torch::nn::init::xavier_uniform_(linear->weight);
torch::nn::init::xavier_normal_(linear->weight);

// Kaiming/He initialization
torch::nn::init::kaiming_uniform_(conv->weight, /*a=*/0, torch::kFanIn, torch::kReLU);
torch::nn::init::kaiming_normal_(conv->weight);

// Other initializations
torch::nn::init::zeros_(linear->bias);
torch::nn::init::ones_(bn->weight);
torch::nn::init::constant_(linear->bias, 0.1);
torch::nn::init::normal_(linear->weight, /*mean=*/0, /*std=*/0.01);
torch::nn::init::uniform_(linear->weight, /*a=*/-0.1, /*b=*/0.1);
torch::nn::init::orthogonal_(rnn->weight_hh);

Cloneable

All torch::nn modules inherit from Cloneable, enabling deep copies:

auto model = torch::nn::Linear(10, 5);
auto model_copy = std::dynamic_pointer_cast<torch::nn::LinearImpl>(model->clone());

AnyModule

AnyModule provides type-erased storage for any module, allowing you to store heterogeneous modules in a single container.

:members:
:undoc-members:

Example:

torch::nn::AnyModule any_module(torch::nn::Linear(10, 5));
auto output = any_module.forward(input);

Functional

Wraps a function or callable as a module, useful for inserting arbitrary functions into a Sequential container.

:members:
:undoc-members:

ModuleHolder

:members:
:undoc-members:

CosineSimilarity

:members:
:undoc-members:

PairwiseDistance

:members:
:undoc-members:

PackedSequence


Holds the data and list of `batch_sizes` of a packed sequence.
All RNN modules accept packed sequences as inputs.

Returns the packed tensor containing all sequence elements.

Returns a 1D tensor of batch sizes at each time step.

Returns indices used to sort sequences by length (descending).

Returns indices to restore the original sequence order.

Moves the packed sequence to the specified device.

See also: torch::nn::utils::rnn::pack_padded_sequence and torch::nn::utils::rnn::pad_packed_sequence.

Padding Layers

ReflectionPad1d / ReflectionPad2d / ReflectionPad3d

:members:
:undoc-members:
:members:
:undoc-members:
:members:
:undoc-members:

ReplicationPad1d / ReplicationPad2d / ReplicationPad3d

:members:
:undoc-members:
:members:
:undoc-members:
:members:
:undoc-members:

ZeroPad1d / ZeroPad2d / ZeroPad3d

:members:
:undoc-members:
:members:
:undoc-members:
:members:
:undoc-members:

ConstantPad1d / ConstantPad2d / ConstantPad3d

:members:
:undoc-members:
:members:
:undoc-members:
:members:
:undoc-members:

Vision Layers

PixelShuffle

:members:
:undoc-members:
:members:
:undoc-members:

PixelUnshuffle

:members:
:undoc-members:
:members:
:undoc-members:

Upsample

:members:
:undoc-members:
:members:
:undoc-members:

Fold / Unfold

:members:
:undoc-members:
:members:
:undoc-members:
:members:
:undoc-members:
:members:
:undoc-members: