mirror of
https://github.com/pytorch/pytorch.git
synced 2026-11-20 09:04:25 +08:00
19da3da3d3
The fix-cpp-build branch is a major restructuring that: - Drops Exhale (which auto-generated API docs but caused OOM in CI) in favor of hand-written RST pages using Breathe directives - Reorganizes notes into a hierarchical API reference with proper categories - Adds new content: XPU docs, stable ABI docs, DeviceGuard/Stream docs - Always runs Doxygen instead of gating on an env var Pull Request resolved: https://github.com/pytorch/pytorch/pull/174096 Approved by: https://github.com/janeyx99, https://github.com/seemethere, https://github.com/albanD
262 lines
5.0 KiB
Markdown
262 lines
5.0 KiB
Markdown
---
|
|
myst:
|
|
html_meta:
|
|
description: Neural network utilities in PyTorch C++ — parameter initialization, module cloning, padding layers, and vision utilities.
|
|
keywords: PyTorch, C++, nn utilities, init, Cloneable, AnyModule, padding, PixelShuffle, Upsample
|
|
---
|
|
|
|
# Utilities
|
|
|
|
Additional utilities for building neural networks: parameter initialization,
|
|
module cloning, type-erased containers, padding layers, and vision utilities.
|
|
|
|
## Parameter Initialization
|
|
|
|
The `torch::nn::init` namespace provides functions for initializing module parameters:
|
|
|
|
```cpp
|
|
#include <torch/nn/init.h>
|
|
|
|
// Xavier/Glorot initialization
|
|
torch::nn::init::xavier_uniform_(linear->weight);
|
|
torch::nn::init::xavier_normal_(linear->weight);
|
|
|
|
// Kaiming/He initialization
|
|
torch::nn::init::kaiming_uniform_(conv->weight, /*a=*/0, torch::kFanIn, torch::kReLU);
|
|
torch::nn::init::kaiming_normal_(conv->weight);
|
|
|
|
// Other initializations
|
|
torch::nn::init::zeros_(linear->bias);
|
|
torch::nn::init::ones_(bn->weight);
|
|
torch::nn::init::constant_(linear->bias, 0.1);
|
|
torch::nn::init::normal_(linear->weight, /*mean=*/0, /*std=*/0.01);
|
|
torch::nn::init::uniform_(linear->weight, /*a=*/-0.1, /*b=*/0.1);
|
|
torch::nn::init::orthogonal_(rnn->weight_hh);
|
|
```
|
|
|
|
## Cloneable
|
|
|
|
```{doxygenclass} torch::nn::Cloneable
|
|
```
|
|
|
|
All `torch::nn` modules inherit from `Cloneable`, enabling deep copies:
|
|
|
|
```cpp
|
|
auto model = torch::nn::Linear(10, 5);
|
|
auto model_copy = std::dynamic_pointer_cast<torch::nn::LinearImpl>(model->clone());
|
|
```
|
|
|
|
## AnyModule
|
|
|
|
`AnyModule` provides type-erased storage for any module, allowing you to
|
|
store heterogeneous modules in a single container.
|
|
|
|
```{doxygenclass} torch::nn::AnyModule
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
**Example:**
|
|
|
|
```cpp
|
|
torch::nn::AnyModule any_module(torch::nn::Linear(10, 5));
|
|
auto output = any_module.forward(input);
|
|
```
|
|
|
|
## Functional
|
|
|
|
Wraps a function or callable as a module, useful for inserting arbitrary
|
|
functions into a `Sequential` container.
|
|
|
|
```{doxygenclass} torch::nn::FunctionalImpl
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
## ModuleHolder
|
|
|
|
```{doxygenclass} torch::nn::ModuleHolder
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
## CosineSimilarity
|
|
|
|
```{doxygenclass} torch::nn::CosineSimilarity
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
## PairwiseDistance
|
|
|
|
```{doxygenclass} torch::nn::PairwiseDistance
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
## PackedSequence
|
|
|
|
```{cpp:class} torch::nn::utils::rnn::PackedSequence
|
|
|
|
Holds the data and list of `batch_sizes` of a packed sequence.
|
|
All RNN modules accept packed sequences as inputs.
|
|
```
|
|
|
|
```{cpp:function} const Tensor& data() const
|
|
|
|
Returns the packed tensor containing all sequence elements.
|
|
```
|
|
|
|
```{cpp:function} const Tensor& batch_sizes() const
|
|
|
|
Returns a 1D tensor of batch sizes at each time step.
|
|
```
|
|
|
|
```{cpp:function} const Tensor& sorted_indices() const
|
|
|
|
Returns indices used to sort sequences by length (descending).
|
|
```
|
|
|
|
```{cpp:function} const Tensor& unsorted_indices() const
|
|
|
|
Returns indices to restore the original sequence order.
|
|
```
|
|
|
|
```{cpp:function} PackedSequence to(torch::Device device) const
|
|
|
|
Moves the packed sequence to the specified device.
|
|
```
|
|
|
|
See also: `torch::nn::utils::rnn::pack_padded_sequence` and
|
|
`torch::nn::utils::rnn::pad_packed_sequence`.
|
|
|
|
## Padding Layers
|
|
|
|
### ReflectionPad1d / ReflectionPad2d / ReflectionPad3d
|
|
|
|
```{doxygenclass} torch::nn::ReflectionPad1d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenclass} torch::nn::ReflectionPad2d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenclass} torch::nn::ReflectionPad3d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
### ReplicationPad1d / ReplicationPad2d / ReplicationPad3d
|
|
|
|
```{doxygenclass} torch::nn::ReplicationPad1d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenclass} torch::nn::ReplicationPad2d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenclass} torch::nn::ReplicationPad3d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
### ZeroPad1d / ZeroPad2d / ZeroPad3d
|
|
|
|
```{doxygenclass} torch::nn::ZeroPad1d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenclass} torch::nn::ZeroPad2d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenclass} torch::nn::ZeroPad3d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
### ConstantPad1d / ConstantPad2d / ConstantPad3d
|
|
|
|
```{doxygenclass} torch::nn::ConstantPad1d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenclass} torch::nn::ConstantPad2d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenclass} torch::nn::ConstantPad3d
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
## Vision Layers
|
|
|
|
### PixelShuffle
|
|
|
|
```{doxygenclass} torch::nn::PixelShuffle
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenstruct} torch::nn::PixelShuffleOptions
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
### PixelUnshuffle
|
|
|
|
```{doxygenclass} torch::nn::PixelUnshuffle
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenstruct} torch::nn::PixelUnshuffleOptions
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
### Upsample
|
|
|
|
```{doxygenclass} torch::nn::Upsample
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenstruct} torch::nn::UpsampleOptions
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
### Fold / Unfold
|
|
|
|
```{doxygenclass} torch::nn::Fold
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenstruct} torch::nn::FoldOptions
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenclass} torch::nn::Unfold
|
|
:members:
|
|
:undoc-members:
|
|
```
|
|
|
|
```{doxygenstruct} torch::nn::UnfoldOptions
|
|
:members:
|
|
:undoc-members:
|
|
```
|