mirror of
https://github.com/pytorch/pytorch.git
synced 2026-11-20 09:04:25 +08:00
19da3da3d3
The fix-cpp-build branch is a major restructuring that: - Drops Exhale (which auto-generated API docs but caused OOM in CI) in favor of hand-written RST pages using Breathe directives - Reorganizes notes into a hierarchical API reference with proper categories - Adds new content: XPU docs, stable ABI docs, DeviceGuard/Stream docs - Always runs Doxygen instead of gating on an env var Pull Request resolved: https://github.com/pytorch/pytorch/pull/174096 Approved by: https://github.com/janeyx99, https://github.com/seemethere, https://github.com/albanD
67 lines
1.9 KiB
Markdown
67 lines
1.9 KiB
Markdown
---
|
|
myst:
|
|
html_meta:
|
|
description: PyTorch C++ data loading API — datasets, data loaders, samplers, and transforms.
|
|
keywords: PyTorch, C++, data, DataLoader, Dataset, sampler, transform
|
|
---
|
|
|
|
# Data Loading (torch::data)
|
|
|
|
The `torch::data` namespace provides utilities for loading and processing
|
|
datasets during training. It includes dataset abstractions, data loaders for
|
|
batching and shuffling, samplers for controlling data access patterns, and
|
|
transforms for data augmentation.
|
|
|
|
**When to use torch::data:**
|
|
|
|
- When loading training data in batches
|
|
- When you need parallel data loading with multiple workers
|
|
- When implementing custom datasets or transforms
|
|
|
|
**Components overview:**
|
|
|
|
- **Dataset**: Defines how to access individual samples (implement `get()` and `size()`)
|
|
- **DataLoader**: Batches samples and optionally shuffles/parallelizes loading
|
|
- **Sampler**: Controls the order in which samples are accessed
|
|
- **Transform**: Applies preprocessing (normalization, augmentation) to samples
|
|
|
|
**Basic usage:**
|
|
|
|
```cpp
|
|
#include <torch/torch.h>
|
|
|
|
// Load built-in dataset
|
|
auto dataset = torch::data::datasets::MNIST("./data")
|
|
.map(torch::data::transforms::Normalize<>(0.1307, 0.3081))
|
|
.map(torch::data::transforms::Stack<>());
|
|
|
|
// Create data loader with batching and shuffling
|
|
auto data_loader = torch::data::make_data_loader(
|
|
std::move(dataset),
|
|
torch::data::DataLoaderOptions().batch_size(64).workers(4));
|
|
|
|
// Iterate over batches
|
|
for (auto& batch : *data_loader) {
|
|
auto images = batch.data; // Shape: [64, 1, 28, 28]
|
|
auto labels = batch.target; // Shape: [64]
|
|
}
|
|
```
|
|
|
|
## Header Files
|
|
|
|
- `torch/csrc/api/include/torch/data.h` - Main data header
|
|
- `torch/csrc/api/include/torch/data/dataloader.h` - DataLoader
|
|
- `torch/csrc/api/include/torch/data/datasets.h` - Dataset classes
|
|
- `torch/csrc/api/include/torch/data/samplers.h` - Samplers
|
|
|
|
## Module Categories
|
|
|
|
```{toctree}
|
|
:maxdepth: 1
|
|
|
|
datasets
|
|
dataloader
|
|
samplers
|
|
transforms
|
|
```
|