Files
pytorch/docs/cpp/source/api/data/datasets.md
T
Svetlana Karslioglu 19da3da3d3 Restructure C++ docs (#174096)
The fix-cpp-build branch is a major restructuring that:

- Drops Exhale (which auto-generated API docs but caused OOM in CI) in favor of hand-written RST pages using Breathe directives
- Reorganizes notes into a hierarchical API reference with proper categories
- Adds new content: XPU docs, stable ABI docs, DeviceGuard/Stream docs
- Always runs Doxygen instead of gating on an env var

Pull Request resolved: https://github.com/pytorch/pytorch/pull/174096
Approved by: https://github.com/janeyx99, https://github.com/seemethere, https://github.com/albanD
2026-04-07 17:24:38 +00:00

111 lines
2.2 KiB
Markdown

---
myst:
html_meta:
description: Dataset classes in PyTorch C++ — Dataset, MapDataset, StreamDataset, and built-in datasets like MNIST.
keywords: PyTorch, C++, Dataset, MapDataset, StreamDataset, MNIST, data
---
# Datasets
The dataset abstraction defines how to access individual samples in your data.
All datasets inherit from `Dataset` and must implement `get()` and `size()`.
## Dataset Base Class
```{doxygenclass} torch::data::datasets::Dataset
:members:
:undoc-members:
```
```{doxygenclass} torch::data::datasets::BatchDataset
:members:
:undoc-members:
```
## StatefulDataset
A dataset that manages its own state across batches (e.g., position in a stream).
Unlike `Dataset`, it produces batches directly without external samplers.
```{doxygenclass} torch::data::datasets::StatefulDataset
:members:
:undoc-members:
```
## ChunkDataReader
Interface for reading chunks of data from a data source. Used with
`ChunkDataset` for large-scale data loading.
```{doxygenclass} torch::data::datasets::ChunkDataReader
:members:
:undoc-members:
```
## Custom Dataset Example
```cpp
class CustomDataset : public torch::data::datasets::Dataset<CustomDataset> {
public:
explicit CustomDataset(const std::string& root) {
// Load data from root directory
}
torch::data::Example<> get(size_t index) override {
return {images_[index], labels_[index]};
}
torch::optional<size_t> size() const override {
return images_.size(0);
}
private:
torch::Tensor images_, labels_;
};
```
## MapDataset
```{doxygenclass} torch::data::datasets::MapDataset
:members:
:undoc-members:
```
## ChunkDataset
```{doxygenclass} torch::data::datasets::ChunkDataset
:members:
:undoc-members:
```
## SharedBatchDataset
```{doxygenclass} torch::data::datasets::SharedBatchDataset
:members:
:undoc-members:
```
## Built-in Datasets
### MNIST
```{doxygenclass} torch::data::datasets::MNIST
:members:
:undoc-members:
```
**Example:**
```cpp
auto dataset = torch::data::datasets::MNIST("./data")
.map(torch::data::transforms::Normalize<>(0.1307, 0.3081))
.map(torch::data::transforms::Stack<>());
```
## Example Struct
```{doxygenstruct} torch::data::Example
:members:
:undoc-members:
```