mirror of
https://github.com/pytorch/pytorch.git
synced 2026-11-20 09:04:25 +08:00
9ae0ecec7d
# Motivation `torch.cuda.mem_get_info` and `torch.xpu.mem_get_info` are widely used in other popular repos, such as - https://github.com/sgl-project/sglang/blob/076313bd099ac1ee484ee77009eaae864eacf396/python/sglang/srt/utils.py#L378, - https://github.com/huggingface/accelerate/blob/7ecc2d7f394fc0686062a18d46128a8bd97c7dad/src/accelerate/utils/modeling.py#L822, - https://github.com/vllm-project/vllm/blob/7ba34b1241ada58f8212f350a8b17382cb412cf2/vllm/worker/worker.py#L150. - This PR introduces a unified API `torch.accelerator.get_memory_info` to cover this scenario. Pull Request resolved: https://github.com/pytorch/pytorch/pull/156812 Approved by: https://github.com/albanD
860 B
860 B
torch.accelerator
.. automodule:: torch.accelerator
.. currentmodule:: torch.accelerator
.. autosummary::
:toctree: generated
:nosignatures:
device_count
is_available
current_accelerator
set_device_index
set_device_idx
current_device_index
current_device_idx
set_stream
current_stream
synchronize
device_index
.. automodule:: torch.accelerator.memory
.. currentmodule:: torch.accelerator.memory
Memory management
.. autosummary::
:toctree: generated
:nosignatures:
empty_cache
get_memory_info
max_memory_allocated
max_memory_reserved
memory_allocated
memory_reserved
memory_stats
reset_accumulated_memory_stats
reset_peak_memory_stats