.. dist-utils documentation master file, created by
   sphinx-quickstart on Fri Sep 22 07:05:03 2023.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to torch-dist-utils's documentation!
============================================

Utilities for PyTorch distributed.

.. toctree::
   :maxdepth: 2
   :caption: Contents:

Module torch_dist_utils
-----------------------

Before using ``torch_dist_utils`` functions, you must either call :func:`torch_dist_utils.init_distributed()` or initialize the default process group yourself. :func:`torch_dist_utils.init_distributed()` can be called even if you did not start the script with ``torchrun``: if you did not, it will assume it is the only process and create a process group with a single member.

.. automodule:: torch_dist_utils
   :members:
   :undoc-members:

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`