Files
pytorch/docs/source
Mikayla Gawarecki 487b6d40ec Add RMSNorm module (#121364)
Similar to https://github.com/facebookresearch/multimodal/blob/dbeed9724bc9099be173c86871df7d41f3b7e58c/torchmultimodal/modules/layers/normalizations.py#L51

**The implementation here is not optimized and we welcome pull requests to improve this**

- Use `normalized_shape` instead of singular integer `dim` to be aligned with the `nn.LayerNorm` implementation
- Remove the [upcast to float and downcast
](https://github.com/facebookresearch/multimodal/blob/dbeed9724bc9099be173c86871df7d41f3b7e58c/torchmultimodal/modules/layers/normalizations.py#L73)

Differential Revision: [](https://our.internmc.facebook.com/intern/diff/)

Differential Revision: [D55485840](https://our.internmc.facebook.com/intern/diff/D55485840)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/121364
Approved by: https://github.com/albanD
2024-03-29 18:05:28 +00:00
..
…
2024-01-29 23:42:35 +00:00
2024-01-29 21:51:33 +00:00
2024-03-15 20:35:55 +00:00
…
2023-02-09 15:45:08 +00:00
2023-01-23 18:07:22 +00:00
2023-10-06 14:16:01 +00:00
2023-10-06 14:16:01 +00:00
2023-10-06 14:16:01 +00:00
2024-03-29 18:05:28 +00:00
2024-03-29 18:05:28 +00:00
2023-10-06 14:16:01 +00:00
2023-10-06 14:16:01 +00:00
2024-03-19 00:17:19 +00:00
2024-03-15 19:49:37 +00:00
2023-12-08 18:49:35 +00:00