C++ API: torch::nn::BatchNorm1d (#28176)

Summary:
Add torch::nn::BatchNorm1d function/module support for the C++ API.
torch::nn::BatchNorm{2,3}d will be added after this PR is merged.

Related Issue: https://github.com/pytorch/pytorch/issues/25883

Reviewer: yf225

I would like to discuss about below items.

* Necessity of `num_batches_tracked` in `BatchNormImplBase`
  * `num_batches_tracked` is needed to calculate `momentum` when we do not feed `momentum` argument in Python API. But in C++ API, `momentum` argument has a default value.
  * `num_batches_tracked` is only used for counting up `BatchNorm1d::foward()` call. I think it is no necessary for user anymore.
* The design of `BatchNorm{1,2,3}dOptions`
  * We have already `BatchNormOptions` used for deprecated `BatchNorm` module. However, it is hard to use it for `BatchNorm{1,2,3}dOptions` because of the arguments disagreement of each modules.
  * In this PR, I introduce `BatchNormOptionsv2` template class for the `BatchNorm{1,2,3}dOptions`. But I'm not sure this design is good or not.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/28176

Differential Revision: D18196843

Pulled By: yf225

fbshipit-source-id: 667e2b5de4150d5776c41b9088c9e6c2ead24cd4
This commit is contained in:
nuka137
2019-10-29 17:25:56 -07:00
committed by Facebook Github Bot
parent 8f1564b8ab
commit cbc234bceb
11 changed files with 341 additions and 37 deletions
+1 -1
View File
@@ -281,7 +281,7 @@ TEST_F(ModuleListTest, PrettyPrintModuleList) {
" (0): torch::nn::Linear(in_features=10, out_features=3, bias=true)\n"
" (1): torch::nn::Conv2d(input_channels=1, output_channels=2, kernel_size=[3, 3], stride=[1, 1])\n"
" (2): torch::nn::Dropout(rate=0.5)\n"
" (3): torch::nn::BatchNorm(features=5, eps=1e-05, momentum=0.1, affine=true, stateful=true)\n"
" (3): torch::nn::BatchNorm(num_features=5, eps=1e-05, momentum=0.1, affine=true, track_running_stats=true)\n"
" (4): torch::nn::Embedding(num_embeddings=4, embedding_dim=10)\n"
" (5): torch::nn::LSTM(input_size=4, hidden_size=5, layers=1, dropout=0)\n"
")");