[C++ API] RNN / GRU / LSTM layer refactoring (#34322)

Summary:
This PR refactors RNN / GRU / LSTM layers in C++ API to exactly match the implementation in Python API.

**BC-breaking changes:**
- Instead of returning `RNNOutput`, RNN / GRU forward method now returns `std::tuple<Tensor, Tensor>`, and LSTM forward method now returns `std::tuple<Tensor, std::tuple<Tensor, Tensor>>`, matching Python API.
- RNN / LSTM / GRU forward method now accepts the same inputs (input tensor and optionally hidden state), matching Python API.
- RNN / LSTM / GRU now has `forward_with_packed_input` method which accepts `PackedSequence` as input and optionally hidden state, matching the `forward(PackedSequence, ...)` variant in Python API.
- In `RNNOptions`
    - `tanh()` / `relu()` / `activation` are removed. Instead, `nonlinearity` is added which takes either `torch::kTanh` or `torch::kReLU`
    - `layers` -> `num_layers`
    - `with_bias` -> `bias`
- In `LSTMOptions`
    - `layers` -> `num_layers`
    - `with_bias` -> `bias`
- In `GRUOptions`
    - `layers` -> `num_layers`
    - `with_bias` -> `bias`

The majority of the changes in this PR focused on refactoring the implementations in `torch/csrc/api/src/nn/modules/rnn.cpp` to match the Python API. RNN tests are then changed to reflected the revised API design.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/34322

Differential Revision: D20311699

Pulled By: yf225

fbshipit-source-id: e2b60fc7bac64367a8434647d74c08568a7b28f7
This commit is contained in:
Will Feng
2020-03-14 12:05:39 -07:00
committed by Facebook GitHub Bot
parent 5710374e4e
commit e23a9dc140
11 changed files with 952 additions and 460 deletions
+9 -1
View File
@@ -43,7 +43,11 @@ TEST(EnumTest, AllEnums) {
torch::enumtype::kBatchMean,
torch::enumtype::kZeros,
torch::enumtype::kBorder,
torch::enumtype::kReflection
torch::enumtype::kReflection,
torch::enumtype::kRNN_TANH,
torch::enumtype::kRNN_RELU,
torch::enumtype::kLSTM,
torch::enumtype::kGRU
> v;
TORCH_ENUM_PRETTY_PRINT_TEST(Linear)
@@ -76,4 +80,8 @@ TEST(EnumTest, AllEnums) {
TORCH_ENUM_PRETTY_PRINT_TEST(Zeros)
TORCH_ENUM_PRETTY_PRINT_TEST(Border)
TORCH_ENUM_PRETTY_PRINT_TEST(Reflection)
TORCH_ENUM_PRETTY_PRINT_TEST(RNN_TANH)
TORCH_ENUM_PRETTY_PRINT_TEST(RNN_RELU)
TORCH_ENUM_PRETTY_PRINT_TEST(LSTM)
TORCH_ENUM_PRETTY_PRINT_TEST(GRU)
}