A few additions (#9837)

Summary:
This PR provides 4 fixes / features:

1. torch::nn::Cloneable inherits virtually from torch::nn::Module. We want to pass around a module with new functions, and the best way to do this is to do a diamond inheritance pattern, i.e.

```c++
struct MySuperModuleImpl : virtual public torch::nn::Module {
  virtual void myFunction() = 0;
}

struct MySuperModule : public torch::nn::Cloneable<MySuperModule>, MySuperModuleImple {};

struct MyModule : public MySuperModule<MyModule> {
  void myFunction() override;
};
```

This way, we can simply pass around MySuperModuleImpl around instead of torch::nn::Module.

2. Optimizer options are public now, since there's no way to decay the LR or modify it during training otherwise
3. Serialization functions creates autograd history and calls copy_! Bad!
4. Optimizers did not create buffers after add_parameters was called.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/9837

Reviewed By: goldsborough

Differential Revision: D9199746

Pulled By: ebetica

fbshipit-source-id: 76d6b22e589a42637b7cc0b5bcd3c6b6662fb299
This commit is contained in:
Zeming Lin
2018-08-13 10:11:45 -07:00
committed by Facebook Github Bot
parent 0a39a9cfbc
commit b8530dc1f0
21 changed files with 141 additions and 120 deletions
+2 -2
View File
@@ -278,7 +278,7 @@ TEST_CASE("sequential") {
SECTION("Is cloneable") {
Sequential sequential(Linear(3, 4), Functional(torch::relu), BatchNorm(3));
Sequential clone =
std::static_pointer_cast<SequentialImpl>(sequential->clone());
std::dynamic_pointer_cast<SequentialImpl>(sequential->clone());
REQUIRE(sequential->size() == clone->size());
for (size_t i = 0; i < sequential->size(); ++i) {
@@ -309,7 +309,7 @@ TEST_CASE("sequential/clone-to-device", "[cuda]") {
Sequential sequential(Linear(3, 4), Functional(torch::relu), BatchNorm(3));
torch::Device device(torch::kCUDA, 0);
Sequential clone =
std::static_pointer_cast<SequentialImpl>(sequential->clone(device));
std::dynamic_pointer_cast<SequentialImpl>(sequential->clone(device));
for (const auto& p : clone->parameters()) {
REQUIRE(p->device() == device);
}