torch入门笔记20:torch实现freeze layer的操作

You can set the learning rate of certain layers to zero by overriding their updateParameters and accGradParameters to zero. You don't necessarily have to subclass, but it is cleaner.

If you are building the model for the first time, you simply get the initialized layer's handle and overrride the mentioned functions.

If you have a pre-trained network, and you want to freeze certain layers, you can either use the generic :apply function, that applies a closure to each layer, such as:

https://gist.github.com/soumith/5010de75f7a6805d33c9

Or you can traverse them via traversing the model's .modules table:

https://gist.github.com/soumith/b910efc4dd559736a9b0

Finally, here's an example freezing the first 3 layers of a net:

https://gist.github.com/soumith/6cd0f9b8462d0507a91b