To make it simple, I would first note that so-called feature maps (1D, 2D, etc.) is our regular channels. Let’s look at examples: Dropout(): Let’s define 2D input: [[1, 1, 1], [2, 2, 2]]. Dropout will consider every element independently, and may result in something like [[1, 0, 1], [0, 2, 2]] SpatialDropout1D(): In this … Read more
The technical differences have already been shown in the other answer. However the main difference is that nn.Dropout is a torch Module itself which bears some convenience: A short example for illustration of some differences: import torch import torch.nn as nn class Model1(nn.Module): # Model 1 using functional dropout def __init__(self, p=0.0): super().__init__() self.p = … Read more
I suggest taking a look at (the first part of) this paper. Regular dropout is applied on the inputs and/or the outputs, meaning the vertical arrows from x_t and to h_t. In your case, if you add it as an argument to your layer, it will mask the inputs; you can add a Dropout layer … Read more