List of common errors that people do when they train NNs.
[Dropout Usage] remember to use dropout only during training! (and remember to scale by the dropout probaility during training, see the original paper here). There are also works that use dropout also at test time, but this is not done for the same reason as the initial idea of dropout (regularization, ...)
[Batch Norm Usage] remeber to set bias=False on the layer just before the batch normalization: the bias is alredy inclueded in the batch norm computation.
[PyTorch Train vs Eval mode]: remeber to call model.train() before training and model.eval() before validation/test.