Related to Model/Framework(s)
Tensorflow/LanguageModelling/BERT LAMB optimizer
Describe the bug
On this line the step is incremented every time the new variable is updated but it is supposed to be the global step
which leads to incorrect bias correction.
Related to Model/Framework(s)
Tensorflow/LanguageModelling/BERT LAMB optimizer
Describe the bug
On this line the step is incremented every time the new variable is updated but it is supposed to be the global step
which leads to incorrect bias correction.