Skip to content

[Tensorflow/LanguageModelling/BERT] Step is incremented incorrectly in LAMB optimizer #391

Description

@TIXFeniks

Related to Model/Framework(s)
Tensorflow/LanguageModelling/BERT LAMB optimizer

Describe the bug
On this line the step is incremented every time the new variable is updated but it is supposed to be the global step
which leads to incorrect bias correction.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions