Skip to content

BERT存在两个严重问题 #86

@yuanyuan19

Description

@yuanyuan19

1.随机替换成词表中的其他词时应该排除[PAD],[CLS],[SEP],[MASK],否则预测模型会学会预测这些词。
2.学习率开大了,始终难以收敛,开1e-6很合适。

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions