Since my issue was closed and wasn't reopen when I comment:
I followed the steps described in the quickstart in Bert's Readme:
[(https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT#quick-start-guide.)]
After a long preprocessing fight , I finally got to pertaining part I used this line:
bash scripts/run_pretraining_lamb.sh 64 8 8 7.5e-4 5e-4 fp32 true 8 2000 200 7820 100 128 512 large
I'm getting two errors;
Traceback (most recent call last):
File "/workspace/bert/run_pretraining.py", line 30, in
import utils.dllogger_class
ModuleNotFoundError: No module named 'utils.dllogger_class'
I tried to google it but I got only java stuff..
And :
Error! /results/tf_bert_pretraining_lamb_large_fp32_gbs165536_gbs232768_200514070735/phase_1/model.ckpt-7038.meta directory missing. Please mount correctly
of course that directory :/results/tf_bert_pretraining_lamb_large_fp32_gbs165536_gbs232768_200514070735/phase_1/
does exists.
The fix:
#502
Did not solve my problem.
To be sure, I even re-cloned the repo.
Still getting the same errors.
I'm using 8 V100 (not DGX) with 16gb.
The container is as requested.
**tf version:**1.14
I'll be happy for your help.
Thanks.
Since my issue was closed and wasn't reopen when I comment:
I followed the steps described in the quickstart in Bert's Readme:
[(https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT#quick-start-guide.)]
After a long preprocessing fight , I finally got to pertaining part I used this line:
bash scripts/run_pretraining_lamb.sh 64 8 8 7.5e-4 5e-4 fp32 true 8 2000 200 7820 100 128 512 largeI'm getting two errors;
I tried to google it but I got only java stuff..
And :
of course that directory :/results/tf_bert_pretraining_lamb_large_fp32_gbs165536_gbs232768_200514070735/phase_1/
does exists.
The fix:
#502
Did not solve my problem.
To be sure, I even re-cloned the repo.
Still getting the same errors.
I'm using 8 V100 (not DGX) with 16gb.
The container is as requested.
**tf version:**1.14
I'll be happy for your help.
Thanks.