ValueError: Shape of variable bert/embeddings/LayerNorm/beta:0 ((768,)) doesn't match with shape of tensor bert/embeddings/LayerNorm/beta ([128]) from checkpoint reader.
kusumlata123 opened this issue · comments
when i used model.ckpt from pretrained IndicBERT then i got this error
ile "extract_features.py", line 339, in
tf.compat.v1.app.run()
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "extract_features.py", line 305, in main
for result in estimator.predict(input_fn, yield_single_examples=True):
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3078, in predict
rendezvous.raise_errors()
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/error_handling.py", line 136, in raise_errors
six.reraise(typ, value, traceback)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3072, in predict
yield_single_examples=yield_single_examples):
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 622, in predict
features, None, ModeKeys.PREDICT, self.config)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2857, in _call_model_fn
config)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1149, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3126, in _model_fn
features, labels, is_export_mode=is_export_mode)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 1663, in call_without_tpu
return self._call_model_fn(features, labels, is_export_mode=is_export_mode)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 1994, in _call_model_fn
estimator_spec = self._model_fn(features=features, **kwargs)
File "extract_features.py", line 153, in model_fn
tf.train.init_from_checkpoint(init_checkpoint, assignment_map)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_core/python/training/checkpoint_utils.py", line 291, in init_from_checkpoint
init_from_checkpoint_fn)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 1940, in merge_call
return self._merge_call(merge_fn, args, kwargs)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 1947, in _merge_call
return merge_fn(self._strategy, *args, **kwargs)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_core/python/training/checkpoint_utils.py", line 286, in
ckpt_dir_or_file, assignment_map)
File "/home/dr/anaconda3/envs/hcoref/lib/python2.7/site-packages/tensorflow_core/python/training/checkpoint_utils.py", line 329, in _init_from_checkpoint
tensor_name_in_ckpt, str(variable_map[tensor_name_in_ckpt])
ValueError: Shape of variable bert/embeddings/LayerNorm/beta:0 ((768,)) doesn't match with shape of tensor bert/embeddings/LayerNorm/beta ([128]) from checkpoint reader.
The issue seems to be a shape mismatch of checkpoints ie You are loading checkpoint weights into a model but both have different shapes.
Can you please send us a colab notebook with the steps you followed to reproduce the error and share it with us?
I solved it by myself