Giters

mozilla / DeepSpeech

DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Positional arguments error during evaluation

kleekaai opened this issue · comments

commented

I am trying to train and evaluate deepspeech model on my custom data. I am able to complete the train step but get error during the evaluate process.

  • Have I written custom code (as opposed to running examples on an unmodified clone of the repository):
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu
  • TensorFlow installed from (our builds, or upstream TensorFlow): pip
  • TensorFlow version (use command below): 1.14.0
  • Python version: 3.7.10
  • Bazel version (if compiling from source):
  • GCC/Compiler version (if compiling from source):
  • CUDA/cuDNN version: cuda_11.0_bu.TC445_37.28845127_0
  • GPU model and memory: 16 gb
  • Exact command to reproduce:
  • deepspeech version: 0.6.0
/content/DeepSpeech
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/usr/local/lib/python3.7/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/bin/sh: 1: sox: not found
SoX could not be found!

    If you do not have SoX, proceed here:
     - - - http://sox.sourceforge.net/ - - -

    If you do (or think that you should) have SoX, double-check your
    path variables.
    
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py:494: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2.
    - tf.py_function takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    - tf.numpy_function maintains the semantics of the deprecated tf.py_func
    (it is not differentiable, and manipulates numpy arrays). It drops the
    stateful argument making all functions stateful.
    
W0318 14:14:25.379205 140604048189312 deprecation.py:323] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py:494: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2.
    - tf.py_function takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    - tf.numpy_function maintains the semantics of the deprecated tf.py_func
    (it is not differentiable, and manipulates numpy arrays). It drops the
    stateful argument making all functions stateful.
    
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/iterator_ops.py:348: Iterator.output_types (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_types(iterator)`.
W0318 14:14:25.452815 140604048189312 deprecation.py:323] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/iterator_ops.py:348: Iterator.output_types (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_types(iterator)`.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/iterator_ops.py:349: Iterator.output_shapes (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_shapes(iterator)`.
W0318 14:14:25.453055 140604048189312 deprecation.py:323] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/iterator_ops.py:349: Iterator.output_shapes (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_shapes(iterator)`.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/iterator_ops.py:351: Iterator.output_classes (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_classes(iterator)`.
W0318 14:14:25.453202 140604048189312 deprecation.py:323] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/iterator_ops.py:351: Iterator.output_classes (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_classes(iterator)`.
I0318 14:14:26.814549 140604048189312 utils.py:157] NumExpr defaulting to 2 threads.
WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

W0318 14:14:27.066528 140604048189312 lazy_loader.py:50] 
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/contrib/cudnn_rnn/python/layers/cudnn_rnn.py:342: calling GlorotUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0318 14:14:27.068302 140604048189312 deprecation.py:506] From /usr/local/lib/python3.7/dist-packages/tensorflow/contrib/cudnn_rnn/python/layers/cudnn_rnn.py:342: calling GlorotUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0318 14:14:27.068591 140604048189312 deprecation.py:506] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/contrib/cudnn_rnn/python/layers/cudnn_rnn.py:345: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0318 14:14:27.068861 140604048189312 deprecation.py:506] From /usr/local/lib/python3.7/dist-packages/tensorflow/contrib/cudnn_rnn/python/layers/cudnn_rnn.py:345: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:Entity <bound method _CudnnRNN.call of <tensorflow.contrib.cudnn_rnn.python.layers.cudnn_rnn.CudnnLSTM object at 0x7fe0811d5e50>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method _CudnnRNN.call of <tensorflow.contrib.cudnn_rnn.python.layers.cudnn_rnn.CudnnLSTM object at 0x7fe0811d5e50>>: AttributeError: module 'gast' has no attribute 'Num'
W0318 14:14:27.137216 140604048189312 ag_logging.py:145] Entity <bound method _CudnnRNN.call of <tensorflow.contrib.cudnn_rnn.python.layers.cudnn_rnn.CudnnLSTM object at 0x7fe0811d5e50>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method _CudnnRNN.call of <tensorflow.contrib.cudnn_rnn.python.layers.cudnn_rnn.CudnnLSTM object at 0x7fe0811d5e50>>: AttributeError: module 'gast' has no attribute 'Num'
WARNING:tensorflow:From DeepSpeech.py:234: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W0318 14:14:27.188672 140604048189312 deprecation.py:323] From DeepSpeech.py:234: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/training/slot_creator.py:193: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
W0318 14:14:27.613036 140604048189312 deprecation.py:323] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/training/slot_creator.py:193: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
I Initializing variables...
I STARTING Optimization
Epoch 0 |   Training | Elapsed Time: 0:00:28 | Steps: 9 | Loss: 314.592078      
Epoch 0 | Validation | Elapsed Time: 0:00:08 | Steps: 3 | Loss: 257.292587 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 257.292587 to: /content/model_checkpoints/best_dev-9
Epoch 1 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 251.725584      
Epoch 1 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 245.925207 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/training/saver.py:960: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to delete files with this prefix.
W0318 14:15:42.440679 140604048189312 deprecation.py:323] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/training/saver.py:960: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to delete files with this prefix.
I Saved new best validating model with loss 245.925207 to: /content/model_checkpoints/best_dev-18
Epoch 2 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 246.932817      
Epoch 2 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 243.106898 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 243.106898 to: /content/model_checkpoints/best_dev-27
Epoch 3 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 245.290102      
Epoch 3 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 241.580933 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 241.580933 to: /content/model_checkpoints/best_dev-36
Epoch 4 |   Training | Elapsed Time: 0:00:05 | Steps: 9 | Loss: 244.127940      
Epoch 4 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 240.559397 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 240.559397 to: /content/model_checkpoints/best_dev-45
Epoch 5 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 243.274753      
Epoch 5 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 240.224253 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 240.224253 to: /content/model_checkpoints/best_dev-54
Epoch 6 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 242.502513      
Epoch 6 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 239.868525 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 239.868525 to: /content/model_checkpoints/best_dev-63
Epoch 7 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 241.796366      
Epoch 7 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 239.239080 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 239.239080 to: /content/model_checkpoints/best_dev-72
Epoch 8 |   Training | Elapsed Time: 0:00:07 | Steps: 9 | Loss: 240.841281      
Epoch 8 | Validation | Elapsed Time: 0:00:01 | Steps: 3 | Loss: 238.824458 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 238.824458 to: /content/model_checkpoints/best_dev-81
Epoch 9 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 239.520710      
Epoch 9 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 237.998851 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 237.998851 to: /content/model_checkpoints/best_dev-90
Epoch 10 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 238.024426     
Epoch 10 | Validation | Elapsed Time: 0:00:01 | Steps: 3 | Loss: 237.242701 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 237.242701 to: /content/model_checkpoints/best_dev-99
Epoch 11 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 236.040066     
Epoch 11 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 236.218455 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 236.218455 to: /content/model_checkpoints/best_dev-108
Epoch 12 |   Training | Elapsed Time: 0:00:06 | Steps: 9 | Loss: 233.862506     
Epoch 12 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 235.194346 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 235.194346 to: /content/model_checkpoints/best_dev-117
Epoch 13 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 230.941633     
Epoch 13 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 233.699132 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 233.699132 to: /content/model_checkpoints/best_dev-126
Epoch 14 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 227.210553     
Epoch 14 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 233.728556 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
Epoch 15 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 223.478024     
Epoch 15 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 232.976186 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 232.976186 to: /content/model_checkpoints/best_dev-144
Epoch 16 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 219.291668     
Epoch 16 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 230.690994 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Saved new best validating model with loss 230.690994 to: /content/model_checkpoints/best_dev-153
Epoch 17 |   Training | Elapsed Time: 0:00:04 | Steps: 9 | Loss: 214.615779     
Epoch 17 | Validation | Elapsed Time: 0:00:00 | Steps: 3 | Loss: 234.124181 | Dataset: /content/drive/MyDrive/training_csvs/knee_society_data/dev.csv
I Early stop triggered as (for last 4 steps) validation loss: 234.124181 with standard deviation: 1.291637 and mean: 232.465246
I FINISHED optimization in 0:09:55.027340
Traceback (most recent call last):
  File "DeepSpeech.py", line 965, in <module>
    absl.app.run(main)
  File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "DeepSpeech.py", line 942, in main
    test()
  File "DeepSpeech.py", line 675, in test
    samples = evaluate(FLAGS.test_files.split(','), create_model, try_loading)
  File "/content/DeepSpeech/evaluate.py", line 48, in evaluate
    Config.alphabet)
TypeError: __init__() takes from 1 to 5 positional arguments but 6 were given

I am adding the evaluate function here. The error is coming from the line. It contains five arguments. Can anyone look into this issue.

def evaluate(test_csvs, create_model, try_loading):
    if FLAGS.lm_binary_path:
        scorer = Scorer(FLAGS.lm_alpha, FLAGS.lm_beta,
                        FLAGS.lm_binary_path, FLAGS.lm_trie_path,
                        Config.alphabet)
    else:
        scorer = None
commented

Please use discourse for support request. Also, deepspeech 0.6.0 is quite old.