Invalid Save Path

[Vijay Viswanathan]

Hi dsindex,

Firstly thank you so much for writing your script! It actually works, unlike the official SyntaxNet documentation.

I'm running ./train.sh -v -v and getting the following error"
INFO:tensorflow:Building training network with parameters: feature_sizes: [8 2 3 3] domain_sizes: [2126 5 1701 2008] Traceback (most recent call last): File "/home/ubuntu/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/syntaxnet/parser_eval.py", line 149, in <module> tf.app.run() File "/home/ubuntu/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/external/tf/tensorflow/python/platform/app.py", line 30, in run sys.exit(main(sys.argv)) File "/home/ubuntu/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/syntaxnet/parser_eval.py", line 145, in main Eval(sess, num_actions, feature_sizes, domain_sizes, embedding_dims) File "/home/ubuntu/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/syntaxnet/parser_eval.py", line 98, in Eval parser.saver.restore(sess, FLAGS.model_path) File "/home/ubuntu/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/external/tf/tensorflow/python/training/saver.py", line 1102, in restore raise ValueError("Restore called with invalid save path %s" % save_path) ValueError: Restore called with invalid save path /home/ubuntu/models/syntaxnet/work/UD_English/tmp/syntaxnet-output/brain_pos/greedy/128-0.08-3600-0.9-0/model

For reference, I replaced UD_English/*.conll with shorter files (500 sentences instead of 10000+) so I could train faster, and because my next step is training SyntaxNet on a Twitter treebank (even smaller than 500 sentences for train, tune, and dev).

Thanks in advance for your help!

[Myungchul Shin]

• i checked error messages

vi bazel-bin/syntaxnet/parser_eval.runfiles/syntaxnet/parser_eval.py
...
parser.saver.restore(sess, FLAGS.model_path)
...

vi bazel-bin/syntaxnet/parser_eval.runfiles/external/tf/tensorflow/python/training/saver.py
...
from tensorflow.python.platform import gfile
...
if not gfile.Glob(save_path):
      raise ValueError("Restore called with invalid save path %s" % save_path)
...

• and i tested in my environment

python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from tensorflow.python.platform import gfile
>>> save_path="/path/to/models/syntaxnet/work/UD_English/tmp/syntaxnet-output/brain_pos/greedy/128-0.08-3600-0.9-0/model"
>>> print gfile.Glob(save_path)
['/path/to/models/syntaxnet/work/UD_English/tmp/syntaxnet-output/brain_pos/greedy/128-0.08-3600-0.9-0/model']

so what about yours?

[Vijay Viswanathan]

from tensorflow.python.platform import gfile
save_path="models/syntaxnet/work/UD_English/tmp/syntaxnet-output/brain_pos/greedy/128-0.08-3600-0.9-0/model"
print gfile.Glob(save_path)
[ ]

[Myungchul Shin]

@vijayyak did you omit '/path/to/' from '/path/to/models/syntaxnet/work/UD_English/tmp/syntaxnet-output/brain_pos/greedy/128-0.08-3600-0.9-0/model' for privacy issue? (just guess)
if not, the path is not properly set.

[Jiří Kerner]

I had this problem when my dataset was to small. I did the same thing as vijayyak. Made my dataset smaller in order to test the training process faster. When I used bigger dataset, everything worked fine. Dont know why this happened, it is just what I observed.

[Myungchul Shin]

@jiriker @vijayyak

i got what the problem is.
when i use a small training corpus, train_pos_tagger does not generate model because it can't be reached the time to save model.
so i modified train.sh

function train_pos_tagger {
    ${BINDIR}/parser_trainer \
      --task_context=${CONTEXT} \
      --arg_prefix=brain_pos \
      --compute_lexicon \
      --graph_builder=greedy \
      --training_corpus=training-corpus \
      --tuning_corpus=tuning-corpus \
      --output_path=${TMP_DIR} \
      --batch_size=32 \
      --decay_steps=3600 \
      --hidden_layer_sizes=128 \
      --learning_rate=0.08 \
      --momentum=0.9 \
      --seed=0 \
      --params=${POS_PARAMS} \
      --num_epochs=12 \
      --report_every=100 \
      --checkpoint_every=1000 \
      --logtostderr
}