--dev

[Hala Salameh]

what is the use of --dev?
what does early stopping mean?

[himkt]

As you mentioned, the help comment says dev_file is used to do early stopping.
However, current implementation does not seem to stop training loop by watching metrics on dev data.

dev_file is used to decide whether deep-crf creates a snapshot or not for each epoch, I think.

deep-crf/deepcrf/main.py

Lines 484 to 494 in ff24ecc

 if prev_dev_f < dev_f: 
     logging.info(' [update best model on dev set!]') 
     dev_list = [prev_dev_f, dev_f] 
     dev_str = '       ' + ' => '.join(map(str, dev_list)) 
     logging.info(dev_str) 
     prev_dev_f = dev_f 
  
     # Save model 
     model_filename = save_name + '_epoch' + str(epoch) 
     serializers.save_hdf5(model_filename + '.model', net) 
     serializers.save_hdf5(model_filename + '.state', opt) 

[Hala Salameh]

As you mentioned, the help comment says dev_file is used to do early stopping.
However, current implementation does not seem to stop training loop by watching metrics on dev data.

dev_file is used to decide whether deep-crf creates a snapshot or not for each epoch, I think.

deep-crf/deepcrf/main.py

Lines 484 to 494 in ff24ecc

 if prev_dev_f < dev_f: 
     logging.info(' [update best model on dev set!]') 
     dev_list = [prev_dev_f, dev_f] 
     dev_str = '       ' + ' => '.join(map(str, dev_list)) 
     logging.info(dev_str) 
     prev_dev_f = dev_f 
  
     # Save model 
     model_filename = save_name + '_epoch' + str(epoch) 
     serializers.save_hdf5(model_filename + '.model', net) 
     serializers.save_hdf5(model_filename + '.state', opt) 

the code throws an error if there is no dev_file argument and i dont know what file i should give it so i just give it the same file as the input dataset