[BUG] No such file or directory
Farhad-Heybati opened this issue · comments
Description
tc_bert_azureml.ipynb
Error message:
wc: /mnt/batch/tasks/shared/LS_root/jobs/gaia-ml-wks/azureml/3576038c-f399-4522-8674-383ad7cd316b/mounts/workspaceblobstore/azureml/88f85fcf-eaf4-4d58-bff2-712b35cb50aa/train0,: No such file or directory
How do we replicate the bug?
Just run the notebook tc_bert_azureml
Expected behavior (i.e. solution)
Other Comments
I think the issue is coming from the fact that the output of preprocessing is defined by PipelineData:
output_data = PipelineData(name="train{}".format(str(i)), datastore=ds,
output_path_on_compute='mnli_data/processed_train/batch{}.csv'.format(str(i)))
By default, the path to the data file is based on an environment variable: $AZUREML_DATAREFERENCE
for example:
processed_train_files[0] => $AZUREML_DATAREFERENCE_train0
When I print the value of the file path in preprocessing step and in the training step, these are differents:
preprocess step:
/mnt/batch/tasks/shared/LS_root/jobs/gaia-ml-wks/azureml/273296fe-21e3-44fa-84d1-6a6b33c04031/mounts/workspaceblobstore/azureml/273296fe-21e3-44fa-84d1-6a6b33c04031/train0'
Bert training step:
/mnt/batch/tasks/shared/LS_root/jobs/gaia-ml-wks/azureml/51dd8b67-13a7-40a1-a9ff-eef158a300c5/mounts/workspaceblobstore/azureml/273296fe-21e3-44fa-84d1-6a6b33c04031/train0
The $AZUREML_DATAREFERENCE value change based on the runID