Running weather-dl in dry run mode might corrupt the actual data.

[Rahul Mahrsee]

Context:

Running a weather-dl job with the actual config in dry run mode. And some files are already there in the target path.

Two stages of weather-dl pipeline:

Stage A (Fetch->Download Stage):

• Actual fetching & downloading of partition chunks will not take place as in dry run mode it uses FakeClient which just logs the message "Fetching ...." & "Downloading..." .

Stage B (Upload Stage):

• There is no check in the upload stage for dry run mode hence it will corrupt the actual data by uploading an empty temp file to the targeted path.

Furthermore, dry run mode uses TempFileStore() instead of FSStore() hence skipping logic will not work as well.