Exception handling in `ic_season.get_published_sens_ref_dir()`

[Massimiliano Lincetto]

It is not clear to me how the exception handling is supposed to work in the following:

flarestack/flarestack/data/icecube/ic_season.py

Lines 58 to 68 in ca3cf40

 def get_published_sens_ref_dir(): 
     try: 
         return ref_dir_7yr, ref_10yr 
     except NameError: 
         logger.error( 
             "No reference sensitivity directory found. " 
             "Please create one at {0}".format( 
                 icecube_dataset_dir + "mirror-7year-PS-sens/" 
             ) 
         ) 
         raise 

From the rest of the module, I see no way a NameError can be caught there, but ref_dir_7yr, ref_10yr will rather have None values.

Once we agree on the desired behaviour I am happy to follow-up on the issue.

[Jannis Necker]

I guess then it should rather be

if isinstance(ref_dir_7yr, typne(None)) or isinstance(ref_10yr, type(None)):
    logger.error("<message>")
else:
    return ref_dir_7yr, ref_10yr 

If you want to fix it feel free to open a pull request. I guess that would be the best way of handling it, no?

[Massimiliano Lincetto]

Probably just using if [...] is None instead of resorting to isinstance() is better. And maybe we still want to raise the error to the calling function. I will take care of it! Thanks for the feedback.

[Massimiliano Lincetto]

To be fair, this is not the only inconsistency. ref_dir_7yr is initialised in:

flarestack/flarestack/data/icecube/ic_season.py

Lines 16 to 23 in 517d44f

 try: 
     icecube_dataset_dir = os.environ["FLARESTACK_DATASET_DIR"] 
  
     if os.path.isdir(icecube_dataset_dir + "mirror-7year-PS-sens/"): 
         ref_dir_7yr = icecube_dataset_dir + "mirror-7year-PS-sens/" 
     logger.info(f"Loading datasets from {icecube_dataset_dir} (local)") 
 except KeyError: 
     icecube_dataset_dir = None 

and immediately overridden by:

flarestack/flarestack/data/icecube/ic_season.py

Lines 25 to 33 in 517d44f

 try: 
     ref_dir_7yr = os.environ["7YR_SENS_REF"] 
 except KeyError: 
     ref_dir_7yr = None 
  
 try: 
     ref_10yr = os.environ["10YR_SENS_REF"] 
 except KeyError: 
     ref_10yr = None 

These two environment variables are not referenced anywhere else in the code so I guess L25-33 can be dropped.

[Jannis Necker]

You are right. Then also again override by

flarestack/flarestack/data/icecube/ic_season.py

Lines 35 to 55 in 517d44f

 if icecube_dataset_dir is None: 
     if host_server == "DESY": 
         icecube_dataset_dir = "/lustre/fs22/group/icecube/data_mirror/" 
         ref_dir_7yr = icecube_dataset_dir + "ref_sensitivity/mirror-7year-PS-sens/" 
         ref_10yr = ( 
             icecube_dataset_dir 
             + "ref_sensitivity/TenYr_E2andE3_sensitivity_and_discpot.npy" 
         ) 
         logger.info(f"Loading datasets from {icecube_dataset_dir} (DESY)") 
     elif host_server == "WIPAC": 
         icecube_dataset_dir = "/data/ana/analyses/" 
         ref_dir_7yr = ( 
             "/data/ana/PointSource/PS/version-002-p01/results/time_integrated_fullsky/" 
         ) 
         ref_10yr = "/data/user/tcarver/skylab_scripts/skylab_trunk/doc/analyses/combined_tracks/TenYr_E2andE3_sensitivity_and_discpot.npy" 
         logger.info(f"Loading datasets from {icecube_dataset_dir} (WIPAC)") 
     else: 
         raise ImportError( 
             "No IceCube data directory found. Run: \n" 
             "export FLARESTACK_DATASET_DIR=/path/to/IceCube/data" 
         ) 

Feel free to think of anything that's more consistent. The important thing is to keep it backwards compatible.

[Massimiliano Lincetto]

I think I will rewrite this part, indeed.

Any objections if I use pathlib? It allows for more robust and elegant handling of paths in comparison to os.path.

[Robert David Stein]

Though I may well have been the one to write this in the first place, I think the code in ic_season lines 25 to 33 should probably just read:

ref_dir_7yr = os.environ.get("7YR_SENS_REF")
ref_10yr = os.environ.get("10YR_SENS_REF")

since that will set the refs to None if the environment variables are not set, rather than raising a KeyError. Basically I think we should never do os.environ["x/y/x"] in the code.

And yeah I agree that Pathlib is a good option, though honestly even os.path would be more robust than + for strings.

[Massimiliano Lincetto]

Though I may well have been the one to write this in the first place, I think the code in ic_season lines 25 to 33 should probably just read:

ref_dir_7yr = os.environ.get("7YR_SENS_REF")
ref_10yr = os.environ.get("10YR_SENS_REF")

Sure, this make sense. I found no other reference in the code or in the documentation to the possibility of passing these as individual environment variables, but we can keep supporting it as it does not cost much.

[Jannis Necker]

Thanks @mlincett! Can I close this now?