IPL-UV / latentNoise_krr

latentNoise

Latent Noise imputation bivariate causal discovery method (LNc, ANMh & LNc-ANMh) from "On Additivity and Model Complexity for Causal Discovery Manuscript".

Main Function files

1. src/latentNoise_funcs_gen.py: contains functions to estimate latent noise, fit KRR and calculate complexity measures
2. src/processResults.py: contains functions to pair up and weight models and to calculate performance measures

Additional function files

1. src/ANLSMN_genfuncs.py: generate artificial data from AN, AN-s, LS, LS-s and MN-U classes
2. src/slope.py: SLOPE bivariate causal algorithm (from Marx & Vreeken 2019)
3. src/func_entropy_v1: different entropy estimation methods.
4. src/kdpee: kdp entropy estimation c files called from func_entropy_v1

Main script files

1. src/slurm_script.py: load data, carry out one LNc experiment and save.
2. src/experiment.py: carry out one LNc experiment (called by slurm_script.py)
3. src/slurm_script_van.py: load data, carry out one ANMh experiment and save
4. src/experiment_van.py: carry one ANMh experiment (called by slurm_script_van.py)
5. src/slurm_pairUp.py: read in consolidated LNc database and pair-up models by chosen pair-up strategies
6. src/slurm_acc.py: read in cpaired-up database, calculate model-pair weights by chosen weighting strategies, calculate decision and accuracy results

Other script files

1. src/null_hsicRX_script.py: obtain one null statistic for additivity test using AN-s additive class simulation

Main slurm files

1. src/latent_job.sh: carry out all LNc experiments by calling slurm_script.py for all parameters
2. src/latent_job_van.sh: carry out all ANMh experiments by calling slurm_script_van.py for all parameters
3. src/postProcess_job.sh: carry out pairing-up, weighting and performance calculation by calling either slurm_pairUp.py or slurm_acc.py

Other slurm files

1. src/null_dists.sh: obtain a null distribution for additivity test by calling null_hsicRX_script.py a few times.

Notebooks

1. notebooks/consolidate_results.ipynb: read in results generated with slurm_script.latent_job.sh & latent_job_van.sh, consolidate and save in respective pickle files to be read in by slurm_pairUp.py and slurm_acc.py. Also calculates additivity p-values so nullDists.ipynb must have been run previously.
2. notebooks/pipeline_results.ipynb: read in different pairUp and weighing strategy results generated by postProcess_job.sh and explore results.
3. notebooks/combineDecisions.ipynb: read in chosen pairUp and weighing strategy, combine LNc and ANMh results and format LNc and LNc-ANMh results
4. notebooks/nullDists.ipynb: read in results generated with null_dists.sh and save additivity null distribution in pickle file to be read in in consolidate_results.ipynb

Data Files

1. data.zip: contains data for 1002 bivariate datasets (Tub, SIM, SIMc, SIMG, SIMln, AN, AN-s, LS, LS-s, MN-U)