fel-thomas / Sobol-Attribution-Method

👋 Code for the paper: "Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis" (NeurIPS 2021)

Home Page:https://github.com/deel-ai/xplique

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

question about sampler

abcde-1447 opened this issue · comments

Hi,

Thanks for this interesting work!

I have been reading the code, and I am curious about some details of the sampler:

  1. in https://github.com/fel-thomas/Sobol-Attribution-Method/blob/main/sobol_attribution_method/sampling.py#L98-L99, is there any specific reason why we set scramble=False? I tested scramble=True, and it seems to be able to give correct results, so I am wondering whether there are other considerations?

  2. can we replace sampling function with any random number generator, such as np.random.random? Is the main purpose of QMC to generate relatively evenly distributed data?

Thank you!

Hello ;)

1 - this is indeed a good question, I guess I should be able to modify the code to allow scrambling. On my side, I haven't seen better results (Deletion, Insertion). Moreover, with scrambling the convergence of the estimator to a final state (e.g n=8, n=16, n=32...) is slower because the points are not deterministic anymore.
Nevertheless I agree that several papers note the beneficial side of scrambling the elements: On the Scrambled Sobol Sequence.

2 - Yep you can create your own purely monte-carlo sampler, and yes you're right again, the goal of QMC is to have a "better covering" of the manifold. This is also something I could add.

Don't hesitate if you have more questions

3603a2d Allow to scramble Halton & Sobol, 33e8529 introduce a pure Monte-Carlo sampling :)

@fel-thomas That makes a lot of sense, thanks for the explanation! I did some experiments and yes pure MC converges more slowly.