A streamlit application that helps calculate relevant statistics before, during, and after an experiment is conducted.
This tool is designed to evaluate the statistical significance of differences in either proportions or means, suitable for one-tailed or two-tailed tests. It supports decision-making in various fields such as marketing, clinical trials, or education by providing a clear indication of whether observed differences in data are statistically significant.
For experiments involving rates or percentages:
python sig_test.py --test_type proportion --tail two --test_value 0.507 --control_value 0.4728 --n_test 25000 --n_control 25000 --confidence 0.95For experiments involving continuous outcomes:
python sig_test.py --test_type mean --tail one --test_value 50 --control_value 45 --std_test 10 --std_control 10 --n_test 25000 --n_control 25000 --confidence 0.90--test_type: Specify the type of data: 'proportion' for rates or percentages, 'mean' for continuous outcomes.--tail: Specify the type of test: 'one' for a one-tailed test or 'two' for a two-tailed test.--test_value: Value observed in the test group (proportion or mean).--control_value: Value observed in the control group (proportion or mean).--std_test: Standard deviation of the test group (required for mean type tests).--std_control: Standard deviation of the control group (required for mean type tests).--n_test: Sample size for the test group.--n_control: Sample size for the control group.--confidence: Confidence level for the test, typically set at 0.95.
After running the tool, it will print a detailed report, including:
Results:
----------------------------
Test Type: Proportion Test
Tail Type: Two-tailed
Test Group Value: 0.507
Control Group Value: 0.4728
Test Group Size: 25000
Control Group Size: 25000
Confidence Level: 0.95
Z-Score: 2.556
P-Value: 0.0107
Significant: Yes
This tool is designed to calculate the necessary sample size for a statistical experiment to detect differences in either proportions or means, suitable for one-tailed or two-tailed tests. It supports decision-making in various fields such as marketing, clinical trials, or education.
For experiments involving rates or percentages:
python samplesize.py --type proportion --tail two --baseline 0.10 --effect_size 0.15 --alpha 0.05 --power 0.8 --split_ratio 0.5For experiments involving continuous outcomes:
python samplesize.py --type mean --tail one --delta 5 --sigma 20 --alpha 0.05 --power 0.8 --split_ratio 0.5--type: Specify the type of data: 'proportion' for rates or percentages, 'mean' for continuous outcomes.--tail: Specify the type of test: 'one' for a one-tailed test or 'two' for a two-tailed test.--delta: The desired difference in means for 'mean' type.--sigma: Standard deviation of the measurements for 'mean' type.--baseline: Baseline value (control group rate) for proportion type experiments.--effect_size: Expected outcome rate in the experimental group for proportion type experiments.--alpha: Significance level (alpha).--power: Statistical power.--split_ratio: The ratio of the sample size allocated to the control group versus the experimental group.
After running the tool, a detailed report will be generated including:
Sample Size Calculation Report
----------------------------
Experiment Type: Proportion Difference
Test Type: Two-tailed
Baseline Proportion: 0.1
Desired Proportion: 0.15
Significance Level (Alpha): 0.05
Statistical Power: 0.8
Control Group Ratio: 0.5
Experimental Group Ratio: 0.5
Total Sample Size Required: 2732
Control Group Sample Size: 1366
Experimental Group Sample Size: 1366