Benchmarking Process
antoinecarme opened this issue · comments
Need to run a benchmarking process to review the current state of PyAF.
In as first time, we will see this as a sanity check (correct some bugs here and there ;).
In a second time, a report is generated with performance figures.
sanity check :
The benchmarks M1, M2, M3, M4 , NN3 and NN5 are now working.
Added a new repository PyAF_Benchmarks to store benchamrk data under :
Added an internal benchmark (specific to PyAF) based on 4818 stock values of the following indices:
['aex', 'aord', 'bvsp', 'cac40', 'currency', 'dow_jones', 'eurindex', 'euronext', 'eurostoxx50', 'exch', 'ftse100', 'ftse250', 'ftseall', 'gdaxi', 'ibex35', 'ibexnm', 'ipc', 'kospi', 'mc', 'merval', 'mib30', 'mibtel', 'midex', 'my_test', 'nasdaq', 'nasdaqbio', 'ny100', 'nysecomp', 'nyworldlead', 'othindex', 'smi', 'sp500', 'spmib', 'sse', 'ta100', 'tsx', 'usindex']
sanity check :
These benchmarks allow building models on around 20000 time series from different businesses:
M1 : 1001 series
M2 : 29 series
M3 : 3003 series
M4 : 10000 series
NN3 : 111 series
NN5 : 111 series
YahooStocks : 4818
logs are available under https://github.com/antoinecarme/PyAF_Benchmarks
Added a jupyter notebook to generate benchmark reports
https://github.com/antoinecarme/PyAF_Benchmarks/blob/master/reporting/bench-debrief.ipynb
Added first report generation data :
https://github.com/antoinecarme/PyAF_Benchmarks/tree/master/reporting/data