Paul-Lez / tropicalnn

Repository with code for computing the linear regions of tropical rational functions

Tropical NN

This repository countains some code for a project on tropical geometry and neural networks. The content is organised as follows:

• General purpose code (e.g. the computational framework for Puiseux rational functions, etc) goes in src
• Code for running the paper's experiments goes in paper/experiments
• Code for generating e.g. plots of our data and other paper specific tools should go in paper/tools
• The experimental outputs and data that will be used in the paper should be put in paper/experimental_outcomes
• Other pieces of data generated by experiments/used for experiments should go in data
• Code used for testing should go in tests

Languages used:

• Julia with Oscar. Follow installation instructions here: https://www.oscar-system.org/install/
• Python.
• Matlab.

Instructions for running the experiments

Julia experiments

• Symbolic computations of linear regions of neural networks (section 5.2). Run file experiment_launchers/experiment_5_2_symbolic.jl in Julia using julia experiment_launchers/experiment_5_2_symbolic.jl. The output for this experiment is a .json file containing a dictionary with the followign entries:
  • Linear regions: a dictionary where the entry "Architecture i" is a dictionary storing the average number of linear regions obtained over all sampled neural networks with architecture specified by the i-th entry of the array architectures, and the number of linear regions for each individual sample.
  • Monomials: a dictionary where the entry "Architecture i" is a dictionary storing the average number of monomials obtained over all sampled neural networks with architecture specified by the i-th entry of the array architectures, and the number of monomials for each individual sample.
  • Compute times: a dictionary where the entry "Architecture i" is a dictionary storing the average runtime for the computation over all sampled neural networks with architecture specified by the i-th entry of the array architectures, and the runtimes for each individual sample.
  • input: the array architectures from the lauch file containing the architectures that were used as inputs in the experiment.
  • time: the time each experiment took.
• Linear regions for rational functions (section 5.1). Run experiment_5_1_1.jl (computation for 3 variables) and experiment_5_1_2.jl (computation for 4 variables) in Julia using julia experiment_launchers/experiment_5_1_1.jl and julia experiment_launchers/experiment_5_1_1.jl. For each of these, the output is a .json file containing a dictionary with the following entries:
  • Number of linear regions: a dictionary where the entry "Computation i" is a tuple. The first entry of this tuple is the average number of regions attained for the samples whose number of monomial is given by the i-th entry of the array n_terms (the input data). The second entry of this tuple is an array containing the outputs for the individual computations.
  • compute times: a dictionary where the entry "Computation i" is a tuple. The first entry of this tuple is the average compute time for the samples whose number of monomial is given by the i-th entry of the array n_terms. The second entry of this tuple is an array containing the compute times for the individual computations.
  • input: the array n_terms which stores the number of monomials we are allowing for each computation.
  • time: the time each experiment took.

Python experiment

• Numerical computation of linear regions of neural networks (section 5.2). Run the Jupyter notebook experiment_launchers/deep_set_experiments.ipynb.

Matlab experiments

The MATLAB code for computing and estimating Hoffman constants of tropical Puiseux rational maps is based on the MATLAB code and scripts from https://www.andrew.cmu.edu/user/jfp/hoffman.html The MATLAB code requires the Optimization Toolbox is needed. A version of MATLAB 2023a or higher is recommended. The computation of the Hoffman constant, estimation of lower/upper bounds, comparison of computational time is bundled in function ```trop_test'''. To run the function, in MATLAB console do trop_test(m_p,m_q,n) where `m_p` is the number of monomials in the numerator, `m_q` is the number of monomials in the denominator, and `n` is the dimension of variables. To display the results, in MATLAB console do `disp_results` All results shown in the paper are stored in `.mat` files and can be shown by `disp_results` function.