keras MAC cell
Keras adaptation of the article Compositional Attention Networks for Machine Reasoning. The original TensorFlow code is here.
How to use it
The .zip CLEVR_v1.0.zip can be downloaded from here (18 Gb) or from the official website, and has to be saved in the data folder for the data generator to find everything automatically.
To run the full experiment, run the main.py file. To pick the MAC cell for another task, probably the easiest is to initialize the class completeMACmodel_simple, and retrieve the model as:
MAC = completeMACmodel_simple()
model = MAC.modelto compile and fit a generator or a dataset after.
DONEs and TODOs
Still work in progress, but maybe making it public can help me figure out what is missing.
List of DONEs:
- numpy arrays dimensions pass correctly before fitting
- gradients pass correctly
- training works, but the performance is still low
List of main TODOs:
- the low speed and low performance might be due to a faulty implementation of the generator
- otherwise figure out what's the source of the low performance
- ResNet50 to ResNet101