Code accompanying the paper "Biologically inspired deep learning model for efficient foveal-peripheral vision" by Hristofor Lukanov, Peter König and Gordon Pipa, Institute of Cognitive science, University of Osnabrück.
- Python 3.6
- tensorflow-gpu 1.14.0
- Keras 2.2.4
- numpy 1.15.4
- SharedArray 3.2.1
- albumentations 0.4.5
- opencv-python 4.2.0.32
- h5py 2.9.0
This repository consists of four main folders:
fov
- contains tools for creating a sampling matrix, as well as precompiled mappings (forward and reverse) for foveationpreprocessing
contains tools for preprocessing the datasetsweights
contains weights for models trained on imagenet and core50utils
contains tools for performing foveation and utilities helpful for training and testing
The dataset needs to be in *.hdf5
format and be named imagenet.hdf5
or core50.hdf5
.
The structure of the hdf5 file is the following:
- Key
train/images
includes the train image set with shape(TOTAL_IMAGES,H,W,C)
inuint8
datatype, where H=W=256 and C=3.TOTAL_IMAGES
is the total number of images in the dataset,H
andW
stand for respectively height and width, andC
stands for color channels. - Key
train/labels
contains the labels for training data for ILSVRC'10 in onehot format.train/labels10
,train/labels50
respectively contain labels with 10 classes or 50 classes for Core50. - Key
test/images
is identical totrain/images
but with the test set. - Keys
test/labels
,test/labels10
andtest/labels50
are identical to their training counterparts but for test data.
The dataset can be downloaded from here. The validation set in ILSVRC 2010 is directly downloaded from the source page.
All images are resized such that the smaller dimension equals 256 pixels and cropped to a 256x256 pixel square images.
Core50 can be downloaded from here.
The test set of Core50 contains the following folders:
s1: o47, o17, o6, o13, o28, o37, o42, o35
s2: o43, o39, o42, o25, o46, o13, o22, o34
s3: o50, o21, o47, o28, o15, o3, o49, o9
s4: o39, o3, o38, o26, o30, o5, o33, o29
s5: o28, o31, o37, o3, o11, o10, o23
s6: o14, o17, o2, o21, o34, o35, o18, o42
s7: o36, o7, o22, o45, o12, o4, o49
s8: o19, o34, o12, o20, o15, o38
s9: o29, o16, o19, o18, o24, o31
s10: o10, o48, o25, o27, o44, o38, o49
s11: o25, o9, o27, o19, o18, o40, o7
All images are resized to 256x256 pixel size.
To train execute:
python main.py --model=[imagenet|core50] --task=train
To test execute:
python main.py --model=[imagenet|core50] --task=test
[to be added]
[to be added]