my python comments

graph and ml
pytorch
prog
ml

Implements\PythonCodes

Git
Tensorflow
- Save model during training
- Keras
Import Data or a module
ToDo

Miscellaneous comments

Git

Convert an existing non-empty directory into a Github repository ref

cd <localdir>
git init
git add .
git commit -m 'message'
git branch -M main
# create a github repository and use its URL in the below command.
git remote add origin <url> # Example: git remote add origin https://github.com/ashkan-abbasi66/vf-PyVisualField.git
git push -u origin main

Shrink .git folder: git gc --aggressive --prune ref.

Tensorflow

Folder: tf-example

number of parameters of tensorflow model here
compute receptive field of a network. here
tensorflow basics: tf_basics.ipynb
some simple train examples: tf_train_examples.ipynb
- A two layer network using pure numpy
- using tensorflow's Gradient Descent optimizer
tf_train_save_restore.ipynb contains examples for saving and restoring models.
- Restore a variable with a different name
perform convolution for spatial filtering with two simple filters tensorflow_filtering.py
comparisson between gpu and cpu computations matrixmult_cpu_versus_gpu.py
control cpu cores or gpu usage control_gpu_cpu.py
How to design and code nn layers:

building_nnlayers_CAN24.py

building_nnlayers_DPED_Resnet.py

Save model during training

sess.run(tf.global_variables_initializer())

# create a saver object
saver=tf.train.Saver(max_to_keep=1000)

# get `checkpoint` file if it is available in the directory `checkpoint_dir`
ckpt=tf.train.get_checkpoint_state(checkpoint_dir) # <<<-------------
if ckpt:
    print('loaded '+ckpt.model_checkpoint_path)
    saver.restore(sess,ckpt.model_checkpoint_path)
for epoch in range(1,N_epochs):

    # if a model is loaded, we can continue training with the loaded model.
    if os.path.isdir("%d"%epoch):
        continue
    
    # Do computations for each epoch
    
    # After each epoch, create a directory & save the model using saver object.
    os.makedirs("%d"%epoch)
    saver.save(sess,"%d/model.ckpt"%epoch)
    saver.save(sess,checkpoint_dir) # <<<------------- 
    
    # it is a good idea to use "%s/%d/model.ckpt"%(checkpoint_dir,epoch) instead of "%d/model.ckpt"%epoch
    
    # [optional] At the end of each epoch, it is a good idea to evaluate the obtained model.

A good structure is:

checkpoint_dir or the output directory
- After last epoch, save the obtained model here.
checkpoint_dir/epoch number/
- some statistics about each epoch.
- obtained validation/test results during training.
- save the obtained model in that epoch.

What do they usually save using saver.save(sess,path)?

- a file named `checkpoint` (may contain CheckpointState proto).
- a file named `model.ckpt.data-00000-of-00001`. - a file named `model.ckpt.index`. - a file named `model.ckpt.meta`. At this time, I don't know much about those files!

Keras

Model - MiniGoogleLeNet

Model - Allconvnet or ALLCNN

Search for Tf-KERAS-*

Import Data or a module

load_dataset.py

load image pairs
load a random subset of patch pairs.

Load a module

module_path='.../module_name.py'
from importlib.machinery import SourceFileLoader
module_name = SourceFileLoader("module_name",module_path).load_module()

Load Numpy data

np.load and np.save => a platform-independent way of saving and loading a Numpy arrays

ToDo

indexing and slicing techniques: indexing.py

my OS Notes - NOT completed

ashkan-abbasi66 / python-comments