[onert/training] Add a feature of setting trainable op to circle_plus_gen

[SeungHui Youn]

background

The fine-tuning feature is being introduced to the onert based on this draft ( #12951. thanks to @mbencer).
Since circle+ file is a model file for training, It is natural that circle+ holds a fine-tuning configuration.
And already, metadata schema holds a trainable_ops for fine-tuning.

ONE/runtime/libs/circle-schema/include/circle_traininfo.fbs

Lines 110 to 122 in 2deec39

 table ModelTraining { 
   // Version of the schema. 
   version:uint; 
   // For training 
   optimizer: Optimizer; 
   optimizer_opt: OptimizerOptions; 
   lossfn: LossFn; 
   lossfn_opt: LossFnOptions; 
   epochs: int; 
   batch_size: int; 
   loss_reduction_type : LossReductionType; 
   trainable_ops : [int]; 
 } 

what to do

In order to make easy use of fine-tuning feature, Let's support generating a circle+ file with a trainable_ops.

reference

• #12984

/cc @jyoungyun @mbencer

[SeungHui Youn]

how to do

I'm still thinking about its' interface.
How about this kind of interface?

# set last 2 layers are trainable for input.circle file 
circle_plus_generator/main.py input.circle --fine-tune -2

# set last 2 layers are trainable for input.circle file 
circle_plus_generator/main.py input.circle --train-only -2

[SeungHui Youn]

how to do

# set full training (default, if nothing is given)
circle_plus_generator/main.py input.circle --train all 

# set last 2 layers are trainable for input.circle file 
circle_plus_generator/main.py input.circle --train last2

# not training (might be used for inference) 
circle_plus_generator/main.py input.circle --train none

# select a specific layer to be trained
# hidden feature for developer, not going to write a document about this
circle_plus_generator/main.py input.circle --train 1-5, 7, 9

[chunseoklee]

@zetwhite circle_plus_generator/main.py input.circle --train-last 2 generates trainable_ops : [8,9] in metadata if 10 layers in seq ?

[SeungHui Youn]

@zetwhite circle_plus_generator/main.py input.circle --train-last 2 generates trainable_ops : [8,9] in metadata if 10 layers in seq ?

I thought so.

[SeungHui Youn]

better to have similar interface with #13140 (comment)

+) So, updated interface a bit #13261 (comment)