johnolafenwa / transformer_layer_swap

An experiment demonstrating the layers of a transformers model can be swapped without breaking the model

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Transformer_layer_swap

An experiment demonstrating the layers of a transformers model can be swapped without breaking the model

Open the layer_swap.ipynb notebook to run an example code that swaps the layers of phi1.5 without loss in accuracy

About

An experiment demonstrating the layers of a transformers model can be swapped without breaking the model

License:MIT License


Languages

Language:Jupyter Notebook 100.0%