MLMD Model Card add colab example with more information

[Marc Ferradou]

Expected Behavior

We should have a colab that can display the model architecture (model.summary() if using keras) as part of the model card.
In addition tensorflow data validation also output general statistics see screenshot:

Finally, It would be nice to show an example where we connect some tensorboard chart (ie. loss chart).

Actual Behavior

No example

Steps to Reproduce the Problem

This is a feature request not a bug

[Suzen Fylke]

Hi @zippeurfou, I'm not sure I fully understand your suggestion. Is the idea to create a notebook on Colab then link to it from the model card? Or perhaps to create a notebook that's in itself a model card and contains interactive components? Or something else?

[Marc Ferradou]

Thanks for getting back to me.
This is the latter.
The idea is to add more tensorflow components as mentioned before in the model card and create an example or tutorial explaining how to do so in a reproducible Google colab.

[Marc Ferradou]

@codesue here is some sample code that can display what I mean.
Assuming keras:
Add model architecture:

from tf.keras.utils import plot_model
import io
model = get_model(...)
model_plot_fn = 'model.png'
def get_model_architecture():
    # Plot the model and save it to a file
    plot_model(model, to_file=model_plot_fn, show_shapes=True, show_layer_names=True)

    # Encode the image as base64
    with open(model_plot_fn, 'rb') as img_file:
        base64_image = base64.b64encode(img_file.read()).decode('utf-8')
    return base64_image


def get_model_param():
    buffer = io.StringIO()
    model.summary(print_fn=lambda x: buffer.write(x + '\n\n'))
    model_summary = buffer.getvalue()

    # Close the buffer to free up resources
    buffer.close()
    return model_summary

base64_image = get_model_architecture()
model_summary = get_model_param()
model_output = create_model_card()
model_output.model_parameters.data[0].graphics.collection.insert(0,mctlib.Graphic(name='Model architecture', image=base64_image)) 
model_output.model_parameters.model_architecture = model_summary
mct.update_model_card(model_output)

Of course a lot more can be done but this is just an example.

[Suzen Fylke]

Hi @zippeurfou, I like the general ideas here! I think we could flesh out the design a bit more and confirm whether the tfdv and tensorboard widgets would work outside of a notebook. Would you be willing to contribute these features?

[Marc Ferradou]

thanks @codesue

Would you be willing to contribute these features?

To be frank it depends, I have limited availabilities but I do feel like this would be a great addition.
One main reason is to follow metaflow model card where they create a model card by execution.
We have everything (with tfx) to allow us to compare production runs within the model card and I think this would be really helpful.
Starting with including Tensorboard information would be fantastic.