suggestion: display base64 content as image when appropriate

[Benjamin RAIBAUD]

It would be great to display base64 (returned from kernel I believe) as images so (for example) that matplolib chart could be displayed in the chat as well.

If what's returned from kernel is a valid image once base64 decoded, then the message.kind should be set to image and handled appropriatly in chat.tsx.

I just don't know react well enough to contribute.

[Rick Lamers]

That should work! Maybe someone else can help :)

https://stackoverflow.com/a/42399865

Perhaps this example can serve as a starting point!

[stefw]

Same to me. but... > Just ask to GPT to return image in JPG or PNG format, and it works well.

Thanks to the "Diesel_price" column, find the 5 "City" with a "Zip code" that starts with "59", with the cheapest "Diesel_price". Displays the results as a bar graph, **in JPG format**, with the cities on the abscissa and the prices on the ordinate

import pandas as pd
import matplotlib.pyplot as plt

# Télécharge le fichier CSV
url = "https://data.economie.gouv.fr/api/explore/v2.1/catalog/datasets/prix-des-carburants-en-france-flux-instantane-v2/exports/csv?lang=fr&timezone=Europe%2FParis&use_labels=true&delimiter=%3B"
data = pd.read_csv(url, sep=';')

# Filtre les données avec un code postal commençant par 59
data_59 = data[data['Code postal'].astype(str).str.startswith('59')]

# Trouve les 5 villes avec le Gazole le moins cher
data_59_sorted = data_59.sort_values(by='Gazole_prix').head(5)

# Crée un graphique à barres avec les villes en abscisse et les prix en ordonnée
plt.figure(figsize=(10, 5))
plt.bar(data_59_sorted['Ville'], data_59_sorted['Gazole_prix'])
plt.xlabel('Villes')
plt.ylabel('Prix du Gazole')
plt.title('Top 5 des villes avec le Gazole le moins cher (Code postal 59)')

# Sauvegarde le graphique en format JPG
plt.savefig('gazole_prix_59.jpg')

# Affiche le lien de téléchargement
print("<a href='/download?file=gazole_prix_59.jpg'>Download file</a>")

And gpt-code-ui generates well the image

[Benjamin RAIBAUD]

@stefw, there is no guarantee that GPT will produce code that creates that type of output consistently (e.g. using the same prompt, I did not get the last line of code, resulting in a base64 output once again).
The type of output returned by the kernel should be identified and processed accordingly.

[Rick Lamers]

Had some spare time. This is in v0.42.14 and up.