Question: Using SHAP with GPT-4 via API

[Marco Diomedi]

Good evening,

I'm trying to use SHAP for my graduation thesis with various LLM, as Falcon, Mistral and GPT 4 and Llama, and i'm trying to follow the instruction with GPT-2 as in documentation but it's not possible to apply it to GPT-4, is there any solution to obtain Shapley Values with GPT-4 using SHAP?

I put an example of my code

import json
from langchain.chat_models import ChatOpenAI
import shap

with open('path/to/apikey.json', 'r') as file_json:
dati_json = json.load(file_json)
openai.api_key = dati_json['OPENAI_API_KEY']

chat = ChatOpenAI(temperature=0.0, openai_api_key =dati_json["OPENAI_API_KEY"] )

review_template = ["I enjoy walking with my cute dog"]
explainer = shap.Explainer(chat)
shap_values = explainer(t)

Thanks in advance!

[Tobias Pitters]

No, there is no possibility to extract shap values from models that are extracted through APIs since one typically just gets the model outputs (and not even the tokenization or something) from these models.

[Tobias Pitters]

Seems like no discussion is getting on here and as already said, this cannot be implemented.

[gabriel chua]

Is there a way to leverage the existing SHAP API/examples?

The OpenAI models do return the log_probs for up to the top 20 tokens at each generation.

completion = client.chat.completions.create(
  model="gpt-4o-mini",
  messages=[
    {"role": "user", "content": "Hello world!"}
  ],
  logprobs=True,
  top_logprobs=20
)