Integrating Custom Callback to Output Agent Steps in Recode Format

[YAMON.IO]

I would like to incorporate a custom callback into the ToolCallingAgentComponent to output the agent's steps in a Recode format. Below is my current source code. Could you guide me on which part to modify?

def print_callback(step_output):
    print(f"Step: {step_output.step}")
    print(f"Action: {step_output.action}")
    print(f"Observation: {step_output.observation}")
    print(f"Thought: {step_output.thought}")

if "input" not in user_prompt:
    raise ValueError("Prompt must contain 'input' key.")
messages = [
    ("system", system_message),
    (
        "placeholder",
        "{chat_history}",
    ),
    ("human", user_prompt),
    ("placeholder", "{agent_scratchpad}"),
]

# activity_diagram = diagram.activity_diagram_callback(note_max_length=2000)
# sequence_diagram = diagram.sequence_diagram_callback(note_max_length=2000)
# handler = StdOutCallbackHandler()

prompt = ChatPromptTemplate.from_messages(messages)
agent = create_tool_calling_agent(llm, tools, prompt)
result = await self.run_agent(agent, input_value, tools, message_history, handle_parsing_errors)
self.status = result

[dosubot]

To integrate a custom callback into the ToolCallingAgentComponent to output the agent's steps in a Recode format, you need to modify the build method to include the custom callback logic. Here is the part of the code you need to modify:

1. Define a custom callback class that inherits from AsyncCallbackHandler.
2. Modify the run_agent method to accept and use the custom callback.

Here is the updated code:

from langchain_core.callbacks.base import AsyncCallbackHandler
from langchain_core.agents import AgentAction, AgentFinish

class CustomCallback(AsyncCallbackHandler):
    async def on_agent_action(self, action: AgentAction, **kwargs: Any):
        log = f"Recode Step: {action.log}"
        print(log)  # Or handle the output as needed

    async def on_agent_finish(self, finish: AgentFinish, **kwargs: Any):
        log = f"Recode Step: {finish.log}"
        print(log)  # Or handle the output as needed

custom_callback = CustomCallback()

if "input" not in user_prompt:
    raise ValueError("Prompt must contain 'input' key.")
messages = [
    ("system", system_message),
    ("placeholder", "{chat_history}"),
    ("human", user_prompt),
    ("placeholder", "{agent_scratchpad}"),
]

prompt = ChatPromptTemplate.from_messages(messages)
agent = create_tool_calling_agent(llm, tools, prompt)
result = await self.run_agent(agent, input_value, tools, message_history, handle_parsing_errors, callback=custom_callback)
self.status = result

Additionally, you need to ensure that the run_agent method supports passing a callback parameter. Here is the modified run_agent method:

async def run_agent(
    self,
    agent: Union[Runnable, BaseSingleActionAgent, BaseMultiActionAgent, AgentExecutor],
    inputs: str,
    tools: List[Tool],
    message_history: Optional[List[Record]] = None,
    handle_parsing_errors: bool = True,
    output_key: str = "output",
    callback: Optional[Callable] = None  # Add a callback parameter
) -> Text:
    if isinstance(agent, AgentExecutor):
        runnable = agent
    else:
        runnable = AgentExecutor.from_agent_and_tools(
            agent=agent,  # type: ignore
            tools=tools,
            verbose=True,
            handle_parsing_errors=handle_parsing_errors,
        )
    input_dict: dict[str, str | list[BaseMessage]] = {"input": inputs}
    if message_history:
        input_dict["chat_history"] = records_to_messages(message_history)
    
    # Pass the callback to the acall method
    result = await runnable.acall(input_dict, callbacks=[callback] if callback else [])
    
    self.status = result
    if output_key in result:
        return cast(str, result.get(output_key))
    elif "output" not in result:
        if output_key != "output":
            raise ValueError(f"Output key not found in result. Tried '{output_key}' and 'output'.")
        else:
            raise ValueError("Output key not found in result. Tried 'output'.")

    return cast(str, result.get("output"))

This modification allows the run_agent method to accept a custom callback function that can be used during the execution of the agent ^[1][2].

To continue talking to Dosu, mention @dosu.

[YAMON.IO]

Based on Dosubot's suggestion, I have deployed the Yamon Tool Calling Agent, referencing the ToolCallingAgentComponent, to the Store. It seems the Langflow team intends to implement the callback in the Service layer, so I temporarily overrode the run_agent method of LCAgentComponent to implement it.

[Carlos Coelho]

Hi @YamonBot ,

Do you need any assistance with this case? If not, please let us know if this issue can be closed.

[YAMON.IO]

@carlosrcoelho

Hello,

I have posted a similar request on Discord regarding this topic. I am currently opposed to the logging methods implemented in LangSmith or LangWatch. These methods also use LangChain callbacks for logging, which makes it difficult to respond to updates of each solution. As the updates of the solutions are based on LangChain, each solution has its own specialized way of handling it. This configuration should be managed at the callback component level. The component class should have a member defined to handle callbacks by default.

https://discord.com/channels/1116803230643527710/1117938353761947678/1260442118208491622