Giters

gofireflyio / aiac

Artificial Intelligence Infrastructure-as-Code Generator.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Using local inference results in "Failed generating code: result was truncated: length"

remmen-io opened this issue · comments

commented

I'm using a local inference service using huggingface tgi, which is compatible with the openai api.

I get the following error using it with aiac

aiac get --url="https://deepseek.myinference.ch/v1" --api-key="none" terraform code for a s3 bucket on aws
Failed generating code: result was truncated: length

[R/r]: retry same prompt
[Y/y]: copy to clipboard
[Q/q]: quit

Using the inference api with curl or other AI tools works flawless

commented

Hi, sorry for the delay. Can you post the full JSON output you get when running the same exact query via curl? It seems the API is telling us it did not return the full generated text.

commented

Version 5.0.0 no longer fails on truncated output from the API. Instead, the library returns the stop reason for the user to decide how to proceed.