在使用异步流返回时，并不是实时刷新逐条返回结果，而是一次返回结果
alivy opened this issue · comments
Steps to Reproduce the Problem
- 当响应体中有text/event-stream时，返回结果为一次性加载返回，而并非实时刷新
- 即使我加载了自己的Middleware添加到OcelotPipelineExtensions时，也不能实时刷新，以异步流的形式收到结果
自定义拓展代码如下
`
public class CustomResponseMiddleware : OcelotMiddleware
{
private readonly RequestDelegate _next;
public CustomResponseMiddleware(RequestDelegate next, IOcelotLoggerFactory loggerFactory)
: base(loggerFactory.CreateLogger<CustomResponseMiddleware>())
{
_next = next;
}
public async Task Invoke(HttpContext httpContext)
{
var response = httpContext.Items.DownstreamResponse();
const string streamingContent = "text/event-stream";
if (httpContext.Response.Headers["Content-Type"] == streamingContent
|| httpContext.Response.ContentType == streamingContent)
{
//await ForwardStream(httpContext.Response.Body, response);
var stream = await response.Content.ReadAsStreamAsync();
await WriteServerStreamEventsAsync(httpContext, stream);
}
await _next.Invoke(httpContext);
}
private static async Task WriteServerStreamEventsAsync(HttpContext httpContext, Stream contentStream)
{
var responseStream = httpContext.Response.Body;
await contentStream.CopyToAsync(responseStream, httpContext.RequestAborted);
await responseStream.FlushAsync(httpContext.RequestAborted);
}
}
`
Dear @alivy !
The official language of the project is English. Most of our teammates and even our contributors/developers from all over the world do not understand Chinese!
Please update the description using English texts, otherwise the issue will be closed!
When there is text/event-stream in the response body, the returned result is loaded and returned at once, not refreshed in real time.
Even when I load my own Middleware and add it to OcelotPipelineExtensions, it doesn't refresh in real time, and the results are received as an asynchronous stream.
The custom extension code is as follows
@alivy @raman-m 我不懂中文，所以可能无法用中文表达清楚。不过我想学习你们的语言。我认为，您应该等待推送到开发分支的最新更改。我们不再直接复制请求正文和响应正文。
Due to environmental factors, the code cannot be made public, but I can describe the scenario to you ：
When I used Ocelot to handle OpenAI's dialogue interface v1/chat/completion, the direct request was to refresh each field in real-time and return it in asynchronous flow form. However, when I used the Ocelot gateway, the response result was also returned in asynchronous flow form, but all dialogue information was returned at once
请使用英文文本更新描述，否则问题将被关闭！
The scene is as follows:
When I used Ocelot to handle OpenAI's dialogue interface v1/chat/completion, the direct request was to refresh each field in real-time and return it in asynchronous flow form. However, when I used the Ocelot gateway, the response result was also returned in asynchronous flow form, but all dialogue information was returned at once
When I tried to modify the addition of middleware in the pipeline to respond to data asynchronously, an error occurred;
public class CustomResponseMiddleware : OcelotMiddleware
{
private readonly RequestDelegate _next;
public CustomResponseMiddleware(RequestDelegate next, IOcelotLoggerFactory loggerFactory)
: base(loggerFactory.CreateLogger<CustomResponseMiddleware>())
{
_next = next;
}
public async Task Invoke(HttpContext httpContext)
{
var response = httpContext.Items.DownstreamResponse();
const string streamingContent = "text/event-stream";
if (httpContext.Response.Headers["Content-Type"] == streamingContent
|| httpContext.Response.ContentType == streamingContent
|| response.Content.Headers.ContentType?.ToString() == streamingContent)
{
//await ForwardStream(httpContext.Response.Body, response);
var stream = await response.Content.ReadAsStreamAsync();
await WriteServerStreamEventsAsync(httpContext, stream);
}
await _next.Invoke(httpContext);
}
private static async Task WriteServerStreamEventsAsync(HttpContext httpContext, Stream contentStream)
{
var responseStream = httpContext.Response.Body;
await contentStream.CopyToAsync(responseStream, httpContext.RequestAborted);
await responseStream.FlushAsync(httpContext.RequestAborted);
}
}The error message is as follows：
Microsoft.AspNetCore.Server.Kestrel[13]
Connection id "0HN0RN6PE4IOV", Request id "0HN0RN6PE4IOV:00000002": An unhandled exception was thrown by the application.
System.InvalidOperationException: Response Content-Length mismatch: too few bytes written (0 of 71354).
@alivy could you tell us which Ocelot version you are using?