warmcat / libwebsockets

canonical libwebsockets.org networking library

Home Page:https://libwebsockets.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Data loss at beginning of read call output in libev integration, any offsets to consider apart from LWS_PRE when received data is read?

alptugp opened this issue · comments

I built a lws client that receives data from a server. I integrated libev with it using my own foreign event loop. (My end goal is doing the socket read syscalls for the lws client asynchronously.)

To achieve this, I'm using a libev loop callback socket_cb that reads data from the server using SSL_read (with the socket fd and the SSL object my lws client uses, which I accessed using lws_get_socket_fd and lws_get_ssl respectively.). However, I noticed that a portion of the data is missing at the beginning of each read operation. For example, in the first SSL_read call, the initial '{"in' part of the data is lost. I provided the terminal output example below.

I suspect that this issue might be related to not considering other offsets apart from LWS_PRE. Could someone provide guidance on how to ensure that I read the full data in each SSL_read call without losing any information at the beginning?

libev Event Loop Callback:

static void socket_cb (EV_P_ ev_io *w, int revents) {
  if (revents & EV_READ) { 
        puts ("SOCKET ready for reading\n");
	int n;

	char buffer[1024 + LWS_PRE];	
	char *px = buffer + LWS_PRE;
        int lenx = sizeof(buffer) - LWS_PRE;

	do {
            printf("%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%");
            n = SSL_read(ssl, &px, lenx);
            printf("BYTES READ: %d\n", n);
            if (n > 0) {
                printf("%s\n", buffer);
            }
        } while (n > 0);	
    }
}

Terminal Output Example:

SOCKET ready for reading

%%%%%%%%%%%%%%%%%%BYTES READ: 1024
fo":"Welcome to the BitMEX Realtime API.","version":"2.0.0","timestamp":"2024-05-09T16:00:09.272Z","docs":"https://www.bitmex.com/app/wsAPI","heartbeatEnabled":false,"limit":{"remaining":29}}�r{"success":true,"subscribe":"orderBookL2_25:XBTUSDT","request":{"op":"subscribe","args":"orderBookL2_25:XBTUSDT"}}�~D{"table":"orderBookL2_25","action":"partial","keys":["symbol","id","side"],"types":{"symbol":"symbol","id":"long","side":"symbol","size":"long","price":"float","timestamp":"timestamp"},"filter":{"symbol":"XBTUSDT"},"data":[{"symbol":"XBTUSDT","id":1001065,"side":"Sell","size":40000,"price":470000.0,"timestamp":"2024-05-09T16:00:09.273Z"},{"symbol":"XBTUSDT","id":1000491,"side":"Sell","size":420000,"price":430000.0,"timestamp":"2024-05-09T16:00:09.273Z"},{"symbol":"XBTUSDT","id":1053338723,"side":"Sell","size":3000000,"price":428243.6,"timestamp":"2024-05-09T16:00:09.273Z"},{"symbol":"XBTUSDT","id":1053339985,"side":"Sell","size":200000,"price":417266.2,"timestamp":"2024-05-09T16:00:09.273Z"},{"sy
%%%%%%%%%%%%%%%%%%BYTES READ: 1024
BTUSDT","id":1053339845,"side":"Sell","size":200000,"price":412875.2,"timestamp":"2024-05-09T16:00:09.273Z"},{"symbol":"XBTUSDT","id":1053339839,"side":"Sell","size":200000,"price":402995.5,"timestamp":"2024-05-09T16:00:09.273Z"},{"symbol":"XBTUSDT","id":1053339851,"side":"Sell","size":200000,"price":381040.6,"timestamp":"2024-05-09T16:00:09.273Z"},{"symbol":"XBTUSDT","id":1053339635,"side":"Sell","size":200000,"price":362378.9,"timestamp":"2024-05-09T16:00:09.273Z"},{"symbol":"XBTUSDT","id":1122605976,"side":"Sell","size":1000000,"price":77100.3,"timestamp":"2024-05-09T16:00:09.273Z"},{"symbol":"XBTUSDT","id":1266334504,"side":"Sell","size":17588000,"price":76830.0,"timestamp":"2024-05-09T16:00:09.273Z"},{"symbol":"XBTUSDT","id":1270833050,"side":"Sell","size":1000000,"price":76827.5,"timestamp":"2024-05-09T16:00:09.273Z"},{"symbol":"XBTUSDT","id":1270833052,"side":"Sell","size":1000000,"price":76825.0,"timestamp":"2024-05-09T16:00:09.273Z"},{"symbol":"XBTUSDT","id":1270885744,"side":"Sell","size":97
...

That's not how lws works... nobody has time to follow whatever you want to do and explain why that doesn't work... the examples are there to provide working solutions, please base off those.