benlesh / rxjs-for-await

A library for making RxJS support async-await for-await loops via AsyncIterables

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Better explanation of memory issues and difference between methods

HKalbasi opened this issue · comments

What is exactly the cost of eachValueFrom ?

If the observable emits faster than your loop can consume them, this may result in a memory leak.

Memory leak? Why memory leak? Do you mean items are queued up in memory and so this may have negative impact on performance? This is not memory leak (there is no leak, those items are needed and should stored on memory) and I think this term is misleading.

This is the same "cost" as concatMap would be in RxJS in some cases.

Basically, if the source produces values faster than the loop can consume them, the values will build up in memory.

import { interval, map } from 'rxjs';
import { eachValueFrom } from 'rxjs-for-await';

// Source pushes a new value, that's a large-ish array, once per second.
const source$ = interval(1000).pipe(
   map(() => new Array(1e6).fill({}))
);

for await (const value of eachValueFrom(source$)) {
   // loop only comes back to the top and reads once every THREE seconds.
   // That means that every minute the buffer grows by 40 values.
   await sleep(3000);
   console.log(value);
}

function sleep(ms: number) {
  return new Promise(res => setTimeout(res), ms);
}

The underlying buffer will grow and grow with no hope of every being cleaned up. On a long enough timeline, the application will run out of memory. "memory leak" "excess, unbounded memory consumption", whatever you'd like to call it. The result is identical and not confusing at all.