NodeJS Stream pipeline() not receiving parsed data from csv-parse transformer in callback

[Keshav Sharma]

Describe the bug
I'm using streams.pipeline in order to read a csv file and parse it, however the callback function does not receive any value. It does however receives any error that may occur.

csv looks like

a,b
1,2

and my code looks like

import {parse} from "csv-parse";
import {createReadStream} from "fs";
import {pipeline} from "stream";

//this function is expected to read csv and return data as an array of objects.
async readFile(fileLocation) {

var csvParsePromise = new Promise((resolve, reject) => {
  pipeline(
    createReadStream(fileLocation),
    parse({columns:true}),
    (err, val) => {
      if(err) {
        return reject(err);
      }
      resolve(val); //val is undefined, expected [{a:1}, {b:2}]
    }
  );
});
var csvData = await csvParsePromise.catch(err=>{}); //the resolve returns undefined.
return csvData || [];
}

I expect csvParsePromise to resolve with an array [{a:1}, {b:2}] but instead it resolves with "undefined" value.
To Reproduce
I've provided a sample code above on how this can be reproduced. Node Version v14.19.3

Update:
So i looked into "stream-transform" npm module, and used it to fix the issue for me, but i don't understand why callback does not receives the value (the official nodejs docs mention that cb accepts 2 args, an error and the final value (val))

this is the code that i use now to read the csv

imports...

const readFile = async (fileLocation) { 
   let mergedResult = [];
   const transformer = transform((data) => { mergedResult.push(data); return data; }
   const csvPromise = new Promise((resolve, reject) => {
     pipeline(
       createReadStream(fileLocation),
       parse({columns:true}),
       transformer,
       (err) => {
         if(err) { return reject(err); }
         resolve(true);
       }
     )
   });
   
   await csvPromise.catch(err=>{});
   return  mergedResult || [];
}

so it seems parse() does indeed forward the data in the pipeline, not sure then why callback not receiving value in the first case. Maybe I've not understood streams properly or there is a bug.