gnieh / fs2-data

streaming data parsing and transformation library

Home Page:https://fs2-data.gnieh.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Wrong line number when CSV contains empty lines

usommerl opened this issue · comments

I'm using the fs2.data.csv.lowlevel.rows pipe to parse CSV files. I wanted to provide meaningful error information to my users and was hoping that I can use the line information from the RowF class. However, the provided line number may be incorrect if the CSV file contains empty lines. I understand that the rows pipe skips empty lines, but I didn't expect that the line numbers of subsequent lines do not correspond with the line numbers of the file. I'm not sure whether this is the intended behaviour.

Example:

import fs2._
import fs2.data.csv._
import cats.implicits._

val input = """A,B,C
              |D,E,F
              |
              |G,H,I
              |
              |J,K,L
              |""".stripMargin

Stream.emit(input)
      .covary[Fallible]
      .through(lowlevel.rows[Fallible, String]())
      .map(r => s"""${r.line.orEmpty}: ${r.values.toList.mkString}""")
      .compile
      .toList  

// actual result  : Right(List(1: ABC, 2: DEF, 3: GHI, 4: JKL))
// my expectation : Right(List(1: ABC, 2: DEF, 4: GHI, 6: JKL))

fs2-data version: 1.6.1

Hey, thank you for reporting this issue!
I agree with your interpretation of line numbers and believe I have a fix in #462. Also included your example as a test case.