ASCII files are read incorrectly if EOL characters are missing
yruslan opened this issue · comments
Ruslan Yushchenko commented
Describe the bug
Although this is a very rare corner case, it would be nice to fix it.
To Reproduce
//Text file = "AABBCC"
val df = spark
.read
.format("cobol")
.option("copybook_contents", copybook)
.option("pedantic", "true")
.option("record_format", "D")
.load("MyTextFile.txt")
df.show
Got
+---+
| A|
+---+
| AB|
| CC|
+---+
Expected
+---+
| A|
+---+
| AA|
| BB|
| CC|
+---+