queryverse / TextParse.jl

A bunch of fast text parsing tools

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Impossible to load CSV files with text containing the '\n' character

arnaudmgh opened this issue · comments

Not sure if related to escaping and issue #85 , but I am unable to read a CSV file with a text field, it chokes when hitting a multiline text entry (f. ex "Dear Mike,\n I hope you are doing well ... "). I think it interprets the '\n' as a newline in the CSV file, and therefore counts a wrong number of fields in the row.

Full error message below:

previous rows had 6 fields but row 8 has 3 error(::String) at error.jl:33 guesscolparsers(::TextParse.VectorBackedUTF8String, ::Array{String,1}, ::TextParse.LocalOpts, ::Int64, ::Int64, ::Array{Any,1}, ::Array{String,1}, ::Nothing) at csv.jl:495 #_csvread_internal#26(::Bool, ::Char, ::Char, ::Type, ::Bool, ::Int64, ::Nothing, ::Nothing, ::Int64, ::Nothing, ::Bool, ::Array{String,1}, ::Array{String,1}, ::OrderedCollections.OrderedDict{Union{Int64, String},AbstractArray{T,1} where T}, ::Int64, ::Nothing, ::Array{Any,1}, ::String, ::Int64, ::typeof(TextParse._csvread_internal), ::TextParse.VectorBackedUTF8String, ::Char) at csv.jl:225 (::getfield(TextParse, Symbol("#kw##_csvread_internal")))(::NamedTuple{(:filename,),Tuple{String}}, ::typeof(TextParse._csvread_internal), ::TextParse.VectorBackedUTF8String, ::Char) at none:0 #22 at csv.jl:103 [inlined] #open#294(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::getfield(TextParse, Symbol("##22#24")){Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},String,Char}, ::String, ::Vararg{String,N} where N) at iostream.jl:369 open at iostream.jl:367 [inlined] #_csvread_f#20 at csv.jl:101 [inlined] _csvread_f at csv.jl:93 [inlined] #csvread#16(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::Char) at csv.jl:76 csvread(::String, ::Char) at csv.jl:76 _loaddata(::CSVFiles.CSVFile) at CSVFiles.jl:71 getiterator(::CSVFiles.CSVFile) at CSVFiles.jl:76 show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::CSVFiles.CSVFile) at CSVFiles.jl:22 show at sysimg.jl:194 [inlined] (::getfield(Atom, Symbol("##27#28")){CSVFiles.CSVFile})(::Base.GenericIOBuffer{Array{UInt8,1}}) at display.jl:17 #sprint#324(::Nothing, ::Int64, ::Function, ::Function) at io.jl:101 sprint at io.jl:97 [inlined] render at display.jl:16 [inlined] Type at types.jl:39 [inlined] Type at types.jl:40 [inlined] render at display.jl:19 [inlined] displayandrender(::CSVFiles.CSVFile) at showdisplay.jl:127 (::getfield(Atom, Symbol("##115#120")){String})() at eval.jl:102 #invokelatest#1 at essentials.jl:697 [inlined] invokelatest at essentials.jl:696 [inlined] macro expansion at dynamic.jl:24 [inlined] (::getfield(Atom, Symbol("##111#116")))(::Dict{String,Any}) at eval.jl:86 handlemsg(::Dict{String,Any}, ::Dict{String,Any}) at comm.jl:164 (::getfield(Atom, Symbol("##19#21")){Array{Any,1}})() at task.jl:259