Optional columns

[Jakub Hampl]

In our application there are some required columns the user submit and some are optional.

At the moment I encode this like this:

optional :Decoder a -> Decoder (Maybe a -> b) -> Decoder b
optional default childDecoder =
    pipeline (Csv.oneOf (Csv.map Just childDecoder) [ Csv.succeed Nothing ])

decoder =
    Csv.into Tuple.pair 
         |> pipeline (Csv.field "requiredColumn" Csv.float)
         |> optional (Csv.field "optionalColumn" Csv.float)

This sort of works, but has some subtle behavioural gotchas:

requiredColumn	optionalColumn
1	2
2	four
3	6

would successfully decode into:

[ ( 1, Just 2 ), ( 2, Nothing ), (3, Just 6) ]

which is not what I want. In this case I'd like to flag the error to the user, but I'd still like to mark some columns as optional. I can't really see a way to achieve this with the current API. So I'd propose something like

optionalField : String -> Decoder a -> Decoder (Maybe a)

which would only go to the Nothing case iff the column was not present at all in the CSV.

[Brian Hicks]

Hmm, that's an interesting question. It probably makes sense to do this and to rename pipeline to required. Would these be the semantics?

input	required (field "a" int)	optional (field "a" int)
"0"	Ok 0	Ok (Just 0)
"a"	Err ...	Err ...
""	Err ...	Ok Nothing

[Jakub Hampl]

Would these be the semantics?

That's not really what I'm after. You can already achieve that with the existing API:

emptyString : Decoder String
emptyString =
    Decode.string 
        |> Decode.andThen (\str ->
            case str of
                 "" -> Decode.succeed ""
                 _ -> Decode.fail "Expected empty string"
        )

optionalField : String -> Decoder a -> Decoder (Maybe a )
optionalField key default childDecoder =
    Csv.oneOf (Csv.map Just childDecoder) [ Csv.map (always Nothing) emptyString  ]

Here I'd like the optionalField to succeed with Nothing only if the field is actually not present in the file.

[Brian Hicks]

oh, oops, the third row there is mean to be missing, not blank. I think we're saying the same thing.

[Brian Hicks]

if you want to make that happen, go for it—but let's change pipeline to required at the same time for symmetry.