Column oriented decoding

[Jakub Hampl]

At work we represent our data in a column oriented format, i.e. something like:

type alias Data =
  { time : List Float
  , speed : List Float 
  , altitude: Maybe (List Float)
  }

rather than the more conventional row oriented structure:

type alias Data =
  List 
     { time : Float
     , speed : Float 
     , altitude: Maybe Float
     }

(note that the data formats have some subtly different invariants encoded into them; also the column oriented structure is more space efficient in the case of missing altitude)

---

Now obviously I can write a decoder for the row oriented data, then write some function that would transpose it to the column oriented data and call it a day.

But I was wondering if there wasn't a way to directly decode column oriented data straight from the CSV?

[Brian Hicks]

hey Jakub, you'll have to write the transformer here. CSV is a row-oriented format, so we have to parse the entire document even when you only want a couple of columns. We don't have a way to speed up that transformation. We could make it more ergonomic, but I don't think it makes a lot of sense for a general-purpose CSV library, since I haven't heard anyone else wanting to do this.

If you wanted to rewrite your input data to be columns-on-rows, you could use Csv.Parser.parse and then transform the values with String.toFloat (for instance.) You wouldn't get to use the Decoders, though!

[Brian Hicks]

@gampleman was there anything else we needed to do here? If not, I'm going to close this issue.

[Jakub Hampl]

No, I think your reasoning makes sense. Thanks for taking the time to reply.