bep / tmc

Provides basic roundtrip JSON etc. encoding/decoding of a map[string]any with custom type adapters.

Codec for a Typed Map

Provides round-trip serialization of typed Go maps.

How to Use

See the GoDoc for some basic examples and how to configure custom codec, adapters etc.

Why?

Text based serialization formats like JSON and YAML are convenient, but when used with Go maps, most type information gets lost in translation.

Listed below is a round-trip example in JSON (see https://play.golang.org/p/zxt-wi4Ljz3 for a runnable version):

package main

import (
	"encoding/json"
	"log"
	"math/big"
	"time"

	"github.com/kr/pretty"
)

func main() {
	mi := map[string]interface{}{
		"vstring":   "Hello",
		"vint":      32,
		"vrat":      big.NewRat(1, 2),
		"vtime":     time.Now(),
		"vduration": 3 * time.Second,
		"vsliceint": []int{1, 3, 4},
		"nested": map[string]interface{}{
			"vint":      55,
			"vduration": 5 * time.Second,
		},
		"nested-typed-int": map[string]int{
			"vint": 42,
		},
		"nested-typed-duration": map[string]time.Duration{
			"v1": 5 * time.Second,
			"v2": 10 * time.Second,
		},
	}

	data, err := json.Marshal(mi)
	if err != nil {
		log.Fatal(err)
	}
	m := make(map[string]interface{})
	if err := json.Unmarshal(data, &m); err != nil {
		log.Fatal(err)
	}

	pretty.Print(m)

}

This prints:

map[string]interface {}{
    "vint":      float64(32),
    "vrat":      "1/2",
    "vtime":     "2009-11-10T23:00:00Z",
    "vduration": float64(3e+09),
    "vsliceint": []interface {}{
        float64(1),
        float64(3),
        float64(4),
    },
    "vstring": "Hello",
    "nested":  map[string]interface {}{
        "vduration": float64(5e+09),
        "vint":      float64(55),
    },
    "nested-typed-duration": map[string]interface {}{
        "v2": float64(1e+10),
        "v1": float64(5e+09),
    },
    "nested-typed-int": map[string]interface {}{
        "vint": float64(42),
    },
}

And that is very different from the origin:

• All numbers are now float64
• time.Duration is also float64
• time.Now and *big.Rat are strings
• Slices are []interface {}, maps map[string]interface {}

So, for structs, you can work around some of the limitations above with custom MarshalJSON, UnmarshalJSON, MarshalText and UnmarshalText.

For the commonly used flexible and schema-less map[string]interface {} this is, as I'm aware of, not an option.

Using this library, the above can be written to (see https://play.golang.org/p/PlDetQP5aWd for a runnable example):

package main

import (
	"log"
	"math/big"
	"time"

	"github.com/bep/tmc"

	"github.com/kr/pretty"
)

func main() {
	mi := map[string]interface{}{
		"vstring":   "Hello",
		"vint":      32,
		"vrat":      big.NewRat(1, 2),
		"vtime":     time.Now(),
		"vduration": 3 * time.Second,
		"vsliceint": []int{1, 3, 4},
		"nested": map[string]interface{}{
			"vint":      55,
			"vduration": 5 * time.Second,
		},
		"nested-typed-int": map[string]int{
			"vint": 42,
		},
		"nested-typed-duration": map[string]time.Duration{
			"v1": 5 * time.Second,
			"v2": 10 * time.Second,
		},
	}

	c, err := tmc.New()
	if err != nil {
		log.Fatal(err)
	}

	data, err := c.Marshal(mi)
	if err != nil {
		log.Fatal(err)
	}
	m := make(map[string]interface{})
	if err := c.Unmarshal(data, &m); err != nil {
		log.Fatal(err)
	}

	pretty.Print(m)

}

This prints:

map[string]interface {}{
    "vduration":        time.Duration(3000000000),
    "vint":             int(32),
    "nested-typed-int": map[string]int{"vint":42},
    "vsliceint":        []int{1, 3, 4},
    "vstring":          "Hello",
    "vtime":            time.Time{
        wall: 0x0,
        ext:  63393490800,
        loc:  (*time.Location)(nil),
    },
    "nested": map[string]interface {}{
        "vduration": time.Duration(5000000000),
        "vint":      int(55),
    },
    "nested-typed-duration": map[string]time.Duration{"v1":5000000000, "v2":10000000000},
    "vrat":                  &big.Rat{
        a:  big.Int{
            neg: false,
            abs: {0x1},
        },
        b:  big.Int{
            neg: false,
            abs: {0x2},
        },
    },
}

Performance

The implementation is easy to reason about (it uses reflection), but It's not particulary fast and probably not suited for big data. A simple benchmark with a roundtrip marshal/unmarshal is included. On my MacBook it shows:

BenchmarkCodec/JSON_regular-16         	   63921	     16261 ns/op	    6486 B/op	     163 allocs/op
BenchmarkCodec/JSON_typed-16           	   31791	     37396 ns/op	   14538 B/op	     387 allocs/op