BinaryDefense.JsonWrapper

What

A plugin for Myriad for generating statically typed lossless wrappers around JToken given a schema.

Why

When serializing JSON directly into known record types, this can cause the underlying dataset to be lost. Given a JSON object:

{
    "Name" : "James Kirk",
    "Age"  : 32
}

and a record type:

type Person = {
    Name : string
    Age : uint32
}

serializing is fine. However, if the underlying json adds more information in it's payload such as:

{
    "Name" : "James Kirk",
    "Age"  : 32,
    "UniformColor": "Gold"
}

If that type is then serialized and saved somewhere, our record type will lose the UniformColor field, causing this to be a lossy conversion.

This Myriad plugin will instead generate a class given a record type as a schema.

How

Add BinaryDefense.Myriad.Plugins.JsonWrapper to your project.

Given our record above as a schema, we need to add the attribute [<Generator.JsonWrapper>] to it.

[<Generator.JsonWrapper>]
type Person = {
    Name : string
    Age : uint32
}

Additionally, an entry to the fsproj file must be added:

<Compile Include="MyGeneratedFile.fs">
    <MyriadFile>MyTypes.fs</MyriadFile>
    <MyriadNameSpace>GeneratedNamespace</MyriadNameSpace>
</Compile>

MyTypes.fs is where the Person record type lives. MyGeneratedFile.fs will be the file generated with the output. GeneratedNamespace will be the namespace the generated code lives in. On build, Myriad will generate the code An example of it's output:

type Person(jtoken: JToken, serializer: JsonSerializer) =
    member this.Name
        with get () =
            let selectedToken = jtoken.["Name"]
            selectedToken.ToObject<string> serializer
        and set (newValue: string) =
            jtoken.["Name"] <- JToken.FromObject(newValue, serializer)

    member this.Age
        with get () =
            let selectedToken = jtoken.["Age"]
            selectedToken.ToObject<uint32> serializer
        and set (newValue: uint32) =
            jtoken.["Age"] <- JToken.FromObject(newValue, serializer)

    override this.GetHashCode () = jtoken.GetHashCode()

    override this.Equals(objToCompare: obj) =
        match objToCompare with
        | :? IHaveJToken as jTokenToCompare -> JToken.DeepEquals(jTokenToCompare.InnerData, jtoken)
        | _ -> false

    ///This allows the class to be pattern matched against
    member this.Deconstruct(Name: outref<int>, Age: outref<string>) =
        Name <- this.Name
        Age <- this.Age

    interface IHaveJToken with
        override this.InnerData = jtoken

When using this type, you'll need to also add custom converters to the JsonSerializer you are using throughout your application.

let converters = Converters.recommendedConverters

let serializationSettings =
    let s = JsonSerializerSettings()
    scrubDefaultDUConverter s.Converters
    for c in converters do s.Converters.Add c
    s

let jsonSettings = serializationSettings
let jsonSerializer =JsonSerializer.CreateDefault jsonSettings

This will be using the IHaveJTokenConverter to ensure the serialization is lossless.

Additional reading on this topic:

Features

Lossless
Override backing field name
Enforce Required fields
Destructuring
Structural equals

Builds

GitHub Actions

NuGet

Package	Stable	Prerelease
BinaryDefense.JsonWrapper.Core
BinaryDefense.Myriad.Plugins.JsonWrapper

Developing

Make sure the following requirements are installed on your system:

dotnet SDK 3.0 or higher
Mono if you're on Linux or macOS.

VSCode Dev Container

Environment Variables

CONFIGURATION will set the configuration of the dotnet commands. If not set, it will default to Release.
- CONFIGURATION=Debug ./build.sh will result in -c additions to commands such as in dotnet build -c Debug
GITHUB_TOKEN will be used to upload release notes and Nuget packages to GitHub.
- Be sure to set this before releasing
DISABLE_COVERAGE Will disable running code coverage metrics. AltCover can have severe performance degradation so it's worth disabling when looking to do a quicker feedback loop.
- DISABLE_COVERAGE=1 ./build.sh

Building

> build.cmd <optional buildtarget> // on windows
$ ./build.sh  <optional buildtarget>// on unix

The bin of your library should look similar to:

$ tree src/MyCoolNewLib/bin/
src/MyCoolNewLib/bin/
└── Debug
    ├── net461
    │   ├── FSharp.Core.dll
    │   ├── MyCoolNewLib.dll
    │   ├── MyCoolNewLib.pdb
    │   ├── MyCoolNewLib.xml
    └── netstandard2.1
        ├── MyCoolNewLib.deps.json
        ├── MyCoolNewLib.dll
        ├── MyCoolNewLib.pdb
        └── MyCoolNewLib.xml

Build Targets

Clean - Cleans artifact and temp directories.
DotnetRestore - Runs dotnet restore on the solution file.
DotnetBuild - Runs dotnet build on the solution file.
DotnetTest - Runs dotnet test on the solution file.
GenerateCoverageReport - Code coverage is run during DotnetTest and this generates a report via ReportGenerator.
WatchTests - Runs dotnet watch with the test projects. Useful for rapid feedback loops.
GenerateAssemblyInfo - Generates AssemblyInfo for libraries.
DotnetPack - Runs dotnet pack. This includes running Source Link.
SourceLinkTest - Runs a Source Link test tool to verify Source Links were properly generated.
PublishToNuGet - Publishes the NuGet packages generated in DotnetPack to NuGet via paket push.
GitRelease - Creates a commit message with the Release Notes and a git tag via the version in the Release Notes.
GitHubRelease - Publishes a GitHub Release with the Release Notes and any NuGet packages.
FormatCode - Runs Fantomas on the solution file.
BuildDocs - Generates Documentation from docsSrc and the XML Documentation Comments from your libraries in src.
WatchDocs - Generates documentation and starts a webserver locally. It will rebuild and hot reload if it detects any changes made to docsSrc files, libraries in src, or the docsTool itself.
ReleaseDocs - Will stage, commit, and push docs generated in the BuildDocs target.
Release - Task that runs all release type tasks such as PublishToNuGet, GitRelease, ReleaseDocs, and GitHubRelease. Make sure to read Releasing to setup your environment correctly for releases.

Releasing

Start a git repo with a remote

git add .
git commit -m "Scaffold"
git remote add origin https://github.com/user/MyCoolNewLib.git
git push -u origin master

Create your NuGeT API key
- Add your NuGet API key to paket
```
paket config add-token "https://www.nuget.org" 4003d786-cc37-4004-bfdf-c4f3e8ef9b3a
```
- or set the environment variable NUGET_TOKEN to your key
Create a GitHub OAuth Token
- You can then set the environment variable GITHUB_TOKEN to upload release notes and artifacts to github
- Otherwise it will fallback to username/password
Then update the CHANGELOG.md with an "Unreleased" section containing release notes for this version, in KeepAChangelog format.

NOTE: Its highly recommend to add a link to the Pull Request next to the release note that it affects. The reason for this is when the RELEASE target is run, it will add these new notes into the body of git commit. GitHub will notice the links and will update the Pull Request with what commit referenced it saying "added a commit that referenced this pull request". Since the build script automates the commit message, it will say "Bump Version to x.y.z". The benefit of this is when users goto a Pull Request, it will be clear when and which version those code changes released. Also when reading the CHANGELOG, if someone is curious about how or why those changes were made, they can easily discover the work and discussions.

Here's an example of adding an "Unreleased" section to a CHANGELOG.md with a 0.1.0 section already released.

## [Unreleased]

### Added
- Does cool stuff!

### Fixed
- Fixes that silly oversight

## [0.1.0] - 2017-03-17
First release

### Added
- This release already has lots of features

[Unreleased]: https://github.com/user/MyCoolNewLib.git/compare/v0.1.0...HEAD
[0.1.0]: https://github.com/user/MyCoolNewLib.git/releases/tag/v0.1.0

You can then use the Release target, specifying the version number either in the RELEASE_VERSION environment variable, or else as a parameter after the target name. This will:
- update CHANGELOG.md, moving changes from the Unreleased section into a new 0.2.0 section
  - if there were any prerelease versions of 0.2.0 in the changelog, it will also collect their changes into the final 0.2.0 entry
- make a commit bumping the version: Bump version to 0.2.0 and adds the new changelog section to the commit's body
- publish the package to NuGet
- push a git tag
- create a GitHub release for that git tag

macOS/Linux Parameter:

./build.sh Release 0.2.0

macOS/Linux Environment Variable:

RELEASE_VERSION=0.2.0 ./build.sh Release

BinaryDefense / JsonWrapper