Myriad

Myriad is a code generator, put plainly it takes an abstract syntax tree from a source and uses that to produce F# code.

Myriad can be used from either an MSBuild extension or from its CLI tool.

The idea behind Myriad is to un-complicate, as far as possible, the ability to generate and do meta-programming in F#. By meta-programming in F# I mean generating actual F# code like discriminated unions and records, not just IL output.

Myriad is an evolution of the ideas I developed while working with F#'s type providers and other meta-programming functionality like quotations and AST manipulation. Myriad aims to make it easy to extend the compiler via Myriad plugins rather than modifying or adjusting Type Providers and other F# improvement that would be a long time to be developed and released. The idea is you write a Myriad plugin that works on a fragment of AST input, and the plugin supplies AST output with the final form being source code that is built into your project. Thus the compiler can optimise and tooling can operate effectively too.

MSBuild usage

To use Myriad via its MSBuild support you add the Myriad.Core and Myriad.Sdk package references:

    <ItemGroup>
      <PackageReference Include="Myriad.Core" Version="0.2.4" />
      <PackageReference Include="Myriad.Sdk" Version="0.2.4" />
    </ItemGroup>

An input file is specified by using the usual Compile element:

<Compile Include="Generated.fs">
    <MyriadFile>Library.fs</MyriadFile>
    <MyriadNameSpace>Test</MyriadNameSpace>
</Compile>

This is configuring Myriad so that a file called Generated.fs will be included in the build using Library.fs as input to the Myriad and Test as the namespace.

Myriad works by using plugins to generate code. A plugin called fields is included with Myriad which takes inspiration from OCamls ppx_fields_conv plugin of the same name.

The input file in this example Library.fs looks like this:

namespace Example
open Myriad.Plugins

[<Generator.Fields>]
type Test1 = { one: int; two: string; three: float; four: float32 }
type Test2 = { one: Test1; two: string }

An attribute is used by the plugin so that the code generator plugin knows which parts of the input AST are to be used by the plugin. If you had several records and you only want the fields plugin to operate on Test1 then the attrivute would be used like in the example about to only apply Generator.Fields to Test1. Note, if you wanted a plugin that just needs the whole input AST then there is no need to provide an input. Myriad aims to be a library rather than a full framework that ties you to the mechanism used to input and generate code.

The fields plugin in this example will generate the following code at prebuild time and compile the code into your assembly:

//------------------------------------------------------------------------------
//        This code was generated by myriad.
//        Changes to this file will be lost when the code is regenerated.
//------------------------------------------------------------------------------
namespace rec Test

module Test1 =
    open Example

    let one (x : Test1) = x.one
    let two (x : Test1) = x.two
    let three (x : Test1) = x.three
    let four (x : Test1) = x.four

    let create (one : Test1) (two : string) (three : float) (four : float32) : Test1 =
        { one = one
          two = two
          three = three
          four = four }

    let map (mapone : int -> int) (maptwo : string -> string) (mapthree : float -> float) (mapfour : float32 -> float32) (record': Test1) =
      { record' with
          one = mapone record'.one
          two = maptwo record'.two
          three = mapthree record'.three
          four = mapfour record'.four }

The fields plugin generates a map for each field in the input record, a create function taking each field, and a map function that takes one function per field in the input record.

The map functions for each field are useful in situations where you just want to use a single field from a record in a lambda like a list of records:

let records = [{one = "a"; two = "aa"; three = 42.0; four = 172.0f}
               {one = "b"; two = "bb"; three = 42.0; four = 172.0f}]
 records |> List.sortBy Test1.one

The full fsproj is detail below:

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>netcoreapp3.0</TargetFramework>
    </PropertyGroup>
    <ItemGroup>
        <Compile Include="Library.fs" />
        <Compile Include="Generated.fs">
            <MyriadFile>Library.fs</MyriadFile>
            <MyriadNameSpace>Test</MyriadNameSpace>
        </Compile>
    </ItemGroup>
    <ItemGroup>
      <PackageReference Include="Myriad.Core" Version="0.2.4" />
      <PackageReference Include="Myriad.Sdk" Version="0.2.4" />
    </ItemGroup>
</Project>

Plugins

Plugins for Myriad are supplied by simply including the nuget package in your project, the nuget infrastructure supplies the necessary MSBuild props and targets so that the plugin is used by Myriad automatically. Following the source for the fields plugin can be used as reference until more details about authoring plugins is created.

Using external Plugins

To consume external plugins that aren't included in the Myriad.Plugins package, you must register them with Myriad. If you are using the CLI tool then the way to do this is by passing in the --plugin <path to dll> command-line argument. If you are using MSBuild then this can be done by adding to the MyriadSdkGenerator property to your project file:

<ItemGroup>
    <MyriadSdkGenerator Include="<path to plugin dll>" />
</ItemGroup>

For example, if you had a project layout like this:

\src
-\GeneratorLib
 - Generator.fs
 - Generator.fsproj
-\GeneratorTests
 - Tests.fs
 - GeneratorTests.fsproj

You would add the following to Generator.fsproj:

  <ItemGroup>
    <Content Include="build\Generator.props">
      <Pack>true</Pack>
      <PackagePath>%(Identity)</PackagePath>
      <Visible>true</Visible>
    </Content>
  </ItemGroup>

Then add a new folder build with the Generator.props file within:

<Project>
    <ItemGroup>
        <MyriadSdkGenerator Include="$(MSBuildThisFileDirectory)/../lib/netstandard2.1/Generator.dll" />
    </ItemGroup>
</Project>

Often an additional props file (In this smaple the file would be Generator.InTest.props) is used to make testing easier. The matching element for the tests fsproj would be something like this:

<Project>
    <ItemGroup>
        <MyriadSdkGenerator Include="$(MSBuildThisFileDirectory)/../bin/$(Configuration)/netstandard2.1/Generator.dll" />
    </ItemGroup>
</Project>

Notice the Include path is pointing locally rather than within the packaged nuget folder structure.

In your testing fsproj you would add the following to allow the plugin to be used locally rather that having to consume a nuget package:

<!-- include plugin -->
<Import Project="<Path to Generator plugin location>\build\Myriad.Plugins.InTest.props" />

Debugging

To debug Myriad, you can use the following two command line options:

--verbose - write diagnostic logs out to standard out
--wait-for-debugger - causes myriad to wait for a debugger to attach to the myriad process

These can be triggered from msbuild by the <MyriadSdkVerboseOutput>true</MyriadSdkVerboseOutput> and <MyriadSdkWaitForDebugger>true</MyriadSdkWaitForDebugger> properties, respectively.

Nuget

The nuget package for Myriad can be found here: Nuget package

How to build and test

Make sure you have .Net Core SDK installed - check required version in global.json
Run dotnet tool restore
Run dotnet fake build

How to release new version

Update CHANGELOG.md by adding new entry (## [0.X.X]) and commit it.
Create version tag (git tag v0.X.X)
Run dotnet fake build -t Pack to create the nuget package and test/examine it locally.
Push the tag to the repo git push origin v0.X.X - this will start CI process that will create GitHub release and put generated NuGet packages in it
Upload generated packages into NuGet.org

Caelestian / myriad