How to check for unsatisfied tokens

[Bernhard Millauer]

I need to check if a template might have tokens that are not provided with the value object.

Based on the example from the readme I tried to capture the missing tokens with a token expander but I wasn't able to find a solution...

// You can add support for Partials via Token Expanders.
// Token Expanders can be used to extend Mustachio for many other use cases, such as: Date/Time formatters, Localization, etc., allowing also custom Token Render functions.

var sourceTemplate = "known = {{known}}, nested known = {{whatever.a}}, unknown = {{nothing}}";
var tokenExpander = new TokenExpander
{
	RegEx = new Regex("{{.*}}"), // you can also use Mustache syntax: {{> content }}
	Renderer = (string s, Queue<TokenTuple> q, ParsingOptions po, InferredTemplateModel itm) =>
		(sb, co) => {
			var ss = s;
			var qq = q;
			var poo = po;
			var itmm = itm;
			
			// somewhere here
		},
	Precedence = Precedence.Low
};
var parsingOptions = new ParsingOptions { TokenExpanders = new[] { tokenExpander } };
var template = Mustachio.Parser.Parse(sourceTemplate, parsingOptions);

// Create the values for the template model:
dynamic model = new ExpandoObject();
model.known = "Test";
model.whatever = new ExpandoObject();
model.whatever.a = "aa";

// Combine the model with the template to get content:
string content = template(model);

Console.WriteLine(content);

Could you guide me please?

[Vlad-Cosmin Sandu]

Hey @SeriousM ! I'm glad to see that you are using Mustachio. Below is a solution that I hope covers your use case. Let me know if you have any other questions!

var sourceTemplate = "known = {{known}}, nested known = {{whatever.a}}, unknown = {{nothing}}";
var tokenExpander = new TokenExpander
{
    RegEx = new Regex("{{.*}}"),
    ExpandTokens = (s, baseOptions) => Tokenizer.Tokenize(s, new ParsingOptions()), // let Mustachio render everything it can normally
    Renderer = (tokenString, tokenQueue, options, inferredModel) =>
        (stringBuilder, context) =>
        {
            var tokenPath = tokenString.TrimStart('{').TrimEnd('}').Trim(); // grab the path. e.g.: nothing
            var foundContext = context.GetContextForPath(tokenPath); // searches the model for a value for the given path

            if (foundContext.Value == null) // if it isn't found then you have a missing token in the model, so render or do anything you need
            {
                stringBuilder.Append("-MISSING VALUE-");
            }
        },
    Precedence = Precedence.Low
};
var parsingOptions = new ParsingOptions { TokenExpanders = new[] { tokenExpander } };
var template = Mustachio.Parser.Parse(sourceTemplate, parsingOptions);

// Create the values for the template model:
dynamic model = new ExpandoObject();
model.known = "Test";
model.whatever = new ExpandoObject();
model.whatever.a = "aa";

// Combine the model with the template to get content:
string content = template(model);

Console.WriteLine(content); // known = Test, nested known = aa, unknown = -MISSING VALUE-

[Bernhard Millauer]

That worked like a charm!

As a big thank you I would like to give you my implementation of a renderer that fails if a token is missing:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using Mustachio;

public class MustachioRenderer
{
  private readonly ParsingOptions parsingOptions;
  private Action<string> missingTokenCallback;

  private MustachioRenderer()
  {
    // https://github.com/wildbit/mustachio/issues/31

    var tokenExpander = new TokenExpander
    {
      RegEx = new Regex("{{.*}}"),
      ExpandTokens = (s, baseOptions) => Tokenizer.Tokenize(s, new ParsingOptions()), // let Mustachio render everything it can normally
      Renderer = (tokenString, tokenQueue, options, inferredModel) =>
        (stringBuilder, context) =>
        {
          var tokenPath = tokenString.TrimStart('{').TrimEnd('}').Trim(); // grab the path. e.g.: test
          var foundContext = context.GetContextForPath(tokenPath); // searches the model for a value for the given path

          if (foundContext.Value == null) // if it isn't found then you have a missing token in the model, so render or do anything you need
          {
            missingTokenCallback?.Invoke(tokenPath);
            stringBuilder.Append($"[MISSING TOKEN: '{tokenPath}']");
          }
	  },
	  Precedence = Precedence.Low
  };
		parsingOptions = new ParsingOptions { TokenExpanders = new[] { tokenExpander } };
	}

	public static RenderResult Render(PostmarkTemplate template, IDictionary<string, object> tokens)
	{
		return render(template, tokens, true);
	}

	public static bool TryRender(PostmarkTemplate template, IDictionary<string, object> tokens, out RenderResult renderResult)
	{
		renderResult = render(template, tokens, false);

		return !renderResult.HasErrors;
	}

	private static RenderResult render(PostmarkTemplate template, IDictionary<string, object> tokens, bool throwOnError)
	{
		var self = new MustachioRenderer();

		var result = new RenderResult();

		self.missingTokenCallback = missingToken => result.MissingTokens[nameof(RenderResult.Subject)].Add(missingToken);
		result.Subject = Parser.Parse(template.Subject, self.parsingOptions).Invoke(tokens);

		self.missingTokenCallback = missingToken => result.MissingTokens[nameof(RenderResult.Html)].Add(missingToken);
		result.Html = Parser.Parse(template.HtmlBody, self.parsingOptions).Invoke(tokens);

		self.missingTokenCallback = missingToken => result.MissingTokens[nameof(RenderResult.Text)].Add(missingToken);
		result.Text = Parser.Parse(template.TextBody, self.parsingOptions).Invoke(tokens);

		if (throwOnError)
		{
			throwExceptionOnError(result);
		}

		return result;
	}

	private static void throwExceptionOnError(RenderResult result)
	{
		if (!result.HasErrors)
		{
			return;
		}

		var sb = new StringBuilder();

		sb.AppendLine("Render of postmark templates failed because one or more tokens were missing:");
		reportMissingTokens(sb, result, nameof(RenderResult.Subject));
		reportMissingTokens(sb, result, nameof(RenderResult.Html));
		reportMissingTokens(sb, result, nameof(RenderResult.Text));

		throw new InvalidOperationException(sb.ToString());
	}

	private static void reportMissingTokens(StringBuilder sb, RenderResult result, string key)
	{
		var list = result.MissingTokens[key];
		if (list.Any())
		{
			sb.AppendLine($"{key}: {string.Join(", ", list)}");
		}
	}

	public class RenderResult
	{
		public string Subject { get; set; }
		public string Html { get; set; }
		public string Text { get; set; }

		public Dictionary<string, List<string>> MissingTokens { get; set; } = new Dictionary<string, List<string>>
		{
			[nameof(Subject)] = new List<string>(),
			[nameof(Html)] = new List<string>(),
			[nameof(Text)] = new List<string>()
		};

		public bool HasErrors => MissingTokens.Values.Any(l => l.Any());
	}
}