Fornax can't support files with non url safe characters

[Robert Pickering]

Describe the bug
Fornax can't support files with non url safe characters. If a mark down file in the posts directory contains a space in the file name, it will break the default template.

To Reproduce
Add a file named 'post with spaces.md' to the post directory
Test the template, it will fail to load.

Expected behaviour
The template should load

Environment (please complete the following information):

• OS: Windows 10
• Fornex version: build from latest master
• dotnet SDK version: 3.1.101

Additional context
The url encoding seems to come from this method:

    let relative toPath fromPath =
        let toUri = Uri(toPath)
        let fromUri = Uri(fromPath)
        toUri.MakeRelativeUri(fromUri).OriginalString

Building the site will fail with this error:

An unexpected error happend: System.IO.FileNotFoundException: Could not find file 'C:\code\Fornax\src\Fornax.Template\posts\post%20with%20spaces.md'.
File name: 'C:\code\Fornax\src\Fornax.Template\posts\post%20with%20spaces.md'
   at System.IO.FileStream.ValidateFileHandle(SafeFileHandle fileHandle)
   at System.IO.FileStream.CreateFileOpenHandle(FileMode mode, FileShare share, FileOptions options)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
   at System.IO.StreamReader.ValidateArgsAndOpenPath(String path, Encoding encoding, Int32 bufferSize)
   at System.IO.StreamReader..ctor(String path, Encoding encoding, Boolean detectEncodingFromByteOrderMarks)
   at System.IO.File.InternalReadAllText(String path, Encoding encoding)
   at System.IO.File.ReadAllText(String path)
   at FSI_0001.Config.postPredicate(String projectRoot, String page)
   at [e-mail](Tuple`2 tupledArg)
   at Microsoft.FSharp.Collections.ListModule.TryFind[T](FSharpFunc`2 predicate, FSharpList`1 list) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\list.fs:line 389
   at Generator.pickGenerator(Config cfg, SiteContents siteContent, String projectRoot, String page) in C:\code\Fornax\src\Fornax\Generator.fs:line 284
   at Generator.generate(FsiEvaluationSession fsi, Config cfg, SiteContents siteContent, String projectRoot, String page) in C:\code\Fornax\src\Fornax\Generator.fs:line 315
   at Generator.action@1(String projectRoot, FsiEvaluationSession fsi, Config config, SiteContents sc, String filePath) in C:\code\Fornax\src\Fornax\Generator.fs:line 483
   at Generator.generateFolder(String projectRoot) in C:\code\Fornax\src\Fornax\Generator.fs:line 482
   at Fornax.guardedGenerate@168(String cwd, Unit unitVar0) in C:\code\Fornax\src\Fornax\Fornax.fs:line 170
Finished (Failed) 'TestTemplate' in 00:00:07.9363273

We could fix this by calling System.Web.HttpUtility.UrlDecode, but that results in the following error:

An unexpected error happend: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.Collections.Generic.KeyNotFoundException: An index satisfying the predicate was not found in the collection.
   at Microsoft.FSharp.Collections.SeqModule.Find[T](FSharpFunc`2 predicate, IEnumerable`1 source) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 677
   at FSI_0034.Post.generate'(SiteContents ctx, String page)
   at lambda_method(Closure , Unit , SiteContents , String , String )
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Generator.EvaluatorHelpers.helper@53(Object next, FSharpList`1 args) in C:\code\Fornax\src\Fornax\Generator.fs:line 63
   at Generator.EvaluatorHelpers.invokeFunction(Object f, IEnumerable`1 args) in C:\code\Fornax\src\Fornax\Generator.fs:line 67
   at [e-mail](FsiValue ft) in C:\code\Fornax\src\Fornax\Generator.fs:line 177
   at Generator.generate(FsiEvaluationSession fsi, Config cfg, SiteContents siteContent, String projectRoot, String page) in C:\code\Fornax\src\Fornax\Generator.fs:line 322
   at Generator.action@1(String projectRoot, FsiEvaluationSession fsi, Config config, SiteContents sc, String filePath) in C:\code\Fornax\src\Fornax\Generator.fs:line 483
   at Generator.generateFolder(String projectRoot) in C:\code\Fornax\src\Fornax\Generator.fs:line 482
   at Fornax.guardedGenerate@168(String cwd, Unit unitVar0) in C:\code\Fornax\src\Fornax\Fornax.fs:line 170

Again, we could fix this by calling System.Web.HttpUtility.UrlDecode, but means the generated Url for the page is incorrect because it has not been correctly url encoded. This is harder to do, as we need to encode the file and directory names but the directory separator characters.

[Robert Pickering]

I was thinking about this, the issue seems to stem from the way the loaders talk to the generators. First the loads are run and enumerate the file system and populate the SiteContents with what ever data they feel necessary then the file system is enumerated by fornax itself and each pages is generated, and during the generation phase it's given chance to look up if a loader stored any data for it.

Wouldn't it be better to have the data from sitecontent passed directly to the generator? Eliminating the double enumeration of the file system. It would mean static content would also need to loaded in someway, but I think that's do able.

It's a big change, so I thought I discuss here before trying to implement.