"include-in-header" File <filename> not found in resource path

[Paul Brandt]

I have created a setup to organise different project files in distinct subdirectories. One of those relates to the file that I want to include in the header:

./
+templates/myHdr.sty

I've modified the pandocomatic.yaml accordingly:

...
   pandoc:
      include-in-header: './templates/myHdr.sty' # To easily include packages.
...

This works fine until the source markdown file is moved into its own subdir, e.g.,

./
+templates/myHdr.sty
+src/docs/mydoc.md

This results in the error File myHdr.sty not found in resource path

Apparently, the resource path is established relative to the source markdown location. This is confirmed by a flawless operation again when changing the setup in pandocomatic.yaml into:

...
   pandoc:
      include-in-header: './../../templates/myHdr.sty' # To easily include packages.
...

This is somewhat counterintuitive, where one, or at least I, would expect the paths to be established from the working directory, i.e., the root of the project that houses pandocomatic.yaml. Is the current operation based on an explicit design decision, or is it a by-effect of the current code?

[Huub de Beer]

I think the latter, but I have to look into this. Which in itself suggests that the behavior is not as consistent as expected or hoped for.

Unless we decide to change behavior, any change will result in breaking behavior. And If I'm changing this behavior, I might as well rewrite the whole path resolution behavior from scratch, but now well-reasoned and consistently. For example, it might make sense to separate the "single file conversion" behavior from the "convert directory / static site generation" behavior.

[Paul Brandt]

Thanks for looking into this.

If anything, there currently exists a work-around or "correct way" to go about this, i.e., by explicitly including the path to the subject file in the pandocomatic.yaml specification. So the community can proceed with it, turning this issue into a not-so-high-priority one.

I'm available for discussions or testing or ..., but only if you consider such useful.

[Huub de Beer]

I appreciate your input! I'm just one user with one way of working with pandoc/pandocomatic. If I want to improve pandocomatic, I need input from all users, or at least a representative set of them.

[Paul Brandt]

IGNORE (the previous contents of) THIS COMMENT

Apologies - my fault: What I thought I had done and had written in this comment as additional error messages, was not what I ACTUALLY had done. Incorrect source files, incorrect commands and more resulted in the additional, weird error messages. Apparently coding and whiskey don't go well together.

The File myHdr.sty not found in resource path comment is still valid, though.

[Huub de Beer]

Looking at your error message, I also wonder: why isn't there an input and output file in the verbose pandoc command at the top. Something is going very wrong here in pandocomatic!

[Paul Brandt]

Some more information for bug-hunting:

It appears that the -i path/to/input/file command line argument is considered the root for the path specified in parameter include-in-header: in the pandocomatic.yaml file. This is counter-intuitive, not only for projects like mine that have a directory structure to manage source documents, template files, bib files, image files, header files, generated results, etc., but more specifically because that behaviour is inconsistent with the bibliography: parameter that does not take the document folder as root for the location of the bib file. I always start pandocomatic from the root of the project, indicating all files relative to that root, which is successful for the bib-file.

Hence, could you remove the assumption for the include-in-header: parameter that its root folder equals the folder where the input document is located. In fact, it would be useful to assure that the behaviour on file location specification is consistent for each file that is specified in the pandocomatic.yaml configuration (and consistent with that of the command line, as well (not sure whether that is the case now or not)).

[Huub de Beer]

Looking in pandocomatic's code, I see that both bibliography and include-in-header are marked as "option with path". However, pandocomatic has different behavior for options that can occur more than once, like include-in-header, and options that can occur only once, like bibliography. The former, uses the "absolute destination path" when selecting the path to use, and the latter uses the specified destination path.

I don't know why this difference exists, so I suspect one of them is an error. I can try to align the behaviors and then prefer the behavior of bibliography, but this will affect all options that can occur more than once and have a path parameter.

[Huub de Beer]

I've started looking into this issue. If I understand the problem correctly, there's some confusion what a local path means in a template. There seem three different interpretations:

1. the current working directory, or
2. the input source file we're currently processing, which might be located in a subdirectory of or even completely elsewhere from the current working directory, or
3. the file the template is defined in

Observations:

• For an internal template, option 2 is the only one that makes sense.
• If a template is defined in a configuration file that's located in the current working directory, 1 and 3 are the same.
• If a template is defined elsewhere, all three can make sense depending on our interpretation.

According to the manual, the following should hold:

All local paths start with ./. These paths are local to the document being converted. When converting a directory tree, the current directory is being prepended to the path minus the ./.

Which seem to suggest that local paths follow interpretation 2.

I am not sure this is the most natural choice, particularly for templates used all over the place. Changing this behavior seems dangerous, though, as it would not be backwards compatible. And for global used templates, we already have the data directory we can set. Still, more control about where a path points to might be convenient.

Like we did for the root directory, we could a tag to a path to mark it as "start from this file". Say, $TEMPLATE$, or so. And also one for the working directory, like $WORKING_DIR$.

What do you think about this?

[Huub de Beer]

I've been thinking about my suggested solution a bit more, and found some issues:

• Either suggestion for special path is potentially dangerous because the input directory and output directory can be totally separate. For example, when using the --css option in pandoc that's relative to the working directory or template directory, you could end up with a path pointing to a directory outside of the regular scope of the output file's path.

  This wouldn't be an issue for preprocessors and the like that use used during the conversion but don't show up in the output. However, for URLs and paths that end up in the output, like includes, css, before/after, this is problematic.

  I don't want to introduce different path behavior for different situations, because that's confusing.

• Although a path relative to template makes sense from the point of a template author, that's less true for the path relative to the working directory. That variant would be more clear to the document writer, I suppose.

Furthermore, using the data directory relative paths or the new environment variable substitution feature (#103), I think template writers have enough power to specify paths. Similarly, document writers can always use local paths in their local or internal templates that are easy to understand and configure.

For now, I will not implement these two special directories $TEMPLATE$ and #WORKING_DIRECTORY$ because I am not convinced they are needed or will introduce more confusion than they resolve.