Pug templates need better filter support

Question

Pug templates need better filter support

kaceo opened this issue 3 years ago · comments

Kace at Telematique commented 3 years ago

Is your feature request related to a problem? Please describe.
One of the best feature of Eleventy is the ability to mix and match different template languages to produce a static site. This advantage falls short when some template languages are more empowered than others.

Currently Pug does not support Eleventy universal filters. This makes it difficult for Pug users to adapt to the Eleventy environment, or for a template in one language to be modified into a different language.

Describe the solution you'd like
In njk/liquid/md a data is evaluated like this: {{ myData | myFilter }}

In Pug a data is used like this: #{ myData }. It does not understand this expression #{ myData | myFilter } as only one variable name should be inside the parenthesis.

My solution is for Eleventy to run one pre-processing step in front of the Pug processing.

During this pre-processing step, Eleventy searches for these expressions #{ *** } and check if they need evaluation (whether there is a | inside the string):

if there is no filter, just skip this variable. It will be Pug's responsibility to perform substitution during run.
if there is a filter expression, Eleventy evaluates the filter and generate a temporary unique variable, and place the variable/value into the data stack, and replace the Pug template variable name.

As Eleventy is a compiler, data values can be determined without side effects during template evaluation. So the invention of temporary variables to store the evaluated eexpressions is safe.

Describe alternatives you've considered
Similar requests but that suggests a solution that requires extending Pug with user-customisation. In my opinion, that is a strategy that will lead to incompatibility between Pug and other template languages.

Additional context
I come into Eleventy because it is described as a "Jekyll in Nodejs". I tried to convert a complete site from Jekyll into Eleventy. Filters are the most problematic issues, as not all Jekyll filters have equivalent (eg. absolute_url, relative_url etc). But this is solveable by plugins.

Pug as my preferred template language further complicate the development, as it has only some of the expressive power of the other languages. Although I can rewrite the logic, I want to keep the Eleventy idioms as much as possible, and that means filters should be retained inside Pug.

Cosmo Myzrail Gorynych aka CoMiGo · Answer 1 · Wed Nov 25 2020 09:57:55 GMT+0800 (China Standard Time)

Why shouldn't filters be added as functions visible to pug, with globals?
https://pugjs.org/api/reference.html#options
and an example pugjs/pug#3198 (comment)

For example, we can have global.filters with all the registered filters from e11y, and they will be usable as #{filters.toc(content)} in pug compared to {{content | toc}}. Yes, it is global object pollution, but who cares if your project is a self-contained and self-sufficient site generator?

Cosmo Myzrail Gorynych aka CoMiGo · Answer 2 · Wed Nov 25 2020 10:19:57 GMT+0800 (China Standard Time)

.eleventy.js

// taking https://www.npmjs.com/package/eleventy-plugin-toc as an example
const pluginTOC = require('eleventy-plugin-toc');
const markdownIt = require('markdown-it')
const markdownItAnchor = require('markdown-it-anchor');

module.exports = function(eleventyConfig) {
    eleventyConfig.setLibrary('md', markdownIt().use(markdownItAnchor));
    eleventyConfig.addPlugin(pluginTOC);
    global.filters = eleventyConfig.javascriptFunctions; // magic happens here
    eleventyConfig.setPugOptions({ // and here
        globals: ['filters']
    });
};

anyPugFile.pug

aside.aPageNavigation!=filters.toc(content)

or

aside.aPageNavigation !{filters.toc(content)}

And it works! dab noises

Another example:

{% set previousPost = collections.posts | getPreviousCollectionItem(page) %}
{% set nextPost = collections.posts | getNextCollectionItem(page) %}
{% if previousPost %}Previous Blog Post: <a href="{{ previousPost.url }}">{{ previousPost.data.title }}</a>{% endif %}
{% if nextPost %}Next Blog Post: <a href="{{ nextPost.url }}">{{ nextPost.data.title }}</a>{% endif %}

becomes

- var prev = filters.getPreviousCollectionItem(collections.posts, page)
- var next = filters.getNextCollectionItem(collections.posts, page)
if prev
    a(href=prev.url)=prev.data.title
if next
    a(href=next.url)=next.data.title

chill-cod3r · Answer 3 · Sat Feb 20 2021 00:18:27 GMT+0800 (China Standard Time)

I think it's worth mentioning that the above solution solves the problem just fine. Additionally, the built in eleventy filters that you see in so many of the docs' examples like: {{ myThing | url }} will all "just work" if you do the above. In pug it's like:

a(href=filters.url('/my-unsafe-url')) My Link That Will Be Correct

Federico Peralta · Answer 4 · Sat Aug 28 2021 02:41:31 GMT+0800 (China Standard Time)

@wolfejw86 @CosmoMyzrailGorynych Amazing!! The above should be in docs!!

Shakeel Mohamed · Answer 5 · Fri Jan 28 2022 19:48:30 GMT+0800 (China Standard Time)

Has anyone taken a stab at this yet? 🤔

Adding a similar addPugGlobal() function to align with addNunjucksGlobal() seems doable (maybe with a global.pug namespace).

Adding an addPugFilter() function to user config would look something like below (untested, test changes needed, etc.).
I’m happy to get it ready for a PR if there’s interest—let me know! 🤠

diff --git a/src/Engines/Pug.js b/src/Engines/Pug.js
index 100b0c6..6788254 100644
--- a/src/Engines/Pug.js
+++ b/src/Engines/Pug.js
@@ -13,6 +13,8 @@ class Pug extends TemplateEngine {
   setLibrary(override) {
     this.pugLib = override || PugLib;
     this.setEngineLib(this.pugLib);
+
+    this.addFilters(this.config.pugFilters);
   }
 
   getPugOptions() {
@@ -27,6 +29,16 @@ class Pug extends TemplateEngine {
     );
   }
 
+  addFilters(filters) {
+    for (let name in filters) {
+      this.addFilter(name, filters[name]);
+    }
+  }
+
+  addFilter(name, filter) {
+    this.pugOptions.filters[name] = filter;
+  }
+
   async compile(str, inputPath) {
     let options = this.getPugOptions();
     if (!inputPath || inputPath === "pug" || inputPath === "md") {
diff --git a/src/UserConfig.js b/src/UserConfig.js
index e639b0f..2755ad0 100644
--- a/src/UserConfig.js
+++ b/src/UserConfig.js
@@ -49,6 +49,7 @@ class UserConfig {
     this.handlebarsPairedShortcodes = {};
     this.javascriptFunctions = {};
     this.pugOptions = {};
+    this.pugFilters = {};
     this.ejsOptions = {};
     this.markdownHighlighter = null;
     this.libraryOverrides = {};
@@ -222,6 +223,24 @@ class UserConfig {
     );
   }
 
+  addPugFilter(name, callback) {
+    name = this.getNamespacedName(name);
+
+    if (this.pugFilters[name]) {
+      debug(
+        chalk.yellow(
+          "Warning, overwriting a Pug filter with `addPugFilter(%o)`."
+        ),
+        name
+      );
+    }
+
+    this.pugFilters[name] = this.benchmarks.config.add(
+      `"${name}" Pug filter`,
+      callback
+    );
+  }
+
   addFilter(name, callback) {
     debug("Adding universal filter %o", this.getNamespacedName(name));
 
@@ -232,6 +251,8 @@ class UserConfig {
 
     // TODO remove Handlebars helpers in Universal Filters. Use shortcodes instead (the Handlebars template syntax is the same).
     this.addHandlebarsHelper(name, callback);
+
+    this.addPugFilter(name, callback);
   }
 
   getFilter(name) {
@@ -239,7 +260,8 @@ class UserConfig {
       this.javascriptFunctions[name] ||
       this.nunjucksFilters[name] ||
       this.liquidFilters[name] ||
-      this.handlebarsHelpers[name]
+      this.handlebarsHelpers[name] ||
+      this.pugFilters[name]
     );
   }
 
@@ -768,6 +790,7 @@ class UserConfig {
       handlebarsPairedShortcodes: this.handlebarsPairedShortcodes,
       javascriptFunctions: this.javascriptFunctions,
       pugOptions: this.pugOptions,
+      pugFilters: this.pugFilters,
       ejsOptions: this.ejsOptions,
       markdownHighlighter: this.markdownHighlighter,
       libraryOverrides: this.libraryOverrides,

bever1337 · Answer 6 · Tue Apr 05 2022 13:06:30 GMT+0800 (China Standard Time)

Because shakeelmohamed mentioned precedence for 11ty providing the global, it would be nice if pug users didn't require a userland solution. I think this would be a breaking change, but a very appreciated one as a pug user.

Shakeel Mohamed · Answer 7 · Thu Apr 07 2022 12:36:29 GMT+0800 (China Standard Time)

it would be nice if pug users didn't require a userland solution

I totally agree, @bever1337.
I’ll wait until this wheel gets squeakier before I commit any time to making a PR though.

Marvin Jérôme Stephan · Answer 8 · Sat Sep 24 2022 19:04:52 GMT+0800 (China Standard Time)

Has anyone taken a stab at this yet? 🤔

Adding a similar addPugGlobal() function to align with addNunjucksGlobal() seems doable (maybe with a global.pug namespace).

Adding an addPugFilter() function to user config would look something like below (untested, test changes needed, etc.). I’m happy to get it ready for a PR if there’s interest—let me know! 🤠

I just tried to incorporate this solution but got the error "Cannot set property 'slug' of undefined (via TypeError)"

npx @11ty/eleventy --serve [11ty] Problem writing Eleventy templates: (more in DEBUG output) [11ty] Cannot set property 'slug' of undefined (via TypeError) [11ty] [11ty] Original error stack trace: TypeError: Cannot set property 'slug' of undefined [11ty] at Pug.addFilter (/Users/Jerome/Documents/11ty/eleventy_local/node_modules/@11ty/eleventy/src/Engines/Pug.js:38:35) [11ty] at Pug.addFilters (/Users/Jerome/Documents/11ty/eleventy_local/node_modules/@11ty/eleventy/src/Engines/Pug.js:33:12) [11ty] at Pug.setLibrary (/Users/Jerome/Documents/11ty/eleventy_local/node_modules/@11ty/eleventy/src/Engines/Pug.js:16:10) [11ty] at new Pug (/Users/Jerome/Documents/11ty/eleventy_local/node_modules/@11ty/eleventy/src/Engines/Pug.js:10:10) [11ty] at TemplateEngineManager.getEngine (/Users/Jerome/Documents/11ty/eleventy_local/node_modules/@11ty/eleventy/src/TemplateEngineManager.js:92:20) [11ty] at TemplateRender.init (/Users/Jerome/Documents/11ty/eleventy_local/node_modules/@11ty/eleventy/src/TemplateRender.js:71:52) [11ty] at TemplateRender.get engine [as engine] (/Users/Jerome/Documents/11ty/eleventy_local/node_modules/@11ty/eleventy/src/TemplateRender.js:93:12) [11ty] at Template.get engine [as engine] (/Users/Jerome/Documents/11ty/eleventy_local/node_modules/@11ty/eleventy/src/TemplateContent.js:80:32) [11ty] at Template.getInputContent (/Users/Jerome/Documents/11ty/eleventy_local/node_modules/@11ty/eleventy/src/TemplateContent.js:167:15) [11ty] at Template.read (/Users/Jerome/Documents/11ty/eleventy_local/node_modules/@11ty/eleventy/src/TemplateContent.js:108:38) [11ty] Wrote 0 files in 0.25 seconds (v1.0.2)

Shakeel Mohamed · Answer 9 · Sun Sep 25 2022 04:37:07 GMT+0800 (China Standard Time)

Hi @HerzogVonWiesel, the key word in my previous comment was untested 😅. There’s probably a few issues with that patch.

Marvin Jérôme Stephan · Answer 10 · Sun Sep 25 2022 16:06:55 GMT+0800 (China Standard Time)

Hey @shakeelmohamed, for sure! Wasn't expecting for it to work, just pointing out an issue I found so we know what to fix when working further on this :)

Zearin · Answer 11 · Sun Mar 26 2023 01:50:58 GMT+0800 (China Standard Time)

(Psst! This Issue should be tagged with template-language:pug)

bever1337 · Answer 12 · Thu Mar 30 2023 05:17:42 GMT+0800 (China Standard Time)

What changed in 11ty 2.0 to break this established solution? Is there something pug users should tweak?

Edit: This could sound vague to non-pug users. I'll try and come up with a reproduction. I've got a lot of incentive to upgrade

Kace at Telematique · Answer 13 · Mon May 22 2023 08:45:56 GMT+0800 (China Standard Time)

Currently I am getting around the problems in Pug filters by using them only in the frontmatter of a pug layout, as documented. E.g.

eleventyComputed:
  nicedate: "{{ page.date | htmlDateString }}"

But this method fail when objects (not strings) need to be computed, such as the previous/next page links in a blog post:

eleventyComputed:
  previousPost: {{ collections.posts | getPreviousCollectionItem }}

For me this is the biggest weakness in the Pug + filter problem, a change in the template language from njk to pug requires major architectural redesign in the variables and expressions logic.

Ideally the frontmatter "eleventyComputed" should allow one fixed liquid-based filter expressions which should work for both strings and objects, and this must be independent of the rest of the template language.