Enhancement: Support for subfolders for locale content

[Tony OHagan]

Is your feature request related to a problem? Please describe.

I'm creating web sites that will support content translation with posts and other content types that each have a subfolder for each locale supported. For example ...

• content/posts/en
• content/posts/de
• content/posts/de_CH

Describe the solution you'd like

I like to be able to specify that a Front Matter pageFolder contains locale specific subfolders. I'd like to then select a locale in the Dashboard to list, create and edit content in corresponding locale subfolders.

Describe alternatives you've considered

I know there is some support for subfolders but after the reading the documentation I'm a bit confused about how this might be used for this common scenario. If that can be used instead then perhaps it needs a recipe in the documentation.

[Tony OHagan]

I'd recommend that the locale codes be configurable rather than hard wired into Front Matter as I've found that different tools and deployment environments will use slightly different locale codes. Also a given project may only wish to support a limited list of languages and locales.

[Elio Struyf]

Internationalization/multilingual content has been on the list for a long time; also a discussion was created a while ago: #527

The biggest challenge is that we will have an opinion on how this content needs to be created in separate folders or linking pages in the front matter are my preferred options.

In Astro, the following structure is used for internationalized content:

- src
  - pages
    - about.astro
    - index.astro
    - es
      - about.astro
      - index.astro
    - pt-br
      - about.astro
      - index.astro

The default content is created on the root of the pages directory, and all localized content is created in their corresponding language directory.

How it could work in Front Matter CMS

Configuration

First, on the page folder level, I'm thinking of adding a new property that allows you to define the root locale folder and content:

{
  "frontMatter.content.pageFolders": [
    {
      "title": "posts",
      "path": "[[workspace]]/src/content/posts",
      "defaultLocale": true
    }
  ]
}

For the other languages, the easiest option is to add a new setting where you can define the locales:

{
  "frontMatter.content.i18n": [
    {
      "title": "German",
      "locale": "de",
      "folder": "de"
    },
    {
      "title": "Nederlands",
      "locale": "nl",
      "folder": "nl"
    }
  ]
}

In this setting, you can define any locale like you suggested (the title would be optional, only used to show it during the content creation).

How it works

Once you've configured it, the flow will be as follows:

1. Create new content in the default language
2. Have a new command available where you can say: create new localized content
3. You get the locale options to pick from
4. Once you've selected the locale, it creates a copy of the original file in the locale folder (if the folder does not yet exist, it will be created at the same time)
5. Now you can start translating the content

What do you think about this approach?

[Elio Struyf]

Todo

• Highlight i18n content (tag/label on content card)
• New command for i18n content creation
  • Disable the command when page is not a default language page
• New panel action for i18n content creation (when on a default language page)

[Elio Struyf]

[Tony OHagan]

I suspect other Front Matters users will wish to make different choices on i18n folder structure to the one recently added by Astro.

In my case, I've built a multi-lingual site that needs to put translated content in .md or .mdx files with .astro files only used to structure the i18n content using dynamic routes. My pages support tags and categories that cross link pages all while supporting i18. To do this you need consistent route paths that place the [locale] in the path in a consistent manner with en being in a subfolder.

Another important reason I can't put are en English in a parent folder is because I use a CDN Network (Firebase) that supports i18 translations but only if the English resides in a en subfolder - not in the parent folder.

I use Front Matter to edit .mdx and .md files structured like so ...

• src
  • content
    • posts
      • en
        • article1.mdx
      • es
        • article1.mdx
      • pt-br
        • article1.mdx
    • site
      • en
        • terms.md
        • privacy.md
      • es
        • terms.md
        • privacy.md
      • pt-br
        • terms.md
        • privacy.md

[Elio Struyf]

@tohagan this will be supported. There will be a global i18n setting (as defined above), or you can define it on page folder level.

Here is an example I'm currenlty testing:

{
  "frontMatter.content.pageFolders": [
    {
      "title": "EN post",
      "path": "[[workspace]]/src/content/posts/en",
      "contentTypes": [
        "simple"
      ],
      "defaultLocale": "en",
      "locales": [
        {
          "title": "English",
          "locale": "en"
        },
        {
          "title": "German",
          "locale": "de",
          "path": "../de"
        },
        {
          "title": "Nederlands",
          "locale": "nl",
          "path": "../nl"
        }
      ]
    }
  ]
}

[Tony OHagan]

You could just have a language selector on the toolbar. A reason is that you often want to stay working on the one post but switch languages or (better) view 2 translations side by side so you can check source and target language together. Changing posts could then mean that you keep the same selected language(s). You don't want to have to drill down from language => post every time.

[Elio Struyf]

That will also be the goal. Have the ability to see all or just one language.

[Tony OHagan]

Awesome! I have a non-tech volunteer assisting my current project. Front Matter will be extremely useful to help him contribute and not get lost in the Astro jungle.

You might even inspire me to open source the auto-translation engine I've developed that incrementally translates .md and .mdx files as you save them. It includes a feature to improve translation quality + other components that are locale aware.

[Elio Struyf]

There are two options to show the content:

Any preferences between A and B?

[Tony OHagan]

I think you'd want it to be part of the filter (Option B) as that gives the user the most feature richest solution. In my case, I'd want to combine a filter of Language + Type when selecting the initial post to edit.

Now .. once you've selected and opened a post in the markdown editor; I think you then want a way to switch languages without having to go back to the filtered cards. To support side by side review/editing, a user could open side-by-side panels in VSCode but the downside of that is that you'd then need to repeatedly find and select the same post in each of the two panels which you'd want to avoid.

One idea I had was that editor included dropdown list of languages that supported multi-selection (or limit to 1, 2 maybe 3 selections). Drop down label needs to hint at this so it's discoverable. You might come up with something better.

Finally a translation reviewer/editor would likely want to preview all the filtered posts from the editor. To do this, the editor toolbar might have < Prev and > Next buttons to navigate between filtered posts without changing the selected languages so the languages in the side by side views remain the same.

[Elio Struyf]

In the latest beta, the multilingual features are now available. Information on how you can configure and use it can be found in the beta version of the docs: https://beta.frontmatter.codes/docs/content-creation/multilingual

In case of questions, feel free to reach out.

[Tony OHagan]

I have another suggestion that would simply the paths. If you make the path default to "./locale" then you can use this configuration that sets the root path to be "[[workspace]]/docs"

For my case I now no longer need to set the path property as it will default correctly for all the locales.

For the the one added by Astro, most locale folders will be correct with the exception of English which would have path "." .

For the example in your docs ...

{
  "frontMatter.content.pageFolders": [
    {
      "title": "Multilingual",
      "path": "[[workspace]]/docs",
      "filePrefix": "",
      "defaultLocale": "en",
      "locales": [
        {
          "title": "English",
          "locale": "en",
          "path": "."
        },
        {
          "title": "Nederlands",
          "locale": "nl"
        },
        {
          "title": "German",
          "locale": "de"
        }
      ]
    }
  ]
}

[Tony OHagan]

The translation engine I've built supports multiple translation APIs. I've added Google and Azure APIs but if I was doing European languages then I'd add DeepL of course. Google Translation has the downside that you're required to include attribution and is twice the price of Azure. I have rules to parse MD and MDX files and selectively translate front matter YAML attributes + I have a couple of custom components related to my specific needs that are locale aware.

[Elio Struyf]

As you suggested, I changed the logic of the paths.

The documentation has been updated.

[Elio Struyf]

The translation engine I've built supports multiple translation APIs. I've added Google and Azure APIs but if I was doing European languages then I'd add DeepL of course. Google Translation has the downside that you're required to include attribution and is twice the price of Azure. I have rules to parse MD and MDX files and selectively translate front matter YAML attributes + I have a couple of custom components related to my specific needs that are locale aware.

Azure Translation Service is also on the list, not yet sure if it will make it into the current release. DeepL is used as the first integration sample, can be expanded in the future or with custom scripts.

[Hrihorii Ilin]

I was waiting for this moment for a looooong Time...
"ask AI how to create astro website with Ukrainian localization 2 years ago.. " - mission complete!

I am use GitLocalize for my new projects.. And 5 days ago sync your CMS with Obsidian, and its working well (with frontmatter plugins and excalidraw). Thank you for awesome work!!