translations not loading automatically

Question

translations not loading automatically

HoneyryderChuck opened this issue 3 years ago · comments

Hi Janko,

I started the work of integrating rodauth-i18n support into rodauth-oauth. It's mostly going well, except for these 2 things I'd want to report you, and see if these are intended behaviour or actual bugs.

The first issue is that the translations overall do not load until I call Rodauth::I18n.add (see the WIP commit which allows the tests to work). This doesn't seem intuitive IMO. I'd expect this to be taken care of out-of-the-box as soon as enable :i18n is called. Why isn't this happening, and the rationale for the current behaviour?

The second one has to do with the rake task to copy the translations to the expected dir for rails. Will this rake task pick up on other plugins files (such as the ones being defined for rodauth-oauth?

Janko Marohnić · Answer 1 · Sun Nov 28 2021 01:23:51 GMT+0800 (China Standard Time)

Hi Tiago, thanks for reporting.

The first issue is that the translations overall do not load until I call Rodauth::I18n.add (see the WIP commit which allows the tests to work). This doesn't seem intuitive IMO. I'd expect this to be taken care of out-of-the-box as soon as enable :i18n is called. Why isn't this happening, and the rationale for the current behaviour?

That's a good point. The reason I took a more conservative approach is that AFAICT I18n.load_path cannot be modified anymore after translations have been loaded, and in an environment that uses autoloading (e.g. Zeitwerk), I wasn't sure whether the Rodauth app could ever be loaded after I18n.translate was already called (which lazily loads translations). I couldn't do it on gem require, because I18n.available_locales will likely be modified after requiring gems (which rodauth-i18n uses to load only a subset of translations).

In any case, Rodauth::Rails.add should only be called by the user, gems should only register locale directories. Also, it's probably safer to extend Rodauth::Rails.directories as soon as the gem is required, because at least in rodauth-rails, the Rodauth app is autoloaded after the Rails app is initialized (at which point Rodauth::Rails.add is already called, extending I18n.load_path).

The second one has to do with the rake task to copy the translations to the expected dir for rails. Will this rake task pick up on other plugins files (such as the ones being defined for rodauth-oauth?

Good catch, not at the moment. I'm not sure if it's better to create a new file (e.g. rodauth-oauth.en.yml) or merge translations into rodauth.en.yml (more difficult). In case of the former, rodauth-i18n would somehow need to know the name of the gem registering its translations.

Tiago · Answer 2 · Sun Nov 28 2021 22:52:03 GMT+0800 (China Standard Time)

The reason I took a more conservative approach is that AFAICT I18n.load_path cannot be modified anymore after translations have been loaded, and in an environment that uses autoloading (e.g. Zeitwerk), I wasn't sure whether the Rodauth app could ever be loaded after I18n.translate was already called (which lazily loads translations).

I guess this is the reason why you're not suggesting loading it all in a rodauth hook, is it? Because this hook might be called multiple times? I might have a couple of thoughts about it, more below. However, I don't find anything in i18n code preventing this from happening, unless you mean some cache in between?

In any case, Rodauth::Rails.add should only be called by the user, gems should only register locale directories.

I don't agree, I'd change this to "should be called once", which is the requirement I think you're trying to enforce here. i18n integration in rails doesn't require explicit calls by the user, and the same experience should be targeted here.

Also, it's probably safer to extend Rodauth::Rails.directories as soon as the gem is required, because at least in rodauth-rails,...

I understand that, but it seems more like a choice based on a rails constraint, and a bit of a design deviation from what rodauth usually goes for, which is, to use rodauth hooks as much as possible. I believe that this could be addressed in one of 2 ways:

either modify the suggestion to Rodauth::I18n.directories << File.expand_path("#{__dir__}/../locales") unless Rodauth::I18n.directories.include?(File.expand_path("#{__dir__}/../locales"))
change directories to a set.

resetting i18n structures could also happen if possible, and perhaps makes more sense on autoloading, as tearind down would pick up new translations, which might not happen now?

#######

I believe that all of the above could be solved better if rodauth would allow features to have single "on feature load" hooks, which would be called once for the lifetime of a process, kind of like roda has load and configure.

Alternatively, rodauth-i18n could hook on roda`s callbacks if it were a "mainline rodauth" plugin, which I still believe it'd make sense to happen tbh (let me know if I should start lobbying in the mailing list :) ).

Janko Marohnić · Answer 3 · Sun Nov 28 2021 23:38:21 GMT+0800 (China Standard Time)

However, I don't find anything in i18n code preventing this from happening, unless you mean some cache in between?

Extending I18n.load_path (which Rodauth::Rails.add does) doesn't have any affect after the translations have already been loaded, which is shown in the following example:

require "rodauth"
require "rodauth/i18n"
require "sequel"

DB = Sequel.sqlite

I18n.available_locales = [:hr]
I18n.locale = :hr

# called before Rodauth app is loaded
I18n.translate("foo") # loads translations

rodauth = Rodauth.lib { enable :i18n }

# called in feature's #post_configure
Rodauth::I18n.add

p rodauth.allocate.login_label
# "translation missing: hr.rodauth.login_label"

For app that uses autoloading in development, I thought it's theoretically possible that I18n.translate is called before the Rodauth app is loaded. But I'm probably just paranoid, perhaps we can count with Rodauth app being loaded before each request.

I don't agree, I'd change this to "should be called once", which is the requirement I think you're trying to enforce here. i18n integration in rails doesn't require explicit calls by the user, and the same experience should be targeted here.

I agree, I also want to eliminate the user having to manually call Rodauth::Rails.add. What I was saying is that 3rd-party gems should definitely not be calling it, as it's meant to be called by the user or internally in rodauth-i18n.

I understand that, but it seems more like a choice based on a rails constraint, and a bit of a design deviation from what rodauth usually goes for, which is, to use rodauth hooks as much as possible.

I was imagining any app that uses autoloading in development (Zeitwerk is not Rails-specific).

either modify the suggestion to Rodauth::I18n.directories << File.expand_path("#{__dir__}/../locales") unless Rodauth::I18n.directories.include?(File.expand_path("#{__dir__}/../locales"))

change directories to a set.

I like the idea of using a set 👍🏻

resetting i18n structures could also happen if possible, and perhaps makes more sense on autoloading, as tearind down would pick up new translations, which might not happen now?

That's the thing, I don't know how to reset I18n without forcing translations to be loaded. I18n.reload! drops the cache, but also loads translations, and I don't want rodauth-i18n to affect when translations are loaded.

I believe that all of the above could be solved better if rodauth would allow features to have single "on feature load" hooks, which would be called once for the lifetime of a process, kind of like roda has load and configure.

I think making Rodauth::Rails.add and changing Rodauth::Rails.directories idempotent could work around #post_configure being called multiple times.

Alternatively, rodauth-i18n could hook on roda`s callbacks if it were a "mainline rodauth" plugin, which I still believe it'd make sense to happen tbh (let me know if I should start lobbying in the mailing list :) ).

Because of Rails generators, and also this being a repository for translations in different languages (where lots of PRs are expected adding/correcting various languages), I think it might make more sense to keep it as a separate gem. I'm not against it, though, I am curious what Jeremy thinks of it.

Tiago · Answer 4 · Mon Nov 29 2021 00:30:20 GMT+0800 (China Standard Time)

For app that uses autoloading in development, I thought it's theoretically possible that I18n.translate is called before the Rodauth app is loaded.

I see, good point. I think rails circumvents this by loading translations during a railtie on_load hook, could something like this be specifically done in this gem's railtie as well?

What I was saying is that 3rd-party gems should definitely not be calling it, as it's meant to be called by the user or internally in rodauth-i18n.

Indeed, the commit was just to show my current in-progress workaround (which changed already btw).

I was imagining any app that uses autoloading in development (Zeitwerk is not Rails-specific).

Just a tangent, roda isn't that compatible with zeitwerk anyway, for all features, as per this template change suggestion. Perhaps it does not apply fully to rodauth though.

That's the thing, I don't know how to reset I18n without forcing translations to be loaded.

Rails tests seem to do I18n.load_path.clear. Not great, I guess, but as autoloading goes, doable(?)

Because of Rails generators, and also this being a repository for translations in different languages (where lots of PRs are expected adding/correcting various languages), I think it might make more sense to keep it as a separate gem.

I believe that the rails generator could be moved to rodauth-rails if i18n could become a default plugin. Nevertheless, point taken about "several language files being added". I still think we could bring this argument, as rodauth has a lot of people following the project, and could in theory attract more potential translation contributors than this fresh repository (at the expense of passing the burden of maintenance to Jeremy, which is perhaps what you want to avoid?).

About the roda configure callback as a potential candidate to call .add on, would you agree that it would fix the "single call" problem? Any other possible workaround?

Janko Marohnić · Answer 5 · Mon Nov 29 2021 02:41:50 GMT+0800 (China Standard Time)

I think rails circumvents this by loading translations during a railtie on_load hook, could something like this be specifically done in this gem's railtie as well?

Yes, this is exactly what rodauth-i18n currently does.

Just a tangent, roda isn't that compatible with zeitwerk anyway, for all features, as per this template change suggestion. Perhaps it does not apply fully to rodauth though.

Interesting to read the discussion.

I believe that the rails generator could be moved to rodauth-rails if i18n could become a default plugin.

Good point 👍🏻

I still think we could bring this argument, as rodauth has a lot of people following the project, and could in theory attract more potential translation contributors than this fresh repository (at the expense of passing the burden of maintenance to Jeremy, which is perhaps what you want to avoid?).

Yeah, once we figure out whether we can automatically load built-in translations using Rodauth hooks (removing the need to call Rodauth::Rails.add), I would feel comfortable proposing to merge rodauth-i18n into Rodauth.

About the roda configure callback as a potential candidate to call .add on, would you agree that it would fix the "single call" problem? Any other possible workaround?

I don't think it's really a problem that post_configure is called multiple times, as long as the code for registering locale directories and extending I18n.load_path is idempotent (which shouldn't be too difficult). I wouldn't know how to hook up to roda plugin's configure callback.

Tiago · Answer 6 · Mon Nov 29 2021 21:17:01 GMT+0800 (China Standard Time)

Yes, this is exactly what rodauth-i18n currently does.

Totally missed it 🙈

After doing a bit more research, I think I understand the problem better now. I do agree now, there is no hook for calling .add in rodauth, as the call should be owned by the framework you're running it (such as in the rails initializer case).

However, let me propose something completely different altogether: Rodauth::I18n.add could be ditched altogether, or have its semantics changed.

I was looking how i18n is set up in rails, and it seems that the different modules just add paths to load_paths, having the assurance that, under the rails initializer on_load, they'd be called only once. rodauth doesn't have a similar callback (post_configure gets called multiple times), but a combination of the suggestions made above could provide a similar semantic. Consider a hypotethicall Rodauth::I18n.register_translations_dir, that could be called like this at post_configure time:

def post_configure
  Rodauth::I18n.register_translations_dir(File.expand_path("#{__dir__}/../locales")
end

In order for this to comply with the "call exactly once" semantics necessary, this routine could store paths into a set, and immediately load them to the i18n paths the first-time:

module Rodauth
  module I18n
    def self.register_translations_dir(path)
      return if @directories.include?(path)

      @directories << path
      ::I18n.load_path.concat(files(path)) # .files would receive the explicit directory
    end
  end
end

This suggestion has 2 assumptions baked in:

I'm implying it's theoretically possible that I18n.translate is called before the Rodauth app is loaded isn't an issue, as rodauth-rails bundles itself as a rack middleware which has to be loaded and traversed on all requests, so technically nothing gets translated before that happens (HTML-wise at least).
The .add implementation has an optional locales parameter. This doesn't seem to be actionable in practice, as locale constraints seem to belong to feature config, rather than something that you might want to pass as argument at load-time (and in fact the railtie doesn't make any use of it). However, if there is some actual use-case for it, might make the semantics of my suggestion above more complicated.

Janko Marohnić · Answer 7 · Wed Dec 01 2021 03:50:31 GMT+0800 (China Standard Time)

Hmm, I was now playing with extending I18n.load_path only in post_configure, and in Rails with the Rodauth app being autoloaded on first request, the user-defined translations unfortunately get loaded before rodauth-i18n's translations, which causes rodauth-i18n's translations to overwrite user-defined translations 😕

Tiago · Answer 8 · Thu Dec 02 2021 21:17:15 GMT+0800 (China Standard Time)

That's interesting 🤔 this means that RodauthApp is never fully loaded unless the path is part of rodauth routes. I'm not sure if there are other implications of breaking with this assumption, so I assume this can't be changed.

I was thinking that, in order to fix the order of translations, that rodauth plugin translations could be append to the beginning of I18n.load_paths, instead of at the end, which is the behaviour happening if they'd be registered in post_configure. That'd fix the order, but not be picked upon if translations were cached (after I18n.translate is used for the first time). This could be however fixed by a rodauth-i18n post_configure hook which would tear down cache (ensuring that "at beginning appended rodauth translations" would be picked upon come next translate calll).

I'm not sure if this is a path you think is worth exploring, though, so let me know what you think of the description above. As far as the current implementation goes, I think the current suggestion (load translations on gem require) is reasonable enough, even if dependent on Bundler.require being used, which not all frameworks do by default.

Janko Marohnić · Answer 9 · Fri Dec 03 2021 01:43:47 GMT+0800 (China Standard Time)

I was thinking that, in order to fix the order of translations, that rodauth plugin translations could be append to the beginning of I18n.load_paths, instead of at the end, which is the behaviour happening if they'd be registered in post_configure.

It seems we both arrived to the same idea 🤘🏻. I've submitted #5 that moves translation file registration in the feature, where locale files are prepended to the I18n.load_path. Let me know if this is what you had in mind overall.

Janko Marohnić · Answer 10 · Fri Dec 03 2021 01:51:04 GMT+0800 (China Standard Time)

this means that RodauthApp is never fully loaded unless the path is part of rodauth routes.

It is actually fully loaded on the first request, regardless of the request path, it's just that user-defined locales are loaded when the application is initialized 😉

Tiago · Answer 11 · Fri Dec 03 2021 20:43:29 GMT+0800 (China Standard Time)

Lol, inception achieved! 😂

Had a look at the PR, don't think I could do better than you did 👍

It is actually fully loaded on the first request, regardless of the request path, it's just that user-defined locales are loaded when the application is initialized 😉

Gotcha, makes sense!