dep on ethereal can cause circular dependences - blame is on ethereal/tenplus1

[wsor4035]

ethereal yet again is still nuking things so a dep on it is needed to prevent this.
known circular dep tree: ethereal --> bakedclay --> flowerpot --> moretrees --> ethereal.
other issue trackers mentioning this: https://gitea.your-land.de/your-land/bugtracker/issues/3013 (thanks Niklp)

status: need to talk to tenplus1

[fluxionary]

ethereal yet again is still nuking things so a dep on it is needed to prevent this.

isn't the "nuking" (clearing biomes/decorations and re-registering them) now a configurable behavior?

[wsor4035]

iirc its now a setting, but it should still properly clear and reregister decorations

[Awkanimus]

The "nuking" is in "biome_init.lua" in ethereal's biome_ini.lua which is only triggered when ethereal.clear_default_biomes is set true (which it is by default). I happened to be looking at it last night.

The logic seems to be

1. create a map (table?) of existing biomes
2. create a map (another table?) of existing decorations
3. clear both
4. reregister biomes not on a discard-list,
5. reregister decorations if they are not in any of the discard-list's biomes.

I'm quite new here and to this codebase; there are of-course probably reasons why ... however I don't quite get why minetest.unregister_biome isn't used selectively on items in the discard list nor why decoration removal isn't the same. Also seems that the current method deprives all biomes of certain decos if they appear in even one discard-list biome. Feels like it's removing too much from the game at first glance.

[Awkanimus]

I'd be happy to try to solve this in ethereal and submit a pull request there. Does anyone know off-hand one-or-more decos which are disappearing or should I just dump them and look for a moretrees: prefix?

[wsor4035]

the simple way to test is clone both mods, remove the ethereal dep from moretrees, and add a moretrees dep to ethereal, and then see whats missing

[wsor4035]

ideally ethereal should keep its hands off anything that isnt made by it, and/or its directly modifying (like default)

[Awkanimus]

Based on my tests, I cannot find any indication that Ethereal is removing moretrees decos. If I missed a case or you'd like something else validated, please let me know.

Test setup:

1. Brand new world created with MTG (by chance... on a fresh computer with an empty mods folder)
2. Ethereal mod cloned fresh from TenPlus1's ethereal repo
3. Moretrees mod cloned fresh from here
4. Wrote a mod of my own created which uses minetest.register_on_mods_loaded to dump deco info after all mods are loaded, ethereal, and moretrees

Sampling procedure:

1. Delete debug.txt
2. Start minetest
3. Toggle the mods on/off to cover the cases
4. Start the world until I move around in it
5. Saved debug.txt with names related to the case
6. Extract deco info from logs and compare the results

Cases sampled:

1. Only my own mod loaded
2. Only moretrees loaded
3. Moretrees and ethereal loaded with no modifications
4. Moretrees and ethereal loaded with dependencies reversed (where ethereal deps on moretrees)

Moretrees adds a "decoration" field with a "moretrees:" prefix. This field is included in the extracted log info

General Conclusions:

1. In case 2 there are 17 decos with the moretrees prefix
2. In case 3 there are 17 decos with the moretrees prefix
3. In case 4 there are 17 decos with "moretrees:" are loaded
4. Comparing decos in case 2 vs case 3, there are 56 non-moretrees decos removed
5. Comparing decos in case 2 vs case 4, there are 56 non-moretrees decos removed.
6. I have manually validated that all 56 removed decos are from biomes included in ethereal's list of biomes to be removed (so ones it's directly modifying).

Attached are the lists of decos found in each case in case someone wants to double check my analysis 'cause it's fairly late here. I've also attached a zip with the debug mod I wrote for testing this.

decos_case_4_moretrees_and_ethereal_deps_reversed.txt
decos_ethereal_alone.txt
decos_case_1_none.txt
decos_case_2_moretrees.txt
decos_case_3_moretrees_and_ethereal.txt
awk_stats.zip

Edit: Missed the starting of the world in the sampling procedure.

[Awkanimus]

Assuming I haven't missed anything, can we remove that dependency on ethereal from moretrees? Is there something else I should try out? I'm happy to reopen my pull request which effects this change if it's helpful.

[fluxionary]

+1 on just removing the dependency. what is to be done about ethereal is a separate discussion.

[wsor4035]

@Niklp09 since you made the original claim #11 (comment) here, is this still a valid issue?

[Awkanimus]

Looked into ethereal's biome_init.lua's history a bit and commit ff99917e is very informative. The behavior @Niklp09 described is corrected in this commit; but anyone who has the previous version of ethereal and hasn't updated it would still fall pray. I can imagine this causing confusion long after the bug is fixed.

In the above commit, ethereal's version is changed to 20220424; If we want moretrees users to avoid this confusion and think it's worth it, we could do a version check and dump a warning. I'm imagining something like:

if ethereal then
    ethereal_ver = tonumber(ethereal.version)
    if (ethereal_ver and ethereal_ver < 20220424) then
        minetest.log("warning", "The version of ethereal detected can result in parts of this mod and others disappearing due to mod load order. Please update ethereal.");
    end
end

I'm not sure it's worth it though as the first thing we can ask is what version of ethereal they have if someone reports the issue. Depends on the expected frequency that people will have an old version of ethereal and a new version of moretrees.

[wsor4035]

your code would need to be wrapped in a register on mods loaded to always work without a dep on ethereal

[wsor4035]

proposal:

unless @Niklp09 can show the situation is still broken

reopen #21 - request a version of #22 (comment) snippet be added wrapped in register on mods loaded.

[wsor4035]

personal opinion on the snippet, iirc, that will only write to debug.txt which is more dev focused. instead or in addition to it should write something into the chat for the first player warning them, or similar to i3 does, block the server process from starting, since its not going to work properly anyways.

perhaps that the above would be better suited for a review in the pr if the above proposal is adopted

[Buckaroo Banzai]

block the server process from starting, since its not going to work properly anyways.

this ^ would be the way imo (if we are going the hacky route)

[wsor4035]

granted no one has said anything here/discord/irc/matrix in 15h, just going to reopen the pr, and request changes

[Awkanimus]

@wsor4035 all done in the pull request. Not sure how to mark this as resolved by that. Please let me know if more is needed.