Can't import base packages from a nested import

[FascinatedBox]

Better support for multi-file packages

This is what happens when I currently try import example:

SyntaxError: Cannot import 'example':
    no preloaded package 'example'
    no file './example.lily'
    no file './example.so'
    no file './packages/example/src/example.lily'
    no file './packages/example/src/example.so'

Right now, Lily's import behavior is to try paths relative to the first module that was loaded. If you're working with a project in isolation, then that's totally fine. Deeper directories just mean having to import a longer path since there's no relative paths right now.

But what happens when garden "installs" a local package. For single-file packages, it doesn't matter because they don't run imports since there's no reason to. But let's say there IS a complex package. That package should be able to run imports without knowing where it's been installed.

The current import strategy can be described as:

• Based on the first module loaded (looking for a plain file or library)
• Based on the first module loaded (looking for a package file or a package library)

The new strategy would add the concept of a 'root module'. A root module is the first module loaded in a packages directory. The first module loaded would automatically be a root.

Now the import paths would be more like:

• From the last root module (for plain file or library)
• From the last root module (package file or package library)

Internally, this would be done by changing the import hook. I'm thinking the import hook could return a status of either failed, success (plain file), sucess (package). Bubbling up root information wouldn't be that difficult.

[FascinatedBox]

I don't quite understand what's going on here. The interpreter loads modules relative to the current one which...seems fine? I created a fake package containing one module that imports another and that seems to work. Past me didn't leave a good example to go off of. The import system seems okay.

Closing this because there's not enough to go off of.

[FascinatedBox]

I think I figured it out. Past me identified the wrong problem. The real problem is this:

Right now, imports run relative to the current file. That usually works, because importing a deeper file means adding a slash. Importing a file upward means putting leading dots.

Let's say we've got a package organized like so:

foo.lily
packages
    postgres
        src
            postgres.so/dll
bar
    abc.lily

Let's say in this scenario that abc.lily wants to use postgres.

Too bad.

It it tries to use ../postgres, the path it tries will be ./packages/../postgres/src/../postgres.so

That path isn't going to work.

What if it tries import postgres. Well, since imports are relative and postgres is in the wrong directory, that isn't going to work either.

The current import system makes it impossible for abc.lily to make use of postgres.

The solution to this comes in several parts:

• Imports must be run relative to the root of a package. An import is considered the root if it's the first file opened, or if it is along a packages path.

• Two new api calls will be written: lily_load_package_{file,string}. These calls will indicate that the module is a root, and imports from it will run relative to the path it was given.

• The directory given to the import hook will be the root directory of the current module, not a relative path.

A downside of this is that imports will need to specify slashed paths where they may have not needed to. For example, if abc.lily wants to import a sibling file in the same directory, it needs to use the slashed path and not a simple import <name>.

This is really close to the release window of 1.3. I deeply dislike having this kind of a change so close to a release, but I don't see a better alternative. I don't want to delay a release, or worse, have a release that has an utterly broken mechanic. I don't want a huge theme of 1.4 to be "please download the bleeding edge because I was dumb and something important is broken".

It's not all bad though! Fixing this is easy in core (modules get a shallow copy root path), and from there it's a matter of getting tests to work again plus documenting the new api/tutorial.