natemcmaster / DotNetCorePlugins

Hi,

first of all thank you for your effort on this project. This is a really nice project and I try to adapt it for our plugin system.

Since I need plugins that can have a reference to some other plugins I looked at this issue #60

I encountered some problems by trying the two suggested solutions and wanted to try another way.

Starting with my Idea I found that this following lines will not resolve the type defined in my created AssemblyLoadContext, but the type defined in the Assembly instance loaded by the PluginLoader.

string? pluginName = Path.GetFileName(pluginDirectory);
string assemblyFile = Path.Combine(pluginDirectory, pluginName + ".dll");

AssemblyLoadContext pluginAssemblyLoadContext = new AssemblyLoadContext(pluginName);
pluginAssemblyLoadContext.LoadFromAssemblyPath(assemblyFile);

PluginLoader loader = PluginLoader.CreateFromAssemblyFile(
    assemblyFile: assemblyFile,
    true,
    sharedTypes: Array.Empty<Type>(),
    config =>
    {
        config.DefaultContext = pluginAssemblyLoadContext;
        config.PreferSharedTypes = true;
    });

Assembly pluginAssembly = loader.LoadDefaultAssembly();
Type[] allTypes = pluginAssembly.GetTypes();

IEnumerable<Type> pluginTypes = allTypes.Where(t => typeof(IPlugin).IsAssignableFrom(t) && !t.IsAbstract);

Why is this? I'm not sure if I found all the available documentation about the ALC and this PluginLoader lib stuff, so maybe I just didn't understand all that ALC fallback things right.
My assumption was that it should be possible to build a tree of ALCs that has the default ALC (implicitly) as root.

I thought here

DotNetCorePlugins/src/Plugins/Loader/ManagedLoadContext.cs

Line 100 in 8312d70

    
           if ((_preferDefaultLoadContext || _defaultAssemblies.Contains(assemblyName.Name)) && !_privateAssemblies.Contains(assemblyName.Name))

this "routing" is done - but it seems that I missed something.

Can you share a complete minimal repro? The code snippet above is a good start, but insufficient to reproduce your problem. For example, how does this line work? new AssemblyLoadContext(pluginName). AssemblyLoadContext is an abstract type, so this direct constructor use wouldn't compile.

Thank you for your response.

Since .net core 3.0 AssemblyLoadContext is not anymore a abstract type.

See this issue

At least this code "runs on my machine" ;-)

I'm using .net core 3.1.

I made a minimal repo for my complete idea - which is possibly a bit more complex than needed. I'll extract this part into a separate project and provide a link after it is done.

Interesting, didn't know that had changed. Looks like Microsoft's API docs are out of date. Looking forward to the repro.

Just FYI - plugin-to-plugin sharing is intentionally not supported right now. I haven't found a good model that makes it easy, safe, and reliable for users, but I appreciate your exploration in this space and hope to learn from your sample.

Sorry for the delay - kids, the dog and some other stuff (I'm pretty sure you know 😄 )

Repro project

I uploaded my concept repo and added a minimal repro project called "AlcTest".

Just run the exe and provide a path to an assembly.
This assembly gets loaded into a new AssemblyLoadContext and a PluginLoader is created based on that assembly.
Then the test program prints out the type information for the types found in the ALC and the types from the PluginLoader.
The hash values are different for the "same" type - so they reside in different ALCs. (which is my problem)

Shared plugin concept

The concept stuff (the other projects) won't work at the moment since the behavior I described in this issue is not that behavior I expected. (So no chance to test the further things)

The core idea is to put only the (shared) plugin assemblies into a separate ALC, then sharing this ALC with the "consuming" plugins (plugins with references to other plugins). I'd like to achieve this by providing the created shared ALC as the default fallback ALC for all the PluginLoaders.
My assumption (well it's more like a hope 😁 ) is now that all other dependencies of the plugin should be resolved inside the dedicated PluginLoaders but not the plugin types the consuming plugins will use.

If this works we could do type checks and all that stuff we need to do with plugin types inside the consuming plugins. For example when giving them a new plugin reference to use (consume).

On the other side the shared ALC has (as far as I understand the ALC concept) a default (auto) fallback to the AssemblyLoadContext.Default ALC. So the contract types like the plugin interfaces would be used from that context.

I hope this is not too confusing - I'm working on a sketch to visualize it.

All this I tried to build in the SharedPlugins project - if you like I'd appreciate your opinion and advice.

As I said, I'm not sure if I understand all those things right - so maybe this is just a chimera.

Good and bad news.

The "good" news:

I got it working by defacing your code 🙈

public Assembly LoadAssemblyFromFilePath(string path)
{
#if FEATURE_NATIVE_RESOLVER
    if (_preferDefaultLoadContext)
    {
        var a = Assembly.LoadFile(path);
        var resolvedFromDefaultContext = _defaultLoadContext.Assemblies.FirstOrDefault(assembly => assembly.FullName == a.FullName);

        if (resolvedFromDefaultContext != null)
        {
            return resolvedFromDefaultContext;
        }
    }
#endif
    if (!_loadInMemory)
    {
        return LoadFromAssemblyPath(path);
    }

    using var file = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.Read);
    return LoadFromStream(file);
}

Of course we should discuss this.

The bad news:

My assumption (well it's more like a hope 😁 ) is now that all other dependencies of the plugin should be resolved inside the dedicated PluginLoaders but not the plugin types the consuming plugins will use.

Well, hope dies last, but this hope nerver was really alive.

As soon as a producer has dependencies - the current construct is not able to resolve them (in retrospec this was clear)

My first guess to fix this would be something like having a dictionary where all producers are registered. The consumer then has to propagate the type he can consume (a interface or a concrete type).
Then we stop providing a shared ALC. Instead we provide the aggregation of all ALCs from producers (only those that match the consumers expected producer type) and register this types to the service collection of each consumer plugin.

The really bad news:

Even if this works - your goal

I haven't found a good model that makes it easy, safe, and reliable for users, but I appreciate your exploration in this space and hope to learn from your sample.

will not be achieved.

I made the drafts:

This one is for the not working one I mentioned above:

And this one is for my current concept:

The "SharedPlugin router" in this second approach would be something like the dictionary I mentioned in my last comment. Like Dictionary<string, AssemblyLoadContext> where the string key is the Type.FullName.

EDIT: The magenta arrows are only the reference "flow" - this should be done under the hood inside the "MagicContext" if a plugin assembly is loaded by the "MyConsumerPlugin ManagedContext" (which use config.DefaultContext = magicContext; with config.PreferSharedTypes = true;)

Got it working. I did it with my own custom ALC to make clear what the idea is.

Concept looks now like this:

Concept code draft is here.

Thanks for the detailed comments and pictures! I have some follow-up questions.

I uploaded my concept repo and added a minimal repro project called "AlcTest".

Thanks for the concept! I looked at the code and think I see what's going on. Your sample is trying to put the "default" (aka entrypoint) assembly into the DefaultContext, but this code is going to force a fresh copy to be loaded anyways, ignoring your default context.

DotNetCorePlugins/src/Plugins/PluginLoader.cs

Lines 255 to 259 in 8312d70

    
           public Assembly LoadDefaultAssembly() 
        
           { 
        
               EnsureNotDisposed(); 
        
               return _context.LoadAssemblyFromFilePath(_config.MainAssemblyPath); 
        
           }

DefaultContext was meant to manage shared dependencies. If you want to share a single instance of the 'entrypoint' assembly, Assembly.LoadFile is probably an easier way to manage this.

I got it working by defacing your code 🙈

public Assembly LoadAssemblyFromFilePath(string path)
{
#if FEATURE_NATIVE_RESOLVER
    if (_preferDefaultLoadContext)
    {
        var a = Assembly.LoadFile(path);
        var resolvedFromDefaultContext = _defaultLoadContext.Assemblies.FirstOrDefault(assembly => assembly.FullName == a.FullName);

        if (resolvedFromDefaultContext != null)
        {
            return resolvedFromDefaultContext;
        }
    }
#endif
    if (!_loadInMemory)
    {
        return LoadFromAssemblyPath(path);
    }

    using var file = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.Read);
    return LoadFromStream(file);
}

Of course we should discuss this.

It looks like the main difference between this and my library that this line instead calls _defaultLoadContext.LoadFromAssemblyName. Is that the key here?

DotNetCorePlugins/src/Plugins/Loader/ManagedLoadContext.cs

Lines 100 to 111 in 8312d70

    
           if ((_preferDefaultLoadContext || _defaultAssemblies.Contains(assemblyName.Name)) && !_privateAssemblies.Contains(assemblyName.Name)) 
        
           { 
        
               // If default context is preferred, check first for types in the default context unless the dependency has been declared as private 
        
               try 
        
               { 
        
                   var defaultAssembly = _defaultLoadContext.LoadFromAssemblyName(assemblyName); 
        
                   if (defaultAssembly != null) 
        
                   { 
        
                       // Older versions used to return null here such that returned assembly would be resolved from the default ALC. 
        
                       // However, with the addition of custom default ALCs, the Default ALC may not be the user's chosen ALC when 
        
                       // this context was built. As such, we simply return the Assembly from the user's chosen default load context. 
        
                       return defaultAssembly;

The really bad news:

Even if this works - your goal

I haven't found a good model that makes it easy, safe, and reliable for users, but I appreciate your exploration in this space and hope to learn from your sample.

will not be achieved.

It's somewhat comforting to have someone else dive into the bowels of AssemblyLoadContext and dependency resolution and arrive at a similar conclusion. IMO assembly loading remains one of the more mysterious and perplexing areas of .NET. Ironically, this project grew organically from trying to demonstrate to the CoreCLR team why no sane developer would implement AssemblyLoadContext. They made some improvements in .NET Core 3.0 (like adding AssemblyDependencyResolver), but it's still fundamentally messy. To combat that, I'm intentionally keeping DotNetCorePlugins as simple as possible, even if that means sacrificing some more advanced features.

Your sample is trying to put the "default" (aka entrypoint) assembly into the DefaultContext, but this code is going to force a fresh copy to be loaded anyways, ignoring your default context.

Yes I realized this, too. After figuring out what exactly the idea behind your concept of providing a default context was, I dropped my first concept. In retrospec it is... well lets say "not so very smart" what I did there.

Maybe I did not make that clear: My new idea is this one:

Got it working. I did it with my own custom ALC to make clear what the idea is.

Concept looks now like this:

Concept code draft is here.

This is working at least for my plugin system.
The implementation of that concept is in the "SharedPlugins" project and the referenced projects.

My custom ALC PluginLoadContext lacks all your libraries features and only implements the "SharedPlugins" feature.

What do you think about the concept and its implementation?

Maybe we can find a way to make it possible to use this easy enough? Something like providing a second "entry point" for this library (additionally to PluginLoader) that uses the PluginLoader internally and can profit in that way from all those features that are already available in this library.

I'd appreciate your opinion!

Thanks for the updated diagrams and explanation. It was useful to learn from your example. I'm not planning on making changes to this library at the moment. I think your code is a good example of how to use AssemblyLoadContext if you have scenarios that extend beyond the current capability of this library, but I don't see enough need for this right now to add anything to the library. I'll leave this issue open for additional discussion. If there is more demand for something like this in the future, I'll reconsider, but for now, it doesn't appear to warrant the effort.

Thanks,
Nate

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please comment if you believe this should remain open, otherwise it will be closed in 14 days. Thank you for your contributions to this project.

Closing because there was no response to the previous comment.
If you are looking at this issue in the future and think it should be reopened, please make a comment and mention natemcmaster so he sees it.

	public Assembly LoadDefaultAssembly()
	{
	EnsureNotDisposed();
	return _context.LoadAssemblyFromFilePath(_config.MainAssemblyPath);
	}

	if ((_preferDefaultLoadContext \|\| _defaultAssemblies.Contains(assemblyName.Name)) && !_privateAssemblies.Contains(assemblyName.Name))
	{
	// If default context is preferred, check first for types in the default context unless the dependency has been declared as private
	try
	{
	var defaultAssembly = _defaultLoadContext.LoadFromAssemblyName(assemblyName);
	if (defaultAssembly != null)
	{
	// Older versions used to return null here such that returned assembly would be resolved from the default ALC.
	// However, with the addition of custom default ALCs, the Default ALC may not be the user's chosen ALC when
	// this context was built. As such, we simply return the Assembly from the user's chosen default load context.
	return defaultAssembly;

[Question] Why is not the given default context resolving the type if PreferSharedTypes is set to true

Repro project

Shared plugin concept