Metadata ordering issues
toptensoftware opened this issue · comments
Some of the metadata tables have requirements on sort order. ILPack doesn't currently ensure these rules aren't violated. The main two that seem to be causing issues is the order of the generic parameters table and the interface map.
These are difficult to reproduce in the unit tests because the ordering and interleaving of things can be affected by other things in the project. Here's how to reproduce both issues in Sandbox.
Add the following to SandboxSubject\MyClass.cs, just below MyClass:
public class GPOrder1
{
public void Function1<T>()
{
}
public void Function2<T>()
{
}
}
public class GPOrder2<T>
{
}
Result:
Metadata table GenericParam not sorted.
at System.Reflection.Throw.InvalidOperation_TableNotSorted(TableIndex tableIndex)
at System.Reflection.Metadata.Ecma335.MetadataBuilder.ValidateGenericParamTable()
at System.Reflection.Metadata.Ecma335.MetadataBuilder.ValidateOrder()
at System.Reflection.Metadata.Ecma335.MetadataRootBuilder.Serialize(BlobBuilder builder, Int32 methodBodyStreamRva, Int32 mappedFieldDataStreamRva)
For the interface map problem, remove the above code and add this instead:
interface Itf1
{
}
interface Itf2
{
}
class MyImpl : Itf2, Itf1
{
}
Result:
Metadata table InterfaceImpl not sorted.
at System.Reflection.Throw.InvalidOperation_TableNotSorted(TableIndex tableIndex)
at System.Reflection.Metadata.Ecma335.MetadataBuilder.ValidateInterfaceImplTable()
at System.Reflection.Metadata.Ecma335.MetadataBuilder.ValidateOrder()
at System.Reflection.Metadata.Ecma335.MetadataRootBuilder.Serialize(BlobBuilder builder, Int32 methodBodyStreamRva, Int32 mappedFieldDataStreamRva)
Cecil uses sorted table (with merge sort) to avoid such issues as following:
As a second approach, I confirm that generic parameter issue was resolved by separating generic parameter metadata emitting from type metadata emitting. Something like that:
// ...
foreach (var type in types) {
ReserveType(type);
}
foreach (var type in types) {
CreateGenericParametersForType(type);
}
foreach (var type in types) {
CreateType(type);
}Interface sorting can be done with a simple sort over the result of GetInterfaces() method.
What do you think?
Not convinced the approach you suggested for generic parameters will work. They need to be generated in CodedIndex order, which for generic parameters puts the type/method flag in the lowest bit. ie: methods and types will be interleaved. Your suggestion may have worked for my example, but it depends on the number of methods, the number of types and the order they're defined in. Pretty sure a sort is the only way to fix it.
For interfaces, yep, sorting GetInterfaces should work.
I'm running into this same issue with InterfaceImpl. In my case, I have something like this:
interface IFoo {}
class Base : IFoo {}
interface IBar {}
class Child : Base, IBar {}Child is the type I'm trying to dynamically define (IFoo, Base and IBar are all in external assemblies), and when I call GenerateAssembly it throws saying the InterfaceImpl table is not sorted.
Any idea when this might be fixed?
I'm facing this too. I built ILPack locally to target .NET Standard 2.1 and things seem to work, but sorting issue remain, also on InterfaceImpl table.
An unhandled exception of type 'System.InvalidOperationException' occurred in System.Private.CoreLib.dll: 'Metadata table InterfaceImpl not sorted.'
at System.Reflection.Throw.InvalidOperation_TableNotSorted(TableIndex tableIndex)
at System.Reflection.Metadata.Ecma335.MetadataBuilder.ValidateInterfaceImplTable()
at System.Reflection.Metadata.Ecma335.MetadataBuilder.ValidateOrder()
at System.Reflection.Metadata.Ecma335.MetadataRootBuilder.Serialize(BlobBuilder builder, Int32 methodBodyStreamRva, Int32 mappedFieldDataStreamRva)
at System.Reflection.PortableExecutable.ManagedPEBuilder.SerializeTextSection(SectionLocation location)
at System.Reflection.PortableExecutable.ManagedPEBuilder.SerializeSection(String name, SectionLocation location)
at System.Reflection.PortableExecutable.PEBuilder.SerializeSections()
at System.Reflection.PortableExecutable.PEBuilder.Serialize(BlobBuilder builder)
at Lokad.ILPack.AssemblyGenerator.GenerateAssemblyBytes(Assembly assembly) in C:\Users\Per\Git\Lokad\ILPack\src\AssemblyGenerator.cs:line 118
at Lokad.ILPack.AssemblyGenerator.GenerateAssembly(Assembly assembly, String path) in C:\Users\Per\Git\Lokad\ILPack\src\AssemblyGenerator.cs:line 125
For interfaces, yep, sorting GetInterfaces should work.
Any pointer to required order?
I was able to get around this by making the following change in AssemblyGenerator.Types.cs (around line 108):
// Add implemented interfaces (not for enums though - eg: IComparable etc...)
if (!type.IsEnum)
{
foreach (var itf in type.GetInterfaces().OrderBy(t => CodedIndex.TypeDefOrRefOrSpec(_metadata.GetTypeHandle(t))))
{
_metadata.Builder.AddInterfaceImplementation(typeHandle, _metadata.GetTypeHandle(itf));
}
}The added bit is the .OrderBy(t => CodedIndex.TypeDefOrRefOrSpec(_metadata.GetTypeHandle(t))) call, which sorts the interfaces in the order expected.
@SvenGroot, Thank you! I had the same issue and your change fixed it!
I was able to get around this by making the following change in AssemblyGenerator.Types.cs
Cool @SvenGroot, worked for me too. I tweaked and built a local version, even targeting standard 2.1, and that works.
Can't judge the production quality of that o/c, but for our needs right now, it's about enough.
PR maybe? 😎
Anyway, thanks a lot. 👍
@SvenGroot Thanks a lot for spotting a solution! have extended the unit tests as well, and indeed, we do increase the coverage with your fix.
@vermorel I've tested this and can confirm that the original problem is already solved so we can close this issue.