How to reflect all subclasses of one common abstract base class?

[MLorenz42]

Hi,

I am completely new to this framework and I have some questions.

I have

1. An abstract base class with a few getters:

abstract class MyBaseClass {
    String get name;
    List<MyValueType> get values;
}

2. Several classes that implement MyBaseClass:

class A implements MyBaseClass {
   @override
   String name = 'AClass';
   
   @override
   List<MyValueType> = [MyValueType.X, MyValueType.Y]; 
}

class B implements MyBaseClass {
   @override
   String name = 'BClass';
   
   @override
   List<MyValueType> = []; 
}

My goal is to fetch all classes that implement MyBaseClass and read their properties.

So, I created:

class Reflector extends Reflectable {
  const Reflector()
      : super(invokingCapability); 
}

const reflector = const Reflector();

1. How do I fetch a list of classes? I only found the InstanceMirror.reflect() which only delivers one result, not many.
2. It is not clear, how the annotation must be set. When trying to fetch all MyBaseClass implementations, do I need to annotate only my abstract MyBaseClass or do I need to annotate classes A and B or do I need to annotate all three classes?
3. Which capabilities do I need? In my test case I got this exception: NoSuchCapabilityError: no capability to invoke the getter "name" but was not able to solve this.

Thanks in advance, any help is appreciated!

[Dimitrios Begnis]

Hey @S-Man42,
in order to be able to reflect instance members, you need the capability instanceInvokeCapability. The invokingCapability you have used gives support for reflecting top level and static members.

For reflection of subtypes/subclasses you need the subtypeQuantifyCapability capability. If your reflector has this capability it is sufficent to annotate you base class and all subclasses will be reflectable. You can view an example usage of this capability in this test.

[Erik Ernst]

Thanks, @Dimibe! invokingCapability actually covers all of InstanceInvokeCapability, StaticInvokeCapability, and NewInstanceCapability. I don't know the context well enough to know why there is a no-such-capability error, so we'll need more info in order to deal with that (if it occurs again after other changes have been made).

@S-Man42, you may wish to take a look at my answer on https://stackoverflow.com/questions/72633562/dart-reflectable-how-to-get-all-subclasses-from-an-abstract-base-class/72642414#72642414 as well.

I'll close this issue because it seems that there is nothing further to do. @S-Man42, please create a new issue if needed.

[MLorenz42]

Hi, thanks for answers. Since I believe here's the better place to discuss than on SO, I'll try to answer here, if it's ok:

1. Since MyBaseClass is orignally abstract, reflector.reflect(MyBaseClass()) does not work (cannot instanciate an abstract class). So, is there a way, to reflect subtypes from an abstract class?

Meanwhile I changed MyBaseClass to non-abstract this way and added the @reflector annotation:

@reflector
class MyBaseClass {
  String get name {return null;}
  List<MyValueType> get values {return null;}
}

Additionally I added this annotation to classes A and B as well.

The Reflector annotation is defined as following (incl. the subtypeQuantifyCapability you mentioned on your great SO answer):

class Reflector extends Reflectable {
  const Reflector()
      : super(
      invokingCapability,
      subtypeQuantifyCapability
  ); // Request the capability to invoke methods.
}

const reflector = const Reflector();

2. It's not clear to me which classes must be annotated to be found by the reflection. Do I need to annotate all three classes (as your SO example shows)?

Afterwards I executed:

flutter packages pub run build_runner build DIR

using the ./build.yaml file:

targets:
  $default:
    builders:
      reflectable:
        generate_for:
          - lib/main.dart

3. When this build was done I called:

    reflector.reflect(MyBaseClass());
    reflector.annotatedClasses.forEach((ClassMirror element) {
        print(element.invokeGetter('name'));
    });
   

which still results in this exception:

The following ReflectableNoSuchMethodError was thrown building MainView(dirty, dependencies: [_LocalizationsScope-[GlobalKey#7ed56]], state: _MainViewState#efaf8):
NoSuchCapabilityError: no capability to invoke the getter "name"
Receiver: MyBaseClass
Arguments: []
Named arguments: {}

When the exception was thrown, this was the stack: 
#0      reflectableNoSuchGetterError (package:reflectable/capability.dart:593:3)
#1      ClassMirrorBase.invokeGetter (package:reflectable/src/reflectable_builder_based.dart:696:13)
#2      initializeRegistry.<anonymous closure> (package:gc_wizard/widgets/registry.dart:3717:19)
#3      List.forEach (dart:core-patch/array.dart:309:8)
#4      initializeRegistry (package:gc_wizard/widgets/registry.dart:3715:30)
#5      _MainViewState.build (package:gc_wizard/widgets/main_view.dart:337:34)
#6      StatefulElement.build (package:flutter/src/widgets/framework.dart:4870:27)
#7      ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4754:15)
#8      StatefulElement.performRebuild (package:flutter/src/widgets/framework.dart:4928:11)
#9      Element.rebuild (package:flutter/src/widgets/framework.dart:4477:5)
#10     ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:4735:5)
#11     StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:4919:11)
#12     ComponentElement.mount (package:flutter/src/widgets/framework.dart:4729:5)
...     Normal element mounting (171 frames)

[Erik Ernst]

I'll try to answer here, if it's ok:

Sure!

Since MyBaseClass is orignally abstract, reflector.reflect(MyBaseClass()) does not work

You could use reflector.reflectType(MyBaseClass) to get a class mirror for MyBaseClass, so there's no need to change MyBaseClass to be concrete in order to obtain a class mirror.

However, that won't help you immediately, because the class mirror doesn't have any members that will allow you to traverse the entire subtype graph (that is, classes that implement or extend MyBaseClass, directly or indirectly).

So you would use reflector.annotatedClasses to get access to all classes supported by reflector. If you have quantified this to be exactly all subtypes of a specific class like MyBaseClass then you're done. This is the situation as far as I can see, but in a more complex realistic setting it might not be true. For instance, you could use subtypeQuantifyCapability and put @reflector on several classes, rather than just on MyBaseClass.

If this is true (that is, if reflector.annotatedClasses contains additional classes, not just the subtypes of MyBaseClass) then you'd need to iterate over reflector.annotatedClasses and include just the ones that are isSubtypeOf(MyBaseClassMirror) where MyBaseClassMirror is obtained from reflector.reflectType(MyBaseClass).

Do I need to annotate all three classes (as your SO example shows)

(...I mentioned subtype_quantify_test.dart which is all about subtypeQuantifyCapability, and it annotates just the type of the subtype graph, here that would be: just MyBaseClass. The other example annotates several classes, but that example is about annotatedClasses, which can of course also be used without subtype quantification ...)

No, the obvious thing to do is to have @reflector as metadata on the declaration of MyBaseClass. You would not have it on any subtypes (like the classes A and B in the original example). It wouldn't hurt, but it seems to defeat the whole purpose of finding the subtypes programmatically.

reflector.reflect(MyBaseClass());

Executing this as a statement (that is, ignoring the returned class mirror) is a no-op, so there's no reason to do that.

still results in this exception ...

You're invoking a getter on a class mirror, and that will only work if there is a static getter with the name name. So reflectable can't find such a getter, and it causes the exception. (It can't see whether it is missing because we did not ask for support for invoking that particular static getter, or if it's because there is no such static getter in the first place, so you get the 'no capability' error in both cases, even though the message is a bit misleading when the getter just isn't there.)

If you want to invoke an instance getter then you need to do it on an instance (with reflection: on an instance mirror),
something like (reflector.reflect(A()) as InstanceMirror).invokeGetter('name').

But if you actually just want to get the name of the class you might use myClassMirror.simpleName.

[MLorenz42]

Thanks for your awesome reply and your patience.

As you said, I used the abstract MyBaseClass again and removed the @reflector annotation from A and B.

As you can see in my comments above, I already used the reflector.annotedClasses method you also mentioned. Well now I struggle with this:

  reflector.reflectType(MyBaseClass);
  reflector.annotatedClasses.forEach((ClassMirror element) {
      reflector.reflect(element as InstanceMirror).invokeGetter('name');
  });

With reflectType I try to get all subclasses from MyBaseClass. Then I am using the annotedClasses method to iterate all found classes. To fetch the value of the field name I try to case the iteration's element into an InstanceMirror, but this seems not work (and seems clear to me, because I believe one cannot cast ClassMirror into InstanceMirror, right?). Resulting exception:

type 'NonGenericClassMirrorImpl' is not a subtype of type 'InstanceMirror' in type cast

In your reply you wrote:

(reflector.reflect(A()) as InstanceMirror).invokeGetter('name')

But this uses the concrete class' name directly, which is exactly what I am trying to avoid by using reflection. So, is there no way to achieve my goal or do I still miss something? Could you suggest an alternative way?

---

Maybe we can solve the problems, when you actually know completely, what's the problem I am trying to solve:

1. I need to iterate several classes (like A and B) by a certain criterion (e.g. implementing the same abstract class like MyBaseClass) without actually knowing their class names or how many exist
2. I need to read the fields of these classes (e.g. name and values as shown above) during a startup routine.
3. When a new class of that kind will be created, it should be ensured, that these field getters will be implemented, to avoid crashes during step 2 ;) (That's why I decided that they need to implement the abstract base class and their abstract getters)

At the moment I have a huge "registry" class, which knows every single class of that kind. Every time I create a new one, I need to add it to the registry class. It's like this:

var myRegisteredClasses = <MyRegisterItem>[];

myRegisteredClasses.add(MyRegisterItem({'name': A().name, 'values': 'A().values'});
myRegisteredClasses.add(MyRegisterItem({'name': B().name, 'values': 'B().values'});
myRegisteredClasses.add(MyRegisterItem({'name': C().name, 'values': 'C().values'});
...

For each new class, there are a few lines to add and import the concrete classes. That makes a few hundred classes resulting in a VERY huge registry class. So I want to make this class a bit more intelligent by finding the relevant classes on its own using reflection mechanisms. You can see it here (of course, in reality it's bit more complex than the examples I gave here):

https://github.com/S-Man42/GCWizard/blob/master/lib/widgets/registry.dart

My idea is to move the parameters in the GCWTool constructors into separate classes (e.g. A and B) implementing MyBaseClass. So the registry can read the fields from MyBaseClass of all found subclasses without knowing them beforehand.

Is my approach wrong? Is there a better or more correct way to achieve this?

Again, thanks for your great patience!

[Erik Ernst]

Thanks for the kind words!

The execution of reflector.reflectType(MyBaseClass) as a statement doesn't do anything. That's just like doing 2 + 2 as a statement: You're computing a result and then discarding it. Nothing useful happened, but it did take some CPU cycles. But you might want to do var classMirror = reflector.reflectType(MyBaseClass) as ClassMirror;, which would initialize the variable classMirror to refer to a class mirror that reflects on MyBaseClass. You could then do someOtherClassMirror.isSubtypeOf(classMirror) to determine whether any given class mirror someOtherClassMirror reflects on a subtype of MyBaseClass.

Next, you can't actually do reflector.reflect(element as InstanceMirror).invokeGetter('name'): That's an attempt to obtain a mirror of a mirror, and then invoking a getter named 'name' on the mirror (and, just guessing, I don't think you have enabled reflection on mirrors, and they don't have a getter named name anyway). So (element as InstanceMirror).invokeGetter('name') would make more sense. But you can't do that, either, because element isn't an InstanceMirror (and a type cast will never change an object, it will just ask whether the object is already of the requested type, and it will throw an exception if it isn't).

In general, you can't call an instance method of an object unless you have that object. In this case you have a class (MyBaseClass), and you have a mirror of it (classMirror), so you can do class-things (like calling static methods, or create new instances), but you can't do instance-things (like calling an instance getter like name).

But if you just want to know the name of the class then you can call ClassMirrors method simpleName. For instance:

for (var element in reflector.annotatedClasses) {
  print(element.simpleName);
}

But this uses the concrete class' name directly, which is exactly what I am trying to avoid by using reflection.

For the example, (reflector.reflect(x) as InstanceMirror).invokeGetter('name'), x just needs to be an instance of the given class (I wrote A() to create the instance right there, but you could have obtained that object from anywhere.

So in order to use this approach you'd need to get hold of an instance of each of the classes that you want to iterate over.

But why would you want to do that? Isn't it sufficient to do class mirror things, e.g., simpleName, or to call a static method that each class defines? I don't understand why you'd want to insist on doing something that you can only do on an instance if all you want to do is to learn something about the set of classes...

I need to read the fields of these classes

That will never work: Classes don't have instance variables, only instances of the classes have them. Classes have some other variables, namely the ones marked static, and they can be evaluated on a class (and they can be evaluated via reflection on a class mirror).

So if you want to read some instance variables then you'll have to create some instances of those classes. You can use newInstance to do that, on each of the class mirrors. In order to do that (without hitting a run time exception) you'll need to make sure the given classes actually have a constructor that accepts the given list of actual arguments, but if you make sure that all the classes have a constructor that have the same list of formal parameters (e.g., ()) then it should work.

It's like this:

It looks like you could actually do something like this (I haven't tried it, so surely there are mistakes, but it shows the idea):

for (var classMirror in reflector.annotatedClasses) {
  if (classMirror.isAbstract) continue;
  var instance = classMirror.newInstance('', [], {}) as MyBaseClass;
  myRegisteredClasses.add(MyRegisterItem({'name': instance.name, 'values': instance.values});
}

[MLorenz42]

That's it! Awesome!

It was the newInstance() method which finally made it!

Thank you so much, sir!

[MLorenz42]

Hm... too fast enjoyed, unfortunately...

I believe, the calling code is now running and correct. It worked on my first tests properly. But it only worked as long as I didn't touch anything else :'( Sorry, I need your help again, hope I do not annoy you. So sorry about it...

I created more subclasses and renamed reflector to gcw_tool_reflector and then I re-executed the class generation with:

flutter packages pub run build_runner build DIR

As written above, I have a build.yaml file in the project root with this content:

targets:
  $default:
    builders:
      reflectable:
        generate_for:
          - lib/main.dart

However, when executing this, following log occures:

[INFO] 5.4s elapsed, 0/16 actions completed.
[INFO] 6.5s elapsed, 0/16 actions completed.
[INFO] 18.9s elapsed, 0/16 actions completed.
[WARNING] No actions completed for 18.9s, waiting on:
  - built_value_generator:built_value on lib/configuration/abstract_tool_registration.dart
  .. and 11 more
  
// [many lines more]
 
[INFO] Running build completed, took 1m 26s

[INFO] Caching finalized dependency graph...
[INFO] Caching finalized dependency graph completed, took 477ms

[INFO] Succeeded after 1m 27s with 1 outputs (614 actions)

( lib/configuration/abstract_tool_registration.dart is the abstract class)

Afterwards the annotatedClasses only has one element (the abstract class, I guess)

1. Is this command correct? If so, do you have an idea, what went wrong?
2. Could you please tell me WHEN do I have to execute the command? Every time I create a subclass?

[MLorenz42]

Hm, I tried much now. Even with manually removing of the main.reflectable.dart and following

flutter clean
flutter pub get

and again executing

flutter packages pub run build_runner build lib
(I changed the previously used DIR into lib, but the reportet WARNING still occurs)

I still get only the abstract class in my annotatedClasses list. Even if I annoted all subclasses as well.

It's still not clean to me, why it worked at the first tries, and what did I wrong afterwards on rebuilding... I am not getting the subclasses into the list anymore... What am I doing wrong now? :'(

May I get your help once more, please? I believe I am missing something really obvious...

---

./build.yaml: https://github.com/S-Man42/GCWizard/blob/reflection/build.yaml
./lib/main.dart: https://github.com/S-Man42/GCWizard/blob/reflection/lib/main.dart
Reflector: https://github.com/S-Man42/GCWizard/blob/reflection/lib/configuration/reflectors/gcw_tool_reflector.dart
Abstract Class: https://github.com/S-Man42/GCWizard/blob/reflection/lib/configuration/abstract_tool_registration.dart
Subclass 1: https://github.com/S-Man42/GCWizard/blob/reflection/lib/tools/crypto_and_encodings/abaddon/config/abaddon_registration.dart
Subclass 2: https://github.com/S-Man42/GCWizard/blob/reflection/lib/tools/crypto_and_encodings/enigma/configuration/enigma_registration.dart

Reflection Initialization: https://github.com/S-Man42/GCWizard/blob/cf600976489b47335660ac28a5ed83b2b24ab1de/lib/widgets/main_view.dart#L335
Reflection Call: https://github.com/S-Man42/GCWizard/blob/cf600976489b47335660ac28a5ed83b2b24ab1de/lib/widgets/registry.dart#L3692

[Erik Ernst]

Hm... too fast enjoyed, unfortunately...

Eager enjoyment would be a very useful element of good mental health! ;-)

Anyway, I think the problem could be associated with the sequencing of the code generation steps: If there is some kind of clean-up that removes all the generated code, and then reflectable gets to generate code for the hand-written code, and then built_value code generation takes place, then reflectable doesn't get access to the code generated by the built_value code generator, and that would cause the classes generated by built_value to be omitted from reflection support.

I haven't used this particular combination so I haven't seen the issue, but you might be able to learn something about how to control the ordering of built_value code generation relative to other code generators from the built_value documentation/community. The general build_runner approach is briefly mentioned in https://pub.dev/packages/build_config#adjusting-builder-ordering.

[Erik Ernst]

Another possibility: Your lib/main.dart is used as the entry point library with the reflectable code generator, but that library does not import (directly or indirectly) certain libraries like GCWizard/lib/tools/crypto_and_encodings/abaddon/config/abaddon_registration.dart. If that is true then a class like AbaddonRegistration is not known to the reflectable code generator, so there's no way it could generate code to handle it.

[MLorenz42]

Thanks again, sir!

Yes, this is true. There's absolutely no class which directly imports this class. That's what I want to achieve. Currently I have a huge registry which imports all classes. I thought, with the reflection it is possible to avoid such mass imports.

Do I really need a point, where my registration classes must be explicitly known? I hoped, adding the annotation is enough. I hoped the reflection magic would find all annotated classes nonetheless.

If not, then the reflection plugin seems not to solve my original problem, unfortunately, right? I originally wanted to remove all imports in the registry class... Is it true, that I really must import the classes somewhere? It cannot be found, if they are only annotated?

[Erik Ernst]

The classes that you wish to cover don't have to be mentioned at any location, but the classes need to be included in the input which is passed to the reflectable code generator. There is no way a code generator could scan the whole world to find all libraries that might declare a subtype of any given type, and the set of libraries available to code generation is exactly the ones which are reachable (via a direct or indirect import path) from the given entry point.

[MLorenz42]

Do I understand you correctly? My registry class needs a list of

import '../../abaddon_registration.dart';
import '../../enigma_registration.dart';
...

But then the Android Studio code AI will mark them as unused imports, correct?

Or what did you mean with "included in input"?

[MLorenz42]

Yes, this works! :)

Thank you so much for your patience. Now I have to think about whether it makes life easier or not, since I still need some sort of registry. Maybe I can write some kind of pre-compiler or similar, which searches for the annotated classes and writes a list of class names which then can be included automatically.

However, you helped me a lot. Thanks for all your effort! Hope, I will not face any problems here anymore!
Thank you, sir!

[MLorenz42]

I managed to solve the problem of manually register the classes by creating a bash script, which needs to be executed before running your build command. It searches for all classes in a directory that implement my base class and create a "registry" class with lots of imports automatically :) So no need for manual importing anymore and the IDE cannot warn for unused imports anymore because it's a separate file ;)

[Erik Ernst]

Thank you for the kind words again!

It searches for all classes in a directory that implement my base class and create
a "registry" class with lots of imports automatically

That was exactly what I was half-done writing in response to your previous comment.

However, that idea is somewhat dangerous because you'll obtain a bash_generated.reflectable.dart library which will work with that bash-generated file bash_generated.dart, and bash_generated.reflectable.dart will contain references to declarations in every single one of those imported libraries that contains one or more declarations of subtypes of the AbstractToolRegistration.

So if you import bash_generated.reflectable.dart in bin/main.dart (or in any other Dart program), it will import all of those libraries. If that's a resource problem (that is, if the program gets too large or too slow because it contains a bunch of otherwise unused classes) then you'd need to use some other approach. In any case, it doesn't make sense to have reflection support for a class that you don't intend to use at all in any given program.

[MLorenz42]

I thanks for your warnings. However, I guess, I it will be no problem here because there are no classes implementing my base class which will be unused. All classes that will implement it, must be reflected. The base class will not be assigned to any other class, so the generated class will exactly include what I need, no more, no less :)