Huge performance hits when having a bigger application with a lot of service definitions

[Michiel Devriese]

We have a big application that has a lot of service definitions registered on the container. In combination with a lot of routes on a Slim 3 application, this causes has a huge impact on our performance (170ms additional latency).

Because of this impact we are still on league/container:^2.x but would very much like to upgrade to the latest version.

The cause is actually very simple, it is located in the DefinitionAggregate on the methods ::has & getDefinition.
Both methods use a foreach on the definitions the aggregate knows about, and with a lot of active definitions combined with a lot of ContainerInterface->get usages this results in a lot of loops being executed.

At first I was confused to why the $definitions array was not using $id as indices but I understand that a foreach is used because a definition can actually be mutated through DefinitionInterface->setAlias, which surprised me, I was expecting immutability.

The discussion of immutability a-side, what would be the recommended approach to address this issue?

Should we implement our own DefinitionAggregateInterface that uses the $id as indices and thus making the definitions their alias immutable?

For clarification, this implementation makes the performance hit go away:

	public function add(string $id, $definition): DefinitionInterface
	{
		if (false === ($definition instanceof DefinitionInterface)) {
			$definition = new Definition($id, $definition);
		}

		$this->definitions[$id] = $definition->setAlias($id);

		return $definition;
	}

	public function has(string $id): bool
	{
		return array_key_exists($id, $this->definitions);
	}

	public function getDefinition(string $id): DefinitionInterface
	{
		if (!array_key_exists($id, $this->definitions)) {
			throw new NotFoundException(sprintf('Alias (%s) is not being handled as a definition.', $id));
		}

		$definition = $this->definitions[$id];

		return $definition->setContainer($this->getContainer());
	}

Thank you for your time.

[Phil Bennett]

Hi, for now, yes, your own implementation would be my suggestion.

However, I'm working on a new feature that will allow you to compile the container ready for production, that will generate a container implementation that just instantly instantiates what you want, based on what you have defined. I've had some health issues recently so development has been slow, but hoping it's only a couple of months out.

[Michiel Devriese]

Hi @philipobenito

Thank you for your swift reply!

I'm already more than happy with your quick answer confirming my proposal.

A precompiled definition set sounds awesome, I'm eager to see what it becomes!

---

I'll provide some insights as well on how we use the container since those are always valuable :)

We generally only use the ->addShared method with a callback as second argument which can instantiate the argument, it is super simple to use.

We also use BootableServiceProviderInterface to avoid having to define everything on the provides method (I realise that this is probably the main reason that we have such a big performance impact), it is that, or running into mistakes because a service was not exposed.

We also wrote a test that can find all definitions and checks if it can actually construct those instances (part of avoiding those other mistakes that can happen 😄)

Container factory test

final class ContainerFactoryTest extends TestCase
{
	private static ?Container $container = null;

	public static function getContainer(): Container
	{
		if (self::$container === null) {
			self::$container = ContainerFactory::load();
		}

		return self::$container;
	}

	/**
	 * @test
	 * @dataProvider dataProvider
	 */
	public function itShouldProvide(string $class): void
	{
		$actual = self::getContainer()->get($class);

		if (class_exists($class)) {
			$this->assertInstanceOf($class, $actual);
		}

		$this->assertNotNull($actual);
	}

	public function dataProvider(): array
	{
		$classes = [];
		foreach ($this->getAllContainerDefinitions() as $definition) {
			$classes[$definition->getAlias()] = [$definition->getAlias()];
		}

		return $classes;
	}

	/**
	 * @return DefinitionInterface[]
	 */
	public function getAllContainerDefinitions(): iterable
	{
		/** @var DefinitionAggregateInterface $providerAggregate */
		$providerAggregate = $this->getPrivateProperty(self::getContainer(), 'definitions');

		return $providerAggregate->getIterator();
	}

	public function getPrivateProperty($class, string $propertyName)
	{
		$property = new ReflectionProperty($class, $propertyName);

		$property->setAccessible(true);

		return $property->getValue($class);
	}
}

---

PS. Feel free to close this issue as you see fit.

I wish you a speedy recovery @philipobenito!

[Phil Bennett]

@mrtus yeah, the bootable service provider is where your big hit is going to come from, but I understand it can be quite annoying if you miss something in provides, the approach with that was really to try and provide a balance, it's less convenient, but allows for lazy loading which hugely improves performance for big applications, as it will only ever define what is needed in the current process.

Hope that gives you a little context on it.

The compiled container is my main priority for the package right now, but I may also look at how I might improve the iteration of definitions, storing against keys is unfortunately not the answer, as it doesn't solve the problem of knowing what's defined directly with the container and what is in a service provider, all about balance again with that, but may be able to squeeze some more performance out of it somewhere.

I'll close this as you mentioned but I'll pop a comment here when I have have something you can play with with the compiler, your app might actually be a great test case if it's big.

[Michiel Devriese]

Yes, I was making the same conclusion about the provides, I understand the use of the provides but it also is a hell in maintenance. Maybe we are simply not conditioned enough as developers to actually use it 🙈.
We stepped away from it because we caused a couple of incidents with our app a couple of years back, because of missing definitions, hence also the additional container factory test.

Some more details then;
At boot we have about ~600 dependencies configured.

Additional service can be added dynamically as well (which won't be solvable through a compiler) and will always be created through reflection and auto-wiring.

Taking a look at ReflectionContainer the cacheResolution could be a solution, also with tags. Although maybe that is a bandaid on a different problem 😁.

[Phil Bennett]

Yeah you'd sort of just be moving the problem with that, reflection performance may well be just as bad with that many definitions.

It is kind of a no win choice, convenience vs performance :-(

[Phil Bennett]

Saying that though, you'd only actually be reflecting on what's needed in the process, so you may be better off, you'd lose any and all customisation options that way though