pkgcore.restrictions needs refactoring/cleanup
ferringb opened this issue · comments
The restriction subsystem is basically built around the notion that all restrictions can be represented in CNF/DNF- while a nice notion, this breaks down in the face of conditionals a fair bit. For transitive use dependencies in particular, this becomes a problem- to render the restriction in cnf or dnf, each branch must be converted to a non-transitive conditional restriction.
The impact of this is that if an atom has multiple transitive use deps, the current implementation of pkgcore.ebuild.atom.transitive_use_atom._recurse_transitive_use_conds ( https://github.com/pkgcore/pkgcore/blob/master/pkgcore/ebuild/atom.py#L706 ) basically goes exponential. This has been the underlying cause as to why pkgcheck was exhausting all memory when scanning certain packages.
The design needs reworking. Realistically, conditionals likely need to be broken out as their own set of restrictions- much as how packages is a separate type from values. Doing this sort of separation would allow the repository.itermatch logic to still rely on cnf/dnf, but explicitly fail if given a conditional restriction to query for (said query would make no sense).
The hope there is that doing this will simplify all pathways involved, and allow for a custom API/protocol to be developed for conditional restriction types that is more able to introspected by the resolver/pkgcheck w/out having to force cnf/dnf exponential expansions for transitive use atoms.
Just updating with recent examples- llvm, samba. Should be confirmed, but are likely to trigger it.
Marking as "can't repro" at this point. My strong suspicion is either @arthurzam or @radhermit fixed the combinatorial explosion inherent in the transitive_use_atom parsing which was the cause of this.
I can't induce that anymore; as an additional datum, I noted I couldn't trigger it ~2 years back, but other issues precluded me from being clearer that the issue seemed resolved.
Either way, the cases mentioned (and my own search to verify) aren't triggering this, so I'm closing this as no longer reproducible. Reopen if needed