apache-airflow resolution

[Tim de Jager]

When running

cargo r apache-airflow

we get the following error message.

thread 'main' panicked at 'internal error: entered unreachable code: when we get here it means that all candidates have been assigned false. This should not be able to happen at this point because during propagation the solvable should have been assigned false as well.', /Users/tdejager/.cargo/registry/src/index.crates.io-6f17d22bba15001f/resolvo-0.2.0/src/solver/mod.rs:584:53
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Some impossible statement is being hit, is being hit inside of resolvo.

[Tim de Jager]

@baszalmstra can you update with any progress made in resolvo :)?

[Bas Zalmstra]

I spent half a day debugging the issue. It looks like at some point something is not properly propagated but I cant figure out what causes it. I'll have to get back to it later.

[Wolf Vollprecht]

I think we should:

• write something in resolvo so that we can store all the rules, and recreate the state
• then debug this

I also looked into it and could not yet figure out the root cause.

[Sumanth]

Not sure if this helps,

cargo run openmetadata-managed-apis

which has dependency of apache-airflow>=2.2 works and installs apache-airflow version 2.6.3 which is valid since all the newer versions need opentelemetry-api and opentelemetry-exporter-otlp which cannot be installed as one of their dependency (opentelemetry-semantic-conventions) has only pre-release versions available

From the logs on the cargo run apache-airflow, I see rip

• Trying to install the latest 2.8.0 version of airflow
• Couldn't because of opentelemetry-api and opentelemetry-exporter-otlp deps
• Backtracks to prev version 2.7.3 which exhausts all the stable variants of both opentelemetry deps and thereby panicking.

This should ideally backtracks to next version of 2.7.2 and so on.

Wish I could dig more on this, still at early days of Rust and understanding SAT solvers.

[Sumanth]

I added a minimal integration test on resolvo which replicates this,

https://github.com/sumanth-manchala/resolvo/blob/airflow-int-test/tests/solver.rs#L462

@baszalmstra

[Wolf Vollprecht]

Hi @sumanth-manchala this is amazing. I also tried to make a reproducer the other day, but didn't manage so I am really happy to see that you managed to do it!

Did you – perhaps – also work on a fix? This issue is certainly one that we would like to fix sooner rather than later :)

[Wolf Vollprecht]

PS: do you want to make a PR with your changes? We could mark it as "expected fail" for now, too. Would love to give you credit for the test!

[Sumanth]

Thanks @wolfv. Have added the integration test to expect panic, will ping here if I could figure out the fix

[Tim de Jager]

Yeah that's absolutely great work, @sumanth-manchala :) Jumping head first into such a gnarly problem couldn't have been easy! Thanks a lot 😄

[Adolfo Ochagavía]

Just in case anyone out there is trying to debug this further, I'm pretty sure I've found the root cause of the issue and hope to have a fix soon.