LLVM compatibility policy
sebastianpoeplau opened this issue · comments
I would like to propose that we agree on some policy for LLVM compatibility. The README currently says that we support any version between 8 and 15, and our CI tests 10 through 15. The code has been accumulating quite a few #ifdef statements, and keeping compatible with all those versions is getting more and more tedious. For example, we have to deal with the old and the new pass manager, but also with LLVM versions that don't have the new one yet, and Clang versions that rely on only the old or only the new pass manager to set up compiler passes.
Given the fast pace of LLVM development (one major release every six months), I think it's reasonable to drop compatibility with old LLVM versions after some time. However, I think we should be careful to make sure that we're always compatible with a version of LLVM that the current Ubuntu LTS release provides via packages (because it seems to be a common platform for developers to use), and we'll probably want to be compatible with the previous Ubuntu LTS for some time. Luckily, new LLVM versions are added to current Ubuntu releases shortly after they come out (e.g., LLVM 15 has recently become available in Ubuntu 22.04); they seem to stop adding new LLVM packages a few months before the next LTS version is released (e.g., the most recent LLVM in Ubuntu 20.04 is version 12, released in spring 2021).
I propose that we support the three most recent LLVM releases. If we'd done this in the past, we would have dropped support for LLVM 12, and hence Ubuntu 20.04 LTS, with the release of LLVM 15 in fall 2022. (Note that people can always build later LLVM versions on old operating systems or get prebuilt packages from third parties if they want; this is just about achieving a reasonable level of convenience for those who don't.)
@aurelf what do you think? Also, everyone, please feel free to comment on whether or not this scheme would work for you!
Hi Sebastian,
you are right it's not needed, and painful, to keep compatibility with all LLVM versions.
I like the idea to keep compatibility with the last 2 Ubuntu LTS versions, but keeping last 3 LLVM versions does not guaranty this... Also some people may run Debian stable which is at version 11 (although LLVM provides some package, I don't know if they introduce some issues).
How about we keep 3 LLVM versions but when we drop the support for an Ubuntu LTS+debian stable we make a branch for this version ?
This will potentially make life easier to anyone who needs a specific compatibility (like if SymCC is combined with another tool that doesn't support up to date LLVM). We can put a warning that the branch is for convenience, without any guaranty/support, and only accept some pull requests proposing backports from main branch that pass CI tests (with specific distribution in the CI).
Or will this be too much overhead?
I like the idea of marking the last commit that is compatible with a given LLVM version. But I think you might be right that a dedicated branch with backports will cause a lot of overhead. Shall we just put a label on the commit? It shouldn't be a huge problem for people to build Clang from source if they need or want to stay on an old operating system for some reason.
I like the idea of marking the last commit that is compatible with a given LLVM version. But I think you might be right that a dedicated branch with backports will cause a lot of overhead. Shall we just put a label on the commit? It shouldn't be a huge problem for people to build Clang from source if they need or want to stay on an old operating system for some reason.
OK for me !
Aurélien
I strongly agree that trying to maintain support for ever-changing LLVM APIs is a losing battle.
I don't have much to add, but is maintaining fuzzbench support important to symcc? I say this because it is by far the most commonly used fuzzing benchmark, so ensuring compatibility seems important (eg for building and expanding on symcc, and evaluating their fuzzers against symcc). I note that currently fuzzbench uses a symcc fork. I have no idea how much work this would involve, but potentially something to think about.
That's a good point. I'm not sure how much the fork has diverged, but I'll have a look to see which base image is used there.
If I understand the FuzzBench configuration correctly, they use a fork by AdaLogics. The fork doesn't seem to follow our changes, so I'm not sure whether we would actually break anything by dropping support for old LLVM versions. However, it still seems a good idea to stay compatible with the FuzzBench base image so that people can use our version of SymCC in FuzzBench if they want. The base image uses Ubuntu 20.04, so we may have to keep LLVM 12 in the list of supported versions until FuzzBench moves to Ubuntu 22.04.