Implement a proper testing framework
danigm opened this issue · comments
Project Title: Implement a proper testing framework
Description: Currently the majority of rpmlint tests are handled via running rpmlint for built RPM packages and comparing the results. This solution has disadvantages. It's hard to maintain it, adding of new RPM test packages requires lots of effort, it doesn't scale much further and last but not least RPM packages take space in rpmlint repository.
The goal of this project is to implement a proper testing framework that would use mocking for creating new testcases instead of requiring full binary RPMs as input. The second part is to transform existing RPMs test packages into the new framework.
Deliverable: The result of this project will be multiple pull requests to the rpmlint repository that implement the testing framework.
Mentor: @danigm
Communication channels:
- Matrix direct message: @danigm:gnome.org
- Matrix openSUSE chat channel: mention me (danigm) openSUSE Chat
- Email: daniel.garcia@suse.com
Be patient and wait one or two days before reping, I'll try to answer as soon as possible, but it's not always possible answer in the same day.
Skills: python, RPM, packaging, git, Linux
Skill Level: Medium/Hard
Get started:
- Get familiar with the rpmlint code base and current way of testing: https://github.com/rpm-software-management/rpmlint
- Study python testing and get familiar with options we can utilize
Same project as #129, but with me as mentor.
if this #129 is closed then why this with different mentor
if this #129 is closed then why this with different mentor
There are still a lot of tests using binary packages in rpmlint, the idea is to extend the testing framework to remove the need of binaries for tests, just the machinery to mock those binary packages.
Hello, I have basic knowledge of python, git and linux. Is it possible for me to get involved in this project ?
Hello, I have basic knowledge of python, git and linux. Is it possible for me to get involved in this project ?
I can guide and teach the basic about testing, the current test framework and how to improve, so I think that a basic knowledge of python is good enough if you are ready to learn during the first weeks.
hi @danigm , I'm definitely ready to learn for this project . So, how do I start ?
The first steps could be to get the rpmlint repo, install the required python modules, using your base system or a virtualenv and run the tests with pytest.
You'll need to have rpm installed, so it'll be easier if you have openSUSE Tumbleweed to work on this. It could work installed in your hardware, as a virtual machine or as a docker container.
Once all of that is done, and you're able to run the tests with the latest code from github, we can start to look for a simple issue to solve, to get familiar with the project code.
The first steps could be to get the rpmlint repo, install the required python modules, using your base system or a virtualenv and run the tests with pytest.
Hi @danigm, do I need to clone the repo in my local machine or fork the repo ?
You'll need to have
rpminstalled, so it'll be easier if you have openSUSETumbleweedto work on this. It could work installed in your hardware, as a virtual machine or as a docker container.
I have installed docker and the docker image of tubleweed. I have install all the required python modules too. How my docker image will interact with pytest ? Do I need to install all the requirements inside the docker image and create a container or the pytest will automatically pic all the necessary things ?
Hey @danigm , I would love to work on this issue, is there any way I can connect with you regarding this issue.
Hello @danigm, I'm interested with this project. How to get contact with you? (Email? GitHub?)
Email is an option, but for a more direct communication we can use https://matrix.to/#/#space:opensuse.org, ping me in the openSUSE Chat channel (@danigm) or send me a direct message to @danigm:gnome.org
Be patient and wait one or two days before reping, I'll try to answer as soon as possible, but it's not always possible answer in the same day.
The first steps could be to get the rpmlint repo, install the required python modules, using your base system or a virtualenv and run the tests with pytest.
Hi @danigm, do I need to clone the repo in my local machine or fork the repo ?
You'll need to have
rpminstalled, so it'll be easier if you have openSUSETumbleweedto work on this. It could work installed in your hardware, as a virtual machine or as a docker container.I have installed docker and the docker image of tubleweed. I have install all the required python modules too. How my docker image will interact with pytest ? Do I need to install all the requirements inside the docker image and create a container or the pytest will automatically pic all the necessary things ?
You'll need to install all the python requirements, that can be done in the container directly, installing the python310-MODULE, or with a virtualenv.
Once everything is installed you can try to run the tests with:
python3 -m pytest
Hi, @danigm I am interested in this idea and would like to work on it. As of now, I have created a dock container, installed rpm and got python3 -m pytest working. I am new to tests but am curious to learn. Could you point me to some learning resources.
Hi, @danigm I am interested in this idea and would like to work on it. As of now, I have created a dock container, installed rpm and got python3 -m pytest working. I am new to tests but am curious to learn. Could you point me to some learning resources.
On the python side, you can start to look at the pytest framework, because the project consists in improving the tests for rpmlint mocking the .rpm files.
Take a look to the current rpmlint project tests, there's a lot of tests that uses binary .rpm files for testing and we want to provide a way to be able to create tests similar to that but without the need of a real binary file, using a fixture or a mock that emulates the behavior.
Hi @danigm, can I contact you to contribute too?
Hi @danigm, can I contact you to contribute too?
Yes, sure, the communication channels information is there, so feel free to ask me anything if you need help
Hi @danigm, I was trying to install rpm-python binding, but I couldn't. From official docs of rpm, however the python binding of RPM is unavailable.
http://www.ukuug.org/events/linux2004/programme/paper-PNasrat-1/rpm-python-slides/frames.html.
I am using MacOS (ventura 13.2) with Apple Silicon (M1). And was able to install all the remaining dependencies, libraries and packages including rpm expect for rpm-python binding.
It says that I don't have module 'rpm'. Could you please help me with this. Thanks.
I am using MacOS (ventura 13.2) with Apple Silicon (M1). And was able to install all the remaining dependencies, libraries and packages including rpm expect for rpm-python binding.
I don't think that this could work on MacOS, my recomendation is to use a virtual machine or a docker container with openSUSE Tumbleweed or another rpm based distribution, and it will be easier. The rpm module can be installed using virtualenv but the bindings are not there, from the pypi module doc:
RPM Python bindings are tied to system RPM installation and are not available as a Python package (on PyPI or elsewhere). This shim module makes it possible to import and use the bindings in a virtualenv.
Hello @danigm .
I want to work on this issue.
I have prior experience in testing in python(using pytest).
Can you please guide me for the same?
Hi @danigm.
I've come here from the google summer of code,
and I'm very interested in contributing to this project as a student!
I have prior experience with python as a student in computer science, but nothing beyond assigned work.
I've got a few years of linux experience under my belt and wanted to try my hand at contributing 😄
Any guidance would be appreciated, if I'm not too late to the party 😅
Kind regards.
@RedSunOverParadise there are some comments in this issue with first steps you can follow, for example: