Unittest framework
marekmosna opened this issue · comments
What do you think to involve any unit testing framework like catch2, gtest, ...
It's really helpful and it doesn't provide such a printf battle and even all CIs support such a approach. For instance GitHub Actions can do a great job for you
Hi @marekmosna ,
What do you mean by "printf battle"?
Print to the output inside the test shouldn't be. Unit test frameworks provide you standardized output based on testing values. You can after decide if you want to abort the test or continue and just fire up the message, it depends.
@marekmosna I'm in favor of having a nice unit test framework, as well as tests which exercise and stress test specific parts of the code! I can say that overall, since this is a single header project, I'd like whatever framework that is used to be as light weight as possible.
Honestly, in the past, I've often used a series of target executables which runs tests and assert on failure that are just driven with cmake's built in ability to run tests with a make tests. But I'm open to sometime more "official" too so long as it's not too heavy :-)
Catch2 is for instance headers only testing framework. I'm used to integrate it as an External project. It's couple of lines in the CMakeLists.txt and rest is job for the cmake invocation. It's pretty useful.
I was thinking a bit about your requirement @eteran and the most important is the comfort of writing the tests. It isn't involved in production so it doesn't matter how much memory or compile time does it take. I've got experiences with boost, catch2 and gtest and they do the great job.
@marekmosna Undersood, and I agree. It's more of a guideline than a requirement since "too heavy" is also a matter of taste :-).
Any one of those frameworks would certainly do the trick.
Unrelated, but I asked on the other PR, how come you closed it? It was planning on merging sometime this week if you still wanted to get it in.
I had to work on it to be able to integrate it so, now there is much more changes than just added cvector_reserve. I can again make just this simple one for you.
Check my fork and if you will be interested in any feature I made, just let me know.
Now I'm planning to involve the unit test framework but another situation comes up. According to the issue this can solved first on your repo and I can just update mine
@marekmosna I like your fork! I actually just updated the repo to have a .clang-format that reflects my style (but uses spaces!).
But also I think that your fork has a flaw in the assert and malloc related features. Basically, your code unconditionally uses cvector_malloc (and related), but that macro is only defined if CVECTOR_CUSTOM_MALLOC is specified. This effectively requires that CVECTOR_CUSTOM_MALLOC be set to even use the library.
I've fixed this in my repo in that cvector_malloc is defined if not already defined by the user, which i think is what we really want here. We want it to default to using malloc and family, but be overridable should the user want it. There's a similar concern for the custom assert stuff you've done.
So I'm certainly intersted in more PRs :-) Especially involving the unit test stuff you spoke about.
That being said, I'm not a fan of the BEGIN_BLOCK / END_BLOCK stuff as I think, for me, it's an unessary abstraction for such a small header only library.
Either way, thanks for the contributions!
And what about doctest?
Seems simple enough to work with, I like it's simplicity!
Are you gonna integrate any by yourself? There is a bunch of possibilities how to make it and I don't like be in position where I didn't read mind properly and the way I made it is just "not liked".
Glad you asked because I was assuming you'd just make a PR. But now that I understand your preference, I can spend some time next week on it making some unit-tests for this. Shouldn't take took long because it's a pretty simple library.
Glad you asked because I was assuming you'd just make a PR. But now that I understand your preference, I can spend some time next week on it making some unit-tests for this. Shouldn't take took long because it's a pretty simple library.
Thank you @eteran. Please do what you can. It's yours repo. I can adapt to anything you'll make. The way is not important for me, the result is. When you integrate the framework I can rewrite some tests for you.
Hi @eteran
How are you doing? Are you gonna integrate any framework? I was forced to leave your implementation due to lack of tests. In case you won't, please close this issue.
Thanks
Marek
I'm doing alright. Have been unfortunately very busy in personal life recently, but definitely do plan to get a test framework in there soon.
It's just been a bit hectic. I hope to get it done soon 🙂
@marekmosna OK, so I've done some digging into unit test stuff and... it's harder than one might think because this codebase is C.
While I can technically use something like catch2 or doctest, but because they are C++ specific, that would require adding a whole bunch of casts between void* and the somehow detected target pointer type. Basically, int *p = malloc(4) is legal C, but not legal C++.
So, I'm looking for a framework that lets me write the tests in C, and almost found one that seemed good. (cester), which unfortunately seems to throw false positives on it's automagical memory leak detection (confirmed with valgrind, there is no leak) :-/.
So I think that one is a non-starter.
So I guess one question is, do it have to be a framework? What if we just used cmake's ctest capabilities natively? Where each test is it's own source file with a main, and just asserts what must be true? And we'd run the tests with just like make test or ctest
Sure, it would be nice if the tests could be all in one file for such a small library, but I don't see many great options for unit testing C code that "just work".
Thoughts?
@marekmosna actually I spoke too soon! I found utest which seems to work pretty nicely :-) I'll have a few base line test up today and if you're still interested in added to that, that would be great. If not, I also understand.
@marekmosna (sorry for the rapid messages if they are a bother).
OK, so I've added some unit tests! They are based on the main test code, but actually do exercise most of the code pretty well as far as I can tell.
You can run them with any of:
make && make test
make check
or if you'd like to see the per-test output:
make
./unit-tests
The last of which outputs the following:
$ ./unit-tests
[==========] Running 7 test cases.
[ RUN ] test.vector_empty
[ OK ] test.vector_empty (900ns)
[ RUN ] test.vector_push_pop
[ OK ] test.vector_push_pop (1600ns)
[ RUN ] test.vector_iterator
[ OK ] test.vector_iterator (1500ns)
[ RUN ] test.vector_index
[ OK ] test.vector_index (1200ns)
[ RUN ] test.vector_insert_delete
[ OK ] test.vector_insert_delete (1700ns)
[ RUN ] test.vector_copy
[ OK ] test.vector_copy (2400ns)
[ RUN ] test.vector_reserve
[ OK ] test.vector_reserve (1900ns)
[==========] 7 test cases ran.
[ PASSED ] 7 tests.
I hope this works for your needs :-)
I'll be closing this issue, but do feel free to re-open if I messed something up, or open another issue if there is other stuff that you feel is needed.
Thanks!