[major] number of lines covered not accurate?
timotheecour opened this issue · comments
- why are there 27 lines considered as executed? I only see 4 lines in blue (in another test file, the number of blue lines was matching the number of lines considered as executed)
- why is line 12 in blue? inconsistent: this means if you have N 1-line funcitons, you get N+1 executed lines?
Note: this was after running:
git clone https://github.com/timotheecour/vitanim
cd vitanim/drange
coco
other example where I don't understand line count:
should we have an option to discount it from line/function/branch coverage stats? otherwise it inflates the numbers
I'm still digging this and it seems actually GCOV/LCOV might not be the perfect solution for code coverage.
Simple example, 1 test file imports 1 function from another Nim file like so:
## Minimal nim file used in tests for checking compilation and coverage
import ../coco
echo get_cache_folder("foo.nim", "bar", 42)
After compilation, we have a C file containing only the required function, not the whole Coco library:
#line 18 "/Users/sam/Code/nim-coverage/coco.nim"
N_LIB_PRIVATE N_NIMCALL(NimStringDesc*, get_cache_folder_2hKxOP9bo6u9buKgLZs3GK2Q)(NimStringDesc* filename, NimStringDesc* nimcache, NI increment) {
NimStringDesc* result; NimStringDesc* fmtRes; nimfr_("get_cache_folder", "coco.nim"); result = (NimStringDesc*)0;
#line 19 "/Users/sam/Code/nim-coverage/coco.nim"
nimln_(19, "coco.nim");
#line 19 "/Users/sam/Code/nim-coverage/coco.nim"
#line 19 "/Users/sam/Code/nim-coverage/coco.nim"
fmtRes = rawNewString(((NI) 67));
# and so on....
And the LCOV.info contains only the following:
SF:/Users/sam/Code/nim-coverage/coco.nim
FN:18,get_cache_folder_2hKxOP9bo6u9buKgLZs3GK2Q
FN:20,coco_cocoInit000
FN:26,coco_cocoDatInit000
FNDA:1,get_cache_folder_2hKxOP9bo6u9buKgLZs3GK2Q
FNDA:1,coco_cocoInit000
FNDA:1,coco_cocoDatInit000
FNF:3
FNH:3
DA:19,4
DA:22,2
DA:24,1
DA:27,2
DA:260,6
DA:268,1
LF:6
LH:6
end_of_record
Resulting in a wrong coverage because there's only one function instead of everything.
So either I need to parse Nim files by hand to consolidate the results or try to find X (compiler flags?) to generate C files more in sync with the Nim sources.
parse Nim files by hand
whatever the right way is, this isn't it ;-)
- random completely untested idea off the top of my head:
for each file bar.nim you want coverage on, generate a coverage_bar.nim with contents:
include bar.nim
So either I need to parse Nim files by hand to consolidate the results or try to find X (compiler flags?) to generate C files more in sync with the Nim sources.
with nim-lang/Nim#9560 at list you could list public symbols; but may not be exactly what's needed here
Is the nim compiler removing dead code?
--deadCodeElim:off
Is the nim compiler removing dead code?
--deadCodeElim:off
Thanks for your suggestion. I tried the flag but it didn't help. Unfortunately I think this project is out of my reach as I'm totally new to Nim.