Add stacktrace in output to directly jump to failed check
Eistroll opened this issue · comments
I just found this adapter and like it's configurability very much so I was wondering if the subject can be done with this adapter...
When I use the cppunittestframework and have a failure there is:
Source: link to the test and
StackTrace: link to the failed check
Could this be done with the adapter so that we would have
Test Name: TestMath.AddFails
Test FullName: TestMath.AddFails
Test Source: c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp : line 29
Test Outcome: Failed
Test Duration: 0:00:00,002
Result StackTrace: at Test_TestMath::AddFails() in c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp : line 30
Result Message:
Value of: Add(10, 10)
Actual: 20
Expected: 1000
instead of
Test Name: TestMath.AddFails
Test FullName: TestMath.AddFails
Test Source: c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp : line 29
Test Outcome: Failed
Test Duration: 0:00:00,002Result Message:
c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp:30
Value of: Add(10, 10)
Actual: 20
Expected: 1000
Has someone already tried to do this?
Best Regards
Thomas
Hey Thomas, thanks for your feedback! Unfortunately, there's no documentation of the VS test adapter framework, so we have to find out on our own how this can be done. However, I guess that if other test adapters support this, then it should also be possible for our adapter.
We will look into it, but it will take some time, since we are both rather busy... In the meantime, feel free to submit a pull request :-)
Ok, I was curious about how to get that running, and it is indeed possible. However, it is a bit tricky since VS seems to parse the stacktrace in a localized way in order to generate a link to the according source file location. I have worked around this, but would like to get feedback from users with different languages (system and visual studio) before merging this into the master branch.
Can anybody try the feature and add any feedback to this issue? You can download a VSIX containing the patch here.
Great that I got your interest :-) and I just tried it out.
I am working in German time zone with English VS.
I got the Stacktrace text but the link is not working:
Result StackTrace: at testcopexecutor.cpp:1144 in c:\users\thomas.roller\documents\work\git\win\wg2\base\test\testcopexecutor.cpp:line 1144.
This is working:
Result StackTrace: at Test_AdvLocalTime::AdvLocalTime_AddMonths() in c:\users\thomas.roller\documents\work\git\win\wg2\shared\test\test_advlocaltime.cpp:line 145
Maybe it is the . at the end?
I just tried it with the "." removed at the end and it worked :-)
Thanks a lot
Maybe it is possible to add several lines to the stacktrace? When I have two failed EXPECTS only the first fail is in the stacktrace.
Since cppunittestframework only has asserts it is there never a point...
How did you try without the "." - building and running? And, btw, which Windows/VS versions are you running if I may ask?
I in fact reopened the bug because of what you suggested in your last comment - that's indeed what I'd like to try. I just want to be reasonably sure that the patch works as desired before working this out... I'll let you know if there's something to test.
Yeah, that's where the fun starts: If I remove the . at the end of the stacktrace string, it doesn't work any more for me. Which renders the questions I asked above more important :-)
Thanks Christian. I will try it today.
I am using VS 2015 on Windows 10
Ok, thanks! Please go for this one, otherwise VS will update the extension automatically
@csoltenborn I tried it and it works... Great.
If we have several stack traces we would need the relation to the message.
Either we could leave the file and line in the message or have the "Expected: ..." as the stacktrace link, or ...
What do you think?
Yeah, that's true. I have something more compact in mind: I tried last night, and in the stacktrace section I can produce links such as "_#1 myfile.cpp:42". I.e., I'd like to relate message and link via "#1", "#_2" etc. - does that sound like a good idea?
Yes sounds good to see the relation between the message and the link.
I have checked in another patch - would you give it one more try? Here's the link...
Updated link... (minor improvements)
This should in general be possible, but I'd like to deal with that in another issue. May I ask you to create one? Maybe you can even attach some example tests making use of scoped traces.