nimble test fails

Question

nimble test fails

timotheecour opened this issue 4 years ago · comments

Timothee Cour commented 4 years ago

@0atman
2 bugs:

bug 1

unittest.require had a long standing bug which I'm fixing in nim-lang/Nim#14676

this:

require getResponse(inputStream, outputStream, defLines) == "A == type string"

used to show:

[Suite] Interface Tests
    /Users/timothee/git_clone/nim/temp/INim/tests/test_interface.nim(34, 61): Check failed: getResponse(inputStream, outputStream, defLines) == "A == type string"
    getResponse(inputStream, outputStream, defLines) was A  : string

but did not actually make the test fail; after that PR the test fails, as it should; that indicates a bug in inim

bug 2

after fixing above issue, nimble test hangs in:

[Suite] INim Test Suite
  [OK] Get Nim Version
  [OK] Indent triggers
Hint: 47462 lines; 0.049s; 61.297MiB peakmem; Debug build; proj: /Users/timothee/git_clone/nim/temp/INim/tests/test_interface.nim; out: /Users/timothee/git_clone/nim/temp/INim/tests/test_interface [SuccessX]
/Users/timothee/git_clone/nim/Nim_devel/compiler/main.nim(402, 15) compiler msg initiated here [MsgOrigin]

[Suite] Interface Tests

Timothee Cour commented 4 years ago

yes

Tristram Oaten · Answer 1 · Tue Jun 16 2020 18:58:09 GMT+0800 (China Standard Time)

Thanks Timothee, I admit I don't really understand this, but thank you for helping us track this bug!

Ryan Cotter · Answer 2 · Fri Jun 26 2020 10:43:53 GMT+0800 (China Standard Time)

This is on the development branch of Nim, right @timotheecour ?

Ryan Cotter · Answer 3 · Fri Jun 26 2020 13:22:48 GMT+0800 (China Standard Time)

@0atman Maybe we should change the CI to use the devel branch instead of stable. I can probably start developing in devel and rely on local tests to check against the stable branch if we can check against both versions at once.

Tristram Oaten · Answer 4 · Sat Jun 27 2020 15:04:12 GMT+0800 (China Standard Time)

For this exact case, using the devel branch would make the tests pass, but do we want to be testing against a version of nim our users aren't using? Probably not. Perhaps we should fork our tests based on the capability of the compiler?

Timothee Cour · Answer 5 · Sat Jun 27 2020 15:40:11 GMT+0800 (China Standard Time)

a lot of packages test against both latest devel and latest realease of nim

Ryan Cotter · Answer 6 · Tue Jun 30 2020 14:45:42 GMT+0800 (China Standard Time)

If we test on both, we can prevent errors that might happen in the future. @timotheecour Does nimble test against stable and devel? If it does, we should be testing both in INim @0atman , so long as tests against devel in nimble don't break builds if the test passes in the latest stable. This way, we can fix upcoming bugs as nim releases new version without getting caught out by a new release

Tristram Oaten · Answer 7 · Tue Jun 30 2020 14:46:54 GMT+0800 (China Standard Time)

I get it now - Sounds really great! 👌

Timothee Cour · Answer 8 · Tue Jun 30 2020 15:08:45 GMT+0800 (China Standard Time)

Does nimble test against stable and devel?

that depends on how you setup your CI (ie, how you install nim in your CI), not on nimble

Ryan Cotter · Answer 9 · Tue Jun 30 2020 15:14:05 GMT+0800 (China Standard Time)

Sweet, so I guess we configure the CI to test against both and fail if one fails. I haven't worked on CI's much, but I'd love to help out if you want an extra set of eyes on it @0atman

Tristram Oaten · Answer 10 · Tue Jun 30 2020 15:20:04 GMT+0800 (China Standard Time)

@Tangdongle Absolutely, I reckon we can do it together. I've set up many a CI in my time!

I think we'll have two github actions, one for stable, one for devel. The default behaviour is to require all steps to pass.

I'm working on it here #99

Tristram Oaten · Answer 11 · Tue Jun 30 2020 19:09:07 GMT+0800 (China Standard Time)

This is now fixed by building the devel branch using choosenim!