org-super-agenda-test--:auto-map test failure
Apteryks opened this issue · comments
Hello!
Attempting to update this package on Guix, I stumbled on the following test failure:
starting phase `check'
DEBUG: ARGS: --debug --
DEBUG: Remaining args:
DEBUG: Local time: Wed 16 Dec 2020 09:35:54 PM UTC
DEBUG: UTC: Wed 16 Dec 2020 09:35:54 PM UTC
DEBUG: TESTING...
Org version: 9.4.2
Test results loaded
Running 66 tests (2020-12-16 21:35:55+0000, selector ‘t’)
org-super-agenda-mode enabled.
passed 1/66 org-super-agenda-test--:and (0.245904 sec)
org-super-agenda-mode enabled.
passed 2/66 org-super-agenda-test--:anything (0.037873 sec)
org-super-agenda-mode enabled.
passed 3/66 org-super-agenda-test--:auto-category (0.024705 sec)
org-super-agenda-mode enabled.
passed 4/66 org-super-agenda-test--:auto-group (0.038509 sec)
org-super-agenda-mode enabled.
Test org-super-agenda-test--:auto-map backtrace:
signal(error ("Empty result for body:307f68fcbbad5bc27a9bb104acbf...
error("Empty result for body:%s\nSTORED-RESULT:%s\nNEW-RESU..." "307
(if (and stored-result new-result) nil (error "Empty result for body
(or (equal stored-result new-result) (if (and stored-result new-resu
(let ((stored-result (ht-get org-super-agenda-test-results body-grou
(if org-super-agenda-test-show-results nil (let ((stored-result (ht-
(let ((body-groups-hash (secure-hash 'md5 (format "%S" (list '(org-a
(progn (if (> (ht-size org-super-agenda-test-results) 0) nil (org-su
(setq value-125 (progn (if (> (ht-size org-super-agenda-test-results
(unwind-protect (setq value-125 (progn (if (> (ht-size org-super-age
(if (unwind-protect (setq value-125 (progn (if (> (ht-size org-super
(let (form-description-126) (if (unwind-protect (setq value-125 (pro
(let ((value-125 (gensym "ert-form-evaluation-aborted-"))) (let (for
(closure (t) nil (let ((value-125 (gensym "ert-form-evaluation-abort
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name org-super-agenda-test--:auto-map :doc
ert-run-or-rerun-test(#s(ert--stats :selector t :tests ... :test-map
ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
ert-run-tests-batch(nil)
ert-run-tests-batch-and-exit()
command-line-1(("--eval" "(message \"Org version: %s\" (org-version)
command-line()
normal-top-level()
Test org-super-agenda-test--:auto-map condition:
(error "Empty result for body:307f68fcbbad5bc27a9bb104acbf6997
STORED-RESULT:nil
NEW-RESULT:Day-agenda (W27):
Wednesday 5 July 2017
Face: org-agenda-calendar-sexp
test: 7:02...... Sunrise (12:04 of daylight)
test: 19:07...... Sunset
Face: org-scheduled-previously
ambition: Sched. 1x: TODO [#A] Skype with president of Antarctica :universe:ambition:world::meetings:
Face: org-scheduled-today
test: 18:00...... Scheduled: TODO Order a pizza :food:dinner:
test: Scheduled: TODO [#B] Fix flux capacitor :spaceship:shopping:@computer:
test: Scheduled: TODO Shop for groceries :food:shopping:@town:
ideas: Scheduled: SOMEDAY Rewrite Emacs in Common Lisp :Emacs:elisp:computers:software:programming:
test: Scheduled: TODO [#C] Get haircut :personal:@town:
ambition: TODO Practice leaping tall ! :universe:ambition::personal:
Face: org-upcoming-deadline
ambition: In 2 d.: TODO [#A] Take over the world :universe:ambition::world:
ambition: In 10 d.: TODO [#A] Take over the universe :universe:ambition:
Face: org-upcoming-distant-deadline
test: In 27 d.: TODO [#A] Spaceship lease :bills:spaceship:
ambition: In 5 d.: TODO [#B] Renew membership in supervillain club :universe:ambition::
test: In 16 d.: TODO [#B] Internet :bills:
ambition: In 53 d.: WAITING Visit the moon :universe:ambition::space:travel:
ambition: In 77 d.: TODO Visit Mars :universe:ambition::space:travel:planet:
Face: org-warning
test: Deadline: CHECK /r/emacs :website:Emacs:
Other items
8:00...... ----------------
10:00...... ----------------
12:00...... now - - - - - - - - - - - - - - - - - - - - - - - - -
12:00...... ----------------
14:00...... ----------------
16:00...... ----------------
18:00...... ----------------
20:00...... ----------------
")
FAILED 5/66 org-super-agenda-test--:auto-map (0.024856 sec)
org-super-agenda-mode enabled.
passed 6/66 org-super-agenda-test--:auto-planning (0.026880 sec)
org-super-agenda-mode enabled.
passed 7/66 org-super-agenda-test--:auto-tags (0.025303 sec)
org-super-agenda-mode enabled.
passed 8/66 org-super-agenda-test--:auto-ts (0.038638 sec)
org-super-agenda-mode enabled.
passed 9/66 org-super-agenda-test--:category (0.038265 sec)
org-super-agenda-mode enabled.
passed 10/66 org-super-agenda-test--:category-multi (0.024736 sec)
org-super-agenda-mode enabled.
passed 11/66 org-super-agenda-test--:children-nil (0.038294 sec)
org-super-agenda-mode enabled.
passed 12/66 org-super-agenda-test--:children-string (0.024958 sec)
org-super-agenda-mode enabled.
passed 13/66 org-super-agenda-test--:children-t (0.038230 sec)
org-super-agenda-mode enabled.
passed 14/66 org-super-agenda-test--:children-todo (0.025011 sec)
org-super-agenda-mode enabled.
passed 15/66 org-super-agenda-test--:date (0.038042 sec)
org-super-agenda-mode enabled.
passed 16/66 org-super-agenda-test--:deadline-after (0.025028 sec)
org-super-agenda-mode enabled.
passed 17/66 org-super-agenda-test--:deadline-before (0.038469 sec)
org-super-agenda-mode enabled.
passed 18/66 org-super-agenda-test--:deadline-future (0.024987 sec)
org-super-agenda-mode enabled.
passed 19/66 org-super-agenda-test--:deadline-nil (0.037246 sec)
org-super-agenda-mode enabled.
passed 20/66 org-super-agenda-test--:deadline-nil-agenda (0.024985 sec)
org-super-agenda-mode enabled.
passed 21/66 org-super-agenda-test--:deadline-past (0.038632 sec)
org-super-agenda-mode enabled.
passed 22/66 org-super-agenda-test--:deadline-t (0.025152 sec)
org-super-agenda-mode enabled.
passed 23/66 org-super-agenda-test--:deadline-today (0.038798 sec)
org-super-agenda-mode enabled.
passed 24/66 org-super-agenda-test--:discard (0.006047 sec)
org-super-agenda-mode enabled.
passed 25/66 org-super-agenda-test--:effort< (0.038394 sec)
org-super-agenda-mode enabled.
passed 26/66 org-super-agenda-test--:effort> (0.024929 sec)
org-super-agenda-mode enabled.
passed 27/66 org-super-agenda-test--:file-path (0.038147 sec)
org-super-agenda-mode enabled.
passed 28/66 org-super-agenda-test--:habit (0.025105 sec)
org-super-agenda-mode enabled.
passed 29/66 org-super-agenda-test--:heading-regexp (0.038570 sec)
org-super-agenda-mode enabled.
passed 30/66 org-super-agenda-test--:log (0.024789 sec)
org-super-agenda-mode enabled.
passed 31/66 org-super-agenda-test--:not (0.038210 sec)
org-super-agenda-mode enabled.
passed 32/66 org-super-agenda-test--:order (0.024982 sec)
org-super-agenda-mode enabled.
passed 33/66 org-super-agenda-test--:order-multi (0.038345 sec)
org-super-agenda-mode enabled.
passed 34/66 org-super-agenda-test--:pred (0.025472 sec)
org-super-agenda-mode enabled.
passed 35/66 org-super-agenda-test--:priority (0.039220 sec)
org-super-agenda-mode enabled.
passed 36/66 org-super-agenda-test--:priority< (0.025757 sec)
org-super-agenda-mode enabled.
passed 37/66 org-super-agenda-test--:priority<= (0.039076 sec)
org-super-agenda-mode enabled.
passed 38/66 org-super-agenda-test--:priority> (0.025865 sec)
org-super-agenda-mode enabled.
passed 39/66 org-super-agenda-test--:priority>= (0.039209 sec)
org-super-agenda-mode enabled.
passed 40/66 org-super-agenda-test--:regexp (0.024956 sec)
org-super-agenda-mode enabled.
passed 41/66 org-super-agenda-test--:scheduled-after (0.038465 sec)
org-super-agenda-mode enabled.
passed 42/66 org-super-agenda-test--:scheduled-before (0.025016 sec)
org-super-agenda-mode enabled.
passed 43/66 org-super-agenda-test--:scheduled-future (0.039905 sec)
org-super-agenda-mode enabled.
passed 44/66 org-super-agenda-test--:scheduled-nil (0.024881 sec)
org-super-agenda-mode enabled.
passed 45/66 org-super-agenda-test--:scheduled-past (0.038335 sec)
org-super-agenda-mode enabled.
passed 46/66 org-super-agenda-test--:scheduled-t (0.038723 sec)
org-super-agenda-mode enabled.
passed 47/66 org-super-agenda-test--:scheduled-today (0.025100 sec)
org-super-agenda-mode enabled.
passed 48/66 org-super-agenda-test--:tag (0.038430 sec)
org-super-agenda-mode enabled.
passed 49/66 org-super-agenda-test--:tag-with-inheritance (0.024425 sec)
org-super-agenda-mode enabled.
passed 50/66 org-super-agenda-test--:time-grid (0.024751 sec)
org-super-agenda-mode enabled.
passed 51/66 org-super-agenda-test--:todo (0.038259 sec)
org-super-agenda-mode enabled.
passed 52/66 org-super-agenda-test--agenda-with-grid-and-todo-with-children (0.043301 sec)
org-super-agenda-mode enabled.
passed 53/66 org-super-agenda-test--auto-dir-name (0.038344 sec)
org-super-agenda-mode enabled.
passed 54/66 org-super-agenda-test--auto-groups (0.025837 sec)
org-super-agenda-mode enabled.
passed 55/66 org-super-agenda-test--auto-outline-path (0.038898 sec)
org-super-agenda-mode enabled.
passed 56/66 org-super-agenda-test--auto-parent (0.025320 sec)
org-super-agenda-mode enabled.
passed 57/66 org-super-agenda-test--auto-property (0.039044 sec)
org-super-agenda-mode enabled.
passed 58/66 org-super-agenda-test--discard-with-2-regexps (0.024579 sec)
org-super-agenda-mode enabled.
passed 59/66 org-super-agenda-test--forward-looking (0.039219 sec)
org-super-agenda-mode enabled.
passed 60/66 org-super-agenda-test--heading-regexp (0.025122 sec)
org-super-agenda-mode enabled.
passed 61/66 org-super-agenda-test--main-example (0.039689 sec)
org-super-agenda-mode enabled.
passed 62/66 org-super-agenda-test--no-groups (0.025011 sec)
org-super-agenda-mode enabled.
passed 63/66 org-super-agenda-test--priority<B-with-order-100 (0.039515 sec)
org-super-agenda-mode enabled.
passed 64/66 org-super-agenda-test--priority>=B (0.025843 sec)
org-super-agenda-mode enabled.
passed 65/66 org-super-agenda-test--someday-tags-view-Emacs (0.016425 sec)
org-super-agenda-mode enabled.
passed 66/66 org-super-agenda-test--time-grid-with-unprioritized-order-100 (0.025172 sec)
Ran 66 tests, 65 results as expected, 1 unexpected (2020-12-16 21:35:57+0000, 2.403023 sec)
1 unexpected results:
FAILED org-super-agenda-test--:auto-map
command "test/run" "--debug" failed with status 1
It may be due to using dependencies newer than those listed in the Package-Requires header:
$ ./pre-inst-env guix show emacs-org-super-agenda
name: emacs-org-super-agenda
version: 1.2
outputs: out
systems: x86_64-linux i686-linux
dependencies: emacs-dash@2.17.0 emacs-f@0.20.0 emacs-ht@2.2 emacs-org@9.4.2 emacs-s@1.12.0 emacs-ts@0.2
+ util-linux@2.35.1
location: /home/maxim/src/guix/gnu/packages/emacs-xyz.scm:14859:2
homepage: https://github.com/alphapapa/org-super-agenda
license: GPL 3+
synopsis: Supercharged Org agenda
description: This package allows items in the Org agenda to be grouped into sections while preserving
+ the structure imposed by any timestamps.
Thank you!
This happens on Emacs >= 27.1 because it hashes lambdas differently than Emacs <= 26.3, and since the test's argument is a lambda, that is what the test result is keyed on, so the test fails to find the test result in the test data.
This can't easily be fixed without either dropping support for Emacs <= 26.3 (which shouldn't be done for, probably, years to come), or making a likely convoluted change to the test suite to handle Emacs versions differently.
I might try the latter at some point. In the meantime, this test failure is harmless and a false negative.
I disabled the test and pushed the package update in Guix with this commit: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=d6dd120be6a527494e9204fa227842db466171e6.
Thanks for the prompt feedback!
Thanks. A couple of comments:
+ (modify-phases %standard-phases
+ (add-after 'unpack 'prepare-for-tests.el
+ (lambda _
+ (make-file-writable "test/test.el")
+ (emacs-substitute-variables "test/test.el"
+ ("org-super-agenda-test-results-file"
+ (string-append (getcwd) "/test/results.el")))
What is the purpose of modifying that variable? I guess it's to correct the CWD while running the tests, however that seems like an awkward way of doing it, and it may indicate doing something wrong.
Specifically, I see earlier in the diff:
+ #:test-command '("test/run" "--debug")
That's not the way I'm running the tests now. Please see the Makefile
in the repo. To run the tests, you should only need to run make tests
from the repo root. If you use that instead, that variable substitution will probably not be required. (I should probably remove that old test/run
script now that I'm using makem.sh
here.)
Also:
+ ;; The following test fail (see:
+ ;; https://github.com/alphapapa/org-super-agenda/issues/183).
Would you please note in the comment that the test only fails on Emacs >= 27.1?
Thanks for your work on this.
Thanks. A couple of comments:
+ (modify-phases %standard-phases + (add-after 'unpack 'prepare-for-tests.el + (lambda _ + (make-file-writable "test/test.el") + (emacs-substitute-variables "test/test.el" + ("org-super-agenda-test-results-file" + (string-append (getcwd) "/test/results.el")))What is the purpose of modifying that variable? I guess it's to correct the CWD while running the tests, however that seems like an awkward way of doing it, and it may indicate doing something wrong.
Without adapting the org-super-agenda-test-results-file
variable, the test suite fails with:
Opening input file: No such file or directory, /tmp/guix-build-emacs-org-super-agenda-1.2.drv-0/source/test/test/results.el
I'm guessing that's due to the default value being resolved based on the location of the '.git' file in the project, which doesn't exist in the Guix checkout of the project (for reasons mentioned alphapapa/org-ql#166 (comment)).
Specifically, I see earlier in the diff:
+ #:test-command '("test/run" "--debug")
That's not the way I'm running the tests now. Please see the
Makefile
in the repo. To run the tests, you should only need to runmake tests
from the repo root. If you use that instead, that variable substitution will probably not be required. (I should probably remove that oldtest/run
script now that I'm usingmakem.sh
here.)
I'd like to, but I can't, because the makem.sh depends on the git metadata being present.
Also:
+ ;; The following test fail (see: + ;; https://github.com/alphapapa/org-super-agenda/issues/183).Would you please note in the comment that the test only fails on Emacs >= 27.1?
There's currently only one version of Emacs package in Guix, and that's 27.1. So adding this information wouldn't bring much in this context.
Thanks for your work on this.
My pleasure!
Thanks for your helpful replies.