iree-e2e-matmul-test duplicate flags
Groverkss opened this issue · comments
What happened?
On running:
<BUILD_DIR>/tools/testing/e2e/iree-e2e-matmul-test --help
there are duplicate flag options for test_utils.h:
# ===----------------------------------------------------------------------===
# Flags in iree/tools/testing/e2e/test_utils.h
# ===----------------------------------------------------------------------===
# Requires floating point result elements to match exactly.
--require_exact_results=true
# Requires floating point result elements to match exactly.
--require_exact_results=true
# Maximum absolute difference allowed with inexact floating point results.
--acceptable_fp_delta=1e-05
# Maximum absolute difference allowed with inexact floating point results.
--acceptable_fp_delta=1e-05
# Maximum number of tensor elements to check for the given test. For larger buffers, only every n-th element will be checked for some n chosed to stay just under that threshold and to avoid being a divisor of the inner dimension size to avoid special patterns. As the check uses a slow reference implementation, this is a trade-off between test latency and coverage. The value 0 means check all elements.
--max_elements_to_check=10000
# Maximum number of tensor elements to check for the given test. For larger buffers, only every n-th element will be checked for some n chosed to stay just under that threshold and to avoid being a divisor of the inner dimension size to avoid special patterns. As the check uses a slow reference implementation, this is a trade-off between test latency and coverage. The value 0 means check all elements.
--max_elements_to_check=10000
On changing the flags, --help only changes one of the flags:
iree-e2e-matmul-test --require_exact_results=false --help
# ===----------------------------------------------------------------------===
# Flags in iree/tools/testing/e2e/test_utils.h
# ===----------------------------------------------------------------------===
# Requires floating point result elements to match exactly.
--require_exact_results=false
# Requires floating point result elements to match exactly.
--require_exact_results=true
# Maximum absolute difference allowed with inexact floating point results.
--acceptable_fp_delta=1e-05
# Maximum absolute difference allowed with inexact floating point results.
--acceptable_fp_delta=1e-05
# Maximum number of tensor elements to check for the given test. For larger buffers, only every n-th element will be checked for some n chosed to stay just under that threshold and to avoid being a divisor of the inner dimension size to avoid special patterns. As the check uses a slow reference implementation, this is a trade-off between test latency and coverage. The value 0 means check all elements.
--max_elements_to_check=10000
# Maximum number of tensor elements to check for the given test. For larger buffers, only every n-th element will be checked for some n chosed to stay just under that threshold and to avoid being a divisor of the inner dimension size to avoid special patterns. As the check uses a slow reference implementation, this is a trade-off between test latency and coverage. The value 0 means check all elements.
--max_elements_to_check=10000
Steps to reproduce your issue
ninja iree-test-deps
<BUILD_DIR>/tools/testing/e2e/iree-e2e-matmul-test --help
What component(s) does this issue relate to?
No response
Version information
Commit Hash: c1cfbfc
Additional context
No response
that test_utils.h is defining flags in the header - that's invalid - that means every file that includes the .h will define the flags. the header should expose some accessor methods and keep the flags in the test_utils.c file:
in test_utils.h:
bool iree_test_utils_require_exact_results(void);in test_utils.c:
IREE_FLAG(...);
bool iree_test_utils_require_exact_results(void) { return FLAG_require_exact_results; }