host_processor_vcpu_count in <system_desc_id>.json
szutenberg opened this issue · comments
According to https://github.com/mlcommons/policies/blob/master/submission_rules.adoc#57-system_desc_idjson-metadata it's required to provide field host_processor_vcpu_count.
I see that relevant checks are missing in https://github.com/mlcommons/inference/blob/master/tools/submission/submission_checker.py so the submitter get
submission_checker.py:3240 WARNING] <redacted>, field host_processor_vcpu_count is unknown
.
My proposal is to remove host_processor_vcpu_count
from the submission rules because this rule is not enforced anyway. In inference_results_v3.1 this field is present only in closed/Intel-HabanaLabs/systems/HLS-Gaudi2-PT.json
.
If we provide only host_processor_vcpu_count instead of host_processor_core_count then submission checker fails:
submission_checker.py:3220 ERROR] <REDACTED>, field host_processor_core_count is missing
submission_checker.py:3240 WARNING] <REDACTED>, field host_processor_vcpu_count is unknown
A related issue: #1351