Null-Value Validation Results prevent push with no explanation

[jason]

Please go to Stack Overflow for help and support:

https://stackoverflow.com/questions/tagged/tensorflow-model-analysis

If you open a GitHub issue, here is our policy:

1. It must be a bug, a feature request, or a significant problem with
   documentation (for small docs fixes please send a PR instead).
2. The form below must be filled out.

Here's why we have that policy: TensorFlow Model Analysis developers respond
to issues. We want to focus on work that benefits the whole community, e.g.,
fixing bugs and adding features. Support only helps individuals. GitHub also
notifies thousands of people when issues are filed. We want them to see you
communicating an interesting problem, rather than being redirected to Stack
Overflow.

---

System information

Lambda Quad; 4 2080 RTX Cards, 128 Gb RAM, 24 Virtual CPUs

• Have I written custom code (as opposed to using a stock example script
  provided in TensorFlow Model Analysis):

No

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):

UBUNTU 18.04

• TensorFlow Model Analysis installed from (source or binary):

pip install tensorflow-tfx

• TensorFlow Model Analysis version (use command below):

TensorFlow version: 2.1.0
TFX version: 0.21.4
pandas version: 1.0.3
TF Data Validation: 0.21.5
TF Model Analysis: 0.21.5

• Python version:

Python 3.7.7

• Jupyter Notebook version:

The version of the notebook server is: 5.7.8

• Exact command to reproduce:
  So I'm looking at what appears to be a TFMA bug. Every so often when training a model, it fails to push even though Tensorboard would suggest a successful passing model. Validation results as you can seee below are for some reason excluding the offending value.

You can obtain the TensorFlow Model Analysis version with

python -c "import tensorflow_model_analysis as tfma; print(tfma.version.VERSION)"

0.21.5

Describe the problem

Describe the problem clearly here. Be sure to convey here why it's a bug in
TensorFlow Model Analysis or a feature request.

Source code / logs

Include any logs or source code that would be helpful to diagnose the problem.
If including tracebacks, please include the full traceback. Large logs and files
should be attached. Try to provide a reproducible test case that is the bare
minimum necessary to generate the problem.

[jason]

for search indexing, results in text:

`
metric_validations_per_slice {
slice_key {
}
failures {
metric_key {
name: "precision"
model_name: "candidate"
}
metric_threshold {
value_threshold {
lower_bound {
value: 0.9
}
upper_bound {
value: 1.1
}
}
}
metric_value {
double_value {
}
}
}
}

`

[Mike Dreves]

This is an artifact of proto3 where when a value is 0 it is not displayed. The validation appears to be working correctly since a value of 0 is below the lower_bound threshold of 0.9. Need to investigate why precision is 0.

[jason]

ok thanks! i'll close ticket.