Llama2-70b Loadgen Issue in Offline Scenario

[Mustafa Eyceoz]

Since the fixes for token measuring in the server scenario have been introduced, there is now a new issue in the offline scenario:

:::MLLOG {"key": "error_runtime", "value": "n_tokens argument missing or attempted to record 0 as number of tokens", "time_ms": 0.151550, "namespace": "mlperf::logging", "event_type": "POINT_IN_TIME", "metadata": {"is_error": true, "is_warning": false, "file": "logging.cc", "line_no": 442, "pid": 394, "tid": 419}}

Every time we call

                response = [lg.QuerySampleResponse(qitem[i].id, bi[0], bi[1])]
                lg.QuerySamplesComplete(response)

in offline, we are missing the new n_tokens field