Question about gpt2 model

[saramohammadinejad]

Ask a Question

How should be the input when we want to do prediction on batch_size more than 1?

Question

Explain your question here.
this one:
texts = ["Here is some text to encode : Hello World!", "Here is some text to encode : Hello World!"]
input_ids_1 = [[tokenizer.encode(text, add_special_tokens=True) for text in texts]]

or this one:
texts = ["Here is some text to encode : Hello World!", "Here is some text to encode : Hello World!"]
input_ids_1 = [[tokenizer.encode(text, add_special_tokens=True)] for text in texts]

The get_inputs() API for onnx model returns: ['input1_dynamic_axes_1', 'input1_dynamic_axes_2', 'input1_dynamic_axes_3'] as input shape but it is not obvious which one corresponds to batch size and which one to sequence length? What is the third dimension?

Further information

Relevant Area (e.g. model usage, backend, best practices, pre-/post- processing, converters):

Is this issue related to a specific model? gpt2 model https://github.com/onnx/models/blob/main/text/machine_comprehension/gpt-2/model/gpt2-10.onnx
Model name (e.g. mnist):
Model opset (e.g. 7):

Notes

Any additional information, code snippets.

[zhaoyang-star]

Also have the same question.