Inputs to TensorFlow serving REST API

[Vallabh Mahajan]

I have build a seq2seq model by referring to this tutorial.
https://keras.io/examples/nlp/lstm_seq2seq/

After training the model, instead of saving the model directly, I saved the encoder_model and decoder_model separately.I'm using TensorFlow serving to deploy these two models.

Encoder model input was a numpy array of length 1, so I converted that to JSON to pass it to REST API. This worked and I got the desired result.

But, for decoder this method didn't worked as length of decoder_input is 3 and data_type is numpy array.

Following the block of decode_sequence function where decoder_model is called.

dec_model_url = "http://localhost:8400/v1/models/dec_model:predict"
headers = {
    'content-type': "application/json;charset=UTF-8'",
    'cache-control': "no-cache",
    'Accept':'application/json'
    }
    while not stop_condition:
        decoder_ip = ([target_seq] + states_value)
        target_seq1 = target_seq.tolist()
        target_seq1=[target_seq1]

        states_value1 = states_value
        states_value1[0] = states_value1[0].tolist()
        states_value1[1] = states_value1[1].tolist()


        decoder_ip1 = (target_seq1 + states_value1[0] + states_value1[1])
        start_main = '{"instances":'
        end_main = '}'
        decoder_ip1 = start_main + str(decoder_ip1) +end_main


        output_tokens, h = requests.request("POST", dec_model_url, data=decoder_ip1, headers=headers)

When I run this, I got the following error.

{"error": "instances is a plain list, but expecting list of objects as multiple input tensors required as per tensorinfo_map'

What is the correct way to pass the decoder_model inputs with REST API?

[gaikwadrahul8]

@VallabhMahajan1

Apologies for the delay and As you mentioned above length of decoder_input is 3 and data_type is Numpy array so could you please try Specifying input tensors in column format and In the columnar format, inputs are keyed to inputs key in the JSON request.

For your reference, I have added official documentation here so please follow it and I hope your issue will be resolved

Please let us know if you need any further assistance?

Thank You!

[Vallabh Mahajan]

Yes, it is working. Thank you so much for the help.

[gaikwadrahul8]

Hi, @VallabhMahajan1

You're welcome and Good to hear that your issue got resolved, so could you please close this issue now ?

Thank you!

[gaikwadrahul8]

Hi, @VallabhMahajan1

Closing this issue due to lack of recent activity for couple of weeks and also your issue got resolved. Please feel free to reopen the issue, if you need any further assistance or update

Thank you!