Erorr occured when I modify real-time inference to batch transform
isoyaoya opened this issue · comments
hello,
Thanks for your amazing job~ I followed your code to build end-to-end Hugging Face Transformers with Sagemaker Pipeline. This code use real-time inference and create an endpoint, but I want to achieve batch transform inference.
My code looks like below.
`
from sagemaker.huggingface.model import HuggingFaceModel
from sagemaker.s3 import S3Uploader,s3_path_join
from sagemaker.inputs import TransformInput
from sagemaker.workflow.steps import TransformStep
huggingfacemodel = HuggingFaceModel(
model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts, # path to your trained sagemaker mod
role=role,
transformers_version=_transformers_version,
pytorch_version=_pytorch_version,
py_version=_py_version,
sagemaker_session=sagemaker_session,
)
output_s3_path = s3_path_join("s3://",'sagemaker-us-east-1-7348***',"**y-datalab-modela/output")
transformer_instance_type=ParameterString(name="Fp16", default_value="False")
transformer = huggingfacemodel.transformer(
instance_count=1,
instance_type="ml.m5.4xlarge",
output_path=output_s3_path,
strategy='SingleRecord',
)
batch_data = ParameterString(
name="BatchData",
default_value='s3://sagemaker-us-east-1-73483***/**y-datalab-modela/test/stary_data.jsonl',
)
transform_step = TransformStep(
name="Batch",
transformer=transformer,
inputs=TransformInput(
data=batch_data,
content_type='application/json',
split_type="Line",
)
)
`
but when I run the code, it returns error: ‘AttributeError: 'Properties' object has no attribute 'decode'’.
Could you please help in this issue or could you please provide some sample code about how to use batch transform pipeline? Thank you!
Could you please format your code to read it better and also share the error you are seeing? And is that the complete code for your example?
Have you tried batch transform inference rather than realtime inference before? Could you please provide some sample codes?
I have attached the whole notebook. Now there are some other errors:
Could you please add the whole error stack it seems that there is something missing.
Attach the whole error stack, thank you so much!
From error it seems that you are sending malformed payloads. Here is an example for batch transform: https://github.com/huggingface/notebooks/blob/main/sagemaker/12_batch_transform_inference/sagemaker-notebook.ipynb
I have seen this code before. Of course, if I do not use Sagemaker pipeline, I can create batch transform job without error and output correct predict results. However, I was required to use Sagemaker pipeline and batch transform to achieve automaticlly process.
Have you tried to modify your code from real-time inference to batch transform with Sagemaker pipeline before? In some NLP applications, batch transform is often used.
Use a lambda, create a lambda step to deploy registered model could solve the issue.
@isoyaoya can you share the how you configured lambda step to do batch transform please?
You can find the code here: https://github.com/philschmid/huggingface-sagemaker-workshop-series/tree/main/workshop_3_mlops/utils