Activites cannot return None (the JSON object must be str, bytes or bytearray, not NoneType)
evanlouie opened this issue Β· comments
π Describe the bug
Activities cannot return None
.
If an activity returns None
, the following exception is raised when the orchestrator attempts to parse the returned value.:
System.Private.CoreLib: Exception while executing function: Functions.hello_orchestrator. System.Private.CoreLib: Orchestrator function 'hello_orchestrator' failed: One or more errors occurred. (Result: Failure
Exception: TypeError: the JSON object must be str, bytes or bytearray, not NoneType
Stack: File "/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.5198/workers/python/3.10/OSX/Arm64/azure_functions_worker/dispatcher.py", line 479, in _handle__invocation_request
call_result = await self._loop.run_in_executor(
File "/Users/***/.pyenv/versions/3.10.12/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.5198/workers/python/3.10/OSX/Arm64/azure_functions_worker/dispatcher.py", line 752, in _run_sync_func
return ExtensionManager.get_sync_invocation_wrapper(context,
File "/opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.5198/workers/python/3.10/OSX/Arm64/azure_functions_worker/extension.py", line 215, in _raw_invocation_wrapper
result = function(**args)
File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/orchestrator.py", line 69, in handle
return Orchestrator(fn).handle(DurableOrchestrationContext.from_json(context_body))
File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/orchestrator.py", line 47, in handle
return self.task_orchestration_executor.execute(context, context.histories, self.fn)
File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/models/TaskOrchestrationExecutor.py", line 93, in execute
self.process_event(event)
File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/models/TaskOrchestrationExecutor.py", line 144, in process_event
self.set_task_value(event, is_success, id_key)
File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/models/TaskOrchestrationExecutor.py", line 197, in set_task_value
new_value = parse_history_event(event)
File "/Users/***/workspace/tmp/durable/.venv/lib/python3.10/site-packages/azure/durable_functions/models/TaskOrchestrationExecutor.py", line 171, in parse_history_event
return json.loads(directive_result.Result, object_hook=_deserialize_custom_object)
File "/Users/***/.pyenv/versions/3.10.12/lib/python3.10/json/__init__.py", line 339, in loads
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
).
π€ Expected behavior
As per the docs, activiies do not need to return a value.
Activity function: It's called by the orchestrator function, performs work, and optionally returns a value.
β Steps to reproduce
import azure.functions as func
import azure.durable_functions as df
app = df.DFApp(http_auth_level=func.AuthLevel.ANONYMOUS)
# An HTTP-Triggered Function with a Durable Functions Client binding
@app.route(route="orchestrators/{functionName}")
@app.durable_client_input(client_name="client")
async def http_start(req: func.HttpRequest, client):
function_name = req.route_params.get("functionName")
instance_id = await client.start_new(function_name)
response = client.create_check_status_response(req, instance_id)
return response
# Orchestrator
@app.orchestration_trigger(context_name="context")
def hello_orchestrator(context):
result1 = yield context.call_activity("hello", "Seattle")
result2 = yield context.call_activity("hello", "Tokyo")
result3 = yield context.call_activity("hello", "London")
return [result1, result2, result3]
# Activity
@app.activity_trigger(input_name="city")
def hello(city: str):
if city == "London":
return None
return "Hello " + city
Then call the orchestrator:
curl http://localhost:7071/api/orchestrators/hello_orchestrator
Other Notes
Their seems to be a misalignment between the durable functions runtime and the azure-functions-durable
library:
azure-functions-durable
is expecting theResult
of tasks to ALWAYS be a JSON serializeable string so it canjson.loads(...)
it (seeTaskOrchestrationExecutor
).- The
Result
property for events in table storage is a string. But when an activty returnsNone
, instead of storing"null"
in the row, it saves nothing.
Either azure-functions-durable
needs to check for None
before doing json.loads(...)
in TaskOrchestrationExecutor.set_task_value
or the durable functions runtime needs to start emitting "null"
for null result values.
probably related: #260
Hi @evanlouie, thanks for reaching out and thanks for the detailed bug report.
As per the docs, activiies do not need to return a value.
You're correct, that's a documentation bug. I'm noting that down as a follow up item.
azure-functions-durable is expecting the Result of tasks to ALWAYS be a JSON serializeable string so it can json.loads(...) it (see TaskOrchestrationExecutor).
Yes, that's correct, there's an assumption that inputs and outputs can be JSON serialized.
The Result property for events in table storage is a string. But when an activty returns None, instead of storing "null" in the row, it saves nothing.
Just to confirm, are you actually triggering a bug where you receive a None
result and the result isn't properly assigned to a task? To my understanding, since activities are failing at the point of returning None
, then the error is occuring before we even attempt to set a task's output value, right?
Just to confirm, are you actually triggering a bug where you receive a None result and the result isn't properly assigned to a task? To my understanding, since activities are failing at the point of returning None, then the error is occuring before we even attempt to set a task's output value, right?
The activities which return None
actually complete successfully. If I look the storage account in the History
table (i.e TestHubNameHistory
on local azurite), the activities which return None
will have EventType
== TaskCompleted
with Result
== null
.
then the error is occuring before we even attempt to set a task's output value, right?
So from the looks of it, the durable functions runtime allow activities to return None (and is stored as null
in table), but the TaskOrchestrationExecutor
which the orchestrator calls does not deserialize the null
as it expects a JSON string.
@davidmrdavid @lilyjma I recently root caused another customer issue to this bug. It would be great if we could prioritize fixing it, as it seems like a trivially simple issue that is really hard to debug.
Close the issue as the PR is merged.
(Unpinning issue since it's fixed.)