Observability and IncompleteEvents

[Pablo Curiel]

When events are resubmitted, their original tracing context is lost.

Keeping this information is specially relevant when events are externalized.

Any thoughts on persisting context on the DB and restoring it when the event is read? Maybe using nullable column(s) and extracting context via an optional Bean, configured by spring-modulith-observability when this module is added.

[Oliver Drotbohm]

Does it really make sense to fully reestablish that context? Wouldn't those traces then look like an actual request having been performed?

[Pablo Curiel]

I think it makes sense regarding event traceability. I will try to explain why with an example.

Given this scenario:

• Service A receives a request with Trace ID ABC
• As a part of that request, it publishes MyEvent via RabbitMQ.
• Service B receives MyEvent and does whatever.
• Service Creceives MyEvent and does whatever.

Considering happy-path execution, Trace ID ABC would enable you to trace up from the initial request to Service A, down to MyEvent reception and processing in Service B and Service C.

However, if anything goes wrong publishing MyEvent (e.g., RabbitMQ is down for a few seconds) and it is finally published when retried from the EventRepository, it will be published with another Trace ID, and we won't be able to trace it down to Service B and Service C.

However, persisting the context and restoring it when retrying the event would solve this issue.
(Note: to simplify the example I'm referring only to the Trace ID, but the same would apply to other Trace Context info).