[enhancement]: introspect cloud-init internal stages and config merge status

[Brett Holman]

Enhancement

A common theme for external consumers of cloud-init is a requirement for the ability to know the following information:

1. whether the configuration has been merged yet
2. whether a module will run at any point in cloud-init's boot
3. whether a specific module has already completed

One way to implement this might be to:

# cloud-init status --format json
{
  "boot_status_code": "enabled-by-generator",
  "datasource": "nocloud",
  "detail": "DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net]",
  "errors": [],
  "extended_status": "done",
+ "completed_modules": [<list of completed modules>]
+ "planned_modules": [<list of all scheduled modules - regardless of run status>]
+ "configuration_merged": true
  "init": {
    "errors": [],
    "finished": 1706709472.5883076,

Note that first requirement is an implicit requirement of the second requirement. This is because there would be no way for a user to tell from "planned_modules: []" whether a) the the configuration is simply unavailable yet or b) the configuration is available and no modules are expected to run.

The benefit of this approach is that external projects which make use of cloud-init could transition from importing from cloud-init directly (which is not supported by cloud-init) to a more loosely coupled approach.