Accumulate outputs for iterator

[Tracy Li]

Current behavior:
There is no way to accumulate the outputs for iterator over an activity.
Expected behavior:
There should be a way to collect all output and let the user use in downstream mapping
What is the motivation / use case for changing the behavior?
New Feature
Additional information you deem important (e.g. I need this tomorrow):

[Tracy Li]

1. Add another task setting field to turn on accumulate on iterator or feature dowhile.

           "settings": {
              "iterate": 6,
              "accumulate": true
            },

2. I came up with a solution where we can use $accumulate[activityName].outputs to get all accumulate data for an activity, where outputs is a key and value will be all iterate outputs.

@mellistibco and @flogo-oss Any thoughts?

Here is an example flogo.json where it iterator RestInvoke and print all accumulated data in downstream log activity.

{
  "imports": [
    "github.com/project-flogo/flow",
    "github.com/project-flogo/contrib/activity/rest",
    "github.com/project-flogo/contrib/activity/log",
    "github.com/project-flogo/contrib/activity/actreturn",
    "github.com/project-flogo/contrib/trigger/timer",
    "github.com/project-flogo/contrib/function/coerce"
  ],
  "name": "iterator",
  "description": " ",
  "version": "1.1.0",
  "type": "flogo:app",
  "appModel": "1.1.1",
  "feVersion": "2.7.0",
  "triggers": [
    {
      "ref": "#timer",
      "name": "flogo-timer",
      "description": "Simple Timer trigger",
      "settings": {},
      "id": "Timer",
      "handlers": [
        {
          "description": "ddd",
          "settings": {},
          "action": {
            "ref": "github.com/project-flogo/flow",
            "settings": {
              "flowURI": "res://flow:ddd"
            }
          }
        }
      ]
    }
  ],
  "resources": [
    {
      "id": "flow:ddd",
      "data": {
        "name": "ddd",
        "description": "ddd",
        "links": [
          {
            "id": 1,
            "from": "RESTInvoke",
            "to": "Log",
            "type": "default"
          },
          {
            "id": 2,
            "from": "Log",
            "to": "Return",
            "type": "default"
          }
        ],
        "tasks": [
          {
            "id": "RESTInvoke",
            "name": "RESTInvoke",
            "description": "Invokes a REST Service",
            "settings": {
              "iterate": 6,
              "accumulate": true
            },
            "type": "iterator",
            "activity": {
              "ref": "#rest",
              "settings": {
                "method": "GET",
                "uri": "https://petstore.swagger.io/v2/pet/2",
                "headers": "",
                "proxy": "",
                "timeout": 0,
                "sslConfig": ""
              },
              "input": {
                "pathParams": "",
                "queryParams": "",
                "headers": ""
              }
            }
          },
          {
            "id": "Log",
            "name": "Log",
            "description": "Logs a message",
            "activity": {
              "ref": "#log",
              "input": {
                "message": "=coerce.toString($accumulate[\"RESTInvoke\"].outputs)",
                "addDetails": false
              }
            }
          },
          {
            "id": "Return",
            "name": "Return",
            "description": "Simple Return Activity",
            "activity": {
              "ref": "#actreturn"
            }
          }
        ],
        "metadata": {
          "input": [],
          "output": [],
          "fe_metadata": {}
        }
      }
    }
  ]
}

[Tracy Li]

We made the final decision with @mellistibco @fm-tibco and @vijaynalawade.

To use $activity[RESTInvoke], it returns all accumulated outputs and user can access each by $activity[RESTInvoke][0].data or using array mapping on $activity[RESTInvoke]

Note: Once enabled accumulate for an activity, the activity's output will not available for the user. User have to use $activity[RESTInvoke][0].data