why in tumbling_local_aggregator module, rx not equal to tx?

[space_xx]

In First pipeline case, why tumbling_local_aggregator_6 module events rx = 92eps, but tx = 32eps?
When I copy the pipeline, make nexmark source as 10000/s, tumbling_local_aggregator_6 module events rx = 9242eps, but tx = 641eps?

[Jackson Newhouse]

Hi there! Thanks for the question. What's happening is that the tumbling local aggregator does some partial aggregation before sending data onto the downstream node. As that edge is a shuffle, it will, in general, cross a network boundary, requiring serialization and deserialization. This is similar to the combiner that you see in some map-reduce systems, including Hadoop.

Hope this answers your question!

[space_xx]

Hi there! Thanks for the question. What's happening is that the tumbling local aggregator does some partial aggregation before sending data onto the downstream node. As that edge is a shuffle, it will, in general, cross a network boundary, requiring serialization and deserialization. This is similar to the combiner that you see in some map-reduce systems, including Hadoop.

Hope this answers your question!

Thank u.