algorand / conduit

Algorand's data pipeline framework.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Pipeline the Pipeline

tzaffi opened this issue · comments

Problem

Following up from #11. With the design agreed upon, implement the design.

Solution

Follow the approach defined in the Next Steps section of the doc:

During the standup of 7/24 we decided that this makes a good choice for a future improvement. However, we should NOT attempt to accommodate any Cross Round Concurrency in the first version of the work. In other words, we default to concurrentRounds==1 for all the conduit components and don’t employ the importer fan-out/fan-in approach shown in the figure.

Dependencies

Draft PR: #128

Urgency

High