Make it easier to set-up the required amount of retrievers
durner opened this issue · comments
- Use default throughput per core values, that can be adapted by the user
- Use default requests per retriever values, that can be adapted by the user
- Use the network throughput information from the instance to calculate concurrency and outstanding requests
- Build pipeline to automatically get the information for the cloud vendors instances
- Integrate that into a config helper class that can be used easily, e.g.,
- config.requiredThreadCount() to give the user the number of retriever threads that need to started
- New constructor TaskSendReceiverGroup(config) to initialize the values more easily