Issue : Inconsistent Environment Variable Usage for Kafka Credentials in Flink Jobs
PrinceSajjadHussain opened this issue · comments
The aggregation_job.py and start_job.py files use slightly different methods to pass kafka credentials to the flink connector. start_job.py creates the sasl_config variable, which is formatted in a manner that is different from the approach taken by aggregation_job.py. The formatting of these values could break if the env vars contain special characters, or if the underlying flink library has a quirk. This may lead to connection issues with Kafka. The configuration should be consistent across files to reduce the risk of errors and improve maintainability.