Email report generation is not working
mdmohsin2005khan opened this issue · comments
I have seen 2 issues with email report generation
- email having dot(.) is considered as invalid, example: firstname.lastname@gmail.com (my office email has this pattern & when I tried generating email, I was getting invalid email in yarn logs). Tried validating this regex https://github.com/qubole/sparklens/blob/7fa57b9606abaace2ff7459028a6bf5a68fd5fa9/src/main/scala/com/qubole/sparklens/helper/EmailReportHelper.scala#L15 through https://regex101.com/ for above mentioned email and I'm getting full match as lastname@gmail.com
- I tried generating with regular email id (without having any dots in email) and even then I was not getting any email reports.
Same with me..
Hi, thanks for reporting the issues.
I have created the PR for the fix: #60
It would be great if you can confirm if the new regex covers the cases. Since many users must be facing an issue because of this, we will do a new release in the coming days. In the meantime, you can build the package from the source once this is merged: sbt clean assembly
I see another issue where I'm unable to save Sparklens JSON file to S3. It gets saved only to HDFS.
I tried giving s3a instead of just s3 and it worked for me, sparklens was writing the metrics to s3 location.
Example: --conf spark.sparklens.data.dir=s3a://bucket-name/prefixes/sparklens/