JVM Memory Increase in Seatunnel
shrikhaparul opened this issue · comments
Search before asking
- I had searched in the issues and found no similar issues.
What happened
I am running the seatunnel cluster continuously, we are running the seatunnel on ec2 and after each run of a job for eg jdbc to jdbc the jvm size increases,....after each run the size increases continuously untill the memory in the ram is being occupied....if more space is required...it crashes the job...
I have tried to manage the jvm otptions as well....increased the heap memory size as well but no memory inprovements i have seen....
history-job-expire-minutes : also i have set it up to 5 mins....still no change
Is this a seatunnel bug or it can be resolved?
SeaTunnel Version
2.3.3
SeaTunnel Config
{
"env": {
"job.name": "TGT_NEWYELLOW_TRIPDATA3CSV",
"execution.parallelism": 2,
"job.mode": "BATCH"
},
"source": [
{
"plugin_name": "Jdbc",
"parallelism": 2,
"url": "",
"connection_check_timeout_sec": 1000,
"driver": "oracle.jdbc.OracleDriver",
"user": "",
"password": "",
"database": "XEPDB1",
"query": "select * from TEST.TGT_NEWYELLOW_TRIPDATA3CSV ",
"fetch_size": 10000
}
],
"sink": [
{
"plugin_name": "LocalFile",
"path": "/home/pshrikh/test_files/files/",
"field_delimiter": ",",
"file_format_type": "csv",
"file_name_expression": "tgt_TGT_NEWYELLOW_TRIPDATA3CSV_2024",
"custom_filename": "true",
"batchsize": 10000,
"is_enable_transaction": "false"
}
]
}
Running Command
/home/apache-seatunnel-2.3.3/bin/seatunnel.sh --config /home/scripts/seatunnel/ora_beverages_filecsv.json
Error Exception
na
Zeta or Flink or Spark Version
zeta
Java or Scala Version
java
Screenshots
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct