PAB not syncing/integration test melts RAM
lemmon-714 opened this issue · comments
Summary
Following the instructions here: https://github.com/input-output-hk/plutus-apps/tree/main/plutus-pab/test-node
...yields two unexpected behaviours, depending on how you look at it: Either PAB does not display expected behaviour (finish syncing*), or integration test crashes it.
*Hunch: PAB does not need to sync, it just fails at displaying things, because something is not connected properly.
Steps to reproduce the behavior
Follow https://github.com/input-output-hk/plutus-apps/tree/main/plutus-pab/test-node. I am also restarting all the steps often enough, and nothing else is happening on that machine.
Actual Result
A) At step 8: PAB is not syncing; meaning: Instead of
...
Current block: 327295. Current slot: 41749136.
Current block: 327295. Current slot: 41749150.
it is stuck at
Starting PAB backend server on port 9080
for hours without change.
B) At step 9: Running the integration test without waiting for the PAB to sync increases RAM-usage until it crashes (15GB RAM + 64GB swap), without printing "Tx confirmed. Integration test complete."
Expected Result
PAB printing "Tx confirmed. Integration test complete." without crashing.
Describe the approach you would take to fix this
No response
System info
- Ubuntu 20.04.3 LTS
- almost fresh instance; not much else happened before on that machine
- chain-index is synced
- node is running
- wallet-backend is running
One thought I had is that my solution here did not actually fix the issue, just the error message, and this some kind of resulting misbehaviour. Although it does appear that 'msc' has been just renamed to 'psc'.
Could you please try to synchronize with very small history by setting --rollback-history 1 option?
Could you please try to synchronize with very small history by setting
--rollback-history 1option?
Thank you, I will. Which part are you referring to, though? Node, chain-index, wallet-backend or PAB?
That's PAB's CLI flag.
Hi,
Update from my side.
- Yesterday - before your last message - I restarted everything (except the integration test) and just let it run, in the hopes that was just being impatient. This morning however I found that PAB still kills itself without any message exceeding
Starting PAB backend server on port 9080(and a 'killed' of course). - Then tried your advice ~3.5h ago, after everything else was fully synced. Still stuck at
[pab:Info:11] [2022-01-26 09:43:13.76 UTC] Restoring PAB state ...
[pab:Info:11] [2022-01-26 09:43:13.76 UTC] PAB state restored with 6 contract instances.
[pab:Info:11] [2022-01-26 09:43:13.76 UTC] Starting PAB backend server on port 9080
RAM usage is at around 15GB still.
A number of questions.
- Is this to be expected?
- Could the issue be the 6 restored contract instances?
- How can I kill them?
Thank you and best
Update from my side. 21h later no change (at least no crash). Turning off the instance now and delegating the task.
EDIT: I lied, I did leave the instance running.
Update: After several days it shows one of those messages, in particular Current block: 100000. Current slot: 27243687
This happened after I started the integration test, and consumes about 15g RAM + 15g swap. Also all eight cores are under heavy load.