Testing procedure for FMI 2.0 interface and setting/getting state support
ghorwin opened this issue · comments
Slowly the number of FMI 2.0 supporting FMUs increases and quite a few provide the feature of storing/restoring the state. I suggest adding the following test procedure to the compliance checker for Co-Simulation FMUs with FMI 2.0 support:
-
store/restore state check
- After "exitInitializationMode" ask FMU to store its state via fmi2GetFMUstateXXX (compliance checker remembers handle)
- Simulation is done for a few steps, results are logged in file 1
- tell FMU to restore its state to saved state via fmi2SetFMUstate
- Simulation is done again for same number of steps, results are compared to those stored in file 1 -> exakt match is required (since even multi-step integrators will not have a history at simulation start)
-
multiple state handling checks
- start simulation, run until say 10% of simulation time
- store handle 1
- continue simulation, run until say 20% of simulation time
- store handle 2
- continue simulation, run until say 30% of simulation time
- restore state to handle 2
- run again until 30% -> compare results -> close match is required in interval 20-30% compared to first run
- restore state to handle 1
- run again until 30% -> compare results -> close match is required in interval 20-30% compared to first run
Note about "close match" requirement:
See discussion about restoring state of error controlled, multi-step methods: When the stored state does not include full history, or when last step was not synchronized with communication interval end - the continued simulation will likely have different results; for example, the simulation could be continued with a "cold-start", i.e. method order 1, smallest time step, hereby taking completely different steps within the communication interval as in the first iteration, thus yielding (slightly) different results.
- Serialization test (only for FMUs that support this; mainly necessary to check for correct re-initialization of external resources being used by the FMU)
- run full simulation once
- re-init (completely from scratch)
- start simulation, run until say 10% of simulation time
- retrieve FMU state and store to file
- re-init (completely from scratch)
- read FMU state from file and set state in FMU
- continue simulation until end -> close match is required for results in both runs
At last, some consistency checks - may require use of valgrind or other memory leak checkers.
- (optional) memory allocation/consistency check:
Motivation: often, problems with resource leaks are not detected in small test cases. However, when an iterative master algorithm is used for a whole simulation, the get/set-state procedure may be repeated many 1000 times, and even a small mistake in memory/resource allocation will cause problems here.- procedure 1 (with short simulation interval in between) is repeated at n times, with intermittend calls to fmi2FreeFMUstateXXX
- memory footprint of FMU model should not grow uncontrolled (a growth of memory footprint would indicate a resource leak) -> alternatively check with valgrind or other leak checker