[Question] doubt in usage of polar_redo_set_buffer_oldest_lsn function during recovery
srinathv2 opened this issue · comments
i was going through the use of flushlist during recovery then i came across polar_redo_set_buffer_oldest_lsn function which puts the buffer into flushlist with valid lsn as buffer oldest lsn during recovery
can't we move polar_redo_set_buffer_oldest_lsn into markbufferdirty during recovery since we have a check RecoveryInProgress inside polar_set_buffer_fake_oldest_lsn if we are in recovery it will be true(to be precise RecoveryInProgress will be true when we don't know recovery completed we need to check the shared value of that) so we will return and use polar_redo_set_buffer_oldest_lsn
wouldn't it be clean if we have a check of recovery in markbufferdirty if we are in recovery we can use polar_redo_set_buffer_oldest_lsn else polar_set_buffer_fake_oldest_lsn instead of calling polar_redo_set_buffer_oldest_lsn outside markbufferdirty in many places like heap_xlog_insert?
Hi @srinathv2 ~ Thanks for opening this issue! 🎉
Please make sure you have provided enough information for subsequent discussion.
We will get back to you as soon as possible. ❤️
@srinathv2 Thanks for your advice, it sure is! We have also noticed this issue earlier in 2022, and has fixed it on newer PolarDB-PG 14. On PolarDB-PG 11, we will leave it now for stability.
ok 👍 thanks for your reply, my other doubt related to this is why we are not using copy buffer during recovery inside polar_redo_set_buffer_oldest_lsn?, we get hot buffers in recovery mode also, so to move the consistent lsn forward we need to copy the hot buffer, even after that buffer is used we need to adjust that buffer in flushlist by removing it and appending to tail with the latest lsn as oldest lsn of that buffer this is all done while not in recovery inside polar_set_buffer_fake_oldest_lsn
@srinathv2 The purpose of the copy buffer is to allow hot buffers to be flushed to disk, thereby advancing the consistent_lsn. Furthermore, the size of the copy buffer is limited, so we only allocate a copy buffer if that hot buffer cannot be flushed to disk, i.e., the latest lsn of the buffer is greater than the oldest apply lsn, indicating that the buffer is blocking the advancement of the consistent_lsn.
Generally, when RecoveryInProgress is true, it means that it's either a primary node during crash recovery or a hot_standby node; in this scenario, sync of hot buffers is usually not blocked. Even if the current buffer satisfies the conditions for allocating a copy buffer in polar_redo_set_buffer_oldest_lsn, by the time it is going to be flushed to disk, the oldest apply lsn may have already advanced, causing the corresponding conditions may no longer be met, rendering the previously allocated copy buffer unnecessary.
i got it 👍 , thanks for the reply, can you please let me know when are you guys planning to open source PolarDB-PG 14
i got it 👍 , thanks for the reply, can you please let me know when are you guys planning to open source PolarDB-PG 14
There is no concreate date. Please pay attention to the official announcement. 😄