[Incremental Ingestion Backend] Incorrect query used for deletion

[Taras Mankovski]

We found that the query we used to identify records to delete was incorrect. After ingestion, we were getting a query error.

We fixed the problem in the closed source version that we need to bring into the open source.

You need to change CHANGE_ENTITY_NAME.

        const removed: { entity: string; ref: string }[] = await tx('final_entities')
          .select(tx.ref('final_entity').as('entity'), tx.ref('refresh_state.entity_ref').as('ref'))
          .join(tx.raw('refresh_state ON refresh_state.entity_id = final_entities.entity_id'))
          .whereRaw(`((final_entity::json #>> '{metadata, annotations, CHANGE_ENTITY_NAME}')) = ?`, [
            provider.getProviderName(),
          ])
          .whereNotIn(
            'entity_ref',
            tx('ingestion.ingestion_marks')
              .join(
                'ingestion.ingestion_mark_entities',
                'ingestion.ingestion_marks.id',
                'ingestion.ingestion_mark_entities.ingestion_mark_id',
              )
              .select('ingestion.ingestion_mark_entities.ref')
              .where('ingestion.ingestion_marks.ingestion_id', id),
          );

[Dmitriy Lazarev]

I noticed that in closed source version was added .where('ingestion.ingestion_marks.ingestion_id', id), this line. Do we need sync closed and open source version? Or we just want to move this change with CHANGE_ENTITY_NAME?