NIH AIM:4 YR:2 TASK:1B | 2.4.1B | (started yr1) Resolve OAI-PMH harvesting issues

[Mike Reekie (iq.harvard.edu)]

Continuation of: #10

[C. Boyd]

Updated title from NIH AIM:4 YR:2 TASK:1A | 2.4.1A | (started yr1) Resolve OAI-PMH harvesting issues to NIH AIM:4 YR:2 TASK:1B | 2.4.1B | (started yr1) Resolve OAI-PMH harvesting issues

[C. Boyd]

The following issues identified in 4 | 1.4.1 | Resolve OAI-PMH harvesting issues | 5 #10 remain open at this time. Issues in bold are in NIH Backlog Queue:

• Refactor the OAI code, Step 2 #8842
• Revisit/reimplement the concept of a "Harvested file" dataverse#8629
• Harvest: Set export can temporarily be adversely affected by full (clean) reindex. #3437
• Harvest: Do not list a set as available in ListSets until it has been successfully exported at least once. #3322
• Feature Request/Idea: Sanitize languages controlled vocabulary values #8243
• Harvesting: OAI sets are not updated when datasets are deleted. #8005
• Remaining mapping problems when harvesting from a repository using ISO 639-3 language codes #8578

[Philip Durbin]

Perhaps we could fix this one too:

• IQSS/dataverse#9275

[C. Boyd]

@pdurbin Happy to add to bottom of prioritized list, here, but wary of taking on too much given that this task was slated for completion last year and has carried over. See original curated list here, for background.

[Julian Gautier]

Hey @cmbz, here are those GitHub issues that I was referring to and you asked me to list here when we were Slacking with @landreev today. And below each issue I added the related email conversations with admins of other repositories.

1. IQSS/dataverse.harvard.edu#92

• https://help.hmdc.harvard.edu/Ticket/Display.html?id=324230
• https://help.hmdc.harvard.edu/Ticket/Display.html?id=329544
• https://help.hmdc.harvard.edu/Ticket/Display.html?id=349681

2. IQSS/dataverse.harvard.edu#142

• https://help.hmdc.harvard.edu/Ticket/Display.html?id=314258

3. IQSS/dataverse.harvard.edu#153

• https://help.hmdc.harvard.edu/Ticket/Display.html?id=314258

4. IQSS/dataverse.harvard.edu#172

• https://help.hmdc.harvard.edu/Ticket/Display.html?id=322153

5. IQSS/dataverse#7624

• https://help.hmdc.harvard.edu/Ticket/Display.html?id=287243

6. IQSS/dataverse.harvard.edu#204

• https://help.hmdc.harvard.edu/Ticket/Display.html?id=330637

I think the second GitHub issue, IQSS/dataverse.harvard.edu#142, is a duplicate and more specific version of the third GitHub issue, IQSS/dataverse.harvard.edu#153, so maybe the second one can be closed?

[Dimitri Szabo]

This would also be important for harvesting (blocks the most datasets for us)

• IQSS/dataverse#9992

[C. Boyd]

2024/01/29

• I have moved all the remaining, open harvesting issues mentioned here to the following epic for tracking: #171
• Please follow the new epic for updates about the individual issues.