Will published datasets be somehow updated? Thanks!

[Yue XU]

Hi there,

Thanks for all the data you've shared and the maintaining effort behind :)

Just a quick check: may I know if the published datasets could be rolling updated? For example, will new files append to or replace the existing ones under the published folders (e.g. sources)?

Thanks and wish your team all the best.
Nino

[Niranj Chandrasekaran]

Hi Nino,

Thank you for your interest in our dataset!

For the most part, files that are currently publicly available will not be replaced, unless there are errors that need to be corrected. But we will continue to add new folders and files as and when they are ready to be uploaded.

[Yue XU]

Hi Nino,

Thank you for your interest in our dataset!

For the most part, files that are currently publicly available will not be replaced, unless there are errors that need to be corrected. But we will continue to add new folders and files as and when they are ready to be uploaded.

Hi Niranj,

Thank you for the reply.

Do you mind further specifying how the new folders will be organized? Is there any possibility that the new files are added to the now existing paths, e.g. sources/batches?

Thanks and best regards,
Nino

[Niranj Chandrasekaran]

We will be adding another source of data soon. Other than that, we may have additional embeddings, additional versions of processed embeddings etc. These will likely alongside the existing embeddings in the same folders. But I should stress that things are still in flux, so reorganization of the folders is a possibility.

[Yue XU]

We will be adding another source of data soon. Other than that, we may have additional embeddings, additional versions of processed embeddings etc. These will likely alongside the existing embeddings in the same folders. But I should stress that things are still in flux, so reorganization of the folders is a possibility.

Got it. Thanks!