United Nations Publications

Question

United Nations Publications

cfoster0 opened this issue 4 years ago · comments

Languages: English, French, Spanish, Arabic, Russian, Chinese (should have translations for all of these)
Date ranges: 1946-2020
Size: 700,000 publications
Link to UN digital library.

Outstanding questions:

How many of these are downloadable through the portal?
Are all of the documents available in all languages?
What is the total corpus size (in bytes) we should expect from this?

Stella Biderman commented 4 years ago

Awesome!

Charles Foster · Answer 1 · Fri Sep 11 2020 02:09:47 GMT+0800 (China Standard Time)

Splitting out the speeches into a separate Issue #39 .

Charles Foster · Answer 2 · Tue Sep 22 2020 11:47:41 GMT+0800 (China Standard Time)

The PDF translations of a given document in the library are listed at https://digitallibrary.un.org/record/[NUMBER]/files/

I'm not entirely sure yet how they're ordered.

Charles Foster · Answer 3 · Thu Sep 24 2020 12:15:58 GMT+0800 (China Standard Time)

@StellaAthena Feel free to assign this to me.

Charles Foster · Answer 4 · Sat Oct 03 2020 20:08:13 GMT+0800 (China Standard Time)

I've completed the url-collection portion of this. There are a bit over 1.8M downloadable PDFs in the database, spread fairly evenly across the 6 official languages.

Stella Biderman · Answer 5 · Mon Nov 30 2020 09:40:15 GMT+0800 (China Standard Time)

@cfoster0 What ended up happening with this?

Charles Foster · Answer 6 · Sat Dec 12 2020 09:07:23 GMT+0800 (China Standard Time)

Nothing new. If someone is interested in downloading the docs and/or converting them to text, I'd be happy to share. Was waiting for v1 work to finish, otherwise.