General overview of the Wikidata universe for newcomers to get more familiar and self-reliant in this (sometimes) confusing ecosystem. Collected and curated by KB, national library of the Netherlands.
This page is a textual summary of the (Dutch language) course Guide to Wikidata for employeees of KB, national library of the Netherlands on 6th June 2023. The (rather long) full slidedeck for this course is available on Wikimedia Commons and Zenodo as PDFs.
Contact
This page is maintained by Olaf Janssen, Wikimedia coordinator of KB. See his Wikidata user page and expert page on kb.nl for contact details.
Reuse and licensing
This overview can be reused freely and openly, it is available under the CC-BY 4.0 license, so attribution is required. Use something like
Wikidata general overview, Olaf Janssen & KB national library of the Netherlands, https://github.com/KBNLwikimedia/Wikidata-General-Overview/
See also
For more in-depth insights into how Wikidata is used specifically by/in/for the services of KB, national library of the Netherlands, see
- the (Dutch language) slide deck In-depth: Wikidata and KB of 14 November 2023, or
- the equivalent textual summary in English available from Wikidata & KB national library of the Netherlands, an overview here on Github.
Latest updates
Latest update: 5 January 2024
- To provide a broad overview of the Wikidata landscape, including both the technical and non-technical (social, community) aspects.
- To foster and stimulate more self-reliance in exploring the Wikidata universe
- 1) What is Wikidata?
- 2) The principles of Wikidata
- 3) How things are described in Wikidata
- 4) How to discover Qs and Ps in Wikidata
- 5) How to request data from Wikidata
- 6) How to add data to Wikidata
- 7) Who creates Wikidata?
- 8) Wikidata community Tools & Projects
- 9) How to stay informed
- 10) Where to find help (passively and actively)
- 11) How to proceed?
Table of contents generated with markdown-toc
- Various answers: A collection of structured data - A linked open database - Something to do with the sematic web? - Data for people and machines - A sister/brother of Wikipedia - Data for everyone - Public counterpart to commercial big tech data collections - A hub for 1000s of other databases - A free public utility for LOD - A worldwide enthusiastic community - The coolest LOD project ever! - Linked data, but understandable - LOD, but without the complexity of LOD - A training tool to develop yourself in the field of (linked open) data - An (academic) research object
- Prompts for ChatGTP, 2023-05-02: "What can you tell me about Wikidata?" and "Can you give me 7 perspectives on Wikidata?"
- Random Wikidata items. Prompts for ChatGTP, 02-05-2023: "Can you give me 50 random 'obscure' things described in Wikidata"
- In addition to being a broad database, Wikidata can also be very specific, e.g. 887 Van Gogh paintings (d.d. 02-05-2023)
- Wikidata contains structured descriptions of 108M things, starting from October 2012 (source, per 4 Jan 2024)
- Things included in Wikidata: Scientific articles - People - Animals and plants - Events - Countries, provinces, municipalities, cities, villages - Streets, roads, squares - Buildings - Vehicles - Companies and institutions - Works of art (film, music, paintings, etc.) - Chemical substances - Astromic objects - Genes - etc. etc.
- Wikidata items with geo locations, June 2023. Addshore, CC0, via Wikimedia Commons
- Scientific articles on Wikidata:
- From Freebase to Wikidata: The Great Migration, Thomas Pellissier Tanon, Denny Vrandecic, Sebastian Schaffert, Thomas Steiner and Lydia Pintscher
- Wikidata from a Research Perspective - A Systematic Mapping Study of Wikidata, Mariam Farda-Sarbas and Claudia Müller-Birn
- Queer Identities, Normative Databases: Challenges to Capturing Queerness On Wikidata, Katy Weathington and Jed R. Brubaker
- Investigating the potential of the semantic web for education: Exploring Wikidata as a learning platform, Shani Evenstein Sigalov and Rafi Nachmias
- What we talk about when we talk about Wikidata quality: a literature survey, Alessandro Piscopo and Elena Simperl
- Widaug. Data augmentation for named entity recognition using Wikidata, Pablo Calleja, Alberto Sanchez and Oscar Corcho
- Improving Candidate Retrieval with Entity Profile Generation for Wikidata Entity Linking, Tuan Lai, Heng Ji and ChengXiang Zhai
- Wikidata: A platform for data integration and dissemination for the life sciences and beyond, Elvira Mitraka, Andra Waagmeester, Sebastian Burgstaller-Muehlbacher, Lynn M. Schriml, Andrew I. Su and Benjamin M. Good
- Mathematics in Wikidata?, Philipp Scharpf, Moritz Schubotz and Bela Gipp
- More articles on Wikidata: Wikidata:Bibliography of Wikidata and Google Scholar
See Wikidata:Introduction and Wikidata:Philosophy
- Structured descriptions of things
- Central storage (vs distributed), no data silos
- Multilingual (200+ languages): Description of the Eiffel Tower, check the Dutch, English, Portuguese and Japanese interfaces.
- Linked data
- Things, not strings - No plain text, but clickable links
- Interconnected: Eiffel Tower and Gustave Eiffel
- Connected to external databases (external IDs): https://www.skyscrapercenter.com/building/wd/9410
- Open & free
- Free, no trackers, no ads
- No copyright or database rights, CC0 license
- Everyone can reuse data: query, share, copy, edit, download, sell, etc.
- Everyone may contribute/edit data, add, improve, delete, merge, etc. --> Community, see below
- Community
- International, 24K editors
- Under the banner of the Wikimedia Foundation --> sister project of Wikipedia, Commons etc.
- More in section 7: Who creates Wikidata?
- For humans and machines
- Human readable, human writable --> Data available via GUIs in HTML, https://www.wikidata.org/wiki/Q1526131 (KB)
- Machine readable, machine writable --> Data available via APIs in JSON, XML/RDF, CSV etc.,
- Or, formulated slightly differently: a free public utility for data - One stop shop for Linked open data - LOD, but understandable
Overview: Help:About data
- "Mount Everest is the highest point in the world" : Earth (Q2) (item) --> highest point (P610) (property) --> Mount Everest (Q513) (value)
- Triple: Earth (Q2) --> highest point (P610) --> Mount Everest (Q513)
- KB on Wikidata: Concept URI: http://www.wikidata.org/entity/Q1526131 (http: and /entity/), redirecting to https://www.wikidata.org/wiki/Q1526131 (https: and /wiki/)
- Unique identifier (Qxxxxxx)
- Multilingual fingerprint: Label, Description, Aliases (Also known as)
- Statements
- Qualifiers
- Source references
- External identifiers: descriptions of KB in external databases
- In summary: Simplified Wikidata data model
- Version history of Koninklijke Bibliotheek (Q1526131): https://www.wikidata.org/w/index.php?title=Q1526131&action=history --> Every mutation is stored! And can be rolled back. Everything is public, for ever!
- Further reading & help: How does Wikidata work? + Wikidata:Glossary
- Wikidata intro videos on Youtube: Searches for Introduction Wikidata, Wikidata 101 and Wikidata help
- Wikidata contains 103M Qitems + 11K Properties (dated 25-05-2023)
- Searching for Qs en Ps: Special:Search + search results for 'konink' (Ps and Qs)
- Browsing all Items (Qs): Special:AllPages (but not very useful for 103M items)
- Browsing all Properties (Ps): List of all Properties + All pages (Property namespace) + List of properties/all + Properties by theme (Anatomy, Ancient world, Astronomical objects etc.) + Properties summary table
- Search properties by keyword
- Wikidata as hub of 8.000+ external databases (select 'External identifiers')
Overview: Wikidata:Data access
- https://www.wikidata.org/wiki/Wikidata:Data_access#Search - based on ElasticSearch
- Special:Search - Search results for 'koninklijke bibliotheek' (Qs and Ps)
- Linked Data Interface (URI): provides access to individual Q-entities via URI: http://www.wikidata.org/entity/Q???
- Concept URI = the reference to the 'thing' in the real world. For instance Koninklijke Bibliotheek (Q1526131): http://www.wikidata.org/entity/Q1526131 (note 'http' and 'entity' instead of 'https' and 'wiki')
- Content negotiation: https://wikidata.org/wiki/Special:EntityData/Q1526131. When accessing a resource in the Special:EntityData namespace, the special page applies content negotiation to determine the output format.
- Result in web browser: the reference to the description of the KB: https://www.wikidata.org/wiki/Q1526131 (note the 'https' and 'wiki')
- See https://www.wikidata.org/wiki/Wikidata:Data_access#Details_2
- Request full Wikidata items in various output formats directly from the Qnumber via a Special:EntityData URL.
- The ouput can be obtained in seven different formats: HTML, JSON, JSON-LD, RDF, NT, TTL or N3 and PHP
- If you don't want to depend on content negotiation (e.g. view non-HTML content in a web browser), you can actively request alternative formats by appendig a format suffix to the URL, eg. to retrieve JSON: https://www.wikidata.org/wiki/Special:EntityData/Q1526131.json.
- Other available formats are JSON-LD, RDF, NT, TTL or N3 and PHP.
- Equivalant URLs for these requests use the format argument, e.g. : Special:EntityData?id=Q1526131&format=rdf.
- Regular Wikidata interface (KB): http://www.wikidata.org/entity/Q1526131
- Reasonator Wikidata GUI: https://reasonator.toolforge.org/?q=Q1526131
- SQID Wikidata GUI: https://sqid.toolforge.org/#/view?id=Q1526131
- MediaWiki Action API - the common API for all Wikimedia projects (Wikidata, Wikipedia, Wikimedia Commons etc.)
- Base URL/API endpoint: https://wikidata.org/w/api.php
- Examples of API calls (bottom of the page)
- Examples KB (Q1526131):
- Wikidata REST API - a specialized API for Wikidata
- Advantage: Cleaner, flatter structure in response data
- Base URL/API endpoint: https://www.wikidata.org/w/rest.php/wikibase/v0/
- Documentation (OpenAPI Swagger) --> GET to request data
- Examples KB (Q1526131):
- Entire item about KB, as JSON
- Lat-long of KB, as JSON
- Wikidata:Database download
- Weekly Wikidata full dumps
- WDumper - create custom Wikidata RDF dumps
Wikidata principles = Central data storage + all kinds of topics + everything connected --> Ask anything with SPARQL
- SPARQL = Language to search in Wikidata (and other LOD databases)
- SPARQL query examples:
- Images of domestic cats
- Cases named after French presidents, outside France, taken form this article
- Causes of Death of US Presidents
- Dutch nominated for Nobel Prize, but never won. See also The Man Who Was Nominated for the Nobel Prize 84 Times, But Never Won
- Paintings with alliterative titles
- 10 largest cities with female mayors
- Birthday calendar of women who studied at Oxford
- Birthplaces of deceased economists, by time period
- More SPARQL examples
- Want to get started with SPARQL in Wikidata yourself?
- Create new Qitem + Help: Creating a new item + Wikidata:Tours, shows how Wikidata works and teach you how to add data.
- Help: Adding to an existing item page
- MediaWiki Action API: wbcreateclaim, wbsetaliases, wbsetclaim, wbsetclaimvalue, wbsetdescription, wbsetlabel, wbsetqualifier, wbsetreference - and further methods to add/modify data.
- Wikidata REST API : REST API endpoint + REST API testbench (OpenAPI Swagger documentation --> PUT, POST and PATCH to add/modify data)
- OpenRefine: "From Excel sheet to Wikidata"
- https://www.openrefine.org
- See the dedicated KB OpenRefine-Wikidata Introduction Workshop, a practical 90 minutes workshop (in Dutch) to learn how to work with OpenRefine and Wikidata at a basic level.
- QuickStatements (QS): "From (formatted) .txt to Wikidata"
- QuickStatements tool + QuickStatements help
- Photo of ±1% of all people globally working on Wikidata, Pierre-Selim Huard, CC BY 4.0 via Wikimedia Commons.
- "What can you tell me about the Wikidata community?" - Prompt for ChatGTP, 02-05-2023
- Wikidata = Sister project/community of Wikipedia, Wikimedia Commons etc.: Common core values are: Open/free knowledge, Collaboration, Knowledge sharing for the benefit of all
- Wikidata in other Wikimedia projects
- Wikidata Statistics: 103M items (since Oct 2012) - 1.9B edits - 24K editors (dated 30-05-2023)
- Roles within the Wikidata community can include: Data editor - Data modeler - Writer of documentation/howtos - Bug reporter - Content domain expert - Software & tools developer - SPARQL expert - Translator - Ambassador - Event organizer/supporter - Writer of project proposals/funding - Conflict mediator - Liaison between community and GLAMs - Communication/outreach - Teacher/trainer/speaker - And many more.
- Wikidata community events + Events archive + Media files related to Wikidata events. See also Category:Wikidata Events
- WikidataCon: WikidataCon 2021 + WikidataCon 2019 + Media files related to WikidataCon.
- Wikidata workshops & trainings examples: Atelier Wikidata à Montréal - Wikidata and Wikibase Workshop 2019 - Wikidata for beginners - EuropeanaTech 2018 Wikidata workshop day
- Wikidata hackathon & editathon examples: Wiki Techstorm 2018 - Wikidata Zurich Hackathon 2019 - GLOW Edithaton
- Wikidata Birthdays: Wikidata's 10th birthday homepage - Events to celebrate Wikidata's 10th birthday worldwide - Wikidata 7th birthday presents
- Wikidata swag: T-shirts, socks, caps, pens, wallets, stickers, beer openers, key chains etc.
- What is a tool?
- Discover Wikidata tools: Wikidata:Tools + on Toolhub + in Hay's Tools Directory
- Tool examples:
- Duplicate items
- VizQuery to build SPARQL queries visually - via Wikidata:Tools/Query data
- Histropedia, Wikidata powered timelines. Eg. a timeline of Apollo missions - via Wikidata:Tools/Visualize data
- Wikidata Firefox add-ons
- Wikidata for Google Sheets
- Coolest Tool Awards + 2019 awards + more background info
- A WikiProject (on Wikidata) is a group of contributors who want to work together as a team to improve Wikidata. Editors join WikiProjects to improve specific areas of Wikidata and build the community.
- Wikidata professional training module about WikiProjects
- Wikidata projects examples: COVID-19 - Books - Tennis - Sum of all paintings, to make a LOD description for every notable painting in the world (excellent project, check ik out!!!)
- Wikidata projects overviews:
- Cultural Wikiprojects, GLAM WikiProjects and Library WikiProjects
- KB: Current Wikidata projects + Category:WikiProjects National Library of the Netherlands
- Wikidata Weekly Summary (newsletter), e.g. 15 May 2023
- Wikidata mailinglist + archives
- Wikidata project chat
- Wikidata De Kroeg (Dutch discussions)
- Wikidata @Telegram:
- Wikidata @Twitter
- Wikidata @Facebook + Wikidata Community @Facebook + Wikidata.GLAM @Facebook
- Wikidata documentation portal, the place where you can get help contributing to Wikidata.
- Wikidata Community portal
- Wikidata Professional Development Training Modules, a set of modules covering the basics of contributing to Wikidata. See the module about the Wikidata community.
- Via Telegram, Mailinglists, Project chat, Twitter, Facebook: see Section 9 above
- Live chat
- Email : info@wikidata.org
- StackExchange (technical help)
- Help by User:OlafJanssen. See also his expert page on kb.nl
- Create an account on Wikidata
- Get started with Wikidata yourself
- Check the other Wikimedia related courses of the KB
- Wikidata & KB, an overview, an overview of how Wikidata is used by/in/for both the linked open datasets (thesauri) and public domain heritage collections of the KB.
- OpenRefine-Wikidata Introduction Workshop, a practical 90 minutes workshop (in Dutch) to learn how to work with OpenRefine and Wikidata at a basic level.
- Wikibase resources, a collection of resources, overviews, links and knowlegde related to Wikibase, collected and curated by KB.
- Wikidata & SPAQRL workshop (2024, to be announced)
- Questions? Looking for support or extra explanations? Contact Olaf Janssen, see above for his details