Latest update: 22nd October 2019
Prerequisites for this Module
- Wikimedia account
- Working knowledge of Wikidata
- Basic understanding of SPARQL and the Wikidata Query Service (WDQS)
- A Wikidata set of items with geo coordinates P625
If you do not have, or cannot create such a Wikidata set, you can use one of these:
- Public libraries in The Netherlands (incomplete set, per 12-8-2019)
- National heritage sites of the Netherlands (limited to 1000 items)
- Big cities in the world
- Volcanos
- Airports around the equator ( between -10 and +10 degrees latitude)
Many examples in this workshop are based on the first dataset Public libraries in The Netherlands
- Public libraries in The Netherlands: https://w.wiki/6dx
- Default map view: https://w.wiki/6eq
- Relation between query and popup
- Hiding fields from popup: https://w.wiki/6mU
- Clustered flat map: https://w.wiki/6ew
- Dutch public libraries, layered by province: https://w.wiki/6gJ - With non-custom layer names, the layer names are the labels from the Wikidata items
- Libraries layered by postal code zone: https://bit.ly/3368DPk
- Big cities, by population: https://w.wiki/AaA
- Disasters, by type: https://w.wiki/6gY
- Medieval churches, by century of construction: https://w.wiki/6ga
- Dutch municipalities, by number of public libraries: https://w.wiki/6gb
Prerequisites for this Module (in addtition to those in Module 1)
Nice to have some knowledge about
- OpenStreetMap (OSM)
- GeoJSON
For embedding maps we'll be using the Mediawiki Kartographer extension
All embedding examples below are available on https://nl.wikipedia.org/wiki/Gebruiker:OlafJanssen/KladblokMapMakingWorkshopWM2019
- Basic embedded map of The Netherlands, no data: https://nl.wikipedia.org/wiki/Gebruiker:OlafJanssen/KladblokMapMakingWorkshopWM2019#/map/0 (View source)
- One public library in Amsterdam, geo coordinates hard-coded: https://nl.wikipedia.org/wiki/Gebruiker:OlafJanssen/KladblokMapMakingWorkshopWM2019#/map/1. (View source)
- Styling of markers https://github.com/mapbox/simplestyle-spec/tree/master/1.1.0#3-client-behavior and https://www.mediawiki.org/wiki/Help:Extension:Kartographer
- Library, museum and archive in Amsterdam, geo coordinates hard-coded: https://nl.wikipedia.org/wiki/Gebruiker:OlafJanssen/KladblokMapMakingWorkshopWM2019#/map/2. (View source)
- Outline (GeoShape) of Amsterdam public library OBA Oosterdok, using Wikidata and Open Street Map: https://nl.wikipedia.org/wiki/Gebruiker:OlafJanssen/KladblokMapMakingWorkshopWM2019#/map/3. (View source)
- Wikidata item of Amsterdam Public Library OBA Oosterdok: https://www.wikidata.org/wiki/Q50413436
- OBA Oosterdok on OSM: https://www.openstreetmap.org/way/240467636#map=19/52.37605/4.90853. Notice the Wikidata=Q50413436 in the tags on the left hand side.
- Map of Dutch public libraries: https://nl.wikipedia.org/wiki/Gebruiker:OlafJanssen/KladblokMapMakingWorkshopWM2019#/map/4. (View source)
- Corresponding .map file on Wikimedia Commons: https://commons.wikimedia.org/wiki/Data:DutchPublicLibraries.map (View source)
- Help: Map Data: https://www.mediawiki.org/wiki/Help:Map_Data
- http://geojson.io
- SPARQL query to make .tsv file as input for Geojson.io: https://bit.ly/2YG48aY. Don't forget to rename the headers (replace '_' with '-')
- https://www.mediawiki.org/wiki/Help:Extension:Kartographer#Combining_multiple_data_types
- Combining GeoShape (with geomask) and .map: https://nl.wikipedia.org/wiki/Gebruiker:OlafJanssen/KladblokMapMakingWorkshopWM2019#/map/5. (View source)
- Map of public libraries in the Dutch province of North-Holland: https://commons.wikimedia.org/wiki/Data:DutchPublicLibrariesNorthHolland.map
- https://www.mediawiki.org/wiki/Help:Extension:Kartographer#GeoShapes_via_Wikidata_Query
- Map with GeoShapes based on SPARQL queries in Wikidata: https://nl.wikipedia.org/wiki/Gebruiker:OlafJanssen/KladblokMapMakingWorkshopWM2019#/map/6. (View source)
- Note: in the source code of this map all newlines need to be stripped from the SPARQL query, otherwise the map gets broken. See https://phabricator.wikimedia.org/T214984 Do I get this right, to make the following showcase map work without Lua, now there isn't a more readable way than removing all newlines from SPARQL?
- GeoShape of Libya (Q1016) on OSM: https://www.openstreetmap.org/relation/192758#map=5/22.086/20.039
- SPARQL query to generate GeoShapes of African countries: https://bit.ly/2YBxV4t
- Variables must have names ?id ?title ?description ?fill ?stroke
- In the SPARQL query only use single quotes, otherwise you’ll get a JSON parse error
- Railway lines around Klang Valley, Malaysia: https://nl.wikipedia.org/wiki/Gebruiker:OlafJanssen/KladblokMapMakingWorkshopWM2019#/map/7. (View source)
- Governors of US states with their party affiliation: https://www.mediawiki.org/wiki/Help:Extension:Kartographer#/map/4 and https://www.mediawiki.org/wiki/Help:Extension:Kartographer#GeoShapes_via_Wikidata_Query
- Examples by Simon Cobb: https://www.wikidata.org/wiki/User:Sic19#GeoShapes
Prerequisites for this Module (in addtition to those in Modules 1 and 2)
- Mediawiki account - https://www.mediawiki.org
Nice to have basic working knowledge about
- Python
- PAWS / Jupyter Notebooks: https://www.mediawiki.org/wiki/PAWS + https://realpython.com/jupyter-notebook-introduction/
And nice to have heard about
- Jupyter-Leaflet: https://github.com/jupyter-widgets/ipyleaflet + https://ipyleaflet.readthedocs.io/en/latest/
- Jupyter-widgets: https://github.com/jupyter-widgets/ipywidgets + https://ipywidgets.readthedocs.io/en/latest/
- Important PAWS pages: https://www.mediawiki.org/wiki/PAWS + https://www.mediawiki.org/wiki/Manual:Pywikibot/PAWS + https://wikitech.wikimedia.org/wiki/PAWS
- Starting your PAWS server: https://paws.wmflabs.org/paws/hub + https://paws.wmflabs.org/paws/hub/user/USERNAME
- Running your PAWS server: https://paws.wmflabs.org/paws/user/USERNAME/tree
- Your PAWS terminal(s): https://paws.wmflabs.org/paws/user/USERNAME/terminals/1 (/2, /3, /4 etc for additional terminals)
- Stopping your PAWS server: https://paws.wmflabs.org/paws/hub/home
- Your PAWS public folder (even when server is offline) https://paws-public.wmflabs.org/paws-public/User:USERNAME
- PAWS alternative, local Jupyter Notebook installation via Anaconda: https://jupyter.org/install + https://dataiseasy.com/2019/03/how-to-install-anaconda-and-jupyter-notebook
- Go to PAWS public folder of User:OlafJanssen: https://paws-public.wmflabs.org/paws-public/User:OlafJanssen
- Go to folder WikidataMapMakingWorkshop
- Save raw json file WikidataMapMakingWorkshop.ipynb to your PC: https://paws-public.wmflabs.org/paws-public/User:OlafJanssen/WikidataMapMakingWorkshop/WikidataMapMakingWorkshop.ipynb?format=raw
- Upload this raw json file to your own PAWS server
- Example notebook to play around with on your own server: https://paws.wmflabs.org/paws/user/USERNAME/notebooks/WikidataMapMakingWorkshop.ipynb
- Full instructions are given in the notebook. The notebook should be self-explanatory, so adapt, play, understand, get frustrated, learn.... Make your own interactive, layered map
- The resulting map from the default notebook looks like this:
- Exporting the map to stand-alone off-Wiki HTML file: https://ipywidgets.readthedocs.io/en/latest/embedding.html#python-interface
- The addition of
python state=dependency_state([m])keeps the html file growing too large - Python code:
python embed_minimal_html('NetherlandsPublicLibrariesHeatmap.html', views=[m], state=dependency_state([m]), title='Heat map of Dutch public libraries, Wikimania map making workshop, 18th August 2019') - Off-Wiki version of NetherlandsPublicLibrariesHeatmap.html
Feedback on this workshop is more than welcome, please let me know your suggestions, improvements, dislikes etc.
- Olaf Janssen, Wikimedia coordinator of the national library of The Netherlands - https://www.kb.nl/wikipedia
- olaf.janssen@kb.nl
- User:OlafJanssen on (Dutch) Wikipedia, Wikimedia Commons and Wikidata
- Twitter: @ookgezellig