Commons:Wiki Loves Monuments 2011/Monuments database

From Wikimedia Commons, the free media repository
Jump to: navigation, search

The presentation of the database is now located at Commons:Monuments_database

This page contains an overview of the monuments database used in Wiki Loves Monuments 2011. This database is used in (international) tools, for example to display the monuments in maps and keeping statistics.

Contents

Overview [edit]

Wikipedia's which have structured lists of monuments can be added as a source of the big monuments database. For each source a table is created in the form monuments_<country code>_(<language code>). The table contains fields based on the Wikipedia row template in the structured list. One big table (monuments_all) contains the combined result of all individual sources. For this fields are mapped from the source table to the monuments_all table.

Monuments fields [edit]

The monuments_all table contains the following fields:

  • country - Iso code of the country or region
  • lang - the language of the entry
  • id - Id of the monument
  • name - Name of the monument
  • address - Address of the monument
  • municipality - Municipality of the monument
  • lat - The latitude of the monument
  • lon - The longitude of the monument
  • image - Image of the monument
  • source - Source of this data (permalink to a Wikipedia page)
  • changed - Timestamp when this data was updated in the database
  • monument_article - wiki article about the monument, formatted as [1] (contains also section links like: 'Wiener_Wienflussbrücken#Brücken')

Other fields could be added in the future. I'm open to suggestions!

Sources overview [edit]

(removed outdated table)

To add [edit]

If you would like to have a source added, just add it to this list.

Not ready [edit]

  • Switzerland in German: "monuments_ch_(de)"
  • Switzerland in French: "monuments_ch_(fr)"
  • Wallonia in French: "monuments_be-wal_(fr)" (deleted)
  • French monuments historiques in German (de:Vorlage:Ligne de tableau MH). Fields need to be translated first.

Things to improve [edit]

  • Belarus is in project namespace (4), should be main
  • Estonia is in project namespace (4), should be main
  • Norway is in Wiki Loves Monuments 2011 project namespace (4), should be main
  • No images found in Norway
  • Poland is in namespace 102 (wikiproject?)?
  • Russia is in project namespace (4), should be main
  • No images found in Russia

Adding a new source [edit]

How to add another source to the monuments database? This can only be done by someone who has access to the "erfgoed" account at the Toolserver.

  1. You first need structured lists. This guide assumes you have those already. We need the header and row template.
  2. Open https://svn.toolserver.org/svnroot/p_erfgoed/erfgoedbot/monuments_config.py
  3. Copy a country you like and start filling out fields. It's easiest to just copy and modify the configuration of an existing country. All fields are described here.
    • project : This is always wikipedia ('wikipedia')
    • lang : The Wikipedia language code ('nl')
    • headerTemplate : The header template for the lists ('Tabelkop rijksmonumenten')
    • rowTemplate : The row template for the lists ('Tabelrij rijksmonument')
    • commonsTemplate : The template here at Commons to track images ('Rijksmonument'). You can leave this empty
    • commonsTrackerCategory : The category added by the previous template ('Rijksmonumenten with known IDs'). You can leave this empty
    • commonsCategoryBase : The base of the category tree at Commons ('Rijksmonumenten'). You can leave this empty.
    • autoGeocode : Do you want to do auto geocoding (False/True). Always start with False
    • unusedImagesPage : Page on Wikipedia where to report unused images ('Wikipedia:Wikiproject/Erfgoed/Nederlandse Erfgoed Inventarisatie/Ongebruikte foto\'s'). You can leave this empty.
    • imagesWithoutIdPage : Page with a list of images without an identifier template at Commons ('Wikipedia:Wikiproject/Erfgoed/Nederlandse Erfgoed Inventarisatie/Foto\'s zonder id'). You can leave this empty.
    • namespaces : Namespaces to work on at Wikipedia ([0]).
    • table : Name of the table to store everything in (u'monuments_nl_(nl)'). The convention is monuments_<countrycode>_(<lang>). Please keep this in line.
    • truncate : To empty out the table on update (False). You need this if you don't have strong identifiers.
    • primkey : The primary key in the table ('objrijksnr'). This should be the identifier.
    • fields : All the fields the bot can find
      • source : The name of the field in the row template ('objrijksnr')
      • dest : The destination field in the sql table (u'objrijksnr'). Please keep this ascii to prevent problems. If you leave it empty this information will just be dropped.
      • conv : Do we want to do any conversions? Deprecated leave empty please.
    • Save the file
  4. Run python monument_tables.py, that will create the sql file for local monuments table based on config in monuments_config.py file
  5. Run mysql < monuments_xx_(yy).sql (where xx is your countrycode and yy the language, this should match the configuration) to create your table. You have to do this as user erfgoed
  6. Edit fill_table_monuments_all.sql. This is the mysql code to put everything in one big table.
  7. Copy a section and put it in the right location. You get the relevant fields from monuments_xx_(yy).sql and match them to the shared fields (id, name, address, municipality, lat, lon, image, source, changed & monument_article)
  8. Run python update_monuments.py -lang:yy -countrycode:xx -fullupdate as erfgoed and see if nothing strange happens
  9. Run sql p_erfgoed_p <fill_table_monuments_all.sql to put everything in the big table
  10. If everything worked, commit to svn. Don't forget to provide an edit summary!
  11. Update Commons:Wiki Loves Monuments 2011/Monuments database#Sources overview to reflect your changes.

Using the database [edit]

Now we have all this data at the Toolserver, but it's not much fun unless you start doing something with it! The database can be accessed at the toolserver. Connect to the database "p_erfgoed_p" at "sql.toolserver.org". If you don't have a Toolserver account or you want an offline copy, you can download a full dump of the database.

Statistics [edit]

country lang total name address municipality coordinates image commonscat adm0 adm1 adm2 adm3 adm4 source pages
ad ca 111 111 (100.0%) 109 (98.2%) 111 (100.0%) 111 (100.0%) 111 (100.0%) 102 (91.89%) 111 (100.0%) 111 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 1
aq en 158 158 (100.0%) 158 (100.0%) 158 (100.0%) 155 (98.1%) 15 (9.49%) 7 (4.43%) 158 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 1
ar es 1807 1807 (100.0%) 1512 (83.67%) 1785 (98.78%) 1162 (64.31%) 822 (45.49%) 0 (0.0%) 1807 (100.0%) 1807 (100.0%) 0 (0.0%) 1785 (98.78%) 1771 (98.01%) 27
at de 36953 36953 (100.0%) 30561 (82.7%) 36839 (99.69%) 36429 (98.58%) 30923 (83.68%) 9729 (26.33%) 36953 (100.0%) 36953 (100.0%) 36839 (99.69%) 36839 (99.69%) 36953 (100.0%) 2405
be-bru nl 1167 1166 (99.91%) 1166 (99.91%) 1164 (99.74%) 2 (0.17%) 468 (40.1%) 0 (0.0%) 1167 (100.0%) 1167 (100.0%) 1164 (99.74%) 0 (0.0%) 0 (0.0%) 19
be-vlg en 1854 1854 (100.0%) 1854 (100.0%) 0 (0.0%) 1817 (98.0%) 540 (29.13%) 0 (0.0%) 1854 (100.0%) 1854 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 20
be-vlg fr 70747 70747 (100.0%) 70747 (100.0%) 70747 (100.0%) 65632 (92.77%) 3793 (5.36%) 0 (0.0%) 70747 (100.0%) 70747 (100.0%) 0 (0.0%) 70747 (100.0%) 0 (0.0%) 376
be-vlg nl 72231 72231 (100.0%) 72231 (100.0%) 72231 (100.0%) 65728 (91.0%) 4630 (6.41%) 0 (0.0%) 72231 (100.0%) 72231 (100.0%) 71579 (99.1%) 72231 (100.0%) 0 (0.0%) 397
be-wal en 3948 3948 (100.0%) 1981 (50.18%) 3777 (95.67%) 3948 (100.0%) 1309 (33.16%) 0 (0.0%) 3948 (100.0%) 3948 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 237
be-wal fr 4204 4048 (96.29%) 2189 (52.07%) 4204 (100.0%) 4204 (100.0%) 1403 (33.37%) 0 (0.0%) 4204 (100.0%) 4204 (100.0%) 0 (0.0%) 4204 (100.0%) 0 (0.0%) 264
be-wal nl 4210 4208 (99.95%) 2085 (49.52%) 4202 (99.81%) 4210 (100.0%) 1445 (34.32%) 0 (0.0%) 4210 (100.0%) 4210 (100.0%) 4198 (99.71%) 4202 (99.81%) 0 (0.0%) 262
by be-x-old 4650 4650 (100.0%) 3150 (67.74%) 4621 (99.38%) 739 (15.89%) 595 (12.8%) 0 (0.0%) 4650 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 125
ca en 8303 8303 (100.0%) 8032 (96.74%) 2366 (28.5%) 8201 (98.77%) 1837 (22.12%) 0 (0.0%) 8303 (100.0%) 8303 (100.0%) 2366 (28.5%) 0 (0.0%) 0 (0.0%) 12
ca fr 12450 12450 (100.0%) 10319 (82.88%) 12449 (99.99%) 12387 (99.49%) 2419 (19.43%) 0 (0.0%) 12450 (100.0%) 12450 (100.0%) 12449 (99.99%) 0 (0.0%) 0 (0.0%) 147
ch de 2919 2919 (100.0%) 2292 (78.52%) 2919 (100.0%) 2916 (99.9%) 2457 (84.17%) 0 (0.0%) 2919 (100.0%) 2919 (100.0%) 0 (0.0%) 2919 (100.0%) 0 (0.0%) 26
ch en 2942 2942 (100.0%) 2298 (78.11%) 2942 (100.0%) 2930 (99.59%) 2468 (83.89%) 0 (0.0%) 2942 (100.0%) 2942 (100.0%) 0 (0.0%) 2942 (100.0%) 0 (0.0%) 27
ch fr 1424 1424 (100.0%) 1156 (81.18%) 1424 (100.0%) 1424 (100.0%) 918 (64.47%) 0 (0.0%) 1424 (100.0%) 1424 (100.0%) 0 (0.0%) 1424 (100.0%) 0 (0.0%) 15
ch it 331 331 (100.0%) 134 (40.48%) 331 (100.0%) 331 (100.0%) 198 (59.82%) 0 (0.0%) 331 (100.0%) 331 (100.0%) 0 (0.0%) 331 (100.0%) 0 (0.0%) 2
cl es 803 803 (100.0%) 735 (91.53%) 803 (100.0%) 803 (100.0%) 550 (68.49%) 0 (0.0%) 803 (100.0%) 803 (100.0%) 803 (100.0%) 0 (0.0%) 0 (0.0%) 15
co es 1023 1023 (100.0%) 675 (65.98%) 1023 (100.0%) 105 (10.26%) 402 (39.3%) 0 (0.0%) 1023 (100.0%) 1023 (100.0%) 1023 (100.0%) 0 (0.0%) 0 (0.0%) 27
cz cs 40076 40076 (100.0%) 39988 (99.78%) 40066 (99.98%) 10633 (26.53%) 15246 (38.04%) 3344 (8.34%) 40076 (100.0%) 40076 (100.0%) 40073 (99.99%) 40066 (99.98%) 0 (0.0%) 450
de-by de 104099 98508 (94.63%) 103225 (99.16%) 96321 (92.53%) 74714 (71.77%) 26162 (25.13%) 5290 (5.08%) 104099 (100.0%) 104099 (100.0%) 0 (0.0%) 0 (0.0%) 96321 (92.53%) 2127
de-he de 11311 5096 (45.05%) 11182 (98.86%) 11311 (100.0%) 2685 (23.74%) 4021 (35.55%) 1175 (10.39%) 11311 (100.0%) 11311 (100.0%) 0 (0.0%) 0 (0.0%) 11311 (100.0%) 153
de-nrw de 58144 53412 (91.86%) 56589 (97.33%) 38087 (65.5%) 11041 (18.99%) 20314 (34.94%) 0 (0.0%) 58144 (100.0%) 58144 (100.0%) 58144 (100.0%) 0 (0.0%) 0 (0.0%) 640
de-nrw-bm de 249 164 (65.86%) 193 (77.51%) 249 (100.0%) 224 (89.96%) 167 (67.07%) 89 (35.74%) 249 (100.0%) 249 (100.0%) 249 (100.0%) 249 (100.0%) 249 (100.0%) 15
de-nrw-k de 8475 8475 (100.0%) 8475 (100.0%) 8464 (99.87%) 3203 (37.79%) 4493 (53.01%) 350 (4.13%) 8475 (100.0%) 8475 (100.0%) 8475 (100.0%) 8475 (100.0%) 8464 (99.87%) 86
dk-bygning da 7057 7057 (100.0%) 7057 (100.0%) 7057 (100.0%) 7041 (99.77%) 451 (6.39%) 0 (0.0%) 7057 (100.0%) 7056 (99.99%) 7056 (99.99%) 0 (0.0%) 0 (0.0%) 98
dk-fortids da 31320 31320 (100.0%) 0 (0.0%) 0 (0.0%) 31320 (100.0%) 71 (0.23%) 0 (0.0%) 31320 (100.0%) 29826 (95.23%) 29826 (95.23%) 0 (0.0%) 0 (0.0%) 100
ee et 24619 24619 (100.0%) 24616 (99.99%) 24582 (99.85%) 11216 (45.56%) 3327 (13.51%) 164 (0.67%) 24619 (100.0%) 24582 (99.85%) 24582 (99.85%) 0 (0.0%) 0 (0.0%) 227
es ca 35022 35022 (100.0%) 29427 (84.02%) 35022 (100.0%) 29401 (83.95%) 11590 (33.09%) 7362 (21.02%) 35022 (100.0%) 35022 (100.0%) 34962 (99.83%) 35022 (100.0%) 0 (0.0%) 571
es es 14080 14068 (99.91%) 3790 (26.92%) 14079 (99.99%) 7676 (54.52%) 6055 (43.0%) 189 (1.34%) 14080 (100.0%) 12740 (90.48%) 12740 (90.48%) 14079 (99.99%) 0 (0.0%) 136
es gl 494 494 (100.0%) 267 (54.05%) 494 (100.0%) 203 (41.09%) 208 (42.11%) 0 (0.0%) 494 (100.0%) 494 (100.0%) 494 (100.0%) 494 (100.0%) 0 (0.0%) 4
fr ca 4595 4595 (100.0%) 1680 (36.56%) 4595 (100.0%) 1756 (38.22%) 1776 (38.65%) 1279 (27.83%) 4595 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 4595 (100.0%) 40
fr fr 44025 44025 (100.0%) 22359 (50.79%) 43696 (99.25%) 35402 (80.41%) 24195 (54.96%) 0 (0.0%) 44025 (100.0%) 42217 (95.89%) 42217 (95.89%) 43696 (99.25%) 0 (0.0%) 415
gh en 52 52 (100.0%) 0 (0.0%) 49 (94.23%) 29 (55.77%) 20 (38.46%) 0 (0.0%) 52 (100.0%) 52 (100.0%) 49 (94.23%) 0 (0.0%) 0 (0.0%) 1
ie en 619 619 (100.0%) 0 (0.0%) 619 (100.0%) 176 (28.43%) 225 (36.35%) 0 (0.0%) 619 (100.0%) 619 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 27
il he 774 774 (100.0%) 646 (83.46%) 645 (83.33%) 668 (86.3%) 485 (62.66%) 470 (60.72%) 774 (100.0%) 766 (98.97%) 645 (83.33%) 0 (0.0%) 0 (0.0%) 6
in en 7478 7478 (100.0%) 128 (1.71%) 6350 (84.92%) 204 (2.73%) 1355 (18.12%) 0 (0.0%) 7478 (100.0%) 7478 (100.0%) 421 (5.63%) 0 (0.0%) 0 (0.0%) 74
it it 963 963 (100.0%) 845 (87.75%) 963 (100.0%) 882 (91.59%) 0 (0.0%) 0 (0.0%) 963 (100.0%) 963 (100.0%) 963 (100.0%) 963 (100.0%) 0 (0.0%) 29
it-88 ca 20 20 (100.0%) 20 (100.0%) 20 (100.0%) 20 (100.0%) 6 (30.0%) 4 (20.0%) 20 (100.0%) 20 (100.0%) 0 (0.0%) 20 (100.0%) 0 (0.0%) 1
it-bz de 4815 4815 (100.0%) 1401 (29.1%) 4815 (100.0%) 4747 (98.59%) 1780 (36.97%) 1786 (37.09%) 4815 (100.0%) 4815 (100.0%) 4815 (100.0%) 4815 (100.0%) 0 (0.0%) 116
ke en 208 208 (100.0%) 5 (2.4%) 206 (99.04%) 16 (7.69%) 25 (12.02%) 0 (0.0%) 208 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 1
lu lb 43 43 (100.0%) 38 (88.37%) 23 (53.49%) 31 (72.09%) 32 (74.42%) 0 (0.0%) 43 (100.0%) 43 (100.0%) 0 (0.0%) 23 (53.49%) 0 (0.0%) 5
mt de 115 113 (98.26%) 100 (86.96%) 115 (100.0%) 60 (52.17%) 28 (24.35%) 0 (0.0%) 115 (100.0%) 115 (100.0%) 115 (100.0%) 0 (0.0%) 0 (0.0%) 1
mx es 9950 9949 (99.99%) 8981 (90.26%) 9950 (100.0%) 9905 (99.55%) 654 (6.57%) 0 (0.0%) 9950 (100.0%) 9950 (100.0%) 9950 (100.0%) 9899 (99.49%) 0 (0.0%) 211
nl nl 63053 61966 (98.28%) 62140 (98.55%) 63053 (100.0%) 61360 (97.31%) 49946 (79.21%) 1556 (2.47%) 63053 (100.0%) 63053 (100.0%) 63053 (100.0%) 0 (0.0%) 0 (0.0%) 1624
nl-gem nl 13117 12050 (91.87%) 13114 (99.98%) 12980 (98.96%) 7869 (59.99%) 1281 (9.77%) 192 (1.46%) 13117 (100.0%) 12980 (98.96%) 12980 (98.96%) 0 (0.0%) 0 (0.0%) 155
no no 11193 10198 (91.11%) 0 (0.0%) 11193 (100.0%) 11188 (99.96%) 2262 (20.21%) 0 (0.0%) 11193 (100.0%) 11193 (100.0%) 11193 (100.0%) 0 (0.0%) 0 (0.0%) 463
pa es 218 218 (100.0%) 174 (79.82%) 216 (99.08%) 135 (61.93%) 194 (88.99%) 0 (0.0%) 218 (100.0%) 217 (99.54%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 1
ph en 687 687 (100.0%) 234 (34.06%) 686 (99.85%) 4 (0.58%) 249 (36.24%) 0 (0.0%) 687 (100.0%) 687 (100.0%) 447 (65.07%) 686 (99.85%) 0 (0.0%) 16
pl pl 83379 83378 (100.0%) 83244 (99.84%) 83133 (99.7%) 23585 (28.29%) 19112 (22.92%) 10083 (12.09%) 83379 (100.0%) 38677 (46.39%) 36888 (44.24%) 83133 (99.7%) 0 (0.0%) 3701
pt pt 4418 4418 (100.0%) 0 (0.0%) 4418 (100.0%) 3750 (84.88%) 1573 (35.6%) 0 (0.0%) 4418 (100.0%) 4418 (100.0%) 4418 (100.0%) 4415 (99.93%) 0 (0.0%) 18
ro ro 29638 29638 (100.0%) 26799 (90.42%) 29630 (99.97%) 1228 (4.14%) 4587 (15.48%) 783 (2.64%) 29638 (100.0%) 29029 (97.95%) 29630 (99.97%) 0 (0.0%) 0 (0.0%) 149
rs sr 2410 2410 (100.0%) 1232 (51.12%) 2401 (99.63%) 1984 (82.32%) 987 (40.95%) 0 (0.0%) 2410 (100.0%) 298 (12.37%) 2410 (100.0%) 2401 (99.63%) 341 (14.15%) 34
ru ru 4995 4995 (100.0%) 4880 (97.7%) 4995 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 4995 (100.0%) 4995 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 28
sct en 47663 47663 (100.0%) 0 (0.0%) 47663 (100.0%) 47663 (100.0%) 6 (0.01%) 0 (0.0%) 47663 (100.0%) 47663 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 1015
se-bbr sv 5609 5609 (100.0%) 756 (13.48%) 5609 (100.0%) 5569 (99.29%) 3883 (69.23%) 0 (0.0%) 5609 (100.0%) 5609 (100.0%) 5609 (100.0%) 0 (0.0%) 0 (0.0%) 42
se-fornmin sv 145136 9708 (6.69%) 987 (0.68%) 145136 (100.0%) 145119 (99.99%) 1248 (0.86%) 0 (0.0%) 145136 (100.0%) 145136 (100.0%) 145136 (100.0%) 0 (0.0%) 0 (0.0%) 967
sk de 14510 12775 (88.04%) 11064 (76.25%) 14509 (99.99%) 1314 (9.06%) 2560 (17.64%) 1006 (6.93%) 14510 (100.0%) 14510 (100.0%) 14509 (99.99%) 14509 (99.99%) 14510 (100.0%) 1848
sk sk 14506 14506 (100.0%) 11124 (76.69%) 14487 (99.87%) 1540 (10.62%) 2643 (18.22%) 835 (5.76%) 14506 (100.0%) 14506 (100.0%) 14487 (99.87%) 14487 (99.87%) 14506 (100.0%) 1846
ua uk 41924 41916 (99.98%) 35556 (84.81%) 41856 (99.84%) 6587 (15.71%) 8196 (19.55%) 0 (0.0%) 41924 (100.0%) 41766 (99.62%) 41677 (99.41%) 0 (0.0%) 0 (0.0%) 606
us en 87420 87420 (100.0%) 87420 (100.0%) 87417 (100.0%) 80585 (92.18%) 48308 (55.26%) 0 (0.0%) 87420 (100.0%) 86747 (99.23%) 87417 (100.0%) 0 (0.0%) 0 (0.0%) 2731
us-ca en 1024 1024 (100.0%) 1024 (100.0%) 1024 (100.0%) 791 (77.25%) 379 (37.01%) 0 (0.0%) 1024 (100.0%) 1024 (100.0%) 1024 (100.0%) 1011 (98.73%) 0 (0.0%) 58
za en 3549 3549 (100.0%) 3549 (100.0%) 3123 (88.0%) 1600 (45.08%) 661 (18.62%) 0 (0.0%) 3549 (100.0%) 3549 (100.0%) 3545 (99.89%) 3123 (88.0%) 0 (0.0%) 26
1215287 1058169 (87.07%) 877693 (72.22%) 1145403 (94.25%) 858358 (70.63%) 328064 (26.99%) 45795 (3.77%) 1215287 (100.0%) 1152596 (94.84%) 880620 (72.46%) 479190 (39.43%) 189021 (15.55%) 25267
Wiki Loves Monuments 2011
Concept - Timeline - Progress - Monuments database - Tools - Action points - Upload wizard - Sponsors - Press - Highlighted pictures


Lists per country - May Meeting - European Heritage Days - Commons infrastructure - Structured lists
Andorra - Austria - Belgium & Luxembourg - Denmark - Estonia - France - Germany - Hungary - Netherlands - Norway - Poland - Portugal - Romania - Russia - Spain - Sweden - Switzerland

2010 competition

LUSITANA WLM 2011 d.svg