Jobs: Loading Large Tables

Some tables from the database are quite large and the API provides them in a different way:

  1. The standard request is rejected with code 98

  2. However, setting the query parameter job=true starts a background task.

  3. The final table is then available under data/resultfile instead of data/tablefile.

``pystatis`` is handling all these details and the whole process for you!. Thus, you will only notice a larger runtime when requesting larger tables (currently about 1 minute of extra timeout).

[ ]:

import logging

from pystatis import Table

logging.basicConfig(level=logging.INFO)

Let’s assume we want to download the table “AI001-2-5”. If the table is not cached yet, a warning message tells us that a background job has been started.

[ ]:

t = Table(name="AI001-2-5")
t.get_data()

Requesting the same table again will provide the table instantly from the cache.

[3]:

t = Table(name="AI001-2-5")
t.get_data()

INFO:pystatis.http_helper:Code 0: erfolgreich

[4]:

t.data.head()

[4]:
Jahr Amtlicher Gemeindeschlüssel (AGS) Gemeinden Anteil_der_Flaeche_fuer_Siedlung_an_Gesamtflaeche__Prozent Anteil_der_Flaeche_fuer_Verkehr_an_Gesamtflaeche__Prozent Anteil_der_Flaeche_fuer_Landwirtschaft_an_Gesamtflaeche__Prozent Anteil_der_Flaeche_fuer_Wald_an_Gesamtflaeche__Prozent Anteil_Sport-,_Freizeit-,_Erholungsflaeche_an_Gesamtflaeche__Prozent Anteil_Siedlungs-_und_Verkehrsflaeche_an_Gesamtflaeche__Prozent Siedlungs-_und_Verkehrsflaeche_je_EW__qm Freiraumflaeche_je_EW__qm
0 2022 DG Deutschland 9.5 5.1 50.4 29.9 1.5 14.1 599.4 3623.7
1 2022 01 Schleswig-Holstein 9.3 4.4 68.3 10.3 1.3 13.4 718.3 4618.0
2 2022 01001 Flensburg, kreisfreie Stadt 39.8 13.8 22.1 6.6 4.4 53.5 327.9 284.1
3 2022 01002 Kiel, kreisfreie Stadt 44.9 13.9 23.4 5.6 10.3 58.8 281.5 197.4
4 2022 01003 Lübeck, kreisfreie Stadt, Hansestadt 28.9 9.2 30.2 14.7 6.2 37.9 371.9 607.5