{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"# Jobs: Loading Large Tables\n",
"\n",
"Some tables from the database are quite large and the API provides them in a different way:\n",
"\n",
"1. The standard request is rejected with code 98\n",
"2. However, setting the query parameter `job=true` starts a background task.\n",
"3. The final table is then available under `data/resultfile` instead of `data/tablefile`.\n",
"\n",
"_`pystatis` is handling all these details and the whole process for you!. Thus, you will only notice a larger runtime when requesting larger tables (currently about 1 minute of extra timeout)._\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import logging\n",
"\n",
"from pystatis import Table\n",
"\n",
"logging.basicConfig(level=logging.INFO)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's assume we want to download the table \"AI001-2-5\".\n",
"If the table is not cached yet, a warning message tells us that a background job has been started.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"t = Table(name=\"AI001-2-5\")\n",
"t.get_data()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Requesting the same table again will provide the table instantly from the cache.\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:pystatis.http_helper:Code 0: erfolgreich\n"
]
}
],
"source": [
"t = Table(name=\"AI001-2-5\")\n",
"t.get_data()"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Jahr</th>\n",
" <th>Amtlicher Gemeindeschlüssel (AGS)</th>\n",
" <th>Gemeinden</th>\n",
" <th>Anteil_der_Flaeche_fuer_Siedlung_an_Gesamtflaeche__Prozent</th>\n",
" <th>Anteil_der_Flaeche_fuer_Verkehr_an_Gesamtflaeche__Prozent</th>\n",
" <th>Anteil_der_Flaeche_fuer_Landwirtschaft_an_Gesamtflaeche__Prozent</th>\n",
" <th>Anteil_der_Flaeche_fuer_Wald_an_Gesamtflaeche__Prozent</th>\n",
" <th>Anteil_Sport-,_Freizeit-,_Erholungsflaeche_an_Gesamtflaeche__Prozent</th>\n",
" <th>Anteil_Siedlungs-_und_Verkehrsflaeche_an_Gesamtflaeche__Prozent</th>\n",
" <th>Siedlungs-_und_Verkehrsflaeche_je_EW__qm</th>\n",
" <th>Freiraumflaeche_je_EW__qm</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>2022</td>\n",
" <td>DG</td>\n",
" <td>Deutschland</td>\n",
" <td>9.5</td>\n",
" <td>5.1</td>\n",
" <td>50.4</td>\n",
" <td>29.9</td>\n",
" <td>1.5</td>\n",
" <td>14.1</td>\n",
" <td>599.4</td>\n",
" <td>3623.7</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>2022</td>\n",
" <td>01</td>\n",
" <td>Schleswig-Holstein</td>\n",
" <td>9.3</td>\n",
" <td>4.4</td>\n",
" <td>68.3</td>\n",
" <td>10.3</td>\n",
" <td>1.3</td>\n",
" <td>13.4</td>\n",
" <td>718.3</td>\n",
" <td>4618.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>2022</td>\n",
" <td>01001</td>\n",
" <td>Flensburg, kreisfreie Stadt</td>\n",
" <td>39.8</td>\n",
" <td>13.8</td>\n",
" <td>22.1</td>\n",
" <td>6.6</td>\n",
" <td>4.4</td>\n",
" <td>53.5</td>\n",
" <td>327.9</td>\n",
" <td>284.1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>2022</td>\n",
" <td>01002</td>\n",
" <td>Kiel, kreisfreie Stadt</td>\n",
" <td>44.9</td>\n",
" <td>13.9</td>\n",
" <td>23.4</td>\n",
" <td>5.6</td>\n",
" <td>10.3</td>\n",
" <td>58.8</td>\n",
" <td>281.5</td>\n",
" <td>197.4</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>2022</td>\n",
" <td>01003</td>\n",
" <td>Lübeck, kreisfreie Stadt, Hansestadt</td>\n",
" <td>28.9</td>\n",
" <td>9.2</td>\n",
" <td>30.2</td>\n",
" <td>14.7</td>\n",
" <td>6.2</td>\n",
" <td>37.9</td>\n",
" <td>371.9</td>\n",
" <td>607.5</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Jahr Amtlicher Gemeindeschlüssel (AGS) Gemeinden Anteil_der_Flaeche_fuer_Siedlung_an_Gesamtflaeche__Prozent Anteil_der_Flaeche_fuer_Verkehr_an_Gesamtflaeche__Prozent Anteil_der_Flaeche_fuer_Landwirtschaft_an_Gesamtflaeche__Prozent Anteil_der_Flaeche_fuer_Wald_an_Gesamtflaeche__Prozent Anteil_Sport-,_Freizeit-,_Erholungsflaeche_an_Gesamtflaeche__Prozent Anteil_Siedlungs-_und_Verkehrsflaeche_an_Gesamtflaeche__Prozent Siedlungs-_und_Verkehrsflaeche_je_EW__qm Freiraumflaeche_je_EW__qm\n",
"0 2022 DG Deutschland 9.5 5.1 50.4 29.9 1.5 14.1 599.4 3623.7\n",
"1 2022 01 Schleswig-Holstein 9.3 4.4 68.3 10.3 1.3 13.4 718.3 4618.0\n",
"2 2022 01001 Flensburg, kreisfreie Stadt 39.8 13.8 22.1 6.6 4.4 53.5 327.9 284.1\n",
"3 2022 01002 Kiel, kreisfreie Stadt 44.9 13.9 23.4 5.6 10.3 58.8 281.5 197.4\n",
"4 2022 01003 Lübeck, kreisfreie Stadt, Hansestadt 28.9 9.2 30.2 14.7 6.2 37.9 371.9 607.5"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"t.data.head()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.9.7 ('.venv': poetry)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.8"
},
"vscode": {
"interpreter": {
"hash": "02e23b522f8c3795158421909d41ced4ef90521258d58d1c53bee449d96f71e3"
}
}
},
"nbformat": 4,
"nbformat_minor": 1
}