nb/01_table.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "165da3a1",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "logging.basicConfig(level=logging.INFO)\n",
    "\n",
    "from pystatis import Table"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e14f4db",
   "metadata": {},
   "source": [
    "# The `Table` class\n",
    "\n",
    "The `Table` class in `pystatis` is the main interface for users to interact with the different databases and download the data/tables in form of `pandas` `DataFrames`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07f3dee4",
   "metadata": {},
   "source": [
    "To use the class, you have to pass only a single parameter: the `name` of the table you want to download."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "5d25c79a",
   "metadata": {},
   "outputs": [],
   "source": [
    "t = Table(name=\"81000-0001\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ca8127a",
   "metadata": {},
   "source": [
    "## Downloading data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d841f94",
   "metadata": {},
   "source": [
    "However, creating a new `Table` instance does not automatically retrieve the data from the database (or cache). Instead, you have to call another method: `get_data()`. The reason for this decision was to give you full control over the download process and avoid unnecessary downloads of big tables unless you are certain you want to start the download."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "632fc783",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    }
   ],
   "source": [
    "t.get_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2370bd5e",
   "metadata": {},
   "source": [
    "You can access the name of a table via the `.name` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f5f1aded",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'81000-0001'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.name"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e050eed",
   "metadata": {},
   "source": [
    "After a successful download (or cache retrieval), you can always access the raw data, that is the original response from the web API as a string, via the `.raw_data` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8fede338",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistik_Code;Statistik_Label;Zeit_Code;Zeit_Label;Zeit;1_Merkmal_Code;1_Merkmal_Label;1_Auspraegung_Code;1_Auspraegung_Label;2_Merkmal_Code;2_Merkmal_Label;2_Auspraegung_Code;2_Auspraegung_Label;BWS001__Bruttowertschoepfung__jew._ME;STR006__Guetersteuern_abzuegl._Guetersubventionen__jew._ME;STR020_______Guetersteuern__jew._ME;SUB003_______Guetersubventionen__jew._ME;VGR014__Bruttoinlandsprodukt__jew._ME;BIP005__nachr.:_Bruttoinlandsprodukt_(Veraenderung_in_%)__Prozent;BIP004__nachr.:_Bruttoinlandsprodukt_je_Einwohner__jew._ME\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2014;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRJPM;in jeweiligen Preisen (Mrd. EUR);2635,393;292,037;298,774;6,737;2927,430;4,1;36149,000\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2014;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPKM;preisbereinigt, Kettenindex (2015=100);98,810;96,150;96,250;100,970;98,530;2,2;99,380\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2014;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVK;preisbereinigt, verkettete Volumenang. (Mrd. EUR);2689,628;-;-;-;2981,695;2,2;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2014;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVU;preisbereinigt, unverkettete Volumenang.(Mrd. EUR);2584,829;-;-;-;2873,722;-;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2015;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRJPM;in jeweiligen Preisen (Mrd. EUR);2722,020;304,160;310,942;6,782;3026,180;3,4;37046,000\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2015;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPKM;preisbereinigt, Kettenindex (2015=100);100,000;100,000;100,000;100,000;100,000;1,5;100,000\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2015;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVK;preisbereinigt, verkettete Volumenang. (Mrd. EUR);2722,020;-;-;-;3026,180;1,5;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2015;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVU;preisbereinigt, unverkettete Volumenang.(Mrd. EUR);2667,231;-;-;-;2970,965;-;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2016;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRJPM;in jeweiligen Preisen (Mrd. EUR);2822,443;312,297;319,143;6,846;3134,740;3,6;38067,000\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2016;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPKM;preisbereinigt, Kettenindex (2015=100);102,250;102,030;101,990;100,240;102,230;2,2;101,410\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2016;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVK;preisbereinigt, verkettete Volumenang. (Mrd. EUR);2783,265;-;-;-;3093,664;2,2;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2016;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVU;preisbereinigt, unverkettete Volumenang.(Mrd. EUR);2783,371;-;-;-;3093,710;-;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2017;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRJPM;in jeweiligen Preisen (Mrd. EUR);2944,074;323,086;329,847;6,761;3267,160;4,2;39527,000\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2017;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPKM;preisbereinigt, Kettenindex (2015=100);105,110;103,760;103,700;101,290;104,970;2,7;103,740\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2017;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVK;preisbereinigt, verkettete Volumenang. (Mrd. EUR);2861,115;-;-;-;3176,581;2,7;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2017;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVU;preisbereinigt, unverkettete Volumenang.(Mrd. EUR);2901,242;-;-;-;3218,826;-;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2018;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRJPM;in jeweiligen Preisen (Mrd. EUR);3032,736;332,714;339,600;6,886;3365,450;3,0;40594,000\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2018;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPKM;preisbereinigt, Kettenindex (2015=100);106,100;105,160;105,070;101,020;106,000;1,0;104,440\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2018;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVK;preisbereinigt, verkettete Volumenang. (Mrd. EUR);2888,063;-;-;-;3207,751;1,0;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2018;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVU;preisbereinigt, unverkettete Volumenang.(Mrd. EUR);2971,786;-;-;-;3299,232;-;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2019;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRJPM;in jeweiligen Preisen (Mrd. EUR);3130,567;343,543;350,942;7,399;3474,110;3,2;41810,000\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2019;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPKM;preisbereinigt, Kettenindex (2015=100);107,000;108,390;108,240;101,420;107,140;1,1;105,330\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2019;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVK;preisbereinigt, verkettete Volumenang. (Mrd. EUR);2912,561;-;-;-;3242,249;1,1;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2019;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVU;preisbereinigt, unverkettete Volumenang.(Mrd. EUR);3058,504;-;-;-;3401,440;-;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2020;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRJPM;in jeweiligen Preisen (Mrd. EUR);3086,377;317,353;325,967;8,614;3403,730;-2,0;40929,000\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2020;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPKM;preisbereinigt, Kettenindex (2015=100);102,670;106,400;105,560;68,580;103,040;-3,8;101,220\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2020;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVK;preisbereinigt, verkettete Volumenang. (Mrd. EUR);2794,698;-;-;-;3118,176;-3,8;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2020;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVU;preisbereinigt, unverkettete Volumenang.(Mrd. EUR);3003,872;-;-;-;3341,107;-;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2021;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRJPM;in jeweiligen Preisen (Mrd. EUR);3276,377;341,073;365,141;24,068;3617,450;6,3;43481,000\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2021;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPKM;preisbereinigt, Kettenindex (2015=100);106,060;108,380;107,530;70,100;106,300;3,2;104,370\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2021;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVK;preisbereinigt, verkettete Volumenang. (Mrd. EUR);2886,974;-;-;-;3216,829;3,2;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2021;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVU;preisbereinigt, unverkettete Volumenang.(Mrd. EUR);3188,250;-;-;-;3511,508;-;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2022;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRJPM;in jeweiligen Preisen (Mrd. EUR);3509,628;367,182;390,670;23,488;3876,810;7,2;46264,000\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2022;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPKM;preisbereinigt, Kettenindex (2015=100);107,860;111,470;108,370;50,060;108,220;1,8;105,500\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2022;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVK;preisbereinigt, verkettete Volumenang. (Mrd. EUR);2935,971;-;-;-;3274,932;1,8;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2022;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVU;preisbereinigt, unverkettete Volumenang.(Mrd. EUR);3331,942;-;-;-;3682,747;-;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2023;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRJPM;in jeweiligen Preisen (Mrd. EUR);3767,909;354,301;390,660;36,359;4122,210;6,3;48775,000\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2023;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPKM;preisbereinigt, Kettenindex (2015=100);107,930;108,460;104,270;39,670;108,010;-0,2;104,400\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2023;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVK;preisbereinigt, verkettete Volumenang. (Mrd. EUR);2937,876;-;-;-;3268,577;-0,2;-\n",
      "81000;Volkswirtschaftliche Gesamtrechnungen des Bundes;JAHR;Jahr;2023;DINSG;Deutschland insgesamt;DG;Deutschland;VGRPB5;Preisbasis;VGRPVU;preisbereinigt, unverkettete Volumenang.(Mrd. EUR);3511,999;-;-;-;3869,273;-;-\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(t.raw_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ae90416",
   "metadata": {},
   "source": [
    "More likely, you are interested in the `pandas` `DataFrame`, which is accessible via the `.data` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "874bbbb9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Jahr</th>\n",
       "      <th>Deutschland insgesamt</th>\n",
       "      <th>Preisbasis</th>\n",
       "      <th>Bruttowertschoepfung__jew._ME</th>\n",
       "      <th>Guetersteuern_abzuegl._Guetersubventionen__jew._ME</th>\n",
       "      <th>Guetersteuern__jew._ME</th>\n",
       "      <th>Guetersubventionen__jew._ME</th>\n",
       "      <th>Bruttoinlandsprodukt__jew._ME</th>\n",
       "      <th>nachr.:_Bruttoinlandsprodukt_(Veraenderung_in_%)__Prozent</th>\n",
       "      <th>nachr.:_Bruttoinlandsprodukt_je_Einwohner__jew._ME</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2014</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>in jeweiligen Preisen (Mrd. EUR)</td>\n",
       "      <td>2635.393</td>\n",
       "      <td>292.037</td>\n",
       "      <td>298.774</td>\n",
       "      <td>6.737</td>\n",
       "      <td>2927.430</td>\n",
       "      <td>4.1</td>\n",
       "      <td>36149.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2014</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>preisbereinigt, Kettenindex (2015=100)</td>\n",
       "      <td>98.810</td>\n",
       "      <td>96.150</td>\n",
       "      <td>96.250</td>\n",
       "      <td>100.970</td>\n",
       "      <td>98.530</td>\n",
       "      <td>2.2</td>\n",
       "      <td>99.38</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2014</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>preisbereinigt, verkettete Volumenang. (Mrd. EUR)</td>\n",
       "      <td>2689.628</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2981.695</td>\n",
       "      <td>2.2</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2014</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>preisbereinigt, unverkettete Volumenang.(Mrd. EUR)</td>\n",
       "      <td>2584.829</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2873.722</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2015</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>in jeweiligen Preisen (Mrd. EUR)</td>\n",
       "      <td>2722.020</td>\n",
       "      <td>304.160</td>\n",
       "      <td>310.942</td>\n",
       "      <td>6.782</td>\n",
       "      <td>3026.180</td>\n",
       "      <td>3.4</td>\n",
       "      <td>37046.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Jahr Deutschland insgesamt                                          Preisbasis  Bruttowertschoepfung__jew._ME  Guetersteuern_abzuegl._Guetersubventionen__jew._ME  Guetersteuern__jew._ME  Guetersubventionen__jew._ME  Bruttoinlandsprodukt__jew._ME  nachr.:_Bruttoinlandsprodukt_(Veraenderung_in_%)__Prozent  nachr.:_Bruttoinlandsprodukt_je_Einwohner__jew._ME\n",
       "0  2014           Deutschland                    in jeweiligen Preisen (Mrd. EUR)                       2635.393                                             292.037                 298.774                        6.737                       2927.430                                                        4.1                                            36149.00\n",
       "1  2014           Deutschland              preisbereinigt, Kettenindex (2015=100)                         98.810                                              96.150                  96.250                      100.970                         98.530                                                        2.2                                               99.38\n",
       "2  2014           Deutschland   preisbereinigt, verkettete Volumenang. (Mrd. EUR)                       2689.628                                                 NaN                     NaN                          NaN                       2981.695                                                        2.2                                                 NaN\n",
       "3  2014           Deutschland  preisbereinigt, unverkettete Volumenang.(Mrd. EUR)                       2584.829                                                 NaN                     NaN                          NaN                       2873.722                                                        NaN                                                 NaN\n",
       "4  2015           Deutschland                    in jeweiligen Preisen (Mrd. EUR)                       2722.020                                             304.160                 310.942                        6.782                       3026.180                                                        3.4                                            37046.00"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "677d68b7",
   "metadata": {},
   "source": [
    "Finally, you can also access the metadata for this table via the `.metadata` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "5f3672e9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'Copyright': '© Statistisches Bundesamt (Destatis), 2024',\n",
      " 'Ident': {'Method': 'table', 'Service': 'metadata'},\n",
      " 'Object': {'Code': '81000-0001',\n",
      "            'Content': 'VGR des Bundes - Bruttowertschöpfung, '\n",
      "                       'Bruttoinlandsprodukt\\n'\n",
      "                       '(nominal/preisbereinigt): Deutschland, Jahre',\n",
      "            'Structure': {'Columns': [{'Code': 'JAHR',\n",
      "                                       'Content': 'Jahr',\n",
      "                                       'Selected': '10',\n",
      "                                       'Structure': None,\n",
      "                                       'Type': 'Merkmal',\n",
      "                                       'Updated': 'see parent',\n",
      "                                       'Values': '10'}],\n",
      "                          'Head': {'Code': '81000',\n",
      "                                   'Content': 'Volkswirtschaftliche '\n",
      "                                              'Gesamtrechnungen des Bundes',\n",
      "                                   'Selected': None,\n",
      "                                   'Structure': [{'Code': 'DINSG',\n",
      "                                                  'Content': 'Deutschland '\n",
      "                                                             'insgesamt',\n",
      "                                                  'Selected': '1',\n",
      "                                                  'Structure': None,\n",
      "                                                  'Type': 'Merkmal',\n",
      "                                                  'Updated': 'see parent',\n",
      "                                                  'Values': '1'}],\n",
      "                                   'Type': 'Statistik',\n",
      "                                   'Updated': 'see parent',\n",
      "                                   'Values': None},\n",
      "                          'Rows': [{'Code': 'BWS001',\n",
      "                                    'Content': 'Bruttowertschöpfung',\n",
      "                                    'Selected': None,\n",
      "                                    'Structure': None,\n",
      "                                    'Type': 'Merkmal',\n",
      "                                    'Updated': 'see parent',\n",
      "                                    'Values': None},\n",
      "                                   {'Code': 'STR006',\n",
      "                                    'Content': 'Gütersteuern abzügl. '\n",
      "                                               'Gütersubventionen',\n",
      "                                    'Selected': None,\n",
      "                                    'Structure': None,\n",
      "                                    'Type': 'Merkmal',\n",
      "                                    'Updated': 'see parent',\n",
      "                                    'Values': None},\n",
      "                                   {'Code': 'STR020',\n",
      "                                    'Content': 'Gütersteuern',\n",
      "                                    'Selected': None,\n",
      "                                    'Structure': None,\n",
      "                                    'Type': 'Merkmal',\n",
      "                                    'Updated': 'see parent',\n",
      "                                    'Values': None},\n",
      "                                   {'Code': 'SUB003',\n",
      "                                    'Content': 'Gütersubventionen',\n",
      "                                    'Selected': None,\n",
      "                                    'Structure': None,\n",
      "                                    'Type': 'Merkmal',\n",
      "                                    'Updated': 'see parent',\n",
      "                                    'Values': None},\n",
      "                                   {'Code': 'VGR014',\n",
      "                                    'Content': 'Bruttoinlandsprodukt',\n",
      "                                    'Selected': None,\n",
      "                                    'Structure': None,\n",
      "                                    'Type': 'Merkmal',\n",
      "                                    'Updated': 'see parent',\n",
      "                                    'Values': None},\n",
      "                                   {'Code': 'BIP005',\n",
      "                                    'Content': 'Bruttoinlandsprodukt '\n",
      "                                               '(Veränderung in %)',\n",
      "                                    'Selected': None,\n",
      "                                    'Structure': None,\n",
      "                                    'Type': 'Merkmal',\n",
      "                                    'Updated': 'see parent',\n",
      "                                    'Values': None},\n",
      "                                   {'Code': 'BIP004',\n",
      "                                    'Content': 'Bruttoinlandsprodukt je '\n",
      "                                               'Einwohner',\n",
      "                                    'Selected': None,\n",
      "                                    'Structure': None,\n",
      "                                    'Type': 'Merkmal',\n",
      "                                    'Updated': 'see parent',\n",
      "                                    'Values': None}],\n",
      "                          'Subheading': {'Code': 'VGRPB5',\n",
      "                                         'Content': 'Preisbasis (jeweilige '\n",
      "                                                    'Preise / preisbereinigt)',\n",
      "                                         'Selected': None,\n",
      "                                         'Structure': None,\n",
      "                                         'Type': 'Merkmal',\n",
      "                                         'Updated': 'see parent',\n",
      "                                         'Values': None},\n",
      "                          'Subtitel': None},\n",
      "            'Time': {'From': '1991', 'To': '2023'},\n",
      "            'Updated': '28.12.2022 17:19:48h',\n",
      "            'Valid': 'false'},\n",
      " 'Parameter': {'area': 'Alle',\n",
      "               'language': 'de',\n",
      "               'name': '81000-0001',\n",
      "               'password': '********************',\n",
      "               'username': '********************'},\n",
      " 'Status': {'Code': 0, 'Content': 'erfolgreich', 'Type': 'Information'}}\n"
     ]
    }
   ],
   "source": [
    "from pprint import pprint\n",
    "\n",
    "pprint(t.metadata)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "953d7cb2",
   "metadata": {},
   "source": [
    "## How `pystatis` prepares the data for you"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22b075b6",
   "metadata": {},
   "source": [
    "As you can notice from a comparison between the `.raw_data` and `.data` formats, `pystatis` is doing a lot behind the scenes to provide you with a format that is hopefully the most useful for you. You will see and learn that there are a few parameters that you can use to actually change this behavior and adjust the table to your needs. \n",
    "\n",
    "But first we would like to explain to you how `pystatis` is preparing the data by default so you have a better understanding of the underlying process."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "889e27db",
   "metadata": {},
   "source": [
    "When we look at the header of the raw data, we can notice a few things:\n",
    "- Many columns always come in a pair of `*_Code` and `*_Label` columns. Both contain the same information, only provided differently.\n",
    "- There are columns that don't have a direct use as they contain information not needed in the table, like the `Statistik_Code` and `Statistik_Label` columns at the beginning. You already know the statistic from the name of the table and this information is the same for each and every row anyway.\n",
    "- There is always a time dimension, broken down into three different columns `Zeit_Code`, `Zeit_Label` and `Zeit` (or `time_*` in English).\n",
    "- The other dimensions are called variables (German \"Merkmale\") and they always come in groups of four columns: `N_Merkmal_Code`, `N_Merkmal_Label`, `N_Auspraegung_Code`, and `N_Auspraegung_Label` (English: variable code and label and variable value code and label).\n",
    "- The actual measurements or values are at the end of the table after the variables and each measurement has one column. The name of this column follows the format `<CODE>__<LABEL>__<UNIT>`, e.g. \"BWS001__Bruttowertschoepfung__jew._ME\". \"BWS001\" is the unique code for this variable, \"Bruttowertschoepfung\" is the human readable label of the variable, and \"jew._ME\" is the unit the measurement was recorded in.\n",
    "\n",
    "**Note** This is only true for tables from Genesis and Regionalstatistik, the format of the Zensus tables is noticeably different from this. However, we follow a similar approach to provide you the same convenient output format."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef48ba95",
   "metadata": {},
   "source": [
    "The following table hopefully makes it a little bit clearer what is happening when going from the raw data string to the pandas `DataFrame`. The example is showing the Table \"11111-02-01-4\" from Regionalstatistik, but remember, that Genesis and Regionalstatistik have identically formats. The table has a time dimension, one attribute and one value.\n",
    "\n",
    "| Statistik_Code | Statistik_Label | Zeit_Code | Zeit_Label | Zeit | 1_Merkmal_Code | 1_Merkmal_Label | 1_Auspraegung_Code | 1_Auspraegung_Label | GEM001__Zahl_der_Gemeinden__Anzahl |\n",
    "|----------------|-----------------|-----------|------------|------|----------------|-----------------|---------------------|----------------------|-----------------------------------|\n",
    "|11111|Feststellung des Gebietsstandes|STAG|Stichtag|31.12.2022|KREISE|Kreise und kreisfreie Städte|DG|Deutschland|10786|\n",
    "|11111|Feststellung des Gebietsstandes|STAG|Stichtag|31.12.2022|KREISE|Kreise und kreisfreie Städte|01|  Schleswig-Holstein|1106|"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "663d6fd9",
   "metadata": {},
   "source": [
    "The same table has the following pandas representation after being \"prettified\" by `pystatis`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "483a1a0e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Stichtag</th>\n",
       "      <th>Amtlicher Gemeindeschlüssel (AGS)</th>\n",
       "      <th>Kreise und kreisfreie Städte</th>\n",
       "      <th>Zahl_der_Gemeinden__Anzahl</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022-12-31</td>\n",
       "      <td>DG</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>10786.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2022-12-31</td>\n",
       "      <td>01</td>\n",
       "      <td>Schleswig-Holstein</td>\n",
       "      <td>1106.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Stichtag Amtlicher Gemeindeschlüssel (AGS) Kreise und kreisfreie Städte  Zahl_der_Gemeinden__Anzahl\n",
       "0 2022-12-31                                DG                  Deutschland                     10786.0\n",
       "1 2022-12-31                                01           Schleswig-Holstein                      1106.0"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"11111-02-01-4\")\n",
    "t.get_data()\n",
    "t.data.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08ee995a",
   "metadata": {},
   "source": [
    "As you can see and hopefully agree, the pandas version (what we call \"prettified\") provides the same information, actually even more, because the header column names have become meaningful and there is a lot less noise that you need to filter out before you can get to the actual data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e048322",
   "metadata": {},
   "source": [
    "For Zensus `pystatis` is basically doing the same, but in a slightly different way because since the release of Zensus 2022 the API no longer returns each measurement as a single column but only a single column for all values. `pystatis` is transforming this long data format back into a wide data format, so you can work with a tidy data set. See the following example of Table \"4000W-1002\" to understand what is going on.\n",
    "\n",
    "|statistics_code|statistics_label|time_code|time_label|time|1_variable_code|1_variable_label|1_variable_attribute_code|1_variable_attribute_label|2_variable_code|2_variable_label|2_variable_attribute_code|2_variable_attribute_label|value|value_unit|value_variable_code|value_variable_label|\n",
    "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
    "|4000W|Wohnungen (Gebietsstand 15.05.2022)|STAG|Stichtag|2022-05-15|GEODL1|Deutschland|DG|Deutschland|WHGFL2|Fläche der Wohnung (10 m²-Intervalle)|WFL170B179|170 - 179 m²|1,2|%|WHG002|Wohnungen in Gebäuden mit Wohnraum|\n",
    "|4000W|Wohnungen (Gebietsstand 15.05.2022)|STAG|Stichtag|2022-05-15|GEODL1|Deutschland|DG|Deutschland|WHGFL2|Fläche der Wohnung (10 m²-Intervalle)|WFL170B179|170 - 179 m²|509041|Anzahl|WHG002|Wohnungen in Gebäuden mit Wohnraum|\n",
    "|4000W|Wohnungen (Gebietsstand 15.05.2022)|STAG|Stichtag|2022-05-15|GEODL1|Deutschland|DG|Deutschland|WHGFL2|Fläche der Wohnung (10 m²-Intervalle)|WFL090B099|90 - 99 m²|7,2|%|WHG002|Wohnungen in Gebäuden mit Wohnraum|\n",
    "|4000W|Wohnungen (Gebietsstand 15.05.2022)|STAG|Stichtag|2022-05-15|GEODL1|Deutschland|DG|Deutschland|WHGFL2|Fläche der Wohnung (10 m²-Intervalle)|WFL090B099|90 - 99 m²|3082890|Anzahl|WHG002|Wohnungen in Gebäuden mit Wohnraum|\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b1c811c2",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.cache:Data was successfully cached under /Users/miay/.pystatis/data/4000W-1002/3d1d8e69a7d4397b08b0/20240713.zip.\n",
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Stichtag</th>\n",
       "      <th>Deutschland</th>\n",
       "      <th>Fläche der Wohnung (10 m²-Intervalle)</th>\n",
       "      <th>Wohnungen in Gebäuden mit Wohnraum__%</th>\n",
       "      <th>Wohnungen in Gebäuden mit Wohnraum__Anzahl</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022-05-15</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>Insgesamt</td>\n",
       "      <td>100.0</td>\n",
       "      <td>43106589.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2022-05-15</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>Unter 30 m²</td>\n",
       "      <td>2.1</td>\n",
       "      <td>909120.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Stichtag  Deutschland Fläche der Wohnung (10 m²-Intervalle)  Wohnungen in Gebäuden mit Wohnraum__%  Wohnungen in Gebäuden mit Wohnraum__Anzahl\n",
       "0  2022-05-15  Deutschland                             Insgesamt                                  100.0                                  43106589.0\n",
       "1  2022-05-15  Deutschland                           Unter 30 m²                                    2.1                                    909120.0"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"4000W-1002\")\n",
    "t.get_data()\n",
    "t.data.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61d6df19",
   "metadata": {},
   "source": [
    "As you can see, `pystatis` is not only increasing readability and making data access easy, it also reduces the amount of data you have to work with. Going from a long format back to a tidy wide format means cutting the number of rows to 1/3 because all three measurements get back their own column."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57acff63",
   "metadata": {},
   "source": [
    "`pystatis` is doing the following things (by default) when parsing the original raw string:\n",
    "- remove the information about the statistic\n",
    "- for all variables: only keep the value column and choose the variable label as the column name\n",
    "- for all measurements: remove the variable code from the column name, only keep label and unit\n",
    "- set the proper data types (`datetime` for the time variable, if appropriate; `str` for regional codes)\n",
    "- handling missing values (i.e. replacing characters \"...\", \".\", \"-\", \"/\" and \"x\" by proper `NaN` values) and special characters\n",
    "- choosing the right decimal character depending on the specified language (German: \",\", English: \".\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "051de4fa",
   "metadata": {},
   "source": [
    "All of this happens behind the scenes when you are downloading the data with `get_data()` and access it via the `Table.data` attribute."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c9a8e49",
   "metadata": {},
   "source": [
    "## All `get_data()` parameters explained"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbd4c6bb",
   "metadata": {},
   "source": [
    "You can find a list of all parameters in the [documentation](https://correlaid.github.io/pystatis/dev/pystatis.html#pystatis.table.Table.get_data) or in the docstring. All parameters are keyword parameters only (fancy Python star syntax: `f(*, everything from here on has to be a keyword only parameter)`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e67dcfb7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;31mSignature:\u001b[0m\n",
      "\u001b[0mt\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_data\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[1;33m*\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mprettify\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mbool\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;32mTrue\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0marea\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m'all'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mstartyear\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m''\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mendyear\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m''\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mtimeslices\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m''\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mregionalvariable\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m''\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mregionalkey\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m''\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mstand\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m''\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mlanguage\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m'de'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m    \u001b[0mquality\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mbool\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;32mFalse\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\n",
      "\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mDocstring:\u001b[0m\n",
      "Downloads raw data and metadata from GENESIS-Online.\n",
      "\n",
      "Additional keyword arguments are passed on to the GENESIS-Online GET request for tablefile.\n",
      "\n",
      "Args:\n",
      "    prettify (bool, optional): Reformats the table into a readable format. Defaults to True.\n",
      "    area (str, optional): Area to search for the object in GENESIS-Online. Defaults to \"all\".\n",
      "    startyear (str, optional): Data beginning with that year will be returned.\n",
      "        Parameter is cumulative to `timeslices`. Supports 4 digits (jjjj) or 4+2 digits (jjjj/jj).\n",
      "        Accepts values between \"1900\" and \"2100\".\n",
      "    endyear (str, optional): Data ending with that year will be returned.\n",
      "        Parameter is cumulative to `timeslices`. Supports 4 digits (jjjj) or 4+2 digits (jjjj/jj).\n",
      "        Accepts values between \"1900\" and \"2100\".\n",
      "    timeslices (str, optional): Number of time slices to be returned.\n",
      "        This parameter is cumulative to `startyear` and `endyear`.\n",
      "    regionalvariable (str, optional): \"code\" of the regional classification (RKMerkmal),\n",
      "        to which the selection using `regionalkey` is to be applied.\n",
      "        Accepts 1-6 characters.\n",
      "        Possible values:\n",
      "        - Regionalstatistik (only for tables ending with \"B\", see /catalogue/variables):\n",
      "            - \"DG\" (Deutschland, 1) -> will not return extra column\n",
      "            - \"DLAND\" (Bundesländer, 16)\n",
      "            - \"REGBEZ\" (Regierungsbezirke, 44)\n",
      "            - \"KREISE\" (Kreise und kreisfreie Städte, 489)\n",
      "            - \"GEMEIN\" (Gemeinden, 13564)\n",
      "        - Zensusdatenbank (for all tables, see /catalogue/variables):\n",
      "            - \"GEODL1\" (Deutschland, 1) -> will not return extra column\n",
      "            - \"GEODL3\" (Deutschland, 1) -> will not return extra column\n",
      "            - \"GEOBL1\" (Bundesländer, 16)\n",
      "            - \"GEOBL3\" (Bundesländer, 16)\n",
      "            - \"GEOGM1\" (Gemeinden, 11340)\n",
      "            - \"GEOGM2\" (Gemeinden mit min. 10_000 Einwohnern, 1574)\n",
      "            - \"GEOGM3\" (Gemeinden mit min. 10_000 Einwohnern, 1574)\n",
      "            - \"GEOLK1\" (Landkreise und kreisfreie Städte, 412)\n",
      "            - \"GEOLK3\" (Landkreise und kreisfreie Städte, 412)\n",
      "            - \"GEORB1\" (Regierungsbezirke/Statistische Regionen, 36)\n",
      "            - \"GEORB3\" (Regierungsbezirke/Statistische Regionen, 36)\n",
      "            - \"GEOVB1\" (Gemeindeverbände, 1333)\n",
      "            - \"GEOVB2\" (Gemeindeverbände mit mindestens 10 000 Einwohnern, 338)\n",
      "            - \"GEOVB3\" (Gemeindeverbände mit mindestens 10 000 Einwohnern, 157)\n",
      "    regionalkey (str, optional): Official municipality key (AGS).\n",
      "        Multiple values can be passed as a comma-separated list.\n",
      "        Accepts 1-12 characters. \"*\" can be used as wildcard.\n",
      "    stand (str, optional): Only download the table if it is newer than the status date.\n",
      "        \"tt.mm.jjjj hh:mm\" or \"tt.mm.jjjj\". Example: \"24.12.2001 19:15\".\n",
      "    language (str, optional): Messages and data descriptions are supplied in this language.\n",
      "    quality (bool, optional): If True, Value-adding quality labels are issued.\n",
      "\u001b[1;31mFile:\u001b[0m      c:\\users\\mdick\\documents\\github\\pystatis\\src\\pystatis\\table.py\n",
      "\u001b[1;31mType:\u001b[0m      method"
     ]
    }
   ],
   "source": [
    "?t.get_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27f2deb1",
   "metadata": {},
   "source": [
    "### `prettify`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f2dcb9e",
   "metadata": {},
   "source": [
    "`prettify` is a boolean and can only be `True` or `False`. The default is `True` because `prettify` is basically doing all the above mentioned work behind the scenes to transform the raw data into the nicer tidy version. However, as we don't know what specific requirements you have, it can always be the case that we are not doing what you want to do or we are doing it in a wrong way. Instead of starting from scratch with the raw string, `prettify=False` will still give you a pandas `DataFrame` but without the transformations described in the previous sections. Basically, `prettify=False` gives you the raw data as a pandas `DataFrame` instead of a string without any transformation from our side."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "5953416f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>statistics_code</th>\n",
       "      <th>statistics_label</th>\n",
       "      <th>time_code</th>\n",
       "      <th>time_label</th>\n",
       "      <th>time</th>\n",
       "      <th>1_variable_code</th>\n",
       "      <th>1_variable_label</th>\n",
       "      <th>1_variable_attribute_code</th>\n",
       "      <th>1_variable_attribute_label</th>\n",
       "      <th>value</th>\n",
       "      <th>value_unit</th>\n",
       "      <th>value_variable_code</th>\n",
       "      <th>value_variable_label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000A</td>\n",
       "      <td>Bevölkerung kompakt (Gebietsstand 15.05.2022)</td>\n",
       "      <td>STAG</td>\n",
       "      <td>Stichtag</td>\n",
       "      <td>2022-05-15</td>\n",
       "      <td>GEOGM4</td>\n",
       "      <td>Gemeinden (Gebietsstand 15.05.2022)</td>\n",
       "      <td>092760130130</td>\n",
       "      <td>Lindberg</td>\n",
       "      <td>2294</td>\n",
       "      <td>Anzahl</td>\n",
       "      <td>PRS018</td>\n",
       "      <td>Personen</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000A</td>\n",
       "      <td>Bevölkerung kompakt (Gebietsstand 15.05.2022)</td>\n",
       "      <td>STAG</td>\n",
       "      <td>Stichtag</td>\n",
       "      <td>2022-05-15</td>\n",
       "      <td>GEOGM4</td>\n",
       "      <td>Gemeinden (Gebietsstand 15.05.2022)</td>\n",
       "      <td>073355011022</td>\n",
       "      <td>Landstuhl, Sickingenstadt, Stadt</td>\n",
       "      <td>8305</td>\n",
       "      <td>Anzahl</td>\n",
       "      <td>PRS018</td>\n",
       "      <td>Personen</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000A</td>\n",
       "      <td>Bevölkerung kompakt (Gebietsstand 15.05.2022)</td>\n",
       "      <td>STAG</td>\n",
       "      <td>Stichtag</td>\n",
       "      <td>2022-05-15</td>\n",
       "      <td>GEOGM4</td>\n",
       "      <td>Gemeinden (Gebietsstand 15.05.2022)</td>\n",
       "      <td>130765654053</td>\n",
       "      <td>Grebs-Niendorf</td>\n",
       "      <td>546</td>\n",
       "      <td>Anzahl</td>\n",
       "      <td>PRS018</td>\n",
       "      <td>Personen</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  statistics_code                               statistics_label time_code time_label        time 1_variable_code                     1_variable_label 1_variable_attribute_code        1_variable_attribute_label  value value_unit value_variable_code value_variable_label\n",
       "0           1000A  Bevölkerung kompakt (Gebietsstand 15.05.2022)      STAG   Stichtag  2022-05-15          GEOGM4  Gemeinden (Gebietsstand 15.05.2022)              092760130130                          Lindberg   2294     Anzahl              PRS018             Personen\n",
       "1           1000A  Bevölkerung kompakt (Gebietsstand 15.05.2022)      STAG   Stichtag  2022-05-15          GEOGM4  Gemeinden (Gebietsstand 15.05.2022)              073355011022  Landstuhl, Sickingenstadt, Stadt   8305     Anzahl              PRS018             Personen\n",
       "2           1000A  Bevölkerung kompakt (Gebietsstand 15.05.2022)      STAG   Stichtag  2022-05-15          GEOGM4  Gemeinden (Gebietsstand 15.05.2022)              130765654053                    Grebs-Niendorf    546     Anzahl              PRS018             Personen"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"1000A-0000\")\n",
    "t.get_data(prettify=False)\n",
    "t.data.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "993660a3",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Stichtag</th>\n",
       "      <th>Amtlicher Regionalschlüssel (ARS)</th>\n",
       "      <th>Gemeinden (Gebietsstand 15.05.2022)</th>\n",
       "      <th>Personen__Anzahl</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4816</th>\n",
       "      <td>2022-05-15</td>\n",
       "      <td>073355011022</td>\n",
       "      <td>Landstuhl, Sickingenstadt, Stadt</td>\n",
       "      <td>8305</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6934</th>\n",
       "      <td>2022-05-15</td>\n",
       "      <td>092760130130</td>\n",
       "      <td>Lindberg</td>\n",
       "      <td>2294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9394</th>\n",
       "      <td>2022-05-15</td>\n",
       "      <td>130765654053</td>\n",
       "      <td>Grebs-Niendorf</td>\n",
       "      <td>546</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Stichtag Amtlicher Regionalschlüssel (ARS) Gemeinden (Gebietsstand 15.05.2022)  Personen__Anzahl\n",
       "4816  2022-05-15                      073355011022    Landstuhl, Sickingenstadt, Stadt              8305\n",
       "6934  2022-05-15                      092760130130                            Lindberg              2294\n",
       "9394  2022-05-15                      130765654053                      Grebs-Niendorf               546"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# don't be confused by the query, we have to query by ARS in this example because prettify=True sorts the data by ARS and the order is different from above\n",
    "t = Table(\"1000A-0000\")\n",
    "t.get_data(prettify=True)\n",
    "t.data[t.data[\"Amtlicher Regionalschlüssel (ARS)\"].isin([\"092760130130\", \"073355011022\", \"130765654053\"])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "caef5440",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 10787 entries, 0 to 10786\n",
      "Data columns (total 4 columns):\n",
      " #   Column                               Non-Null Count  Dtype \n",
      "---  ------                               --------------  ----- \n",
      " 0   Stichtag                             10787 non-null  object\n",
      " 1   Amtlicher Regionalschlüssel (ARS)    10787 non-null  object\n",
      " 2   Gemeinden (Gebietsstand 15.05.2022)  10787 non-null  object\n",
      " 3   Personen__Anzahl                     10787 non-null  int64 \n",
      "dtypes: int64(1), object(3)\n",
      "memory usage: 337.2+ KB\n"
     ]
    }
   ],
   "source": [
    "t.data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a9d87bb",
   "metadata": {},
   "source": [
    "### `area`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91c8216b",
   "metadata": {},
   "source": [
    "We don't have a good explanation for this one, so if you have a concrete use case, please let us know!\n",
    "\n",
    "Here is the description from the official [documentation](https://www-genesis.destatis.de/genesis/misc/GENESIS-Webservices_Einfuehrung.pdf):\n",
    "\n",
    "The area query parameter specifies the area in which the object is stored, which is analogous to online navigation. Here is the breakdown:\n",
    "\n",
    "For internal users:\n",
    "\n",
    "- Meine/Benutzer\n",
    "- Gruppe\n",
    "- Amt\n",
    "- Katalog/Öffentlich\n",
    "- Alle\n",
    "\n",
    "For external users:\n",
    "\n",
    "- Meine/Benutzer\n",
    "- Katalog/Öffentlich\n",
    "\n",
    "This parameter corresponds to:\n",
    "\n",
    "- Bereich=Benutzer as Bereich=Meine\n",
    "- Bereich=Öffentlich as Bereich=Katalog"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1bc5047",
   "metadata": {},
   "source": [
    "### `startyear`, `endyear` and `timeslices`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e72f9b69",
   "metadata": {},
   "source": [
    "All three parameters can be used to fetch data of a certain time range for the given Table. The default is Table specific and has to be checked for each Table, often it is just the latest period of time available.\n",
    "\n",
    "The important thing here is that `timeslices` is **cumulative** to the other two options, meaning that `timeslices=N` will give you N years after `startyear` or before `endyear`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d21d8bf4",
   "metadata": {},
   "source": [
    "Let's say you are interested in school-leaving qualifications over the years in Germany. Then Table [21111-0004](https://www-genesis.destatis.de/genesis//online?operation=table&code=21111-0004) might be of interest to you. The description of the table mentions that data is available for the years 1997/98 - 2021/22. But what will the API return if you specify no time parameter? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c1e32498",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.cache:Data was successfully cached under /Users/miay/.pystatis/data/21111-0004/7f787b175d83ae25ee55/20240630.csv.\n",
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array(['2020/21', '2021/22'], dtype=object)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"21111-0004\")\n",
    "t.get_data()\n",
    "t.data[\"Schuljahr\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f687bd08",
   "metadata": {},
   "source": [
    "As you can see, `pystatis` only returns you, for whatever reason, the years 2020/21 and 2021/22. How can you get the ten latest years? Let's see:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9b34306a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.cache:Data was successfully cached under /Users/miay/.pystatis/data/21111-0004/188b2885044566b1329d/20240630.csv.\n",
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array(['2012/13', '2013/14', '2014/15', '2015/16', '2016/17', '2017/18',\n",
       "       '2018/19', '2019/20', '2020/21', '2021/22'], dtype=object)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.get_data(timeslices=10)\n",
    "t.data[\"Schuljahr\"].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "037dfbd8",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.cache:Data was successfully cached under /Users/miay/.pystatis/data/21111-0004/2cd6fefd6a6c423b70bd/20240630.csv.\n",
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array(['2012/13', '2013/14', '2014/15', '2015/16', '2016/17', '2017/18',\n",
       "       '2018/19', '2019/20', '2020/21', '2021/22'], dtype=object)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.get_data(startyear=\"2012\")\n",
    "t.data[\"Schuljahr\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "630d5fff",
   "metadata": {},
   "source": [
    "If you are only interested in a time period somewhere in between, you need to use both `startyear` and `endyear`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d0b70aad",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.cache:Data was successfully cached under /Users/miay/.pystatis/data/21111-0004/23e95897ca6a80fd2617/20240630.csv.\n",
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array(['2012/13', '2013/14', '2014/15', '2015/16'], dtype=object)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.get_data(startyear=\"2012\", endyear=\"2015\")\n",
    "t.data[\"Schuljahr\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6edd09e4",
   "metadata": {},
   "source": [
    "You might expect that using `startyear` and `timeslices` might give the same result, but it turns out that this is not the case and quite misleading. In fact, `timeslices` is always coming on top of whatever you have selected with `startyear` and `endyear`. Is that confusing? We definitely think so!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "9bc604d4",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.cache:Data was successfully cached under /Users/miay/.pystatis/data/21111-0004/3286f8f1af46f895cd26/20240630.csv.\n",
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array(['2012/13', '2013/14', '2014/15', '2015/16', '2019/20', '2020/21',\n",
       "       '2021/22'], dtype=object)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.get_data(startyear=\"2012\", endyear=\"2015\", timeslices=3)  # gives everything between 2012 and 2015 three more years\n",
    "t.data[\"Schuljahr\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31cfa070",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "b3bbe0ed",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.cache:Data was successfully cached under /Users/miay/.pystatis/data/21111-0004/6e1a814d2cac7d1233f1/20240630.csv.\n",
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array(['1997/98', '1998/99', '1999/00', '2000/01', '2001/02', '2002/03',\n",
       "       '2003/04', '2004/05', '2005/06', '2006/07', '2007/08', '2008/09',\n",
       "       '2009/10', '2010/11', '2011/12', '2012/13', '2013/14', '2014/15',\n",
       "       '2015/16', '2019/20', '2020/21', '2021/22'], dtype=object)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.get_data(endyear=\"2015\", timeslices=3)  # gives everything up to 2015 and three more years\n",
    "t.data[\"Schuljahr\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1293aa0f",
   "metadata": {},
   "source": [
    "### `regionalvariable` and `regionalkey`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "561f5029",
   "metadata": {},
   "source": [
    "Tables that end with a \"B\" in Regionalstatistik are special: They allow to change the regional depth of the data, meaning that you can fetch data for different regional areas depending on these two variables. The same is true for all Zensus tables. \n",
    "\n",
    "To select a specific region area, you can either specify `regionalvariable` and pass one of the reserved codes for this geo variable, or you can directly select a specific region via its key. Let's see some examples, so let's analyze Table [12613-01-01-5-B](https://www.regionalstatistik.de/genesis//online?operation=table&code=12613-01-01-5-B):\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0a5ac832",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Jahr</th>\n",
       "      <th>Amtlicher Gemeindeschlüssel (AGS)</th>\n",
       "      <th>Gemeinden</th>\n",
       "      <th>Geschlecht</th>\n",
       "      <th>Gestorbene__Anzahl</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022</td>\n",
       "      <td>01001000</td>\n",
       "      <td>Flensburg, kreisfreie Stadt</td>\n",
       "      <td>Insgesamt</td>\n",
       "      <td>1200.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2022</td>\n",
       "      <td>01001000</td>\n",
       "      <td>Flensburg, kreisfreie Stadt</td>\n",
       "      <td>männlich</td>\n",
       "      <td>594.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2022</td>\n",
       "      <td>01001000</td>\n",
       "      <td>Flensburg, kreisfreie Stadt</td>\n",
       "      <td>weiblich</td>\n",
       "      <td>606.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2022</td>\n",
       "      <td>01002000</td>\n",
       "      <td>Kiel, kreisfreie Stadt, Landeshauptstadt</td>\n",
       "      <td>Insgesamt</td>\n",
       "      <td>2996.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2022</td>\n",
       "      <td>01002000</td>\n",
       "      <td>Kiel, kreisfreie Stadt, Landeshauptstadt</td>\n",
       "      <td>männlich</td>\n",
       "      <td>1479.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Jahr Amtlicher Gemeindeschlüssel (AGS)                                 Gemeinden Geschlecht  Gestorbene__Anzahl\n",
       "0  2022                          01001000               Flensburg, kreisfreie Stadt  Insgesamt              1200.0\n",
       "1  2022                          01001000               Flensburg, kreisfreie Stadt   männlich               594.0\n",
       "2  2022                          01001000               Flensburg, kreisfreie Stadt   weiblich               606.0\n",
       "3  2022                          01002000  Kiel, kreisfreie Stadt, Landeshauptstadt  Insgesamt              2996.0\n",
       "4  2022                          01002000  Kiel, kreisfreie Stadt, Landeshauptstadt   männlich              1479.0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"12613-01-01-5-B\")\n",
    "t.get_data()\n",
    "t.data.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e173637e",
   "metadata": {},
   "source": [
    "Instead of fetching the data for all municipalities, we can choose a different regional depth (see the codes [here](https://correlaid.github.io/pystatis/dev/pystatis.html#module-pystatis.table)), for example \"KRESIE\", one level above \"GEMEINDE\", which is the default for this table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ad1ee900",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Jahr</th>\n",
       "      <th>Amtlicher Gemeindeschlüssel (AGS)</th>\n",
       "      <th>Kreise und kreisfreie Städte</th>\n",
       "      <th>Geschlecht</th>\n",
       "      <th>Gestorbene__Anzahl</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022</td>\n",
       "      <td>01001</td>\n",
       "      <td>Flensburg, kreisfreie Stadt</td>\n",
       "      <td>Insgesamt</td>\n",
       "      <td>1200.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2022</td>\n",
       "      <td>01001</td>\n",
       "      <td>Flensburg, kreisfreie Stadt</td>\n",
       "      <td>männlich</td>\n",
       "      <td>594.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2022</td>\n",
       "      <td>01001</td>\n",
       "      <td>Flensburg, kreisfreie Stadt</td>\n",
       "      <td>weiblich</td>\n",
       "      <td>606.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2022</td>\n",
       "      <td>01002</td>\n",
       "      <td>Kiel, kreisfreie Stadt</td>\n",
       "      <td>Insgesamt</td>\n",
       "      <td>2996.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2022</td>\n",
       "      <td>01002</td>\n",
       "      <td>Kiel, kreisfreie Stadt</td>\n",
       "      <td>männlich</td>\n",
       "      <td>1479.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Jahr Amtlicher Gemeindeschlüssel (AGS) Kreise und kreisfreie Städte Geschlecht  Gestorbene__Anzahl\n",
       "0  2022                             01001  Flensburg, kreisfreie Stadt  Insgesamt              1200.0\n",
       "1  2022                             01001  Flensburg, kreisfreie Stadt   männlich               594.0\n",
       "2  2022                             01001  Flensburg, kreisfreie Stadt   weiblich               606.0\n",
       "3  2022                             01002       Kiel, kreisfreie Stadt  Insgesamt              2996.0\n",
       "4  2022                             01002       Kiel, kreisfreie Stadt   männlich              1479.0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"12613-01-01-5-B\")\n",
    "t.get_data(regionalvariable=\"KREISE\")\n",
    "t.data.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "783cbae6",
   "metadata": {},
   "source": [
    "`regionalkey` can be used to fetch only certain areas, see <https://datengui.de/statistik-erklaert/ags>. We now fetch only municipalities in Baden-Württemberg:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "84329699",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Jahr</th>\n",
       "      <th>Amtlicher Gemeindeschlüssel (AGS)</th>\n",
       "      <th>Gemeinden</th>\n",
       "      <th>Geschlecht</th>\n",
       "      <th>Gestorbene__Anzahl</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022</td>\n",
       "      <td>08111000</td>\n",
       "      <td>Stuttgart, Landeshauptstadt</td>\n",
       "      <td>Insgesamt</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2022</td>\n",
       "      <td>08111000</td>\n",
       "      <td>Stuttgart, Landeshauptstadt</td>\n",
       "      <td>männlich</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2022</td>\n",
       "      <td>08111000</td>\n",
       "      <td>Stuttgart, Landeshauptstadt</td>\n",
       "      <td>weiblich</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2022</td>\n",
       "      <td>08115001</td>\n",
       "      <td>Aidlingen</td>\n",
       "      <td>Insgesamt</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2022</td>\n",
       "      <td>08115001</td>\n",
       "      <td>Aidlingen</td>\n",
       "      <td>männlich</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Jahr Amtlicher Gemeindeschlüssel (AGS)                    Gemeinden Geschlecht  Gestorbene__Anzahl\n",
       "0  2022                          08111000  Stuttgart, Landeshauptstadt  Insgesamt                 NaN\n",
       "1  2022                          08111000  Stuttgart, Landeshauptstadt   männlich                 NaN\n",
       "2  2022                          08111000  Stuttgart, Landeshauptstadt   weiblich                 NaN\n",
       "3  2022                          08115001                    Aidlingen  Insgesamt                 NaN\n",
       "4  2022                          08115001                    Aidlingen   männlich                 NaN"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"12613-01-01-5-B\")\n",
    "t.get_data(regionalkey=\"08*\")\n",
    "t.data.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1497a9a",
   "metadata": {},
   "source": [
    "### `stand`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5aa0d3d6",
   "metadata": {},
   "source": [
    "Can be used to only download tables that have a version newer than the given date."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "055850e8",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Schuljahr</th>\n",
       "      <th>Deutschland insgesamt</th>\n",
       "      <th>Geschlecht</th>\n",
       "      <th>Schulart</th>\n",
       "      <th>Schulabschlüsse</th>\n",
       "      <th>Absolventen_und_Abgaenger__Anzahl</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020/21</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>männlich</td>\n",
       "      <td>Hauptschulen</td>\n",
       "      <td>Ohne Hauptschulabschluss</td>\n",
       "      <td>4086.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020/21</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>männlich</td>\n",
       "      <td>Hauptschulen</td>\n",
       "      <td>Hauptschulabschluss</td>\n",
       "      <td>22784.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020/21</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>männlich</td>\n",
       "      <td>Hauptschulen</td>\n",
       "      <td>Mittlerer Schulabschluss</td>\n",
       "      <td>12700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020/21</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>männlich</td>\n",
       "      <td>Hauptschulen</td>\n",
       "      <td>Fachhochschulreife</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020/21</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>männlich</td>\n",
       "      <td>Hauptschulen</td>\n",
       "      <td>Allgemeine Hochschulreife</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Schuljahr Deutschland insgesamt Geschlecht      Schulart            Schulabschlüsse  Absolventen_und_Abgaenger__Anzahl\n",
       "0   2020/21           Deutschland   männlich  Hauptschulen   Ohne Hauptschulabschluss                             4086.0\n",
       "1   2020/21           Deutschland   männlich  Hauptschulen        Hauptschulabschluss                            22784.0\n",
       "2   2020/21           Deutschland   männlich  Hauptschulen   Mittlerer Schulabschluss                            12700.0\n",
       "3   2020/21           Deutschland   männlich  Hauptschulen         Fachhochschulreife                                NaN\n",
       "4   2020/21           Deutschland   männlich  Hauptschulen  Allgemeine Hochschulreife                                NaN"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"21111-0004\")\n",
    "t.get_data()\n",
    "t.data.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8928e0de",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'24.11.2023 14:49:25h'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.metadata[\"Object\"][\"Updated\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4bb78e4f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Schuljahr</th>\n",
       "      <th>Deutschland insgesamt</th>\n",
       "      <th>Geschlecht</th>\n",
       "      <th>Schulart</th>\n",
       "      <th>Schulabschlüsse</th>\n",
       "      <th>Absolventen_und_Abgaenger__Anzahl</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020/21</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>männlich</td>\n",
       "      <td>Hauptschulen</td>\n",
       "      <td>Ohne Hauptschulabschluss</td>\n",
       "      <td>4086.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020/21</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>männlich</td>\n",
       "      <td>Hauptschulen</td>\n",
       "      <td>Hauptschulabschluss</td>\n",
       "      <td>22784.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020/21</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>männlich</td>\n",
       "      <td>Hauptschulen</td>\n",
       "      <td>Mittlerer Schulabschluss</td>\n",
       "      <td>12700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020/21</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>männlich</td>\n",
       "      <td>Hauptschulen</td>\n",
       "      <td>Fachhochschulreife</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020/21</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>männlich</td>\n",
       "      <td>Hauptschulen</td>\n",
       "      <td>Allgemeine Hochschulreife</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Schuljahr Deutschland insgesamt Geschlecht      Schulart            Schulabschlüsse  Absolventen_und_Abgaenger__Anzahl\n",
       "0   2020/21           Deutschland   männlich  Hauptschulen   Ohne Hauptschulabschluss                             4086.0\n",
       "1   2020/21           Deutschland   männlich  Hauptschulen        Hauptschulabschluss                            22784.0\n",
       "2   2020/21           Deutschland   männlich  Hauptschulen   Mittlerer Schulabschluss                            12700.0\n",
       "3   2020/21           Deutschland   männlich  Hauptschulen         Fachhochschulreife                                NaN\n",
       "4   2020/21           Deutschland   männlich  Hauptschulen  Allgemeine Hochschulreife                                NaN"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.get_data(stand=\"01.01.2023\")  # before updated date, so should return data\n",
    "t.data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b40994e3",
   "metadata": {},
   "outputs": [
    {
     "ename": "NoNewerDataError",
     "evalue": "Keine aktualisierten Daten vorhanden. (Mindestens ein Parameter enthält ungültige Werte. Er wurde angepasst, um den Service starten zu können.: stand)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNoNewerDataError\u001b[0m                          Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[7], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mt\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mget_data\u001b[49m\u001b[43m(\u001b[49m\u001b[43mstand\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m01.01.2024\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m \u001b[38;5;66;03m# after updated date, so error\u001b[39;00m\n\u001b[1;32m      2\u001b[0m t\u001b[38;5;241m.\u001b[39mdata\u001b[38;5;241m.\u001b[39mhead()\n",
      "File \u001b[0;32m~/git/github/CorrelAid/pystatis/src/pystatis/table.py:118\u001b[0m, in \u001b[0;36mTable.get_data\u001b[0;34m(self, prettify, area, startyear, endyear, timeslices, regionalvariable, regionalkey, stand, language, quality)\u001b[0m\n\u001b[1;32m    115\u001b[0m db_matches \u001b[38;5;241m=\u001b[39m db\u001b[38;5;241m.\u001b[39midentify_db_matches(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mname)\n\u001b[1;32m    116\u001b[0m db_name \u001b[38;5;241m=\u001b[39m db\u001b[38;5;241m.\u001b[39mselect_db_by_credentials(db_matches)\n\u001b[0;32m--> 118\u001b[0m raw_data_bytes \u001b[38;5;241m=\u001b[39m \u001b[43mload_data\u001b[49m\u001b[43m(\u001b[49m\u001b[43mendpoint\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mdata\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmethod\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtablefile\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mparams\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mparams\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdb_name\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdb_name\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    119\u001b[0m \u001b[38;5;28;01massert\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(raw_data_bytes, \u001b[38;5;28mbytes\u001b[39m)  \u001b[38;5;66;03m# nosec assert_used\u001b[39;00m\n\u001b[1;32m    120\u001b[0m raw_data_str \u001b[38;5;241m=\u001b[39m raw_data_bytes\u001b[38;5;241m.\u001b[39mdecode(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mutf-8-sig\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
      "File \u001b[0;32m~/git/github/CorrelAid/pystatis/src/pystatis/http_helper.py:51\u001b[0m, in \u001b[0;36mload_data\u001b[0;34m(endpoint, method, params, db_name)\u001b[0m\n\u001b[1;32m     49\u001b[0m     data \u001b[38;5;241m=\u001b[39m read_from_cache(cache_dir, name, params)\n\u001b[1;32m     50\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m---> 51\u001b[0m     response \u001b[38;5;241m=\u001b[39m \u001b[43mget_data_from_endpoint\u001b[49m\u001b[43m(\u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mparams\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdb_name\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m     52\u001b[0m     content_type \u001b[38;5;241m=\u001b[39m response\u001b[38;5;241m.\u001b[39mheaders\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mContent-Type\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtext/csv\u001b[39m\u001b[38;5;124m\"\u001b[39m)\u001b[38;5;241m.\u001b[39msplit(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m/\u001b[39m\u001b[38;5;124m\"\u001b[39m)[\u001b[38;5;241m-\u001b[39m\u001b[38;5;241m1\u001b[39m]\n\u001b[1;32m     53\u001b[0m     data \u001b[38;5;241m=\u001b[39m response\u001b[38;5;241m.\u001b[39mcontent\n",
      "File \u001b[0;32m~/git/github/CorrelAid/pystatis/src/pystatis/http_helper.py:124\u001b[0m, in \u001b[0;36mget_data_from_endpoint\u001b[0;34m(endpoint, method, params, db_name)\u001b[0m\n\u001b[1;32m    122\u001b[0m response\u001b[38;5;241m.\u001b[39mencoding \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mUTF-8\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    123\u001b[0m _check_invalid_status_code(response)\n\u001b[0;32m--> 124\u001b[0m \u001b[43m_check_invalid_destatis_status_code\u001b[49m\u001b[43m(\u001b[49m\u001b[43mresponse\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    126\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m response\n",
      "File \u001b[0;32m~/git/github/CorrelAid/pystatis/src/pystatis/http_helper.py:266\u001b[0m, in \u001b[0;36m_check_invalid_destatis_status_code\u001b[0;34m(response)\u001b[0m\n\u001b[1;32m    263\u001b[0m     response_dict \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m    265\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m response_dict \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m--> 266\u001b[0m     \u001b[43m_check_destatis_status\u001b[49m\u001b[43m(\u001b[49m\u001b[43mresponse_dict\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mget\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mStatus\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m{\u001b[49m\u001b[43m}\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/git/github/CorrelAid/pystatis/src/pystatis/http_helper.py:305\u001b[0m, in \u001b[0;36m_check_destatis_status\u001b[0;34m(destatis_status)\u001b[0m\n\u001b[1;32m    303\u001b[0m     \u001b[38;5;28;01mpass\u001b[39;00m\n\u001b[1;32m    304\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m destatis_status_code \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m50\u001b[39m:\n\u001b[0;32m--> 305\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m NoNewerDataError(destatis_status_content) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m    306\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m    307\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m DestatisStatusError(destatis_status_content) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n",
      "\u001b[0;31mNoNewerDataError\u001b[0m: Keine aktualisierten Daten vorhanden. (Mindestens ein Parameter enthält ungültige Werte. Er wurde angepasst, um den Service starten zu können.: stand)"
     ]
    }
   ],
   "source": [
    "t.get_data(stand=\"01.01.2024\")  # after updated date, so error\n",
    "t.data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1040396",
   "metadata": {},
   "source": [
    "### `language`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43f5a88b",
   "metadata": {},
   "source": [
    "`language` can either be \"de\" or \"en, with \"de\" being the default, obviously. Regionalstatistik is not supporting \"en\" and will not translate any data, Genesis and Zensus have some support for English, but you have to check for yourself, if the data is translated and to what extend."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a714ddf8",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Jahr</th>\n",
       "      <th>Deutschland insgesamt</th>\n",
       "      <th>Preisbasis</th>\n",
       "      <th>Bruttowertschoepfung__jew._ME</th>\n",
       "      <th>Guetersteuern_abzuegl._Guetersubventionen__jew._ME</th>\n",
       "      <th>Guetersteuern__jew._ME</th>\n",
       "      <th>Guetersubventionen__jew._ME</th>\n",
       "      <th>Bruttoinlandsprodukt__jew._ME</th>\n",
       "      <th>nachr.:_Bruttoinlandsprodukt_(Veraenderung_in_%)__Prozent</th>\n",
       "      <th>nachr.:_Bruttoinlandsprodukt_je_Einwohner__jew._ME</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2014</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>in jeweiligen Preisen (Mrd. EUR)</td>\n",
       "      <td>2635.393</td>\n",
       "      <td>292.037</td>\n",
       "      <td>298.774</td>\n",
       "      <td>6.737</td>\n",
       "      <td>2927.43</td>\n",
       "      <td>4.1</td>\n",
       "      <td>36149.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Jahr Deutschland insgesamt                        Preisbasis  Bruttowertschoepfung__jew._ME  Guetersteuern_abzuegl._Guetersubventionen__jew._ME  Guetersteuern__jew._ME  Guetersubventionen__jew._ME  Bruttoinlandsprodukt__jew._ME  nachr.:_Bruttoinlandsprodukt_(Veraenderung_in_%)__Prozent  nachr.:_Bruttoinlandsprodukt_je_Einwohner__jew._ME\n",
       "0  2014           Deutschland  in jeweiligen Preisen (Mrd. EUR)                       2635.393                                             292.037                 298.774                        6.737                        2927.43                                                        4.1                                             36149.0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"81000-0001\")\n",
    "t.get_data()\n",
    "t.data.head(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "49f4d094",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: successfull\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Germany</th>\n",
       "      <th>Price base</th>\n",
       "      <th>Gross_value_added__jew._ME</th>\n",
       "      <th>Taxes_on_products_less_subsidies__jew._ME</th>\n",
       "      <th>Taxes_on_products__jew._ME</th>\n",
       "      <th>Subsidies_on_products__jew._ME</th>\n",
       "      <th>Gross_domestic_product__jew._ME</th>\n",
       "      <th>Mem._item:_Gross_domestic_product_(change_in_%)__percent</th>\n",
       "      <th>Mem._item:_Gross_domestic_product_per_inhabitant__jew._ME</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2014</td>\n",
       "      <td>Germany</td>\n",
       "      <td>At current prices (bn EUR)</td>\n",
       "      <td>2635.393</td>\n",
       "      <td>292.037</td>\n",
       "      <td>298.774</td>\n",
       "      <td>6.737</td>\n",
       "      <td>2927.43</td>\n",
       "      <td>4.1</td>\n",
       "      <td>36149.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Year  Germany                  Price base  Gross_value_added__jew._ME  Taxes_on_products_less_subsidies__jew._ME  Taxes_on_products__jew._ME  Subsidies_on_products__jew._ME  Gross_domestic_product__jew._ME  Mem._item:_Gross_domestic_product_(change_in_%)__percent  Mem._item:_Gross_domestic_product_per_inhabitant__jew._ME\n",
       "0  2014  Germany  At current prices (bn EUR)                    2635.393                                    292.037                     298.774                           6.737                          2927.43                                                       4.1                                                    36149.0"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"81000-0001\")\n",
    "t.get_data(language=\"en\")\n",
    "t.data.head(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6fa9de37",
   "metadata": {},
   "source": [
    "### `quality`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "762e1744",
   "metadata": {},
   "source": [
    "`quality` can be either \"on\" or \"off\", with \"off\" being the default. When switching to \"on\", the downloaded table has additional quality columns \"__q\" for each value column with quality symbols. Check [Explanation of symbols](https://www-genesis.destatis.de/genesis/online?operation=ergebnistabelleQualitaet&language=en&levelindex=3&levelid=1719342760835#abreadcrumb.) Not supported for all tables or databases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f3467e20",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Jahr</th>\n",
       "      <th>Deutschland insgesamt</th>\n",
       "      <th>Beschäftigtengrößenklassen</th>\n",
       "      <th>WZ2008 (Abschnitte): URS</th>\n",
       "      <th>Unternehmen_(EU)__Anzahl</th>\n",
       "      <th>Unternehmen_(EU)__q</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2021</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>0 bis unter 10 abhängig Beschäftigte</td>\n",
       "      <td>Bergbau und Gewinnung von Steinen und Erden</td>\n",
       "      <td>1065</td>\n",
       "      <td>e</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Jahr Deutschland insgesamt            Beschäftigtengrößenklassen                     WZ2008 (Abschnitte): URS  Unternehmen_(EU)__Anzahl Unternehmen_(EU)__q\n",
       "0  2021           Deutschland  0 bis unter 10 abhängig Beschäftigte  Bergbau und Gewinnung von Steinen und Erden                      1065                   e"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"52111-0001\")\n",
    "t.get_data(quality=\"on\")\n",
    "t.data.head(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "955f782d",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Jahr</th>\n",
       "      <th>Amtlicher Gemeindeschlüssel (AGS)</th>\n",
       "      <th>Kreise und kreisfreie Städte</th>\n",
       "      <th>Art der Lebensform</th>\n",
       "      <th>Lebensformen_am_Hauptwohnort__1000</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2019</td>\n",
       "      <td>DG</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>Insgesamt</td>\n",
       "      <td>42059.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Jahr Amtlicher Gemeindeschlüssel (AGS) Kreise und kreisfreie Städte Art der Lebensform  Lebensformen_am_Hauptwohnort__1000\n",
       "0  2019                                DG                  Deutschland          Insgesamt                             42059.0"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"12211-Z-11\")\n",
    "t.get_data(quality=\"on\")  # not supported, ignored, but also no warning\n",
    "t.data.head(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f5700bc3",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pystatis.cache:Data was successfully cached under /Users/miay/.pystatis/data/1000A-0000/ec488574ddc014f6340b/20240713.zip.\n",
      "INFO:pystatis.http_helper:Code 0: erfolgreich\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Stichtag</th>\n",
       "      <th>Amtlicher Regionalschlüssel (ARS)</th>\n",
       "      <th>Gemeinden (Gebietsstand 15.05.2022)</th>\n",
       "      <th>Personen__Anzahl</th>\n",
       "      <th>Personen__Anzahl__q</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022-05-15</td>\n",
       "      <td>DG</td>\n",
       "      <td>Deutschland</td>\n",
       "      <td>82719540</td>\n",
       "      <td>e</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Stichtag Amtlicher Regionalschlüssel (ARS) Gemeinden (Gebietsstand 15.05.2022)  Personen__Anzahl Personen__Anzahl__q\n",
       "0  2022-05-15                                DG                         Deutschland          82719540                   e"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Table(\"1000A-0000\")\n",
    "t.get_data(quality=\"on\")\n",
    "t.data.head(1)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.7 ('.venv': poetry)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  },
  "vscode": {
   "interpreter": {
    "hash": "02e23b522f8c3795158421909d41ced4ef90521258d58d1c53bee449d96f71e3"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}