{ "cells": [ { "cell_type": "markdown", "id": "8d207bc3", "metadata": { "id": "21P-6caix_ma" }, "source": [ "# III. Harmonizing Year Name Formulae and Processing Dates\n", "\n", "We now will convert the date field to a series of numerical entries representing the year, month, and day, as well as various special values, recorded on the tablet. This will allow us to perform time analysis on the data." ] }, { "cell_type": "code", "execution_count": null, "id": "2a604c9e", "metadata": { "id": "sH8jC6z00vB9" }, "outputs": [], "source": [ "# import necessary libraries\n", "import pandas as pd" ] }, { "cell_type": "code", "execution_count": null, "id": "b3fd08d8", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 946 }, "id": "78n2_xd41bv1", "outputId": "064e5962-eb58-4143-fff8-8bc68bd94fa0" }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
date_of_origindates_referencedcollectionprimary_publicationmuseum_noproveniencemetadata_source
P100041SSXX - 00 - 00SSXX - 00 - 00Louvre Museum, Paris, FranceAAS 053AO 20313Puzriš-DagānBDTNS
P100189SH46 - 08 - 05SH46 - 08 - 05Louvre Museum, Paris, FranceAAS 211AO 20039Puzriš-DagānBDTNS
P100190SH47 - 07 - 29SH47 - 07 - 29Louvre Museum, Paris, FranceAAS 212AO 20051Puzriš-DagānBDTNS
P100191AS01 - 03 - 24AS01 - 03 - 24Louvre Museum, Paris, FranceAAS 213AO 20074Puzriš-DagānBDTNS
P100211AS01 - 12 - 11AS01 - 12 - 11Museum of Fine Arts, Budapest, HungaryActSocHun Or 5-12, 156 2MHBA 51.2400Puzriš-DagānBDTNS
........................
P456164NaNNaNNaNCDLI Seals 003454 (composite)NaNPuzriš-Dagan (mod. Drehem)ORACC
P459158Ibbi-Suen.00.00.00Ibbi-Suen.00.00.00private: anonymous, unlocatedCDLI Seals 006338 (physical)Anonymous 459158Puzriš-Dagan (mod. Drehem)ORACC
P481391SH46 - 02 - 24SH46 - 02 - 24Department of Classics, University of Cincinna...unpublished unassigned ?UC CSC 1950Puzriš-DagānBDTNS
P481395SS02 - 02 - 00SS02 - 02 - 00Department of Classics, University of Cincinna...unpublished unassigned ?UC CSC 1954Puzriš-DagānBDTNS
P517012NaNNaNNaNCDLI Seals 013964 (composite)NaNPuzriš-Dagan (mod. Drehem)ORACC
\n", "

15671 rows × 7 columns

\n", "
" ], "text/plain": [ " date_of_origin ... metadata_source\n", "P100041 SSXX - 00 - 00 ... BDTNS\n", "P100189 SH46 - 08 - 05 ... BDTNS\n", "P100190 SH47 - 07 - 29 ... BDTNS\n", "P100191 AS01 - 03 - 24 ... BDTNS\n", "P100211 AS01 - 12 - 11 ... BDTNS\n", "... ... ... ...\n", "P456164 NaN ... ORACC\n", "P459158 Ibbi-Suen.00.00.00 ... ORACC\n", "P481391 SH46 - 02 - 24 ... BDTNS\n", "P481395 SS02 - 02 - 00 ... BDTNS\n", "P517012 NaN ... ORACC\n", "\n", "[15671 rows x 7 columns]" ] }, "execution_count": 9, "metadata": { "tags": [] }, "output_type": "execute_result" } ], "source": [ "words_df = pd.read_pickle('https://gitlab.com/yashila.bordag/sumnet-data/-/raw/main/part_2_output.p') # uncomment to read from online file\n", "#words_df = pd.read_pickle('output/part_2_output.p') #uncomment to read from local file\n", "\n", "catalogue_data = pd.read_pickle('https://gitlab.com/yashila.bordag/sumnet-data/-/raw/main/part_1_catalogue.p')\n", "#catalogue_data = pd.read_pickle('output/part_1_catalogue.p')" ] }, { "cell_type": "markdown", "id": "ee7430ef", "metadata": { "id": "ZeF5OQ06lE4G" }, "source": [ "## 1 Normalizing BDTNS Dates\n", "\n", "In the ORACC metadata the field `date_of_origin` is formatted as `Amar-Suen.05.10.03`, meaning \"5th regnal year of Amar-Suen; month 10, day 3\". The ORACC metadata are directly imported from CDLI. In general, BDTNS metadata tend to be more reliable than CDLI metadata. As such, wherever possible we will use the BDTNS date rather than the CDLI date. \n", "\n", "However, the strings for the dates are formatted differently in BDTNS. If the date were the same as the example given above, in the BDTNS format it would be represented by the string `AS05 - 10 - 03`. Moreover, BDTNS dates can sometimes contain additional information, like if the month was a diri month, or multiple dates on the same tablet. When present, this information could be of value later on, so we attempt to include as much of it as possible. In the following we will write two functions, the first to convert from BDTNS format to several numerical and boolean fields and the second to do the same for the CDLI format." ] }, { "cell_type": "code", "execution_count": null, "id": "9f5b2564", "metadata": { "id": "N5f9Jx0-eKdP" }, "outputs": [], "source": [ "def inner_normalize_bdtns_date(bdtns_date):\n", " # the normalized time each king's reign began, with Ur-Namma's first year\n", " # of rule as year 0.\n", " reigns = {\n", " 'UN' : 0, \n", " 'SH' : 18, \n", " 'AS': 66, \n", " 'SS' : 75, \n", " 'IS' : 84\n", " }\n", " \n", " # Sometimes a date string contains multiple dates seperated by '//'.\n", " # In such a case we set the range of dates to go from the earliest\n", " # date to the latest date.\n", " if '//' in bdtns_date:\n", " dates = [normalize_bdtns_date(date) for date in bdtns_date.split('//')]\n", " return {\n", " 'min_year': min([date['min_year'] for date in dates]),\n", " 'max_year': max([date['max_year'] for date in dates]),\n", " 'min_month': min([date['min_month'] for date in dates]),\n", " 'max_month': max([date['max_month'] for date in dates]),\n", " 'diri_month': any([date['diri_month'] for date in dates]),\n", " 'min_day': min([date['min_day'] for date in dates]),\n", " 'max_day': max([date['max_day'] for date in dates]),\n", " 'questionable': any([date['questionable'] for date in dates]),\n", " 'other_meta_chars': any([date['other_meta_chars'] for date in dates]),\n", " 'multiple_dates': True\n", " }\n", "\n", " # there are several characters used as markers for metatextual information\n", " # we remove them from our string and handle them seperately.\n", " chars = ' d?]m+()l'\n", " reduced_string = bdtns_date.upper()\n", " for c in chars.upper():\n", " reduced_string = reduced_string.replace(c, '')\n", " date_list = reduced_string.split('-')\n", " date_list = ['nan' if 'XX' in s else s for s in date_list]\n", " try:\n", " year = reigns[date_list[0][:2]] + float(date_list[0][-2:])\n", " except KeyError:\n", " year = float('nan')\n", " month = float(date_list[1])\n", " try:\n", " day = float(date_list[2])\n", " except IndexError:\n", " day = float('nan')\n", " return {\n", " 'min_year': year,\n", " 'max_year': year + 1,\n", " 'min_month': month,\n", " 'max_month': month + 1,\n", " 'diri_month': 'd' in bdtns_date,\n", " 'min_day': day,\n", " 'max_day': day + 1,\n", " 'questionable': '?' in bdtns_date,\n", " 'other_meta_chars': any([c in bdtns_date for c in chars[3:]]),\n", " 'multiple_dates': False\n", " }\n", "\n", "# Lastly, we define a wrapper function to catch any errors thrown by our main function.\n", "# This is useful because the string was manually entered, so there are many edge cases,\n", "# each with only a few instances.\n", "def normalize_bdtns_date(bdtns_date):\n", " try:\n", " return inner_normalize_bdtns_date(bdtns_date)\n", " except:\n", " return {\n", " 'min_year' : None,\n", " 'max_year' : None,\n", " 'min_month' : None,\n", " 'max_month' : None,\n", " 'min_day' : None,\n", " 'max_day' : None,\n", " 'diri_month' : None,\n", " 'questionable' : None,\n", " 'other_meta_chars': None,\n", " 'multiple_dates' : None\n", " }" ] }, { "cell_type": "markdown", "id": "2ffdb970", "metadata": { "id": "ZQge9XB4XU08" }, "source": [ "## 2 Normalizing CDLI Date Strings" ] }, { "cell_type": "markdown", "id": "f7bd2d5f", "metadata": { "id": "7VGCnQY7NqpF" }, "source": [ "The next step is to convert the CDLI string representing the date into a numerical format. For months and days, this process is relatively straightforward: so long as the entry is legible we can convert the substring into a number. If it is illegible, we set it to `None`.\n", "\n", "When it comes to the year, however, the process is a bit more involved. Since the year number in the date string represents the year of the current king's reign, it does not by itself give us the absolute year. We account for this by adding the year the king began his reign to the year given in the date. The table below contains the values used to achieve this.\n", "\n", "| king | normalized years | regnal years|\n", "| ----- | ---------------- | ---------- |\n", "| Ur-Namma | 1-18 | 18 |\n", "| Šulgi | 19-66 | 48 |\n", "| Amar-Suen | 67-75 | 9 |\n", "| Šū-Suen | 76-84 | 9 |\n", "| Ibbi-Suen | 85-108 | 24 |" ] }, { "cell_type": "code", "execution_count": null, "id": "436cd8a8", "metadata": { "id": "Pus9Kx-5Uq2O" }, "outputs": [], "source": [ "def normalize_cdli_date(cdli_date):\n", " # the normalized time each king's reign began, with Ur-Namma's first year\n", " # of rule as year 0.\n", " reigns = {'Ur-Namma' : 0, \n", " 'Šulgi' : 18, \n", " 'Amar-Suen': 66, \n", " 'Šū-Suen' : 75, \n", " 'Ibbi-Suen' : 84 \n", " }\n", "\n", " # break if NaN or None (this means the date is illegible)\n", " if type(cdli_date) is not str: \n", " out = {\n", " 'min_year' : None,\n", " 'max_year' : None,\n", " 'min_month' : None,\n", " 'max_month' : None,\n", " 'min_day' : None,\n", " 'max_day' : None,\n", " 'diri_month' : None,\n", " 'questionable' : None,\n", " 'other_meta_chars': None,\n", " 'multiple_dates' : None\n", " }\n", " return out\n", "\n", " decomposed_date = cdli_date.split('.')\n", " decomposed_date[0] = reigns.get(decomposed_date[0])\n", " if decomposed_date[0] is None:\n", " decomposed_date[0] = float('nan')\n", "\n", " # if a section of the year/month/day is illegible we replace that entry with NaN,\n", " # otherwise we convert it to a float.\n", " decomposed_date[1:] = list(map(lambda x: float(x) if x.isdigit() else float('nan'), decomposed_date[1:]))\n", " try:\n", " out = {\n", " 'min_year' : decomposed_date[0] + decomposed_date[1],\n", " 'max_year' : decomposed_date[0] + decomposed_date[1] + 1,\n", " 'min_month' : decomposed_date[2],\n", " 'max_month' : decomposed_date[2] + 1,\n", " 'min_day' : decomposed_date[3],\n", " 'max_day' : decomposed_date[3] + 1,\n", " 'diri_month' : False,\n", " 'questionable' : False,\n", " 'other_meta_chars': False,\n", " 'multiple_dates' : False\n", " }\n", " return out\n", "\n", " except: # due to some edge cases in the formatting, the list can occasionally\n", " # have length shorter than 4. If so, the date is partially illegible\n", " # and can be ignored.\n", " out = {\n", " 'min_year' : None,\n", " 'max_year' : None,\n", " 'min_month' : None,\n", " 'max_month' : None,\n", " 'min_day' : None,\n", " 'max_day' : None,\n", " 'diri_month' : None,\n", " 'questionable' : None,\n", " 'other_meta_chars': None,\n", " 'multiple_dates' : None\n", " }\n", " return out" ] }, { "cell_type": "markdown", "id": "8b27cf16", "metadata": { "id": "ReAKcEhfd41X" }, "source": [ "## 3 Putting it all Together" ] }, { "cell_type": "markdown", "id": "6c3071db", "metadata": { "id": "k_v_rBKiRZiX" }, "source": [ "Now that we have a way to convert our strings in either format, we are ready to apply it to the data. To do this we will\n", "\n", "1. loop through each row of the input DataFrame\n", "2. use the `metadata_source` field to determine the format of the date string\n", "3. apply the appropriate function to the `date_of_origin` field to get a dictionary of our new fields\n", "4. compile the results for each row into a list\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "id": "b9a21463", "metadata": { "id": "veFm7uHmd3NY" }, "outputs": [], "source": [ "def add_normalized_columns(df):\n", " row_list = []\n", " # go through each row of the DataFrame\n", " for index in df.index:\n", " # get the date_of_origin string\n", " date = df['date_of_origin'][index]\n", " # select the appropriate function to convert using metadata_source\n", " if df['metadata_source'][index] == 'BDTNS':\n", " row = normalize_bdtns_date(date)\n", " else:\n", " row = normalize_cdli_date(date)\n", " row['metadata_source'] = df['metadata_source'][index]\n", " # add the result to a list\n", " row_list.append(row)\n", " return row_list" ] }, { "cell_type": "markdown", "id": "2e3c7fe9", "metadata": { "id": "JTVMOoafSDCA" }, "source": [ "Next, we process the dates given in `catalogue_data`. From this we will both add the information to our `words_df` variable, as well as include it in a standalone variable, `time_data`." ] }, { "cell_type": "code", "execution_count": null, "id": "13d83780", "metadata": { "id": "uppJLRuSshRL" }, "outputs": [], "source": [ " result = add_normalized_columns(catalogue_data)\n", " time_data = pd.DataFrame(result, index=catalogue_data.index)\n", " words_df = words_df.merge(time_data, left_on='id_text', right_index=True)" ] }, { "cell_type": "markdown", "id": "febaff24", "metadata": { "id": "KHtnB1DQSkAz" }, "source": [ "At this stage, we can already get a bit more insight into our data using our newly added fields. For example, we can look at the number of tablets over time - even removing cases with uncertain or multiple dates." ] }, { "cell_type": "code", "execution_count": null, "id": "138b0514", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 283 }, "id": "-V8HH0jlWc47", "outputId": "4f5a31a4-101d-4fd0-c8e0-d08bfc88eac8" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAaSklEQVR4nO3de7hddX3n8fcHRCoXueU0jSEYxqY61FHEFLE4U0q8ICigRS6tGBls7PNgi6PPQEQ7aGst3mCo06FNDSW0lIiIhSmpNhO5yIygQVEIAcnQIIm5QSCANwx85o/1O3TnuM51n7332ud8Xs+zn73377f2Wt+9z/md7/ld1tqyTURExFC79TqAiIhopiSIiIiolQQRERG1kiAiIqJWEkRERNRKgoiIiFpJEBERUSsJoockrZf0E0lPSXpM0o2S5pS6KyQ9LenJcrtH0p9L2q/UX1Be95Skn0p6puX5mrKNJd0tabeWY35c0hUtz8+WdF85xhZJKyTt2+WPIgIASe8uv7M/lrRZ0mWS9i91H5X08/I7/rik/yvptTX7uELSTkmzaureJOnW8vu+TdItkk5sqZ8laamkTWWb+yR9TNLenX3nzZQE0Xtvtb0PMAvYAnyupe5TtvcFBoCzgKOA/yNpb9ufsL1Pee0fAN8YfG7711v28SLg9LoDS/ot4BPAGeU4/x74wmS/wYixkPRB4JPAfwX2o/p9fzGwUtLzy2ZfKL/zM4CbgC8O2cfewO8AO4B3Dqk7pWx/JXAwMBP4b8BbS/2BwDeAFwCvLW3iDcD+wEsm+e32hSSIhrD9U+Ba4LC6OtvfAk4EDqJKFmP1KeBjkp5XU/cbVInlO+U4220vs/3kuN9ARBskvRD4GPCHtr9i++e21wOnAnMZ8sfe9k7gKmC2pIGWqt8BHgf+BFjYsn8BFwN/avvztnfYftb2LbZ/v2z2AeBJ4J3l2Nh+2Pa5tr836W+6DyRBNISkvYDTgNuH26b84V4J/Mdx7Po64Ang3TV1dwBvKl3ooyXtOY79Rkym3wR+ier39Tm2nwJWUP0n/5zSo3gX8CjwWEvVQuBqYDnwMkmvLuUvBeZQ/RM2nNcD19l+duJvY2pJgui9f5T0OFWX+A3Ap0fZ/ofAgePYv4E/Bv64pZteVdhfB94OHAHcCDwq6WJJu49j/xGTYQbwSOkZDLWp1AOcWtrLT4DfB04ZfI2kQ4DfBv7B9hZgFVUSgarnPbiv4Rw0Sv20kwTReyfb3p/qv6f3AbdI+pURtp8NbB/PAWyvADYA762p+2fbb6VKOidR9TTeM579R0yCR4AZwwyFzir1ANeU9jITuAd4dct2ZwJrbd9Vnl8F/K6kPah6GoP7Gs6jo9RPO0kQDWH7GdvXAc8Ar6vbRtI+VN3gr0/gEB8GLgD2Gub4z9peBXwNePkE9h/Rjm8AP6Pq0T6n/M6/mao38BzbjwCLgI+2rFZ6F/DvyuqnzVRzDjOA44H7gYep5iiG87+Bt7Wu+pvu8kE0hConAQcAa4fU7VnGUv+Rarz1b8e7f9s3U/3H1Tpxd5Kk0yUdUI5/JPBbjDAPEtEJtndQTVJ/TtJxkvaQNBe4hqr3+3c1r7kf+CpwXlnu+hLgSODwcns58A/Au1x9r8EHqIZaz5L0Qkm7SXqdpCVllxcDLwSWSXoxgKTZZdj1FR178w1W152L7vpfkp6hmit4CFhoe0216ILzJJ0LqNT9E9WY648meKyPsOsf/8eAPwL+B7An1fjrp21fNcH9R0yY7U9JehT4DNUf+yeo/in6Pds/K21iqE9T9Xr3A663fXdrpaRLga9LOtD2tZKeoupNf45qHmNN2Qe2t0v6TeDjwB1lyexGqknvdZP+hvuA8oVBERFRJ0NMERFRKwkiIiJqJUFEREStJIiIiKiVBBHRQZJ+SdI3JX1X0hpJHyvlh0q6Q9I6SV8YepZ7RBM0ehXTjBkzPHfu3F6HEVPQnXfe+YjtgdG3bE+5SNzetp8qZ/TeBpxLtSb/OtvLJf0V8F3bl420r7SH6ISR2kKjz4OYO3cuq1ev7nUYMQVJeqgbxyknaD1Vnu5RbgaOBX63lC8DPgqMmCDSHqITRmoLGWKK6DBJu0u6C9hKdTXe/wc83nJhug1U19iKaJQkiIgOK9fZOpzqS2qOBF421tdKWiRptaTV27Zt61iMEXWSICK6xPbjVN+C9lpg/5Yrlx5MdUmHutcssT3f9vyBgY5PmUTsIgkiooMkDbR8p/ILqL7zYy1VojilbLYQuL43EUYMr9GT1BFTwCyqq4PuTvUP2TW2/0nSvcBySR8HvgMs7WWQEXWSICI6qHyX8atqyh+kmo+IaKwMMUVERK1RE4SkyyVtlXRPS9mnJd0n6XuSvjw4xlrqPlTODr1f0ptayo8rZeskLZ78txIREZNpLD2IK4DjhpStBF5u+xXA94EPAUg6DDgd+PXymv9Z1oDvDvwl1VcHHgacUbaNPjR38Y3P3SKitzrZFkdNELZvBbYPKfuXlpN8bqdapgfVl94vt/0z2/9K9S1MR5bbOtsP2n4aWF62jYiIhpqMOYj/DPxzeTyb6ovBBw2eITpceURENFRbCULSh4GdwKR9h3HOHI2IaIYJJwhJ7wbeQvWF4oOXhN0IzGnZbPAM0eHKf0HOHI2IaIYJJQhJxwHnASfa/nFL1Q3A6ZL2lHQoMA/4JvAtYF65Bv7zqSayb2gv9IiI6KRRT5STdDVwDDBD0gbgQqpVS3sCK6vL3XO77T+wvUbSNcC9VENP59h+puznfcBXgd2By22v6cD7iYiISTJqgrB9Rk3xsJcFsP1nwJ/VlK8AVowruoiI6JmcSR0RQWfPJ+hXSRAREVErCSIiImolQURERK0kiIiIqJUEERERtZIgIiKiVhJERESH9esS2iSIiIiolQQRERG1kiAiIqJWEkRERNRKgoiIiFpJEBERUWvUy31HRExVnVh6OrjP9RedMOn77rb0ICIiolYSRERE1EqCiOggSXMk3STpXklrJJ1byj8qaaOku8rt+F7HGjFU5iAiOmsn8EHb35a0L3CnpJWl7hLbn+lhbBEjSoKI6CDbm4BN5fGTktYCs3sbVcTYZIgpokskzQVeBdxRit4n6XuSLpd0QM8CixhGEkREF0jaB/gS8H7bTwCXAS8BDqfqYXx2mNctkrRa0upt27Z1Ld4ISIKI6DhJe1Alh6tsXwdge4vtZ2w/C/wNcGTda20vsT3f9vyBgYHuBR1BEkRER0kSsBRYa/vilvJZLZu9Dbin27FFjCaT1BGddTRwJnC3pLtK2QXAGZIOBwysB97bm/AihjdqgpB0OfAWYKvtl5eyA4EvAHOpfrlPtf1Y+W/pUuB44MfAu21/u7xmIfCRstuP2142uW8lonls3waopmpFt2OJGK+xDDFdARw3pGwxsMr2PGBVeQ7wZmBeuS2imogbTCgXAq+hGmu9MKs2IiKabdQEYftWYPuQ4pOAwR7AMuDklvIrXbkd2L+Mtb4JWGl7u+3HgJX8YtKJiIgGmegk9cxyAhDAZmBmeTwbeLhluw2lbLjyiIhoqLZXMdk21UTbpMi674iIZphogtgyuEyv3G8t5RuBOS3bHVzKhiv/BVn3HRHRDBNNEDcAC8vjhcD1LeXvUuUoYEcZivoq8EZJB5TJ6TeWsoiIaKixLHO9GjgGmCFpA9VqpIuAaySdDTwEnFo2X0G1xHUd1TLXswBsb5f0p8C3ynZ/YnvoxHdERDTIqAnC9hnDVC2o2dbAOcPs53Lg8nFFFxERPZNLbURERK1caiPGrBNf8B4RzZUeRERE1EqCiIiIWkkQERFRKwkiIiJqJUFEREStJIiIiKiVZa4RER3Q7WXhrcdbf9EJk7LP9CAiIqJWEkRERNRKgoiIiFpJEBERUSsJIiIiaiVBRERErSSIiJg25i6+MVclHockiIiIqJUEERERtZIgIjpI0hxJN0m6V9IaSeeW8gMlrZT0QLk/oNexRgyVBBHRWTuBD9o+DDgKOEfSYcBiYJXtecCq8jyiUZIgIjrI9ibb3y6PnwTWArOBk4BlZbNlwMm9iTBieEkQEV0iaS7wKuAOYKbtTaVqMzCzR2FFDCsJIqILJO0DfAl4v+0nWutsG/Awr1skabWk1du2betCpBH/JgliGspa8O6StAdVcrjK9nWleIukWaV+FrC17rW2l9ieb3v+wMBAdwKOKJIgIjpIkoClwFrbF7dU3QAsLI8XAtd3O7aI0bT1hUGS/gvwHqru8d3AWcAsYDlwEHAncKbtpyXtCVwJvBp4FDjN9vp2jh/t6cQXjMQvOBo4E7hb0l2l7ALgIuAaSWcDDwGn9ii+iGFNOEFImg38EXCY7Z9IugY4HTgeuMT2ckl/BZwNXFbuH7P9q5JOBz4JnNb2O4hoMNu3ARqmekE3Y4kYr3aHmJ4HvEDS84C9gE3AscC1pb51+V7rsr5rgQWl+x1TSOY3IqaOCScI2xuBzwA/oEoMO6iGlB63vbNstoFqzTfl/uHy2p1l+4MmevyIiOisdoaYDqDqFRwKPA58ETiu3YAkLQIWARxyyCHt7i4i4hdMZi93cF/dnsfrRk+9nSGm1wP/anub7Z8D11FNyO1fhpwADgY2lscbgTkApX4/qsnqXWRZX0REM7STIH4AHCVprzKXsAC4F7gJOKVs07p8r3VZ3ynA18oJQhER0UDtzEHcQTXZ/G2qJa67AUuA84EPSFpHNcewtLxkKXBQKf8AuThZRESjtXUehO0LgQuHFD8IHFmz7U+Bd7RzvIiI6J6cSR1tybLWiKkrCSIiImolQUREjGK69pSTICIiolYSRERE1EqCiIiIWkkQERFRKwkiIiJqJUFEREStJIiIiKjV1qU2IiKmo7pzIqbieRLpQURERK0kiIiIqJUhppgUU7F7HTHdpQcRERG1kiAiIqJWEkRERNTKHERETGmZH5u4JIio1dqo1l90Qg8jiYheyRBTRETUSoKI6CBJl0vaKumelrKPStoo6a5yO76XMUYMJwkiorOuAI6rKb/E9uHltqLLMUWMSRJERAfZvhXY3us4IiYiCSKiN94n6XtlCOqAXgcTUScJIqL7LgNeAhwObAI+O9yGkhZJWi1p9bZt27oV37Q2d/GNwy6NHamu3X03UVsJQtL+kq6VdJ+ktZJeK+lASSslPVDuDyjbStJfSFpX/nM6YnLeQkR/sb3F9jO2nwX+BjhyhG2X2J5ve/7AwED3goyg/R7EpcBXbL8MeCWwFlgMrLI9D1hVngO8GZhXbouo/ouKhum3/3D6kaRZLU/fBtwz3LYRvTThE+Uk7Qf8J+DdALafBp6WdBJwTNlsGXAzcD5wEnClbQO3l97HLNubJhx9RMNJupqqPcyQtAG4EDhG0uGAgfXAe3sWYMQI2jmT+lBgG/C3kl4J3AmcC8xs+aO/GZhZHs8GHm55/YZSlgQRU5btM2qKl3Y9kIgJaGeI6XnAEcBltl8F/Ih/G04CoPQWPJ6dZlIuIqIZ2kkQG4ANtu8oz6+lShhbBsdYy/3WUr8RmNPy+oNL2S4yKRcR0QwTHmKyvVnSw5Jeavt+YAFwb7ktBC4q99eXl9xAtfZ7OfAaYEfmHyIidlV3oczBsm5fOLPdq7n+IXCVpOcDDwJnUfVKrpF0NvAQcGrZdgVwPLAO+HHZNiIiGqqtBGH7LmB+TdWCmm0NnNPO8SIiontyJnVERNRKgoiIiFpJEBERUSsJIiIiauU7qWNUuTZTNFWvln+2q65NNfE9pAcRERG1kiAiIqJWEkRERNRKgoiIiFpJEBERUSsJIiIiamWZa0REjSzvTg8iIiKGkQQRERG1kiAiIqJWEkRERNRKgoiIiFpJEBERUSsJIiIiaiVBRERErSSIiIiolQQR0UGSLpe0VdI9LWUHSlop6YFyf0AvY4wYThJERGddARw3pGwxsMr2PGBVeR7ROEkQER1k+1Zg+5Dik4Bl5fEy4OSuBhUxRkkQEd030/am8ngzMLOXwUQMp+2ruUraHVgNbLT9FkmHAsuBg4A7gTNtPy1pT+BK4NXAo8Bptte3e/wYWesVKZv4pejTnW1L8nD1khYBiwAOOeSQrsUVzdDrK8pORg/iXGBty/NPApfY/lXgMeDsUn428Fgpv6RsFzEdbZE0C6Dcbx1uQ9tLbM+3PX9gYKBrAUZAmwlC0sHACcDny3MBxwLXlk1ax1dbx12vBRaU7SOmmxuAheXxQuD6HsYSMax2h5j+O3AesG95fhDwuO2d5fkGYHZ5PBt4GMD2Tkk7yvaPtBlDNFCGtiqSrgaOAWZI2gBcCFwEXCPpbOAh4NTeRRgxvAknCElvAbbavlPSMZMVUMZcYyqxfcYwVQu6GkjEBLQzxHQ0cKKk9VST0scClwL7SxpMPAcDG8vjjcAcgFK/H9Vk9S4y5jp1zV18Y88n3SJi7CacIGx/yPbBtucCpwNfs/17wE3AKWWz1vHV1nHXU8r2w67eiMmXP9ARMR5tL3OtcT6wXNLHge8AS0v5UuDvJK2jOnHo9A4cOyYoiSOaZvB3cjrPYfXapCQI2zcDN5fHDwJH1mzzU+Adk3G8iIjovJxJHRERtZIgIiKiVhJERETU6sQkdcSIchJdRH9IDyIiImqlBxER0Se6vRw9CSL6RoamIrorCSI6LifhRfSnzEFEREStJIiIiKiVBBEREbWSICIiolYSRERE1MoqpohotLEsb84S6M5IgpiipvrS0nxXQETnZYgpIiJqJUFEREStJIiIiKiVBBEREbWSICIiolZWMUVE3xjP6rWpvpKvG9KDiMaau/jGNPKIHkqCiIiIWhliiugRSeuBJ4FngJ225/c2oohdTbgHIWmOpJsk3StpjaRzS/mBklZKeqDcH1DKJekvJK2T9D1JR0zWm4joY79t+/Akh2iidoaYdgIftH0YcBRwjqTDgMXAKtvzgFXlOcCbgXnltgi4rI1jxxSReYaI5prwEJPtTcCm8vhJSWuB2cBJwDFls2XAzcD5pfxK2wZul7S/pFllPxHDmsIJxMC/SDLw17aX9DqgiFaTMgchaS7wKuAOYGbLH/3NwMzyeDbwcMvLNpSyJIiYrl5ne6OkXwZWSrrP9q2tG0haRNXj5pBDDulFjNElTfxHqO1VTJL2Ab4EvN/2E611pbfgce5vkaTVklZv27at3fAiGsv2xnK/FfgycGTNNktsz7c9f2BgoNshxjTXVoKQtAdVcrjK9nWleIukWaV+FrC1lG8E5rS8/OBStos0iJgOJO0tad/Bx8AbgXt6G1XErtpZxSRgKbDW9sUtVTcAC8vjhcD1LeXvKquZjgJ2ZP4hBk3DyeqZwG2Svgt8E7jR9ld6HFPELtqZgzgaOBO4W9JdpewC4CLgGklnAw8Bp5a6FcDxwDrgx8BZbRw7oq/ZfhB4Za/jiBhJO6uYbgM0TPWCmu0NnDPR40VERHflUhsREVErl9qIvpbvpu5vdfNO7f4sp9lcVq3JahfpQURERK0kiIiIqJUEERERtTIHEY2S8eOI5kiCmGLyBzYiJkuGmCIiolZ6EBHRKGPpBaen3B3pQURERK0kiIiIqJUhpikg3e2I6IQkiD6WxBARnZQhpoiIqJUEERERtTLE1IcytBT9YOgVRfN7233tXtU1PYiIiKiVBBEREbWSICIiolbmIPpExm8jotvSg4iIiFpJED00d/GN6RlERGNliKlhsjQwppr8DvevJIgeyOWMI6IfJEE0QJJBRDRR1+cgJB0n6X5J6yQt7vbxI5oibSGarqs9CEm7A38JvAHYAHxL0g227+1mHN2QXkGMZDq1hehf3R5iOhJYZ/tBAEnLgZOANIpoS2tCnuh1Z7osbSEar9sJYjbwcMvzDcBruhxDW+p6Bn3yBymape/bQkx9jZuklrQIWFSePiXp/h6EMQN4ZKwb65MdjGT8xhV7g0xa3GP8ebx4Mo7Vaf3YHhokcRejtIlh20K3E8RGYE7L84NL2XNsLwGWdDOooSSttj2/lzFMVL/G3q9xt2HUtgBpD+1I3O3r9iqmbwHzJB0q6fnA6cANXY4hognSFqLxutqDsL1T0vuArwK7A5fbXtPNGCKaIG0h+kHX5yBsrwBWdPu449TTLn2b+jX2fo17wvqkLUD//mwSd5tku9cxREREA+VqrhERUSsJIiIiaiVBRERErSSIiIio1bgzqbtN0vOAs4G3AS8qxRuB64Gltn/eq9hGImk/4EPAycAvAwa2UsV9ke3HexjeiPr1M5/q+vnn0q/toemf+bRfxSTpauBxYBnV9XCgOqt1IXCg7dN6FdtIJH0V+BqwzPbmUvYrVHEvsP3GXsY3kn79zKe6fv659Gt7aPpnngQhfd/2r423rtck3W/7peOta4J+/cynun7+ufRre2j6Z545CNgu6R2SnvssJO0m6TTgsR7GNZqHJJ0naeZggaSZks5n16uENlG/fuZTXT//XPq1PTT6M0+CqK6BcwqwRdL3JT0AbAbeXuqa6jTgIOAWSY9J2g7cDBwInNrLwMZg8DPfXD7z79Mfn/lU169tAfq3PTS6LUz7IaZWkg4qDy+1/c6eBjMKSa8B7rO9Q9JewGLgCGAN8AnbO3oa4AjKxenOAH4IfBs4DjiaKvYlvZ6Yi/5qC9C/7aHpbWHaJwhJdVfQPJZqwgvbJ3Y3orGRtAZ4Zbno2xLgR8CXgAWl/O09DXAEkq6iWkH3AmAHsDfwZarYZXthD8Obtvq1LUD/toemt4Vpv8yVasXAvcDnqZbGCfgN4LO9DGoMdrO9szyeb/uI8vg2SXf1Kqgx+g+2X1GW+G0EXmT7GUl/D3y3x7FNZ/3aFqB/20Oj20LmIGA+cCfwYWCH7ZuBn9i+xfYtPY1sZPdIOqs8/q6k+QCSfg1o+hDNbqVrvS+wF7BfKd8T2KNnUUW/tgXo3/bQ6LYw7XsQtp8FLpH0xXK/hf74XN4DXCrpI1RfT/gNSQ9Trdh4T08jG91S4D6q70H4MPBFSQ8CRwHLexnYdNbHbQH6tz00ui1M+zmIoSSdABxt+4JexzIWkl4IHErVkDfY3tLjkMZE0osAbP9Q0v7A64Ef2P5mbyOLQf3WFqA/20OT20ISRERE1MocRERE1EqCiIiIWkkQERFRKwkiIiJqJUFERESt/w+HkSXCF+escwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light", "tags": [] }, "output_type": "display_data" } ], "source": [ "ax = time_data[(time_data['questionable'] == False)].hist(column='min_year', by='metadata_source', sharex=True, bins= range(30, 90))" ] }, { "cell_type": "markdown", "id": "494b2c5a", "metadata": { "id": "-53Lkz2mzFe7" }, "source": [ "## 4 Save Results in CSV file and Pickle\n", "Here we will save the `words_df` and `time_data` outputs from parts 1, 2, and 3." ] }, { "cell_type": "code", "execution_count": null, "id": "2242a10b", "metadata": { "id": "SDvcQVykzSh8" }, "outputs": [], "source": [ "words_df.to_csv('output/part_3_words_output.csv')\n", "words_df.to_pickle('output/part_3_words_output.p')\n", "\n", "time_data.to_csv('output/part_3_time_output.csv')\n", "time_data.to_pickle('output/part_3_time_output.p')" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.7" } }, "nbformat": 4, "nbformat_minor": 5 }