# Empirical project 12


## Getting started in Python


Visit the ‘Getting started in Python’ page for help and advice on setting up a Python session to work with. Remember, you can run any page from this book as a *notebook* by downloading the relevant file from this [repository](https://github.com/aeturrell/core_python) and running it on your own computer. Alternatively, you can run pages online in your browser over at [Binder](https://mybinder.org/v2/gh/aeturrell/core_python/HEAD).


### Preliminary settings


Let's import the packages we'll need and also configure the settings we want:


```python
import pandas as pd
import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
from pathlib import Path
import pingouin as pg
from lets_plot import *
from lets_plot.mapping import as_discrete


LetsPlot.setup_html(no_js=True)
```


### *Python walk-through 12.1* Importing a specified range of data from a spreadsheet


We start by importing the data; you will need to download it and put it in a subfolder of your working directory called ‘data’. The data provided in the Excel spreadsheet is not in the usual format of one variable per column (known as ‘long’ format). Instead the first tab contains two separate tables, and we need the second table. We can therefore use the `skiprows=` keyword argument of **pandas**' `pd.read_excel` function to specify the cells in the spreadsheet to import. (Note that variable headers for the years are included.)


```python
income = pd.read_excel(
    Path("data/doing-economics-project-12-datafile.xlsx"), skiprows=12
)


income
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }
.dataframe tbody tr th {
    vertical-align: top;
}
.dataframe thead th {
    text-align: right;
}
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Unnamed: 0</th>
      <th>2009</th>
      <th>2010</th>
      <th>2011</th>
      <th>2012</th>
      <th>2013</th>
      <th>2014</th>
      <th>2015</th>
      <th>2016</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>85th</td>
      <td>43300.0</td>
      <td>43945.31250</td>
      <td>44515.669516</td>
      <td>44544.177789</td>
      <td>45270.209450</td>
      <td>45162.212854</td>
      <td>47659.574562</td>
      <td>47085.549738</td>
    </tr>
    <tr>
      <th>1</th>
      <td>75th</td>
      <td>31000.0</td>
      <td>31250.00000</td>
      <td>32273.860399</td>
      <td>32517.249786</td>
      <td>34166.195812</td>
      <td>33298.950420</td>
      <td>34791.489430</td>
      <td>34906.914962</td>
    </tr>
    <tr>
      <th>2</th>
      <td>50th</td>
      <td>17400.0</td>
      <td>17578.12500</td>
      <td>17806.267806</td>
      <td>17817.671116</td>
      <td>18620.576717</td>
      <td>18490.326277</td>
      <td>19063.829825</td>
      <td>19392.730534</td>
    </tr>
    <tr>
      <th>3</th>
      <td>25th</td>
      <td>8000.0</td>
      <td>8203.12500</td>
      <td>8346.688034</td>
      <td>8819.747202</td>
      <td>8541.548953</td>
      <td>8590.638315</td>
      <td>8737.588670</td>
      <td>8532.801435</td>
    </tr>
    <tr>
      <th>4</th>
      <td>15th</td>
      <td>4500.0</td>
      <td>4394.53125</td>
      <td>4637.048908</td>
      <td>4454.417779</td>
      <td>4356.189966</td>
      <td>4090.780150</td>
      <td>3971.631213</td>
      <td>3878.546107</td>
    </tr>
    <tr>
      <th>5</th>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>6</th>
      <td>Note: Nominal values rounded to the nearest hu...</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
    </tr>
  </tbody>
</table>
</div>


This dataframe still has some rows that we don't need for our analysis (those that are entirely filled with `NaN`s), so we need to get rid of them. There are multiple ways to do this depending on the context.


In this case, the two rows we don't want contain a large number of invalid entries, so we can make use of the row-wise `.dropna()` method to clean up the data. While doing this task, we will also rename the first column (`Unnamed: 0`) to something more relevant.


```python
income = income.dropna(axis=0).rename(columns={"Unnamed: 0": "Percentile"})
income
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }
.dataframe tbody tr th {
    vertical-align: top;
}
.dataframe thead th {
    text-align: right;
}
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Percentile</th>
      <th>2009</th>
      <th>2010</th>
      <th>2011</th>
      <th>2012</th>
      <th>2013</th>
      <th>2014</th>
      <th>2015</th>
      <th>2016</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>85th</td>
      <td>43300.0</td>
      <td>43945.31250</td>
      <td>44515.669516</td>
      <td>44544.177789</td>
      <td>45270.209450</td>
      <td>45162.212854</td>
      <td>47659.574562</td>
      <td>47085.549738</td>
    </tr>
    <tr>
      <th>1</th>
      <td>75th</td>
      <td>31000.0</td>
      <td>31250.00000</td>
      <td>32273.860399</td>
      <td>32517.249786</td>
      <td>34166.195812</td>
      <td>33298.950420</td>
      <td>34791.489430</td>
      <td>34906.914962</td>
    </tr>
    <tr>
      <th>2</th>
      <td>50th</td>
      <td>17400.0</td>
      <td>17578.12500</td>
      <td>17806.267806</td>
      <td>17817.671116</td>
      <td>18620.576717</td>
      <td>18490.326277</td>
      <td>19063.829825</td>
      <td>19392.730534</td>
    </tr>
    <tr>
      <th>3</th>
      <td>25th</td>
      <td>8000.0</td>
      <td>8203.12500</td>
      <td>8346.688034</td>
      <td>8819.747202</td>
      <td>8541.548953</td>
      <td>8590.638315</td>
      <td>8737.588670</td>
      <td>8532.801435</td>
    </tr>
    <tr>
      <th>4</th>
      <td>15th</td>
      <td>4500.0</td>
      <td>4394.53125</td>
      <td>4637.048908</td>
      <td>4454.417779</td>
      <td>4356.189966</td>
      <td>4090.780150</td>
      <td>3971.631213</td>
      <td>3878.546107</td>
    </tr>
  </tbody>
</table>
</div>


Let’s make a quick plot of this data. We'll need to re-orient the data if we want to use the **lets-plot** plotting package (because it expects long-format data). We'll use `pd.melt` to do this re-orientation.


```python
income_melted = pd.melt(
    income, id_vars="Percentile", value_vars=range(2009, 2016), var_name="Year"
)
```


```python
(
    ggplot(income_melted, aes(x=as_discrete("Year"), y="value", color="Percentile"))
    + geom_line(size=2)
    + labs(y="Monthly real household income (HKD)")
    + scale_x_continuous(format="d")
)
```


![png](empirical_project_12_files/empirical_project_12_WT1_1.png)


**Figure 12.1** Monthly real household income over time.


However, plotting all percentile groups using the same scale hides much of the variation within each percentile group. So, we should create a separate chart for each percentile group. As an example, we will plot the chart for the 15th percentile.


First, we use the `pd.melt` function to reshape the data into the format that **lets-plot** uses to plot charts. Then we use the `.loc` function to select data for the 15th percentile only. Finally, we make the line chart as before.


```python
perct_to_use = "15th"


(
    ggplot(
        income_melted.loc[income_melted["Percentile"] == perct_to_use, :],
        aes(x=as_discrete("Year"), y="value"),
    )
    + geom_line(size=1)
    + geom_point(size=3)
    + labs(
        title=f"Monthly real household income for {perct_to_use} percentile",
        y="Household income (HKD)",
    )
    + scale_x_continuous(format="d")
)
```


![png](empirical_project_12_files/empirical_project_12_WT1_2.png)


**Figure 12.2** Monthly real household income for the 15th percentile.


### *Python walk-through 12.2* Calculating cumulative income shares and plotting a Lorenz curve


Questions 2(a)–(c) can be completed in a few steps. To do them, we're going to use several techniques.


First, we use the `.loc` method to get the relevant variables. Remember that the syntax is `.loc[rows, columns]`, so we can create a list of just the columns we want to work with and use `:` to get all rows. Then we're going to use the `assign` function. This function allows us to change dataframes *in-line*, that is, without having to disrupt the flow, and within a *chained set of methods*. As it's an important point, let's take a quick look at this idea in more detail.


Say we had a dataframe, `df`, with a column `number` that just comprised the numbers from 1 to 10. Imagine we wanted to create a second column called `number_add_one`, which comprises the numbers 2 to 11. There are two ways to do this task using `df`. The first is to explicitly create the new column:


```python
df["number_add_one"] = df["number"] + 1
```


The second is to use `assign`:


```python
df = (
    df.assign(
        number_add_one=lambda x: x["number"] + 1
    )
)
```


Neither of these ways is inherently better than the other, they're just different. However, if you're doing many steps in a row, multiple `assign` statements can make the code more readable. As a reminder, `lambda` functions are functions that you don't have to give a name to. They work by defining a variable, by convention `x`, but you could call it anything. In the case of a dataframe assign statement, writing `lambda x:` means that `x` stands in for the dataframe itself, so that writing `lambda x: x["number"]` refers to the ‘number’ column in the dataframe `x`.


Returning to the task at hand: we're first going to remove the ‘th’ suffix in the percentile column values using `.str.split("th", expand=True)[0]` to split all the strings, expand the resulting list into two columns and, with `[0]`, only take the first part (before the ‘th’). The variable containing percentiles is then cast as an integer type.


```python
cols_of_interest = ["Percentile", 2011, 2012]


percentiles = income.loc[:, cols_of_interest].assign(
    Percentile=lambda x: x["Percentile"].str.split("th", expand=True)[0].astype("int")
)
percentiles
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }
.dataframe tbody tr th {
    vertical-align: top;
}
.dataframe thead th {
    text-align: right;
}
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Percentile</th>
      <th>2011</th>
      <th>2012</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>85</td>
      <td>44515.669516</td>
      <td>44544.177789</td>
    </tr>
    <tr>
      <th>1</th>
      <td>75</td>
      <td>32273.860399</td>
      <td>32517.249786</td>
    </tr>
    <tr>
      <th>2</th>
      <td>50</td>
      <td>17806.267806</td>
      <td>17817.671116</td>
    </tr>
    <tr>
      <th>3</th>
      <td>25</td>
      <td>8346.688034</td>
      <td>8819.747202</td>
    </tr>
    <tr>
      <th>4</th>
      <td>15</td>
      <td>4637.048908</td>
      <td>4454.417779</td>
    </tr>
  </tbody>
</table>
</div>


In the next step, we're going to concatenate a dataframe with three zeros to form the zeroth percentile entries for 2011 and 2012: we need this step to ensure our data includes 0.


```python
percentiles = pd.concat(
    [
        percentiles,
        pd.DataFrame([[0, 0, 0]], columns=cols_of_interest, index=[5], dtype="float"),
    ],
    axis=0,
).sort_values(by="Percentile")
percentiles
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }
.dataframe tbody tr th {
    vertical-align: top;
}
.dataframe thead th {
    text-align: right;
}
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Percentile</th>
      <th>2011</th>
      <th>2012</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>5</th>
      <td>0.0</td>
      <td>0.000000</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>4</th>
      <td>15.0</td>
      <td>4637.048908</td>
      <td>4454.417779</td>
    </tr>
    <tr>
      <th>3</th>
      <td>25.0</td>
      <td>8346.688034</td>
      <td>8819.747202</td>
    </tr>
    <tr>
      <th>2</th>
      <td>50.0</td>
      <td>17806.267806</td>
      <td>17817.671116</td>
    </tr>
    <tr>
      <th>1</th>
      <td>75.0</td>
      <td>32273.860399</td>
      <td>32517.249786</td>
    </tr>
    <tr>
      <th>0</th>
      <td>85.0</td>
      <td>44515.669516</td>
      <td>44544.177789</td>
    </tr>
  </tbody>
</table>
</div>


Next, we're going to do a number of operations on the columns using `.assign`, including:


- adding a column representing the number of households in each percentile group (assuming 100 households in the economy)
- creating a new variable called `handout_2012` that adds $6,000 to each value for the year 2012
- converting monthly to annual data (multiplying the 2011 and 2012 values by 12)
- adding the economy-wide income for each percentile group, derived from multiplying the income values by the number of households, and storing these in the variables `income_2011` and `income_2012`
- creating the normalised cumulative income (as a percentage) for each income group.


```python
percentiles = percentiles.assign(
    households=[15, 10, 25, 25, 10, 15],
    handout_2012=lambda x: x[2012]*12 + 6000,
    handout_2011=lambda x: x[2011]*12,
    income_2011=lambda x: x["handout_2011"] * x["households"],
    income_2012=lambda x: x["handout_2012"] * x["households"],
    rel_share_2011=lambda x: 100 * x["handout_2011"].cumsum() / x["handout_2011"].sum(),
    rel_share_2012=lambda x: 100 * x["handout_2012"].cumsum() / x["handout_2012"].sum(),
    Percentile=lambda x: x["households"] + x["Percentile"],
).loc[:, ["Percentile", "rel_share_2011", "rel_share_2012"]]
percentiles
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }


    .dataframe tbody tr th {
        vertical-align: top;
    }


    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Percentile</th>
      <th>rel_share_2011</th>
      <th>rel_share_2012</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>5</th>
      <td>15.0</td>
      <td>0.000000</td>
      <td>0.427639</td>
    </tr>
    <tr>
      <th>4</th>
      <td>25.0</td>
      <td>2.742732</td>
      <td>3.252577</td>
    </tr>
    <tr>
      <th>3</th>
      <td>50.0</td>
      <td>15.085025</td>
      <td>16.537542</td>
    </tr>
    <tr>
      <th>2</th>
      <td>75.0</td>
      <td>41.415250</td>
      <td>42.648724</td>
    </tr>
    <tr>
      <th>1</th>
      <td>85.0</td>
      <td>60.504663</td>
      <td>61.474686</td>
    </tr>
    <tr>
      <th>0</th>
      <td>100.0</td>
      <td>100.000000</td>
      <td>100.000000</td>
    </tr>
  </tbody>
</table>
</div>


Finally, we tidy up the data by ensuring that the 0th percentile has no share of the income and the 100th percentile has 100% of the cumulative income. We do this by adding a row of zeros by concatenating another empty dataframe to `percentiles`. Finally, we sort by the values again.


```python
income_shares = pd.concat(
    [
        percentiles,
        pd.DataFrame(
            [[0, 0, 0]],
            columns=["Percentile", "rel_share_2011", "rel_share_2012"],
            index=[6],
        ),
    ],
    axis=0,
).sort_values(by="Percentile")


income_shares.round(2)
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }


    .dataframe tbody tr th {
        vertical-align: top;
    }


    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Percentile</th>
      <th>rel_share_2011</th>
      <th>rel_share_2012</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>6</th>
      <td>0.0</td>
      <td>0.00</td>
      <td>0.00</td>
    </tr>
    <tr>
      <th>5</th>
      <td>15.0</td>
      <td>0.00</td>
      <td>0.43</td>
    </tr>
    <tr>
      <th>4</th>
      <td>25.0</td>
      <td>2.74</td>
      <td>3.25</td>
    </tr>
    <tr>
      <th>3</th>
      <td>50.0</td>
      <td>15.09</td>
      <td>16.54</td>
    </tr>
    <tr>
      <th>2</th>
      <td>75.0</td>
      <td>41.42</td>
      <td>42.65</td>
    </tr>
    <tr>
      <th>1</th>
      <td>85.0</td>
      <td>60.50</td>
      <td>61.47</td>
    </tr>
    <tr>
      <th>0</th>
      <td>100.0</td>
      <td>100.00</td>
      <td>100.00</td>
    </tr>
  </tbody>
</table>
</div>


With the data from Questions 2(a)–(c), we can plot the Lorenz curve using the **lets-plot** package. Note that we use the `Percentile` variable to draw the line of perfect equality.


We need to have data in long format for this, so we begin by doing a `melt`:


```python
long_incomes_shares = pd.melt(
    income_shares,
    id_vars="Percentile",
    var_name="year",
    value_name="Cumulative share of income (%)",
)
```


```python
# rename the legend labels
long_incomes_shares = long_incomes_shares.replace({"rel_share_2011": "2011", "rel_share_2012": "2012"})
(
    ggplot(
        long_incomes_shares,
        aes(x="Percentile", y="Cumulative share of income (%)", color="Year"),
    )
    + geom_abline(slope=1, color="black", linetype=2, alpha=0.7)
    + geom_line(size=1)
    + geom_point(size=3)
    + labs(x="Cumulative share of the population (%)")
)
```


![png](empirical_project_12_files/empirical_project_12_WT2_1.png)


**Figure 12.4** Lorenz curves (2011 and 2012). 


### *Python walk-through 12.3* Generating Gini coefficients


#### Create a table containing percentiles


To create the 2011, 2012, and 2013 percentiles for every percentile in our 100-household economy, we need to take the income for each percentile group and expand that for every household in the respective percentile group. For example, there are 15 households in the bottom percentile group that have zero income for 2011 and 2013, and $6,000 in 2012. For the 15th percentile group, there are 15 households that will share the same income value, and so on for the other percentile groups.


To achieve this expansion, there are a few steps. First, we're going to select the years we're interested in and add in a row of zeros (for the zero percentiles). Then we’ll convert monthly to annual incomes by multiplying all values by 12. Note that the values in the dataframe are incomes.


```python
years_of_interest = [2011, 2012, 2013]
raw_percentiles = income.loc[:, years_of_interest].sort_values(2011)


raw_percentiles[2011] = raw_percentiles[2011]*12
raw_percentiles[2012] = raw_percentiles[2012]*12
raw_percentiles[2013] = raw_percentiles[2013]*12


raw_percentiles = pd.concat(
    [
        pd.DataFrame(
            [[0, 0, 0]],
            columns=years_of_interest,
            index=[6],
        ),
        raw_percentiles,
    ],
).reset_index(drop=True)
raw_percentiles
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }


    .dataframe tbody tr th {
        vertical-align: top;
    }


    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>2011</th>
      <th>2012</th>
      <th>2013</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0.000000</td>
      <td>0.000000</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>1</th>
      <td>55644.586895</td>
      <td>53453.013347</td>
      <td>52274.279592</td>
    </tr>
    <tr>
      <th>2</th>
      <td>100160.256410</td>
      <td>105836.966428</td>
      <td>102498.587435</td>
    </tr>
    <tr>
      <th>3</th>
      <td>213675.213675</td>
      <td>213812.053389</td>
      <td>223446.920608</td>
    </tr>
    <tr>
      <th>4</th>
      <td>387286.324786</td>
      <td>390206.997436</td>
      <td>409994.349740</td>
    </tr>
    <tr>
      <th>5</th>
      <td>534188.034188</td>
      <td>534530.133473</td>
      <td>543242.513405</td>
    </tr>
  </tbody>
</table>
</div>


Next, we create a list of the numbers of households in each group (`households = [15, 10, 25, 25, 10, 15]`) that will be matched with the six different percentiles we have available in the `raw_percentiles` dataframe. Then we use two `for` loops:


- An inner `for` loop goes over each number in the households list above (the variable is called `num_hhds`), takes the corresponding income value (`num_income`) and repeats it `num_hhds` times. We achieve this repetition using `np.repeat` from the **numpy** package. Each one of these repetitions is concatenated until we have a dataframe with an income value for all 100 percentiles for a specific year.
- An outer `for` loop that goes over the years.


All the results from both dataframes are combined together. We do a couple of other useful things too:


- `hh_percentiles_by_year.index % 100 + 1` creates the percentiles from 1 to 100. The `%` symbol represents the modulo operator which gives the remainder after division.
- For 2012, we add in the 6000 HKD extra payment.


Note that **pandas** uses `0` as its default column name, which is why it appears below.


```python
households = [15, 10, 25, 25, 10, 15]


hh_percentiles = pd.DataFrame()


for year in raw_percentiles.columns:
    hh_percentiles_by_year = pd.DataFrame()
    for i, num_hhds in enumerate(households):
        num_income = raw_percentiles.loc[i, year]
        hh_percentiles_by_year = pd.concat(
            [hh_percentiles_by_year, pd.DataFrame(np.repeat(num_income, num_hhds))],
            axis=0,
        )
        hh_percentiles_by_year["year"] = year


    hh_percentiles_by_year = hh_percentiles_by_year.reset_index(drop=True)
    hh_percentiles_by_year["percentile"] = hh_percentiles_by_year.index % 100 + 1
    if year == 2012:
        hh_percentiles_by_year[0] = hh_percentiles_by_year[0] + 6000


    hh_percentiles = pd.concat([hh_percentiles, hh_percentiles_by_year], axis=0)


hh_percentiles.tail()
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }


    .dataframe tbody tr th {
        vertical-align: top;
    }


    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>0</th>
      <th>year</th>
      <th>percentile</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>95</th>
      <td>543242.513405</td>
      <td>2013</td>
      <td>96</td>
    </tr>
    <tr>
      <th>96</th>
      <td>543242.513405</td>
      <td>2013</td>
      <td>97</td>
    </tr>
    <tr>
      <th>97</th>
      <td>543242.513405</td>
      <td>2013</td>
      <td>98</td>
    </tr>
    <tr>
      <th>98</th>
      <td>543242.513405</td>
      <td>2013</td>
      <td>99</td>
    </tr>
    <tr>
      <th>99</th>
      <td>543242.513405</td>
      <td>2013</td>
      <td>100</td>
    </tr>
  </tbody>
</table>
</div>


We can now `pivot` the table we created to have columns as years and percentiles as the index:


```python
hh_percentiles = hh_percentiles.pivot(columns="year", index=["percentile"], values=0)
hh_percentiles
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }


    .dataframe tbody tr th {
        vertical-align: top;
    }


    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th>year</th>
      <th>2011</th>
      <th>2012</th>
      <th>2013</th>
    </tr>
    <tr>
      <th>percentile</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>1</th>
      <td>0.000000</td>
      <td>6000.000000</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0.000000</td>
      <td>6000.000000</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>3</th>
      <td>0.000000</td>
      <td>6000.000000</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>4</th>
      <td>0.000000</td>
      <td>6000.000000</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>5</th>
      <td>0.000000</td>
      <td>6000.000000</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>96</th>
      <td>534188.034188</td>
      <td>540530.133473</td>
      <td>543242.513405</td>
    </tr>
    <tr>
      <th>97</th>
      <td>534188.034188</td>
      <td>540530.133473</td>
      <td>543242.513405</td>
    </tr>
    <tr>
      <th>98</th>
      <td>534188.034188</td>
      <td>540530.133473</td>
      <td>543242.513405</td>
    </tr>
    <tr>
      <th>99</th>
      <td>534188.034188</td>
      <td>540530.133473</td>
      <td>543242.513405</td>
    </tr>
    <tr>
      <th>100</th>
      <td>534188.034188</td>
      <td>540530.133473</td>
      <td>543242.513405</td>
    </tr>
  </tbody>
</table>
<p>100 rows × 3 columns</p>
</div>


Now we can compute the Gini coefficients for the different years. First we define a function that computes a Gini, given a set of values. (Here we use the exact formula for the Gini coefficient.)


```python
def gini_coefficient(x):
    """Compute Gini coefficient of array of values"""
    x = np.double(x.values)
    x = x / x.sum()
    # Mean absolute difference
    mad = np.abs(np.subtract.outer(x, x)).mean()
    # Relative mean absolute difference
    rmad = mad / np.mean(x)
    # Gini coefficient
    g = 0.5 * rmad
    return g
```


Second, we `apply` this function to each column separately (using `axis=0`) to get our Gini values.


```python
hh_percentiles.apply(lambda x: gini_coefficient(x), axis=0)
```


```
year
2011    0.468760
2012    0.451902
2013    0.469819
dtype: float64
```


## *Extension Python walk-through 12.4* Converting nominal incomes to real incomes


To obtain the real income values, we need to divide the income for each percentile group by the inflation index created in Question 5(a). Recall that we have imported only the real income data from the Excel spreadsheet and not the nominal income data, so we will first import the nominal income data (`nom_income`). Note that, in the code below, the inflation index is entered as a vector (list of numbers) called `inflation` (with the same number of elements as the number of years in the data) and this is multiplied (element-wise) by each row of the income data using the \[todo\] method.


```python
# Import the nominal income data, drop everything beyond first table,
# transpose, and set Percentile as the index
nom_income = (
    pd.read_excel(
        Path("data/doing-economics-project-12-datafile.xlsx"),
        skiprows=2,
    )
    .loc[:4, :]
    .set_index("Percentile")
    .T
)


nom_income
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }
.dataframe tbody tr th {
    vertical-align: top;
}
.dataframe thead th {
    text-align: right;
}
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th>Percentile</th>
      <th>85th</th>
      <th>75th</th>
      <th>50th</th>
      <th>25th</th>
      <th>15th</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>2009</th>
      <td>43300.0</td>
      <td>31000.0</td>
      <td>17400.0</td>
      <td>8000.0</td>
      <td>4500.0</td>
    </tr>
    <tr>
      <th>2010</th>
      <td>45000.0</td>
      <td>32000.0</td>
      <td>18000.0</td>
      <td>8400.0</td>
      <td>4500.0</td>
    </tr>
    <tr>
      <th>2011</th>
      <td>48000.0</td>
      <td>34800.0</td>
      <td>19200.0</td>
      <td>9000.0</td>
      <td>5000.0</td>
    </tr>
    <tr>
      <th>2012</th>
      <td>50000.0</td>
      <td>36500.0</td>
      <td>20000.0</td>
      <td>9900.0</td>
      <td>5000.0</td>
    </tr>
    <tr>
      <th>2013</th>
      <td>53000.0</td>
      <td>40000.0</td>
      <td>21800.0</td>
      <td>10000.0</td>
      <td>5100.0</td>
    </tr>
    <tr>
      <th>2014</th>
      <td>55200.0</td>
      <td>40700.0</td>
      <td>22600.0</td>
      <td>10500.0</td>
      <td>5000.0</td>
    </tr>
    <tr>
      <th>2015</th>
      <td>60000.0</td>
      <td>43800.0</td>
      <td>24000.0</td>
      <td>11000.0</td>
      <td>5000.0</td>
    </tr>
    <tr>
      <th>2016</th>
      <td>60700.0</td>
      <td>45000.0</td>
      <td>25000.0</td>
      <td>11000.0</td>
      <td>5000.0</td>
    </tr>
  </tbody>
</table>
</div>


Now we multiply this data with the inflation data, taking the first element of the inflation data for the first row (excluding the year), the second for the second, and so on. Note that we actually need to divide by the inflation numbers, rather than multiply, which we do with a *list comprehension*. Finally, we transpose back to the original data shape.


`nom_income` is a dataframe with the nominal income observations (years in rows and percentiles in columns). We wish to divide all 2009 data by the first price index number (`1.0`), all 2010 observations by the 2nd (`1.024`), etc. If your dataframe consists of numerical data only, then this can be achieved using the `.div` method. As we want to divide each of the eight rows by a different number, we need to feed in eight different numbers. This is done by feeding in the inflation list and specifying that we want to apply these one for each row `(axis = 0)`.


```python
inflation = [1, 1.024, 1.078, 1.122, 1.171, 1.222, 1.259, 1.289]
nom_income = nom_income.div(inflation, axis=0).T
nom_income.round(2)
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }
.dataframe tbody tr th {
    vertical-align: top;
}
.dataframe thead th {
    text-align: right;
}
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>2009</th>
      <th>2010</th>
      <th>2011</th>
      <th>2012</th>
      <th>2013</th>
      <th>2014</th>
      <th>2015</th>
      <th>2016</th>
    </tr>
    <tr>
      <th>Percentile</th>
      <th></th>
      <th></th>
      <th></th>
      <th></th>
      <th></th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>85th</th>
      <td>43300.0</td>
      <td>43945.31</td>
      <td>44526.90</td>
      <td>44563.28</td>
      <td>45260.46</td>
      <td>45171.85</td>
      <td>47656.87</td>
      <td>47090.77</td>
    </tr>
    <tr>
      <th>75th</th>
      <td>31000.0</td>
      <td>31250.00</td>
      <td>32282.00</td>
      <td>32531.19</td>
      <td>34158.84</td>
      <td>33306.06</td>
      <td>34789.52</td>
      <td>34910.78</td>
    </tr>
    <tr>
      <th>50th</th>
      <td>17400.0</td>
      <td>17578.12</td>
      <td>17810.76</td>
      <td>17825.31</td>
      <td>18616.57</td>
      <td>18494.27</td>
      <td>19062.75</td>
      <td>19394.88</td>
    </tr>
    <tr>
      <th>25th</th>
      <td>8000.0</td>
      <td>8203.12</td>
      <td>8348.79</td>
      <td>8823.53</td>
      <td>8539.71</td>
      <td>8592.47</td>
      <td>8737.09</td>
      <td>8533.75</td>
    </tr>
    <tr>
      <th>15th</th>
      <td>4500.0</td>
      <td>4394.53</td>
      <td>4638.22</td>
      <td>4456.33</td>
      <td>4355.25</td>
      <td>4091.65</td>
      <td>3971.41</td>
      <td>3878.98</td>
    </tr>
  </tbody>
</table>
</div>


### *Python walk-through 12.5* Importing data from Excel 


We have downloaded and saved the data with the filename ‘datatables.xlsx’. From inspecting the Excel file, the data starts in row 2. We will use `pd.read_excel` to import the rows we need, and store this table in a dataframe called `overall`.


```
overall = pd.read_excel(
    Path("data/datatables.xlsx"), skiprows=1
)
overall
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }


    .dataframe tbody tr th {
        vertical-align: top;
    }


    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>調查開始日期Survey Start Date</th>
      <th>調查結束日期Survey End Date</th>
      <th>成功樣本Successful cases</th>
      <th>次樣本人數Subsample</th>
      <th>回應率Response rate</th>
      <th>好滿意Very satisfied</th>
      <th>幾滿意Quite satisfied</th>
      <th>滿意Satisfied</th>
      <th>一半半Half-half</th>
      <th>幾唔滿意Quite dissatisfied</th>
      <th>好唔滿意Very dissatisfied</th>
      <th>唔滿意Dissatisfied</th>
      <th>唔知/難講DK/HS</th>
      <th>總數Total</th>
      <th>淨值Netvalue</th>
      <th>平均量值Mean value</th>
      <th>基數Base</th>
      <th>樣本平均標準誤差Standard Error of Mean</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>2023-12-01 00:00:00</td>
      <td>2023-12-07</td>
      <td>502.0</td>
      <td>502.0</td>
      <td>0.487</td>
      <td>0.126</td>
      <td>0.356</td>
      <td>0.482</td>
      <td>0.175</td>
      <td>0.131</td>
      <td>0.195</td>
      <td>0.327</td>
      <td>0.017</td>
      <td>1.0</td>
      <td>0.156</td>
      <td>3.1</td>
      <td>492.0</td>
      <td>0.1</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2023-11-01 00:00:00</td>
      <td>2023-11-15</td>
      <td>1004.0</td>
      <td>518.0</td>
      <td>0.514</td>
      <td>0.109</td>
      <td>0.316</td>
      <td>0.425</td>
      <td>0.139</td>
      <td>0.180</td>
      <td>0.245</td>
      <td>0.425</td>
      <td>0.011</td>
      <td>1.0</td>
      <td>0.000</td>
      <td>2.9</td>
      <td>510.0</td>
      <td>0.1</td>
    </tr>
    <tr>
      <th>2</th>
      <td>2023-10-03 00:00:00</td>
      <td>2023-10-25</td>
      <td>1003.0</td>
      <td>493.0</td>
      <td>0.536</td>
      <td>0.103</td>
      <td>0.334</td>
      <td>0.438</td>
      <td>0.139</td>
      <td>0.177</td>
      <td>0.237</td>
      <td>0.414</td>
      <td>0.009</td>
      <td>1.0</td>
      <td>0.023</td>
      <td>2.9</td>
      <td>482.0</td>
      <td>0.1</td>
    </tr>
    <tr>
      <th>3</th>
      <td>2023-09-07 00:00:00</td>
      <td>2023-09-19</td>
      <td>1001.0</td>
      <td>512.0</td>
      <td>0.530</td>
      <td>0.138</td>
      <td>0.306</td>
      <td>0.444</td>
      <td>0.158</td>
      <td>0.170</td>
      <td>0.208</td>
      <td>0.378</td>
      <td>0.020</td>
      <td>1.0</td>
      <td>0.066</td>
      <td>3.0</td>
      <td>502.0</td>
      <td>0.1</td>
    </tr>
    <tr>
      <th>4</th>
      <td>2023-08-01 00:00:00</td>
      <td>2023-08-10</td>
      <td>1005.0</td>
      <td>519.0</td>
      <td>0.545</td>
      <td>0.107</td>
      <td>0.315</td>
      <td>0.422</td>
      <td>0.164</td>
      <td>0.183</td>
      <td>0.220</td>
      <td>0.403</td>
      <td>0.011</td>
      <td>1.0</td>
      <td>0.019</td>
      <td>2.9</td>
      <td>513.0</td>
      <td>0.1</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>397</th>
      <td>1997-10-28 00:00:00</td>
      <td>1997-10-29</td>
      <td>536.0</td>
      <td>536.0</td>
      <td>0.445</td>
      <td>0.019</td>
      <td>0.351</td>
      <td>0.370</td>
      <td>0.401</td>
      <td>0.160</td>
      <td>0.019</td>
      <td>0.179</td>
      <td>0.050</td>
      <td>1.0</td>
      <td>0.191</td>
      <td>3.2</td>
      <td>509.0</td>
      <td>0.1</td>
    </tr>
    <tr>
      <th>398</th>
      <td>1997-09-23 00:00:00</td>
      <td>1997-09-24</td>
      <td>512.0</td>
      <td>512.0</td>
      <td>0.416</td>
      <td>0.014</td>
      <td>0.475</td>
      <td>0.489</td>
      <td>0.304</td>
      <td>0.123</td>
      <td>0.008</td>
      <td>0.131</td>
      <td>0.076</td>
      <td>1.0</td>
      <td>0.358</td>
      <td>3.4</td>
      <td>471.0</td>
      <td>0.1</td>
    </tr>
    <tr>
      <th>399</th>
      <td>1997-08-26 00:00:00</td>
      <td>1997-08-27</td>
      <td>532.0</td>
      <td>532.0</td>
      <td>0.424</td>
      <td>0.030</td>
      <td>0.413</td>
      <td>0.443</td>
      <td>0.287</td>
      <td>0.119</td>
      <td>0.011</td>
      <td>0.130</td>
      <td>0.140</td>
      <td>1.0</td>
      <td>0.313</td>
      <td>3.4</td>
      <td>456.0</td>
      <td>0.1</td>
    </tr>
    <tr>
      <th>400</th>
      <td>1997-07-29 00:00:00</td>
      <td>1997-07-30</td>
      <td>564.0</td>
      <td>564.0</td>
      <td>0.397</td>
      <td>0.016</td>
      <td>0.380</td>
      <td>0.396</td>
      <td>0.245</td>
      <td>0.153</td>
      <td>0.018</td>
      <td>0.171</td>
      <td>0.189</td>
      <td>1.0</td>
      <td>0.225</td>
      <td>3.3</td>
      <td>458.0</td>
      <td>0.1</td>
    </tr>
    <tr>
      <th>401</th>
      <td>#2017年9月前以「整體回應比率」顯示，之後以「實效回應比率」顯示。#“Overall r...</td>
      <td>NaT</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
    </tr>
  </tbody>
</table>
<p>402 rows × 18 columns</p>
</div>


The data has been imported correctly - Python recognised the start and end dates as dates (they are even formatted with a timestamp), and all other variables as numeric (num). However, the variable names are quite long and include Chinese characters, so let’s rename the variables we need (survey start date and net value) to English-only variables. We’ll also remove the last row of the dataframe as it only contains missing values.


```
overall = overall.dropna(axis=0).rename(columns={"調查結束日期Survey End Date": "Survey date", "淨值Netvalue": "Net value"})
overall.info()


```


<class 'pandas.core.frame.DataFrame'>
Index: 401 entries, 0 to 400
Data columns (total 18 columns):
 #   Column                          Non-Null Count  Dtype         
---  ------                          --------------  -----         
 0   調查開始日期Survey Start Date         401 non-null    object        
 1   Survey date                     401 non-null    datetime64[ns]
 2   成功樣本Successful cases            401 non-null    float64       
 3   次樣本人數Subsample                  401 non-null    float64       
 4   回應率Response rate                401 non-null    float64       
 5   好滿意Very satisfied               401 non-null    float64       
 6   幾滿意Quite satisfied              401 non-null    float64       
 7   滿意Satisfied                     401 non-null    float64       
 8   一半半Half-half                    401 non-null    float64       
 9   幾唔滿意Quite dissatisfied          401 non-null    float64       
 10  好唔滿意Very dissatisfied           401 non-null    float64       
 11  唔滿意Dissatisfied                 401 non-null    float64       
 12  唔知/難講DK/HS                      401 non-null    float64       
 13  總數Total                         401 non-null    float64       
 14  Net value                       401 non-null    float64       
 15  平均量值Mean value                  401 non-null    float64       
 16  基數Base                          401 non-null    float64       
 17  樣本平均標準誤差Standard Error of Mean  401 non-null    float64       
dtypes: datetime64[ns](1), float64(16), object(1)
memory usage: 59.5+ KB




### *Python walk-through 12.6* Plotting line charts


The data is already in the correct format, but the variable we want to plot (`Net value`) is expressed in proportions instead of percentages. We will convert this variable to percentages and then use `ggplot` to make the line chart.


```
overall["Net value (%)"] = overall["Net value"]*100
(
    ggplot(overall[(overall["Survey date"] > "2006-1-20")], aes(x="Survey date", y="Net value (%)"))
    + geom_line(size=1)
    + labs(x="Survey date", y="Net satisfaction (% positive minus % negative)", title="Overall satisfication with HKSAR Government")
    + scale_x_datetime()
)
```


![png](empirical_project_12_files/empirical_project_12_WT6_1.png)


**Figure 12.7** Net public satisfaction with the government’s performance over time.


For Question 3(b), in this example we use the variable ‘Appraisal of degree of prosperity’. We have downloaded and saved this data with the filename ‘datatables_part2.xlsx’. We will import it into Python as the dataframe `prosperity`.


```
prosperity = pd.read_excel(
    Path("data/datatables_part2.xlsx"), skiprows=1
)
prosperity
```
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }


    .dataframe tbody tr th {
        vertical-align: top;
    }


    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>調查開始日期Survey Start Date</th>
      <th>調查結束日期Survey End Date</th>
      <th>樣本人數Total Sample</th>
      <th>次樣本人數Sub Sample</th>
      <th>回應率Response Rate</th>
      <th>繁榮Prosperity</th>
      <th>標準誤差Standard Error</th>
      <th>評分人數Number of Raters</th>
      <th>認知率Recognition Rate</th>
      <th>備註Remark</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>2023-08-01 00:00:00</td>
      <td>2023-08-10</td>
      <td>1005.0</td>
      <td>492.0</td>
      <td>0.545</td>
      <td>5.42</td>
      <td>0.11</td>
      <td>487.0</td>
      <td>0.990</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2023-05-03 00:00:00</td>
      <td>2023-05-18</td>
      <td>1003.0</td>
      <td>503.0</td>
      <td>0.524</td>
      <td>5.87</td>
      <td>0.13</td>
      <td>500.0</td>
      <td>0.995</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>2</th>
      <td>2023-02-01 00:00:00</td>
      <td>2023-02-09</td>
      <td>1017.0</td>
      <td>528.0</td>
      <td>0.580</td>
      <td>5.83</td>
      <td>0.10</td>
      <td>519.0</td>
      <td>0.983</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>3</th>
      <td>2022-11-02 00:00:00</td>
      <td>2022-11-10</td>
      <td>1001.0</td>
      <td>519.0</td>
      <td>0.489</td>
      <td>5.09</td>
      <td>0.10</td>
      <td>515.0</td>
      <td>0.991</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>4</th>
      <td>2022-08-01 00:00:00</td>
      <td>2022-08-09</td>
      <td>1004.0</td>
      <td>510.0</td>
      <td>0.586</td>
      <td>5.52</td>
      <td>0.09</td>
      <td>501.0</td>
      <td>0.983</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>99</th>
      <td>1997-09-23 00:00:00</td>
      <td>1997-09-24</td>
      <td>511.0</td>
      <td>511.0</td>
      <td>0.416</td>
      <td>7.89</td>
      <td>0.06</td>
      <td>497.0</td>
      <td>0.973</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>100</th>
      <td>1997-08-26 00:00:00</td>
      <td>1997-08-27</td>
      <td>532.0</td>
      <td>532.0</td>
      <td>0.424</td>
      <td>7.86</td>
      <td>0.06</td>
      <td>513.0</td>
      <td>0.964</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>101</th>
      <td>1997-07-29 00:00:00</td>
      <td>1997-07-30</td>
      <td>564.0</td>
      <td>564.0</td>
      <td>0.397</td>
      <td>7.86</td>
      <td>0.07</td>
      <td>540.0</td>
      <td>0.957</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>102</th>
      <td>1997-06-25 00:00:00</td>
      <td>1997-06-25</td>
      <td>548.0</td>
      <td>548.0</td>
      <td>0.467</td>
      <td>8.05</td>
      <td>0.06</td>
      <td>526.0</td>
      <td>0.960</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>103</th>
      <td>* 該調查結果公佈時尚未包括手機樣本。上表結果已更新為固網樣本及手機樣本的合併統計數字。* ...</td>
      <td>NaT</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>NaN</td>
    </tr>
  </tbody>
</table>
<p>104 rows × 10 columns</p>
</div>


```
prosperity = prosperity.rename(columns={"調查結束日期Survey End Date": "Survey date", "繁榮Prosperity": "Prosperity"})
prosperity.info()
```
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 104 entries, 0 to 103
Data columns (total 10 columns):
 #   Column                   Non-Null Count  Dtype         
---  ------                   --------------  -----         
 0   調查開始日期Survey Start Date  104 non-null    object        
 1   Survey date              103 non-null    datetime64[ns]
 2   樣本人數Total Sample         103 non-null    float64       
 3   次樣本人數Sub Sample          103 non-null    float64       
 4   回應率Response Rate         103 non-null    float64       
 5   Prosperity               103 non-null    float64       
 6   標準誤差Standard Error       103 non-null    float64       
 7   評分人數Number of Raters     103 non-null    float64       
 8   認知率Recognition Rate      103 non-null    float64       
 9   備註Remark                 2 non-null      object        
dtypes: datetime64[ns](1), float64(7), object(2)
memory usage: 8.3+ KB


For Question 3(c), we repeat the steps taken for 3(a). The `Prosperity` variable is measured on a 0-10 scale, where higher numbers indicate stronger beliefs that Hong Kong is a prosperous society. 


![png](empirical_project_12_files/empirical_project_12_WT6_2.png)


**Figure 12.8** Belief that Hong Kong is a prosperous society (0-10 scale).