Regression Analysis in Python Using UNICEF MICS Data

If you're trying to learn about data analytics, economics or statistics, you've probably come across the term "regression analysis" before. It is one of the most commonly used statistical methods for estimating the relationships between a dependent variable and one or several independent variables. In this entry, we'll look at how we can use Python to run a regression analysis on panel data from the Multiple Indicator Cluster Survey datasets produced by the UNICEF. We'll illustrate this tutorial with a real-world use case, specifically looking at education indicators in the context of Madagascar. As usual, our program will include code for importing data, cleaning it up, run basic statistical analyses, specify a model and display the results. Let's get to it!

The Multiple Indicator Cluster Survey (aka MICS) datasets produced by the UNICEF are probably some of the greatest, most extensive datasets available as far as micro-level data is concerned. This data is particularly important in the context of developing countries where good statistics are often in short supply. This data is especially helpful to get a good grasp of the human, social and economic challenges a country's population might face. The UNICEF does a great job of collecting this data, keeping it organized and making it available free of charge. If you haven't, I would recommend checking out their website. The data used for this article was kindly provided by the UNICEF MICS Team. Before we begin, a short disclaimer is in order.

Disclaimer: The following code was written in and ran using Visual Studio Code and the latest Anaconda distribution. The data used for this article was kindly provided by the UNICEF MICS Team. The code itself is solely intended for educational purposes and the potential findings may not engage the data provider's responsibility in any capacity.

Author: Johary Razafindratsita, 2022.

The first step is to download the data from the MICS Website. I would recommend registering as a MICS data user first, and applying to get access to the datasets you are interested in. For the purpose of this article, we will be downloading the 2018 MICS datasets for Madagascar. Since the data is made available in SPSS format, the first thing we need to do is install a module that will allow us to read that data in Python. This is where pyreadstat comes in:

pip install pyreadstat

Note that in addition to pyreadstat, we will also be using a specific module called statsmodels to specify our model and run our regression analysis. Now, let's import the different libraries and modules we will be using:

import pandas as pd

import matplotlib.pyplot as plt

import numpy as np

import pyreadstat

import statsmodels.api as sm

import dataframe_image as dfi

We are now ready to import our data into Python. MICS produces several large and extensive datasets. At this stage, it is therefore very important to understand what these datasets are, how they are structured and what each indicator means. I cannot stress this enough. Thankfully, the MICS Team provides a good FAQ as well as supporting documents to help us here. Additionally, reading the associated MICS reports is essential in understanding how these datasets are structured.

Because we're interested in education indicators, we'll be working on the 'fs.sav' file which is the survey related to children aged 5 to 17. Let's import it in our program:

df, meta = pyreadstat.read_sav('./MICS/fs.sav')

At this stage, you can convert it into a .csv file should you want to have a look at the data in say, Excel, for instance (you could also convert it to a regular Excel file, though this will result in a much larger file size):

pd.DataFrame.to_csv(df, 'mics_fs.csv')

Now that we've imported our data, it's time to select the variables we'll be using for our regression analysis. For the purpose of this article, we'll be looking at determinants of school attendance in Madagascar based on information provided by the MICS data.

Our dependent (or exogenous) variable, the one we're looking to explain, will therefore be "School Attendance". This is the CB7 variable in our database, which can take one of 2 values:

• The value "1" if they attended school or pre-primary school at any point during the current school year

• The value "2" if they did not attend school or pre-primary school at all during the current school year

Now let's select our independent (or explanatory) variables and lay out our hypotheses:

1. Our first variable is "Age" (CB3 in our database). Children aged 5 to 17 should normally be attending school. However, in Madagascar and many developing countries, school attendance often decreases with age. This can be due to several factors: there are considerably less secondary schools - and even less higher secondary schools - compared to primary schools; attending secondary schools usually requires traveling longer distances in rural areas, and would often requires children to move out from their parent's home and rent a place in the town where the school is located, resulting in substantial financial costs for the family; the older children are, the more likely they are to work or contribute in a meaningful manner to help their family.

2. Our second variable is "Child Labour" (CL3 in our database). As explained above, adolescents often find themselves in a position where they are asked to work or contribute meaningfully to help their family make ends meet. This can happen at the detriment of their education, causing them to stop attending school. This specific variable measures the time children spent working over the previous week (measured in hours).

3. Our third variable is "Chores" (CL13 in our database). This variable measures the time children spent helping with domestic chores over the previous week (measured in hours). The more time children spend on chores, the less likely they are to attend school. This is particularly true for children living in rural areas where chores can consist of substantial tasks, including fetching water and firewood, cooking, cleaning and helping care for other family members.

4. Our fourth variable is "Farm Work" (CL1A in our database). This is another indicator for child labour which focuses on children's involvement in farming activities. As the vast majority of Madagascar's population still relies on subsistence farming, children are very likely to be asked to help with these types of activities as they grow up. This specific variable, like our exogenous variable, is dichotomous, meaning that it can take one of 2 values (the value "1" if children are doing any farm work and "2" otherwise).

5. Our fifth and final variable is "Other Income Generating Activities" (CL1X in our database). Like the previous variable, this indicator encompasses all other activities that children could be involved in that generate an income. It is also a dichotomous variable (with a value of "1" if children are involved in any such activities and "2" otherwise).

Now that we've identified all our variables, we can build a DataFrame using only these variables:

Table_1 = df[['CB3', 'CB7', 'CL3', 'CL13', 'CL1X','CL1A']]

Table_1 = pd.DataFrame(Table_1)

Now let's run some basic statistical analyses on our newly built DataFrame using the .describe() function and save the output table as an image file on our local drive:

edu_tab1 = Table_1.describe()

dfi.export(edu_tab1, 'edu_tab1.png')