Five sources, three answers to one simple question
1.28% or 1.11%?Average yearly growth of U.S. real GDP per person, 2000–2022, depending on where you download it
FRED, the World Bank and the IMF say 1.28% a year. The Penn World Table says 1.16%, the Maddison Project 1.11%. All five are respected sources. By the end of this deck you will download all five yourself, in a few lines of Python, and know why they differ.
Today: six questions, one per act
Where do macroeconomic data live? Who publishes them, four ways in, three ideas.
How do you get U.S. data from FRED? Three routes: a CSV address, pandas-datareader, fredapi.
How do the World Bank and IMF libraries work?wbgapi and imfp: search, then download.
Can one piece of code reach many agencies? SDMX, Eurostat and DBnomics.
What if a dataset has no library? Read its file: Penn World Table, Maddison, Our World in Data.
Why does one number differ across sources? The comparison, and good habits.
Every tool in this deck is taught the same way, in four steps
What it does, in one sentence
The code, a few lines at a time
The output the notebook printed
The pitfalls, and how to avoid them
Guess before the output appears.
Run a variation yourself.
A common error and its fix.
A map of the data
Act I
Most numbers start at a national statistical office, then get collected
Producers: national statistical offices, such as the U.S. Bureau of Economic Analysis (BEA) and Bureau of Labor Statistics (BLS), measure GDP, prices and jobs.
Collectors: FRED, the World Bank, the IMF, the OECD, Eurostat, the ECB and the BIS gather these numbers, check them and put them in one place.
Research datasets: the Penn World Table and the Maddison Project rebuild the numbers to compare countries and centuries.
You: one line of Python per series.
That is why the same series often appears in several sources, and why the copies can differ.
Seven collectors hold the official statistics
Source
Published by
What it covers
Easiest route
FRED
St. Louis Fed
over 800,000 series, mostly U.S.; daily to annual
CSV address; pandas-datareader; fredapi
WDI
World Bank
about 1,400 indicators, 217 economies, since 1960
wbgapi
WEO, IMF Data
IMF
GDP, inflation, debt for about 190 countries, with forecasts
imfp; sdmx1; DBnomics
OECD Data Explorer
OECD
monthly and quarterly data for 38 members
DBnomics; sdmx1
Eurostat
European Commission
official statistics of the EU
eurostat; sdmx1
ECB Data Portal
European Central Bank
euro-area money, interest and exchange rates
sdmx1
BIS Data Portal
Bank for International Settlements
policy rates, credit, house prices
DBnomics; sdmx1
Four more sources rebuild or regroup the numbers
Source
Published by
What it covers
Easiest route
Penn World Table
University of Groningen
real GDP, capital, jobs, productivity; 185 countries, 1950–2023
read its Stata or Excel file
Maddison Project
University of Groningen
GDP per person and population back to the year 1
read its Excel file
Our World in Data
Oxford and Global Change Data Lab
thousands of charts, each with a CSV download
read its CSV address
DBnomics
Banque de France and partners
many providers through one website and one library
dbnomics
The research datasets have no Python library: pandas reads their files directly (Act V).
There are four ways to get data into Python
Way
Example in this deck
Good
Watch out
1. A specialized library for one source
wbgapi, imfp, fredapi, eurostat
easy; search functions; tidy tables
one library per source; libraries change
2. A general library that speaks SDMX
sdmx1 for the ECB, IMF, OECD, Eurostat
the same code for many agencies
you must learn each dataset’s key
3. A collection of many sources
dbnomics
one library and one website
its copy can lag months behind
4. Read a file from its web address
FRED CSV, PWT, Maddison, OWID
pandas alone; any file online
you must find the address; it can move
Acts II–V walk through all four ways, source by source.
Which way would you use?
You need (a) India’s GDP per person in 1870, (b) the euro–dollar exchange rate every month since 1999, and (c) unemployment in five OECD countries. Which of the four ways fits each?
Way 4: read the Maddison Excel file; there is no library.
Way 2: sdmx1 with the ECB’s key (or way 3, DBnomics).
Way 3: DBnomics, which copies the OECD (or way 2, sdmx1).
Why write code when every site has a Download button?
A skeptic says: “I can click Download and open the file in Excel. Why learn five libraries?”
Reply: code is a record. It says exactly which series, which countries and which dates you used; it reruns in seconds when the data are revised; and the same three lines work for 197 countries as for one.
Idea 1: every series has a short code, and codes are precise
UNRATE
U.S. unemployment rate, FRED
NY.GDP.PCAP.KD
GDP per person, constant dollars, World Bank
Names are vague (“GDP per capita” has seven versions at the World Bank); codes are exact.
Find the code first: the source’s search box, or a search function in Python.
Step one of every download: find the right code.
Idea 2: read the units before you compute anything
Percent or fraction? 4.2 or 0.042
Dollars or thousands of dollars? Maddison counts population in thousands, the Penn World Table in millions.
Current or constant prices? Only constant prices remove inflation.
Monthly, quarterly or annual? FRED’s GDP is quarterly; its unemployment rate is monthly.
Seasonally adjusted or not?
A wrong unit gives a wrong answer that looks right.
Idea 3: numbers are revised, so each download is a vintage
Statistical offices revise their numbers as better information arrives.
The growth rate you download today for 2024 may differ from the one published in 2025.
Each release is a vintage. The IMF even keeps its old forecast editions as separate datasets.
So: always write down when you downloaded your data.
Data downloaded on 2026-10-17
Setup: install the data libraries once per Colab session
# Install the data libraries this appendix uses (about 30 seconds, once per session)%pip install -q wbgapi fredapi pandas-datareader sdmx1 dbnomics eurostat%pip install -q --no-deps imfp
Lines that start with % are commands for the notebook, not Python.
-q (“quiet”) hides most messages.
--no-deps keeps Colab’s own pandas and requests: imfp asks for exact versions that Colab may not have.
Note: you may need to restart the kernel to use updated packages.
This note is harmless: ignore it.
Setup: one cell loads the common libraries and records the date
import numpy as np # numbers and arraysimport pandas as pd # data tablesimport plotly.express as px # quick charts from data tablesimport plotly.io as pio# The prepared data files of the chapters live in the book's GitHub repositoryBOOK_DATA ="https://raw.githubusercontent.com/quarcs-lab/macro102-site/main/data/"# The date of this run: data change over time, so we print it next to our resultsTODAY = pd.Timestamp.today().strftime("%Y-%m-%d")print("Data downloaded on", TODAY)
Each data library is imported in its own section, right where it is first needed.
FRED, three routes
Act II
FRED is the first stop for U.S. data, and the code is in the address
fred.stlouisfed.org/series/UNRATE
FRED (Federal Reserve Economic Data, St. Louis Fed): over 800,000 series, many of them international too.
To find a code, type what you want in the FRED search box.
Three routes into Python:
Way 4: read the CSV file of a series (no library, no key)
Way 1: pandas-datareader (several series in one call)
Way 1: fredapi (the official API, with a free key)
Route 1: a FRED series is a CSV file at a fixed address
pd.read_csv reads a CSV file, and it accepts a web address as easily as a file on your computer.
# The address of a FRED series: a fixed beginning plus the series codeurl ="https://fred.stlouisfed.org/graph/fredgraph.csv?id=UNRATE"unrate = pd.read_csv(url, parse_dates=["observation_date"])unrate.tail()
How many columns will the table have? What will the dates in the last rows look like?
Two columns, monthly dates: the last value is 4.2% for September 2026
Monthly data are dated on the first day of the month.
Row 944 is the 945th month: the series starts in January 1948.
A small function turns one line into a reusable tool
def read_fred(code):"""Download one FRED series as a table with the columns date and <code>.""" url ="https://fred.stlouisfed.org/graph/fredgraph.csv?id="+ code table = pd.read_csv(url, parse_dates=["observation_date"])return table.rename(columns={"observation_date": "date"})cpi = read_fred("CPIAUCSL") # the consumer price index for all urban consumerscpi.tail()
.shift(12) moves the column down by 12 rows, so each row sees the value of 12 months earlier.
The formula on the slide and the line of code are the same thing.
What will the inflation column hold in the first 12 rows of the table?
merge lines up two series by date in one table
# Put unemployment and inflation side by side in one table, matched by datefred = read_fred("UNRATE").merge(cpi[["date", "inflation"]], on="date")fred = fred.rename(columns={"UNRATE": "Unemployment rate","inflation": "Inflation rate (CPI, 12 months)"})fred = fred[fred["date"] >="1960-01-01"]fred.tail()
When were inflation and unemployment at their highest since 1960?
Two series, one line of code each, sixty-six years of history
Data: U.S. Bureau of Labor Statistics through FRED, UNRATE and CPIAUCSL, seasonally adjusted; inflation is our 12-month percent change. Topics of Chapters 7, 8 and 12 of Jones (2024), Macroeconomics, 6th ed.
Inflation peaked around 1980; unemployment spikes in every recession
Inflation peaked in the 1970s and early 1980s, fell for decades, and jumped again in 2021–2022.
Unemployment spikes in every recession; April 2020 (the pandemic) is the highest in the chart.
Latest month in the download: unemployment 4.2%, inflation 3.7% (September 2026).
Two series, one line of code each: the CSV route needs no library and no registration.
Route 2: pandas-datareader asks for several series in one call
It reads FRED (and a few other sources) and lines the series up by date for you.
import pandas_datareader.data as web# Real GDP and real personal consumption, quarterly, billions of chained 2017 dollarsmacro = web.DataReader(["GDPC1", "PCECC96"], "fred", start="2000-01-01")macro["consumption share"] = macro["PCECC96"] / macro["GDPC1"]macro.tail()
You install with a hyphen, but import with an underscore
%pip install pandas-datareader, then import pandas-datareader.
Python names cannot contain hyphens. You installpandas-datareader and importpandas_datareader. The same happens with sdmx1, which you import as sdmx.
Route 3: fredapi uses FRED’s official API, with a free key
An API (application programming interface) is a web address designed for programs rather than people.
fredapi can also search FRED from Python.
It needs an API key: a personal password for programs, 32 letters and digits, free.
Get a key
Create a free FRED account.
Request a key on the API keys page.
Store it in Colab
Key icon (Secrets) → Add new secret: FRED_API_KEY.
Paste the key; switch on Notebook access.
The key is read from Colab Secrets, never typed into the notebook
FRED_API_KEY =Nonetry:from google.colab import userdata FRED_API_KEY = userdata.get("FRED_API_KEY")exceptException:print("No FRED API key found, so this example is skipped. Follow steps 1-3 above to add one.")if FRED_API_KEY:from fredapi import Fred fred_api = Fred(api_key=FRED_API_KEY) found = fred_api.search("real gross domestic product per capita")print(found[["title", "frequency_short", "units_short", "popularity"]].head(10)) gdp = fred_api.get_series("GDPC1", observation_start="2020-01-01")print(gdp.tail())
try: runs a block; except: says what to do if it fails, instead of stopping with an error.
Our run had no key, so the cell skipped politely
No FRED API key found, so this example is skipped. Follow steps 1-3 above to add one.
The notebook was executed outside Colab, without a secret: the except branch ran.
With a key, the cell prints the ten most popular FRED series matching the search, then the last values of GDPC1.
Never paste a key into a notebook you share: anybody who sees it can use it. Never put one on a slide or in a screenshot either.
Pasting a key into a cell is not “just for now”
It is fine to paste my API key into the notebook while I test; I will delete it later.
Notebooks are copied, shared and saved with their history. Anyone who sees the key can use it under your name. Keep it in Colab Secrets from the first run.
Three routes to FRED: pick by what you need
Route
Needs
Strength
Use it when
CSV address (way 4)
pandas only
no library, no key, always works
one series at a time; the book’s choice
pandas-datareader (way 1)
one library
several series, aligned by date
a few series you want in one table
fredapi (way 1)
a free key
search, metadata, the official API
you search for codes from Python
Same data, three doors: the CSV route is the simplest and the one every chapter notebook uses.
Daily or monthly? Check the dates of what you download
Use read_fred to download the 10-year Treasury yield (DGS10) and the federal funds rate (FEDFUNDS). Print the last rows of each. Which one is monthly, and which one is daily?
Agency libraries
Act III
The World Bank’s own library searches before it downloads
World Development Indicators (WDI): about 1,400 indicators for 217 economies, mostly annual since 1960.
The standard source for comparing countries over time.
wbgapi, written by the World Bank, imported as wb.
The workflow: search the indicator → find the country codes → download → reshape.
Search the indicator names first
import wbgapi as wbwb.series.info(q="GDP per capita")
id value
----------------- ---------------------------------------------------
NY.GDP.PCAP.CD GDP per capita (current US$)
NY.GDP.PCAP.CN GDP per capita (current LCU)
NY.GDP.PCAP.KD GDP per capita (constant 2015 US$)
NY.GDP.PCAP.KD.ZG GDP per capita growth (annual %)
NY.GDP.PCAP.KN GDP per capita (constant LCU)
NY.GDP.PCAP.PP.CD GDP per capita, PPP (current international $)
NY.GDP.PCAP.PP.KD GDP per capita, PPP (constant 2021 international $)
7 elements
Which of the seven codes measures growth over time?
You want to compare how fast GDP per person grew in five countries since 1960. Which code do you pick, and why?
NY.GDP.PCAP.KD: constant 2015 U.S. dollars remove inflation, so changes are real growth.
Current dollars (.CD) mix real growth with inflation and exchange-rate swings.
For comparing levels across countries in one year, use PPP (NY.GDP.PCAP.PP.KD); we do that next.
Countries have three-letter codes; search them too
wb.economy.info(q="Korea")
How many economies will match “Korea”?
id value region incomeLevel
---- ------------------------- -------- -------------
KOR Korea, Rep. EAS HIC
PRK Korea, Dem. People's Rep. EAS LIC
2 elements
Two Koreas: KOR (South, high income) and PRK (North, low income). Check the name, not just the word.
One call downloads one indicator for five countries
countries = ["USA", "CHN", "IND", "KOR", "BRA"]gdppc = wb.data.DataFrame("NY.GDP.PCAP.KD", countries, time=range(1960, 2030), numericTimeKeys=True, labels=True)gdppc.iloc[:, :6] # all rows, the first 6 columns
time=range(1960, 2030): every year available in that range.
numericTimeKeys=True: years as numbers (2020), not text ("YR2020").
labels=True: adds the country names.
The table is wide: one row per country, one column per year
Country 1960 1961 1962 \
economy
BRA Brazil 2603.519010 2743.388888 2838.472299
KOR Korea, Rep. 1037.728992 1077.104640 1087.793622
IND India 312.777848 316.739645 318.381946
CHN China 241.382661 177.349529 166.085165
USA United States 18991.544600 19108.935362 19965.143574
1963 1964
economy
BRA 2772.579949 2785.074432
...
In 1960 Korea’s GDP per person (about $1,038) was below Brazil’s (about $2,604), in constant 2015 dollars.
melt again, then a ratio-scale chart
gdppc_long = gdppc.melt(id_vars="Country", var_name="year", value_name="gdp_pc").dropna()print("Years:", gdppc_long["year"].min(), "to", gdppc_long["year"].max())fig = px.line(gdppc_long, x="year", y="gdp_pc", color="Country", log_y=True, title="GDP per person in five economies, World Bank WDI", labels={"year": "", "gdp_pc": "GDP per person, constant 2015 dollars (ratio scale)","Country": ""})fig.update_yaxes(tickvals=LOG_TICKS, tickformat=",")fig.show()
Years: 1960 to 2025
GDP per person in five economies
Which of the five economies grew fastest since 1960, and how can you tell on a ratio scale?
China and Korea grew fastest; Brazil slowed after 1980
China’s and Korea’s lines are the steepest: their incomes grew fastest.
Brazil’s line is much flatter after 1980: its income per person grew slowly.
Korea started below Brazil in 1960 and is now far above it.
One call, five countries, sixty-six years: the World Bank library does the search, the download and the labels.
A snapshot: two indicators for every country in one year
YEAR =2023snapshot = wb.data.DataFrame(["NY.GDP.PCAP.PP.KD", "SP.DYN.LE00.IN"], "all", time=YEAR, skipAggs=True, columns="series", labels=True)snapshot = snapshot.rename(columns={"NY.GDP.PCAP.PP.KD": "gdp_pc_ppp","SP.DYN.LE00.IN": "life_expectancy"})snapshot = snapshot.dropna()print(len(snapshot), "countries have both numbers in", YEAR)snapshot.head()
"all": every economy. skipAggs=True: leave out groups such as “World” or “Euro area”.
columns="series": one column per indicator (instead of one per year).
197 countries have both numbers for 2023
197 countries have both numbers in 2023
Country gdp_pc_ppp life_expectancy
economy
ZWE Zimbabwe 5218.045266 62.775
ZMB Zambia 3673.484197 66.349
PSE West Bank and Gaza 5562.424426 65.170
VNM Viet Nam 13534.799784 74.588
VUT Vanuatu 3782.826490 71.477
The rows come in reverse alphabetical order of the code; order never matters for a scatter plot.
The cloud bends: big gains at low incomes, small gains at high incomes
Among poor countries, life expectancy rises steeply with income.
Among rich countries, the cloud flattens.
The ratio scale matters: each step to the right is a doubling or tripling of income.
The chart shows an association, not a cause: income, health systems and education move together.
Unemployment in Spain, Germany and South Africa
Search the indicator code of the unemployment rate with wb.series.info(q="unemployment"), then download it for Spain, Germany and South Africa (ESP, DEU, ZAF) from 2000 on and show the table.
The IMF’s World Economic Outlook comes with forecasts
Published twice a year, in April and October.
GDP growth, inflation, unemployment and debt for about 190 countries.
Unlike most sources: forecasts for the next five years.
imfp reads it, and every other dataset of the IMF’s data portal, in four steps:
which datasets? → which dimensions? → which codes? → the data
Step 1: which datasets exist?
import imfp# Step 1: the IMF's datasets; .str.contains keeps the rows whose name contains a worddatasets = imfp.imf_get_dataflows()print(len(datasets), "datasets")datasets[datasets["name"].str.contains("World Economic Outlook")][["id", "name"]]
224 datasets
id name
196 WEO_2026_APR_VINTAGE World Economic Outlook (WEO) 2026 April
209 WEO_2025_OCT_VINTAGE World Economic Outlook (WEO) 2025 October
212 WEO World Economic Outlook (WEO)
WEO is the latest edition; the others are vintages, kept to show what the IMF predicted at the time.
Steps 2 and 3: which dimensions, and which codes?
# Step 2: the dimensions of the WEOprint(imfp.imf_get_datastructure("WEO"))# Step 3: the indicator codes; we keep those about GDP at constant pricesindicators = imfp.imf_get_codelists("INDICATOR", "WEO")indicators[indicators["name"].str.contains("constant prices", case=False)][["code", "name"]]
dimension_id type position
0 COUNTRY Dimension 0
1 INDICATOR Dimension 1
2 FREQUENCY Dimension 2
Five codes all start with “GDP, constant prices”. NGDP_R, NGDP_RPCH, NGDPRPC, …: which one is the percent change?
NGDP_RPCH: real GDP, percent change. That is the growth rate.
NGDP_R: the level of real GDP; NGDPRPC: real GDP per capita (used in Act VI).
When the names are cut off, print one in full before you trust it.
Step 4: pick codes for each dimension and download
# Step 4: the dataweo = imfp.imf_get("WEO", {"COUNTRY": ["USA", "CHN", "IND", "DEU", "BRA"],"INDICATOR": "NGDP_RPCH"})weo["year"] = weo["TIME_PERIOD"].astype(int) # the years arrive as text: make them numbersnames = {"USA": "United States", "CHN": "China", "IND": "India", "DEU": "Germany", "BRA": "Brazil"}weo["country"] = weo["COUNTRY"].map(names)weo = weo[weo["year"] >=2000]print("Years:", weo["year"].min(), "to", weo["year"].max())weo[["country", "year", "OBS_VALUE", "COUNTRY_UPDATE_DATE"]].tail()
What will be the last year in the table, in a download made in October 2026?
The table runs to 2031: the last years are forecasts
Years: 2000 to 2031
country year OBS_VALUE COUNTRY_UPDATE_DATE
255 United States 2027 2.292985 9/30/2026
256 United States 2028 2.056286 9/30/2026
257 United States 2029 1.880588 9/30/2026
258 United States 2030 1.753132 9/30/2026
259 United States 2031 1.755000 9/30/2026
Data and forecasts arrive in one column. From the current year on, most values are projections: label them.
Shade the forecast years so nobody mistakes them for data
FIRST_FORECAST = pd.Timestamp.today().year # the current yearfig = px.line(weo, x="year", y="OBS_VALUE", color="country", markers=True, title="Real GDP growth with IMF forecasts, World Economic Outlook", labels={"year": "", "OBS_VALUE": "Real GDP growth, percent per year", "country": ""})fig.add_vrect(x0=FIRST_FORECAST -0.5, x1=weo["year"].max() +0.5, fillcolor=GRAY, opacity=0.15, line_width=0, annotation_text="IMF forecasts", annotation_position="top left")fig.add_hline(y=0, line_color=GRAY, line_width=1)fig.show()
add_vrect shades a vertical band; add_hline draws the zero line.
Real GDP growth with IMF forecasts
How do the IMF’s forecast lines differ in shape from the actual growth of the past 25 years?
Before you look: will the forecast years show recessions, booms, or something else?
Actual growth jumps around; the forecasts are smooth
Forecasts assume a gradual return to normal growth
Actual growth jumps: the 2009 financial crisis and the 2020 pandemic stand out.
The forecast lines are smooth: the IMF assumes each economy returns gradually to its normal growth rate.
U.S. forecast: 2.29% in 2027, easing to 1.76% in 2031.
A forecast is a model’s assumption, not an observation. Shade it, and say which edition it comes from.
Which country has the highest unemployment in the latest forecast year?
Download the WEO unemployment rate (indicator LUR) for the same five countries and draw it with px.line. Which country is highest in the last forecast year?
One standard, one door
Act IV
SDMX: one standard, so one library speaks to many agencies
SDMX (Statistical Data and Metadata eXchange): a standard shared by central banks and statistical agencies.
sdmx1 (imported as sdmx) speaks it: the same few lines work for the ECB, the IMF, the OECD, Eurostat…
Way 2 of the four: learn one library, reach many agencies.
date value
328 2026-05-01 1.167320
329 2026-06-01 1.151800
330 2026-07-01 1.141748
331 2026-08-01 1.159310
332 2026-09-01 1.151323
Lowest and highest: two lines of code find them
print(f"Lowest: {usd['value'].min():.2f} dollars per euro, in {usd.loc[usd['value'].idxmin(), 'date']:%B %Y}")print(f"Highest: {usd['value'].max():.2f} dollars per euro, in {usd.loc[usd['value'].idxmax(), 'date']:%B %Y}")
Guess the range: how low and how high has the euro been, in dollars, since 1999?
Lowest: 0.85 dollars per euro, in June 2001
Highest: 1.58 dollars per euro, in July 2008
The euro in dollars
Has one euro always been worth more than one dollar?
A euro has bought from 0.85 to 1.58 dollars since 1999
Data: European Central Bank, ECB Data Portal, series EXR.M.USD.EUR.SP00.A (euro reference exchange rate, U.S. dollar, monthly average), downloaded with sdmx1. Exchange rates: Chapter 20 of Jones (2024), Macroeconomics, 6th ed.
Exchange rates move a lot, and the quote direction matters
The euro has been worth well below one dollar (0.85, June 2001) and well above 1.5 (1.58, July 2008).
September 2026: 1.15 dollars per euro.
A higher line is a stronger euro: the series is dollars per euro.
Check the quote: “dollars per euro” and “euros per dollar” move in opposite directions.
Same code, another agency: Eurostat through sdmx1
Only the agency (ESTAT), the dataset and the key change. Here the key is one text: the codes in their fixed order, separated by dots; + asks for several codes.
A . Y15-74 . PC_ACT . T . ES+DE
annual · ages 15–74 · percent of the labor force · total (both sexes) · Spain and Germany
estat = sdmx.Client("ESTAT")message = estat.data("une_rt_a", key="A.Y15-74.PC_ACT.T.ES+DE")unemployment = sdmx.to_pandas(message).reset_index()unemployment[["geo", "TIME_PERIOD", "value"]].tail()
Spain’s unemployment rate fell from 14.9% in 2021 to 10.5% in 2025
geo TIME_PERIOD value
29 ES 2021 14.9
30 ES 2022 13.0
31 ES 2023 12.2
32 ES 2024 11.4
33 ES 2025 10.5
The last rows are Spain (ES): the rows of both countries are stacked in one long table.
Two-letter country codes at Eurostat, not three-letter ones.
The yen or the pound against the euro
Change the ECB key to download the monthly exchange rate of the Japanese yen (JPY) or the British pound (GBP) against the euro since 1999, and draw it.
The eurostat package: ask for the dimensions, then filter
Every Eurostat table has a code, shown next to its name on the website: une_rt_a is the annual unemployment rate.
import eurostatprint(eurostat.get_pars("une_rt_a")) # the dimensions of the table
How many dimensions? Think of the SDMX key we just used for this same table.
['freq', 'age', 'unit', 'sex', 'geo']
The same five dimensions as the SDMX key, now in lowercase: it is the same table through another door.
filter_pars keeps only the rows you want
filters = {"age": "Y15-74", "sex": "T", "unit": "PC_ACT"}eu_unemp = eurostat.get_data_df("une_rt_a", filter_pars=filters)# a shorter name for the country columneu_unemp = eu_unemp.rename(columns={"geo\\TIME_PERIOD": "geo"})LAST = eu_unemp.columns[-1] # the last column is the latest yeareu_unemp.iloc[:5, :8]
freq age unit sex geo 2003 2004 2005
0 A Y15-74 PC_ACT T AT NaN NaN NaN
1 A Y15-74 PC_ACT T BA NaN NaN NaN
2 A Y15-74 PC_ACT T BE NaN NaN NaN
3 A Y15-74 PC_ACT T BG NaN NaN NaN
4 A Y15-74 PC_ACT T CH NaN NaN NaN
Empty cells are normal: many series start later
NaN (“not a number”) is how pandas marks a missing value.
In the first years, many countries have no data yet.
Country codes have two letters: EL is Greece; EU27_2020 is the European Union; codes starting with EA are euro-area aggregates.
Missing is not zero: never fill NaN with 0 in an economic series.
Names from a dictionary, then the latest year
geo_names =dict(eurostat.get_dic("une_rt_a", "geo"))eu_unemp["country"] = eu_unemp["geo"].map(geo_names)eu_unemp.loc[eu_unemp["geo"] =="EU27_2020", "country"] ="European Union"# a shorter labellatest = eu_unemp[["geo", "country", LAST]].dropna().sort_values(LAST)latest = latest[~latest["geo"].str.startswith("EA")] # drop the euro-area aggregatesprint("Year:", LAST, "-", len(latest) -1, "countries and the European Union")latest.tail()
Year: 2025 - 34 countries and the European Union
geo country 2025
12 EL Greece 8.9
15 FI Finland 9.7
13 ES Spain 10.5
26 MK North Macedonia 11.5
1 BA Bosnia and Herzegovina 12.2
Unemployment rates in Europe
Where in Europe is unemployment highest, and is the EU average closer to the top or the bottom?
Name the two EU members you expect at the top.
Spain and Finland top the EU; the Western Balkans are higher still
Europe and Japan protected jobs; U.S. firms laid off and rehired
The U.S. rate jumped far higher in 2020 than the others.
European countries and Japan protected jobs with short-time work schemes.
U.S. firms laid workers off and rehired them later.
The same shock, different institutions, different unemployment: one chart from one DBnomics call.
How fresh is the DBnomics copy of the BIS policy rates?
Look up the U.S. series of central bank policy rates from the BIS on DBnomics, address "BIS/WS_CBPOL/M.US". What is the latest month in the DBnomics copy? Try "M.XM" (euro area) and "M.JP" (Japan) too.
rgdpo: real GDP (output side), millions of 2021 dollars at PPP.
emp: people employed, millions. pop: population, millions.
GDP per worker, relative to the United States
\[\text{GDP per worker} = \frac{\texttt{rgdpo}}{\texttt{emp}} \qquad \text{relative to U.S.} = 100 \times \frac{\text{GDP per worker}_{i,t}}{\text{GDP per worker}_{US,t}}\]
pwt["gdp_per_worker"] = pwt["rgdpo"] / pwt["emp"]us = pwt[pwt["countrycode"] =="USA"][["year", "gdp_per_worker"]]us = us.rename(columns={"gdp_per_worker": "us_gdp_per_worker"})pwt = pwt.merge(us, on="year")pwt["relative_to_us"] =100* pwt["gdp_per_worker"] / pwt["us_gdp_per_worker"]catch_up = pwt[pwt["countrycode"].isin(["KOR", "DEU", "MEX", "BRA", "CHN", "IND"])& (pwt["year"] >=1960)]fig = px.line(catch_up, x="year", y="relative_to_us", color="country", title="GDP per worker relative to the United States, Penn World Table 11.0", labels={"year": "", "relative_to_us": "GDP per worker, United States = 100","country": ""})
Catching up with U.S. productivity
Which of these economies has closed the gap with U.S. GDP per worker since 1960, and has any fallen further behind?
Korea, Germany, Mexico, Brazil, China, India: pick one climber and one faller.
Korea climbed from under a tenth to over half of U.S. productivity
131,144 rows from the year 1 to 2022, and population in thousands
Downloading https://dataverse.nl/api/access/datafile/421302 ... (about 5 MB, may take 20 seconds)
131144 rows, 169 countries, 1 to 2022
countrycode country region year gdppc pop
131139 ZWE Zimbabwe Sub Saharan Africa 2018 1900.199191 14096.56
131140 ZWE Zimbabwe Sub Saharan Africa 2019 1753.024449 14344.59
131141 ZWE Zimbabwe Sub Saharan Africa 2020 1585.972804 14614.82
131142 ZWE Zimbabwe Sub Saharan Africa 2021 1687.253154 14900.01
131143 ZWE Zimbabwe Sub Saharan Africa 2022 1703.529373 15204.66
Zimbabwe, 2022: population 15204.66 here (thousands) but 16.07 in the Penn World Table (millions). Same country, same year, different units and estimates.
Two centuries of GDP per person on a ratio scale
long_run = maddison[maddison["countrycode"].isin(["GBR", "USA", "JPN", "CHN", "IND"])]long_run = long_run[long_run["year"] >=1820].dropna(subset=["gdppc"])fig = px.line(long_run, x="year", y="gdppc", color="country", log_y=True, title="Two centuries of GDP per person, Maddison Project Database 2023", labels={"year": "", "gdppc": "GDP per person, 2011 dollars (ratio scale)","country": ""})fig.update_yaxes(tickvals=LOG_TICKS, tickformat=",")fig.show()
dropna(subset=["gdppc"]) drops only the rows where GDP per person is missing; the early years are patchy.
Two centuries of GDP per person
In 1820, how far apart were Britain, the United States, Japan, China and India? Close together, or already far apart?
Around 1820 the five economies were much closer than today
Britain and the U.S. pulled away first; Asia caught up later
Around 1820 the five economies were much closer than today.
The United Kingdom and the United States pulled away first.
Japan caught up in the 20th century; China and India took off only in the last few decades.
One Excel file, one sheet name: two thousand years of data in one line of pandas.
Our World in Data: every chart has a CSV at its address + .csv
Our World in Data (OWID) turns data from many sources into charts.
The chart at ourworldindata.org/grapher/share-of-population-in-extreme-poverty has its data at the same address plus .csv.
OWID asks programs to say who they are, with a User-Agent header (a short label sent with the request).
OWID_URL ="https://ourworldindata.org/grapher/share-of-population-in-extreme-poverty.csv"poverty = pd.read_csv(OWID_URL, storage_options={"User-Agent": "Our World In Data data fetch/1.0"})SHARE = poverty.columns[3] # the column with the poverty rate (its name is long)print("Column:", SHARE)poverty.tail()
The poverty column, and the empty years at the end
Column: Share of population in poverty ($3 a day)
Entity Code Year Share of population in poverty ($3 a day) \
59367 Zimbabwe ZWE 2019 49.219894
59368 Zimbabwe ZWE 2020 NaN
59369 Zimbabwe ZWE 2021 NaN
59370 Zimbabwe ZWE 2022 NaN
59371 Zimbabwe ZWE 2023 NaN
...
The poverty line: $3 a day, in 2021 prices.
Zimbabwe’s last survey value is 2019 (49.2%); later years are empty.
Picking a column by position (columns[3]) breaks if the file gains a column: print its name to check.
Five regions since 1990
regions = ["World", "Sub-Saharan Africa (WB)", "South Asia (WB)", "East Asia and Pacific (WB)","Latin America and Caribbean (WB)"]poor = poverty[poverty["Entity"].isin(regions) & (poverty["Year"] >=1990)]fig = px.line(poor, x="Year", y=SHARE, color="Entity", category_orders={"Entity": regions}, title="Share of people in extreme poverty, by region, Our World in Data", labels={"Year": "", SHARE: "Percent of the population", "Entity": ""})fig.show()
The regions are rows of the same table: their names end in “(WB)”, the World Bank’s regions.
Extreme poverty by region
Where did extreme poverty fall most since 1990, and where do most of the world’s extreme poor live today?
East Asia’s extreme poverty fell from a majority to almost nothing
Growth in China cut poverty; Sub-Saharan Africa now holds most of the poor
Extreme poverty in East Asia fell from a majority of the population in 1990 to almost nothing, mostly because of growth in China.
Sub-Saharan Africa now holds most of the world’s extreme poor.
The most recent years are World Bank projections from older household surveys.
A chart address plus .csv and a User-Agent: that is all OWID needs.
Every chapter of this book reads its data the same way
# The U.S. unemployment rate prepared for Chapter 7 (folder data/ch07)book_unrate = pd.read_csv(BOOK_DATA +"ch07/fig7-2.csv", parse_dates=["date"])print("From", f"{book_unrate['date'].min():%B %Y}", "to", f"{book_unrate['date'].max():%B %Y}")book_unrate.tail()
From January 1950 to September 2026
date unemployment_rate
915 2026-05-01 4.3
916 2026-06-01 4.2
917 2026-07-01 4.1
918 2026-08-01 4.1
919 2026-09-01 4.2
Human capital in Korea, India and the United States
Use the Penn World Table already in memory (pwt) to draw the human capital indexhc for Korea, India and the United States since 1950.
One number, many sources
Act VI
How fast has U.S. income per person grown since 2000? Ask five sources
Source
Series
Unit
FRED
A939RX0Q048SBEA
real GDP per capita, quarterly → averaged over each year
World Bank
NY.GDP.PCAP.KD
constant 2015 dollars
IMF WEO
NGDPRPC
GDP per capita, constant prices, national currency
Penn World Table
rgdpna / pop
real GDP from the national accounts, per person
Maddison Project
gdppc
2011 dollars
We already know how to reach every one of them. Now we put them in one table.
FRED is quarterly: average the four quarters of each year
# FRED: quarterly data, so we average the four quarters of each yearfred_pc = read_fred("A939RX0Q048SBEA")fred_pc["year"] = fred_pc["date"].dt.yearfred_pc = fred_pc[fred_pc["year"] < pd.Timestamp.today().year] # complete years onlyfred_pc = fred_pc.groupby("year")["A939RX0Q048SBEA"].mean().rename("FRED")
Why drop the current year before averaging?
In October 2026 only the first quarters of 2026 exist: their average is not an annual value comparable with the other sources.
The World Bank needs a transpose; the IMF needs numeric years
# World Bankwb_pc = wb.data.DataFrame("NY.GDP.PCAP.KD", "USA", time=range(1990, 2030), numericTimeKeys=True)wb_pc = wb_pc.T["USA"].rename("World Bank") # .T swaps rows and columns: years become rows# IMF World Economic Outlookimf_pc = imfp.imf_get("WEO", {"COUNTRY": "USA", "INDICATOR": "NGDPRPC"})imf_pc["year"] = imf_pc["TIME_PERIOD"].astype(int)imf_pc = imf_pc.set_index("year")["OBS_VALUE"].rename("IMF WEO")print(fred_pc.tail(3), wb_pc.tail(3), imf_pc.tail(3), sep="\n\n")
The World Bank table is wide (one column per year): .T turns years into rows.
Each source becomes a pandas Series indexed by year and named after its source.
The IMF column runs to 2031: forecasts are mixed in
year
2023 67412.25
2024 68788.50
2025 70020.25
Name: FRED
2023 65667.28158
2024 66856.51317
2025 67945.61465
Name: World Bank
year
2029 75400.028306
2030 76495.271236
2031 77616.076824
Name: IMF WEO
FRED and the World Bank end in 2025; the IMF’s last years are forecasts to 2031.
The levels differ too ($70,020 versus $67,946 in 2025): different base years.
Penn World Table and Maddison: reuse the tables already in memory
# Penn World Table and Maddison: already in memory from B.9us_pwt = pwt[pwt["countrycode"] =="USA"].copy()us_pwt["gdp_pc"] = us_pwt["rgdpna"] / us_pwt["pop"]pwt_pc = us_pwt.set_index("year")["gdp_pc"].rename("Penn World Table")pwt_pc.index = pwt_pc.index.astype(int)mad_pc = maddison[maddison["countrycode"] =="USA"].set_index("year")["gdppc"].rename("Maddison")# One table: pd.concat puts the five columns side by side, matching the yearssources = pd.concat([fred_pc, wb_pc, imf_pc, pwt_pc, mad_pc], axis=1).sort_index()sources.loc[2000:]
pd.concat(..., axis=1) matches the five Series on their year index, leaving NaN where a source has no value.
Five columns, five units: the levels cannot be compared directly
FRED World Bank IMF WEO Penn World Table Maddison
2000 49914.50 48616.254628 49933.506536 55171.319764 45886.470498
...
2010 54188.25 52812.682987 54206.538072 59465.698228 49266.915862
...
2020 61127.00 59533.609024 61225.660175 65783.138914 54379.206512
2021 64828.50 63096.785615 64839.516732 69617.537868 57522.704626
2022 66037.25 64322.233282 66057.138042 71079.677954 58487.465861
2023 67412.25 65667.281580 67438.383199 72718.486651 NaN
2024 68788.50 66856.513170 68810.761710 NaN NaN
2025 70020.25 67945.614650 70032.247214 NaN NaN
...
2031 NaN NaN 77616.076824 NaN NaN
In 2000 the levels range from about $45,900 (Maddison) to $55,200 (Penn World Table).
Index each source to 2000 = 100, then compute average growth
END =int(sources.dropna().index.max()) # the last year with all five sourcesindex =100* sources.loc[2000:END] / sources.loc[2000]growth =100* ((sources.loc[END] / sources.loc[2000]) ** (1/ (END -2000)) -1)print(f"Average annual growth of U.S. real GDP per person, 2000-{END}, percent:")print(growth.round(2).to_string())
An index removes the units: every source starts at 100.
\(g\) is the constant yearly growth rate that takes \(y_{2000}\) to \(y_{\text{end}}\).
Will the five sources agree on the growth rate?
Same country, same years, same concept. Will the five average growth rates for 2000–2022 be equal, almost equal, or clearly different? Which sources will agree?
Three sources say 1.28% a year; the research datasets say less
Average annual growth of U.S. real GDP per person, 2000-2022, percent:
FRED 1.28
World Bank 1.28
IMF WEO 1.28
Penn World Table 1.16
Maddison 1.11
FRED, the World Bank and the IMF agree exactly: 1.28%. All three copy the latest official U.S. numbers.
The Penn World Table (1.16%) and the Maddison Project (1.11%) show slower growth.
The gap: 0.17 percentage points a year, every year, for 22 years.
U.S. real GDP per person in five sources
Where in the 22 years do the five lines part ways: at once, slowly, or after one event?
The five sources tell the same story with different numbers
Why sources differ: vintages, population and prices
Vintages. Each source copied the official U.S. numbers at a different time, and the BEA revises GDP every year. FRED updates within days; the Penn World Table and Maddison every few years.
Population. GDP per person divides by population, and the sources use different estimates (Census Bureau, United Nations, their own).
Prices and methods. Different base years and, for the Penn World Table and Maddison, their own international prices.
Always say where a number comes from, and compare sources when a result matters.
Different sources do not mean one of them is wrong
The sources disagree, so one of them must have made a mistake.
All five can be right about what they measure. They copied the official data at different dates, divided by different population estimates and used different prices. The error would be to report a number without its source.
Does the gap shrink after 2010?
Repeat the comparison for 2010 to the last common year. Are the differences between the sources larger or smaller than for 2000 onwards?
Good habit: save what you download, with the date in the name
# Save a table with the download date in its namefile_name =f"us_unemployment_inflation_{TODAY}.csv"fred.to_csv(file_name, index=False)print("Saved", file_name)
Saved us_unemployment_inflation_2026-10-17.csv
A Colab session is temporary: download the file from the Files icon, or save it to Google Drive.
Five habits make a download trustworthy
Save what you download, with the date in the file name.
Write down the source, the series code and the date next to every result.
Check the units and the latest date before you compute anything: .tail() is your friend.
Keep keys secret: Colab Secrets, never the notebook.
Be polite to the servers: download once and reuse the table.
Read the documentation before you trust a series
FRED: the notes under every series chart give the units, the frequency, the seasonal adjustment and the original producer.
World Bank: every indicator page has a definition, a source and its limitations.
Penn World Table: the user guide sits in the file list we downloaded (pwt110_user_guide_to_data_files.pdf).
Maddison: the Notes and Sources sheets of the Excel file.
Eurostat, ECB, OECD: the metadata page of each dataset explains every dimension code.
Five minutes with the documentation saves an afternoon of wrong results.
Cite a dataset as carefully as a paper
Who produced it: the agency or the authors.
What: the dataset and its edition (Penn World Table 11.0; WEO, October 2026).
Which series: the code (UNRATE, NY.GDP.PCAP.KD).
Where: the link or the DOI. When: the download date.
U.S. Bureau of Labor Statistics, Unemployment Rate [UNRATE], retrieved from FRED, Federal Reserve Bank of St. Louis, fred.stlouisfed.org/series/UNRATE, 17 October 2026.
When a download fails, fall back on a copy
try: unemployment_rate = read_fred("UNRATE")print("Downloaded from FRED")exceptExceptionas problem:print("FRED did not answer:", problem) unemployment_rate = pd.read_csv(BOOK_DATA +"ch07/fig7-2.csv", parse_dates=["date"])print("Using the copy stored in the book's repository instead")
Downloaded from FRED
Servers are sometimes down, busy or changed. Wait a minute and run the cell again first.
Predict the fix
read_fred("UNRATES") stops with HTTPError: HTTP Error 404: Not Found. What went wrong, and how do you fix it?
A 404 means the address does not exist: here the series code is misspelled (UNRATES). Check the code on the FRED website and copy it exactly.
Common errors, part 1: installing and addressing
Error message
What it usually means
What to do
ModuleNotFoundError: No module named 'wbgapi'
the library is not installed in this session
run the %pip install cell again
HTTPError: HTTP Error 404: Not Found
the web address or the series code is wrong
check the code on the source’s website; copy it exactly
HTTPError: HTTP Error 403: Forbidden
the server refused the program
add a User-Agent, as for Our World in Data
HTTPError: HTTP Error 429 or 503
too many requests, or the server is busy
wait a minute and try again; download less often
Common errors, part 2: inside the data
Error message
What it usually means
What to do
KeyError: 'value'
a column name that does not exist
print table.columns and use the exact name
an empty table
no data for this combination of codes
check each code of the key; try fewer filters
ValueError in pd.read_excel
wrong sheet name
open the file and check the sheet names
an API key error
no key, or a key not visible to the notebook
add it in Colab Secrets and switch on Notebook access
Read the last line of the error first: it usually names the problem.
Dollars per euro. European Central Bank, ECB Data Portal, EXR.M.USD.EUR.SP00.A, via sdmx1.
Unemployment in Europe. Eurostat, table une_rt_a, via eurostat and sdmx1.
Unemployment in five rich economies. OECD, Key Economic Indicators DSD_KEI@DF_KEI, measure UNEMP, through DBnomics.
Productivity. Feenstra, R. C., R. Inklaar and M. P. Timmer (2015), “The Next Generation of the Penn World Table”, American Economic Review 105(10); Penn World Table 11.0, doi:10.34894/FABVLR.
Five sources compared.A939RX0Q048SBEA (BEA via FRED), NY.GDP.PCAP.KD, NGDPRPC, PWT 11.0 rgdpna/pop, Maddison 2023 gdppc. All data downloaded on 17 October 2026.