Five sources, three answers to one simple question

1.28% or 1.11%? Average yearly growth of U.S. real GDP per person, 2000–2022, depending on where you download it

FRED, the World Bank and the IMF say 1.28% a year. The Penn World Table says 1.16%, the Maddison Project 1.11%. All five are respected sources. By the end of this deck you will download all five yourself, in a few lines of Python, and know why they differ.

Today: six questions, one per act

  • Where do macroeconomic data live? Who publishes them, four ways in, three ideas.
  • How do you get U.S. data from FRED? Three routes: a CSV address, pandas-datareader, fredapi.
  • How do the World Bank and IMF libraries work? wbgapi and imfp: search, then download.
  • Can one piece of code reach many agencies? SDMX, Eurostat and DBnomics.
  • What if a dataset has no library? Read its file: Penn World Table, Maddison, Our World in Data.
  • Why does one number differ across sources? The comparison, and good habits.

Every tool in this deck is taught the same way, in four steps

  1. What it does, in one sentence
  2. The code, a few lines at a time
  3. The output the notebook printed
  4. The pitfalls, and how to avoid them

Guess before the output appears.

Run a variation yourself.

A common error and its fix.

A map of the data

Act I

Most numbers start at a national statistical office, then get collected

  • Producers: national statistical offices, such as the U.S. Bureau of Economic Analysis (BEA) and Bureau of Labor Statistics (BLS), measure GDP, prices and jobs.
  • Collectors: FRED, the World Bank, the IMF, the OECD, Eurostat, the ECB and the BIS gather these numbers, check them and put them in one place.
  • Research datasets: the Penn World Table and the Maddison Project rebuild the numbers to compare countries and centuries.
  • You: one line of Python per series.

That is why the same series often appears in several sources, and why the copies can differ.

Seven collectors hold the official statistics

Source Published by What it covers Easiest route
FRED St. Louis Fed over 800,000 series, mostly U.S.; daily to annual CSV address; pandas-datareader; fredapi
WDI World Bank about 1,400 indicators, 217 economies, since 1960 wbgapi
WEO, IMF Data IMF GDP, inflation, debt for about 190 countries, with forecasts imfp; sdmx1; DBnomics
OECD Data Explorer OECD monthly and quarterly data for 38 members DBnomics; sdmx1
Eurostat European Commission official statistics of the EU eurostat; sdmx1
ECB Data Portal European Central Bank euro-area money, interest and exchange rates sdmx1
BIS Data Portal Bank for International Settlements policy rates, credit, house prices DBnomics; sdmx1

Four more sources rebuild or regroup the numbers

Source Published by What it covers Easiest route
Penn World Table University of Groningen real GDP, capital, jobs, productivity; 185 countries, 1950–2023 read its Stata or Excel file
Maddison Project University of Groningen GDP per person and population back to the year 1 read its Excel file
Our World in Data Oxford and Global Change Data Lab thousands of charts, each with a CSV download read its CSV address
DBnomics Banque de France and partners many providers through one website and one library dbnomics

The research datasets have no Python library: pandas reads their files directly (Act V).

There are four ways to get data into Python

Way Example in this deck Good Watch out
1. A specialized library for one source wbgapi, imfp, fredapi, eurostat easy; search functions; tidy tables one library per source; libraries change
2. A general library that speaks SDMX sdmx1 for the ECB, IMF, OECD, Eurostat the same code for many agencies you must learn each dataset’s key
3. A collection of many sources dbnomics one library and one website its copy can lag months behind
4. Read a file from its web address FRED CSV, PWT, Maddison, OWID pandas alone; any file online you must find the address; it can move

Acts II–V walk through all four ways, source by source.

Which way would you use?

You need (a) India’s GDP per person in 1870, (b) the euro–dollar exchange rate every month since 1999, and (c) unemployment in five OECD countries. Which of the four ways fits each?

    1. Way 4: read the Maddison Excel file; there is no library.
    1. Way 2: sdmx1 with the ECB’s key (or way 3, DBnomics).
    1. Way 3: DBnomics, which copies the OECD (or way 2, sdmx1).

Why write code when every site has a Download button?

A skeptic says: “I can click Download and open the file in Excel. Why learn five libraries?”

Reply: code is a record. It says exactly which series, which countries and which dates you used; it reruns in seconds when the data are revised; and the same three lines work for 197 countries as for one.

Idea 1: every series has a short code, and codes are precise

UNRATE

U.S. unemployment rate, FRED

NY.GDP.PCAP.KD

GDP per person, constant dollars, World Bank

  • Names are vague (“GDP per capita” has seven versions at the World Bank); codes are exact.
  • Find the code first: the source’s search box, or a search function in Python.

Step one of every download: find the right code.

Idea 2: read the units before you compute anything

  • Percent or fraction? 4.2 or 0.042
  • Dollars or thousands of dollars? Maddison counts population in thousands, the Penn World Table in millions.
  • Current or constant prices? Only constant prices remove inflation.
  • Monthly, quarterly or annual? FRED’s GDP is quarterly; its unemployment rate is monthly.
  • Seasonally adjusted or not?

A wrong unit gives a wrong answer that looks right.

Idea 3: numbers are revised, so each download is a vintage

  • Statistical offices revise their numbers as better information arrives.
  • The growth rate you download today for 2024 may differ from the one published in 2025.
  • Each release is a vintage. The IMF even keeps its old forecast editions as separate datasets.
  • So: always write down when you downloaded your data.
Data downloaded on 2026-10-17

Setup: install the data libraries once per Colab session

# Install the data libraries this appendix uses (about 30 seconds, once per session)
%pip install -q wbgapi fredapi pandas-datareader sdmx1 dbnomics eurostat
%pip install -q --no-deps imfp
  • Lines that start with % are commands for the notebook, not Python.
  • -q (“quiet”) hides most messages.
  • --no-deps keeps Colab’s own pandas and requests: imfp asks for exact versions that Colab may not have.
Note: you may need to restart the kernel to use updated packages.

This note is harmless: ignore it.

Setup: one cell loads the common libraries and records the date

import numpy as np                      # numbers and arrays
import pandas as pd                     # data tables
import plotly.express as px             # quick charts from data tables
import plotly.io as pio

# The prepared data files of the chapters live in the book's GitHub repository
BOOK_DATA = "https://raw.githubusercontent.com/quarcs-lab/macro102-site/main/data/"

# The date of this run: data change over time, so we print it next to our results
TODAY = pd.Timestamp.today().strftime("%Y-%m-%d")
print("Data downloaded on", TODAY)

Each data library is imported in its own section, right where it is first needed.

FRED, three routes

Act II

FRED is the first stop for U.S. data, and the code is in the address

fred.stlouisfed.org/series/UNRATE

  • FRED (Federal Reserve Economic Data, St. Louis Fed): over 800,000 series, many of them international too.
  • To find a code, type what you want in the FRED search box.
  • Three routes into Python:
    • Way 4: read the CSV file of a series (no library, no key)
    • Way 1: pandas-datareader (several series in one call)
    • Way 1: fredapi (the official API, with a free key)

Route 1: a FRED series is a CSV file at a fixed address

pd.read_csv reads a CSV file, and it accepts a web address as easily as a file on your computer.

# The address of a FRED series: a fixed beginning plus the series code
url = "https://fred.stlouisfed.org/graph/fredgraph.csv?id=UNRATE"
unrate = pd.read_csv(url, parse_dates=["observation_date"])
unrate.tail()

How many columns will the table have? What will the dates in the last rows look like?

Two columns, monthly dates: the last value is 4.2% for September 2026

    observation_date  UNRATE
940       2026-05-01     4.3
941       2026-06-01     4.2
942       2026-07-01     4.1
943       2026-08-01     4.1
944       2026-09-01     4.2
  • The column is named after the code: UNRATE.
  • Monthly data are dated on the first day of the month.
  • Row 944 is the 945th month: the series starts in January 1948.

A small function turns one line into a reusable tool

def read_fred(code):
    """Download one FRED series as a table with the columns date and <code>."""
    url = "https://fred.stlouisfed.org/graph/fredgraph.csv?id=" + code
    table = pd.read_csv(url, parse_dates=["observation_date"])
    return table.rename(columns={"observation_date": "date"})


cpi = read_fred("CPIAUCSL")     # the consumer price index for all urban consumers
cpi.tail()
          date  CPIAUCSL
952 2026-05-01   333.979
953 2026-06-01   332.568
954 2026-07-01   332.813
955 2026-08-01   334.131
956 2026-09-01   336.298

Inflation is the 12-month growth rate of the price level

\[\text{inflation}_t = \left(\frac{\text{CPI}_t}{\text{CPI}_{t-12}} - 1\right) \times 100\]

cpi["inflation"] = (cpi["CPIAUCSL"] / cpi["CPIAUCSL"].shift(12) - 1) * 100
  • .shift(12) moves the column down by 12 rows, so each row sees the value of 12 months earlier.
  • The formula on the slide and the line of code are the same thing.

What will the inflation column hold in the first 12 rows of the table?

merge lines up two series by date in one table

# Put unemployment and inflation side by side in one table, matched by date
fred = read_fred("UNRATE").merge(cpi[["date", "inflation"]], on="date")
fred = fred.rename(columns={"UNRATE": "Unemployment rate",
                            "inflation": "Inflation rate (CPI, 12 months)"})
fred = fred[fred["date"] >= "1960-01-01"]
fred.tail()
          date  Unemployment rate  Inflation rate (CPI, 12 months)
940 2026-05-01                4.3                         4.166615
941 2026-06-01                4.2                         3.463531
942 2026-07-01                4.1                         3.303856
943 2026-08-01                4.1                         3.353016
944 2026-09-01                4.2                         3.717251

Plotly wants long tables: melt stacks the columns

Wide: one column per series

date Unemployment Inflation
2026-09-01 4.2 3.72

Long: one row per date and series

date series percent
2026-09-01 Unemployment 4.2
2026-09-01 Inflation 3.72
fred_long = fred.melt(id_vars="date", var_name="series", value_name="percent")

Long format lets px.line(..., color="series") draw one line per series.

One more cell prints the latest month and draws the chart

latest = fred.dropna().iloc[-1]          # the last month with both numbers
print(f"Latest month: {latest['date']:%B %Y}")
print(f"Unemployment rate: {latest['Unemployment rate']:.1f} percent")
print(f"Inflation rate: {latest['Inflation rate (CPI, 12 months)']:.1f} percent")

fig = px.line(fred_long, x="date", y="percent", color="series",
              title="U.S. unemployment and inflation, monthly, from FRED",
              labels={"date": "", "percent": "Percent", "series": ""})
fig.show()
Latest month: September 2026
Unemployment rate: 4.2 percent
Inflation rate: 3.7 percent

U.S. unemployment and inflation

When were inflation and unemployment at their highest since 1960?

Two series, one line of code each, sixty-six years of history

Data: U.S. Bureau of Labor Statistics through FRED, UNRATE and CPIAUCSL, seasonally adjusted; inflation is our 12-month percent change. Topics of Chapters 7, 8 and 12 of Jones (2024), Macroeconomics, 6th ed.

Inflation peaked around 1980; unemployment spikes in every recession

  • Inflation peaked in the 1970s and early 1980s, fell for decades, and jumped again in 2021–2022.
  • Unemployment spikes in every recession; April 2020 (the pandemic) is the highest in the chart.
  • Latest month in the download: unemployment 4.2%, inflation 3.7% (September 2026).

Two series, one line of code each: the CSV route needs no library and no registration.

Route 2: pandas-datareader asks for several series in one call

It reads FRED (and a few other sources) and lines the series up by date for you.

import pandas_datareader.data as web

# Real GDP and real personal consumption, quarterly, billions of chained 2017 dollars
macro = web.DataReader(["GDPC1", "PCECC96"], "fred", start="2000-01-01")
macro["consumption share"] = macro["PCECC96"] / macro["GDPC1"]
macro.tail()
                GDPC1    PCECC96  consumption share
DATE
2025-04-01  23884.563  16378.319           0.685728
2025-07-01  24112.966  16526.176           0.685365
2025-10-01  24125.479  16604.582           0.688259
2026-01-01  24274.383  16634.157           0.685256
2026-04-01  24408.011  16788.685           0.687835

You install with a hyphen, but import with an underscore

%pip install pandas-datareader, then import pandas-datareader.

Python names cannot contain hyphens. You install pandas-datareader and import pandas_datareader. The same happens with sdmx1, which you import as sdmx.

Route 3: fredapi uses FRED’s official API, with a free key

  • An API (application programming interface) is a web address designed for programs rather than people.
  • fredapi can also search FRED from Python.
  • It needs an API key: a personal password for programs, 32 letters and digits, free.

Get a key

  1. Create a free FRED account.
  2. Request a key on the API keys page.

Store it in Colab

  1. Key icon (Secrets) → Add new secret: FRED_API_KEY.
  2. Paste the key; switch on Notebook access.

The key is read from Colab Secrets, never typed into the notebook

FRED_API_KEY = None
try:
    from google.colab import userdata
    FRED_API_KEY = userdata.get("FRED_API_KEY")
except Exception:
    print("No FRED API key found, so this example is skipped. Follow steps 1-3 above to add one.")

if FRED_API_KEY:
    from fredapi import Fred
    fred_api = Fred(api_key=FRED_API_KEY)
    found = fred_api.search("real gross domestic product per capita")
    print(found[["title", "frequency_short", "units_short", "popularity"]].head(10))
    gdp = fred_api.get_series("GDPC1", observation_start="2020-01-01")
    print(gdp.tail())

try: runs a block; except: says what to do if it fails, instead of stopping with an error.

Our run had no key, so the cell skipped politely

No FRED API key found, so this example is skipped. Follow steps 1-3 above to add one.
  • The notebook was executed outside Colab, without a secret: the except branch ran.
  • With a key, the cell prints the ten most popular FRED series matching the search, then the last values of GDPC1.

Never paste a key into a notebook you share: anybody who sees it can use it. Never put one on a slide or in a screenshot either.

Pasting a key into a cell is not “just for now”

It is fine to paste my API key into the notebook while I test; I will delete it later.

Notebooks are copied, shared and saved with their history. Anyone who sees the key can use it under your name. Keep it in Colab Secrets from the first run.

Three routes to FRED: pick by what you need

Route Needs Strength Use it when
CSV address (way 4) pandas only no library, no key, always works one series at a time; the book’s choice
pandas-datareader (way 1) one library several series, aligned by date a few series you want in one table
fredapi (way 1) a free key search, metadata, the official API you search for codes from Python

Same data, three doors: the CSV route is the simplest and the one every chapter notebook uses.

Daily or monthly? Check the dates of what you download

Use read_fred to download the 10-year Treasury yield (DGS10) and the federal funds rate (FEDFUNDS). Print the last rows of each. Which one is monthly, and which one is daily?

Agency libraries

Act III

The World Bank’s own library searches before it downloads

  • World Development Indicators (WDI): about 1,400 indicators for 217 economies, mostly annual since 1960.
  • The standard source for comparing countries over time.
  • wbgapi, written by the World Bank, imported as wb.
  • The workflow: search the indicator → find the country codes → download → reshape.

Search the indicator names first

import wbgapi as wb

wb.series.info(q="GDP per capita")
id                 value
-----------------  ---------------------------------------------------
NY.GDP.PCAP.CD     GDP per capita (current US$)
NY.GDP.PCAP.CN     GDP per capita (current LCU)
NY.GDP.PCAP.KD     GDP per capita (constant 2015 US$)
NY.GDP.PCAP.KD.ZG  GDP per capita growth (annual %)
NY.GDP.PCAP.KN     GDP per capita (constant LCU)
NY.GDP.PCAP.PP.CD  GDP per capita, PPP (current international $)
NY.GDP.PCAP.PP.KD  GDP per capita, PPP (constant 2021 international $)
                   7 elements

Which of the seven codes measures growth over time?

You want to compare how fast GDP per person grew in five countries since 1960. Which code do you pick, and why?

  • NY.GDP.PCAP.KD: constant 2015 U.S. dollars remove inflation, so changes are real growth.
  • Current dollars (.CD) mix real growth with inflation and exchange-rate swings.
  • For comparing levels across countries in one year, use PPP (NY.GDP.PCAP.PP.KD); we do that next.

Countries have three-letter codes; search them too

wb.economy.info(q="Korea")

How many economies will match “Korea”?

id    value                      region    incomeLevel
----  -------------------------  --------  -------------
KOR   Korea, Rep.                EAS       HIC
PRK   Korea, Dem. People's Rep.  EAS       LIC
      2 elements

Two Koreas: KOR (South, high income) and PRK (North, low income). Check the name, not just the word.

One call downloads one indicator for five countries

countries = ["USA", "CHN", "IND", "KOR", "BRA"]
gdppc = wb.data.DataFrame("NY.GDP.PCAP.KD", countries, time=range(1960, 2030),
                          numericTimeKeys=True, labels=True)
gdppc.iloc[:, :6]          # all rows, the first 6 columns
  • time=range(1960, 2030): every year available in that range.
  • numericTimeKeys=True: years as numbers (2020), not text ("YR2020").
  • labels=True: adds the country names.

The table is wide: one row per country, one column per year

               Country          1960          1961          1962  \
economy
BRA             Brazil   2603.519010   2743.388888   2838.472299
KOR        Korea, Rep.   1037.728992   1077.104640   1087.793622
IND              India    312.777848    316.739645    318.381946
CHN              China    241.382661    177.349529    166.085165
USA      United States  18991.544600  19108.935362  19965.143574

                 1963          1964
economy
BRA       2772.579949   2785.074432
...

In 1960 Korea’s GDP per person (about $1,038) was below Brazil’s (about $2,604), in constant 2015 dollars.

melt again, then a ratio-scale chart

gdppc_long = gdppc.melt(id_vars="Country", var_name="year", value_name="gdp_pc").dropna()
print("Years:", gdppc_long["year"].min(), "to", gdppc_long["year"].max())

fig = px.line(gdppc_long, x="year", y="gdp_pc", color="Country", log_y=True,
              title="GDP per person in five economies, World Bank WDI",
              labels={"year": "", "gdp_pc": "GDP per person, constant 2015 dollars (ratio scale)",
                      "Country": ""})
fig.update_yaxes(tickvals=LOG_TICKS, tickformat=",")
fig.show()
Years: 1960 to 2025

GDP per person in five economies

Which of the five economies grew fastest since 1960, and how can you tell on a ratio scale?

On a ratio scale the steepest line grew fastest

Data: World Bank, World Development Indicators, NY.GDP.PCAP.KD (constant 2015 U.S. dollars), downloaded with wbgapi. Growth across countries: Chapter 3 of Jones (2024), Macroeconomics, 6th ed.

China and Korea grew fastest; Brazil slowed after 1980

  • China’s and Korea’s lines are the steepest: their incomes grew fastest.
  • Brazil’s line is much flatter after 1980: its income per person grew slowly.
  • Korea started below Brazil in 1960 and is now far above it.

One call, five countries, sixty-six years: the World Bank library does the search, the download and the labels.

A snapshot: two indicators for every country in one year

YEAR = 2023
snapshot = wb.data.DataFrame(["NY.GDP.PCAP.PP.KD", "SP.DYN.LE00.IN"], "all", time=YEAR,
                             skipAggs=True, columns="series", labels=True)
snapshot = snapshot.rename(columns={"NY.GDP.PCAP.PP.KD": "gdp_pc_ppp",
                                    "SP.DYN.LE00.IN": "life_expectancy"})
snapshot = snapshot.dropna()
print(len(snapshot), "countries have both numbers in", YEAR)
snapshot.head()
  • "all": every economy. skipAggs=True: leave out groups such as “World” or “Euro area”.
  • columns="series": one column per indicator (instead of one per year).

197 countries have both numbers for 2023

197 countries have both numbers in 2023
                    Country    gdp_pc_ppp  life_expectancy
economy
ZWE                Zimbabwe   5218.045266           62.775
ZMB                  Zambia   3673.484197           66.349
PSE      West Bank and Gaza   5562.424426           65.170
VNM                Viet Nam  13534.799784           74.588
VUT                 Vanuatu   3782.826490           71.477

The rows come in reverse alphabetical order of the code; order never matters for a scatter plot.

Merge in each country’s income group

economies = wb.economy.DataFrame(skipAggs=True)
income_names = {"HIC": "High income", "UMC": "Upper middle income",
                "LMC": "Lower middle income", "LIC": "Low income"}
economies["income group"] = economies["incomeLevel"].map(income_names)
snapshot = snapshot.merge(economies[["income group"]], left_index=True, right_index=True)
snapshot["income group"].value_counts()
income group
High income            73
Upper middle income    58
Lower middle income    46
Low income             20

Richer countries live longer

As income rises, does life expectancy rise steadily, faster and faster, or more and more slowly?

Sketch the shape of the cloud of 197 countries before you see it: x is GDP per person (ratio scale), y is life expectancy.

Life expectancy rises steeply among poor countries, then flattens

Data: World Bank, WDI, NY.GDP.PCAP.PP.KD (PPP, constant 2021 international dollars) and SP.DYN.LE00.IN, 2023; income groups from the World Bank classification. Related to Chapter 3 of Jones (2024).

The cloud bends: big gains at low incomes, small gains at high incomes

  • Among poor countries, life expectancy rises steeply with income.
  • Among rich countries, the cloud flattens.
  • The ratio scale matters: each step to the right is a doubling or tripling of income.

The chart shows an association, not a cause: income, health systems and education move together.

Unemployment in Spain, Germany and South Africa

Search the indicator code of the unemployment rate with wb.series.info(q="unemployment"), then download it for Spain, Germany and South Africa (ESP, DEU, ZAF) from 2000 on and show the table.

The IMF’s World Economic Outlook comes with forecasts

  • Published twice a year, in April and October.
  • GDP growth, inflation, unemployment and debt for about 190 countries.
  • Unlike most sources: forecasts for the next five years.
  • imfp reads it, and every other dataset of the IMF’s data portal, in four steps:

imf_get_dataflows() → imf_get_datastructure() → imf_get_codelists() → imf_get()

which datasets? → which dimensions? → which codes? → the data

Step 1: which datasets exist?

import imfp

# Step 1: the IMF's datasets; .str.contains keeps the rows whose name contains a word
datasets = imfp.imf_get_dataflows()
print(len(datasets), "datasets")
datasets[datasets["name"].str.contains("World Economic Outlook")][["id", "name"]]
224 datasets
                       id                                       name
196  WEO_2026_APR_VINTAGE    World Economic Outlook (WEO) 2026 April
209  WEO_2025_OCT_VINTAGE  World Economic Outlook (WEO) 2025 October
212                   WEO               World Economic Outlook (WEO)

WEO is the latest edition; the others are vintages, kept to show what the IMF predicted at the time.

Steps 2 and 3: which dimensions, and which codes?

# Step 2: the dimensions of the WEO
print(imfp.imf_get_datastructure("WEO"))

# Step 3: the indicator codes; we keep those about GDP at constant prices
indicators = imfp.imf_get_codelists("INDICATOR", "WEO")
indicators[indicators["name"].str.contains("constant prices", case=False)][["code", "name"]]
  dimension_id       type  position
0      COUNTRY  Dimension         0
1    INDICATOR  Dimension         1
2    FREQUENCY  Dimension         2
           code                                               name
10  NGDP_RPCHMK  Gross domestic product (GDP), Constant prices,...
16    NGDP_RPCH  Gross domestic product (GDP), Constant prices,...
18       NGDP_R  Gross domestic product (GDP), Constant prices,...
48      NGDPRPC  Gross domestic product (GDP), Constant prices,...
91   NGDPRPPPPC  Gross domestic product (GDP), Constant prices,...

Which code is the growth rate of real GDP?

Five codes all start with “GDP, constant prices”. NGDP_R, NGDP_RPCH, NGDPRPC, …: which one is the percent change?

  • NGDP_RPCH: real GDP, percent change. That is the growth rate.
  • NGDP_R: the level of real GDP; NGDPRPC: real GDP per capita (used in Act VI).
  • When the names are cut off, print one in full before you trust it.

Step 4: pick codes for each dimension and download

# Step 4: the data
weo = imfp.imf_get("WEO", {"COUNTRY": ["USA", "CHN", "IND", "DEU", "BRA"],
                           "INDICATOR": "NGDP_RPCH"})
weo["year"] = weo["TIME_PERIOD"].astype(int)   # the years arrive as text: make them numbers
names = {"USA": "United States", "CHN": "China", "IND": "India", "DEU": "Germany", "BRA": "Brazil"}
weo["country"] = weo["COUNTRY"].map(names)
weo = weo[weo["year"] >= 2000]
print("Years:", weo["year"].min(), "to", weo["year"].max())
weo[["country", "year", "OBS_VALUE", "COUNTRY_UPDATE_DATE"]].tail()

What will be the last year in the table, in a download made in October 2026?

The table runs to 2031: the last years are forecasts

Years: 2000 to 2031
           country  year  OBS_VALUE COUNTRY_UPDATE_DATE
255  United States  2027   2.292985           9/30/2026
256  United States  2028   2.056286           9/30/2026
257  United States  2029   1.880588           9/30/2026
258  United States  2030   1.753132           9/30/2026
259  United States  2031   1.755000           9/30/2026

Data and forecasts arrive in one column. From the current year on, most values are projections: label them.

Shade the forecast years so nobody mistakes them for data

FIRST_FORECAST = pd.Timestamp.today().year       # the current year
fig = px.line(weo, x="year", y="OBS_VALUE", color="country", markers=True,
              title="Real GDP growth with IMF forecasts, World Economic Outlook",
              labels={"year": "", "OBS_VALUE": "Real GDP growth, percent per year", "country": ""})
fig.add_vrect(x0=FIRST_FORECAST - 0.5, x1=weo["year"].max() + 0.5, fillcolor=GRAY, opacity=0.15,
              line_width=0, annotation_text="IMF forecasts", annotation_position="top left")
fig.add_hline(y=0, line_color=GRAY, line_width=1)
fig.show()

add_vrect shades a vertical band; add_hline draws the zero line.

Real GDP growth with IMF forecasts

How do the IMF’s forecast lines differ in shape from the actual growth of the past 25 years?

Before you look: will the forecast years show recessions, booms, or something else?

Actual growth jumps around; the forecasts are smooth

Data: International Monetary Fund, World Economic Outlook database, NGDP_RPCH (real GDP, percent change), from the IMF data portal with imfp. Growth over the business cycle: Chapter 9 of Jones (2024), Macroeconomics, 6th ed.

Forecasts assume a gradual return to normal growth

  • Actual growth jumps: the 2009 financial crisis and the 2020 pandemic stand out.
  • The forecast lines are smooth: the IMF assumes each economy returns gradually to its normal growth rate.
  • U.S. forecast: 2.29% in 2027, easing to 1.76% in 2031.

A forecast is a model’s assumption, not an observation. Shade it, and say which edition it comes from.

Which country has the highest unemployment in the latest forecast year?

Download the WEO unemployment rate (indicator LUR) for the same five countries and draw it with px.line. Which country is highest in the last forecast year?

One standard, one door

Act IV

SDMX: one standard, so one library speaks to many agencies

  • SDMX (Statistical Data and Metadata eXchange): a standard shared by central banks and statistical agencies.
  • sdmx1 (imported as sdmx) speaks it: the same few lines work for the ECB, the IMF, the OECD, Eurostat…
  • Way 2 of the four: learn one library, reach many agencies.
import sdmx

print(sdmx.list_sources())
['ABS', 'ABS_JSON', 'AR1', 'BBK', 'BIS', 'COMP', 'ECB', 'EMPL', 'ESTAT', 'ESTAT3',
 'ESTAT_COMEXT', 'GROW', 'ILO', 'IMF', 'IMF_DATA', 'IMF_DATA3', 'INEGI', 'INSEE', 'ISTAT',
 'LSD', 'NB', 'NBB', 'OECD', 'OECD3', 'OECD_JSON', 'SGR', 'SGR3', 'SPC', 'STAT_EE', 'StatCan',
 'UNESCO', 'UNICEF', 'UNSD', 'UY110', 'WB', 'WB_WDI']

In SDMX every series is a key: one code per dimension

M . USD . EUR . SP00 . A

Dimension Code Meaning
FREQ M monthly
CURRENCY USD the U.S. dollar
CURRENCY_DENOM EUR measured against the euro
EXR_TYPE SP00 spot rate
EXR_SUFFIX A average over the period

Browse the series and their keys on the ECB Data Portal (dataset EXR).

Which part of the key would you change for the yen?

You have the key for the monthly dollar–euro rate. Which one code would you change to get the Japanese yen against the euro, and to what?

CURRENCY: change USD to JPY. Everything else stays: monthly, against the euro, spot rate, average.

The ECB client downloads a key in four lines

ecb = sdmx.Client("ECB")
key = {"FREQ": "M", "CURRENCY": "USD", "CURRENCY_DENOM": "EUR",
       "EXR_TYPE": "SP00", "EXR_SUFFIX": "A"}
message = ecb.data("EXR", key=key, params={"startPeriod": "1999"})
usd = sdmx.to_pandas(message).reset_index()
usd["date"] = pd.to_datetime(usd["TIME_PERIOD"])
usd[["date", "value"]].tail()
          date     value
328 2026-05-01  1.167320
329 2026-06-01  1.151800
330 2026-07-01  1.141748
331 2026-08-01  1.159310
332 2026-09-01  1.151323

Lowest and highest: two lines of code find them

print(f"Lowest: {usd['value'].min():.2f} dollars per euro, in {usd.loc[usd['value'].idxmin(), 'date']:%B %Y}")
print(f"Highest: {usd['value'].max():.2f} dollars per euro, in {usd.loc[usd['value'].idxmax(), 'date']:%B %Y}")

Guess the range: how low and how high has the euro been, in dollars, since 1999?

Lowest: 0.85 dollars per euro, in June 2001
Highest: 1.58 dollars per euro, in July 2008

The euro in dollars

Has one euro always been worth more than one dollar?

A euro has bought from 0.85 to 1.58 dollars since 1999

Data: European Central Bank, ECB Data Portal, series EXR.M.USD.EUR.SP00.A (euro reference exchange rate, U.S. dollar, monthly average), downloaded with sdmx1. Exchange rates: Chapter 20 of Jones (2024), Macroeconomics, 6th ed.

Exchange rates move a lot, and the quote direction matters

  • The euro has been worth well below one dollar (0.85, June 2001) and well above 1.5 (1.58, July 2008).
  • September 2026: 1.15 dollars per euro.
  • A higher line is a stronger euro: the series is dollars per euro.

Check the quote: “dollars per euro” and “euros per dollar” move in opposite directions.

Same code, another agency: Eurostat through sdmx1

Only the agency (ESTAT), the dataset and the key change. Here the key is one text: the codes in their fixed order, separated by dots; + asks for several codes.

A . Y15-74 . PC_ACT . T . ES+DE

annual · ages 15–74 · percent of the labor force · total (both sexes) · Spain and Germany

estat = sdmx.Client("ESTAT")
message = estat.data("une_rt_a", key="A.Y15-74.PC_ACT.T.ES+DE")
unemployment = sdmx.to_pandas(message).reset_index()
unemployment[["geo", "TIME_PERIOD", "value"]].tail()

Spain’s unemployment rate fell from 14.9% in 2021 to 10.5% in 2025

   geo TIME_PERIOD  value
29  ES        2021   14.9
30  ES        2022   13.0
31  ES        2023   12.2
32  ES        2024   11.4
33  ES        2025   10.5
  • The last rows are Spain (ES): the rows of both countries are stacked in one long table.
  • Two-letter country codes at Eurostat, not three-letter ones.

The yen or the pound against the euro

Change the ECB key to download the monthly exchange rate of the Japanese yen (JPY) or the British pound (GBP) against the euro since 1999, and draw it.

The eurostat package: ask for the dimensions, then filter

Every Eurostat table has a code, shown next to its name on the website: une_rt_a is the annual unemployment rate.

import eurostat

print(eurostat.get_pars("une_rt_a"))      # the dimensions of the table

How many dimensions? Think of the SDMX key we just used for this same table.

['freq', 'age', 'unit', 'sex', 'geo']

The same five dimensions as the SDMX key, now in lowercase: it is the same table through another door.

filter_pars keeps only the rows you want

filters = {"age": "Y15-74", "sex": "T", "unit": "PC_ACT"}
eu_unemp = eurostat.get_data_df("une_rt_a", filter_pars=filters)
# a shorter name for the country column
eu_unemp = eu_unemp.rename(columns={"geo\\TIME_PERIOD": "geo"})
LAST = eu_unemp.columns[-1]          # the last column is the latest year
eu_unemp.iloc[:5, :8]
  freq     age    unit sex geo  2003  2004  2005
0    A  Y15-74  PC_ACT   T  AT   NaN   NaN   NaN
1    A  Y15-74  PC_ACT   T  BA   NaN   NaN   NaN
2    A  Y15-74  PC_ACT   T  BE   NaN   NaN   NaN
3    A  Y15-74  PC_ACT   T  BG   NaN   NaN   NaN
4    A  Y15-74  PC_ACT   T  CH   NaN   NaN   NaN

Empty cells are normal: many series start later

  • NaN (“not a number”) is how pandas marks a missing value.
  • In the first years, many countries have no data yet.
  • Country codes have two letters: EL is Greece; EU27_2020 is the European Union; codes starting with EA are euro-area aggregates.

Missing is not zero: never fill NaN with 0 in an economic series.

Names from a dictionary, then the latest year

geo_names = dict(eurostat.get_dic("une_rt_a", "geo"))
eu_unemp["country"] = eu_unemp["geo"].map(geo_names)
eu_unemp.loc[eu_unemp["geo"] == "EU27_2020", "country"] = "European Union"   # a shorter label

latest = eu_unemp[["geo", "country", LAST]].dropna().sort_values(LAST)
latest = latest[~latest["geo"].str.startswith("EA")]          # drop the euro-area aggregates
print("Year:", LAST, "-", len(latest) - 1, "countries and the European Union")
latest.tail()
Year: 2025 - 34 countries and the European Union
   geo                 country  2025
12  EL                  Greece   8.9
15  FI                 Finland   9.7
13  ES                   Spain  10.5
26  MK         North Macedonia  11.5
1   BA  Bosnia and Herzegovina  12.2

Unemployment rates in Europe

Where in Europe is unemployment highest, and is the EU average closer to the top or the bottom?

Name the two EU members you expect at the top.

Spain and Finland top the EU; the Western Balkans are higher still

Data: Eurostat, table une_rt_a, ages 15–74, downloaded with the eurostat package. Unemployment: Chapter 7 of Jones (2024), Macroeconomics, 6th ed.

Unemployment differs a lot, even inside one market and one currency

  • Highest in the EU: Spain (10.5%) and Finland (9.7%); Greece is next (8.9%).
  • Outside the Union, North Macedonia (11.5%) and Bosnia and Herzegovina (12.2%) are higher still.
  • One common market, and for many one currency, yet very different labor markets.

The eurostat package: a table code, a few filters, and names from a dictionary.

Youth unemployment

Draw the unemployment rate of youth (ages 15 to 24) instead. Which countries are at the top?

DBnomics: one door to many providers

  • DBnomics copies the data of the IMF, OECD, Eurostat, ECB, BIS, national offices and more.
  • One search website and one Python library: way 3, a collection.
  • Every series has an address: provider / dataset / series, shown on its page.
from dbnomics import fetch_series

us_growth = fetch_series("IMF/WEO:latest/USA.NGDP_RPCH.pcent_change")
print(us_growth.columns.tolist())
us_growth[["series_name", "period", "value"]].tail()

The same IMF series, from DBnomics

['@frequency', 'provider_code', 'dataset_code', 'dataset_name', 'series_code', 'series_name',
 'original_period', 'period', 'original_value', 'value', 'weo-country', 'weo-subject', 'unit',
 'WEO Country', 'WEO Subject', 'Unit']
                                          series_name     period  value
46  United States – Gross domestic product, consta... 2026-01-01  1.744
47  United States – Gross domestic product, consta... 2027-01-01  1.983
48  United States – Gross domestic product, consta... 2028-01-01  2.122
49  United States – Gross domestic product, consta... 2029-01-01  2.122
50  United States – Gross domestic product, consta... 2030-01-01  2.122

Compare with Act III: imfp gave 2.29% for 2027 and ran to 2031. DBnomics gives 1.98% and stops in 2030. Why?

DBnomics keeps a copy, and some copies lag by months

  • The difference is a vintage: the DBnomics copy of “WEO latest” is an older edition than the one imfp read from the IMF.
  • The same happens with any collection: it is only as fresh as its last copy.
  • Always look at the latest date of what you downloaded before you rely on it.

A collection is convenient, not authoritative. When the latest number matters, go to the original provider.

The OECD’s monthly unemployment rates, through DBnomics

oecd = fetch_series("OECD", "DSD_KEI@DF_KEI",
                    dimensions={"REF_AREA": ["USA", "JPN", "DEU", "FRA", "GBR"], "FREQ": ["M"],
                                "MEASURE": ["UNEMP"]})
oecd = oecd[oecd["period"] >= "2000-01-01"]
print("Latest month:", f"{oecd['period'].max():%B %Y}")

FRED’s U.S. unemployment rate ran to September 2026 in the same run. What will the latest month be here?

Latest month: May 2026

Unemployment in five rich economies

When the pandemic hit in 2020, did unemployment jump by the same amount in the U.S., Japan, Germany, France and the U.K.?

In 2020 U.S. unemployment jumped far higher than in Europe or Japan

Data: OECD, Key Economic Indicators (DSD_KEI@DF_KEI, measure UNEMP, monthly), through DBnomics. Unemployment across countries: Chapter 7 of Jones (2024), Macroeconomics, 6th ed.

Europe and Japan protected jobs; U.S. firms laid off and rehired

  • The U.S. rate jumped far higher in 2020 than the others.
  • European countries and Japan protected jobs with short-time work schemes.
  • U.S. firms laid workers off and rehired them later.

The same shock, different institutions, different unemployment: one chart from one DBnomics call.

How fresh is the DBnomics copy of the BIS policy rates?

Look up the U.S. series of central bank policy rates from the BIS on DBnomics, address "BIS/WS_CBPOL/M.US". What is the latest month in the DBnomics copy? Try "M.XM" (euro area) and "M.JP" (Japan) too.

Three doors to the same Eurostat table

Door Way How you name the data
sdmx1 2: SDMX estat.data("une_rt_a", key="A.Y15-74.PC_ACT.T.ES+DE")
eurostat 1: specialized get_data_df("une_rt_a", filter_pars={"age": "Y15-74", ...})
dbnomics 3: collection fetch_series("Eurostat", "une_rt_a", dimensions={"geo": ["ES", "DE"], ...})

Same table, three ways to name it. Learn the dimensions once, and every door opens.

Files on the web

Act V

Some key datasets have no library: pandas reads their files

File type pandas function Example
CSV (.csv) pd.read_csv(address) FRED, Our World in Data, this book’s files
Excel (.xlsx) pd.read_excel(address, sheet_name=...) Maddison Project Database
Stata (.dta) pd.read_stata(address) Penn World Table

The only hard part is finding the address of the file.

The Penn World Table compares productivity across countries

  • Real GDP, employment, hours, capital and productivity, comparable across countries: 185 countries, 1950–2023.
  • The dataset of Chapters 3 to 6.
  • Stored at DataverseNL, a data archive. Each dataset has a permanent identifier (a DOI).
  • Dataverse has an API that lists the files of a dataset with their numbers.

Ask the archive’s API for the list of files

import requests

files_api = "https://dataverse.nl/api/datasets/:persistentId/versions/:latest/files"
reply = requests.get(files_api, params={"persistentId": "doi:10.34894/FABVLR"})   # PWT 11.0
pwt_files = pd.json_normalize(reply.json()["data"])
pwt_files[["dataFile.id", "dataFile.filename", "dataFile.filesize"]]
  • requests downloads anything from the web.
  • .json() reads the reply (nested lists and dictionaries).
  • pd.json_normalize flattens it into a table.

Twelve files; the main one is pwt110.dta, number 554030

    dataFile.id                    dataFile.filename  dataFile.filesize
0        554047          program_package_pwt_110.zip            4883267
1        554026            pwt110_capital_detail.dta            1793875
2        554029           pwt110_capital_detail.xlsx            3948645
3        554030                           pwt110.dta            3739965
4        554028              pwt110_labor_detail.dta            2723875
5        554024                   pwt110_na_data.dta            2092044
...
9        554025  pwt110_user_guide_to_data_files.pdf             119363
10       554031                  pwt110_whatsnew.pdf             394645
11       554105                          pwt110.xlsx            5839841

The download address of a file is https://dataverse.nl/api/access/datafile/ followed by its id.

Look up the id, build the address, read the Stata file

PWT_ID = pwt_files.loc[pwt_files["dataFile.filename"] == "pwt110.dta", "dataFile.id"].iloc[0]
PWT_URL = f"https://dataverse.nl/api/access/datafile/{PWT_ID}"
print("Downloading", PWT_URL, "... (about 4 MB)")
pwt = pd.read_stata(PWT_URL)
print(pwt.shape[0], "rows,", pwt.shape[1], "columns,", pwt["countrycode"].nunique(), "countries,",
      int(pwt["year"].min()), "to", int(pwt["year"].max()))
pwt[["countrycode", "country", "year", "rgdpo", "emp", "pop"]].tail()

How many rows? Think: 185 countries times the years from 1950 to 2023.

13,690 rows: 185 countries × 74 years, in long format

Downloading https://dataverse.nl/api/access/datafile/554030 ... (about 4 MB)
13690 rows, 51 columns, 185 countries, 1950 to 2023
      countrycode   country  year         rgdpo       emp        pop
13685         ZWE  Zimbabwe  2019  45106.468750  5.272343  15.271368
13686         ZWE  Zimbabwe  2020  45983.898438  5.206007  15.526888
13687         ZWE  Zimbabwe  2021  44440.199219  5.298346  15.797210
13688         ZWE  Zimbabwe  2022  48054.238281  5.344455  16.069056
13689         ZWE  Zimbabwe  2023  50540.847656  5.504774  16.340822
  • rgdpo: real GDP (output side), millions of 2021 dollars at PPP.
  • emp: people employed, millions. pop: population, millions.

GDP per worker, relative to the United States

\[\text{GDP per worker} = \frac{\texttt{rgdpo}}{\texttt{emp}} \qquad \text{relative to U.S.} = 100 \times \frac{\text{GDP per worker}_{i,t}}{\text{GDP per worker}_{US,t}}\]

pwt["gdp_per_worker"] = pwt["rgdpo"] / pwt["emp"]
us = pwt[pwt["countrycode"] == "USA"][["year", "gdp_per_worker"]]
us = us.rename(columns={"gdp_per_worker": "us_gdp_per_worker"})
pwt = pwt.merge(us, on="year")
pwt["relative_to_us"] = 100 * pwt["gdp_per_worker"] / pwt["us_gdp_per_worker"]

catch_up = pwt[pwt["countrycode"].isin(["KOR", "DEU", "MEX", "BRA", "CHN", "IND"])
               & (pwt["year"] >= 1960)]
fig = px.line(catch_up, x="year", y="relative_to_us", color="country",
              title="GDP per worker relative to the United States, Penn World Table 11.0",
              labels={"year": "", "relative_to_us": "GDP per worker, United States = 100",
                      "country": ""})

Catching up with U.S. productivity

Which of these economies has closed the gap with U.S. GDP per worker since 1960, and has any fallen further behind?

Korea, Germany, Mexico, Brazil, China, India: pick one climber and one faller.

Korea climbed from under a tenth to over half of U.S. productivity

Data: Feenstra, Inklaar and Timmer (2015), Penn World Table 11.0, pwt110.dta from DataverseNL; GDP per worker is rgdpo / emp. Related to Chapters 3 and 4 of Jones (2024), Macroeconomics, 6th ed.

Korea and China caught up; Mexico fell behind

  • Korea: less than a tenth of U.S. GDP per worker in 1960, well over half by the 2020s.
  • China’s climb starts later, around 2000.
  • Mexico moved the other way: relative to the United States, its workers produce much less today than in 1980.

One API call to find the file, one read_stata to load it: no library needed.

The Maddison Project: GDP per person over two thousand years

  • One Excel file with several sheets: Notes, Sources, GDPpc, Population, Full data, …
  • The sheet Full data is already in long format.
  • Stored at DataverseNL too, so the same kind of address works.
MADDISON_URL = "https://dataverse.nl/api/access/datafile/421302"     # mpd2023_web.xlsx
print("Downloading", MADDISON_URL, "... (about 5 MB, may take 20 seconds)")
maddison = pd.read_excel(MADDISON_URL, sheet_name="Full data")
print(maddison.shape[0], "rows,", maddison["countrycode"].nunique(), "countries,",
      maddison["year"].min(), "to", maddison["year"].max())
maddison.tail()

131,144 rows from the year 1 to 2022, and population in thousands

Downloading https://dataverse.nl/api/access/datafile/421302 ... (about 5 MB, may take 20 seconds)
131144 rows, 169 countries, 1 to 2022
       countrycode   country              region  year        gdppc       pop
131139         ZWE  Zimbabwe  Sub Saharan Africa  2018  1900.199191  14096.56
131140         ZWE  Zimbabwe  Sub Saharan Africa  2019  1753.024449  14344.59
131141         ZWE  Zimbabwe  Sub Saharan Africa  2020  1585.972804  14614.82
131142         ZWE  Zimbabwe  Sub Saharan Africa  2021  1687.253154  14900.01
131143         ZWE  Zimbabwe  Sub Saharan Africa  2022  1703.529373  15204.66

Zimbabwe, 2022: population 15204.66 here (thousands) but 16.07 in the Penn World Table (millions). Same country, same year, different units and estimates.

Two centuries of GDP per person on a ratio scale

long_run = maddison[maddison["countrycode"].isin(["GBR", "USA", "JPN", "CHN", "IND"])]
long_run = long_run[long_run["year"] >= 1820].dropna(subset=["gdppc"])

fig = px.line(long_run, x="year", y="gdppc", color="country", log_y=True,
              title="Two centuries of GDP per person, Maddison Project Database 2023",
              labels={"year": "", "gdppc": "GDP per person, 2011 dollars (ratio scale)",
                      "country": ""})
fig.update_yaxes(tickvals=LOG_TICKS, tickformat=",")
fig.show()

dropna(subset=["gdppc"]) drops only the rows where GDP per person is missing; the early years are patchy.

Two centuries of GDP per person

In 1820, how far apart were Britain, the United States, Japan, China and India? Close together, or already far apart?

Around 1820 the five economies were much closer than today

Data: Bolt and van Zanden (2024), Maddison Project Database 2023, mpd2023_web.xlsx from DataverseNL. Inspired by the long-run growth facts of Chapter 3 of Jones (2024), Macroeconomics, 6th ed.

Britain and the U.S. pulled away first; Asia caught up later

  • Around 1820 the five economies were much closer than today.
  • The United Kingdom and the United States pulled away first.
  • Japan caught up in the 20th century; China and India took off only in the last few decades.

One Excel file, one sheet name: two thousand years of data in one line of pandas.

Our World in Data: every chart has a CSV at its address + .csv

  • Our World in Data (OWID) turns data from many sources into charts.
  • The chart at ourworldindata.org/grapher/share-of-population-in-extreme-poverty has its data at the same address plus .csv.
  • OWID asks programs to say who they are, with a User-Agent header (a short label sent with the request).
OWID_URL = "https://ourworldindata.org/grapher/share-of-population-in-extreme-poverty.csv"
poverty = pd.read_csv(OWID_URL, storage_options={"User-Agent": "Our World In Data data fetch/1.0"})
SHARE = poverty.columns[3]           # the column with the poverty rate (its name is long)
print("Column:", SHARE)
poverty.tail()

The poverty column, and the empty years at the end

Column: Share of population in poverty ($3 a day)
         Entity Code  Year  Share of population in poverty ($3 a day)  \
59367  Zimbabwe  ZWE  2019                                  49.219894
59368  Zimbabwe  ZWE  2020                                        NaN
59369  Zimbabwe  ZWE  2021                                        NaN
59370  Zimbabwe  ZWE  2022                                        NaN
59371  Zimbabwe  ZWE  2023                                        NaN
...
  • The poverty line: $3 a day, in 2021 prices.
  • Zimbabwe’s last survey value is 2019 (49.2%); later years are empty.

Picking a column by position (columns[3]) breaks if the file gains a column: print its name to check.

Five regions since 1990

regions = ["World", "Sub-Saharan Africa (WB)", "South Asia (WB)", "East Asia and Pacific (WB)",
           "Latin America and Caribbean (WB)"]
poor = poverty[poverty["Entity"].isin(regions) & (poverty["Year"] >= 1990)]

fig = px.line(poor, x="Year", y=SHARE, color="Entity",
              category_orders={"Entity": regions},
              title="Share of people in extreme poverty, by region, Our World in Data",
              labels={"Year": "", SHARE: "Percent of the population", "Entity": ""})
fig.show()

The regions are rows of the same table: their names end in “(WB)”, the World Bank’s regions.

Extreme poverty by region

Where did extreme poverty fall most since 1990, and where do most of the world’s extreme poor live today?

East Asia’s extreme poverty fell from a majority to almost nothing

Data: World Bank, Poverty and Inequality Platform, as processed by Our World in Data (below $3 a day, 2021 prices); recent years are World Bank projections from older surveys. Related to Chapter 3 of Jones (2024), Macroeconomics, 6th ed.

Growth in China cut poverty; Sub-Saharan Africa now holds most of the poor

  • Extreme poverty in East Asia fell from a majority of the population in 1990 to almost nothing, mostly because of growth in China.
  • Sub-Saharan Africa now holds most of the world’s extreme poor.
  • The most recent years are World Bank projections from older household surveys.

A chart address plus .csv and a User-Agent: that is all OWID needs.

Every chapter of this book reads its data the same way

# The U.S. unemployment rate prepared for Chapter 7 (folder data/ch07)
book_unrate = pd.read_csv(BOOK_DATA + "ch07/fig7-2.csv", parse_dates=["date"])
print("From", f"{book_unrate['date'].min():%B %Y}", "to", f"{book_unrate['date'].max():%B %Y}")
book_unrate.tail()
From January 1950 to September 2026
          date  unemployment_rate
915 2026-05-01                4.3
916 2026-06-01                4.2
917 2026-07-01                4.1
918 2026-08-01                4.1
919 2026-09-01                4.2

Human capital in Korea, India and the United States

Use the Penn World Table already in memory (pwt) to draw the human capital index hc for Korea, India and the United States since 1950.

One number, many sources

Act VI

How fast has U.S. income per person grown since 2000? Ask five sources

Source Series Unit
FRED A939RX0Q048SBEA real GDP per capita, quarterly → averaged over each year
World Bank NY.GDP.PCAP.KD constant 2015 dollars
IMF WEO NGDPRPC GDP per capita, constant prices, national currency
Penn World Table rgdpna / pop real GDP from the national accounts, per person
Maddison Project gdppc 2011 dollars

We already know how to reach every one of them. Now we put them in one table.

FRED is quarterly: average the four quarters of each year

# FRED: quarterly data, so we average the four quarters of each year
fred_pc = read_fred("A939RX0Q048SBEA")
fred_pc["year"] = fred_pc["date"].dt.year
fred_pc = fred_pc[fred_pc["year"] < pd.Timestamp.today().year]     # complete years only
fred_pc = fred_pc.groupby("year")["A939RX0Q048SBEA"].mean().rename("FRED")

Why drop the current year before averaging?

In October 2026 only the first quarters of 2026 exist: their average is not an annual value comparable with the other sources.

The World Bank needs a transpose; the IMF needs numeric years

# World Bank
wb_pc = wb.data.DataFrame("NY.GDP.PCAP.KD", "USA", time=range(1990, 2030), numericTimeKeys=True)
wb_pc = wb_pc.T["USA"].rename("World Bank")    # .T swaps rows and columns: years become rows

# IMF World Economic Outlook
imf_pc = imfp.imf_get("WEO", {"COUNTRY": "USA", "INDICATOR": "NGDPRPC"})
imf_pc["year"] = imf_pc["TIME_PERIOD"].astype(int)
imf_pc = imf_pc.set_index("year")["OBS_VALUE"].rename("IMF WEO")
print(fred_pc.tail(3), wb_pc.tail(3), imf_pc.tail(3), sep="\n\n")
  • The World Bank table is wide (one column per year): .T turns years into rows.
  • Each source becomes a pandas Series indexed by year and named after its source.

The IMF column runs to 2031: forecasts are mixed in

year
2023    67412.25
2024    68788.50
2025    70020.25
Name: FRED
2023    65667.28158
2024    66856.51317
2025    67945.61465
Name: World Bank
year
2029    75400.028306
2030    76495.271236
2031    77616.076824
Name: IMF WEO
  • FRED and the World Bank end in 2025; the IMF’s last years are forecasts to 2031.
  • The levels differ too ($70,020 versus $67,946 in 2025): different base years.

Penn World Table and Maddison: reuse the tables already in memory

# Penn World Table and Maddison: already in memory from B.9
us_pwt = pwt[pwt["countrycode"] == "USA"].copy()
us_pwt["gdp_pc"] = us_pwt["rgdpna"] / us_pwt["pop"]
pwt_pc = us_pwt.set_index("year")["gdp_pc"].rename("Penn World Table")
pwt_pc.index = pwt_pc.index.astype(int)

mad_pc = maddison[maddison["countrycode"] == "USA"].set_index("year")["gdppc"].rename("Maddison")

# One table: pd.concat puts the five columns side by side, matching the years
sources = pd.concat([fred_pc, wb_pc, imf_pc, pwt_pc, mad_pc], axis=1).sort_index()
sources.loc[2000:]

pd.concat(..., axis=1) matches the five Series on their year index, leaving NaN where a source has no value.

Five columns, five units: the levels cannot be compared directly

          FRED    World Bank       IMF WEO  Penn World Table      Maddison
2000  49914.50  48616.254628  49933.506536      55171.319764  45886.470498
...
2010  54188.25  52812.682987  54206.538072      59465.698228  49266.915862
...
2020  61127.00  59533.609024  61225.660175      65783.138914  54379.206512
2021  64828.50  63096.785615  64839.516732      69617.537868  57522.704626
2022  66037.25  64322.233282  66057.138042      71079.677954  58487.465861
2023  67412.25  65667.281580  67438.383199      72718.486651           NaN
2024  68788.50  66856.513170  68810.761710               NaN           NaN
2025  70020.25  67945.614650  70032.247214               NaN           NaN
...
2031       NaN           NaN  77616.076824               NaN           NaN

In 2000 the levels range from about $45,900 (Maddison) to $55,200 (Penn World Table).

Index each source to 2000 = 100, then compute average growth

\[\text{index}_t = 100 \times \frac{y_t}{y_{2000}} \qquad g = \left(\frac{y_{\text{end}}}{y_{2000}}\right)^{1/(\text{end} - 2000)} - 1\]

END = int(sources.dropna().index.max())       # the last year with all five sources
index = 100 * sources.loc[2000:END] / sources.loc[2000]
growth = 100 * ((sources.loc[END] / sources.loc[2000]) ** (1 / (END - 2000)) - 1)
print(f"Average annual growth of U.S. real GDP per person, 2000-{END}, percent:")
print(growth.round(2).to_string())
  • An index removes the units: every source starts at 100.
  • \(g\) is the constant yearly growth rate that takes \(y_{2000}\) to \(y_{\text{end}}\).

Will the five sources agree on the growth rate?

Same country, same years, same concept. Will the five average growth rates for 2000–2022 be equal, almost equal, or clearly different? Which sources will agree?

Three sources say 1.28% a year; the research datasets say less

Average annual growth of U.S. real GDP per person, 2000-2022, percent:
FRED                1.28
World Bank          1.28
IMF WEO             1.28
Penn World Table    1.16
Maddison            1.11
  • FRED, the World Bank and the IMF agree exactly: 1.28%. All three copy the latest official U.S. numbers.
  • The Penn World Table (1.16%) and the Maddison Project (1.11%) show slower growth.
  • The gap: 0.17 percentage points a year, every year, for 22 years.

U.S. real GDP per person in five sources

Where in the 22 years do the five lines part ways: at once, slowly, or after one event?

The five sources tell the same story with different numbers

Data: BEA through FRED (A939RX0Q048SBEA); World Bank WDI (NY.GDP.PCAP.KD); IMF WEO (NGDPRPC); Penn World Table 11.0 (rgdpna, pop); Maddison Project Database 2023 (gdppc). Related to Chapters 2 and 3 of Jones (2024).

Why sources differ: vintages, population and prices

  • Vintages. Each source copied the official U.S. numbers at a different time, and the BEA revises GDP every year. FRED updates within days; the Penn World Table and Maddison every few years.
  • Population. GDP per person divides by population, and the sources use different estimates (Census Bureau, United Nations, their own).
  • Prices and methods. Different base years and, for the Penn World Table and Maddison, their own international prices.

Always say where a number comes from, and compare sources when a result matters.

Different sources do not mean one of them is wrong

The sources disagree, so one of them must have made a mistake.

All five can be right about what they measure. They copied the official data at different dates, divided by different population estimates and used different prices. The error would be to report a number without its source.

Does the gap shrink after 2010?

Repeat the comparison for 2010 to the last common year. Are the differences between the sources larger or smaller than for 2000 onwards?

Good habit: save what you download, with the date in the name

# Save a table with the download date in its name
file_name = f"us_unemployment_inflation_{TODAY}.csv"
fred.to_csv(file_name, index=False)
print("Saved", file_name)
Saved us_unemployment_inflation_2026-10-17.csv

A Colab session is temporary: download the file from the Files icon, or save it to Google Drive.

Five habits make a download trustworthy

  • Save what you download, with the date in the file name.
  • Write down the source, the series code and the date next to every result.
  • Check the units and the latest date before you compute anything: .tail() is your friend.
  • Keep keys secret: Colab Secrets, never the notebook.
  • Be polite to the servers: download once and reuse the table.

Read the documentation before you trust a series

  • FRED: the notes under every series chart give the units, the frequency, the seasonal adjustment and the original producer.
  • World Bank: every indicator page has a definition, a source and its limitations.
  • Penn World Table: the user guide sits in the file list we downloaded (pwt110_user_guide_to_data_files.pdf).
  • Maddison: the Notes and Sources sheets of the Excel file.
  • Eurostat, ECB, OECD: the metadata page of each dataset explains every dimension code.

Five minutes with the documentation saves an afternoon of wrong results.

Cite a dataset as carefully as a paper

  • Who produced it: the agency or the authors.
  • What: the dataset and its edition (Penn World Table 11.0; WEO, October 2026).
  • Which series: the code (UNRATE, NY.GDP.PCAP.KD).
  • Where: the link or the DOI. When: the download date.

U.S. Bureau of Labor Statistics, Unemployment Rate [UNRATE], retrieved from FRED, Federal Reserve Bank of St. Louis, fred.stlouisfed.org/series/UNRATE, 17 October 2026.

When a download fails, fall back on a copy

try:
    unemployment_rate = read_fred("UNRATE")
    print("Downloaded from FRED")
except Exception as problem:
    print("FRED did not answer:", problem)
    unemployment_rate = pd.read_csv(BOOK_DATA + "ch07/fig7-2.csv", parse_dates=["date"])
    print("Using the copy stored in the book's repository instead")
Downloaded from FRED

Servers are sometimes down, busy or changed. Wait a minute and run the cell again first.

Predict the fix

read_fred("UNRATES") stops with HTTPError: HTTP Error 404: Not Found. What went wrong, and how do you fix it?

A 404 means the address does not exist: here the series code is misspelled (UNRATES). Check the code on the FRED website and copy it exactly.

Common errors, part 1: installing and addressing

Error message What it usually means What to do
ModuleNotFoundError: No module named 'wbgapi' the library is not installed in this session run the %pip install cell again
HTTPError: HTTP Error 404: Not Found the web address or the series code is wrong check the code on the source’s website; copy it exactly
HTTPError: HTTP Error 403: Forbidden the server refused the program add a User-Agent, as for Our World in Data
HTTPError: HTTP Error 429 or 503 too many requests, or the server is busy wait a minute and try again; download less often

Common errors, part 2: inside the data

Error message What it usually means What to do
KeyError: 'value' a column name that does not exist print table.columns and use the exact name
an empty table no data for this combination of codes check each code of the key; try fewer filters
ValueError in pd.read_excel wrong sheet name open the file and check the sheet names
an API key error no key, or a key not visible to the notebook add it in Colab Secrets and switch on Notebook access

Read the last line of the error first: it usually names the problem.

Cheat sheet: one call per source

Source Library (way) The call that downloads
FRED pandas (4) pd.read_csv("https://fred.stlouisfed.org/graph/fredgraph.csv?id=UNRATE")
FRED pandas-datareader (1) web.DataReader(["GDPC1", "PCECC96"], "fred", start="2000-01-01")
FRED fredapi (1, key) Fred(api_key=KEY).get_series("GDPC1")
World Bank wbgapi (1) wb.data.DataFrame("NY.GDP.PCAP.KD", ["USA", "CHN"], time=range(1960, 2030))
IMF imfp (1) imfp.imf_get("WEO", {"COUNTRY": ["USA"], "INDICATOR": "NGDP_RPCH"})
ECB sdmx1 (2) sdmx.Client("ECB").data("EXR", key={...})
Eurostat eurostat (1) eurostat.get_data_df("une_rt_a", filter_pars={...})
OECD, IMF, BIS… dbnomics (3) fetch_series("IMF/WEO:latest/USA.NGDP_RPCH.pcent_change")
Penn World Table pandas (4) pd.read_stata("https://dataverse.nl/api/access/datafile/554030")
Maddison pandas (4) pd.read_excel(".../datafile/421302", sheet_name="Full data")
Our World in Data pandas (4) pd.read_csv(chart_address + ".csv", storage_options={"User-Agent": ...})

Where to learn more

What to remember from this appendix

  • The map: official collectors (FRED, World Bank, IMF, OECD, Eurostat, ECB, BIS) and research datasets (Penn World Table, Maddison, Our World in Data).
  • Four ways in: a specialized library, SDMX, a collection, or a file read straight from its address.
  • Every series has a code: find it first, then check the units and the latest date.
  • Sources differ because of vintages, population estimates and prices: 1.28% versus 1.11% a year.
  • Record where and when you got every number; keep keys secret; save a copy.

Every number has a source, a code, units and a date: record all four, and Python can fetch it again in seconds

Next: Appendix C. Math Review for Macroeconomics

Sources: data shown

Sources: tools and texts