Regional inequality

How unequal Bolivia’s municipalities are, whether that is changing, and whether the inequality lies between departments or inside them: Gini, Theil, the Lorenz curve and Theil’s decomposition.

✨ Run this page yourself: nothing to install on your computer.

▶ Open in Colab

An inequality index puts in one number how far a set of values is from being all the same. This article asks three questions of Bolivia’s municipalities:

  1. How unequal are they? geo.inequality() computes the Gini index, Theil’s index and the coefficient of variation, and draws the Lorenz curve.
  2. Is that changing? With two periods the measures can be compared; with a long panel their trend can be tested.
  3. Is the inequality between departments or inside them? geo.theil_decomposition() splits Theil’s index in two.

It uses two variables on purpose: the Sustainable Development Index, a score between 0 and 100, and population, a count. An inequality index means different things for the two, and the difference is the main thing to take from this page.

!pip install -q geo4bolivia

The data

The dataset sdg has the Sustainable Development Index (index_sdg, IDS) and the census population of the 343 municipalities in two periods, stored as 2012 and 2024.

import geo4bolivia as geo
import numpy as np
import pandas as pd

sdg = geo.load("sdg", groups=["department"])
cols = ["variable", "year", "count", "mean", "std", "min", "median", "max"]
geo.describe(sdg, variables=["index_sdg", "population"])[cols]
variable year count mean std min median max
0 index_sdg 2012 343 50.123032 5.819148 37.2 49.6 70.5
1 index_sdg 2024 343 60.081341 5.263740 47.3 59.8 77.3
2 population 2012 343 29329.026239 107270.841863 509.0 10647.0 1444425.0
3 population 2024 343 33133.915452 116499.083621 875.0 12296.0 1610982.0

The two variables could hardly be more different. The index of 2024 lies between 47.3 and 77.3, around a mean of 60.1. Population in 2024 runs from 875 people to 1.6 million, with a mean (33,134) far above the median (12,296).

NoteInequality between municipalities, not between people

Every measure on this page treats each municipality as one observation, whatever its size: the municipality of 875 people counts as much as the one of 1.6 million. The results describe how different municipalities are. They are not measures of inequality between people or households, which would need data on people.

1. Three measures

res = geo.inequality(sdg, "index_sdg", measures=("gini", "theil", "cv"))
res

Cross-sectional inequality in Sustainable Development Index, IDS (0-100) is tracked across 343 units over 2 years (2012 to 2024).

The Gini index fell from 0.065 to 0.0493 between the first and last year.

The Theil index fell from 0.00663 to 0.00381 between the first and last year.

The coefficient of variation fell from 0.116 to 0.0876 between the first and last year.

With two years (2012 and 2024) the measures can be compared, but their trend cannot be tested: that needs three or more years.

On the Lorenz curve of 2024, the half of the units with the lowest values holds 46% of the total (45% in 2012) and the tenth with the highest values holds 12% (12%); with no inequality they would hold 50% and 10%.

These are associations, not causal effects. A causal reading needs a research design.

All three measures are 0 when every unit has the same value and grow as the values spread out. They summarise the spread differently:

Measure What it is Range Reacts most to
Gini index ("gini") the average difference between two units, over all pairs, divided by twice the mean 0 to 1 differences in the middle of the distribution, where most units are
Theil index ("theil") the average of (value ÷ mean) × ln(value ÷ mean) 0 to ln(number of units) high values more than low ones; it splits exactly into groups (section 6)
Coefficient of variation ("cv") the standard deviation divided by the mean 0 upwards the highest values, because deviations are squared

The Theil index takes logarithms, so it needs strictly positive values; with zeros in the data, use measures=("gini", "cv").

All three are relative measures: they compare the values with their mean. Doubling every value leaves them unchanged. Adding the same amount to every value does not: it lowers all three, because the same gaps are then smaller relative to a higher mean. Keep this in mind for section 3.

.gt is the table of the result, and .df has the same numbers, one row per year:

res.gt
Inequality: Sustainable Development Index, IDS (0-100)
2012 and 2024, 343 units
2012 2024 Change Change (%)
Gini index 0.06501 0.04934 -0.01567 -24.1
Theil index 0.006629 0.003806 -0.002824 -42.6
Coefficient of variation 0.1161 0.08761 -0.02849 -24.5
With two years the measures can be compared, but their trend cannot be tested: that needs three or more years.

By all three measures the index is more evenly spread in 2024: the Gini index fell from 0.065 to 0.049, the Theil index from 0.0066 to 0.0038 and the coefficient of variation from 0.116 to 0.088. With two periods that is all that can be said: the values can be compared, but there is no test of a trend, which needs three or more years (res.notes says so).

2. The Lorenz curve

The Lorenz curve is the picture behind the Gini index. Sort the municipalities from the lowest value to the highest. The curve shows, for each share of municipalities counted from the bottom (horizontal axis), the share of the total of the variable that they hold (vertical axis).

res.fig_lorenz

How to read it:

  • The dashed diagonal is equality: if every municipality had the same value, the lowest 30% of them would hold 30% of the total.
  • The curve can only lie on or below the diagonal. The further it sags, the more unequal the values.
  • The Gini index is the area between the diagonal and the curve, as a share of the whole triangle under the diagonal.
  • When two curves do not cross, the higher one is the more equal by any of the three measures. When they cross, the measures can disagree, because each weighs the parts of the distribution differently.

.lorenz_share(year, share) reads a point off the curve: the share of the total held by the lowest share of units. .lorenz is the table behind the figure.

for year in [2012, 2024]:
    half = res.lorenz_share(year, 0.5)
    top = 1 - res.lorenz_share(year, 0.9)
    print(year, "| lowest half:", round(half, 3), "| highest tenth:", round(top, 3))
2012 | lowest half: 0.454 | highest tenth: 0.123
2024 | lowest half: 0.465 | highest tenth: 0.116

For the index the curves almost lie on the diagonal. The half of the municipalities with the lowest index holds 45.4% of the sum of the index in 2012 and 46.5% in 2024, where equality would give 50%; the tenth with the highest index holds about 12%, where equality would give 10%. The curve of 2024 lies above the curve of 2012 along its whole length, which is why all three measures agree that the spread fell.

3. What an index of a score can and cannot say

Now the same call on population.

pop = geo.inequality(sdg, "population", measures=("gini", "theil", "cv"))
pop.fig_lorenz
print(pop.df.round(3))
half, top = pop.lorenz_share(2024, 0.5), 1 - pop.lorenz_share(2024, 0.9)
print("lowest half:", round(half, 3), "| highest tenth:", round(top, 3))
   year  n_units   gini  theil     cv
0  2012      343  0.705  1.408  3.657
1  2024      343  0.704  1.366  3.516
lowest half: 0.098 | highest tenth: 0.642

The Gini index of population is 0.70 in both censuses. The half of the municipalities with the fewest people holds 9.8% of the population in 2024, and the tenth with the most people holds 64%.

It is tempting to conclude that population is fourteen times more unequal than development (0.70 against 0.05). That comparison means nothing, for three reasons.

A count has a total that is shared out; a score does not. For population the Lorenz curve can be read literally: 64% of the people live in a tenth of the municipalities. For the index, “the total” is the sum of 343 scores, which is not an amount of anything and which no one holds. The curve and the Gini index of a score still describe how spread out the scores are, relative to their mean, but the reading “the lowest half holds 46% of the total” is a statement about arithmetic, not about who has what.

A bounded score cannot be very unequal. Population can differ by a factor of more than a thousand between two municipalities. An index that lies between 47 and 77 cannot: its highest value is 1.6 times its lowest. Its Gini index is bound to be small whatever happens in the country. A small value is a property of the scale, so compare the index with itself, between years or between groups of municipalities, not with other variables.

The direction of the change depends on how the score is written. A score can be turned around: an index of what has been achieved, or an index of what is missing. The two hold the same information, but relative measures do not treat them alike. Multidimensional poverty (sdg1_2_mpi, from 0 to 1, lower is better) shows it clearly. It fell in every municipality between the two censuses.

mpi = geo.load("sdg01", variables="sdg1_2_mpi")
mpi["not_poor"] = 1 - mpi["sdg1_2_mpi"]
rows = []
for variable in ["sdg1_2_mpi", "not_poor"]:
    gini = geo.inequality(mpi, variable, measures="gini", lorenz=False).df
    spread = mpi.groupby("year")[variable].agg(["mean", "std"]).reset_index()
    rows.append(gini.merge(spread, on="year").assign(variable=variable))
pd.concat(rows)[["variable", "year", "mean", "std", "gini"]].round(3)
variable year mean std gini
0 sdg1_2_mpi 2012 0.234 0.106 0.254
1 sdg1_2_mpi 2024 0.063 0.044 0.367
0 not_poor 2012 0.766 0.106 0.078
1 not_poor 2024 0.937 0.044 0.025

The Gini index of poverty rose, from 0.254 to 0.367, while the Gini index of its mirror image, one minus the poverty index, fell, from 0.078 to 0.025. Both describe the same municipalities in the same years. The standard deviation, which is identical for the two, fell from 0.106 to 0.044: the municipalities are closer to each other in index points. But mean poverty fell even faster, so the remaining gaps are larger relative to the mean of poverty, and smaller relative to the mean of its mirror image.

Neither Gini is wrong. “Poverty became more concentrated in a few municipalities” and “municipalities became more alike in how much of their population is out of poverty” are both true. The lesson is to report the mean and the standard deviation next to any relative measure of a score, and to say which way the score runs.

The Sustainable Development Index runs upwards (100 is best), and its mean rose by ten points. How much of the fall of its inequality is the higher mean?

spread = sdg.groupby("year")["index_sdg"].agg(["mean", "std"])
spread["cv"] = spread["std"] / spread["mean"]
print(spread.round(3))
print(round(spread.loc[2012, "std"] / spread.loc[2024, "mean"], 3))
        mean    std     cv
year                      
2012  50.123  5.819  0.116
2024  60.081  5.264  0.088
0.097

The standard deviation fell by about a tenth, from 5.82 to 5.26 points, and the coefficient of variation by about a quarter, from 0.116 to 0.088. Had the gaps in points stayed exactly as in 2012 while the mean rose to 60.1, the coefficient of variation would still have fallen, to 0.097. So about two thirds of its fall comes from the higher level and one third from municipalities moving closer together in points. Both parts are real, and they are different facts: “everyone rose by a similar amount” and “the lowest rose more”. The convergence article looks at the second directly.

4. A long panel: testing the trend

With three or more years, geo.inequality() also estimates the trend of each measure: it regresses the logarithm of the measure on the year, so the slope is the average proportional change per year. The dataset pop has a gridded population estimate (WorldPop) for 333 municipalities in every year from 2001 to 2020.

long = geo.inequality("pop", "pop", measures=("gini", "theil", "cv"))
long

Cross-sectional inequality in Population is tracked across 333 units over 20 years (2001 to 2020).

The Gini index rose from 0.696 to 0.715 between the first and last year.

The Theil index rose from 1.37 to 1.41 between the first and last year.

The coefficient of variation was essentially unchanged at 3.59 between the first and last year.

The log-trend of the Gini index is 0.00143 per year (statistically significant at the 1% level), so inequality by this measure is widening.

The log-trend of the Theil index is 0.00129 per year (statistically significant at the 1% level), so inequality by this measure is widening.

The log-trend of the coefficient of variation is 7.5e-05 per year (not statistically significant at conventional levels), so the movement is not distinguishable from a flat trend.

On the Lorenz curve of 2020, the half of the units with the lowest values holds 9% of the total (10% in 2001) and the tenth with the highest values holds 65% (62%); with no inequality they would hold 50% and 10%.

These are associations, not causal effects. A causal reading needs a research design.

long.summary[["measure", "first", "last", "change", "slope", "se", "pvalue"]]
measure first last change slope se pvalue
0 gini 0.695566 0.714839 0.019273 0.001427 0.000011 0.000000e+00
1 theil 1.373319 1.406630 0.033311 0.001290 0.000066 3.943627e-86
2 cv 3.587710 3.586896 -0.000814 0.000075 0.000081 3.563349e-01

The Gini index of municipal population rose from 0.696 in 2001 to 0.715 in 2020. Its slope is 0.0014: the index grew by 0.14% a year on average, and the trend is statistically significant (the p-value is far below 0.01). The Theil index rose at a similar rate (0.13% a year). The coefficient of variation did not move: its slope is not distinguishable from zero (p = 0.36).

So the measures disagree, and the table in section 1 says why they can. The Gini index reacts to the middle of the distribution and the coefficient of variation to its very top. Read together, they say that population became somewhat more concentrated among the municipalities in general, while the distance between the very largest ones and the average did not change.

Three cautions about this result:

  • The series is a modelled estimate that changes smoothly from year to year, which is why the standard error of the Gini trend is so small. The test says that a straight line fits these twenty values closely; it does not have twenty independent measurements behind it.
  • The 333 municipalities are those whose territory did not change between the municipal maps of 2015 and 2025 (geo.info("pop") explains it). The result describes them.
  • first= and last= restrict the years, for instance geo.inequality("pop", "pop", first=2012). A trend over a different window can differ.

With more than four years the Lorenz figure draws only the first and the last year; .lorenz keeps all of them.

5. The spatial Gini: do neighbours differ less?

The Gini index is an average over all pairs of units. spatial=True separates the pairs into two kinds, neighbours and all the others, and reports the part of the Gini index that comes from pairs of neighbours (gini_spatial). A permutation test then asks whether the values are arranged in space so that neighbours differ less than other pairs: it shuffles the values over the map and compares. The neighbours are those of the spatial weights, queen contiguity by default (Maps and spatial dependence explains them); w= takes others.

sp = geo.inequality(sdg, "index_sdg", spatial=True, permutations=999)
linked = geo.weights("municipality").pct_nonzero
print(sp.df.round(5))
print((sp.df["gini_spatial"] / sp.df["gini"]).round(4).tolist(), round(linked, 2))
   year  n_units     gini    theil  gini_spatial  gini_spatial_p
0  2012      343  0.06501  0.00663       0.00077           0.001
1  2024      343  0.04934  0.00381       0.00057           0.001
[0.0119, 0.0116] 1.61

In 2024, pairs of neighbouring municipalities account for 0.00057 of the Gini index of 0.0493, which is 1.2% of it. That share is small mostly because few pairs are neighbours: 1.6% of all pairs of municipalities. The informative part is the comparison of the two percentages and the test. Neighbours are 1.6% of the pairs but contribute 1.2% of the inequality, so two neighbouring municipalities differ less than two municipalities taken at random, and none of the 999 shuffles gave a pattern as strong (gini_spatial_p = 0.001). The same holds in 2012.

This is the spatial clustering of the index, seen from the side of inequality: most of the differences are between places that are far apart. It says nothing about why neighbours are alike. With spatial=True every measure refers to the units that have data in every year and a boundary; res.notes reports any that were left out.

6. Between and within groups

Is the inequality between municipalities a matter of rich and poor departments, or of differences inside each department? Theil’s index can be split exactly in two parts that add up to it:

  • between: the inequality there would be if every municipality had the average of its group;
  • within: the inequality among the municipalities of the same group, averaged over the groups.

by= names the groups: a column of the table, or a coarser scale that the units nest in.

dep = geo.theil_decomposition(sdg, "index_sdg", by="department",
                              permutations=999)
dep

The Theil index of Sustainable Development Index, IDS (0-100) splits additively into inequality between the 9 department groups and inequality within them (between + within = total, exactly).

In the latest year (2024), the total Theil index is 0.00381: about 15% of it lies between department groups and 85% within them — most inequality plays out among units inside the same department.

Over the window, the between-group share fell from 20% to 15% — group means are pulling closer together relative to the differences inside groups.

The permutation test on the between component (p = 0.001) indicates the observed grouping captures more inequality than random reassignments of units to groups typically do.

These are associations, not causal effects. A causal reading needs a research design.

dep.df.round(4)
year theil between within between_share p_between
0 2012 0.0066 0.0014 0.0053 0.2043 0.001
1 2024 0.0038 0.0006 0.0032 0.1475 0.001

Differences between the averages of the nine departments are 20% of the inequality between municipalities in 2012 and 15% in 2024. Most of it, 85% in 2024, is inside departments. Both parts fell, the between part faster (from 0.0014 to 0.0006) than the within part (from 0.0053 to 0.0032).

Is the between part more than chance?

Any grouping gives a between part above zero, even one drawn from a hat, because group averages always differ a little. permutations= tests the grouping: it reassigns the municipalities to groups of the same sizes at random, many times, and counts how often the random groups give a between part as large as the real ones (p_between). Here none of 999 did (p = 0.001): the departments separate municipalities with higher and lower values more than an arbitrary grouping would, although they hold only 15% of the inequality.

More groups, a larger between share

The same decomposition by province gives a very different number.

prov = geo.theil_decomposition(sdg, "index_sdg", by="province",
                               permutations=999)
prov.df.round(4)
year theil between within between_share p_between
0 2012 0.0066 0.0042 0.0024 0.6380 0.001
1 2024 0.0038 0.0025 0.0013 0.6578 0.001

Between the 113 provinces lies 66% of the inequality in 2024. That does not make provinces four times as important as departments. The between share always grows when the groups are made finer, for a mechanical reason: with fewer units in each group there is less left to differ inside it. At the limit, with every municipality its own group, the between share is 100%. The provinces have three municipalities each on average, and 16 of them have only one, so they contribute nothing to the within part by construction.

How large is the mechanical part? Shuffle the municipalities among the groups, keeping the sizes of the groups, and see what share random groups give. This is what the permutation test does internally.

units = geo.units("municipality")[["geo_id", "department_id", "province_id"]]
rng = np.random.default_rng(1)
rows = []
for by, real in [("department_id", dep), ("province_id", prov)]:
    shares = []
    for _ in range(50):
        shuffled = units.assign(group=rng.permutation(units[by]))
        table = sdg.merge(shuffled[["geo_id", "group"]], on="geo_id")
        random = geo.theil_decomposition(table, "index_sdg", by="group")
        shares.append(random.df["between_share"].iloc[-1])
    rows.append({"groups": by, "number": units[by].nunique(),
                 "between share": real.df["between_share"].iloc[-1],
                 "random groups (mean)": np.mean(shares)})
pd.DataFrame(rows).round(3)
groups number between share random groups (mean)
0 department_id 9 0.147 0.024
1 province_id 113 0.658 0.322

Nine random groups of the sizes of the departments hold about 2% of the inequality; the real departments hold 15%. A hundred and thirteen random groups of the sizes of the provinces hold about a third; the real provinces hold 66%. Both real groupings are well above their random counterpart, and that comparison, not the raw share, is what can be set side by side. Compare between shares only for groupings with a similar number of groups, or against the random benchmark.

The same split for population

people = geo.theil_decomposition(sdg, "population", by="department",
                                 permutations=999)
people.df.round(3)
year theil between within between_share p_between
0 2012 1.408 0.084 1.325 0.059 0.831
1 2024 1.366 0.091 1.275 0.067 0.690

For population the departments hold 6 to 7% of the inequality between municipalities, and random groupings give as much or more in most of the shuffles (p = 0.83 in 2012 and 0.69 in 2024): the between part is not distinguishable from chance. The large differences in population are between the municipalities inside each department, not between the departments’ averages.

In short

  • geo.inequality() gives the Gini index, Theil’s index and the coefficient of variation in each year. They are relative measures and react to different parts of the distribution, so they can disagree, as they do for population between 2001 and 2020.
  • The Lorenz curve shows the whole distribution: the further it sags below the diagonal, the more unequal the values. .lorenz_share() reads points off it.
  • The Sustainable Development Index is more evenly spread across municipalities in 2024 than in 2012 by all three measures (Gini 0.065 to 0.049). About two thirds of the fall of its coefficient of variation comes from the higher mean, one third from smaller gaps in points.
  • An inequality index of a bounded score is small by construction, has no “total” that is shared out, and can change direction when the score is turned around. Compare it with itself, and report the mean and the standard deviation with it.
  • With two periods the measures can be compared; a trend test needs three or more years.
  • The spatial Gini shows that neighbouring municipalities differ less than others.
  • Differences between departments are 15% of the inequality of the index in 2024, more than random groups would give. Between shares grow with the number of groups, so compare them against a random benchmark.
  • The units are municipalities, each counted once. None of this measures inequality between people, and none of it explains where the differences come from.

Where next