Theil index

From Wikipedia, the free encyclopedia

Jump to: navigation, search

The Theil index[1], derived by econometrician Henri Theil, is a statistic used to measure economic inequality.

Contents

The formula is

T=\frac{1}{N}\sum_{i=1}^N \left( \frac{x_i}{\overline{x}} \cdot \ln{\frac{x_i}{\overline{x}}} \right)

where xi is the income of the ith person, \overline{x}= \frac{1}{N} \sum_{i=1}^N x_i is the mean income, and N is the number of people. The first term inside the sum can be considered the individual's share of aggregate income, and the second term is that person's income relative to the mean. If everyone has the same (i.e., mean) income, then the index is 0. If one person has all the income, then the index is ln N.

The Theil index is derived from Shannon's measure of information entropy. Letting T be the Theil index and S be Shannon's information entropy measure,

T=\ln(N)-S. \,

Shannon derived his entropy measure in terms of the probability of an event occurring. This can be interpreted in the Theil index as the probability a dollar drawn at random from the population came from a specific individual. This is the same as the first term, the individual's share of aggregate income.

With reference to information theory[2], Theil's measure is a redundancy rather than an entropy. The redundancy of a system at a given time is the difference between its maximum entropy and its present entropy at that time.[3]

One of the advantages of the Theil index is that it is a weighted average of inequality within subgroups, plus inequality among those subgroups. For example, inequality within the United States is the average inequality within each state, weighted by state income, plus the inequality among states.

If the population is divided into m certain subgroups and sk is the income share of group k, Tk is the Theil index for that subgroup, and \overline{x}_k is the average income in group k, then the Theil index is

T = \sum_{k=1}^m s_k T_k + \sum_{k=1}^m s_k \ln{\frac{\overline{x}_k}{\overline{x}}}.

Another, more popular, measure of inequality is the Gini coefficient. The Gini coefficient is more intuitive to many people since it is based on the Lorenz curve. However, it is not easily decomposable like the Theil.

Theil's index takes an equal distribution for reference which is similar to distributions in statistical physics. An index for an actual system is an actual redundancy, that is, the difference between maximum entropy and actual entropy of that system.

Theil's measure can be converted[3] into one of the indexes of Anthony Barnes Atkinson. The result of the conversion also is called normalized Theil index[4]. James E. Foster[5] used such a measure to replace the Gini coefficient in Amartya Sen's welfare function W=f(income,inequality). The income e.g. is the average income for individuals in a group of income earners. Thus, Foster's welfare function can be computed directly from the Theil index T, if the conversion is included into the computation of the average per capita welfare function:

W = \overline\text{income} \times {e^{-T}}.\,
Map of economic inequality in the United States using the Theil Index. A high positive theil index indicates more income than population while a negative value shows more population than income. A value of zero shows equality between population and income.

Note: This image is not the Theil Index in each area of the United States, but of contributions to the US Theil Index by each area (the Theil Index is always positive, individual contributions to the Theil Index may be negative or positive).

For the income distributions provided by the The World Income Inequality Database (2007-05) the difference between their symmetrized Theil indices and their Hoover indices are plotted over their respective Gini indices. The difference illustrates the impact of the different inequalities on the information generated by them. Negative values occur for Theil indices, which are smaller than the respective Hoover indices.
For the income distributions provided by the The World Income Inequality Database (2007-05)[6] the difference between their symmetrized Theil indices and their Hoover indices are plotted over their respective Gini indices. The difference illustrates the impact of the different inequalities on the information generated by them. Negative values occur for Theil indices, which are smaller than the respective Hoover indices.

The formula for the Hoover index (also called Robin Hood index) H is:

H = {\frac{1}{2}} \sum_{i=1}^N \color{Blue} \left| \color{Black} {\frac{{E}_i}{{E}_\text{total}}} - {\frac{{A}_i}{{A}_\text{total}}} \color{Blue} \right| \color{Black}.

A comparison of the Hoover index and the Theil index gives sense to of both indices:

  • For the Hoover index, the relative deviations in each quantile are summed up. Each deviation is weighted by its own sign (+1 or −1). Thus, the Hoover index is the most simple inequality measure. It has no normative foundations and does not refer to any models from physics or information theory.
  • For the symmetrized Theil index, the relative deviations in each quantile are summed up as well. But each deviation is weighted by its relative information weight. Thus, the Theil index is an indicator not only for the plain relative inequality, it also attempts to indicate how much attention inequality can get.

The following formulas illustrate that difference in the categories symmetry and percevability. For the formulas, a notation[7] is used, where the amount N of quantiles only appears as upper border of summations. Thus, inequities can be computed for quantiles with different widths Ai. For example, Ei could be the income in the quantile #i and Ai could be the amount (absolute or relative) of earners in the quantile #i. Etotal then would be the sum of incomes of all N quantiles and Atotal would be the sum of the income earners in all N quantiles.

Computation of the (asymmetric) Theil index T [8]:

T = \ln{\frac{{A}_\text{total}}{{E}_\text{total}}} - \frac{\sum_{i=1}^N {{E}_i} \ln{\frac{{A}_i}{{E}_i}}}{{E}_\text{total}}.

With normalized data, E'i = Ei / Etotal and A'i = Ai / Atotal would apply. This would simplify the formula:
 \color{Gray} T = 0 - \frac{\sum_{i=1}^N {{E}'_i} \ln{\frac{{A}'_i}{{E}'_i}}}{1} = \sum_{i=1}^N {{E}'_i} \ln{\frac{{E}'_i}{{A}'_i}}

Computation of the symmetrized Theil index Ts:

T_s = \frac{1}{2} \left( \ln{\frac{{A}_\text{total}}{{E}_\text{total}}} - \frac{\sum_{i=1}^N {{E}_i} \ln{\frac{{A}_i}{{E}_i}}}{{E}_\text{total}} + \ln{\frac{{E}_\text{total}}{{A}_\text{total}}} - \frac{\sum_{i=1}^N {{A}_i} \ln{\frac{{E}_i}{{A}_i}}}{{A}_\text{total}} \right).

This leads to:

T_s = {\frac{1}{2}} \sum_{i=1}^N \color{Blue} \ln{\frac{{E}_i}{{A}_i}} \left( \color{Black} {\frac{{E}_i}{{E}_\text{total}}} - {\frac{{A}_i}{{A}_\text{total}}} \color{Blue} \right) \color{Black}.

The difference between the Hoover index and the symmetrized Theil index only is the operation in the deviation from equity Ei / Etotal − Ai / Atotal.

The property of not being a measure with a closed scale between 0 and 1 (or 0% and 100%), like in case of the Gini index, is a barrier, which to overcome seems to be difficult even for famous scientists: Theil's index "is not a measure that is exactly overflowing with intuitive sense," wrote Amartya Sen in a book[5], in which his co-author James Foster used the Theil index nevertheless. One way to overcome this obstacle is the normalized[4] Theil index Tnormalized = 1 − e − T.

The alternative is, not to normalize the index and to use it as it is due to an interesting property of that index: For resource distributions described by only two quantiles, the Theil index is 0 for 50:50 distributions and reaches 1 at 82:18[9], which is very close to a distribution often referred to as "Pareto Principle". Higher inequities yield Theil indices above 1. This leads to a comparison, which yields to intuition:

  • The Gini index is 0 if the distribution is completely equal. It is 1 at maximum inequality.
  • The Theil index is 0 if the distribution is completely equal. It is 1 for an inequality, which is slightly above the equivalent to the frequently cited 80:20 distribution.

A Theil index T can be found for any A:B distribution in societies, which are split into two quantiles. The height A of the 1st quantile is the height B of the 2nd quantile. The width B of the 1st quantile is the width B of the 2nd quantile. First the Gini index G is calculated from the A:B distribution:

G=\left|2A-1 \right|

Then:

T = 2 \cdot G \cdot artanh \left( G \right).

For these computations the range 0 to 1 has to be used for a and b instead of 0% to 100%.

  1. ^ Introduction to the Theil index from the University of Texas
  2. ^ ISO/IEC DIS 2382-16:1996 Information theory
  3. ^ a b http://www.poorcity.richcity.org (Redundancy, Entropy and Inequality Measures)
  4. ^ a b Juana Domínguez-Domínguez, José Javier Núñez-Velázquez: The Evolution of Economic Inequality in the EU Countries During the Nineties, 2005
  5. ^ a b James E. Foster and Amartya Sen, 1996, On Economic Inequality, expanded edition with annexe, ISBN 0-19-828193-5
  6. ^ http://www.wider.unu.edu/wiid/wiid.htm
  7. ^ The notation using E and A follows the notation of a small calculus published by Lionnel Maugis: Inequality Measures in Mathematical Programming for the Air Traffic Flow Management Problem with En-Route Capacities (für IFORS 96), 1996
  8. ^ (1) The first part of the formula is the maximum entropy of the E-A-system. The second part (after the minus symbol) is the real entropy of the E-A-system at a certain time. Such a difference is called redundancy (ISO/IEC DIS 2382-16, information theory).
    (2) This version of Theil's formula allows to process quantiles with different widths Ai. N only serves as summation index.
    (3) Besides mathematical comparison of this formula to the formulas found in many calculuses, you can compare the results 1A and 1B yielded by this formula with the examples 1A and 1B given in The Theoretical Basics of Popular Inequality Measures (Travis Hale, University of Texas Inequality Project, 2003).
  9. ^ Example: 82.4% of the people own 17.6% of all ressources and 17.6% own 82.4% of all ressources. For computation see also http://www.poorcity.richcity.org/calculator/?quantiles=82.4,17.6|17.6,82.4
Retrieved from "http://en.wikipedia.org/wiki/Theil_index"
Advanced Search
Included Web Search Engines


Safe Search

close

Top Matching Results

Occasionally Search.com will highlight specialized results that are based on the context of your query. Examples of specialized results include specific links to news, images, or video.

Top Matching Results may highlight information from other Search.com pages, content from the CNET Network of sites, or third party content. The listings are based purely on relevance. Search.com does not receive payment for listings in this section but our partners that provide this data may get paid for listing these products.

Sponsored Links

This section contains paid listings which have been purchased by companies that want to have their sites appear for specific search terms and related content. These listings are administered, sorted and maintained by a third party and are not endorsed by Search.com.

Search Results

Search.com sends your search query to several search engines at one time and integrates the results into one list which has been sorted by relevance using Search.com's proprietary algorithm. You can customize the list of search engines included in your metasearch from the preferences.

The search engines that are used in your metasearch may allow companies to pay to have their Web sites included within the results. To view the Paid Inclusion policy for a specific search engine, please visit their Web site. Search.com does not accept payment or share revenue with any search engine partner for listings in this section.