Abstract
There are many problems in the social sciences that refer to the evaluation of the relative performance of some populations when their members’ achievements are described by a distribution of outcomes over a set of ordered categories. A new method for the evaluation of this type of problems is presented here. That method, called balanced worth, offers a cardinal, complete and transitive evaluation that is based on the likelihood of getting better results. The evaluation of each society is based on the probability of obtaining better results with respect to the others. The balanced worth is a refinement of “the worth” (Herrero and Villar in PLoS ONE 8(12):e84784, 2013. https://doi.org/10.1371/journal.pone.0084784) that overcomes its excessive sensitivity to the differences, due to the presence of ties. We also discuss how this method can be applied for the case of heterogeneous populations and show how it can be applied in different contexts. An empirical example, regarding life satisfaction in Spain is used to illustrate the working of this method.
This is a preview of subscription content, access via your institution.
Notes
 1.
This happens, for instance, when analysing the citation impact of research articles regarding different disciplines. As the number of citations is rather idiosyncratic, one usually takes the shares of the papers in the different percentiles of each discipline to make an analysis independent of the differences in the mean and the variance of the citations distributions.
 2.
Note that, for a given problem, the probability of ties will depend on the number of admissible categories defined. The difference between the balanced worth and the worth will thus be smaller the finer the grid of possible outcomes and vanishes for continuous distributions.
 3.
We say that group k is fully dominated when \( p_{kj} = 0\,,\,\,\forall \,\,j \ne k \).
 4.
Let us recall here that inequality measures typically give more weight to the realizations in the lower part of the distribution. This makes sense when heterogeneity is bad but this is not always the case. For instance when comparing years of schooling across generations in a given country, one would typically like to find that the young generation has higher values than the old one, so that perfect equality is not the desideratum.
 5.
In the empirical application regarding life satisfaction individual answers have been grouped into a rougher set of categories. One may reasonably wonder what the purpose is of losing information by grouping those data into broader categories when we have all the individual numerical responses. The main reason is that when dealing with subjective evaluations in terms of numerical scales, there is no guarantee that numbers mean the same for different people (your 7 and my 7 may well represent very different things). Moreover, individual scales need not be linear (i.e. an evaluation 8 need not be twice one of 4, even for a single individual). Grouping numerical answers into categories may thus helps illuminate some structural features of the groups, enhance robustness and reduce the comparability assumptions required.
References
Albarrán, P., Herrero, C., RuizCastillo, J., & Villar, A. (2017). The HerreroVillar approach to citation impact. Journal of Informetrics, 11(2), 625–640.
Bellù, L. G., & Liberati, P. (2005). Social welfare analysis of income distributions ranking income distributions with crossing generalised Lorenz curves. Rome: Food and Agriculture Organization of the United Nations.
Border, K. C. (1989). Fixed point theorems with applications to economics and game theory. Cambridge: Cambridge University Press.
Bourguignon, F., Ferreira, F. H. G., & Leite, P. G. (2008). Beyond Oaxaca–Blinder: Accounting for differences in household income distributions. Journal of Economic Inequality, 6(2), 117–148.
Chakravarty, S. R., & Silber, J. (2007). A generalized index of employment segregation. Mathematical Social Sciences, 53(2), 185–195.
Crespo, J. A., Li, Y., & RuizCastillo, J. (2013). The measurement of the effect on citation inequality of differences in citation practices across scientific fields. PLoS ONE, 8(3), e58727. https://doi.org/10.1371/journal.pone.0058727.
Cuhadaroglu, T. (2013). My group beats your group: Evaluating nonincome inequalities. St Andrews: W.P. School of Economics and Finances, University of St Andrews.
Echenique, F., & Fryer, R. G. (2005). On the measurement of segregation (Vol. w11258). Cambridge: National Bureau of Economic Research.
Eurostat. (2015). Quality of life. Facts and views. Luxemburg: Publication Office of the European Union.
Frankel, D. M., & Volij, O. (2011). Measuring school segregation. Journal of Economic Theory, 146(1), 1–38.
Gallén, M. L., & Peraita, C. (2015). A comparison of corporate social responsibility engagement in the OECD countries with categorical data. Applied Economics Letters, 22(12), 1005–1009.
GonzalezDiaz, J., Hendrichx, R., & Lohmann, E. (2014). Paired comparison analysis: An axiomatic approach to ranking methods. Social Choice and Welfare, 42(1), 139–169.
Grannis, R. (2002). Segregation indices and their functional inputs. Sociological Methodology, 32(1), 69–84.
Herrero, C., Méndez, I., & Villar, A. (2014). Analysis of group performance with categorical data when agents are heterogeneous: The evaluation of scholastic performance in the OECD through PISA. Economics of Education Review, 40, 140–151.
Herrero, C., & Villar, A. (2013). On the comparison of group performance with categorical data. PLoS ONE, 8(12), e84784. https://doi.org/10.1371/journal.pone.0084784.
Laband, D. N., & Piette, M. J. (1994). The relative impacts of economics journals: 1970–1990. Journal of Economic Literature, 32(2), 640–666.
Laslier, J. (1997). Tournament solutions and majority voting. Berlin: Springer.
Li, F., Yi, K., & Jestes, J. (2009). Ranking distributed probabilistic data. SIGMOD’09, June 29–July 2.
Lieberson, S. (1976). Ranksum comparisons between groups. Sociological Methodology, 7, 276–291. https://doi.org/10.2307/270713.
MartínezMekler, G., Martínez, R. A., del Río, M. B., Mansilla, R., Miramontes, P., et al. (2009). Universality of rankordering distributions in the arts and sciences. PLoS ONE, 4(3), e4791. https://doi.org/10.1371/journal.pone.0004791.
PalaciosHuerta, I., & Volij, O. (2004). The measurement of intellectual influence. Econometrica, 72(3), 963–977.
Pavot, W., & Diener, E. (2008). The satisfaction with life scale and the emerging construct of life satisfaction. The Journal of Positive Psychology, 3, 137–152.
Pinski, G., & Narin, F. (1976). Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics. Information Processing and Management, 12(5), 297–312.
Reardon, S. F., & Firebaugh, G. (2002). Measures of multigroup segregation. Sociological Methodology, 32, 33–76.
Rosvall, M., & Bergstrom, C. T. (2007). An informationtheoretic framework for resolving community structure in complex networks. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.0611034104.
Sheriff, G., & Maguire, K. (2013). Ranking distributions of environmental outcomes across population groups. National Center for Environmental Economics. August.
Shorrocks, A. F. (1983). Ranking income distributions. Economica, 50(197), 3–17.
Slutzki, G., & Volij, O. (2006). Scoring of web pages and tournaments. Social Choice and Welfare, 26(1), 75–92.
Torregrosa, R. (2015). Medición y evolución del sentimiento autonómico en España. Investigaciones Regionales Journal of Regional Research, 33, 53–70.
Veenhoven, R. (1991). Is happiness relative? Social Indicators Research, 24, 1–34.
Villar, A. (2014). Education and cognitive skills in the Spanish adult population. Intergenerational comparison of mathematical knowledge form PIAAC data. Advances in Social Sciences Research Journal, 1(1), 72–88.
Yalonetzky, G. (2012). A dissimilarity index of multidimensional inequality of opportunity. The Journal of Economic Inequality, 10(3), 343–373.
Acknowledgements
Thanks are also due to Héctor García Peris, for his help in developing the algorithm that computes the evaluation, and to an anonymous referee for very helpful comments and suggestions. Funding was provided by the Spanish Ministerio de Economía y Competitividad (Grant Nos. ECO201565408R, ECO201565820P).
Author information
Affiliations
Corresponding author
Appendix: Existence and uniqueness of the balanced worth
Appendix: Existence and uniqueness of the balanced worth
Here we prove that the balanced worth always exists and that, under very general conditions, it is unique and strictly positive.
Definition
Given a problem A, a group j is fully dominated in A, if for all i ≠ j it happens that \( p_{ij} = 1 \).
When a group j is fully dominated, \( p_{ji} = e_{ji} = 0,\;\forall i \ne j \). That is, all individuals in group j belong to a lower level than any other individual in the rest of the groups. Note that, in practice, the probability of finding a fully dominated group is zero.
Let us formally state that a solution to the system of g equations with g unknowns (3), always exists.
Theorem 1
Let A be an evaluation problem. Then:

(i)
A vector \( {\mathbf{w}} \in {\mathbb{R}}_{ + }^{g} \) exists that solves equation system (3). That is, a vector w* such that:
$$ w_{i}^{*} = \frac{{\sum\nolimits_{j \ne i} {\left( {p_{ij} + \frac{{e_{ij} }}{2}} \right)w_{j}^{*} } }}{{\sum\nolimits_{j \ne i} {\left( {p_{ji} + \frac{{e_{ji} }}{2}} \right)} }},\quad i = 1,2, \ldots ,g $$ 
(ii)
If no group is fully dominated, then the solution is unique (up to a scalar multiplication) and strictly positive.
Proof
(i) Let \( W = \left\{ {{\mathbf{w}} \in {\mathbb{R}}_{ + }^{g} /\sum\nolimits_{i = 1}^{g} {w_{i} = g} } \right\} \) and consider the function \( \varphi :W \to {\mathbb{R}} \), given by:
As \( \sum\nolimits_{j \ne i} {\left( {p_{ji} + \frac{{e_{ij} }}{2}} \right) \le g  1} \), we have:
Moreover,
Note that, by construction,
which means that \( \sum\nolimits_{i = 1}^{g} {\varphi_{i} \left( {\mathbf{w}} \right)} = g \) and hence that function φ maps W into itself. As it is a continuous function and W is a compact convex set, Brouwer’s Theorem (e.g. Border 1989), ensures the existence of a fixpoint, \( {\mathbf{w}}^{*} = \varphi \left( {{\mathbf{w}}^{*}} \right) \). That is,
and, therefore,
(ii) Assume now that there is no fully dominated group, that is, \( (p_{ij} + \frac{{e_{ij} }}{2}) > 0\;\forall i,j \). Then, the solutions must be strictly positive. This is so because both numerator and denominator are strictly positive. To prove uniqueness, suppose there are two strictly positive vectors, w, y, that solve the equation system (3). Then, we can write:
For a given i, this expression can be rewritten as:
where all terms are strictly positive, with \( x_{j} = w_{j} /w_{i} ,\;z_{j} = y_{j} /y_{i} \). But this is the equation of a hyperplane with a given normal, which means that vectors x and z are to be proportional. That is, the solution is unique up to the choice of units.□
Rights and permissions
About this article
Cite this article
Herrero, C., Villar, A. The Balanced Worth: A Procedure to Evaluate Performance in Terms of Ordered Attributes. Soc Indic Res 140, 1279–1300 (2018). https://doi.org/10.1007/s1120501718187
Accepted:
Published:
Issue Date:
Keywords
 Evaluation method
 Categorical variables
 Relative group performance