The demand for indicators that allow comparing the performance of countries' public sectors internationally is increasing. This trend already existed for some time in the development community, where such indicators are used – or at least referred to – in decisions whether or not to give development aid to certain countries.
While some international country competitiveness rankings have included information on the functioning of countries' public sectors for several years, they have always focused on just a very limited aspect of the public sector. Likewise, we have seen a steady increase in the efforts to measure and compare national health and education sectors. Attention for measuring the performance of public sectors in general is much more recent.
The need for such indicators is chiefly motivated by two factors. One is to use the information for naming and shaming countries with a low score and to stimulate them to improve the performance of their public sectors. The other factor is related to countries' reputations as efficient or as bureaucratic. Countries may not always deserve their reputation, but these reputations do influence people: businessmen are likely to refrain from investing in countries notorious for their bureaucracy, highly-skilled expats may be unwilling to work in countries with bad health and education systems, and donors are reluctant to give aid to developing countries known for waste and corruption. Public sector performance indicators may help to reassess these reputations.
Public sector performance: how do European countries rank?
Several studies and indicators have become available in recent years to compare countries' public sectors. One such study is the European Central Bank's working paper “Public sector efficiency: an international comparison”. One of the elements in the comparison, apart from health, education and public infrastructure efficiency, was the performance and efficiency of the public administration. A comparison was made of 23 OECD countries, and resulted in a performance and an efficiency score for these countries. The higher the score, the better the performance and efficiency.
Switzerland is the country with the highest public sector performance and efficiency, while Italy is the country with the lowest overall score in the report. This working paper was one of the first attempts at comparing public sector performance and efficiency across countries in a comprehensive way and deserves praise for that. At the same time, however, the study failed to recognize the problems with the data used to measure the performance of public administrations, and it suffered from a small government bias in its efficiency measurement.
A second indicator is the Government Effectiveness indicator, which is part of the World Bank Governance Indicators dataset. This government effectiveness indicator is currently the most comprehensive indicator for assessing the performance of public administrations, even though the indicator is of a quite general and sometimes even vague nature. The indicator currently covers 213 countries or territories worldwide. It measures government effectiveness by looking at “the quality of public services, the quality of the civil service and the degree of its independence from political pressures, the quality of policy formulation and implementation, and the credibility of the government's commitment to such policies”. According to this indicator, the Scandinavian countries receive the highest scores, while Italy, Poland, and Greece are dangling at the lower end of the ranking.
A final source of information are the Global Competitiveness Report (GCR) and the World Competitiveness Yearbook (WCY), which have a Public Institutions and a Government Efficiency Index, respectively. These publications are primarily concerned with measuring the competitiveness of a country. A public sector that is conductive to business is an important factor in this competitiveness. The figure shows how the public sectors are scored in 19 EU countries, whereby a higher score means a better performing public sector. The Scandinavian countries and Austria get good scores, while Italy, Poland and Slovakia score quite bad on the public institutions index and the government efficiency ranking respectively.
A problem of definition
The indicators above appear to offer a straightforward ranking and measurement of countries' public sectors. They do, nonetheless, have a number of technical and conceptual problems. Developing a suitable indicator depends on deciding on a number of conceptual and political-philosophical issues. These issues, however, cannot easily be solved. When we want to measure the performance, the efficiency or quality of a public sector, we need to agree on a definition of “government” and “the public sector”, and we need to find a means to agree on what we consider as “good” performance.
What is government?
“Government” is a concept that is particularly hard to define and compare: A Scandinavian-style welfare state is clearly different from a minimal state or an Anglo-Saxon regime, yet we seem to want to directly compare their performance. When we want to compare public sector performance or overall bureaucratic quality, we need to agree on a definition of the public sector, the public administration, and government.
“There is no, and there can probably be no, agreement on the current size of the government sector.”
Different criteria may be used for determining whether an institution or service can be considered as part of government: financial relations, legal bonds, self-identification, official names and titles, etc. This differentiation is easier in some sectors than in others. How for instance to deal with schools? Or with non-profits that would most likely not exist without the government's financial support? When public sector efficiency is measured, e.g. using the number of civil servants, who do we then count? Only civil servants appointed for life, or also people employed in QUANGOs? There is no, and there can probably be no, agreement on the current size of the government sector. The popularity of public-private partnerships and agentification has not made this task any easier.
What is good government?
A second major challenge is related to the definition of “good” or “better” performance. Researchers are interested in determining whether government A is performing better than government B. A government that is wasting resources is not performing adequately. One that is not wasting resources is a performing organization. But how do we determine whether a government is wasting resources? Obviously, we do not only want government to do the things right, we also want it to do the right things. But what are “the right things”? And how do we determine whether they are doing the right things and that they are done right? This is more than just a technical question. Obviously, there is no market value for measuring government outputs against.
The apparently easiest approach is to be selective. Rather than measuring all aspects of government performance, we can define a bottom-line: “It doesn't really matter what a government is doing as long as it does X, or doesn't do Y.” Corruption is a good example. Corruption is used as the indicator par excellence. A country's institutions receive good grades when they are found not to allow corruption. This obviously is a quite narrow interpretation, and many will agree that the mere absence of corruption is by no means a guarantee for high performance. Another quite popular approach is to reduce government performance to being conducive to business: good governments are governments that support business by limited regulatory intervention, low taxes, and a high degree of stability or absence of political and administrative risk. Red tape, in this view, is inevitably related to inefficiency. Three of the indicators in the first section of this article fit within this tradition.
Adapting existing indicators
Demand for indicators that allow for comparing the performance of public administrations internationally is increasing. A diverse range of actors is interested in indicators for measuring the quality of bureaucracies at the macro-level, rather than at the micro-level of specific services. These actors include inter- and supranational organizations, the international donor community, and scholars. Demand in comparative public management remains rather limited, but it is to be expected that public management scholars will speed up their efforts to compare the performance of countries' administrations.
The supply of indicators is also rapidly increasing. A growing number of datasets is being developed and promoted, with the World Bank governance indicators as one of the best-known examples. Yet, many of these indicators and datasets suffer from conceptual and data quality problems, making them less suitable for comparative public management scholars.
There is a real risk that the indicator boom may lead to superficial international comparisons. When comparative public management wants to build a research programme around the comparison of the performance of public administrations, it can build on the work that has been going on for decades in other disciplines. Blind reliance on existing data, however, is likely to endanger the quality of research. Public management policy makers need to adapt the existing indicators to new developments in the definition of government, and need to raise the methodological standards of some of the older datasets.
November 2008.
This is a shortened version of “Comparing the performance of national public sectors: conceptual problems”, which originally appeared in International Journal of Productivity and Performance Management, Volume 57 Number 4, 2008.
The author is Steven Van de Walle.