Toward better measures of global ICT adoption and use

Introduction

Efforts to agree on the most appropriate indicators to use for measuring disparities in information and communications technology (ICT) adoption and progress toward information society goals have continued in 2008. However as yet global consensus has not been reached and debate continues over what indicators would best take into account the growing broadband divide, what constitutes “universal access”, and how to accommodate local realities regarding data availability, especially in developing countries.

Current background and status of work on global ICT indicators

In the area of ICTs, constant technology and market change has meant that until recently there was little global agreement on an appropriate set of indicators or indices. As a result, a wide range of ICT-related data has been gathered by national statistical and regulatory agencies, and many regional and international agencies have developed their own measures of ICT uptake over the last fifteen years. ^[1]

By the beginning of the new century, more concrete and universal information society goals were being developed. These began to focus at a global level with the targets of the World Summit on the Information Society (WSIS) Action Plan and the ICT-related components of the Millennium Development Goals (MDGs), which provided further momentum for two important developments in ICT uptake measurement.

First, three indices aimed at measuring and ranking national progress towards becoming information societies were developed and published by the International Telecommunication Union (ITU): the Digital Accessibility Index (DAI), the ICT Opportunity Index (ICT-OI) and the Digital Opportunity Index (DOI). ^[2]Using a small set of mainly ICT infrastructure and human capacity-related indicators such as teledensity and education levels, none of them were directly based on measures of achievement of the WSIS targets. Although they provide interesting general measures of progress toward some information society goals, important aspects were left out, partly because the data are not seen as relevant, or because the data are simply not available for many countries, especially data requiring household surveys.

The ITU has now begun work on a single index which aims to combine the best features of the ICT-OI and the DOI. At the 6th World Telecommunication/ICT Indicators Meeting in December 2007, the options for a single index were discussed but agreement was not reached, and a working group was set up to finalise the index. One of the outstanding issues, which highlights the difficulty of coming up with simple, globally applicable indicators, was the proposed use of international bandwidth as an indicator. Advanced countries isolated by language, such as South Korea or Japan, would not feature highly on use of international bandwidth because most of their traffic would be local. The meeting also considered community access indicators and a number of measures were proposed, including tracking the percentage of localities (villages, towns, etc.) with a public internet access centre, and those that are connected to the public telephone network. In addition, new indicators in the area of mobile/wireless broadband measurement and computer virus infection levels were discussed.

Second, and perhaps of greater significance, has been the formation of the international multi-stakeholder Partnership on Measuring ICT for Development. Established during the 11th United Nations Conference on Trade and Development (UNCTAD) session in 2004, the partnership now comprises the ITU, Organisation for Economic Co-operation and Development (OECD), UNCTAD, United Nations Educational, Scientific and Cultural Organisation (UNESCO), Eurostat, the World Bank Group and the UN regional agencies. ^[3]

The partnership was set up for three key reasons: to achieve a common set of core ICT indicators, agreed upon internationally; to help build the capacities of national statistical offices in developing countries to collect the necessary data; and to develop a global database on ICT indicators and make it available on the internet. Its two main report outputs are: Measuring ICT: The Global Status of ICT Indicators,^[4]andCore ICT Indicators.^[5]The former is the report of a global stocktaking exercise on the availability of ICT indicators. The 47% national response rate to this concerted effort underlines the problems in establishing global indicators – especially with particularly low numbers of responses for Africa and the Asia Pacific countries. ^[6]

The second work,Core ICT Indicators, describes a set of 41 core indicators that were identified during the stocktaking exercise and subsequently endorsed by the UN Statistical Commission in 2007. The core indicators are divided into four groups as follows:

• ICT infrastructure and access (twelve indicators)
• Access to and use of ICT by households and individuals (thirteen indicators)
• Use of ICT by businesses (twelve indicators)
• The ICT sector and trade in ICT goods (four indicators).

The full list of 41core indicators is described in Annex A at the end of the chapter.

Several developing countries have since integrated the core indicators into existing household and business surveys. While the UN endorsement, and the partnership capacity-building activities, should lead to improvements in the number of countries that collect ICT indicators, and in the comparability of the data, there may need to be a rethink about what indicators should be contained in the “core list”. In this respect it should be noted that the partnership does not claim the list to be complete, and identifies the process as continuous and subject to periodic review.

In an ideal world, the core list as proposed by the partnership would certainly provide a useful picture of ICT uptake that covers a large part of the Real Access Framework (RAF) criteria suggested by Bridges.org to assess access to ICTs. (The RAF has been used loosely in the country reports in GISW 2008 to reflect access challenges at a national level). However, lack of data availability from many countries remains a key problem – only a small proportion of countries are able to report on all 41 indicators. In 2005 the partnership found that only about 40 countries worldwide collected ten or more household ICT indicators.

To maximise the number of countries that can report on a common set of indicators, the total number of indicators may have to be reduced, especially those that require user surveys. The core list also has many measures for factors that mainly concern business and trade, which could be reduced relative to those that focus on the general public. Developing countries need indicators which help them formulate regulatory and policy decisions around how to best extend the network using constrained resources. Shared use, community networks, telecentres and so forth are strategies that are not yet fully reflected or measured in the legacy indicators agreed to by the partnership – although the intention to use household survey data does take some steps towards accuracy in this regard.

There are also a number of other important aspects of Real Access that the core list does not explicitly address, including gender disaggregation. These areas are covered in more detail in the following section.

Principles and considerations for selecting future indicators

The number and range of ICTs available today has never been greater, and the interrelationships between them and their indicators are many. In order to effectively evaluate the choice of indicators, it is essential to have a clear conceptual framework on which to base the evaluation. In considering options for choosing indicators, the key considerations and assumptions can be summarised as follows:

• The goal should be to provide universally accepted measures of ICT adoption at a national level that encompasses as many nations as possible, using consistent data definitions and timing for data reporting.
• The selection of indicators should be based on a solid conceptual framework that aims to provide measures of actual uptake and use. The use of factors that attempt to ascribe the potential for access are likely to find less wide acceptance. Similarly, supply-side indicators also tend to reflect potential use rather than actual use.
• Given the framing of the WSIS and MDG goals, the focus should be on personal rather than business use (although ideally in future when more data is available, household use and other types of disaggregation would also be more explicitly included in the indicators).
• To maximise the validity period in the face of evolving technologies, new infrastructure and new services adoption, the indicators need to anticipate the future evolution of ICT infrastructure and services. ^[7]
• Indicator data used should be provided by credible organisations which issue it on a regular basis to allow for longitudinal studies (over time).

Due to the general lack of up-to-date data, the smallest number of indicators is likely to be the most inclusive and comparable across countries. Data freshness is another factor here. Even for the most commonly used data such as teledensity, while an increasing amount of year-end 2007 data is becoming available, overall, 2006 is still the most recent year for globally representative data. This highlights a key problem in selecting a meaningful set of core indicators and also means that for policy-makers there is at least a two-year lag in seeing the results of policy decisions. While more up-to-date information may be available for some indicators, if it is not available for all indicators, the overall value decreases substantially. Since the availability of indicators with broad representation across countries is so small, these considerations also underline a key tension in the construction of the core list: the playoff between accuracy and country representation.

Measures of equipment uptake need careful consideration for inclusion in a core list of indicators because of lack of accurate data in developing countries, and also due to technology change. For example, in considering the use of computer penetration, the definition of what actually constitutes a personal computer (PC) is becoming increasingly blurred because of mobile/PC convergence and the embedding of computing devices in other household equipment such as fridges.

Television (TV) penetration also suffers from similar problems. Data for TV sales is not up to date in many countries, is likely to be inaccurate due to grey market importing, and is currently only available for 85 countries. TV penetration measures are also not future-proof, considering rapid moves toward internet protocol TV (IPTV) and mobile phone TV, so that using traditional TV measures would bias against those countries that have already adopted these technologies. Radio penetration data suffers from the same sort of problems as PC and TV data.

Fixed-line penetration measures may also be problematic, considering that little new cable is being laid and many nations (especially developing countries) are skipping the use of fixed-line infrastructure and moving directly to wireless technologies. As a result, including fixed-line measures would be likely to bias against most developing countries.

In contrast to fixed lines, mobile phone access is becoming the de facto measure of basic access, and this indicator is of particular concern to developing countries where growth is still rapid and has not come close to reaching saturation. In addition, mobile phones are now being used more for internet access than PCs in some countries. ^[8]Mobile subscribers are accurately monitored in 220 countries by Wireless Intelligence, ^[9]the partnership between the GSM [global system for mobile] Association and Ovum. Quarterly data is even available a few months after the end of the quarter ^[10]and the data spans mobile network operators across most technologies, including GSM, wideband code division multiple access (W-CDMA), time division multiple access (TDMA), personal digital cellular (PDC), cdmaOne, CDMA2000 1x, CDMA2000 1xEV-DO, analogue and integrated digital enhanced network (iDEN).

Similarly, a measure of the total number of internet users is an important indicator, but there are some limitations to subscriber data, which is usually provided by operators. This is because there is no clear relationship between the number of internet subscribers (relatively easily obtained) and internetusers, many of whom may share the subscriber’s connection. As a result, much of the available data is based on estimates, for which the level of accuracy is unclear.
Since broadband users, and in particular, wireless and mobile internet users, are becoming an increasingly important component of the internet user base, it may also make sense to include measures of these users, especially as there is now a well-accepted understanding of the importance of broadband for full access to the information society. The need for affordable pervasive access to broadband therefore extends beyond access to information and into active participation, as people with shared interests or problems become significantly active on the web only when broadband is available.
In measuring usage (rather than availability), until more widespread national survey data is available, the use of proxy indicators such as telephone minutes or internet bandwidth will be necessary. The main deficiency with these indicators is a tendency to over-emphasise international usage. Ideally more measures of national usage would be included. However, there is very little national internet traffic data currently available, and although there is some national voice-traffic data, the level of country representation is poor.

Although traffic indicators would appear to only measure usage, they also provide some indication of production of data, although ideally this aspect would be augmented in future by other measures such as numbers of local websites and domain names. These measures are difficult to gather, however, due to the use of generic top-level domains (gTLDs) by many in-country website operators who choose not to use country code top-level domains (ccTLDs). Similarly, the number of secure internet servers has been commonly adopted as an indicator of the extent to which reliable digital transactions are made. However, this indicator does not reflect the fact that many of the most popular online services requiring secure servers are global brands and not specific to any particular country (Amazon, eBay, etc.).
International internet bandwidth per capita has become an increasingly well-recognised indicator following its use at the G8 Dot-Force meeting in Kananaskis in 2002. It is fairly easy to obtain because there are a relatively small number of international internet service providers. Because of the relatively high costs of international bandwidth, it is likely to reflect actual usage rather than being a supply-side indicator based on the size of the pipe. There are also other ways of measuring or cross-checking estimates of internet bandwidth. For example, bandwidth data is gathered by the Stanford University SLAC PingER project.^[11]The PingER project calculates the bandwidth of internet links by measuring the time it takes to send packets of data to internet hosts around the world.. This indicator confirms that international bandwidth reports to the ITU are broadly in line with measured performance, although there are a number of exceptions at a national level that would be worth examining.
Ideally, if total national and international internet bandwidth could be measured, this figure, combined with the total number of internet users, would give a reasonable composite measure of the extent of internet use. However, given the growing importance of networks based on internet protocols and the decreasing use of switched-voice circuits, it will be increasingly important to identify other measures of internet use. IP host numbers have been used, as this is a superficially attractive measure because it is easily available for every country and is relatively up to date. However, due to the prevalent use of private IP numbers behind firewalls, and allocations of numbers not in actual use, this measure is quite misleading. In addition, the transition from IPV4 to IPV6 is changing the entire IP numbering system, and some countries are more advanced in this process. In the long term, however, this will ultimately improve IP host numbers as a measure by eliminating the need for network address translation (NAT) and host address masquerading.

In the interim, a more valid approach would be to use a metric based on autonomous system numbers (AS numbers or ASNs). Unique ASNs are allocated to internet network operators by the regional registries (RIRs) for use in multi-path (BGP) routing (the protocol used to ensure that there is more than one route to the internet provider’s network). The use of ASNs as an indicator was pioneered by OECD researcher Tom Vest, and based on his work, the OECD’s Committee for Information, Computer and Communications Policy (ICCP) has now proposed the use of ASNs for measuring internet uptake in their member countries. ^[12]

Raw ASN information is available on a daily basis via automated file transfer protocol (FTP) download and is therefore the most up-to-date ICT indicator available in the world. The data is hosted by the University of Oregon Route Views Project ^[13]where the daily updated data goes back to 1997. In this respect a key advantage of the ASN metric is that it does not rely upon country reporting and therefore does not further burden developing country national statistical offices (or the national regulator) with further indicator collection responsibilities.
Indicators to measure the level of exclusion from ICTs amongst the public are of special importance. While this has not been the direct focus of the other ICT uptake measurement efforts, the DOI focused on the related concept of opportunity. Of note is that the World Bank’sWorld Development Report 2006advocates taking equity into account when determining development priorities. 

Given current technology trends and long-stated gender concerns, it is becoming increasingly essential to have a clear picture of how the internet and women’s access to ICTs are evolving in developing countries, and indeed throughout the world. So measures of gender-disaggregated access should be included, although currently gender-disaggregated data availability is minimal; for example, only 39 countries feature on the ITU’s STAT page for female internet users. ^[14]There is no doubt that as national-level information society policies prioritise women and girls’ access to and ability to use ICTs, there will be efforts to measure these in order to document progress towards policy goals. But this is only just beginning to happen, and it will be a long time before there is a critical mass of gendered ICT indicators available.

Aside from measures of gender equity, other equity indicators would include the dispersion of public access facilities (telecentres, cybercafés or public phones), mobile coverage areas, mobile and broadband affordability, and basic literacy levels. Measures of network coverage should include national broadband coverage and the proportion of population covered by mobile networks. Ideally, affordability indicators would measure the prices of broadband subscriptions calculated pro-rata for a certain agreed speed of connection per month, such as one megabit per second (Mbps). This would allow comparison of countries with different speeds available and could also be expressed as a percentage of average monthly household income.

Another expression of affordability could be the OECD-defined basket of costs for mobile usage. Because of the complexity and variety of available mobile tariff packages, and the lack of identical packages in different countries, it should be noted that there may be some inherent variation in the data that does not reflect actual costs. In addition, a case could be made for using the medium basket, rather than the low-end user basket, which was defined in the early 1990s when mobile usage was relatively low.

It should also be noted that while the ICT access costs aim to measure affordability, when compared against country wealth they may not correlate fully with use. The poor may spend a much higher proportion of their income on communication costs. Flat-rate subscriptions with monthly minute packages also tend to skew this assessment.

Adult literacy levels are an obvious and well-represented indicator for the degree to which the public can use ICTs, but the measure does suffer from some biases. Mobile phone users do not necessarily have to be literate to use this technology, and intermediaries are often used by the non-literate to obtain information from the internet or to send messages.

References

ITU and UNCTAD (2007)  World Information Society Report 2007: Beyond WSIS.
Available at: www.itu.int/osg/spu/publications/worldinformationsociety/2007

Minges, M. (2005)Evaluation of e-Readiness Indices in Latin America and the Caribbean. Santiago, Chile: ECLAC.
Available at: www.eclac.org/socinfo/publicaciones/xml/8/24228/w73.pdf

Partnership on Measuring ICT for Development (2005) Measuring ICT: The Global Status of ICT Indicators.
Available at: www.itu.int/ITU-D/ict/partnership/material/05-42742%20GLOBAL%20ICT.pdf

Partnership on Measuring ICT for Development (2005) Core ICT Indicators.
Available at:
www.itu.int/ITU-D/ict/partnership/material/CoreICTIndicators.pdf

World Bank (2006)World Development Report 2006: Equity and Development. Washington: World Bank.

Footnotes

^[1]A good comparison of the most important of these can be found in Minges (2005).

^[2]TheWorld Information Society Report 2007: Beyond WSIS, a joint publication by ITU and UNCTAD, details the use of these different indices.

^[3]These are the UN Economic Commission for Africa (UNECA), UN Economic Commission for Latin America and the Caribbean (ECLAC), UN Economic and Social Commission for Asia and the Pacific (UNESCAP), and UN Economic and Social Commission for Western Asia (UNESCWA).

^[4] www.itu.int/ITU-D/ict/partnership/material/05-42742%20GLOBAL%20ICT.pdf

^[5] www.itu.int/ITU-D/ict/partnership/material/CoreICTIndicators.pdf

^[6]Not to mention the absence of some major economies which did not respond to the survey, such as China, Nigeria and South Africa.

^[7]In this respect it is expected that networks will steadily evolve away from a circuit-switched infrastructure to packet-switched/internet protocol-based networks, commonly known as next generation networks (NGNs), which will also increasingly comprise larger numbers of wireless internet users.

^[8]communities-dominate.blogs.com/brands/2007/01/putting_27_bill.html

^[9] www.wirelessintelligence.com

^[10] www.gsmworld.com/news/statistics/index.shtml

^[11] www.slac.stanford.edu/xorg/icfa/icfa-net-paper-jan07

^[12] www.oecd.org/dataoecd/25/54/36462170.pdf

^[13] archive.routeviews.org/oix-route-views

^[14] www.itu.int/ITU-D/ict/statistics/at_glance/f_inet.html