|gw2018_thematic_At the limits of the internet: Technology options for community networks1||1.67 MB|
At the limits of the internet: Technology options for community networks1
Community networks are kinds of networking infrastructures built to provide local and global connectivity to interconnect people and devices and transfer messages and content. As with other critical infrastructures, the challenge is to reach everyone and anything requiring connectivity. This relies on a combination of technologies to optimise affordability, complexity, quality and performance.
This report discusses how connectivity works in general, the specifics of network access and backhaul technology, and the software considerations when setting up a community network. It then offers a list of key software and other resources useful to community networks.
Connectivity – the ability to connect or communicate with others – comes in units of links that are part of the internet. Links bring access to people or devices in a given location or as they move from location to location. Interconnected links spread over geographic areas to provide coverage forming access networks, or autonomous systems. Regions which are at the edges of the internet are connected through routers and long-distance links, also known as the backhaul network, which connects access networks to the core of the internet. In some cases, access providers use internet exchange points (IXPs), where autonomous systems meet and exchange or trade internet traffic to reach local content or transit providers. Transit providers allow customer networks to cross or "transit" the provider's network, usually to reach the rest of the internet. This can take the form of offering backhaul connectivity to networks. They do not offer connectivity to individual end-users. Technology in the form of standards, hardware and software artifacts, and their complexity, restrictions, performance, cost and evolution, determine the availability of connectivity or the lack of it.
The electromagnetic spectrum refers to the range of all frequencies of electromagnetic radiation. The internet relies on devices that generate, carry and read information encoded in this radiation in the form of waves in cables, in the air or even as light waves in optic fibres. Simply speaking, waves are oscillations, and at each oscillation some information is moved. The bandwidth of a signal refers to how wide the frequency range of the oscillation is. A wider frequency range, a broad band, results in higher data transfer speeds. Therefore, broadband is an old term to refer to connectivity from the perspective of the allocated spectrum bandwidth. 2 By being an always-on and faster internet access, fit for a wide range of uses, broadband is differentiated from slow and fragile access through telephone lines (dial-up, or narrowband). The term is carefully defined and politically loaded in the telecom regulation in every region 3 as it affects criteria for digital inclusion policies and public subsidies or investment.
Generally speaking, transmitting two different wireless signals at the same frequency creates interference, and may make it impossible to receive any information. For these reasons the public electromagnetic spectrum is divided in intervals (bands) and each band may be licensed or unlicensed. There are international agreements and regulations regarding this public spectrum, 4 but to use a licensed band one must obtain an authorisation from a local regulator (a public body that has the task of assigning bands of the public spectrum commons to operators, generally at a cost). Unlicensed bands5 are free for use, which means that they must use technical means to survive congestion, and in general are considered less reliable. Cables and fibres create their own “private spectrums” that are separate from the public spectrum and do not interfere with each other.
More speed allows for more communication to happen simultaneously. Adequate connectivity allows running one or multiple apps with no visible degradation, which means not limited by data rate or fluctuations (congestion) at any part of the network path. Using the analogy of roads, if broadband is the asphalt, lanes carry data packets (cars) and roundabouts correspond to routers that do packet switching and queueing. Quality also includes reliability, the quality of being trustworthy or performing consistently well, and latency, the time interval between input and response, which depends on the length of the network path, given the unavoidable physical limits of propagation of electromagnetic signals. 6
Figure 1. Spectrum properties. Frequency ranges (Hz), wavelength (metres), radiation. Wi-Fi is on the range of 109 Hz and light on 1014.
Access generally refers to the first network link between a hardware device (also called a “terminal”) and the network that reaches each user or server device. Mobile phones, and computers in general, link human beings to the internet, but many types of hardware serve as terminal nodes, such as servers, printers, cameras, and environmental sensors. These terminals are typically connected through cables, or without (wireless), with the first option offering a wider bandwidth at the cost of a higher price (cables need ducts to be deployed and reach our houses), and the second option being generally poorer in performance, but cheaper and supporting mobility.
Wired access usually reuses an existing wired cable such as a telephone copper line (dial-up, DSL) or TV distribution cable. The evolution towards more data traffic has led to the replacement of copper network segments with faster fibre 7 – for example, fibre to the cabinet in the neighborhood (FTTC) or to the premises/building (FTTP) extended by reusing existing copper cables to each premises, and full fibre access networks to each unit, business or home (FTTH). As fibre cables and fibre network devices get standardised and become easy and cheap to deploy, community networks have adopted the FTTH access model, with examples of full fibre in the Broadband for the Rural North (B4RN) 8 community network, and, mixed with some participants using wireless while others use full fibre, in the case of guifi.net. 9 These give typical access rates of around 1-10 Gbps.
Wireless cellular access networks: The road to 5G
Mobile devices get connected through some form of GSM (Global System for Mobile communications) standards or evolutions of it, nowadays described in terms of technology generations (from 1G to 5G). Each connectivity provider sets up base stations with omnidirectional antennas that cover a certain area (a “cell”) and pays a fee to have a licence for the exclusive use of a part of the electromagnetic spectrum to serve its customers. The more spectrum allocated to a given mobile provider, the more customers will be able to communicate at the same time in the same cell. That leads to a competitive privatisation of the public spectrum, typically at country level, for commercial usage: optimised to maximise the profit, and therefore prioritising and optimising the most profitable market of dense (urban) population with higher disposable income.
The allocation of public spectrum is a good source of income for governments (through spectrum auctions) and a good source of business for mobile operators, but leaves less spectrum available for other public, community or private uses. In underserved areas, there is the tragedy of lock-out of allocated spectrum, kept idle by the licensee due to lack of profitability and preventing its usage by anyone else.
The evolution of mobile technology has brought faster data rates with more efficient data encodings, better support for mobility, roaming, internet packets, and different coverages for dense (a few metres with pico/femto cells) or sparse macrocell deployments up to a 100 km radius.
The future generation of mobile connectivity (5G) deserves special attention, as it is intended to be not only a technological update but a leap forward. The goal of 5G is to provide a 1,000-fold increase of the aggregate network capacity, with up to 10,000 connected devices per base station. Apparently, 5G will change the way we access the internet. This requires a large spatial densification of the base stations, and therefore a huge investment to install and connect them through high-capacity links (typically fibre or high-capacity directive radio links). Operators will generate revenues selling new kinds of applications that take advantage of the breakthrough in terms of bandwidth and communication delay. The need for densification shows the focus on more speed in smaller areas, as opposed to a focus on increased coverage. This will increase the cost per user and the profit of the service provider in a given location.
It is worth noting that more than eight years after the roll-out of 4G, 10 only 29% of the five billion worldwide mobile subscribers use it. Since 5G calls for an even larger capital expenditure for new infrastructure, it is legitimate to ask: who will mostly benefit from 5G? Will it ever reach the populations in the developing world, especially in rural areas? Or – most likely – will it simply widen the divide between those who are already well connected and can afford better connectivity at a higher price, and those who are still unconnected today, roughly 50% of the world population?
At the other end of the technological spectrum, we find community networks that work with mobile phones as user devices that are cheaper, simpler and lighter than larger devices. These cellular community networks rely on low-cost cellular access points (base stations). Hardware for these base stations is becoming available at lower prices (in the range of a few thousand USD and 50 W of power consumption) and runs open source software. Alongside technical advancements, proposals are being made on innovative ways to access spectrum, such as progressive regulation for spectrum access to promote social rights for communities or secondary spectrum access for digital inclusion. 11
Wi-Fi access networks
The term Wi-Fi refers to a family of technologies for wireless radio components (technically belonging to the IEEE 802.11 standards, with multiple revisions and updates: 11b/g/a/n/ac/ax etc.) that have reached ubiquitous diffusion. Contrary to cellular access networks, Wi-Fi uses unlicensed spectrum, which cuts the deployment costs but also increases the risks of congestion.
At a very low price, a Wi-Fi access point can be used for an access link to another Wi-Fi device or to the internet (coverage of 100 metres or less) or, using directional antennas, Wi-Fi can provide high-performance point-to-point links (between only two devices, separated even by tens of kilometres).
Figure 2: Alternatives for Wi-Fi wireless radio links
Interconnected with access points through wired or wireless point-to-point links, Wi-Fi can expand the coverage of access networks and also create a backhaul network. The multipoint links, with sector antennas, result in a mesh network.
A mesh network is a network topology in which each node is capable of relaying data for any user of the network, not just the node owner. In mesh networks, all nodes cooperate in the distribution of data throughout the network to the mutual benefit of its participants. With each participating node, the reach, throughput and resilience of the network expands. Mesh networks are able to adapt to changes: when a node joins or leaves the network, the others automatically reconfigure to guarantee connectivity to the modified network. In some sense, they can grow “organically” with the growth of the community of people that use/manage them. A key ingredient of mesh networks is the routing protocol that can automatically select routes to enable multi-hop communication between any two nodes on the network. Combined with access points for user devices and internet gateways to reach the internet, mesh networks allow access networks to transform as new participants join, new areas are reached, and more capacity is added to links and internet gateways.
Many community networks based on mesh networks exist today, often led by volunteers who are able to set up networks to give coverage to large areas at a fraction of the cost it would require with cellular technology or cables.
The key observation is that if the price to bootstrap a network is lowered while capacity and cost grow incrementally with the growth of the community, this technology makes it possible to create networks that gradually expand with little planning or human coordination and give the time for the community to face the technical and organisational issues that come up along the way. Without a large initial capital expenditure for a spectrum licence and expensive infrastructure, it is much easier and less risky to create low-cost, bottom-up network infrastructures owned and managed by initially small communities of participants.
Other wireless access technologies
Another opportunity for long-distance communication is the use of the “white spaces” of TV spectrum, so-called TVWS, which are lower frequencies than Wi-Fi, and which were allocated to analog TV broadcasting (UHF and VHF) but are not used anymore. The standards for these radios are IEEE 802.11af and IEEE 802.22, also referred to as "White-Fi" and "Super Wi-Fi". The antennas look like TV antennas (both for access points and users) and have very good coverage, in the range of a radius of tens of kilometres without the need for line-of-sight.
What is known as the “internet of things” (IoT) allows us to connect “slow” devices, such as sensors, using very-long-range transmissions (more than 10 km in rural areas) with low power consumption and very slow data rates. One popular example of this is LoRa/LoRaWAN.12
On the higher part of the spectrum, beyond microwaves, we find millimeter waves, in the range of 30-300 GHz, with one licence-free ISM band13 at 60 GHz. The IEEE 802.11ad standard, also known as wireless gigabit or WiGig, promised very directive in-room or open space multi-Gbps communication in the range of a few to perhaps a few hundred metres. However, this specification has not succeeded in the market, needing expensive and niche devices. Instead there are alternative wireless gigabit proprietary products in the 24 GHz ISM band. 14 IEEE 802.11ay targets even higher speeds of up to 20 Gbps with a final specification expected by 2019 and products a few months later.
Satellite access15 only appears as a competitive solution for low population density areas, in the range of less than a few tens of inhabitants per square kilometre. Beyond that it becomes too costly compared to the alternatives. The added latency comes from propagation delay considering the radius of the orbits, 1,000 km high or 12 ms for low earth orbits (LEO), 10,000 km or 120 ms for medium earth orbits (MEO), and 36,000 km or 480 ms for geostationary orbit (GEO). The satellite latency is a reason in favour of high-altitude planes or balloon networks that operate in the stratosphere, at altitudes around 20 km 16 with less than 1 ms latency.
The service cost for satellite is determined by the number of subscribers in the coverage area, the cost of the satellite in orbit, and the base stations on the supplier side. The frequency bands used for internet traffic are C: 4-8 GHz, Ku: 12-18 GHz, Ka: 26-40 GHz with antennas for users of 2.5 metres in the C band, 1 metre in Ku band or less than that in Ka band. The achievable data rates, assuming the total capacity of the satellite is not saturated, can be up to 16 Mbps for 99.995% of the time for the C band, 64 Mbps for 99.9% of the time for the Ku band, and 512 Mbps for 99.7% of the time for the Ka band in a temperate climate, as rain has more of a fading (degradation) effect on higher frequencies. Satellite internet access has unique characteristics in coverage, but is expensive given the cost of build, launch, capacity and latency. Although an expensive access technology for any individual user, some community networks in remote environments may benefit from sharing the cost and capacity of satellite connectivity as one of the sources of connectivity in the backhaul to reach the internet, but ideally not the only one.
Figure 3. Coverage of different technologies, applicable to licensed and unlicensed bands. Source: Authors
Beyond technological details and choices, both Wi-Fi and mobile technologies evolve side by side: while mobile operators evolve towards LTE and 5G, successive generations of Wi-Fi technology also offer faster and cheaper devices capable of serving more users (e.g. MIMO) with faster data rates (new modulation schemes) in the range of gigabits per second but covering smaller areas. Who will win out between mobile operators or Wi-Fi device vendors? Probably both will coexist and complement each other, but definitely one is based on a “centralised” operator model with its own reserved radio spectrum, and the other is “self-provided” or “decentralised” using shared and unlicensed radio spectrum.
Beyond access networks, network interconnection relies on long distance links that carry aggregate traffic, IP packets, from/to the internet. These links are provided by internet service providers (ISPs), which can be retail providers (one community network sharing one or several retail internet connections: fibre, DSL or satellite), wholesale internet transit providers, with points of presence reachable over fibre or high-speed point-to-point radio links, or IXPs, with the presence of multiple network providers (internet carriers) and content providers (content distribution networks or CDNs). The interconnection fees in these IXPs may depend on the symmetry of the traffic (cheaper or even free for a community network with a balanced mix of content that is generated by the network and users or readers of content on the network, while more expensive for networks that only have users or readers of content, the latter also called “eyeball networks”). Community networks can even formally or informally become IXPs in regions without any. As mentioned before, satellite can be one ingredient of the backhaul for community networks in remote areas, but ideally not the only one.
A growing development to facilitate connectivity is the availability of open access optical regional networks (or fibre-equivalent radio links) that provide wholesale/volume connectivity to reach interconnection points, carriers or build access networks. These shared infrastructures,17 developed cooperatively or competitively, benefit nearly everyone locally, and therefore may be supported by large users such as governments, education institutions or the private sector, and may create economies of scale of competitive dark, active fibre or ethernet circuits to facilitate regional connectivity. Availability and cost efficiency in regional connectivity increase the opportunities to provide more services to more people. Recommendations regarding functional separation can keep incumbents from unhealthy competition and overbuilding, and facilitate community networks to scale up their deployments while reducing the cost. Community networks are effective in aggregating traffic from different stakeholders and sharing internet access, which directly translates into a significant reduction of cost for internet connectivity. This is the case with several community networks that share and rent wholesale open-access fibre for regional connectivity.18
Another barrier for the deployment of backhaul cables is the occupation of public space (through rights of way) by “private” infrastructures for private use. Municipalities are in charge of regulating this. Beyond laws to facilitate deployments, the guifi.net Foundation has developed the universal deployment model, 19 a template for a municipal ordinance to help promote the development of commons infrastructures. This template helps municipalities to avoid any discrimination and facilitate infrastructure deployments that are mutually beneficial for governments and private and community use. The principle is that any cable for private use to be deployed on public land is required to assign fibres for public (municipal) and shared/commons usage. This results in a public and community infrastructure at minimal cost (the private actor takes on the installation and maintenance costs in exchange). Therefore, the universal deployment model simultaneously allows for the three uses described, which results in infrastructure to expand community networks. The model can be extended from municipal land nationally, regionally and internationally (overseas), and even govern the use of undersea fibres. 20
Hardware and software
The behaviour of the building blocks that produce connectivity is controlled by software. There is the software needed to run the network: routing protocols, authentication systems, and wireless/wired drivers for link adapters. There is also the software needed to monitor and manage the network, or network management and planning tools. We do not have a wide range of these in open source, or not as stable as proprietary products. The typical discussion in community networks is on the tradeoffs with regards to openness when choosing between proprietary and open source solutions (efficacy vs lock-in in integrated components).
Hardware is an area where openness is lacking. Mostly anything directly related to it is still kept closed source, with protected intellectual property and product secrets held by the industry for hardware such as radio boards, radio firmware, device drivers, and programming interfaces. Notable exceptions of open hardware are the Mesh Potato, 21 LibreRouter 22 and Turris Omnia 23 routers and the software-defined radios for GSM such as UmTRX 24 or the USRP 25 family.
Proprietary hardware often requires the use of proprietary software, more expensive and potentially less secure due to the lack of public scrutiny. It is also less adaptable because it lacks the possibility of contributing bug fixes or alternative implementations. Most community networks rely on proprietary hardware and software black boxes, or a mix of open and closed source, for the previous reasons. Fortunately, there are a range of standards and public specifications which allow interoperable interconnection of components from different sources. The community of open source developers has made and is working on an impressive list of key solutions for community networks (see our list below for details).
Open specifications and standards,26 in comparison to proprietary specifications, are key to promoting software and hardware alternatives, reducing cost and promoting specialised and optimised components that are interoperable. Public research helps to address the needs of the population, exploring challenges with high societal impact, in contrast to research in industry, which is typically focused on the development of competitive advantages and economic benefits that benefit private profits and shareholders first. In fact, Elinor Ostrom identified this requirement in the task of designing sustainable, complex human-resource systems: “Building respectful collaborations between local users, public officials, and scientific experts is a vital requisite of adaptive governance.” 27
Software and other resources for community networks
The following is a set of typical software and related resources used in community networks.
Wi-Fi access points
OpenWISP: A software platform that can be used to implement a complete Wi-Fi service, including managing access point devices, captive portals, user credentials, accounting data and monitoring. 28
Cellular mobile access
Osmocom: An umbrella project focused on open source mobile communications; includes software and tools implementing a variety of mobile communication standards, including GSM, DECT, TETRA and others. 29
OpenBTS: BTS stands for base transceiver station. OpenBTS is an open source software-based GSM access point, allowing standard GSM-compatible mobile phones to be used as SIP endpoints in VoIP networks. 30
Firmware for routers
OpenWrt: An open source project for an embedded operating system based on Linux, primarily used on embedded devices to route network traffic.31
Quick Mesh Project (qMp): A system for easily deploying Mesh/MANET networks using Wi-Fi technology. qMp has been designed to be used in any scenario, such as free community networks, corporate networks, large social events, quick network deployments, etc. The qMp firmware, based on OpenWrt, works on many embedded Wi-Fi network devices. 32
LibreMesh: LibreMesh is an initiative of community network members from different continents to unite efforts in developing tools to facilitate the deployment of free networks for any community in the world. The main tool is the LibreMesh firmware, based on OpenWrt, which standardises the creation of Wi-Fi communities and provides roaming to existing ones. This project was initiated to merge a number of pre-existing firmware projects: AlterMesh (from AlterMundi, Argentina), qMp (from guifi.net, Catalonia) and eigenNet (from eigenLab, Ninux, Italy). 33
There are two families of protocols: distance-vector routing protocols, based on the Bellman-Ford algorithm, share only aggregated information about the path metrics, whereas link-state routing protocols, based on the Dijkstra algorithm, share the whole view of the network, and the metric of every single link is known by every router. Therefore, in link-state routing, every router has a global map of the network, whereas distance-vector routing only takes into account vectors (arrays) of distances to the other routers in the network. The most popular distance-vector routing protocols in community networks are Babel and BMX, while Batman-adv and OLSR are widely used for link-state routing. 34
Babel: A distance-vector routing protocol for internet protocol packet-switched networks that is designed to be robust and efficient on both wireless mesh networks and wired networks. In June 2016, an Internet Engineering Task Force (IETF) 35 working group was created whose main goal is to produce a standard version of Babel.36
BMX6/7: BMX6 (or the new version BMX7) is an IPv6 native dynamic routing protocol which offers very advanced features and a small network overhead (thanks to the distance-vector strategy and a set of optimisations). Version 7 has, in addition, security routing extensions.37
Batman-adv: B.A.T.M.A.N. (better approach to mobile ad-hoc networking) Advanced, known as Batman-adv, is a link-state mesh routing protocol which runs in the kernel space. Even if the network topology is made by multiple nodes and multiple hops, it abstracts it to a single layer 2 collision domain, so from the user perspective the entire mesh will look like a single LAN. This architecture is very interesting for roaming purposes given that network connections are not lost even when moving and changing access points. 38
OLSR: The Optimised Link State Routing protocol is an IP routing protocol optimised for mobile ad hoc networks. OLSR is a proactive link-state routing protocol, which uses control messages 39 to discover and then disseminate link-state information throughout an ad hoc mobile network. Individual nodes use this topology information to compute next hop destinations for all nodes in the network using shortest hop forwarding paths.40
Prometheus: A metrics collection and monitoring system that is particularly well suited to community networks, with data exporters for network nodes, including network traffic and BMX6/7 routing metadata. 41
FreeSWITCH: A free and open source application server for real-time communication, WebRTC, telecommunications, video and VoIP.43
FiberFy: An application for those who develop fibre networks. It allows implementers to plan deployments and maintenance, define coverage areas, prepare projects and budgets, etc. It allows the sharing of information among actors who can intervene in a fibre deployment: suppliers, local administrations, and retailers. 44
Governance and economics
1 This work has been partially funded by the European Commission, H2020-ICT-2015 Programme, Grant Number 688768 “netCommons” (Network Infrastructure as Commons).
2 Analogous to comparing traffic characteristics of vehicles (packets) on roads (links) referring to the thickness of their asphalt (signal to noise ratio) or width of lanes (narrow or broad).
3 https://en.wikipedia.org/wiki/Broadband; regional definitions – EU: https://ec.europa.eu/digital-single-market/en/broadband-glossary and UK: https://www.ispreview.co.uk/broadband.shtml
6 Less than 300 metres per microsecond (light), comparable to the execution time of a CPU instruction.
13 The industrial, scientific and medical (ISM) radio bands are radio bands (portions of the radio spectrum) reserved internationally for the use of radio frequency energy for industrial, scientific and medical purposes other than telecommunications. https://en.wikipedia.org/wiki/ISM_band
18 See, for example, Waites, W., et al. (2016). RemIX: A Distributed Internet Exchange for Remote and Rural Networks. https://arxiv.org/abs/1603.08978 and Baig, R., et al. (2016). Making Community Networks Economically Sustainable: The guifi.net experience. people.ac.upc.edu/leandro/pubs/baig-sigcomm.pdf
19 Most recent version in English (outdated): pe ople.ac.upc.edu/leandro/docs/ordinancePEIT-rev14-en.pdf ; updated version in Catalan: https://fundacio.guifi.net/web/content/2322?unique=cef4bebe39b45ba50ed5ebb5e2a63ecaf07e6cb4&download=true
20 Navarro, L. (2018). Network deployments for universal connectivity. Presentation at IETF 102, 14-20 July, Montreal, Canada. people.ac.upc.edu/leandro/docs/ietf-102-universal.pdf
26 Such as those promoted by organisations such as the Institute of Electrical and Electronics Engineers (IEEE) or the Internet Engineering Task Force (IETF).
45 Baig, R., Roca, R., Freitag, F., & Navarro, L. (2015). guifi.net, a crowdsourced network infrastructure held in common. Computer Networks, 90. people.ac.upc.edu/leandro/pubs/crowds-guifi-en.pdf
46 Baig, R., Dalmau, L., Roca, R., Navarro, L., Freitag, F., & Sathiaseelan, A. (2016). Making Community Networks E conomically Sustainable: The guifi.net experience. Paper presented at the 2016 Workshop on Global Access to the Internet for All (GAIA), 22-26 August, Florianopolis, Brazil. people.ac.upc.edu/leandro/pubs/baig-sigcomm.pdf