From cloud... to the Edge

Features

The elevator pitch

IoT devices generate time-sensitive high bandwidth data, often as live streaming, and is often consumed close to where they are generated. It is expensive and wasteful to send all that data to the cloud, get computed, and get the results back on time. Services like Netflix stream high bandwidth static data from the cloud, which can easily be streamed locally from servers within an ISP network that is close to users. To solve these problems, we need to scale down and offload cloud services down to the edges of user networks. We are building a decentralized (no servers) platform to do that, by developing converged services that run at the edge. Our platform can scale to thousands of devices spanning large networks of ISPs and enterprises.

Inventions happen when there is a need. As COVID-19 outbreak has shown, we need to conserve internet bandwidth either by reducing cloud service quality or by offloading cloud services to the edge.

The edge

The edge is where user devices connect to. It can be an enterprise wiring closet, a cell base station, a cable headend or even a home wifi router. The edge is largely unattended, devices there are relatively independent requiring minimal deployment or operation support. These devices require minimal maintenance, often replaced as a whole when they fail. Any device designed for the edge should just work without operator support.

The challenge

Modern cloud/business applications are built from distributed microservices. Most of the code written for such an application essentially deals with gluing these services and translating between them, with the business logic implemented in less than 5% of the codebase. These microservices are often black boxes (albeit open source), with 80% of their code dealing with defensive logic, plumbing, and translation, and often have incompatible execution models with other services. Thus, cloud applications requiring hundreds of developers, devops and virtual machines cannot be moved to a standalone unattended environment like the network edge.

Our solution

We build hardware called Edge Service Router, ESR, built using standard PC components. It runs software called State Operating System, StateOS, which provides host and network infrastructure for running decentralized services. StateOS is designed for zero-touch installation and autonomous independent operation, and can automatically scale from a single ESR to thousands of ESRs. StateOS runs edge services, which are converged services built by integrating functionality of distributed services into a single instance. While ESRs expose their services to users using standard protocols, they don't provide APIs for third-party developers.

Symmetric operating system

In a StateOS network, there are no client/server or master/slave devices. All ESRs run the same software, they create a hierarchical network among them to share their state. Any ESR can serve user requests, either from their own resources or from other ESRs in the network. A failure of one ESR doesn't affect the network significantly, because user data is replicated among multiple devices.

Converged services

Converged services vertically integrate different services of a distributed computing infrastructure, without using a central server such as a database or file server. By combining different services into one, it eliminates defensive programming logic, duplicate framework code, duplicate business logic, message queues or RPCs; resulting in a smaller footprint suitable for edge operation.

Open source, but closed.

StateOS doesn't run applications, it only runs services developed/signed by Network State. A StateOS network is a standalone, private, sealed network, it doesn't need or work with other applications/servers like databases or fileservers. Unlike Azure/AWS, StateOS is not for developers, there is no developer API to StateOS. StateOS interacts with end-users through standard protocols like HTTP/HTML/WebRTC.

Fault tolerant.

An ESR is essentially a storage/computing cache device, an ESR is not an exclusive owner of data, it is always backed up collectively at other nodes and ultimately outside the StateOS network. Any data update at a single ESR is always propagated to other ESRs in the network. A failure of one ESR affects performance, but it doesn't affect the functionality or reliability of the network.

List of services.

StateOS currently supports WebRTC based media streaming, future services include GIS maps, Workflow engine (to support CRM, Order/Case management, Smart Contracts) and Web scanning (to convert webpages to structured data). The workflow engine is targeted at business analysts, who define tasks that don't involve flowcharts or programming, the operations team can track and manage a task's progress.

Design

Philosophy

StateOS is a minimalist software, it supports features it has to, no optional features. While the software industry is busy solving problems they caused/created, we go back in time and try to eliminate those problems through better design/architecture. We aim to innovate through integration, by bringing complex software components together and breaking their barriers.

Our strength is that we own our software stack with very little dependence on host OS. We don't fix bugs, we avoid them, we constantly refactor our code to eliminate scope for such bugs. We recognize that edge computing is hard, it takes a while to understand its workflow requirements, but we intend to adapt quickly to what we discover.

Low-level code

StateOS is currently implemented as a low- level kernel module on Windows, with plans to move directly to hardware bypassing Windows. By running at interrupt level and in kernel mode, StateOS has complete control of the device. With innovative design, we avoid memory management and locking problems commonly found in similar software. StateOS scales linearly and predictably within a device to use all available CPU, memory and network bandwidth.

Storage

We do not use file systems or databases to store our data, we store data directly on disks bypassing file systems. Our storage algorithms are optimized for SSD and harddisks. User data is typically stored in a format called Visual Design Language (VDL) which is derived from JSON, XML and HTML. With VDL, documents can be easily converted to display formats like HTML/PDF and at the same time used as a data exchange format much like JSON/XML.

Networking

StateOS uses a custom protocol for internal communication, we mainly focus on layer 2 (ethernet) multicast networking and use UDP tunnels when layer 2 networking is not operational. All ESRs communicate with each other, either directly or through group leaders and gateways, we create a hierarchical network scalable to thousands of devices.

Reliability

StateOS is a replicated environment, having multiple ESRs increases reliability and performance but are not functionally required. While data is not duplicated to every node, it is automatically copied to as many nodes as possible. Network protocols and storage use encryption, most standard IP services are turned off to avoid network intrusion.

Chain of immutable data units

In a StateOS network, user data/object/document is represented as a chain of data units. Each such data unit is self identifiable and immutable, any ESR can rebuild the data object by retrieving all relevant data units and assembling them. Thus, data objects are never centrally stored, they are everywhere. The state of data objects (not the data) are synchronized across the network and is also available on demand.

Workflow engine

Workflow engine is a core component that enables value-added services. Workflow is a business process as defined by a business analyst, is a chain of events/actions applied to an object over time, including actions generated from a timeout. Workflows are declarative tasks that can be executed at any ESR, with changes stored as immutable data units.

Media Service

Market need

We have seen explosive growth in on-demand streaming, like Netflix. Live streaming is still new and expected to grow even bigger. High-resolution IP cameras are now affordable, enabling TV quality broadcasting of events and lectures using multiple cameras. Streaming is still served by centralized cloud-based platforms. Cloud-based streaming solutions put enormous strain on ISPs, and would end up being prohibitively expensive for enterprises and universities. While Twitch and Netflix try to get into ISPs and enterprises, they don't have a decentralized, easily deployable solution that lets ISPs/enterprises deploy devices close to their users.

Product match

Cloud-based streaming is an ideal candidate to move to the edge. StateOS is designed to transport application level data streams to thousands of devices. Unlike the cloud, StateOS doesn't duplicate data streams between devices. StateOS streams media directly to customer devices using standard protocols like HTTP and WebRTC, thus directly streaming to browser-based applications with very low latency. StateOS is a vertically integrated solution, doesn't require third-party tools such as databases and file servers, thus offers a cleaner, simpler, cheaper and superior solution.

Content ingestion

Media streams are ingested into StateOS network through multiple protocols. Browser-based applications can stream media to StateOS through the built-in WebRTC protocol, with wide support for audio and video devices. IP cameras stream media to StateOS through RTSP protocol. Live streaming is sent to StateOS by game consoles and applications like OBS studio using RTMP protocol. Media files can be sent to StateOS through HTTP protocol. Thus, we support WebRTC, RTSP, RTMP, and HTTP for content ingestion.

Content playback

Media streams are sent to end-user devices through WebRTC protocol. Currently, we support H264 video and OPUS audio, we transparently transcode streams from other codecs to H264/OPUS. While we are open to add support for other codecs, we don't see an immediate need. WebRTC is supported natively by all major browsers in major platforms, thus we provide a client free solution.

Content storage

Media content is stored as independent tracks/streams indexed by play/record time. These frames and samples are stored as chunks that are distributed in realtime through the StateOS network. Any ESR in the StateOS network will be able to serve these media streams.

Provider neutral

ISPs and enterprises run the StateOS network, they have the freedom to control access to their network. Ideally, an ISP would store and stream content from original sources like studios and gamers, companies like Netflix and Twitch may sell a license for such content. Enterprises and universities may serve only locally produced content and may choose to restrict access to companies like Netflix and Twitch.

Competition

Streaming providers recognize the need to extend their streaming network into enterprises and ISPs, but they are going to face technical and logistical hurdles. Netflix offers a BGP router that can be placed inside an ISP, but it's far removed from end-users and doesn't help ISPs much. Network companies like Cisco might have more success in this space, but they need to think very differently from what are they are used to.

Licensing.

We focus on selling hardware, our software is available for review at GitHub. We don't grant rights to use our software, as it's not really usable without hardware integration.