Four cloud-native advances that streamline the process

IoT and Edge oriented applications need the high speed capabilities and the robustness of new disaggregated environments to support low latency remotely hosted utilities and applications. This is especially the case in organizations that predominantly rely on traditional monolithic SAN-type storage resources. High throughput, low latency delivery of data services and other large data set backup operations is hard, if near impossible, to achieve on architectures based on large storage SANs.

Remotely hosted IoT and edge service applications such as video monitoring for a retail outlet, or for cell tower based caching for telemetry or analytics, traditionally involve a cacophony of compute cores, OS’s, communication protocols, and other complications. Yet the data center- based applications or databases those end point targets are usually more straightforward to host and manage with standard tools. What results is a complex architecture that is challenging for which to write individual applications, and a nightmare to keep in sync. Dynamic memory, persistent memory and storage are classically small, slow, and expensive.

In addition, remote in situ block storage for such applications has generally been bound to the local compute environment. Any access to that data was constrained to go through that local processor. This bottleneck in data access greatly constrains the ability to use it.

Micro Kubernetes (K8s) change the playing field for the compute environment. All the disparate compute cores and OS needs are homogenized by the cloud native layer that allows developers to focus on the functions they’re developing, rather than how they need to build those functions to suit a specific hosting architecture. The developer can worry about how they are processing the remote workload and the datacenter workloads with the same development tools and restful APIs, not whether there is some nuance of an ARM-based instruction set for a custom SOC requiring them to further adapt.

Micro K8’s, on their own, however, don’t resolve the local remote storage issue. There is still a set of utilities required including CSI and remote manageability features to access and manage the persistent data hosted at the endpoint or remote function caching location.

Key Technologies: NVMe

For the physical environment, NVMe, especially NVMe over Fabric (NVMeoF), is both a tool and a host target that can catalyze such a change. It’s ideal for the microservices and containerization. NVMe can drastically improve application performance over classic implementations. TCP/IP addressing of the storage grants it autonomy from the compute layer. Given the right tools to manage this data access freedom, developers can now transform the process of writing for those implementations to be more robust, more flexible, and faster.

The OpenEBS suite of engines and utilities, from this CNCF Sandbox project, helps developers to to cover this gap. The project’s newest CSI engine, Mayastor, is specifically written for NVMe and NVMeoF storage. An autonomous storage container in a K8 environment can be presented to, accessed by, and remotely managed in concert with the rest of the cloud native infrastructure and utilities. Access speeds for that data are no longer constrained by the classic SATA protocols which can give a 3-5x bump to I/O performance. A suite of both local and remote K8 based utilities can access that stateful data without the constraint of a local processor and shared TCP/IP socket. The latest high speed networks can easily and more flexibly directly share the remote data with a central datacenter and have the hosting complexity abstracted into a Kubernetes environment.

For example, some sectors such as the media & entertainment are heavy I/O users. More nimble storage solutions can help address a wide range of issues – including performance for edge delivery, TCO reductions for render farms, faster throughput for content generation and editing cycles. These are also applications that scream for NVMe speed.

What does this enable? A remote automatic camera monitoring a gate at the Oscars can tell the datacenter that celebrity Foo Bar just passed a gate. The cached image database on local NVMe triggers a real time scoop to the paparazzi database and alerts the photographers. All of that logic now takes place inside a Kubernetes abstraction layer facilitated by high speed, autonomous storage managed by line speed CSI’s.

Deployment of low latency, quick response technology like NVMe also means that organizations don’t require an Apollo moon mission-sized development team. App deployment (VERIFY) can be handled by a small team that is versed in cloud native technologies which have abstracted away the complexity of the physical layers.

Remote manageability functions

Another cloud-native advance that makes developers’ life easier for IoT and edge development is the added simplicity of remote management for media delivery and return analytics that is made possible by Kubernetes environments. Remote cache content delivery for local markets from non datacenter (or central geographic repositories) will be simplified via cloud native containerized storage scaled for specific hosting environments. User analytics collection and caching can be orchestrated with the same storage infrastructure tools across all environments and delivered back to central repositories with homogeneous cloud native CSIs.

Whether that infrastructure is on a cell tower, in a co-lo cloud environment or in the datacenter can be managed with the same block tools and file system delivery utilities across the entire ecosystem. For example, with the OpenEBS software, cross compiled utilities for different specialized compute cores and local specialty OS’s become invisible to everyday operations with the Kubernetes abstraction layer on top, and with other tools to manage that dataflow. Local hostpath or Mayastor-based PVs plus OpenEBS NFS for containers make key building blocks for any implementation.

Cross-compile functions

DevOps teams face an increasing number of OS and target environments they need to build for ARM, GPUs, and other specialized computing environments introduce a host of constraints. These constraints can increase build/dev and testing environment complexity and “back end” time for releases hurting overall productivity.

Cloud-native advances such as CSI drivers natively cross compiled to multiple different instruction set types also help developers streamline IoT and edge development. Such images when cross-compiled are small enough to fit almost all IoT or Edge specialty cores. This allows for one set of container storage utilities for a large heterogenous set of target environments but with one logical type of interface. The decreased complexity makes development simpler and more robust for all cloud native applications.

Abstraction-driven ease

The homogeneous, robust utilities for deploying remote applications that are offered by container-native open source storage applications also save developer time and hassles. High speed analytics at the edge, which are common in telecom applications for either application or traffic heuristics collection, benefit from the same TCO advantages as a datacenter. The manageability paradigm employed for analytics data collection, processing and uplift also simplifies deployment of applications in classically embedded-type equipment and locations.

Making the infrastructure invisible and fast for large scale, broadly dispersed compute needs makes it simpler to provide remote services by requiring less from the deployment teams.

In telephony use cases, for example, infrastructure specialists can focus on deployment of optimal servers to towers, while telemetry and analytics teams need not become deeply trained in these infrastructures but simply write their applications given a few simplified constraints. Using the same cloud native utilities across all greatly accelerates their ramp time.

The steady march away from large, monolithic storage SANs, to more nimble, flexible and faster storage containers can’t come soon enough for organizations where IoT and edge applications are scaling fast. Abstracting away the IoT complexities to cloud native utilities allows developers to focus on the end functions more, and the idiosyncrasies of the hardware less. With an underpinning of the right newer technologies, developers can build functions on a foundation in which their modern architectures can flourish.

-By Michael Turnlund, head of containerized field operations, DataCore Software