Business
How to run data on Kubernetes: 6 starting principles
Kubernetes is fast becoming an industry standard, with up to 94% of organizations deploying their services and applications on the container orchestration platform, per a survey. One of the key reasons businesses deploy on Kubernetes is standardization, which lets advanced users see productivity gains of up to two times.
Standardizing on Kubernetes gives organizations the ability to deploy any workload, anywhere. But there was a missing piece: the technology assumed that workloads were ephemeral, meaning that only stateless workloads could be safely deployed on Kubernetes. However, the community recently changed the paradigm and brought features such as StatefulSets and Storage Classes, which make using data on Kubernetes possible.
While running stateful workloads on Kubernetes is possible, it is still challenging. In this article, I provide ways to make it happen and why it is worth it.
Do it progressively
Kubernetes is on its way to being as popular as Linux and the de facto way of running any application, anywhere, in a distributed fashion. Using Kubernetes involves learning a lot of technical concepts and vocabulary. For instance, newcomers might struggle with the many Kubernetes logical units such as containers, pods, nodes, and clusters.
If you are not running Kubernetes in production yet, don’t jump directly into data workloads. Instead, start with moving stateless applications to avoid losing data when things go sideways.
If you can’t find an operator that matches your needs, don’t worry, because most of them are open-source.
Understand the limitations and specificities
Once you are familiar with general Kubernetes concepts, dive into the specifics for stateful concepts. For example, because applications may have different storage needs, such as performance or capacity requirements, you must provide the correct underlying storage system.
What the industry generally calls storage “profiles” is termed Storage Classes in Kubernetes. They provide a way to describe the different types of classes a Kubernetes cluster can access. Storage classes can have different quality-of-service levels, such as I/O operations per second per GiB, backup policies, or arbitrary policies, such as binding modes and allowed topologies.
Another critical component to understand is StatefulSet. It is the Kubernetes API object used to manage stateful applications, and offers key features such as:
- Stable, unique network identifiers that let you keep track of volume, and detach and reattach them as you please;
- Stable, persistent storage so that your data is safe;
- Ordered, graceful deployment and scaling, which is required for many Day 2 operations.
While StatefulSet has been a successful replacement for the infamous PetSet (now deprecated), it is still imperfect and has limitations. For example, the StatefulSet controller has no built-in support for volume (PVC) resizing — which is a major challenge if the size of your application data set is about to grow above the current allocated storage capacity. There are workarounds, but such limitations must be understood well ahead of time so that the engineering team knows how to handle them.
-
Business7 days ago
Checkfirst raises $1.5M pre-seed, applying AI to remote inspections and audits
-
Business6 days ago
AI chip startup DEEPX secures $80M Series C at a $529M valuation
-
Entertainment4 days ago
Jinkx Monsoon promises ‘the queerest season of ‘Doctor Who’ you’ve ever seen!’
-
Business4 days ago
StrictlyVC London welcomes Phoenix Court and WEX
-
Business6 days ago
Retell AI lets businesses build ‘voice agents’ to answer phone calls
-
Entertainment4 days ago
How to watch every ‘Law and Order’ online in 2024
-
Entertainment4 days ago
BookTok and teens: What parents need to know
-
Business4 days ago
Why Apple’s ‘Crush’ ad is so misguided