Platform Design - Types and MVP

Part 2: Types and MVP

Introduction

If we look back on the schema of the previous blog post we have discussed where kubernetes is placed in that diagram. I want to take the next step and look into the platform aspect of it. We have determined that a platform offers more than a traditional infrastructure. It gives the developers a toolset to help them go faster or abstract some infrastructure away so that he/she doesn’t have to think about it. The problem with this statement is that it is extremely vague. There is no definition behind it. What is the minimal viable product for a platform. What are the overlapping tools that all the platforms need. Day 2 operations comes to mind, but is that the same for every platform? A workflow that is meant for the web has different types of outputs and metrics than a workflow for data intelligence. That makes another question arise what should the platform support. Think about it. An application platform has different needs than a data intelligence platform. These are the two topics I want to discuss in this article.

Types

A type of platform defines what it is used for. So what kind of workflows will land on it. An easy example is an application platform where a web server which exposes a frontend or api would live. What kind of platforms could exist or are already in use let’s try to summarize them.

• Application platform
  • Cloud Foundry
• Data Science platform
  • Kubeflow
• Database platform
• Logging platform
  • Elastic search on K8S ~
• Messaging Platform (ActiveMQ, Kafka)

A lot of these types don’t exist yet or will come in the near future. For example hosting a database on Kubernetes becomes more viable in the future with databases moving their architecture towards a 12-factor idea. CoackroachDB is a nice example of hosting a db on a kubernetes cluster, but just hosting an application on kubernetes doesn’t make it a platform. A platform is something that adds value besides the initial product. An application platform for example offers an easy way to deploy and host and gain insight into the workflow deployed on it.

A database platform could for example give an easy interface to spin up a new database or even a different type, say a NoSQL solution instead of an SQL one. Making a backup of the data could be enabled by default and restored if it would fail. To access it you could for example just connect to it, and the platform would arrange the connection string or even the whole database. The database would be spun up by just requesting it and be bound to it.

What I’m trying to show with these examples is, what kind of acceleration a platform can give. To shift the focus on developing actual business value, instead of worrying about how to build infrastructure or even on how to use that infrastructure. It’s not just a database or elastic search running on a cluster, but more of an abstraction on top of a cluster running a database, or an elastic search cluster, to make the lives of developers easier.

Minimal Viable Product

It comes down then to what are the shared components of a platform. What is the minimal viable product that a platform needs to have. All the above types should have a common denominator which can be used and installed on top of the kubernetes cluster. However, kubernetes is not the limitation. To complete a platform it is possible to use elements of an IAAS and let that create a secret solution for you, or something else that you might need.

The most powerful feature of Kubernetes is the abstraction that it creates. By creating an interface any provider can write a way of interacting with the rest of the components as long as the provider sticks with the interface. Kubernetes dictates in this way how to work with it. The product Kubernetes, becomes more opinionated on how it works by enforcing how to write for it. In programming this concept is an interface used by the Object Oriented Programming, or OOP for short. To have this implemented on a scale of infrastructure and operations is brilliant. Think about it. As long as you have kubernetes running anywhere, it doesn’t matter how you use it. Talking to it stays the same despite the cloud you are running on.

Let’s step up the ladder and think about creating an interface for a platform. A minimal product that can be adapted to all the other platforms. A base layer for all the other types of platforms to build on. What does every platform need and what overlaps enough to pull out of the upper layer and implement in all of the platforms.

A quick list:

• Prometheus
• Grafana
• Filebeat / fluentbit
• Virtual Network with subnets
• A vault to store secrets in
• Helm
• RBAC rules
• Ingress

All of these things are valid, but how are you going to make the choice of which tools and maybe an even better question is: “Who cares!”. This sounds a bit strange, but it is the truth. A platform should take away choices and make them for you. If you think about the image from the previous blog post you see that the further up you go the less control you have. So who does care about these choices. Well I do of course, as a Platform Engineer I want to create a more opinionated platform. As a developer you should care about these things as well, but less. It is not your core skill to offer insights into your app or make sure it is connected in a network. Dealing with secrets or creating a way to expose your secrets will slow you down and lose focus on what is important, creating business value.

All in all a platform is there to make a developer go faster. Ignore the old infra problems and focus more on what is important. Which opens up a bigger question and food for thought for a new blog post. Hook, line and stinker, why it might not be for the best to follow the kubernetes hype. Think about this blog post. Is it for your company a core business to develop a platform. In most cases I would say not. The core business is usually in the apps on top of it. So why pick the lower option in the stack at the CAAS level? Why not pick something at the PAAS level? More about that later in a different blog.