Skip to main content


Story of deploying pod to every node and preventing them from termination

 During the development of one of our new features, I faced an interesting challenge. The feature requests are simple and clear: there is an existing DaemonSet  (workload running on "every" node) on the target Kubernetes  cluster, we have to deploy another workload next to each instance and prevent workload termination under certain conditions. Let's split the problem into two parts; deployment and prevention. From the deployment perspective, another DaemonSet makes lots of sense. If we use the same node selectors as the existing one, Kubernetes would deploy pods to the same nodes. In our case a custom operator is working in the background, so we are able to sync node selectors, but for other kinds of deployments this should be a tricky piece. On the topic of prevention PodDisruptionBudget [PDB] comes into the picture. Without going into the details PDB allows us to define how many of the target pods should be terminated by Kubernetes at once. It has a maxUnavailable fi
Recent posts

Autoscaling Calico Route Reflector topology in Kubernetes

Kubernetes is a great tool to organize your workloads on a low or high scale. It has many nice features in different areas, but it is totally out-sourcing the complexity of the network. Network is one of the key layers of a success story and happily there are many available solutions on the market. Calico is one of them, and it is I think the most used network provider, including big players in public cloud space and has a great community who works day by day to make Calico better. Installing Kubernetes and Calico nowadays is easy as a flick if you are happy with the default configurations. Otherwise, life became tricky very easily, there are so many options, configurations, topologies, automation, etc. Surprise or not, networking is one of the hard parts in high scale, and requires thorough design from the beginning. By default Calico uses IPIP encapsulation and full mesh BGP to share routing information within the cluster. This means every single node in the cluster is connected w

Kubernetes and Calico development environment as easy as a flick

I became an active member of the Calico community so I had to build my own development environment from zero. It wasn't trivial for many reasons but mainly because I have MacOS on my machine and not all of the features of Calico are available on my main operating system. The setup also makes some sense on Linux hosts, because if the node controller runs locally it might make changes to the system, which always has some risk in the playing cards. The other big challenge was that I wanted to start any version of Kubernetes with the ability to do changes in it next to Calico. Exactly I had to prepare two tightly coupled environments. My idea was to create a virtual machine with Linux on it, configure development environments for both projects in the VM and use VSCode 's nice remote development feature for code editing. In this way projects are hosted on the target operating system, I don't risk my system, I don't have to deal with poor file system sync between host a

Connecting non-Kubernetes nodes to Calico overlay network

Kubernetes networking has some basic rules.  In short, every pod has to communicate with every other. Selecting the right network plugin for the cluster is a critical key component when planning and architecting a new cluster. Luckily there are great presentations and blog posts around the topic of Kubernetes cluster networking on the internet, but the available sources are very limited about how to connect external resources that aren’t part of the cluster into the mesh. It all depends on what we would like to achieve, so finally, we have to glue the solutions together. In this post, I would like to tell our story @IBM about converting an existing node to become a full member of our Kubernetes + Calico network. First of all, we had to specify the main goals: Make node full member of the overlay network The external node needs a pod IP to be able to reach it like any regular pod in the system The pod IP must be listenable for services on the external node Service discovery i

First impressions of the new Cloud Native programming language Ballerina

Nowadays everything is Cloud Native; everybody talks about CN tools, frameworks, solutions, and so on. On the other hand, those tools totally changed the way we design, develop, test and release modern applications. I think the number of issues that we solved with the new concepts is equal to the number of new challenges, so in short,     we simply shoveled problems from one hole to the other. Many new tools appeared on the market to make developers' life easier by integrating software with the underlying infrastructure watching file changes and building containers automatically generating resource descriptors on the fly allowing debugging in a running container etc. Next to the new tools, new programming languages such as Metaparticle , Pulumi or Ballerina have been born. The last one had my attention because others are extensions on top of some existing languages, while Ballerina is a brand new programming language, designed from

Advanced testing of Golang applications

Golang has a nice built-in framework for testing production code and you can find many articles on how to use it. In this blog post, I don't want to talk too much about the basics , table-driven testing ,  how to generate code coverage  or detect race conditions . I would like to share my personal experiences with a real-world scenario. Go is a relatively young and modern programming language on one side, and it is an old fashion procedural language on the other. You have to keep in mind that fact when you are writing production code from the beginning, otherwise, your program should become an untestable mess so easily. In a procedural way, your program is executed line by line and functions call other functions without any control of the dependencies. Hard to unit test, because you are testing underlying functions too, which are side effects from the perspective of testing.  It looks like everything is static if you are coming from object-oriented world. There are no depend

How to sign messages in Java and verify in Go

In my company, we use Java and Go as our development platforms and of course, sometimes those projects have to communicate with each other. In the current blog post, I would like to introduce our solution to sign messages on the Java side and verify them in a Go service. First, let's talk a bit about architecture. Our Java application spins up new virtual machines in the cloud, and the base image of the instance contains a small Go service. That service is the main entry point of our configuration management system, and we don't want to allow any operation from untrusted clients to modify nodes. Two-way SSL with a signature in the request sounded fair enough to trust in clients. Both components are open source, we don't have any "secret" in the binaries, so we elected RSA asymmetric key pairs to generate and verify signatures. Java has the private key and Go has the public one. Time to go deeper. Java is an old platform (personally I have many years of experi