Jenkins Pipeline

Introduction


I believe in simplicity. It means I think things should be simple to work with.
It doesn't mean that it always should be simple inside. It could be complex, sometimes very complex.

At the same time, I distinguish two types of complexity:
  1. complexity caused by indirect and non-obvious relationships between components,
  2. complexity caused by mess within components itself and messy relationships between them
First one means product could be implemented in a very smart way, built on large number of implicit assumptions, often non-obvious.

Second one means product's complexity is exaggerated by very confusing, illogical and messed up relationships between components. Unlike the first one, relationships exists, but they make no sense not because they are so over-smartly designed, but because they are spaghetti-like.

Often those two types could be met together in same product. Hope you'd never had a chance to deal with such.

The way to fix complexity type #1 is to remove implicit assumptions by adding smaller components with visible relationships.

However, to fix complexity type #2, you need to understand it deeply. What if thing is complicated b/c you can't understand it yet at this moment? To be able to answer this question, I always start with research. And my research has some kind of diagram. I believe that visualization is the best way to tackle complexity. At least it always works well for me.

Simply put, visualization is one of the best ways to simplify hard and complex things. Even if this is a very basic diagram, it is still better than nothing. Sometimes, it requires a time to find a correct form of visualization within correct level of abstraction, but once you did it, you are half done.

Continuous Deployment


And this post is about visualization; and how it can help to combine parts into a single simple picture. And more specifically, it is about how visualization can help simplifying continuous delivery.

Continuous delivery is a process of getting you product from source code and into production. It usually happens through a pipeline of jobs: first job is compiling source code and building artifacts, then goes integration testing , deploying to staging environment, and eventually to production. For example, in Jenkins job is usually created for each of these steps, where each job triggers next one once it's finished successfully. Fore example, there would be jobs like 'Build XYZ,' 'Test XYZ,' 'Deploy XYZ to Staging,' 'Deploy XYZ to 1-box' and 'Deploy XYZ to Production'.

Default Jenkins View would present those jobs as a list, with no visible relationships between those. But there are relationships between them. And actually all those jobs are here for the single most important goal: get new version into production for the customers! So relationships play important role, which is hidden from us as users.

You might not even feel it immediately, but this presentation of jobs list brings a complexity. There is a sense behind those jobs, but it is normally hidden from viewer, unless one ready to spend time to understand how things work.

Pipelines in Jenkins


But good thing is Jenkins already allows you to remove this complexity. And it's via visualization of the pipeline of jobs you've created.

This support is brought by "Build Pipeline Plugin". This plugin adds a new "View" type called "Build Pipeline View".



In a next step you would need to pick your first job in the pipeline.



Then pipeline would be created based on jobs dependencies: this pipeline will contain jobs that would be triggered by first job, and also jobs which are triggered by that jobs and so on.

And now once someone makes a change into XYZ source, a new job for "Build XYZ" would be triggered. Once this job is finished successfully, it will trigger "Text XYZ" and so on. As you see no functionality change happened here, but with pipelines it is possible to visualize both dependencies between those jobs and what is a current state of the CD process.



That makes things so simple to work with. You can understand build structure and current state with a single glance.

More to that, Jenkins 2.0 comes with built it support for pipelines.

Blue Ocean


I'd also like to mention the initiative called "Blue Ocean" which sets a goal to build a better visualization of pipelines in Jenkins. I'd be happy to see it in live one day.

Other Products


There are bunch of other products that would help you to create build / deployment pipeline:
  1. AWS CodePipeline - https://aws.amazon.com/codepipeline/
  2. Concoure CI - https://concourse.ci/
  3. Bitbucket Pipelines - https://bitbucket.org/product/features/pipelines

Stack trace dump on Go

I'm from Java world. Means I spend most of the time coding things on Java and running them in Java Virtual Machine. This has own benefits and disadvantages. For example, I really wish JVM would use less memory and be faster from the start. It is awesome you can benefit from JIT eventually, but till then you should wait a code to run for thousands times before it can be translated into machine code.

At the same time, Java has bunch of benefits. I believe the most important one is a set of available tools: from small utilities to powerful IDEs to application servers.

One of such utility tools is jstack. It is use to print a stack dump of application threads, including main one. What a useful thing! Saved so many time when I tried to investigate why my application got stuck or is really hard on CPU.

And it is also one of the tools I really needed a few days ago: I had to understand why service written on Go is so eager for CPU resources. I spent some time searching for a tool that would work similarly to jstack but for a Go application. Couldn't find one. Yet found how to get a stack trace with a small change in my Go application.

Go standard library already has everything you need to print a stack dump: signals support to notify application and runtime.Stack function to get stack dump, and print it.

NOTE: signals are primarily supported on UNIX based systems, it means this approach might not work as expected on Windows.


And now you can run you program, and ask for a stack dump by sending a signal using kill command:
$ killall -SIGQUIT accountservice
and you'd see something like this


Output consists of stack traces for each goroutine. Each stack trace starts with 'goroutine', its number and current state. Stack trace, like Java one, shows the most recent operation on top, contains code file name (not a class name), line number, and also function parameters. Also stack trace contains information about goroutine creator.

Now, it should be easy to find why application got stuck, or what is it currently doing so that almost all CPU is used up. However, this solution might not work always: in case all resources are used, it might take a time until SIGQUIT notification is processed by application.

Simple RESTful API on Go (with Gin)

It is very populate to use Go to create microservice applications. And actually there are many benefits when using Go comparing to other technologies: it is easy to start, easy to create, and so cheap to run that on your servers. The last one is one of my favorites. Go application has so small footprint on memory and CPU comparing to Java applications, and the same time runs faster.

In this post, I want to show how to create a simple RESTful web service on Go using awesome Gin library.

First, lets create an main function, that sets up logging, Gin router with registered paths and handlers. Gin provides a simple way to create template-like paths and register a handler for each path.

Now lets create DTOs for our business models, and methods to convert from model to DTO:

Finally, need to create functions to handle requests. Again, super easy with Gin:

And we are done. Well, almost, as those examples don't actual models and functions to work with storage.

Simple configuration on Go

Very often your new tool/service needs to have a configuration defined either via parameters or configuration file.
For example, you want to have different settings in development and production environments, or you just have a separate fleets for different clients.

It is pretty easy to add configuration support to you Go application using YAML for configurations and gopkg.in/yaml.v2 for deserializing YAML files into struct value.

Here is an example YAML configuration file. It contains service name and its listening endpoint, and also database connection URL. Pretty simple!

To read configuration, we first open and read whole configuration file, and then use yaml package to unmarshal it into value of Configuration type.

And finally a part of main function in our application, where we use parameter to pass a path to the configuration file, and load it to use configuration later:

That's it. Simple and fast!

Reinforcing feedback loops

There are two different types of feedback loops recognized in systems:
  1. Balancing feedback loops
  2. Reinforcing feedback loops
Balancing feedback loop controls inflow and outflow of resources. Example of it would be amount of money on the bank account. Amount increases if you put money into, and decreases if you withdraw them.

Reinforcing feedback loop is a different kind of animal. It isn't as strait forward as balancing feedback loop. Example of it would be interest rate for the bank account. The more you have money, the larger interest you get. The larger interest you get, the more money on you bank account you have. Another example of reinforcing feedback loop would be well know dependency between education and social status: if you have money, you can get better education, the better education you have, the more you can earn etc.

Many people are very focused on balancing feedback loops and give little attention to reinforcement feedback loops. At the same time latter one are very important in our life.

Reinforcing feedback loops are very important in our life. They can allow fast growth or decay. They are that magic behind growth of many successful companies.

The better Google's search is, the more people use it. The more people use it, the more data Google has to improve its search.

The better apps on your iPhone, the more you use it. The more you use it, the better apps are becoming. The better apps, the more you use iPhone. Similar thing with AppStore.

The more you invest into some company XTZ, they more money it would have. The more money they have, the more successful they are. The more successful they are, the more stock growths. The more stock growths, the more chances you keep investing.

And so many and many examples.

Reinforcement feedback loop is one behind "Success for successful and fail for failures" trap.

Thus, creating successful enterprise is not only about good old balancing feedback loop. It is always a lot about reinforcing feedback loop behind it.

Wonderful book on this:
Thinking in Systems: A Primer

See it on Goodreads







Links:
  • http://www.systems-thinking.org/theWay/sre/re.htm
  • https://www.youtube.com/watch?v=hdGxIameiM8