Today the release pipeline at work broke. No harm was done, but finding the cause of the problem was difficult. While I analyzed the issue, once again, I learned something surprising, nay horrifying, about YAML.

Our application is deployed to a Kubernetes cluster via Helm. Helm uses so-called Helm charts to package applications and their Kubernetes configuration together. These packages can then be used to install or upgrade applications on Kubernetes clusters. Charts are composed of YAML files that will be processed by a template engine during the deployment. One of our use cases for this feature is the replacement of placeholder tokens with environment-specific values.

Now to the helpful error message that was logged when the error occurred during the deployment. I formatted it for better readability and removed some irrelevant details.

Error: UPGRADE FAILED: YAML parse error on chart/templates/deployment.yaml:
error converting YAML to JSON:
yaml: invalid map key:
map[interface {}]interface {}{
  "required \"A .Values.someValue value is required.\" .Values.appSettings.someValue":
  interface {}(nil)
}

Only two lines were added to the deployment.yaml file mentioned in the error message. These two lines contained a placeholder that should be replaced by .Values.someValue that is passed to the Helm upgrade command. After I completely ignored the error message, I checked the usual suspects. First, I inspected if .Values.someValue contained a value. Second, I ensured that the deployment.yaml file is correctly indented. Neither of these theories checked out, so I finally read the error message and realized that the message was new to me. Unfortunately, I only understood that something went wrong during the parsing of the deployment.yaml file and that Helm is written in Go (interface {} ❤️).

So let’s take a look at the two lines that were added.

- name: "SomeName"
  value: { { required "A .Values.someValue value is required." .Values.someValue } }

It’s actually pretty easy to spot the issue in these two lines. Especially if I tell you that placeholders in Helm chart template files are supposed to start with two opening curly braces {{ and end with two closing curly braces }}. So the issue here is the space between the curly braces { { and } }. After the space was removed, the release worked again.

I could have stopped there, but one thing that left me puzzled was the error message. It isn’t simply saying something like: “Hey, you added some weird braces in a YAML file which isn’t valid YAML, duh!” but it actually tells me something like: “Hey, I tried to put your stuff into a map type but failed, and by the way, I use Go!”. So I assumed the YAML must be at least partially valid if the parser does not stop at the first curly brace it encounters.

It’s YAML specification reading time, yay. Specifically, I looked at the current YAML specification, which at the time of writing is version 1.2.2 and can be found at yaml.org. This is a pretty long document spanning ten chapters. So I did the first thing that came to my mind — activate the browsers search and type in “curly” — Bingo! I found exactly one match for the word, which led me to Chapter 7. Flow Style Production in the specification. If you want to know in detail what YAML’s flow styles are, just read the specification. I will only tell you that the following sample is 100% valid YAML and 97.8% valid JSON. Only the last value of the structure is not valid JSON.

{
  "is": "this",
  "really": "valid",
  "YAML?": true,
  "btw": [
    "this",
    "is",
    "also",
    "YAML"
  ],
  "oh": no
}

This explains why the parser didn’t fail when it encountered curly braces in the YAML but instead tried to do its best with what it got. After looking into this issue in detail, the error message also seems less cryptic. At its core, the error message says “invalid map key”. This means that a map, sometimes called a dictionary or object, is created, and the given key is invalid. The key is the construct inside the outer curly braces, which once again is a map. But a map cannot be used as the key of a map. Therefore, the error is raised.

It’s not over yet. The snippet above can be translated differently depending on the YAML specification that is implemented by the parser. Specifically, the last value, no, is problematic. In YAML version 1.2.2, this would be interpreted as the string "no" because you do not always need quotes around strings in YAML. But in earlier versions, no will be translated to the boolean false (see YAML v1.1 bool draft). Just search for the Norway problem online, in case you want to know more about this hilarious topic.

Once again, I know more about YAML than I did before, which will probably make it easier to avoid and detect bugs in the future. Generally, I think YAML has its place, but it is way more complex than JSON, making mistakes more common. The German phrase “So ein Jammer!” can roughly be translated to “What a pity!” and I often have to say “So ein YAML!” whenever I have to touch YAML.