Introduction to YAMLPath

I want to introduce YAMLPath, a Java DSL framework for reading and manipulating YAML documents fragments. I created this project from scratch and I’m proud of it because it brings a world of possibilities when you need to read fragments of YAML resources or/and manipulate a bunch of YAML files.

A bit of history

Let me try to summarize how the YAMLPath innitiative started.

I was asked to develop the Quarkus Helm extension to automatically generate the Helm charts resources. As an input, I had autogenerated YAML files, for example:

apiVersion: v1
kind: Service
metadata:
  name: example
spec:
  ports:
    - name: http
      port: 80
      targetPort: 8080
  type: ClusterIP

Using the input, I had to produce a full Helm chart folder structure:

  • Chart.yaml
  • values.yaml
  • /charts
  • /templates
  • /templates/deployment.yaml
  • /templates/ingress.yaml
  • /templates/service.yaml
  • /templates/NOTES.txt

To continue with the above example, we’ll focus only in the generated file at /templates/service.yaml. If you’re interested in more about this extension, I will write another post speaking about it or you can go to the official documentation guide which is maintained by me anyway :).

So, to get the maximum benefit from Helm, we need to map some special properties into the values.yaml by replacing locations using the Helm expression: ``:

  • /templates/deployment.yaml
apiVersion: v1
kind: Service
metadata:
  name:  <-- This is the objective
spec:
  ports:
    - name: http
      port: 80
      targetPort: 8080
  type: ClusterIP

How can we do that? It’s quite obvious that we need a DSL parser that we could use to locate parts of the YAML resource using expressions. But, how?

First version: do not reinvent the wheel! If there is already something that does it, use it. And the most similar framework that I found was JSONPath. Even JSONPath has a really cool online editor that you can use to evaluate your expressions.

Actually, the first version of the Quarkus Helm extension worked JSONPath. However, we encountered two major issues with this framework:

  • Obviously: It only works with JSON resources! We had to map the YAML resources into JSON files and then transform back the JSON files into YAML. This is sub-optimal.
  • And the JSONPath expressions are quite quite hard to maintain/use/read/understand/…

For example, the JSONPath expression to locate the part at metadata.name is $..metadata.name, but things get complicater when you want to also use filters: $.[?(@.kind == 'Ingress')].spec.rules..http.paths..backend.service.name or literal values (single quote values) $[?(@.kind == 'Deployment')]['spec']['template']['metadata']['annotations']['app.dekorate.io/commit-id'].

Therefore, I decided to write a solution similar but simpler to JSONPath, but that natively supports YAML resources: YAMLPath was born.

JAVA API

YamlPath is available at the Maven Central Repository. To use it, simply declare the following dependency part of your pom file:

<dependency>
    <groupId>io.github.yaml-path</groupId>
    <artifactId>yaml-path</artifactId>
    <version>${latest version in Maven Central}</version>
</dependency>

The simplest most straight forward way to use YamlPath is via the static API.

String yaml = "apiVersion: v1\n" 
            + "kind: Service\n" 
            + "metadata:\n" 
            + "  name: example\n";

String name = YamlPath.from(yaml).readSingle("metadata.name");
// name == example

And replacements:

String newServiceYaml = YamlPath.from(yaml)
        .write("metadata.name", "")
        .dumpAsString();
// Output of newYamlContent is:
//        ---
//        apiVersion: v1
//        kind: Service
//        metadata:
//          name: ""

Using YAMLPath, you don’t need to learn much about the expressions itself since expressions are rather intuitive, simply follow the YAML tree:

  • metadata.name to select the part at metadata and then name
  • (kind == Service).spec to select a resource which kind is equal to “Service” and then go to the spec part
  • *.spec to select all the elements that contains a part called spec
  • `metadata.labels.[‘value’] to select the label with the literal value “value”

To find more examples, go to the YAMLPath repository.

Command Line

YAMLPath is not only a Java framework, it can also be used from your terminal!

The YAML-Path command line is installed via JBang. So, if you haven’t installed it yet, you need to install it:

curl -Ls https://sh.jbang.dev | bash -s - app setup

Note that you can find more information about how to install it in here.

Next, you need to register the YAMLPath jbang repository:

curl -Ls https://sh.jbang.dev | bash -s - app install --fresh --force yamlpath@yaml-path/jbang

And now, you can have fun with YAMLPath directly from your terminal!

Let’s see some examples about how to use it:

  • Help
> yamlpath --help
Usage: yamlpath [-hV] [-o=<output>] [-r=<replacement>] expression [file]
YAML-Path Expression Language Parser
      expression          YAMLPath expression
      [file]              YAML file
  -h, --help              Show this help message and exit.
  -o, --output=<output>   Sets the output file
  -r, --replace-with=<replacement>
                          Replace matching locations with this value
  -V, --version           Print version information and exit.
  • Find elements using YAMLPath expressions
> yamlpath "spec.selector.matchLabels.'app.kubernetes.io/name'" examples/test.yaml 
[example]

Where the first parameter is the YAMLPath expression and the second parameter is the YAML file (we can also specify a folder).

  • Find elements and replace with a supplied property
> yamlpath --replace-with="anotherValue" metadata.name examples/test.yaml 
---
apiVersion: v1
kind: Service
metadata:
  name: anotherValue
spec:
  ports:
    - name: http
      port: 80
      targetPort: 8080
  type: ClusterIP

In this example, the updated YAML resource will be printed into the standard output. If you want to write the output into a separated file, you can specify the location using the parameter --output:

> yamlpath --replace-with="anotherValue" --output=target/result.yaml metadata.name examples/test.yaml 
Output written in 'target/result.yaml'

Conclusion

YAMLPath is a very new project and it only supports the use cases I needed for developing the Quarkus Helm extensions:

  • Filters
  • Wildcards
  • Expressions with index
  • Literals
  • Replacements

If something is not supported and you think it fits perfect within YAMLPath, I would be happy to assist and collaborate :)

[ Java, YAMLPath ]