Skip to main content

Quickstart

Welcome to the Holos Quickstart guide. Holos is an open source tool to manage software development platforms safely, easily, and consistently. We'll use Holos to manage a fictional bank's platform, the Bank of Holos. In doing so we'll take the time to explain the foundational concepts of Holos.

  1. Platform - Holos breaks a Platform down into Components owned by teams.
  2. Component - Components are CUE wrappers around unmodified upstream vendor Helm Charts, Kustomize Bases, or plain Kubernetes manifests.
  3. CUE - We write CUE to configure the platform. We'll cover the basics of CUE syntax and why Holos uses CUE.
  4. Tree Unification - CUE files are organized into a unified filesystem tree. We'll cover how unification makes it easier and safer for multiple teams to change the platform.

The Bank of Holos provides a good example of how Holos is designed to make it easier for multiple teams to deliver services on a platform. These teams are:

  • Platform
  • Software development
  • Security
  • Quality Assurance

Here's a screenshot of the retail banking application we'll build and deploy on our platform. We'll keep each of these teams in mind as we work through the guides. Each of our guides focuses on different aspects of delivering the Bank of Holos.

Bank of Holos

What you'll need​

This guide is intended to be informative without needing to run the commands. If you'd like to render the platform and apply the manifests to a real Cluster, complete the Local Cluster Guide before this guide.

You'll need the following tools installed to run the commands in this guide.

  1. holos - to build the Platform.
  2. helm - to render Holos Components that wrap Helm charts.
  3. kubectl - to render Holos Components that render with Kustomize.

Install Holos​

Start by installing the holos command line tool with the following command. If you don't have Go, refer to Installation to download the executable.

go install github.com/holos-run/holos/cmd/holos@latest
tip

Nearly all day-to-day platform management tasks use the holos command line tool to render plain Kubernetes manifests.

Fork the Git Repository​

Building a software development platform from scratch takes time so we've published an example for our guides. Fork the Bank of Holos to get started.

Clone the repository to your local machine.

# Change YourName
git clone https://github.com/YourName/bank-of-holos
cd bank-of-holos

Run the rest of the commands in this guide from the root of the repository.

Configure ArgoCD​

The Bank of Holos platform is organized as a collection of software components. Each component represents a piece of software provided by an upstream vendor, for example ArgoCD, or software developed in-house. Components are also used to glue together, or integrate, other components into the platform.

We'll start by changing the platform to point to our fork. We need to do this so we'll be able to see changes in ArgoCD as we make changes with GitOps.

package holos

#ArgoConfig: {
  Enabled: true
  RepoURL: "https://github.com/holos-run/bank-of-holos"
}

Change the RepoURL to the URL of your fork. For example:

diff --git a/projects/argocd-config.cue b/projects/argocd-config.cue
index 5264f48..0214e99 100644
--- a/projects/argocd-config.cue
+++ b/projects/argocd-config.cue
@@ -2,5 +2,5 @@ package holos

 #ArgoConfig: {
        Enabled: true
-       RepoURL: "https://github.com/holos-run/bank-of-holos"
+       RepoURL: "https://github.com/jeffmccune/bank-of-holos"
 }

We need to render the platform manifests after we make changes.

Render the Platform​

Platform rendering is is the process of looping over all the components in the platform and rendering each one into plain kubernetes manifest files. Holos is designed to write plain manifest files which can be applied to Kubernetes, but stops short of applying them so it's easier for team members to review and understand changes before they're made.

holos render platform ./platform

Rendering the platform to plain manifest files allows us to see the changes clearly. We can see this one line change affected dozens ArgoCD Application resources across the platform.

git status

Take a look at the Application resource for the bank-frontend component to see the changed spec.source.repoURL field.

git diff deploy/clusters/workload/gitops/bank-frontend.application.gen.yaml

We'll add, commit, and push this change to our fork then take a little time to explain what happened when we made the change and rendered the platform.

git add .
git commit -m 'quickstart: change argocd repo url to our fork'
git push origin

Platform Rendering Explained​

So what happens when we run holos render platform? We saw holos write plain manifest files, let's dive into how and why we implemented platform rendering like this.

Why do we render the platform?​

We built Holos to make the process of managing a platform safer, easier, and more consistent. Before Holos we used Helm, Kustomize, and scripts to glue together all of the software that goes into a platform. Then we coaxed the output of each tool into something that works with GitOps. This approach has a number of shortcomings. We wanted to see the manifests before ArgoCD or Flux applied them, so we wrote a lot of difficult to maintain scripts to get the template output into something useful. We tried avoiding the scripts by having ArgoCD handle the Helm charts directly, but we could no longer see the changes clearly during code review.

The platform rendering process allows us to have it both ways. We avoid the unsafe text templates and glue scripts by using CUE. We're able to review the exact changes that will be applied during code review because holos renders the whole platform to plain manifest files.

Finally, because we usually make each change by rendering the whole platform, we're able to see and consider how a single-line change, like the one we just made, affects the whole platform. Before we made Holos we were frustrated with how difficult it was to get this zoomed-out, broad perspective of each change we made.

How does platform rendering work?​

Holos is declarative. CUE provides resources that declare what holos needs to do. The output of holos is always the same for the same inputs, so holos is also idempotent.

When we run holos render platform, CUE builds the Platform specification (spec). This is a fancy way of saying a list of software to manage on each cluster in the platform. The CUE files in the platform directory provide the platform spec to holos.

Let's open up two of these CUE files to see how this works. Ignore the other files for now, they behave the same as these two.

package holos

// Manage the component on every cluster in the platform
for Fleet in #Fleets {
  for Cluster in Fleet.clusters {
    #Platform: Components: "\(Cluster.name)/argocd-crds": {
      path:    "projects/platform/components/argocd/crds"
      cluster: Cluster.name
    }
    #Platform: Components: "\(Cluster.name)/argocd": {
      path:    "projects/platform/components/argocd/argocd"
      cluster: Cluster.name
    }
  }
}

There's quite a few new concepts to unpack in these two CUE files.

  1. A Fleet is just a collection of clusters that share a similar, but not identical configuration. Most platforms have a management fleet with one cluster to manage the platform, and a workload fleet for clusters that host the services we deploy onto the platform.
  2. A Cluster is a Kubernetes cluster. Each component is rendered to plain manifests for a cluster.
important

On lines 6 and 10 we see a Component being assigned to the Platform. We also start to dive into the syntax of CUE, which we need to understand a little before going further.

In its simplest form, CUE looks a lot like JSON. This is because CUE is a superset of JSON. Or, put differently: all valid JSON is CUE1.

  1. C-style comments are allowed
  2. field names without special characters don't need to be quoted
  3. commas after a field are optional (and are usually omitted)
  4. commas after the final element of a list are allowed
  5. the outermost curly braces in a CUE file are optional

JSON objects are called structs in CUE. JSON arrays are called lists, Object members are called fields, which link their name, or label, to a value.

There are two important things to know about CUE to understand these two files. First, the curly braces have been omitted which is item 5 on the list above. Second, CUE is all about unification. These files could have been written like this:

package holos

// Manage the component on every cluster in the platform
for Fleet in #Fleets {
  for Cluster in Fleet.clusters {
    #Platform: {
      // Define #Platform.Components
      Components: {
        "\(Cluster.name)/argocd-crds": {
          path:    "projects/platform/components/argocd/crds"
          cluster: Cluster.name
        }
      }

      // Define #Platform.Components again!? Error?
      Components: {
        "\(Cluster.name)/argocd": {
          path:    "projects/platform/components/argocd/argocd"
          cluster: Cluster.name
        }
      }
    }
  }
}
important

Unlike most other languages, it is common to declare the same field in multiple places. CUE unifies the value of the field. We can think of CUE as a Configuration Unification Engine.

Now that we know curly braces can be omitted and values are unified, we can understand how the rest of the CUE files in the platform directory behave.

tip

Each CUE file in the platform directory adds components to the #Platform.Components struct.

The final file in the directory is responsible for producing the Platform spec. It looks like this.

package holos

#Platform.Output

This file provides the value of the #Platform.Output field, the platform spec, to holos.

Let's take a look at that Output value:

cue export --out yaml ./platform
tip

You don't normally need to execute cue, CUE is built into holos. We use it here to gain insight.

We see the platform spec is essentially a list of components, each assigned to a cluster.

important

Notice CUE unifies Components from multiple files into one list.

We'll see this unification behavior again and again. Unification is the defining characteristic of CUE that makes it a unique, powerful, and safe configuration language.

Holos takes this list of components and builds each one by executing:

holos render component --cluster-name="example" "path/to/the/component"

We can think of platform rendering as rendering a list of components, passing the cluster name each time. Rendering each component writes the fully rendered manifest for that component to the deploy/ directory, organized by cluster for GitOps.

Render a Component​

Rendering a component works much the same way as rendering a platform. holos uses CUE to produce a specification, then processes it. The specification of a component is called a BuildPlan. A BuildPlan is a list of zero or more kubernetes resources, Helm charts, Kustomize bases, and additional files to write into the deploy/ directory.

Let's take a look at the cert-manager component. Notice the platform/cert-manager.cue file has the field path: "projects/platform/components/cert-manager". This path indicates where to start working with the cert-manager component.

package holos

// Produce a helm chart build plan.
(#Helm & Chart).BuildPlan

let Chart = {
  Name:      "cert-manager"
  // #CertManager is defined in projects/cert-manager.cue
  Version:   #CertManager.Version
  Namespace: #CertManager.Namespace

  Repo: name: "jetstack"
  Repo: url:  "https://charts.jetstack.io"

  // CUE offers type checking and validation of Helm values.
  Values: installCRDs: true
  Values: startupapicheck: enabled: false
}

This file introduces a few new concepts.

  1. Line 4 indicates this component produces a BuildPlan that wraps a Helm Chart.
  2. On line 6 let binds a name to an expression for the current scope. The current file in this case.
  3. Notice Chart is referenced on line 4 before it's bound on line 6. Order is irrelevant in CUE. Complex changes are simpler and easier when we don't have to think about order.
  4. The chart version and namespace are defined in a different file closer to the root, projects/cert-manager.cue
  5. We define Helm values in CUE to take advantage of strong type checking and manage multiple Helm charts consistently with platform wide values.

Let's take a look at the BuildPlan this CUE configuration defines.

cue export --out yaml ./projects/platform/components/cert-manager
important

Again, you don't normally need to execute cue, it's built into holos. We use it here to show how Holos works with Helm.

Looking at the BuildPlan, we see holos will render the Helm chart into the deploy directory along with an ArgoCD Application resource in the gitops/ directory.

tip

The BuildPlan API is flexible enough to write any file into the deploy/ directory. Holos uses this flexibility to support both Flux and ArgoCD.

When we run cue export, we get back a Core API BuildPlan. The BuildPlan is produced by the #Helm definition on line 4 which is part of the Author API. The Core API is the contract between CUE and holos. As such, it's not as friendly as the Author API. The Author API is the contract component authors and platform engineers use to configure and manage the platform. The Author API is meant for people, the Core API is meant for machines. This explains why we see quite a few fields in the exported BuildPlan we didn't cover in this guide. Day to day we don't need to be concerned with those fields because the Author API handles them for us.

tip

Our intent is to provide an ergonomic way to manage the platform with the Author API.

When the Author API doesn't offer a path forward, authors may use the Core API directly from CUE. We can think of the Core API as an escape hatch for the Author API. We'll see some examples of this in action in the more advanced guides.

Next Steps​

Thank you for finishing the Quickstart guide. Dive deeper with the next guide on how to Deploy a Service which explains how to take one of your existing Helm charts or Deployments and manage it with Holos.

Footnotes​

  1. The Basics of CUE ↩