Building Patterns

How to write and build a simple Pattern.
Categories:

In this tutorial, we’ll write a simple Pattern and build it using Etcha.

Requirements

  • Docker or Podman (we’ll use Docker here, but this should work with Podman, too)
  • Access to pull down Etcha from GitHub’s container registry (ghcr.io)
  • A text editor

Tutorial

Prepare Our Environment

  1. Open a local, empty directory in a shell like bash where we can read/write files and mount them into a container.
  2. Create a temporary bash alias for Etcha so we can use it:
alias etcha='docker run -u $(id -u):$(id -g) --rm -v $(pwd):/work -w /work ghcr.io/candiddev/etcha:latest'
  1. Initialize a new directory with Etcha:
etcha init .
  1. This command created a few new things, run ls or tree to examine the new files, it should look something like this:
$ tree
.
├── etcha
├── lib
│   └── etcha
│       ├── aptKey.libsonnet
│       ├── apt.libsonnet
│       ├── copy.libsonnet
│       ├── dir.libsonnet
│       ├── file.libsonnet
│       ├── mount.libsonnet
│       ├── native.libsonnet
│       ├── password.libsonnet
│       ├── symlink.libsonnet
│       └── systemdUnit.libsonnet
├── patterns
└── README.md

5 directories, 11 files
  1. Lets inspect one of the Libraries, native.libsonnet. We’re going to use this in our Pattern, so we should understand what it’s doing:
$ cat lib/etcha/native.libsonnet
// Generated by Etcha dev+latest

{
  getConfig(): std.native('getConfig')(),
  getEnv(key): std.native('getEnv')(key),
  getPath(path, fallback=null): std.native('getPath')(path, fallback),
  getRecord(type, name, fallback=null): std.native('getRecord')(type, name, fallback),
  randString(length): std.native('randString')(length),
  regexMatch(regex, string): std.native('regexMatch')(regex, string),
}
  1. It’s a lot of Jsonnet code, but it is basically a bunch of helper functions called Native Functions

Writing a Pattern

  1. Lets write a new Pattern using a few of the Etcha libraries. Create a new file under patterns called helloworld.jsonnet. Add in this content:
local n = import '../lib/etcha/native.libsonnet';

{
  build: [
    {
      always: true,
      change: 'echo %s > /work/helloworld' % n.getEnv('HOSTNAME'),
      id: 'write a file',
      onChange: [
        'read file',
      ],
    },
    {
      change: 'cat /work/helloworld',
      id: 'read file',
      onChange: [
        'etcha:buildManifest',
      ],
    },
  ],
  buildExec: {
    command: '/bin/sh -c'
  },
}

This simple Pattern, built using etcha build, will always create a file, helloworld, with the current environment hostname, and read the contents of it when it changes, sending the output to the event buildManifest.

  1. We can view the rendered Pattern by running etcha render
$ etcha render patterns/helloworld.jsonnet
{
  "audience": null,
  "build": [
    {
      "always": true,
      "change": "echo 95a2e5bdb531 \u003e /work/helloworld",
      "envPrefix": "",
      "id": "write a file",
      "onChange": [
        "read file"
      ]
    },
    {
      "change": "cat /work/helloworld",
      "envPrefix": "",
      "id": "read file",
      "onChange": [
        "etcha:buildManifest"
      ]
    }
  ],
  "exec": {
    "allowOverride": false,
    "command": "/bin/sh -c",
    "containerEntrypoint": "",
    "containerImage": "",
    "containerPrivileged": false,
    "containerPull": "",
    "containerUser": "",
    "containerVolumes": null,
    "env": null,
    "envInherit": false,
    "group": "",
    "user": "",
    "workDir": ""
  },
  "expiresInSec": 0,
  "id": "",
  "issuer": "",
  "run": null,
  "runEnv": {},
  "subject": ""
}

We can filter to just see the build config using etcha jq:

$ etcha render patterns/helloworld.jsonnet | etcha jq -r '.build'
[
  {
    "always": true,
    "change": "echo e9b3bf4daba6 \u003e /work/helloworld",
    "envPrefix": "",
    "id": "write a file",
    "onChange": [
      "read file"
    ]
  },
  {
    "change": "cat /work/helloworld",
    "envPrefix": "",
    "id": "read file",
    "onChange": [
      "etcha:buildManifest"
    ]
  }
]
  1. Lets lint this pattern to make sure it’s correct. We need to disable the external build linter, shellcheck, because we’re running in a container:

$ etcha -x build_linters=null lint patterns/helloworld.jsonnet yes

ERROR shared/go/jsonnet/fmt.go:36
files not formatted properly
diff have /patterns/helloworld.jsonnet want /patterns/helloworld.jsonnet
--- have /patterns/helloworld.jsonnet
+++ want /patterns/helloworld.jsonnet
@@ -19,6 +19,6 @@
     },
   ],
   buildExec: {
-    command: '/bin/sh -c'
+    command: '/bin/sh -c',
   },
 }

ERROR candiddev/etcha/go/lint.go:23
found linting errors
patterns/helloworld.jsonnet:
        files not formatted properly
  1. Oh no, our Pattern has invalid formatting. Looks like we’re missing some commas at the end, lets add those and run Lint again:
$ etcha -x build_linters=null lint patterns/helloworld.jsonnet yes
$ $?
0

Phew, the day is saved!

Building a Pattern

  1. We’re almost ready to build. Lets generate a signing key to use:
$ etcha gen-keys sign-verify
{
  "privateKey": "ed25519private:MC4CAQAwBQYDK2VwBCIEIBq+BhDRYk8OJv1ksMwKtf0td5p3FGwypXq96gHKefGS:reqYEklgP4",
  "publicKey": "ed25519public:MCowBQYDK2VwAyEAw7eTEuEH0+TfgtX3zB+JZVnYD0eskY6qn3n7ZCA7wWM=:reqYEklgP4"
}

You can generate your own keys or use the ones above, please don’t ever use these outside of this tutorial!!!

  1. We’ll use the privateKey to build our pattern:
$ etcha -x build_signingKey=ed25519private:MC4CAQAwBQYDK2VwBCIEIBq+BhDRYk8OJv1ksMwKtf0td5p3FGwypXq96gHKefGS:reqYEklgP4 \
    build patterns/helloworld.jsonnet helloworld.jwt
INFO  Always changed write a file [1s]
INFO  Triggered read file via write a file [100ms]

Etcha ran all of our build commands successfully and outputted a JWT in our current directory, helloworld.jwt

  1. Lets inspect the JWT, using the public key we generated for verification:
$ etcha -x run_verifyKeys='["ed25519public:MCowBQYDK2VwAyEAw7eTEuEH0+TfgtX3zB+JZVnYD0eskY6qn3n7ZCA7wWM=:reqYEklgP4"]' \
    jwt helloworld.jwt 
{
  "etchaBuildManifest": "3cb2dfd2ed93",
  "etchaPattern": {
    "entrypoint": "/patterns/helloworld.jsonnet",
    "files": {
      "/lib/etcha/native.libsonnet": "// Generated by Etcha dev+latest\n\n{\n  getConfig(): std.native('getConfig')(),\n  getEnv(key): std.native('getEnv')(key),\n  getPath(path, fallback=null): std.native('getPath')(path, fallback),\n  getRecord(type, name, fallback=null): std.native('getRecord')(type, name, fallback),\n  randString(length): std.native('randString')(length),\n  regexMatch(regex, string): std.native('regexMatch')(regex, string),\n}\n",
      "/patterns/helloworld.jsonnet": "local n = import '../lib/etcha/native.libsonnet';\n\n{\n  build: [\n    {\n      always: true,\n      change: 'echo %s \u003e /work/helloworld' % n.getEnv('HOSTNAME'),\n      id: 'write a file',\n      onChange: [\n        'read file',\n      ],\n    },\n    {\n      change: 'cat /work/helloworld',\n      id: 'read file',\n      onChange: [\n        'etcha:buildManifest',\n      ],\n    },\n  ],\n  buildExec: {\n    command: '/bin/sh -c'\n  },\n}\n"
    }
  },
  "etchaRunEnv": {},
  "etchaVersion": "dev+latest",
  "iat": 1697217289,
  "nbf": 1697217289
}

Etcha built our JWT successfully. It included the output of the buildManifest event in the etchaBuildManifest property, too!

  1. Just for grins, lets try viewing the JWT with a different public key:
$ etcha -x run_verifyKeys='["ed25519public:MCowBQYDK2VwAyEAE6mgQIQNiQM9WA9lX93PQcZIYGJevHp3xxyoxVEfOl8=:zbUeB0b0ni"]' \
    jwt helloworld.jwt
{
  "etchaBuildManifest": "3cb2dfd2ed93",
  "etchaPattern": {
    "entrypoint": "/patterns/helloworld.jsonnet",
    "files": {
      "/lib/etcha/native.libsonnet": "// Generated by Etcha dev+latest\n\n{\n  getConfig(): std.native('getConfig')(),\n  getEnv(key): std.native('getEnv')(key),\n  getPath(path, fallback=null): std.native('getPath')(path, fallback),\n  getRecord(type, name, fallback=null): std.native('getRecord')(type, name, fallback),\n  randString(length): std.native('randString')(length),\n  regexMatch(regex, string): std.native('regexMatch')(regex, string),\n}\n",
      "/patterns/helloworld.jsonnet": "local n = import '../lib/etcha/native.libsonnet';\n\n{\n  build: [\n    {\n      always: true,\n      change: 'echo %s \u003e /work/helloworld' % n.getEnv('HOSTNAME'),\n      id: 'write a file',\n      onChange: [\n        'read file',\n      ],\n    },\n    {\n      change: 'cat /work/helloworld',\n      id: 'read file',\n      onChange: [\n        'etcha:buildManifest',\n      ],\n    },\n  ],\n  buildExec: {\n    command: '/bin/sh -c'\n  },\n}\n"
    }
  },
  "etchaRunEnv": {},
  "etchaVersion": "dev+latest",
  "iat": 1697217289,
  "nbf": 1697217289
}

Etcha return the JWT contents, but it definitely didn’t like the signature! During normal operation, Etcha would’ve ignored this JWT.

You can now remove the current directory if you want.

Summary

We’ve successfully written, linted, and built our Pattern. Next, we’ll look at pushing and pulling Patterns.