By Shaoqing Tan in kwt — Nov 15, 2020

KWT Unveiled: Streamlining CLI Workflows with Modern Dev Practices (Part 1)

I recently improved kwt (GitHub), a CLI utility for developers. I'm writing a blog series to share my thoughts on the project, hoping that either my ideas or the project itself will interest others. While I'm not yet ready to discuss the motivation (which would be a more natural starting point), in part 1, I'll focus on the technical components.

What is kwt?

Kwt makes running commands easier by helping developers avoid typing long commands repeatedly with only minor changes to arguments. It does this by reading from a repository of YAML files containing templated commands, allowing users to render and execute these commands with input arguments. By maintaining command templates, kwt also encourages sharing and version control of commands.

For example, I use vim to edit code, and often I remember only the file name, not the full relative path. Typically, I would first use find to locate the file, then call vim:

vim $(find . -name Apple.java | head -1)

When I need to edit different files, I would recall (Ctrl+R) the previous command, move my cursor to the file name, and replace it with the new one. While this works, it's tedious, error-prone, and doesn't meet the predictiveness I expect from my tools. Instead, I can write a template for this command and use kwt to render an actual command on the fly.

For instance, save this template as $HOME/.kwt/menus/developer-demo.yaml (GitHub):

name: developer-demo
version: v0.1.0
help: Developer commands for demo
actions:
- name: find-vim
  help: Find the first file with given name in directory and edit it with vim
  template: vim $(find . -name '{{.target}}' | head -1)
  params:
  - name: target
    help: Target file name / directory
    value: README.md

Invoke it for Apple.java:

kwt developer-demo find-vim --target Apple.java

Kwt also automatically creates shorthands for commands and arguments:

k d f -t Apple.java

For frequently used commands, I go one step further and create an alias:

alias kdft=k d f -t
kdft Apple.java

Complex code can also be templated. Here's an example of a forex rate retriever written in Python:

name: python-demo
version: v0.1.0
help: Python commands for demo
actions:
- name: forex-rate
  help: Print forex rates
  template: |
    python3 -u -c "
    import urllib.request
    import urllib.parse
    import json
    url = 'https://api.exchangeratesapi.io/latest?base={{.base}}&symbols={{.symbols}}'
    r = json.load(urllib.request.urlopen(url))
    date = r['date']
    rates = r['rates']
    print(f'On {date}')
    print('\n'.join(
      f'1 {{.base}} can buy {rates[symbol]:.2f} {symbol}'
      for symbol in rates
    ))
    "
  params:
  - name: base
    help: Base currency
    value: USD
  - name: symbols
    help: Comma separated currency symbol list
    value: CAD,GBP

Run it for EUR to JPY and AUD:

$ kwt p f -b EUR -s JPY,AUD
On 2020-11-13
1 EUR can buy 123.88 JPY
1 EUR can buy 1.63 AUD

Templates can be shared and ingested from local files or over HTTP:

kwt i -s https://raw.githubusercontent.com/bettercallshao/kwt-menus/master/python-demo.yaml

A quick start guide is available on GitHub.

What is kwtd?

Kwtd is a web frontend for kwt. While functional, its UI design needs improvement. Documentation is also incomplete; in the meantime, there's an outdated blog post available. Kwtd is a crucial part of the system and required significant thought and development. Consequently, it takes more effort to explain, and there's more work to be done.

Technical Decisions

License: MIT

As a simple tool aimed for impact, especially in corporate environments, the MIT license was chosen to minimize friction. I've also written a blog post on Free Software.

Versioning: Tags

I use three-number versioning (e.g., v0.5.0), similar to semantic versioning. The third number is incremented for fixes, the second for significant changes, and the first number will reach 1 when I'm confident it works well for others. Versions are denoted by git tags and injected into the binary at build time.

Merging: Rebase

All changes to the main branch occur through pull requests that are rebase merged (unless I was asleep).

Language: Golang

Golang was chosen primarily for its ease of distribution. Past experiences with Python utilities (e.g., qinvoke) revealed difficulties in getting others to run them with the correct virtual environments and package versions. In some corporate environments, even installing the Python interpreter can be challenging. Positive experiences with kubectl and terraform led to the choice of Golang. The goal is to provide the best installation experience: the binary just runs, without privilege escalation or touching the Windows registry.

Golang also surprised me with its ability to easily build for different architectures. While I personally don't enjoy the $GOPATH aspect of Golang philosophy, Go modules are a must.

Code Structure

Two binaries, kwt and kwtd, are defined in the cmd directory. Components that can be defined and tested are placed in the pkg directory, including alias, channel, cmd, exch, menu, msg, socket, and version. There are no tests for the cmd directory, which is particularly problematic for kwtd due to its significant codebase.

Building: Make

Make (Makefile) is used for building. It was chosen due to its widespread use, although I'm somewhat disappointed by our continued reliance on it. While Make is solid and reliable, I was looking for more modern, full-service options like npm but found none suitable. The building process for kwt is not the simplest (downloading and embedding resources, dynamically generating a version string), and writing the Makefile wasn't enjoyable. To build, run make with the default target. For tests, use make test, which only builds the minimum necessary components.

Release: GoReleaser, Homebrew, Scoop

GoReleaser (config) is used to build for different OS's and architectures, package the binaries, and distribute them on Homebrew (tap) and Scoop (bucket). This makes installation easy for users with access to brew (Linux & Mac) or scoop (Windows):

brew install bettercallshao/tap/kwt

scoop bucket add bettercallshao https://github.com/bettercallshao/scoop-bucket
scoop install bettercallshao/kwt

Delivery: CircleCI

CircleCI (config) runs tests for each commit and executes the GoReleaser routine only for tags.