Run commands easily with kwt

Part 1: software stack, development, and continuous delivery.

bettercallshao
bettercallshao
Run commands easily with kwt

I recently improved kwt (github), a CLI utility for developers. I am writing down my thoughts on the projects in a blog series in hope that either my ideas or the project is interesting to others. I am not ready to talk about the motivation (which is a more natual place to start), therefore in part 1, I will start with the technical components.

What is kwt?

Kwt makes running commands easier, it helps developers avoid typing long commands over and over only changing some arguments in the middle of the long line. Kwt does it by reading from a repository of YAMLs of templated commands, so that the user can render and execute the commands with input arguments. By keeping command templates, kwt also encourages sharing and version control of commands.

For example, I use vim to edit code, and often I only remember the name of the files but not the full relative path. So I would first find the file then call vim.

vim $(find . -name Apple.java | head -1)

When I need to edit a different files, I would recall (Ctrl+R) the previous command, move my cursor to the file name, and replace it with the new file name. It works, but it is rather tedious, error prone, and doesn't meet the predictiveness I expect from my tools. Instead, I can write down a template for this command, and ask kwt to render an actual command on the go.

For example, save template as $HOME/.kwt/menus/developer-demo.yaml (github).

name: developer-demo
version: v0.1.0
help: Developer commands for demo
actions:
- name: find-vim
  help: Find the first file with given name in directory and edit it with vim
  template: vim $(find . -name '{{.target}}' | head -1)
  params:
  - name: target
    help: Target file name / directory
    value: README.md

Invoke it for Apple.java.

kwt developer-demo find-vim --target Apple.java

Kwt also automatically creates short hands for commands and arguments.

k d f -t Apple.java

For the frequently used, I go one step further and make an alias for it.

alias kdft=k d f -t
kdft Apple.java

Complex code can also be templated, for example this forex rate retriever written in Python.

name: python-demo
version: v0.1.0
help: Python commands for demo
actions:
- name: forex-rate
  help: Print forex rates
  template: |
    python3 -u -c "
    import urllib.request
    import urllib.parse
    import json
    url = 'https://api.exchangeratesapi.io/latest?base={{.base}}&symbols={{.symbols}}'
    r = json.load(urllib.request.urlopen(url))
    date = r['date']
    rates = r['rates']
    print(f'On {date}')
    print('\n'.join(
      f'1 {{.base}} can buy {rates[symbol]:.2f} {symbol}'
      for symbol in rates
    ))
    "
  params:
  - name: base
    help: Base currency
    value: USD
  - name: symbols
    help: Comma separated currency symbol list
    value: CAD,GBP

Run it for EUR to JPY and AUD.

$ kwt p f -b EUR -s JPY,AUD
On 2020-11-13
1 EUR can buy 123.88 JPY
1 EUR can buy 1.63 AUD

Templates can be shared and ingested from local files or over HTTP.

kwt i -s https://raw.githubusercontent.com/bettercallshao/kwt-menus/master/python-demo.yaml

A quick start guide is available on github.

What is kwtd?

Kwtd is a web frontend for kwt. It is functional, but the UI design is to be improved. Documentation is also to be completed, in the meantime, there is an outdated blog. Kwtd is a very important part of the system, and it took a lot of thought and development. Consequently, it also takes more effort to explain, and there is more work to be done.

License: MIT

This is a simple tool and I am aiming for impact especially in corporates, so MIT license is used for the least friction. I also wrote a blog on Free Software.

Versioning: tags

I used three numbers for versioning, e.g. v0.5.0, like semantic versions. I basically increment the third number for fixes, increment the second number for significant changes, and not incrementing the first number to 1 until I am sure it works well for others. Versions are denoted by git tags and injected into the binary at build time.

Merging: rebase

Everything on the main branch happens through a pull request that is rebase merged (unless I was asleep).

Language: Golang

Golang is chosen because it is easy to distribute, period. I have made Python utilities (e.g. qinvoke) in the past, and it was a pain to get other to run it with the right virtual environments and package versions. In some corporates, it is even a pain to install the Python interpreter itself. On the other hand, I had great experience installing kubectl and terraform, so Golang was chosen. I am aiming for the best installation experience, the binary just runs, no privilege escalation, no touching Windows registry.

Golang also surprised me with the ability to easily build for different architectures, which is good. I personally don't enjoy the $GOPATH part of the Golang philosophy, so Go module is a must.

Code: divided

Two binaries kwt and kwtd are defined in the cmd directory. Any component that can be defined and tested is put into pkg directory, namely alias, channel, cmd, exch, menu, msg, socket, and version. There are no tests for cmd directory, which is especially bad for kwtd since there is quite a bit of code there.

Building: make

Make (Makefile) is used to run the build. It is chosen because it is what others are using. I am honestly disappointed in our continuing dependency on make. Make is solid and reliable, but I was looking for some more modern and full-service options like npm, but found none. I could honestly just use npm even though this is not Javascript, but that breaks the simplicity. The buliding process for kwt is not the simpliest (downloading and embedding resources, dynamically generating a version string), and I didn't enjoy writing the Makefile. To build, run make with the default target. To run tests, use make test, which only builds the minimum necessary.

Release: GoReleaser Homebrew Scoop

GoReleaser (config) is used to build for different OS's and architectures, packaging the binaries, and distributing them on Homebrew (tap) and Scoop (bucket). Installation is made easy for users with access to brew (Linux & Mac) or scoop (Windows).

brew install bettercallshao/tap/kwt
scoop bucket add bettercallshao https://github.com/bettercallshao/scoop-bucket
scoop install bettercallshao/kwt

Delivery: CircleCI

CircleCI (config) runs tests for each commit, and runs the GoReleaser routine only for tags. CircleCI