Awesome CLI

TL;DR

This post is kind of a sales pitch of using Command Line Interface in programmer’s everyday work. I list all the things I appreciate in using shell, all my favourite tools, tips and tricks related to console with some description why I consider those awesome. I also share my personal config so anyone can have the same setup on own machine in seconds - just take a look at section about dotfiles

Context

I don’t know what was your thought process that led you to becoming software engineer, however I remember the moment when I decided to become one. I was watching the Kung Fury and during the scene, where Hackerman is hacking the time I immediately understood that I want to be like this guy!

When working with other developers I often see that they reject to work with Command Line Interface or they use it only as a last resort. It is very suprising for me because I find the console to be very useful on daily basis - to be honest it’s hard to imagine my work with computer without it. Don’t get me wrong - I do not watch YT or play games inside the terminal! There are some tasks for which GUI is just irreplaceable, however I find that many objectives can be done in much more effective way when they are done through console, such as:

• Working with files (i. e. navigating system)
• Working with text
• Working with GIT (and other tools)
• Automating repeatable tasks

And there is something else…

:plus:1️⃣0️⃣ to respect when somebody looks at your fancy one-liners typed into hacker-like console! 😎

In this blog post I want to elaborate about awesome capabilities that modern CLI is offering us, emphasize how easy it is to setup and how much value it gives for programmers.

Post highly inspired by:

• awesome-shell
• modern-unix
• Julia Evans blog post

Must Have Configuration

So I started learning programming, I got to know that there is something like console and I saw this:

It would be an understatement to say that I was disappointed. What is the most frustrating in above view? It’s ugly. It looks bad. It discouraging. We want to feel something exactly opposite. We want to smile when we think about our terminal! We see something wrong? We know how to easily change it. We see a chance for improvement? We improve it and it stays with us forever.

🚀 Let’s make our console powerful 🚀

Terminal Emulator

Terminal Emulator is just an application which allow us using the shell but it gives us a lot of nice features. We can customize its appearance easily and have this feeling that console becomes interactive. I like how iTerm2 is describing itself:

iTerm2 brings the terminal into the modern age with features you never knew you always wanted.

Features they mean give interactive-like feeling of using console. You can easier choose the options of programs and output of those is much more readable as well. It’s usually highly customizable and have some fireworks 🎆 (very often not that useful 😄) - for example iTerm2 gives you possibility of going back in time or cool suggestions (but actually you can get those with zsh about which I write later).

Another interesting option which I found out during writing this post is Kitty which is much faster and even more customizable. Although I’m missing some features like hotkey window, I choose to use it because of its incredible speed and configuration as a file! Moreover, hotkeywindow can be set easily with usage of other software like hammerspoon .

The most what I like in using Terminal Emulator is easily customizable background, fonts, shourtcuts and this feeling of having interactive terminal.

Oh My Zsh!

Z shell (ZSH) is basically an extended bash. By using it we can achieve feeling of having even more interactive console. It’s higly customizable and extensible. The Oh My Zsh gathers many plugins, themes and helper functions. Things I like to use:

• Theme powerlevel10k which makes huge difference in how you feel in console. Your console will never be boring again!
• Plugins:
  • zsh-autosuggestions - real game changer, Just take a look!
  • zsh-syntax-highlighting - as showed above it shows on green command that can be successfully ended, but when you type something wrong, it screams immediately, so you’re not going too deep with typing this command!
  • thefuck - sometimes you make a mistake, however you don’t need to rewrite everyting!
  • git which saves many keystrokes by giving cool aliases and some useful functions
  • git-open - very useful plugin for jumping straight into github page by just typing git-open!
  • fzf-tab - apply fuzzy search to anything you use with fzf . Just by pressing tab you get suggestions and you can interactively search through them.

Useful links:

• https://medium.com/wearetheledger/oh-my-zsh-made-for-cli-lovers-installation-guide-3131ca5491fb
• https://github.com/unixorn/awesome-zsh-plugins

Tmux

It’s time for setting up Tmux - a Terminal Multiplexer. The idea behind terminal multiplexer is very simple - we need more space which should be easily manageable. Look how my terminal look like in time of writing this post:

In the upper tab, I have running hugo server - thanks to that I can see what this blog looks like immediately I edit the *.md file. However it’s not that important what it is printing, so it’s very small size. On the left I can modify context of post and on the right I can do some additional operations like copying the image that I’ll use in the post from one directory to another one. Switching between those panes are super easy (in my case it’s Control + B and then vim-like direction in which I want to go, so right is l, left is h etc.)

Because I’m a VIM addict I also can easily copy stuff from console by using vim-like controls.

The other cool thing is sessions. All this view can be saved for later so when I decide to work on something else, I just detach from the session and create new one. After I finish I can attach to the previous session and immediately start working on the previous thing!

Moreover, tmux is easily customizable just by changing .tmux.conf.

Awesome resources:

• https://github.com/rothgar/awesome-tmux

Navigating and searching

Last ‘must have’ in console for me is making navigation easy. For finding directory to which I want to go I use enhancd with fzf which is life changer! The result of simple cd command looks like that:

When I am searching for anything on filesystem I go with find <DIRECTORY_NAME> | fzf and it works awesome for single files, however for multiple files I prefer plain find, becuase I can easily pipe the output to other UNIX programs. For example:

find ~/Workshop -type f -name '*.json' | xargs -I {} cp {} /tmp/.

which copies all files that ends with .json to the /tmp directory.

To find files that have some content inside I like to use rg which has nice features - easy configuration of files I want to ignore and nice default display of output. Again - when I want to chain it with some other programs I rather go for simple grep or awk.

Useful links:

• https://pragmaticpineapple.com/four-useful-fzf-tricks-for-your-terminal/
• https://github.com/junegunn/fzf

Dotfiles

If you got to this point, you may see the sense of configuring all of this. Opening the console makes us happy, it opens new possibilities of doing things faster, smoother and in more ellegant way. But what if our computer crash, we work on few machines or we temporarily switched a computer? There has to be a way to restore our favourite set up in seconds! In this case dotfiles come with the rescue!

Most of tools I present here (if not all of them) have written configurations as files (usually those configurations are hidden, starting with ‘.’, that’s why those are named dotsfiles.

Idea is super simple - gather all of those files into git repo, symlink them into your system and whenever you update your configuration just do git push. Add small script which do all the linking and installing all the needed stuff, so you when you start working on new machine you can be done with only two commands:

git clone <LINK_TO_YOUR_DOTFILES_REPO>
bash install.sh

Working with text

At this moment we should smile when we see our terminal. It looks beautiful, it’s easy to use and we already look like hackers. That’s cool. Now it’s time to talk about becoming effective.

Awk

I think that it is the most underrated tool language on the world. When you know awk then you know how to process files. That’s all.

Quoting article awk is the coolest tool you dont know :

In short, awk is a domain-specific language which reads the kind of files you probably have a lot of already, then mutates them or computes something interesting from them.

The cool drawing done by Julia Evans describing awk in few words:

Of course it is much more powerful than only doing those simple things! There is a repository with solutions to Advent Of Code problems which shows how powerful it is, because I consider some of them pretty hard.

The game inside the terminal I mentioned at the beginning of this post is also written in AWK!

But don’t get me wrong, I do not encourage you to write everything in AWK and I think that those 2 examples above are extreme. I just want to point out that it’s incredibly useful when doing a text processing tasks.

Useful links:

• https://github.com/freznicek/awesome-awk
• https://github.com/freznicek/awk-crashcourse/blob/master/README.md - super cool diagrams showing how awk works

JQ (working with jsons)

I remember, that there was a time I didn’t like working with jsons… then I met jq 😅

The basic feature is just coloring json and formatting it:

However it also allow to query json in any way we want and transform to any form we want:

Cool thing is mixing jq with fzf-eval from awesome-fzf , so we can experiment with json and getting it know better :)

Longer transformations should be written to separate file - it’s especially useful when we transform one json format to another.

Useful link:

• https://stedolan.github.io/jq/manual/

Vim

I like VIM because of its super intuitive and expressive language. I would like to briefly go through my favourite things about VIM:

• Navigating the file. I find using h, j, k and l much easier to use than arrows, because of how close they are to other keys. Possibility of going easily to the end of line, beginning of line, to the next apperance of character or to previous appearance of character, jumping to the next and previous paragraph/sentence… But let’s be honest - it’s just something that every text editor should have.
• Vim language. I like to building sentences in vimish when editing the text. Cut inside the " would be ci" and I’m editing everything what is inside the quotes. Insert # 5 times would be 5i#. Isn’t that great?
• Registries. It’s not just about easy copying and pasting, but I can have multiple registries, so I can copy few things at once and keep it for as long as I want.
• Recordings. It’s surprising how often I need to perform the same action again and again. It’s cool that I can just do it once and then repeat it with few keys.
• How fast it is. Is there a 10000000 lines file to process? No problem, I’ll open that for you! And you even won’t notice big difference if it had 10 lines!.
• Searching and replacing patterns. It’s super easy to search and replace things. Moreover the syntax is very similar as using awk/sed.
• Bookmarks. Actually in vim it is called marker. Recently I use it much less in favor of IntelliJ bookmaks, however when working with plain text it is useful.

I also like to use plugin fzf-vim as I don’t have to quit vim to quickly jump to another file.

Must have, when using vim is autosave ! And everyone who lost his notes knew about it very well :)

Useful links:

• https://github.com/akrawchyk/awesome-vim#learning-vim

Tools + CLI = ❤️

Git

Working with git using CLI is super convenient for me. Especially with cool aliases that zsh git plugin provides. It’s very easy to:

• Add files to staging phase and reverse it
• Commit files
• Undo any changes (with use of reflog and git reset)
• Named stashes
• Pulling and fetching
• Moving between branches
• Pushing changes

However, for viewing diffs I use IntelliJ as it seems to be more effective tool.

There is also a git-open plugin worth using - with one command you are on the github ready for reviewing the same branch as in CLI.

Yup, it’s also easy to integrate with fzf, so you can find the command you want quickly…

Marker

Marker is a cool utility for keeping your one-liners as a bookmarks. Anytime you do an action that have a potential to be used later again, just press CONTROL + K and save its name. Later you can find it easily by clicking CONTROL + SPACE:

Boom

Boom is a CLI bookmark manager for your shell commands. It’s especially useful when used with fzf. For example boom all | fzf shows all bookmarks you have and you can interactively search for what you are interested in. Then with boom open <BOOKMARK_NAME> you can open immediately what you want.

Htop

So, your computer started burning and you don’t know why? Just launch htop and find out what is taking so many CPUs or memory! Then you can easily kill it with fzf-kill-processes or simple kill -9.

Many more

It’s hard to believe when you see how many CLI tools people created. It’s worth to go through those lists - even if you won’t find anything interesting. This experience of seeing how many things can be done with console in some way opens a mind for new possibilities. I had at least few little explosions 🤯 inside my head when I saw some of those ideas .

Automating repeatable tasks

Whenever something cumbersome, something that I need to repeat often appears then I like to automate it. I want to show you few examples from my current job.

Accessing often visited places

With have this kibana which I visit pretty often. Instead of clicking through browser, I just type in my console kibana <ENVIRONMENT> <NAME_OF_APP> and it opens immediately the right kibana in google chrome. The function is super simple:

function kibana() {
  env=$1
  service=$2
  if [[ "$env" == "prod" ]];
  then
    url="$LINK_TO_PROD_KIBANA/app/kibana/discover/{}"
  else
    url="$LINK_TO_TEST_KIBANA/app/kibana/discover/{}"
  fi

  cat ~/scripts/kibana-dictionary.txt \\
  | grep $service \\
  | grep $env \\
  | awk '{print $3}'\\
  | xargs -I {} zsh -cil "chrome $url"
}

…and ~/scripts/kibana-dictionary.txt is simple lookup file with each line of format:

APP_NAME ENVIRONMENT ID

Performing actions commonly done

In development process it seems to me that there is a lot of repeatable tasks. For example, I sometimes need to check if something works as expected after changes made on Allegro. Making a B2C or B2B order is a base for further actions. It’s smart to script it then.

I’ll be cheating a little bit, because I already had whole process of making those orders in form of groovy tests in gradle project, however I still think that this small script can be useful (don’t need to open the IntelliJ with this project and search for id I need):

function order() {
  local mode=$1
  local env=$2
  if [ "$mode" = "b2b" ]; then
    bash ~/scripts/allegro/make_b2b_order.sh $env
  elif [ "$mode" = "b2c" ]; then
    bash ~/scripts/allegro/make_b2c_order.sh $env
  else
    echo "Unknown mode: $mode. Allowed modes: [b2b, b2c]"
  fi
}

And subscript make_b2b_order.sh looks like that:

#!/bin/sh -e

env=$1

if [ -z $env  ];
then
  echo "Environment should be chosen [dev, test]"
  exit -1
fi

current_directory=$(pwd)

cd <PATH_TO_PROJECT_WITH_TESTS>
./gradlew test -Penv=$env --info --tests '<NAME_OF_TEST>' | tee /tmp/results
awk '$0 ~ /"orderId"/ {getline; gsub(/"/,""); print $2}' /tmp/results
cd $current_directory

As a result I get also the orderId as the last line of the script which can be chained to other scripts (like opening this order in browser or anything I want). Imagination is our only limit here.

Other tricks & tips

Some things that may not be that obvious or at least they were not obvious for me…

Keep your configurations in repo

You never know when you computer blows up. It’s better, when configurations and scripts on which you were working for years were on place. That’s why I have special repo with configs and to make it easy to modify those, I made symlinks pointing to this repo.

Scripting tips

Since I’m not a script master, things that make scripts easier to use are those, who validate parameters (at least if required parameters are on place) and inform about options user have. It can be as simple as below:

if [ -z $env  ];
then
  echo "Environment have to be chosen. Possible options: [dev, test]"
  exit -1
fi

More often than less it is beneficial to set up the -e flag at the beginning which will end the script whenever any command returns non 0 code (which should signal failure).

Bash isn’t the only way

Probably it’s obvious that there are other languages than bash to write shell scripts. Python is pretty popular one and it gives a lot of benefits like easier syntax to follow. However, if you value static typed languages, Kotlin is someting worth considering !