Unix Reference

Kip Landergren

(Updated: )

My cheat sheet for working with Unix-like operating systems covering utilities, shell builtins, and frequently asked questions.

Contents

Unix Utilities

Note: the following refer to BSD variants unless otherwise noted.

echo

Write arguments to standard output.

Examples

$ echo 'hello world'
hello world
$

$ echo -n 'hello world'
hello world$

$ echo 'hello\nworld'
hello\nworld
$ echo -e 'hello\nworld'
hello
world

file

Determine file type.

% file foo
foo: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=redacted, stripped

find

find [options] [path ...] [expression]

Walk a file hierarchy specified by one or more path arguments. Recursively descend each path and evaluate expression on each file. The evaluation of the expression typically determines whether or not to print the file.

An expression is composed of primaries and operators.

Gotchas

Options
-X for use with xargs; warns and skips files containing delimiting characters

Primaries
-print always evaluates to true, printing the pathname of the current file to stdout. If none of -exec, -ls, -print, -print0, or -ok is specified, the given expression shall be effectively replaced by ( given expression ) -print.
-print0 always evaluates to true, printing the pathname of the current file to stdout followed by an ASCII NUL character (character code 0). For use with xargs -0.
-prune causes find to not descend into the current file; always evaluates to true
-type t true if the file is of type t. file type options: - b - block special - c - character special - d - directory - f - file - l - symbolic link - p - FIFO - s - socket
-depth n true if the depth of the file relative to the starting point is equal to n

Operators
-a / -and the logical AND operator
-o / -or the logical OR operator

Examples

Grouping Multiple Conditions
$ find . -type f \
       -and \( \
         -name '*.html' \
         -or -name '*.css' \
         -or -name '*.ico' \
         -or -name '*.map' \
         -or -name '*.js' \
         -or -name '*.json' \
         -or -name '*.xml' \
         -or -name '*.txt' \
         -or -name '*.svg' \
       \)
Printing files

Print all files in the current directory’s hierarchy, excluding excluded-path and every file underneath excluded-path:

$ find . -path ./excluded-path -prune -or -print

Breakdown:

grep

file pattern searcher

Examples

Recursively search all subdirectories and files:

$ grep -r 'pattern' /path/to/dir

gzip, gunzip, zcat, gzcat

less

View text file contents, paginated, without reading the whole file.

Commands
h display a help summary of less commands
q exit; quit
e or j forward 1 line
y or k backward 1 line
f forward 1 window
b backward 1 window
d forward 1 half-window
u backward 1 half-window
/pattern Search forward for matching line
?pattern Search backward for matching line
&pattern filter file for lines matching pattern; supply empty pattern and hit ↵ Enter to exit
n next match
N previous match

Frequently Asked Questions

How to exit &pattern?

ln

make links

Best Practices

Example:

$ ln -s ../path/to/foo foo

ls

List directory contents.

Examples

Default:

$ ls
my-directory		my-fifo			my-symbolic_link
my-executable		my-normal-file

List contents, distinguishing each by special trailing character:

$ ls -F
my-directory/		my-fifo|		my-symbolic_link@
my-executable*		my-normal-file

List contents, distinguishing each by special trailing character, each on its own line:

$ ls -F1
my-directory/
my-executable*
my-fifo|
my-normal-file
my-symbolic-link@

man

Format and display the system’s on-line manual pages.

Defaults
pager less -is

Manual Sections
1 User Commands
2 System Calls
3 C Library Functions
4 Devices and Special Files
5 File Formats and Conventions
6 Games et al.
7 Miscellanea
8 System Administration tools and Deamons

Examples

Search the system’s whatis database for strings matching the provided pattern:

$ man -k '^intro'
intro(1)                 - introduction to general commands (tools and utilities)
intro(2)                 - introduction to system calls and error numbers
intro(3)                 - introduction to the C libraries
intro(5)                 - introduction to file formats
intro(7)                 - miscellaneous information pages
intro(8)                 - introduction to system maintenance procedures and commands
intro(9)                 - introduction to system kernel interfaces
$ man intro
$ man 1 intro
$ man 2 intro

Frequently Asked Questions

What do the numbers mean after unix commands? What does the (1) mean in git(1)?

The number refers to the section of the manual, described above. In the case of git(1), the (1) indicates that this refers to git the user command. This disambiguation is useful when a string of characters has multiple meanings (e.g. printf).

What happens when I type man git?

Depending on your terminal, shell, and operating system:

How to navigate to links in a man page? How to follow “See Also” links?

When using less as your pager this is not possible. Other man page viewers allow this.

mktemp

Make temporary filenames or directory names.

Examples

Make temporary file:

$ echo $TMPDIR
/tmp
$ mktemp
/tmp/tmp.bN0khCLV
$ mktemp -t some-prefix
/tmp/some-prefix.zDu8khd0
$ mktemp template-string.X
template-string.n
$ mktemp template-string.XX
template-string.yK
$ mktemp template-string.XXX
template-string.BS2
$ mktemp template-string.XXX template-string.XXX
template-string.k0J
template-string.t92

Make temporary directory:

$ echo $TMPDIR
/tmp
$ mktemp -d
/tmp/tmp.p3BbpeXj
$ mktemp -d -t some-prefix
/tmp/some-prefix.6z40Wn1h
$ mktemp -d relative-to-current-directory.XXX
relative-to-current-directory.lQM

Make temporary directory and change directory to it:

$ cd $(mktemp -d relative-to-current-directory.XXX)

rm

rm -rf -- /path/to/delete

Note: the -- is intentional and demarcates the end of options.

sha1sum

Comput and check SHA1 message digest.

Examples

$ echo -n 'hello world' | sha1sum
2aae6c35c94fcfb415dbe95f408b9ce91ee846ed  -

$ echo -n 'hello world' > hello-world.txt
$ sha1sum hello-world.txt
2aae6c35c94fcfb415dbe95f408b9ce91ee846ed  hello-world.txt

stat

Display file status.

Examples

Display verbose file information:

$ stat -x foo
  File: "foo"
  Size: 0            FileType: Regular File
  Mode: (0644/-rw-r--r--)         Uid: (  501/klandergren)  Gid: (   20/   staff)
Device: 1,4   Inode: 8692341193    Links: 1
Access: Fri Jun 25 11:21:52 2021
Modify: Fri Jun 25 11:21:52 2021
Change: Fri Jun 25 11:21:55 2021

tail

display the last part of a file

Examples

Display all but the first line (skip first line; skip n lines):

$ tail -n +1 file.txt

tar

create and manipulate streaming archive files

Examples

Compress my-directory to my-archive.tar:

tar -cvf my-archive.tar my-directory

Uncompress my-archive.tar, changing directory to my-directory first (creating my-directory/my-archive, with caveats that you should review the man page for behavior when my-directory/my-archive already exists):

tar -xvf my-archive.tar -C my-directory

wc

word, line, character, and byte count

Examples

Count the number of lines:

$ cat foo | wc -l
      42
$ wc -l foo
      42 foo
$ wc -l foo bar
      42 foo
      42 bar
      84 total

xargs

construct argument list(s) and execute utility

General Structure

xargs -I {} -t sh -c '[COMMAND USING $1]' sh {}

Gotchas

Selected Options
-I replstr replace one or more occurrences of replstr
-t echo command to be executed before executing

Examples

$ find . -type f | xargs -I {} -t sh -c 'echo "$1"' sh {}
sh -c echo "$1" sh ./foo.txt
./foo.txt
sh -c echo "$1" sh ./bar.txt
./bar.txt

$ find . -type f | xargs -I {} -t sh -c 'echo "$1"; echo "$1"' sh {}
sh -c echo "$1"; echo "$1" sh ./foo.txt
./foo.txt
./foo.txt
sh -c echo "$1"; echo "$1" sh ./bar.txt
./bar.txt
./bar.txt

Other Programs

rsync

faster, flexible replacement for rcp

Examples

delete files at destination:

$ rsync --verbose --archive --human-readable path_src/ path_dest/ --delete

tree

List contents of directories in a tree-like format.

Examples

preferred invocation:

$ tree -a -F --noreport
.
├── .hidden-file
├── bar/
│   ├── bar.a
│   └── baz/
│       └── baz.a
└── foo/
    └── foo.a

default invocation:

$ tree
.
├── bar
│   ├── bar.a
│   └── baz
│       └── baz.a
└── foo
    └── foo.a

3 directories, 3 files

control depth:

$ tree -a --noreport -L 1
.
├── .hidden-file
├── bar
└── foo

exclude files that match multiple patterns:

$ tree -a -F --noreport -I 'baz|*.a'
.
├── .hidden-file
├── bar/
└── foo/

Shell builtins

type

Determine interpretation of text if the text is used as a command name.

Examples

$ type type
type is a shell builtin

$ type -a type
type is a shell builtin
type is /usr/bin/type

true

Always returns a 0 exit code.

Shell Techniques

Redirection Operators

Non-exhaustive. Full list available in the bash manual chapter for redirections.
> redirect output
>> append output
< redirect input

Process Substitution

Piping the output of multiple processes:

$ diff <(echo -e 'hello\nworld') <(echo -e 'goodbye\nworld')
1c1
< hello
---
> goodbye

Exit Codes
0 success
1 error

Beware! This is not the same as boolean types:
0 false
1 true

Signals
HUP
SIGINT
SIGHUP

Frequently Asked Questions (FAQ)

When to use which vs. type?

Prefer using type as it is defined in the POSIX specification and which is not. Additionally, prefer type -a so that you may distinguish all:

$ type -a type
type is a shell builtin
type is /usr/bin/type

How to view the man page for shell builtins?

For bash:

$ man bash

or:

$ help builtin

When to use echo vs. printf?

Use echo when:

Use printf when:

Why do some command examples have a --?

From bash(1) man page:

A -- signals the end of options and disables further option processing. Any arguments after the -- are treated as filenames and arguments. An argument of - is equivalent to --.

This protects against directories named with a leading dash being interpreted as options.

From the zsh User’s Guide with the following example:

print -- '$foo is "'$foo'"'

First, why did I put -- after the print? That's because print, like many UNIX commands, can take options after it which begin with a -. -- says that there are no more options; so if what you're trying to print begins with a -, it will still print out. Actually, in this case you can see it doesn't, so you're safe; but it's a good habit to get into, and I wish I had.