100 Useful Command-Line Utilities
by Oliver; 201435. ps, pstree, jobs, bg, fg, kill, top, htop
Borrowing from An Introduction to the Command-Line (on Unix-like systems) - Processes and Running Processes in the Background: Processes are a deep subject intertwined with topics like memory usage, threads, scheduling, and parallelization. My understanding of this stuff—which encroaches on the domain of operating systems—is narrow, but here are the basics from a unix-eye-view. The command ps displays information about your processes:$ ps # show process info $ ps -f # show verbose process infoWhat if you have a program that takes some time to run? You run the program on the command line but, while it's running, your hands are tied. You can't do any more computing until it's finished. Or can you? You can—by running your program in the background. Let's look at the command sleep which pauses for an amount of time specified in seconds. To run things in the background, use an ampersand:
$ sleep 60 & [1] 6846 $ sleep 30 & [2] 6847The numbers printed are the PIDs or process IDs, which are unique identifiers for every process running on our computer. Look at our processes:
$ ps -f UID PID PPID C STIME TTY TIME CMD 501 5166 5165 0 Wed12PM ttys008 0:00.20 -bash 501 6846 5166 0 10:57PM ttys008 0:00.01 sleep 60 501 6847 5166 0 10:57PM ttys008 0:00.00 sleep 30The header is almost self-explanatory, but what's TTY? Wikipedia tells us that this is an anachronistic name for the terminal that derives from TeleTYpewriter. Observe that bash itself is a process which is running. We also see that this particular bash—PID of 5166—is the parent process of both of our sleep commands (PPID stands for parent process ID). Like directories, processes are hierarchical. Every directory, except for the root directory, has a parent directory and so every process, except for the first process—init in unix or launchd on the Mac—has a parent process. Just as tree shows us the directory hierarchy, pstree shows us the process hierarchy:
$ pstree -+= 00001 root /sbin/launchd |-+= 00132 oliver /sbin/launchd | |-+= 00252 oliver /Applications/Utilities/Terminal.app/Contents/MacOS/Terminal | | \-+= 05165 root login -pf oliver | | \-+= 05166 oliver -bash | | |--= 06846 oliver sleep 30 | | |--= 06847 oliver sleep 60 | | \-+= 06848 oliver pstree | | \--- 06849 root ps -axwwo user,pid,ppid,pgid,commandThis command actually shows every single process on the computer, but I've cheated a bit and cut out everything but the processes we're looking at. We can see the parent-child relationships: sleep and pstree are children of bash, which is a child of login, which is a child of Terminal, and so on. With vanilla ps, every process we saw began in a terminal—that's why it had a TTY ID. However, we're probably running Safari or Chrome. How do we see these processes as well as the myriad others running on our system? To see all processes, not just those spawned by terminals, use the -A flag:
$ ps -A # show all process info $ ps -fA # show all process info (verbose)Returning to our example with sleep, we can see the jobs we're currently running with the command jobs:
$ jobs [1]- Running sleep 60 & [2]+ Running sleep 30 &What if we run our sleep job in the foreground, but it's holding things up and we want to move it to the background? Cntrl-z will pause or stop a job and the commands bg and fg set it running in either the background or the foreground.
$ sleep 60 # pause job with control-z ^Z [1]+ Stopped sleep 60 $ jobs # see jobs [1]+ Stopped sleep 60 $ bg # set running in background [1]+ sleep 60 & $ jobs # see jobs [1]+ Running sleep 60 & $ fg # bring job from background to foreground sleep 60We can also kill a job:
$ sleep 60 & [1] 7161 $ jobs [1]+ Running sleep 60 & $ kill %1 $ [1]+ Terminated: 15 sleep 60In this notation %n refers to the nth job. Recall we can also kill a job in the foreground with Cntrl-c. More generally, if we didn't submit the command ourselves, we can kill any process on our computer using the PID. Suppose we want to kill the terminal in which we are working. Let's grep for it:
$ ps -Af | grep Terminal 501 252 132 0 11:02PM ?? 0:01.66 /Applications/Utilities/Terminal.app/Contents/MacOS/Terminal 501 1653 1532 0 12:09AM ttys000 0:00.00 grep TerminalWe see that this particular process happens to have a PID of 252. grep returned any process with the word Terminal, including the grep itself. We can be more precise with awk and see the header, too:
$ ps -Af | awk 'NR==1 || $2==252' UID PID PPID C STIME TTY TIME CMD 501 252 132 0 11:02PM ?? 0:01.89 /Applications/Utilities/Terminal.app/Contents/MacOS/Terminal(that's print the first row OR anything with the 2nd field equal to 252.) Now let's kill it:
$ kill 252poof!—our terminal is gone.
Running stuff in the background is useful, especially if you have a time-consuming program. If you're scripting a lot, you'll often find yourself running something like this:
$ script.py > out.o 2> out.e &i.e., running something in the background and saving both the output and error.
Two other commands that come to mind are time, which times how long your script takes to run, and nohup ("no hang up"), which allows your script to run even if you quit the terminal:
$ time script.py > out.o 2> out.e & $ nohup script.py > out.o 2> out.e &As we mentioned above, you can also set multiple jobs to run in the background, in parallel, from a loop:
$ for i in 1 2 3; do { echo "***"$i; sleep 60 & } done
(Of course, you'd want to run something more useful than sleep!)
If you're lucky enough to work on a large computer cluster shared by many users—some of whom may be running memory- and time-intensive programs—then scheduling different users' jobs is a daily fact of life.
At work we use a queuing system called the Sun Grid Engine to solve this problem.
I wrote a short SGE wiki here.
To get a dynamic view of your processes, loads of other information, and sort them in different ways, use top:
$ topHere's a screenshot of htop—a souped-up version of top—running on Ubuntu:
htop can show us a traditional top output split-screened with a process tree. We see various users—root, ubuntu, and www-data—and we see various processes, including init which has a PID of 1. htop also shows us the percent usage of each of our cores—here there's only one and we're using just 0.7%. Can you guess what this computer might be doing? I'm using it to host a website, as the process for nginx, a popular web server, gives away.
