As a follow-up to Michael Stutz's excellent
article, this article provides 10 more good habits
to adopt that will improve your UNIX® command-line
efficiency. Learn about common errors and how to
overcome them, and discover exactly why these 10 UNIX
habits are worth picking up!
Let's face it: Bad habits are hard to break. But habits
that you've just become comfortable with can be even more
difficult to overcome. Sometimes, a fresh look at things may
provide you with an "A-ha, I didn't know you could do that!"
moment. Building on Michael Stutz's excellent article, "Learn
10 good UNIX usage habits," this article suggests 10
more UNIX command-line commands, tools, and techniques that
may make you more productive as a UNIX command-line wizard.
The 10 additional good habits you should adopt are:
- Use file name completion.
- Use history expansion.
- Reuse previous arguments.
- Manage directory navigation with
pushd
and popd .
- Find large files.
- Create temporary files without an editor.
- Use the
curl command-line utility.
- Make the most of regular expressions.
- Determine the current user.
- Process data with
awk .
Use file name
completion
Wouldn't it be great if you didn't have to type a long,
convoluted file name at the command prompt? Well, you don't,
as it turns out. You can configure the most popular UNIX
shells for file name completion, instead. This functionality
works a bit differently in each shell, so I show you how to
use file name completion in the most popular shells. File
name completion allows you to type faster and avoid errors.
Lazy? Perhaps. More efficient? Definitely!
Which shell am
I running?
What happens if you don’t know which shell you're
currently running? Although this trick isn't officially part
of the 10 more good habits, it's still pretty useful. As
shown in Listing 1,
you can use the echo $0 or ps –p $$
command to display the shell you're using. In my case, I'm
running the Bash shell.
Listing 1. Determine your shell
$ echo $0
-bash
$ ps –p $$
PID TTY TIME CMD
6344 ttys000 0:00.02 –bash
|
C shell
The C shell supports the most
straightforward file name completion. Setting the
filec variable enables the functionality. (You can
use the command set filec ). After you start
typing the name of a file, you can click Escape, and
the shell fills in the name of the file—or as much as it
can. For example, say you have files named file1, file2,
and file3. If you type f , then click
Escape, file will be filled out, and you'll have
to type the 1 , 2 , or 3
to complete the appropriate file name.
Bash
The Bash shell also provides file name completion but
uses the Tab key instead of the Escape key. You don't need
to set anything to enable file name completion in the Bash
shell; it's set by default. Bash also implements an
additional feature. After typing a portion of a file name,
then clicking Tab, if you reach that point at which
multiple files satisfy your request and you need to add text
to select one of the files, you can click Tab twice
more for a list of the files that match what you have typed
so far. Using the earlier examples of files named file1,
file2, and file3, start by typing f .
When you click Tab once, Bash completes file;
clicking Tab one more time expands the list of
file1 file2 file3.
Korn shell
For Korn shell users, file name completion depends on the
value of the EDITOR variable. If EDITOR
is set to vi, you type part of name, and then click
Escape followed by a backslash (\ )
character. If EDITOR is set to emacs,
you type part of the name, and then click the Escape
key twice to complete the file name.
Use history
expansion
What happens if you're using the same file name for a
series of commands? Well, there's a shortcut that can
quickly retrieve the last file name you used. As shown in
Listing 2, the
!$ command returns the file name that the
previous command used. The file
this-is-a-long-lunch-menu-file.txt is searched for
occurrences of the word pickles. After searching, the
vi command is used to edit the
this-is-a-long-lunch-menu-file.txt file without the need for
retyping the file name. You use the bang, or exclamation
point (! ), to access the history, and the
dollar sign ($ ) returns the last field of the
previous command. It's a great tool if you are using long
file names repeatedly.
Listing 2. Using !$ to retrieve the last
file name used with a command
$ grep pickles this-is-a-long-lunch-menu-file.txt
pastrami on rye with pickles and onions
$ vi !$
|
Reuse previous arguments
The !$ command returns the last argument
used with a command. But what happens if you have a command
that used arguments and you want to reuse just one of them?
The !:1 operator returns the argument used in a
command. The example in Listing 3 shows how you can use this
operator in combination with the !$ operator.
In the first command, a file is renamed to a more meaningful
name, but to preserve use of the original file name, a
symbolic link is created. The file kxp12.c is
renamed in a more readable manner, then the link command is
used to create a symbolic link back to the original file
name, in case it's still used elsewhere. The !$
operator returns the file_system_access.c
argument, and the !:1 operator returns the
kxp12.c argument, which is the first argument
of the previous command.
Listing 3. Using !$ and !:1 in
combination
$ mv kxp12.c file_system_access.c
$ ln –s !$ !:1
|
Manage directory navigation with
pushd and popd
UNIX supports a wide variety of directory-navigation
tools. Two of my favorite productivity tools are pushd
and popd . You're certainly aware that the
cd command changes your current directory. What
happens if you have several directories to navigate, but you
want to be able to quickly return to a location? The
pushd and popd commands create a virtual
directory stack, with the pushd command
changing your current directory and storing it on the stack,
and the popd command removing the directory
from the top of the stack and returning you to that
location. You can use the dirs command to
display the current directory stack without pushing or
popping a new directory.
Listing 4 shows
how you can use the pushd and popd
commands to quickly navigate the directory tree.
Listing 4. Using pushd and popd to
navigate the directory tree
$ pushd .
~ ~
$ pushd /etc
/etc ~ ~
$ pushd /var
/var /etc ~ ~
$ pushd /usr/local/bin
/usr/local/bin /var /etc ~ ~
$ dirs
/usr/local/bin /var /etc ~ ~
$ popd
/var /etc ~ ~
$ popd
/etc ~ ~
$ popd
~ ~
$ popd
|
The pushd and popd commands also support
parameters to manipulate the directory stack. Using the
+n or -n parameter,
where n is a number, you can rotate the stack left or
right, as shown in
Listing 5.
Listing 5. Rotating the directory stack
$ dirs
/usr/local/bin /var /etc ~ ~
$ pushd +1
/var /etc ~ ~ /usr/local/bin
$ pushd -1
~ /usr/local/bin /var /etc ~
|
Find large files
Need to find out where all your free disk space went?
Here are a couple of tools you can use to manage your
storage. As shown in
Listing 6, the df command shows you the
total number of blocks used on each available volume and the
percentage of free space.
Listing 6. Determining volume usage
$ df
Filesystem 512-blocks Used Available Capacity Mounted on
/dev/disk0s2 311909984 267275264 44122720 86% /
devfs 224 224 0 100% /dev
fdesc 2 2 0 100% /dev
map -hosts 0 0 0 100% /net
map auto_home 0 0 0 100% /home
|
Want to find the largest files? Use the find command with
the -size parameter.
Listing 7 shows
how to use the find command to find files
larger than 10MB. Note that the -size parameter
takes a size in kilobytes.
Listing 7. Find all files larger than
10MB
$ find / -size +10000k –xdev –exec ls –lh {}\;
|
Create temporary files without an
editor
This is a simple one: You need to quickly create a simple
temporary file but don't want to fire up your editor. Use
the cat command with the >
file-redirection operator. As shown in
Listing 8, using
the cat command without a file name simply
echoes anything typed to standard input; the >
redirection captures that to the specified file. Note that
you must provide the end-of-file character when you're
finished typing—typically, Ctrl-D.
Listing 8. Quickly create a temporary
file
$ cat > my_temp_file.txt
This is my temp file text
^D
$ cat my_temp_file.txt
This is my temp file text
|
Need to do the same thing but append to an existing file instead of
creating a new one? As shown in
Listing 9, use the
>> operator, instead. The >>
file-redirection operator appends to an existing file.
Listing 9. Quickly append to a file
$ cat >> my_temp_file.txt
More text
^D
$ cat my_temp_file.txt
This is my temp file text
More text
|
Use the curl command-line utility
I can access the Web from the command line? Are you
crazy? No, it's just curl ! The curl
command lets you retrieve data from a server using the HTTP,
HTTPS, FTP, FTPS, Gopher, DICT, TELNET, LDAP, or FILE
protocols. As shown in
Listing 10, I can
use the curl command to access the current
local conditions of the National Weather Service for my
location (Buffalo, NY). When combined with the grep
command, I can retrieve the conditions in Buffalo. Use the
-s command-line option to suppress curl
processing output.
Listing 10. Retrieve the current weather
conditions with curl
$ curl –s http://www.srh.noaa.gov/data/ALY/RWRALY | grep BUFFALO
BUFFALO MOSUNNY 43 22 43 NE13 30.10R
|
As shown in Listing 11, you
can also use the curl command to download
HTTP-hosted files. Use the -o parameter to
specify where the output is saved.
Listing 11. Use curl to download
HTTP-hosted files
$ curl -o archive.tar http://www.somesite.com/archive.tar
|
This is really just a hint of what you can do with curl .
You can start exploring a bit more simply by typing
man curl at your command prompt to display the
complete usage information for the curl
command.
Make the most of
regular expressions
Many UNIX commands use regular expressions as arguments.
Technically speaking, a regular expression is a
string (that is, a sequence of characters composed of
letters, numbers, and symbols) that represents a pattern
defining zero or more strings. A regular expression uses
meta-characters (for example, the asterisk [* ]
and question mark [? ] symbols) to match parts
of or whole other strings. A regular expression doesn't have
to contain wildcards, but wildcards can make regular
expressions useful for searching for patterns and
manipulating files.
Table 1 shows some basic regular expression sequences.
Table 1. Regular expression sequences
Sequence |
Description |
Caret (^ ) |
Matches the expression at the start of a line,
as in ^A
|
Question mark (? ) |
Matches the expression at the end of a line, as
in A?
|
Backslash (\ ) |
Turns off the special meaning of the next
character, as in \^
|
Brackets ([] ) |
Matches any one of the enclosed characters, as
in [aeiou] (Use a hyphen [- ]
for a range, as in [0-9] .) |
[^ ] |
Matches any one character except those enclosed
in brackets, as in [^0-9] |
Period (. ) |
Matches a single character of any value except
end of line |
Asterisk (* ) |
Matches zero or more of the preceding characters
or expressions |
\{x,y\} |
Matches x to y occurrences of the
preceding |
\{x\} |
Matches exactly x occurrences of the
preceding |
\{x,\} |
Matches x or more occurrences of the
preceding |
Listing 12
shows some of the basic regular expressions used with the
grep command.
Listing 12. Using regular expressions
with grep
$ # Lists your mail
$ grep '^From: ' /usr/mail/$USER
$ # Any line with at least one letter
$ grep '[a-zA-Z]' search-file.txt
$ # Anything not a letter or number
$ grep '[^a-zA-Z0-9] search-file.txt
$ # Find phone numbers in the form 999-9999
$ grep '[0-9]\{3\}-[0-9]\{4\}' search-file.txt
$ # Find lines with exactly one character
$ grep '^.$' search-file.txt
$ # Find any line that starts with a period "."
$ grep '^\.' search-file.txt
$ # Find lines that start with a "." and 2 lowercase letters
$ grep '^\.[a-z][a-z]' search-file.txt
|
Many books have been written just about regular expressions. For a more
in-depth look at command-line regular expressions, I suggest
the developerWorks article, "Speaking
UNIX, Part 9: Regular expressions."
Determine the
current user
At times, you may have an administrative script that you
want to make sure a certain user has or has not executed. To
find out, you can use the whoami command to
return the name of the current user.
Listing 13 shows
the whoami command run on its own;
Listing 14 shows
an excerpt from a Bash script using whoami to
make sure the current user isn't root.
Listing 13. Using whoami from the
command line
Listing 14. Using whoami in a script
if [ $(whoami) = "root" ]
then
echo "You cannot run this script as root."
exit 1
fi
|
Process data with awk
The awk command always seems to live in the
shadows of Perl, but it can be a quick, useful tool for
simple command-line-based data manipulation.
Listing 15 shows
how to get started with the awk command. To get
the length of each line in the file text, use the
length() function. To see if the string ing is
present in the file text, use the index()
function, which returns the location of the first occurrence
of ing so that you can use it for further string
processing. To tokenize (that is, split a line into
word-length pieces) a string, use the split()
function.
Listing 15. Basic awk processing
$ cat text
testing the awk command
$ awk '{ i = length($0); print i }' text
23
$ awk '{ i = index($0,”ing”); print i}' text
5
$ awk 'BEGIN { i = 1 } { n = split($0,a," "); while (i <= n) {print a[i]; i++;} }' text
testing
the
awk
command
|
Printing specified fields of text file is a simple awk
task. In Listing 16,
the sales file consists of each salesperson's name
followed by a monthly sales figure. You can use the
awk command to quickly total the sales for each
month. By default, awk treats each
comma-separated value as a different field. You use the
$n operators to access each individual
field.
Listing 16. Using awk for data
summarization
$cat sales
Gene,12,23,7
Dawn,10,25,15
Renee,15,13,18
David,8,21,17
$ awk -F, '{print $1,$2+$3+$4}' sales
Gene 42
Dawn 50
Renee 46
David 46
|
The awk command can be complex and used in a wide variety
of situations. To explore the awk command more
fully, start with the command man awk in
addition to the resources mentioned in the
Resources.
Conclusion
Becoming a command-line wizard takes a bit of practice.
It's easy to keep doing things the same way simply because
you're used to it. Expanding your command-line resources can
provide a big increase in your productivity and propel you
toward becoming a UNIX command line wizard! |
No comments:
Post a Comment