ITEC 400
Shell (Part 2)
George Vaughan
Franklin University
1
Topics
•
•
•
•
•
•
VNC
Root
Special Characters in Shell
Useful Unix Commands for Scripting
computations in Shell Scripts
Flow Control in Shell Scripts
* Note: Your Sys Admin book has a nice description of shell
programming in the appendix
2
VNC
• VNC is a free, handy tool for remotely managing
Windows, Linux or Unix machines
• Virtual Network Computing
• URL: http://www.uk.research.att.com/vnc/
• Allows one to administer the Desktops of many
computers on one computer.
• Example: I can manage my Linux and Windows
desktops (hence the machines themselves) from
my laptop.
3
ROOT
• ROOT is an overloaded term in Unix
• ROOT user is the super-user. Root can
see and do anything. (You must be root to
add new users to the system)
• ROOT directory is the parent of all UNIX
directories (/tmp is under the root directory
‘/’)
4
Special Shell Characters
• The Dot “.” Command
– The Dot Command allows you to run a script in the
current shell (i.e. a new shell is not created).
– Note: using the dot command to run a script can
affect the current shell.
– Example (using 2 telnet sessions):
$ ps -u gvaughan
PID TTY
TIME
1434 pts/0
00:00:00
12594 pts/2
00:00:00
12668 pts/2
00:00:00
12669 pts/0
00:00:00
CMD
bash
bash
sleep
ps
<= Login shell from telnet #1
<= Login shell from telnet #2
<= sleep from script on telnet #2
<= ps on telnet #1
5
Special Shell Characters
• “\” Line Continuation Character
– allows one logical line in shell to fit on more than one
physical line in a file
– Example:
ls | \
wc –l
• will return a count of the files in the current
directory.
• This is equivalent to: ls | wc -l
• Useful for long source lines - improves readability
6
Special Shell Characters
• “;” Command Separator
– Used to separate multiple commands on same
physical line
– Example:
cd bin; ls
• This is equivalent to:
cd bin
ls
– Commands separated by semi-colon are executed
serially. How is this different from the pipe?
– 2 commands separated by a semi-colon are still
executed in 2 distinct processes.
7
Special Shell Characters
• “(…)” Sub Shell Construct
– Allows one or more commands to be executed in a
sub shell.
– Behavior is similar to putting commands in a script.
– Useful to protect current shell form side effects.
– Useful for redirecting output from many commands
– Examples:
cd $HOME; (cd /bin; pwd); pwd
(ls; pwd; ls) > trash #send output of all 3 commands to trash
(ls; pwd; ls) &
#run all cmds in background
8
Special Shell Characters
• Example:
>date; ps
Mon May 5 21:16:38 EDT 2003
PID TTY
TIME CMD
12325 pts/1
00:00:00 bash
12367 pts/1
00:00:00 ps
– Why don’t we see ‘date’ in the ‘ps’ report?
• Example:
>(date; ps)
Mon May 5 21:15:57 EDT 2003
PID TTY
TIME CMD
12325 pts/1
00:00:00 bash
12361 pts/1
00:00:00 bash
12363 pts/1
00:00:00 ps
– Notice the 2 instances of bash
9
Special Shell Characters
• “&” Run command in background
– Will run command in background.
– Example (with 1 telnet session):
>test01 &
[1] 12820 <= process ID of script in background
>ps
PID TTY
TIME CMD
1434 pts/0
00:00:00 bash <= login shell
12820 pts/0
00:00:00 bash <= script shell
12821 pts/0
00:00:00 sleep
12822 pts/0
00:00:00 ps
10
Special Shell Characters
• “|” Pipe
– Take the output of command 1 and use it as
the input of command 2
– Example:
ps -e | wc -l
• the ‘ps’ and ‘wc’ commands are running
concurrently in 2 distinct processes.
– You can pipe multiple commands:
who | awk '{print $1 }' | sort -u | wc -l
• This reports the number of unique active logins on
the system
11
Special Shell Characters
• “>” Redirect Output
– Sends output of command to file.
– Will overwrite file
– Examples:
ls > trash
(date;ps) > trash
• “>>” Append Output
– Appends output of command to end of file.
– Example:
ls >> trash
12
Special Shell Characters
• “<“ Redirect input
– Useful for reading command input from a file.
– Example:
• Use redirection to automatically edit a bunch of
files.
• Assume we have a file named control.vi
• Inside the file control.vi we have the following vi
commands (change bsh to ksh, write file, quit):
:g/bsh/s//ksh
:wq
13
Special Shell Characters
• Redirection Example (continued)
– We can now use the control.vi file to
automatically edit another file such as
myscript:
vi myscript < control.vi
– We can imagine how we might generalize this
using a shell script with loops…
– The sed command is very useful for this
purpose (we will see sed described later…)
14
Special Shell Characters
– We can also use input redirection to generate
a list as arguments for a command.
– Example:
• Assume we have a file “trash” with the following
contents:
/etc
/bin
• We could use the file “trash” as input to “ls”:
ls $(<trash)
#(equivalent to ls /bin /etc)
• ls will list the contents of /etc and /bin.
15
Special Shell Characters
• “.” Current Directory (not the same as the dot
command)
– “.” is a shortcut for referencing the current directory
– Example:
./myApp
#execute the program myApp which exists
#in the current directory.
• “..” Parent Directory
– “..” is a shortcut for referencing the parent directory
– Example:
../myApp
#execute the program myApp which exists
#in the directory above.
16
Useful Unix Commands for Scripting
• Unix has many commands – some are
especially useful for scripts.
• The following commands are feature rich –
check man page.
17
Useful Unix Commands for Scripting
• grep – prints any line in the input stream that
contains a specified substring
– When filtering output, think of grep as tool to make
horizontal cuts (line selection)
– Supports regular expressions with –e option
– Example: Show me all of my processes:
ps –ef | grep $LOGNAME
– Useful for filtering out items of interest
– “grep -v string” will match every line that doesn’t
contain the string.
– grep -i string (ignores case)
18
Useful Unix Commands for Scripting
• awk, nawk – a filter like scripting language
– When filtering output, think of ‘awk’ as tool to make
horizontal cuts (line selection) and vertical cuts (field
selection).
– Quite powerful (it is a programming language)
– Can be used to select and/or rearrange fields in input
– Example:
• ps –ef shows the following (only subset is shown):
> ps -ef | more
UID
PID PPID C
root
0
0 0
root
1
0 0
root
2
0 0
root
3
0 1
STIME TTY
Dec 13 ?
Dec 13 ?
Dec 13 ?
Dec 13 ?
TIME CMD
0:01 sched
1:12 /etc/init 0:00 pageout
529:56 fsflush
19
Useful Unix Commands for Scripting
• awk, nawk (Cont.):
– Assume we only want to see the command and the
owner, in that order:
>ps –ef | awk '{ print $9, $1 }'
sched root
/etc/init root
pageout root
fsflush root
/usr/lib/utmpd root
/usr/sbin/keyserv root
/usr/lib/saf/sac root
/usr/lib/devfsadm/devfseventd root
– Notice the use of single quotes in AWK script (why?)
20
Useful Unix Commands for Scripting
• awk, nawk (Cont.):
• Another example:
– Each line in the password file (/etc/passwd) is
variable length and delimited by a colon ‘:’
pangh:x:118726:100:Hanbin Pang:/export/home/pangh:/bin/ksh (subset of file)
– If I want to retrieve the user’s ID and name (field
positions 1 and 5), I can use awk:
>cat
/etc/passwd | awk '{ FS=":"; print ($1, $5) }'
vaughang George Vaughan (subset of output)
– ‘FS’ in the example above is used to inform
awk what character(s) are used as field
separators (space is the default)
21
Useful Unix Commands for Scripting
• cut- When filtering output, think of ‘cut’ as
tool to make vertical cuts (column
selection).
• When can make vertical cuts based on
character position or by field (awk is better
for this task).
22
Useful Unix Commands for Scripting
•
Example of ‘cut’ by character position:
– The command ‘ls –l’ can be used to show everyone's permissions to a
file:
>ls -l
total 8
-rwxr-xr-x
-rwxr-xr-x
drwxr-xr-x
-rw-r--r--
1
1
2
1
vaughang
vaughang
vaughang
vaughang
faculty
faculty
faculty
faculty
415
151
512
20
Sep
Sep
Sep
Sep
6
9
6
6
09:32
21:27
00:06
00:07
dirsize
ex0280
junk
trash2
– We can use cut based on character position to just display permissions for owner
(info in columns 2 through 4):
>ls -l | grep -v total |
rwx
rwx
rwx
rw-
cut -c2-4
– In the example above, we used ‘grep –v’ to strip out the first line of ‘ls –l’ which
contains information about the total number of disk blocks used by directory.
23
Useful Unix Commands for Scripting
• Example of ‘cut’ by field position:
– Each line in the password file (/etc/passwd) is
variable length and delimited by a colon ‘:’
pangh:x:118726:100:Hanbin Pang:/export/home/pangh:/bin/ksh (subset of file)
– If I want to retrieve the user’s ID and name (field
positions 1 and 5), I can use cut:
>cat /etc/passwd | cut -f1,5 –d’:’
vaughang:George Vaughan (subset of output)
– The ‘-d’ option is used to tell ‘cut’ which character (or
characters) serve as the delimiter (TAB is the default).
– Note: awk is a more powerful tool for making vertical
cuts based on fields.
24
Useful Unix Commands for Scripting
• sort – sorts input.
– can sort on any column (defaults to column
1).
– the –u option will only show unique rows.
– Example:
• Count the number of active processes:
ps -ef | wc –l
• Count the number of unique process owners:
ps -ef | awk '{ print $1 }' | sort -u | wc -l
25
Useful Unix Commands for Scripting
• sed – stream editor
– Used to edit input on the fly, powerful
– Examples:
• Don’t list processes owned by root:
ps -ef | sed '/root/d'
• Strip out blank lines:
cat trash | sed '/^$/d'
• Change each comma to a blank, globally
cat trash | sed 's/,/ /g'
26
Useful Unix Commands for Scripting
• find – recursively search directories for files of
interest
– Book provides a good overview
– Example:
• Count all files in the system that end in .html
find / –name *.html –print | wc –l
• date – print date and time
– contains useful format control useful for creating file
suffices:
– Example:
• Save a listing of my files as of today
FILE_NAME=file_list_`date '+%m%d%y'`
• Produces file name like: file_list_011203
27
Useful Unix Commands for Scripting
• diff – show differences between 2 ASCII (text)
files.
– Output is cryptic, can be used to convert one file to
another, allows SCCS to store deltas.
– If files are the same, no output.
– Example
diff file_1 file_2 | wc –l
#if zero, files are same
• cat – dump the contents of a file to standard out:
– Example:
• Count the number of registered users on system by counting
entries in password file:
cat /etc/passwd | wc -l
28
Useful Unix Commands for Scripting
• touch – change file timestamps.
– useful for ‘marking a file’.
– If file doesn’t exist, it is created with zero
length.
– ‘touched’ file can serve as a record when
something happened.
– Example:
• touch the file ‘last_backup’ when backup
completes:
touch last_backup
29
Useful Unix Commands for Scripting
• basename – strip of the path from a fully
qualified file name.
– Commands like ‘find’ produce lists of files with
pre-appended paths. Use basename if you
want just the filename.
– Example:
echo `basename vaughang/itec400/lecture_notes/notes1.pp`
• produces the output ‘notes1.pp’
30
Useful Unix Commands for Scripting
• which – tells you which copy of command
you will execute if you execute the
command.
– Determined by your PATH variable.
– Useful when you think you are using one copy
when in fact you are using another copy.
– Example:
which ls
– produce the following result: /bin/ls
31
Useful Unix Commands for Scripting
• set –x
– Used to turn on tracing in shell script.
– Useful for debugging
• set +x
– Turns off shell tracing
• Example:
set –x
date
set +x
date
32
Useful Unix Commands for Scripting
• exit – terminate script and return integer for
return code
• Example:
exit 2
– Will cause the script to terminate with a return code of
2.
– Useful for error handling by other scripts.
– Which shell variable will contain the return code from
exit (see notes from previous lecture)?
33
Useful Unix Commands for Scripting
• shift
– The shift command will shift all command line
arguments to the left by 1.
– So, after using shift, $1=$2, $2=$3, $3=$4,…
– shift can be used in a loop.
34
How Script Can Change Parent Shell
Variables (“.” Command)
• By default, shell variables are local to
script.
• How can we make a shell script change
contents of parent shell?
• With the “.” command:
– example: . myScript
• myScript will change variables in parent shell
• why?
35
How Parent Shell Can Initialize Shell
Variables In Script (“export” command)
• When we use the export command on a shell
variable, it becomes visible to subshells and
scripts.
• Exported variable is something like a global
variable.
• In the example below, $MY_VAR is set to 23 and
then exported. $MY_VAR will be visible inside
myScript
export MY_VAR=23
myScript
36
Using “.” and “export” commands
together
• If we use a “.” command on a script and
inside the script we “export” variables,
then exported variables will not only be
visible in parent shell but also in all future
subshells.
• Example: PATH variable in .profile
• Notice that we always execute .profile with
the “.” command (what would happen if we
didn’t?)
37
A Note On Examples
• Source for all examples in this and future lectures are on codd (einstein) at:
/export/home/vaughang/public_html/itec400/examples
• and on the web at:
– http://cs.franklin.edu/~vaughang/itec400/examples/
• All shell examples are written in Korn Shell (ksh) although they should be
quite compatible with Bash.
• In all examples, the ‘greater-than’ symbol, ‘>’ will be used to indicate the
shell prompt. So, if we execute the date command,
>date
Thu Sep 2 20:43:58 EDT 2004
we only type ‘date’ and not ‘>date’.
• All scripts should be able to execute on codd (einstein) which is a Solaris
machine, or any Linux distribution that supports Korn shell, located at
/bin/ksh.
38
Flow Control in Shell Scripts
• We will explore the following flow control
constructs:
– for
– while
– case
39
Flow Control in Shell Scripts
• “for” flow construct
– Format:
for VAR in list
do
statement-body
done
– Loop for each element in list, assigning each
element to $VAR per loop iteration.
40
Example ex0240
0001 #!/bin/ksh
0002
0003 ###########
0004 # File: ex0240
0005 #
0006 # Test each entry in arg
0007 # to see if file or dir
0008 ###########
0009 for i in $1/*
0010 do
0011 file $i
0012 done
• Line 9: For each file in dir
specified by arg, determine file
type.
• OUTPUT:
>ex0240 $PWD
ex0010:
executable /bin/ksh script
ex0020:
executable /bin/ksh script
ex0030:
executable /bin/ksh script
ex0040:
executable /bin/ksh script
41
Example 0250
0001 #!/bin/ksh
0002
0003 ###########
0004 # File: ex0250
0005 #
0006 # Test each entry in file
0007 # to see if file or dir
0008 ###########
0009 for i in $(<$1)
0010 do
0011 file $i
0012 done
• Line 9: Use the file specified
by $1 as the list for the for loop
• OUPUT:
>ls > trash
>ex0250 trash
ex0010:
executable /bin/ksh script
ex0020:
executable /bin/ksh script
ex0030:
executable /bin/ksh script
ex0040:
executable /bin/ksh script
ex0050:
executable /bin/ksh script
trash:
ascii text
42
Flow Control in Shell Scripts
• “while” flow construct
– Format:
while [ expression ]
do
statement-body
done
– Keep looping while expression is true.
43
Flow Control in Shell Scripts
0001 #!/bin/ksh
0002
0003 ###########
0004 # File: ex0260
0005 #
0006 # Create a counter
0007 ###########
0008 COUNTER=0
0009 while [ "$COUNTER" -lt 5 ]
0010 do
0011 COUNTER=$(( $COUNTER + 1 ))
0012 echo $COUNTER
0013 done
•
Line 9: Loop as long as
COUNTER is less than 5.
• Line 11:Math computations are
performed inside $(( ))
• Line 11:Notice space after “$((“
and before “))” – they are
necessary.
• OUTPUT:
>ex0260
1
2
3
4
5
44
Flow Control in Shell Scripts
• “case” flow construct
– Format:
case value in
pattern1)
command1
commandn;;
patternn)
command1
commandn;;
esac
45
Example ex0270
0001: #!/bin/ksh
0002:
0003: ###########
0004: # File: ex0270
0005: #
0006: # translate money
0007: ###########
0008: COIN=$1
0009: case "$COIN" in
0010: 1) echo penny;;
0011: 5) echo nickel;;
0012: 10) echo dime;;
0013: 25) echo quarter;;
0014: *) echo not a coin!;;
0015: esac
•
Line 9: Use ‘case’ to translate coin
value to coin name.
• Output:
$ ex0270 25
quarter
$ ex0270 3
not a coin!
46
More Than 9 Command Line
Arguments
• As we have seen, a shell script can
access command line arguments with
shell variables such as $1, $2, $3, etc.
• Shell however, will only allow us to access
up to 9 command line arguments ($10 will
not work).
• The shift command is one way to address
this, see:
http://www.shu.ac.uk/schools/cms/teaching/ps/unix/params.html
47
More Than 9 Command Line
Arguments
• Another technique is to use a special shell
variable, $*
• $* contains all command line arguments.
• We can use $* to loop through an
indeterminate number of command line
arguments…
48
More Than 9 Command Line
Arguments
0001 #!/bin/ksh
0002 ###########
0003 # File: ex0280
0004 #
0005 # process more than 9 args
0006 ###########
0007 ARG_LIST="$*"
0008 for i in $ARG_LIST
0009 do
0010 printf "%03d " $i
0011 done
0012 printf "\n"
•
Line 7: Store ‘n’ arguments in
$ARG_LIST
• Line 8: Loop across all arguments
• Line 10: print each argument
– without a new line
– right justified
– zero padded
• Line 12: print a “new line”
• OUTPUT:
ex0280 1 2 3 4 5 6 7 8 9 10 11
001 002 003 004 005 006 007 008 009
010 011
49
Shell Arrays
• Shell supports arrays
• Setting Array Elements:
MY_ARRAY[3]=34
MY_ARRAY[$COUNTER]=5
• Accessing Array Elements:
echo ${MY_ARRAY[3]}
echo ${MY_ARRAY[$COUNTER]}
50
Shell Arrays
0001: #!/bin/ksh
• Line 10: initialize each array
element.
0002: ###########
• Line 15: access 2nd array element
0003: # File: ex0290
• OUTPUT:
0004: #
ex0290
0005: # shell array example
0006: ###########
green
0007: COUNTER=1
0008: for i in blue green red yellow
0009: do
0010: MY_ARRAY[$COUNTER]=$i
0011: COUNTER=$(( $COUNTER + 1 ))
0012: done
0013: printf "\n"
0014:
0015: echo ${MY_ARRAY[2]}
51
References
• “Essential System Administration”, by Aeleen
Frisch, 2002
• “UNIX Shell Programming”, by Stephen G.
Kochan and Patrick H. Wood, 1990
• “Learning the bash Shell”, by Cameron Newham and
Bill Rosenblatt, 1998
52