\chapter{A Unix Primer\label{chap:unixprimer}}

In this appendix, you would learn: \\
\begin{itemize}
\item basic concepts of the Unix operating system
\item some basic commands in Unix
\end{itemize}

\section{Introduction}

\subsection{Why Should I Care About Unix?}

You may wonder why I took the time to dedicate an appendix to the Unix operating system in a Perl tutorial.
``Perl is platform-independent'', you said, ``and my Perl programs run fine on my Windows XP!'' Indeed, Perl
programs are largely platform independent (platform-dependent Perl programming is possible, but in most cases
you don't need to, so I am not going into details), and you can develop and test Perl programs entirely on 
your platform. However, a couple of Web server surveys consistently confirm that more than 60\% of the Web 
servers on the Internet run on various flavours of Unix, outweighing Microsoft Windows with the remaining
share of about 30\%. Very likely, your Web host is also running on Unix. Therefore, it is beneficial to you 
if you can get yourself acquainted with this operating system. \\

Also, Perl has a strong Unix culture and tradition. It was intended to be a flexible scripting language like
shell scripting, \mono{awk} and \mono{sed} etc. on Unix platforms. For example, regular expressions have long
been extensively used in various utilities on the Unix platform. Quite a number of Perl's builtin functions
(as seen on the perlfunc manpage) are interfaces to the corresponding Unix commands or closely resemble 
functions in the standard C libraries on Unix (not to mention the fact that Dennis Ritchie, the inventor of 
the C programming language, was also one of the inventors of the core Unix operating system!). \\

\subsection{What Is Unix?}

Unlike Microsoft Windows, which is solely owned and developed by the Microsoft Corporation, the term
``Unix'' does not refer to any specific operating system releases. It is a collective name embracing a family
of operating systems sharing certain common characteristics.\footnote{The Open Group has issued 
\link{http://www.opengroup.org/onlinepubs/007904975/}{The Open Group Base Specifications Issue 6, IEEE Std 1003.1, 2003 Edition}
that defines a standard operating system interface, which can be regarded as the basis of Unix operating systems.}
It was initially developed by Dennis Ritchie and Ken Thompson from the Bell Labs in 1960s. Over more than 30
years of evolution, many variants of the Unix operating systems have emerged. Today, the most prominent names
in Unix include Linux, *BSD (FreeBSD/OpenBSD/NetBSD), Solaris and Mac OS X (Jaguar) as a newcomer. \\

%http://www.bell-labs.com/history/unix/
%
%Without delving into the history of Unix, into which you can research from more authoritative sources online and 
%in technical papers and books in case you are interested, I intend to highlight some special features of Unix in
%general to help you understand the major differences between Unix and other operating systems. \\


This appendix is intended to be an introduction to the Unix computing environment. Therefore, emphasis is placed
on shell commands that are frequently used by users. In particular, X-Windows, the graphical user interface on
Unix, will not be introduced in this appendix. Also, for the sake of generality features or commands that are 
specific to any particular Unix variant will not be included. The commands and concepts described in this chapter
can be applied on many Unix operating systems. Readers are advised to consult other Unix literature for more 
in-depth treatment of the topics covered. \\

\subsection{The Overall Structure}

A system running Unix can generally be envisioned as having a layered architecture. What does this mean? First, the 
bottom layer is the system hardware. This ranges from the central processing unit (CPU), memory system, storage 
devices, graphics adapter to peripheral devices such as keyboard and printers. They are interconnected through
\term{buses}, that are inter-component wires that carry data and control information in between. The operating 
system lies on top of the hardware layer to coordinate the hardware components. This part of the operating system
which interacts with the hardware directly is the core of the operating system, called the \term{kernel}. On 
top of the kernel are the \term{user programs}. These programs do not need to interact with the hardware directly anymore.
Instead, they only need to interface with the kernel in case those services are needed, by invoking the necessary
\term{system calls}, which are functions provided by the kernel. Filesystem operations, for example, are examples
of standard system calls on Unix. The kernel would in turn perform the appropriate actions to instruct the devices
to commit the operation. When you are using a user program, you are presumably the top among the layers, on top
of the user program. The user program provides you with the user interface with which you interact. Commands defined by the user program,
and in case of graphical user interfaces, mouse clicks and their corresponding pixel coordinates are translated into
instructions for the lower layers. This layered architecture allows a high degree of abstraction between layers. 
If the interface (or the Application Programming Interface) of a layer changes, only the layer immediately on top
needs to be modified. Because of abstraction, the user needs not understand the low-level details of the hardware 
circuitry, for example, in order to operate a computer system. This is an important principle in all modern computer
systems. \\

To sum up, the Unix operating system serves to act as an intermediary between a computer user and the computer 
hardware so that activities that occur at different components of a computer system are well coordinated. \\

Unlike many other operating systems, Unix is a multi-user operating system from the very beginning, allowing 
multiple users to work on the system concurrently. Multiple users may log on a machine remotely from other
machines via the traditional telnet protocol, which gives you remote access to the command-line interface of the machine.
Telnet is not secure because data are carried in plain text. SSH (Secure SHell)
is a similar protocol but data are encrypted in transit. There is also a protocol called Virtual Network Computing
(VNC) that lets users have access to the graphical user interface from the remote side. \\

\section{Filesystems and Processes}

\subsection{Overview}
% FIXME Cache

In modern days, large-%
capacity \term{secondary storage} is important. Floppy disks, hard disks and CD-ROM etc. are classified as 
secondary storage devices, in contrast to \term{primary storage}, which is just an alias of the main memory.
Secondary storage media is considered permanent, because data that are written onto the media are retained when
the power is off. When an executable file has to be executed, the operating system arranges for the executable
file to be copied from secondary storage to primary storage (i.e. your RAM). The program is then broken down
into instructions and executed from internal caches. A program in execution
is called a \term{process}. The reason that the executable file is not executed from the secondary storage directly 
is that accesses of secondary storage devices are very slow compared with the main memory because such accesses involve
mechanical movement of disk arms, for example, and the speed of which is subject to mechanical limitations. \\

A disk is simply a large array of disk blocks of fixed size in which data can be stored. It does not mandate any
rules to organize data that are being stored on the disk. In order to better organize storage of data, a large-capacity
storage media is usually divided into multiple \term{partitions}. This is very common in systems with multiple 
operating systems installed, with each operating system installed in its own partition. However, partitioning 
simply marks the beginning and the end of each partition, but does not answer the need for organizing data that 
are written in each partition. Therefore, the second step is to create a \term{filesystem} on each partition.
By creating a filesystem, the data structures that are necessary to index data to allow efficient access are 
written to the partition. This operation is more well-known as ``formatting'' to users of MS-DOS and Windows 
operating systems. Modern Windows systems use either one of the two filesystems, namely \term{File Allocation Table}
(FAT) or \term{New Technology File System} (NTFS). On Unix, various choices of filesystems are available depending
on the operating system variant. Among the Unix variants, Linux supports the largest number of filesystems.
The standard filesystem on Linux is the \term{Second Extended Filesystem} (ext2). Other filesystems in widespread
use include reiserfs and ext3, which are both equipped with journalling capabilities. Other Unix variants 
mostly use the \term{Unix File System} (UFS). \\

The filesystem determines the directory structure, for example, how files are represented and layout on the disk.
To better organize the files, we introduce a hierarchical directory structure. Logically, files are classified and
put into different directories. Files of a similar kind are put into the same directory. Also, directories can be
nested, so that a directory can be created inside a directory. This allows fine-grained organization of files in a
well-structured manner for easy access. Because directories are hierarchical, they are customarily represented in 
the form of a tree. In Computer Science arena, a tree refers to a hierarchical data structure which best shows the 
subordinate relationships of directories. This is a very intuitive concept that readers with some experience with 
operating systems should be familiar with. \\

\image{!hbt}{0.8\textwidth}{!}{images/dirtree.eps}{Directory Structure In Tree Representation}{fig:dirtree}

The root of the directory structure in Unix is represented by /. The directory root is the only directory which 
does not have any parents. All files on the system must rest under the directory root. As you can see in the figure
above, a number of directories appear under the root. Each of these directories serve its specific purposes. 
In Table~\ref{tab:fhs} I list a few more important ones that are present on nearly all Unix systems
\footnote{Please consult the \link{http://www.pathname.com/fhs/}{Filesystem 
Hierarchy Standard}, the recommended scheme for compatibility between different Unix variants.}. \\

\begin{table}[htbp]
\begin{center}
\begin{tabular}{ll}
\hline
\textbf{Directory}	& \textbf{Purpose}	\\
\hline 
	\texttt{/bin}	&	Essential user programs, e.g. shells, file tools \br \\
	\texttt{/boot}	&	Core system files for system bootup \br \\
	\texttt{/dev}	&	Device files (remember devices are represented as files?) \br \\
	\texttt{/etc}	&	System-wide configuration files for system and user programs \br \\
	\texttt{/home}	&	User accounts: files private to individual users \br \\
	\texttt{/mnt}	&	Mount point for foreign filesystems \br \\
	\texttt{/root}	&	User account of system administrator (root) \br \\
	\texttt{/sbin}	&	System administration commands used only by root \br \\
	\texttt{/tmp}	&	Temporary files \br \\
	\texttt{/usr}	&	User programs that are installed system-wide, and related files \br \\
	\texttt{/var}	&	Data files written by system programs, e.g. email and various logs \br \\
\hline 
\end{tabular}
\\[\baselineskip]
\end{center}
\caption{Major Unix Standard Directories and Their Purposes}\label{tab:fhs}
\end{table}

In Unix terminology, a file is not necessarily a regular file. Directories, symbolic links and even devices are also
represented in the same way as a regular file, differentiated simply by a file type indicator in the inode (see below). 
A few other file types are defined, yet they are specialized file types that are seldom used directly, so I 
am not going into details here. \\

\subsection{Symbolic Links and Hard Links}

The concept of links is foreign to users of many other operating systems, e.g. MS-DOS or Windows. Unix supports
two types of links, namely \term{symbolic links} and \term{hard links}. Symbolic links and hard links in Unix are 
similar in sense to symbolic references and references in Perl, respectively. In many situations, symbolic links
and hard links have similar effects. However, under the hood they are implemented in very different ways which
give rise to different behaviours in certain situations. Because the concept of hard links is not very clearly 
explained in many Unix literature, I decided to elaborate a little bit on this concept in this section. \\

Symbolic links are easier to understand. It works like ``shortcuts'' (the .lnk) files on Windows. Basically, a
symbolic link is a special type of file that stores an alternative access path. When the symbolic link is accessed,
the stored path is used for access. Symbolic links are frequently used to create shortcuts to very long paths,
such that a shorter path is used instead. For software distribution sites, they are also used to maintain a static
URL to the latest release. For example, on a certain system there are directories and files as follows: \\

\begin{verbatimtab}[2]
/
	pub/
		download/
			myprog_current.zip -> /usr/share/devel/myprog/current.zip
	usr/
		share/
			devel/
				myprog/
					current.zip -> myprog_3.0.zip
					myprog_1.0.zip
					myprog_2.0.zip
					myprog_2.2.zip
					myprog_3.0.zip
\end{verbatimtab}

The ``\mono{->}'' indicates a symbolic link. On the left is the name of the symbolic link (the alias), and on
the right is the alternative path to follow. Say \mono{myprog} is a software developed. In this example, to 
access the current version of \mono{myprog}, the path is \mono{/pub/download/myprog\_current.zip}. When the file
is accessed through this symbolic link, the system would then try to access it at \mono{/usr/share/devel/myprog/current.zip},
which in turn is also a symbolic link to \mono{myprog\_3.0.zip}, the archive of the latest release. Therefore, 
essentially, the path \mono{/usr/share/devel/myprog/myprog\_3.0.zip} is used to access the file. This example serves
both purposes. It shortens the URL, and users can always get the latest release with the same URL, provided the
symbolic links are properly maintained to point to the latest release. You may also wonder why I create two symbolic
links instead of having \mono{myprog\_current.zip} pointing to \mono{myprog\_3.0.zip} directly. This is for convenience
in management, so that no matter the user is in \mono{/pub/download} or \mono{/usr/share/devel/myprog} he or she can 
still easily locate the latest version, and notice that by doing so the symbolic link in \mono{/pub/download} needs
not be updated when a new release is placed in \mono{/usr/share/devel/myprog}, only \mono{current.zip} needs to be. \\

To create a symbolic link, use the \mono{ln} command, with the \mono{-s} switch. For example, \\

\begin{lstlisting}[numbers=none,backgroundcolor=\color{CodeBackground}]
cbkihong@cbkihong:~/test$ cd pub/download
cbkihong@cbkihong:/pub/download$ ln -s /usr/share/devel/myprog/current.zip myprog_current.zip
\end{lstlisting}
\addvspace{\baselineskip}

Hard links are more difficult to understand. Recall that in the previous section I described what a filesystem is.
By creating a filesystem, indexing facilities that allow the operating system to quickly locate which disk blocks
a certain file occupies are created on the disk. This serves a similar purpose as the map in a traveller's pocket.
When a file is created on the disk, an \term{inode} (index node) is created for the file which contains attributes
such as the owner and group (see next section) identifiers, the times of modification and access, the file type and
locations of disk blocks containing the file content (pointers). Finally, the operating system needs to add to the inode of the
containing directory the pointer to the newly created inode so that the file is added to the directory entry. This
pointer is actually a hard link. Hard links work at the level of inodes to allow direct access of the file being
pointed to. Therefore, for every inode there is at least one hard link that points to it, from the containing directory file
entry. If we create an additional hard link to a file, that means a new pointer to the file inode is added.
Here is a sequence of commands which serves as an example: \\

\begin{lstlisting}[numbers=none,backgroundcolor=\color{CodeBackground}]
cbkihong@cbkihong:~/test$ touch file1.txt
cbkihong@cbkihong:~/test$ ln file1.txt file2.txt
cbkihong@cbkihong:~/test$ touch file3.txt
cbkihong@cbkihong:~/test$ ls -li
 342533 -rw-r--r--    2 cbkihong cbkihong        0 2003-07-30 17:56 file1.txt
 342533 -rw-r--r--    2 cbkihong cbkihong        0 2003-07-30 17:56 file2.txt
 342531 -rw-r--r--    1 cbkihong cbkihong        0 2003-07-30 17:59 file3.txt
cbkihong@cbkihong:~/test$ echo "ABCD" > file1.txt
cbkihong@cbkihong:~/test$ ls -li
 342533 -rw-r--r--    2 cbkihong cbkihong        5 2003-07-30 18:01 file1.txt
 342533 -rw-r--r--    2 cbkihong cbkihong        5 2003-07-30 18:01 file2.txt
 342531 -rw-r--r--    1 cbkihong cbkihong        0 2003-07-30 17:59 file3.txt
\end{lstlisting}
\addvspace{\baselineskip}

The \mono{touch} command creates an empty file. The \mono{ln} command without any switches creates a hard link.
Here, \mono{file2.txt} is created as an additional hard link to the inode of \mono{file1.txt}. The \mono{ls} command
displays a listing of files. Here, two switches \mono{-l} and \mono{-i} are given. If multiple switches are provided
on the command line, you can combine them into \mono{-li} (ordering does not matter). The \mono{-l} option causes
the long listing to be displayed, with the permission values, owner, group, date of the last modification and the file size. If 
the \mono{-i} switch is given, the inode number that the file entry points to is inserted as the first column. We
can see that \mono{file1.txt} and \mono{file2.txt} points to the same inode, while \mono{file3.txt} points to a
different inode. The third column is the number of hard links pointing to the inode. This number is stored at each
inode.  For regular files, this number
is usually one, as explained previously. However, because we have manually created a new hard link as 
\mono{file2.txt}, the number displayed is 2. Also notice that the two entries are exactly identical, except the
name. Here, the text ``ABCD'' (with line terminating character) is written into \mono{file1.txt}, and both entries
are updated. \\

You may wonder why the filesystem has to keep track of the number of hard links pointing to an inode. If you 
have read the chapter on references in Perl you will find a close resemblance with the Perl garbage collection
mechanism. That is, the inode would not be freed (deleted) until the number of hard links to it drops to 0. 
Consider the example again. If at this point we delete \mono{file1.txt}, the number drops to 1, but because we
still have a hard link that points to it from the entry \mono{file2.txt}, the inode is not freed. It is not 
until when \mono{file2.txt} is deleted, that the inode and other disk blocks that are associated with this file 
will eventually be freed. \\

\begin{lstlisting}[numbers=none,backgroundcolor=\color{CodeBackground}]
cbkihong@cbkihong:~/test$ rm file1.txt
cbkihong@cbkihong:~/test$ ls -li
 342533 -rw-r--r--    1 cbkihong cbkihong        5 2003-07-30 18:01 file2.txt
 342531 -rw-r--r--    1 cbkihong cbkihong        0 2003-07-30 17:59 file3.txt
cbkihong@cbkihong:~/test$ rm file2.txt
cbkihong@cbkihong:~/test$ ls -li
 342531 -rw-r--r--    1 cbkihong cbkihong        0 2003-07-30 17:59 file3.txt
\end{lstlisting}
\addvspace{\baselineskip}
 
Occasionally you will find some files (and directories) with the ``.'' name prefix, e.g. ``.kde3/'' and ``.vimrc''. 
These files  (the ``dot'' files) are usually found in private user accounts in the \mono{/home} tree, but in most cases 
you won't see them at all because the ``dot'' files are hidden by default. These files are usually created by user
applications to store user-specific configuration and data files, because the user account is usually the only directory
that applications written by users can write to that is private to the user. The \mono{/etc} tree is owned by root and only system-wide
configuration can be made there by the system administrator, and the \mono{/tmp} tree are for temporary files only 
and need to be periodically cleaned up to avoid accumulation of useless files, thus data files that are to be kept 
cannot be placed into these directories. Due to the large number of applications installed in 
the system, very likely large numbers of these data files have to be created in the user account. To avoid clutter
and prevent the user from accidentally deleting these files, Unix hides these files by default, unless you pass the
\mono{-a} option to the \mono{ls} command. \\

Are you wondering why I suddenly jump from my hard links discussion to ``dot'' files? That is because I am going to
introduce to you two special ``dot'' files that are present in every directory, namely ``.'' and ``..''. MS-DOS 
also has the notion of these two special files. They are automatically created in a directory when the directory 
is created. They are in fact hard links to the current directory file, and the parent directory file respectively.
Therefore, they appear as directories with the \mono{ls -la} command. ``..'' is used to refer to the parent directory
in the directory tree. For example, with the \mono{cd} command, you can specify the name of a subdirectory to go into it, but 
when you need to return to the parent directory, you can use the command \\

\begin{verbatim}
cd ..
\end{verbatim}

As another example, \mono{cat ../../README.txt} outputs the file \mono{README.txt} two levels upward. For example, 
if you are in the directory \mono{/home/cbkihong/docs/perltut} when you type the above command, then the Unix shell will try 
to find the file at \mono{/home/cbkihong/README.txt} and display its content. The ``.'' directory points to the 
current directory. It seems to be not useful at all. However, I can find at least one use of it. For security reasons,
many Unix installations would not put the current directory, that is ``.'' into the environment variable \mono{PATH}, especially for the root user.
Therefore, if you have an executable file, say \mono{myprog}, in your current directory by just typing \mono{myprog}
the program will not be started at all, because \mono{PATH} is the executable search path that is used if the path to the executable is not specified. Directories not listed
in this variable will not be searched for executables. In this case, you need to qualify it with the directory where
it can be found, by \\

\begin{verbatim}
./myprog
\end{verbatim}

If the target file is not an executable, the ``./'' prefix is generally optional and well understood if absent. 
Therefore, \mono{cat docs/README} is generally understood to be \mono{cat ./docs/README}. Because of these two
special ``dot'' files, you will find that the number of hard links pointing to directories is never one. As a 
bare minimum there are two, one due to ``.'' and the other one from the parent directory file inode. If there are
subdirectories, then ``..'' in the subdirectories will add to the number of hard links pointing to the current
directory inode. If there are extra hard links created manually to the current directory, then there are even more.
For example, \\

\begin{verbatimtab}[2]
/
	bin/
	etc/
	home/
		cbkihong/
\end{verbatimtab}

The \mono{/home} directory inode has 3 hard links pointing to it. However, there are 5 for \mono{/}, that is because
there is no parent directory for \mono{/}, and for consistency the ``..'' in \mono{/} also points to itself. Together
with ``.'' and ``..'' from the 3 subdirectories, the hard link count is therefore 5. \\

Symbolic links are more widely used because symbolic links only stores an alternative path name and can be used
provided the alternative path is accessible from the directory tree. That is, symbolic links may point to a destination
that is on a different filesystem. For example, a symbolic link on my Linux reiserfs partition may point to a 
file on the Windows FAT partition, which does not even have the notion of hard links (did I tell you that you cannot
create a hard link on a mounted Windows partition at all?). On the other hand, hard links cannot cross filesystems,
and they are only supported on Unix-compatible filesystems, so their areas of application are rather limited. \\ 

\subsection{Permission and Ownership}

Because Unix is from the ground up a multi-user operating system, a permission and ownership system has to be
in place to control who have access to resources and how they can access them. In Unix filesystems, every file
has an owner, the user who created the file on the filesystem. Apart from the owner, each file is associated
with a group. The \mono{ls} command  with the \mono{-l} switch causes the directory listing to be printed in 
the long format, with the name of the owner and group printed in the third and fourth column, respectively. 
The first column is the access permissions. In this column, the first character indicates the file type.
Customarily, regular files are represented by a hyphen ``-'', directories by ``d'' and symbolic links by ``l''. \\

\begin{lstlisting}[numbers=none,backgroundcolor=\color{CodeBackground}]
cbkihong@cbkihong:~/docs/perltut$ ls -l
drwxr-xr-x    2 cbkihong users         477 2003-06-20 14:25 images
-rw-r--r--    1 cbkihong users      657800 2003-06-20 14:28 perltut.pdf
-rw-r--r--    1 cbkihong users     4012769 2003-06-20 14:28 perltut.ps
-rw-r--r--    1 cbkihong users        3887 2003-06-19 19:17 perltut.tex
\end{lstlisting}
\addvspace{\baselineskip}

In this example, the files have the owner ``cbkihong'' and belong to the group ``users''. To control how different
users can access the files, three sets of permission bits are assigned to each file which specify permissions that
are given to the owner, group members and everybody else. The permission values are reflected by the first column of
the file listing obtained by the \mono{ls} command above. The first character indicate the type of the file.  The remaining nine characters represent
the permission values. Here is a summary of what each character means with respect to a directory and a regular file: \\

\begin{table}[htbp]
\begin{center}
Regular File \\
\begin{tabular}{ccl}
\hline
\textbf{Bit}	& \textbf{Value}	&	\textbf{Meaning}	\\
\hline 
	\texttt{r}	&	4	& Read file content \br \\
	\texttt{w}	&	2	& Modify the file content \br \\
	\texttt{x}	&	1	& Execute the file \br \\
\hline 
\end{tabular}
\\[\baselineskip]
Directory \\
\begin{tabular}{ccl}
\hline
\textbf{Bit}	& \textbf{Value}	&	\textbf{Meaning}	\\
\hline 
	\texttt{r}	&	4	& Read directory listing \br \\
	\texttt{w}	&	2	& Create/delete files in the directory \br \\
	\texttt{x}	&	1	& Enter the directory \br \\
\hline
\end{tabular}
\end{center}
\caption{Filesystem Permission values}\label{tab:chmod}
\end{table}

Let's take the first two entries in the example listing above as an example. For the ``images'' directory, we 
divide the permission values into three sets: \\

\begin{verbatim}
file type  owner  group   everybody else
    d       rwx    r-x         r-x 
\end{verbatim}

Everybody on the system can read the directory listing and enter the directory. However, only the owner ``cbkihong''
can add new files or remove files from the directory. To read the directory listing means you get a list of names 
that represent the files (including subdirectories and symbolic links etc.) in the directory. For example, if you 
enable the ``x'' bit but not the ``r'' bit, the command \mono{ls images/} will fail because this operation involves getting the
directory listing. However, the command \mono{cat images/README.txt} will be successful if the file \mono{README.txt} exists
in the ``images'' directory because directory listing is not involved in the operation. On the other hand, if the 
``r'' bit is enabled but not the ``x'' bit, you can see the list of files in the directory, but you cannot access 
them. Changing into the directory with the command \mono{cd} will also fail. Because such a combination 
of permission bits is somewhat nonsense in practical use, for directories usually the ``r'' and ``x''
bits go together --- either you enable or disable both of them, but not enabling one and not the other. \\

Unix groups are not frequently used in practice, but they can be good for sharing of files among users on 
the system in a simple way. For example, a file server in a company may have a ``managers'' group whose members
consist of managers from all departments. The server may have a directory called ``reports'' containing reports
prepared by the managerial for all staff. If the directory is not in the \mono{/home} tree, it is very likely owned by
the system administrator (root). However, the administrator may set the group to ``managers'' and set the group
``w'' bit to allow the managers to put their reports into the directory, while other staff users only have read
access. A possible configuration is shown below: \\

\begin{lstlisting}[numbers=none,backgroundcolor=\color{CodeBackground}]
drwxrwxr-x    2 root  managers         477 2003-07-20 14:38 reports
\end{lstlisting}
\addvspace{\baselineskip}

For the ``perltut.pdf'' file in the sample file listing above, the permission values are as follows: \\

\begin{verbatim}
file type  owner  group   everybody else
    -       rw-    r--         r-- 
\end{verbatim}
\addvspace{\baselineskip}

That means everybody can read the file, but only the owner can modify it. Some people have misconceptions on Unix permissions
that one needs directory write permission to modify a file in the directory. The fact is only the file write permission
is needed. Also, some may think to delete a file one needs writable permission to that file. Only the directory
write permission is needed in this case. If you understand my description above fully, you are not going to make 
these kinds of mistakes. \\

Note that internally (for example, in the inodes) owners and groups are represented by numbers instead of names 
such as ``cbkihong'' and ``users''. That is because storage of integers uses less space compared with names. On
most systems, the \mono{/etc/passwd} stores the mapping between user ID and username, while \mono{/etc/group} stores
the mapping between group ID and group name. They are used to resolve the mnemonic names for display as output of
commands such as \mono{ps} and \mono{ls} etc. \\

%poses no restriction to the administrator, who 

In Table~\ref{tab:chmod} you see a column with the heading ``Value''. Internally, to store the permission values in
a more compact form the permissions are encoded as a number. Take the ``images'' directory as an example. The
permission value ``drwxr-xr-x'' may be converted to numeric representation as follows: \\

Owner (rwx): $4 + 2 + 1 = 7$ \\
Group (r-x): $4 + 1 = 5$ \\
Everybody else (r-x): $4 + 1 = 5$ \\

As already described, the character ``d'' is only an indication that this is a directory, and it's not a permission value.
Therefore, the numeric representation is 755, by concatenating the permission values for the owner, group and everybody
else. Please note that permission values of symbolic links are not used in practice, and are set to 777 (lrwxrwxrwx) by
default. \\

%However, there are 3 extra permission bits that can be applied to files. They
%are seldom needed, and their use should be avoided unless with good reason. These bits will be briefly covered 
%in a later section. \\

\subsection{Processes}

Once executable permission is applied to a file, it can then be executed by the system. A program in execution is
called a process. Each process has an owner and is associated with a group, similar to the case of files on a 
filesystem. Each process is associated with four identifiers. Apart from the user ID and group ID, a process also
has an \term{effective user ID} and an \term{effective group ID}. In general, when an executable file is being 
executed, the user ID is that of the user who executed the program, and the group ID is that assigned to the user
when the user account was created. In general, the effective user ID is the same as user ID, and the effective group ID is 
the same as group ID. They are different only if the file being executed has either the setuid or setgid bit set,
which are two additional permission bits that are useful only to executable files and will be described in the
next section. They are seldom needed, and their use are usually not justified unless with good reasons. \\

The enforcement of an ownership system on processes prevents unauthorized users from modifying the state of the
processes, for example, to terminate them. In general, only root or the users whose user ID matches the user ID
or effective user ID of a process are allowed to change the state of a process. You can change the state of a
process by sending it a \term{signal}, that is, a message sent from the operating system kernel to a process.
From a user's point of view, a process can be in one of several states: running, suspended or terminated. We can
use the \term{kill} command to send a signal to a process. The signals that are most frequently used include 
\mono{SIGHUP} (1), \mono{SIGINT} (2), \mono{SIGKILL} (9), \mono{SIGSTOP} (19) and \mono{SIGCONT} (18). The 
way \mono{SIGHUP} is handled is process-specific. It is generally widely supported that when a daemon process 
receives this signal, it rereads its configuration files. This is convenient for system administrators to 
effectuate changes made to the configuration files without restarting the daemon process. The \mono{SIGINT} 
signal is what is sent to a process when the user presses Ctrl-C. In most cases, the process is terminated. 
However, some processes are defined to catch the signal and thus prevent it from being terminated. For example, 
Ctrl-C is a combination key defined in emacs, so it has to catch the signal. In this case, you may try to send
the \mono{SIGTERM} (15) signal. Runaway processes can generally be abruptly terminated by the \mono{SIGKILL} 
signal. The \mono{SIGSTOP} signal causes the process to be suspended. Both \mono{SIGKILL} and \mono{SIGSTOP} 
cannot be caught by any processes, and therefore provided you have the permission to change the state of the
process these two signals should succeed, at least in theory. However, there are a few occasions when the
process, or even other parts of the operating system are ignoring these signals. This is a sign of inauspiciousness
and you are advised to restart your system if this happens to you (but probably by then it is already too late).
For a suspended process, you can put it back to running state by sending it a \mono{SIGCONT} signal. You can
send a signal using \mono{kill} in several forms, as shown in the following examples: \\

\begin{verbatim}
kill -SIGHUP 826
kill -HUP 826
kill -1 826
\end{verbatim}

You can use the name of the signal or the signal ID to specify the signal to send to the process. Because all
Unix signal names actually start with the prefix ``SIG'', you may omit this prefix to reduce the amount of typing.
The last argument to \mono{kill} is the process ID. Each process has an ID to uniquely identify a process. 
You should check the process ID by using the \mono{ps} command, which is displayed on the far left. Most
systems also have a \mono{killall} command which frees you from the need of looking up the process ID. For example, \\

\begin{verbatim}
killall -SIGINT myprog
\end{verbatim}

which tries to terminate all instances of the program myprog. However, \mono{kill} is a more reliable choice
on some Unix variants, or where \mono{killall} is not available. \\

A side note about the executable and read permission of executable files. I have recently seen a question raised
on a Unix forum about making a shell script executable but not readable by other users on the system. The answer
is that is not possible to use the permission 711 (rwx--x--x). There are two main types of executable programs.
Either they are compiled into an executable object code (binary) format that can be executed directly (e.g., compiled
programs written in C), or shell scripts (e.g. perl or sh scripts). If the executable is in an object format
supported by the operating system kernel, it can be directly loaded and executed by the kernel. Today two main
binary formats are supported on Unix, namely \term{a.out} and \term{ELF} (Executable and Linkable Format). a.out
is well supported, but is quickly replaced by ELF on many Unix platforms. In the case of executable scripts, on 
the other hand, the kernel needs to load the interpreter instead, and the interpreter executes the script instead.
To execute the script, the interpreter needs to have read access of the script. That's why you can't use 711 as
the script permission if everybody needs to execute it. This works for compiled programs, though. \\

\subsection{The Special Permission Bits}

% TODO: setuid, setgid, sticky
Apart from the 9 permission bits that can be set for every file, there are three special option bits that 
may be applied, two of which pertain to executable files. These bits are known as \term{setuid}, \term{setgid} and \term{sticky}
respectively. They may be set using the \mono{chmod} command. \\

If you are granted access to a Unix shell account, possibly one of the first commands you will be asked to use is 
\mono{passwd}. This command lets you change your password. On some Unix platforms, some other functionalities are also
available, for example, to change the login shell. Traditionally on Unix, user information are stored in the file
\mono{/etc/passwd}, containing the username, user ID, home directory, login shell and an encrypted password of users.
As encrypted passwords in traditional Unixes are well known to be easy to break, today many systems are configured
to use \term{shadowed passwords}, whose encrypted (strictly speaking, hashed by a cryptographically strong algorithm
such as MD5) passwords are stored in a separate file \mono{/etc/shadow} which is only readable by root, in contrast
to \mono{/etc/passwd} which is readable by everybody. The \mono{passwd} file has to be readable by everybody because
its file content is read by a number of standard Unix programs, such as \mono{finger}, which can be run by all users
on the system to display information about the user (it may also be executed remotely, but it's usually disabled to 
minimize security risks). The \mono{passwd} file is owned by root. Have you wondered how the \mono{passwd} program
which is executed by a regular user can modify this file which is only writable by root? The fact is, the \mono{passwd}
is one of the few programs in a standard Unix installation that has the setuid bit set, which allows it to take on the
privilege of the root user. \\

When an executable with the setuid bit set is being executed, the effective user is set to the owner of executable
instead of the user who executed the program. Similar is the case for the setgid bit. Therefore, essentially, the 
setuid and setgid bits are used to escalate the privileges of the executable. It is also for this reason that 
setuid and setgid bits should not be set unless with strong reasons, and programs with these bits on should
be examined closely to ensure they do not have any inherent vulnerabilities that may be taken advantage by attackers.
For Perl programs, for example, the program \mono{suidperl} which is itself a setuid version of the perl interpreter
is actually used to execute a Perl program with the setuid or setgid bit set. As mentioned in Chapter~\ref{chap:eval},
runtime taint check is performed automatically. An executable may be set setuid or setgid using the \mono{chmod} command: \\

\begin{verbatimtab}[4]
chmod u+s executable	# setuid
chmod g+s executable	# setgid
\end{verbatimtab}

For example, \mono{passwd}, a setuid program shows up as this: \\

\begin{lstlisting}[numbers=none,backgroundcolor=\color{CodeBackground}]
-rwsr-xr-x    1 root     root        24376 2003-09-14 05:53 /usr/bin/passwd
\end{lstlisting}
\addvspace{\baselineskip}

The sticky bit serves a totally different purpose. Many programs in execution need to create large number of temporary
files. In order to avoid cluttering the home directory of the respective users, they are customarily created under
the \mono{/tmp} tree. A check of the \mono{/tmp} directory shows that it is owned by root, and the group is root.
Here is the entry displayed by \mono{ls -l /}: \\

\begin{lstlisting}[numbers=none,backgroundcolor=\color{CodeBackground}]
drwxrwxrwt    8 root     root         4096 2003-09-22 14:47 tmp
\end{lstlisting}
\addvspace{\baselineskip}

Because the directory is writable by everybody (the w bit is enabled for others), programs executed by regular users 
can use it as a swap space to create temporary files. However, this also creates a problem. Note that while any regular user
may create anything in this directory, they may as well remove anything in this directory including files and directories
created by another user. A malicious user may keep on removing other users' files to inconvenience them. The workaround
is a sticky bit applied on the \mono{/tmp} directory. If the sticky bit is enabled, the system will not allow you to
modify any entries in the directory unless you are the owner specified at the entry, or the root user. You may enable
sticky mode by the command shown below: \\

\begin{verbatim}
chmod +t somedir/
\end{verbatim}

t stands for a sticky bit. You can see from above that it appears instead of x in the listing of \mono{/tmp}. Here is an example
of some entries in \mono{/tmp} on my Debian Linux system: \\

\begin{lstlisting}[numbers=none,backgroundcolor=\color{CodeBackground}]
cbkihong@cbkihong:/$ ls -l /tmp
-rw-r--r--    1 cbkihong cbkihong        0 2003-09-22 14:25 AcroEiBBIX
drwx------    2 cbkihong cbkihong     4096 2003-09-22 14:23 kde-cbkihong
\end{lstlisting}
\addvspace{\baselineskip}

Unless you are the owner, that is ``cbkihong'', you cannot rename or remove these two entries. \\
