\chapter{Files and Filehandles\label{chap:files}}

\section{Introduction}

The ability to read from and write to files is nearly always essential to computer programs. Output is frequently 
generated in the course of execution of a program. However, if they are not stored in secondary storage media such
as disks, they will disappear once the power is switched off. Therefore, file access is an important element of the
input/output system. In this chapter, we will explore the general Perl input/output system and the functions we
can use to access the filesystem. \\

Similar to the C standard and to be in line with Unix concepts, Perl uses the concept of \term{filehandles} to 
represent an opened file. They are also known as \term{file descriptors} in programming languages like C. Although I 
did not mention explicitly, actually you have been working with filehandles throughout the tutorial. The statement \\

\begin{verbatim}
print "Hello World!";
\end{verbatim}

Appeared in the very first Perl program covered in this tutorial, it implicitly uses a filehandle to get the string
\mono{print}ed on the screen. This statement is actually written as \\

\begin{verbatim}
print STDOUT "Hello World!";
\end{verbatim}

However, the filehandle \mono{STDOUT} is assumed by default, so we have been omitting it all the way up to this 
chapter. Unix has a generalized view of input and output. Apart from files, hardware devices are also represented
as files on a Unix filesystem. Therefore, input and output of hardware devices can be represented by input and output
of files, through a filehandle. This generalized view helps abstract a user or programmer from the peculiarities
of the implementation of each device, which are left to the device drivers that exist in the operating system kernel, 
allowing users to interact with each device in a uniform manner. With a strong Unix tradition, Perl adopts a similar
notion as well. Even if you don't use any flavours of Unix at all it does not mean you don't have to read this chapter,
because Perl uses the same concepts on every platform it runs on, so that a considerable degree of platform 
independence can be achieved. \\

\section{Filehandles}

Perl has a number of predefined filehandles, namely \mono{STDIN}, \mono{STDOUT} and \mono{STDERR} that you can 
use in command line applications for input/output on screen. \mono{STDIN}, or standard input, is the filehandle 
from which keyboard input can be read. \mono{STDOUT}, or standard output, is where you should send the output
of your program. \mono{STDERR}, or standard error, is mostly used for outputting warning or error messages because
sometimes people wouldn't like error messages to be displayed. By dumping error messages to a separate filehandle
the user may decide whether to display them or instead redirect them to, for example, a log file on disk for
diagnosis at a later time. These three filehandles are always ready for you to use and you don't need to create
them manually. \\

Filehandle is one of the data types available in Perl. As you have learnt in the chapter on references, filehandle
has a separate slot in a symbol table entry. However, unlike other data types you do not have to prefix a filehandle
with a type symbol. By convention, filehandles are in all capital characters to make them visually stand out from
names of functions and subroutines etc. \\

\subsection{\mono{open} a File}

Unless you are working on one of the predefined filehandles, the first step you should take is to populate a 
filehandle. A populated filehandle represents an active \term{stream} which allows input or/and output of data. 
For the case of file access, a populated filehandle represents an \mono{open}ed file, which is then used by the
input/output functions to read from or write to the file. Filehandle is also used in Perl socket programming 
which is used to represent a socket. However, due to time constraints network programming is not covered in 
the first edition of this tutorial. \\

Unix supports two sets of file access functions. One set is provided by the operating system kernel, and the
other set is provided by the standard C libraries installed on the system, which is implemented on top of the
version provided by the kernel. The file access functions in Perl are actually interfaces to these functions.
Perl allows access to both, through the \mono{open()} function which invokes the version provided by the C
libraries, and the \mono{sysopen()} function invokes the operating system version. The use of \mono{open()}
is generally preferred to \mono{sysopen()}, because it is simpler to use, but \mono{sysopen()} is more powerful. 
\mono{sysopen()} is not covered in this edition of the tutorial. \\

You use the \mono{open()} function to open a file. Usually, the \mono{open()} function takes on one of the 
following forms: \\

\mono{%
\textbf{open} FILEHANDLE, EXPR \\
\textbf{open} FILEHANDLE, MODE, EXPR  \\
}

\mono{FILEHANDLE} is either a filehandle or a lexical variable with the \mono{undef} value, which is used 
by the \mono{open()} function to store the reference to the filehandle created. \mono{EXPR} is a scalar
expression which contains the name of file to \mono{open()}, and \mono{MODE} describes the access mode 
to apply to the file, for example, whether read or write access are allowed on the file. If \mono{MODE}
is missing, it defaults to ``\mono{<}'', the read-only mode. Otherwise, \mono{MODE} should be prepended 
to \mono{EXPR} in the first form, if it is not given as a standalone argument. If \mono{open()} is successful, it returns
a nonzero value. Otherwise, \mono{undef} is returned. You should always check the return value of file access
functions and handle cases of failure to ensure your program is fault tolerant, especially if your program
is to be used by other people instead of you, such as CGI scripts. \\

Traditionally, a filehandle is usually used instead of a lexical filehandle reference. For example, \\

\begin{verbatim}
my $retval = open(LOG, "<command.log");
\end{verbatim}

which creates \mono{LOG} permanently on the symbol table of your current package. This may be acceptable
to you, but you may wish to \mono{local}ize it to a certain subroutine, for example. As a recapitulation,
this is the way to do it: \\

\begin{verbatim}
local *LOG;
my $retval = open(LOG, "<command.log");
\end{verbatim}

However, because filehandles themselves cannot be lexical, and many Perl programmers are not familiar with 
the use of typeglobs, a better way would be to use a lexical filehandle reference, which you can easily pass
around without needing any knowledge in typeglobs. An example is \\

\begin{verbatim}
my $fhLOG;
my $retval = open($fhLOG, "<command.log");
\end{verbatim}

Here is a summary of the 6 access modes provided: \\

\begin{table}[htbp]
\begin{center}
\begin{tabular}{ll}
\hline
\textbf{\texttt{MODE}}	& \textbf{Description} \\
\hline 
	\texttt{<} &	Read-only access. Specified file must exist. \br \\
	\texttt{+<} &	Read-write access. Specified file must exist. \br \\
	\texttt{>} &	Write-only access. File emptied	if exists; created otherwise. \br \\
	\texttt{+>} &	Read-write access. File emptied	if exists; created otherwise. \br \\
	\texttt{>>} &	Append-only access (file pointer at end-of-file). File created if not exist. \br \\
	\texttt{+>>} &	Read-Append access (file pointer at end-of-file). File created if not exist. \br \\
\hline 
\end{tabular}
\\[\baselineskip]
\end{center}
\caption{File Access Modes for \mono{open()}}\label{tab:iomode}
\end{table}

These access modes determine the operations that can be applied on a file. If you \mono{open()} a file
with read-only access but you try to write data to the corresponding filehandle, a runtime error will
occur. \\

Every open file has a \term{file pointer}, which determines the position of the next character read or
write. The first four modes listed above position a file pointer at the beginning of a file, so
that read/writes occur at the beginning of the file. \mono{<} grants only reads access to the file.
\mono{>} grants only write access to the file, which is automatically created if it does not exist, 
and empties it before writing. Both \mono{+<} and \mono{+>} grants read-write access
to the file, so you may read from as  well as write to the filehandle. The difference between \mono{+<} 
and \mono{+>} is that for \mono{+<}, the specified file must exist, while for \mono{+>} the file is
automatically created if it does not exist, and empties the content before writing. The last two modes
place the file pointer at the end of a file. Therefore, data are written at the end of the file. A
file opened with either of these two modes is created where necessary. \\

Note that \mono{+<} does in-place writing. If file write occurs in the middle of a file, it simply overwrites
the exact number of characters from the file pointer required by the write, growing the file as necessary and other characters in the file
are unaffected. The effect is similar to putting your text editor in ``replace'' or ``overwrite'' mode and typing 
characters at a cursor to overwrite the old text. \\


\subsection{Output Redirection}

Output redirection allows you to redirect output sent to a certain filehandle to another filehandle.
This is frequently used by shell script authors on Unix systems to redirect error messages to log files
or simply to throw them away as if they have not been output at all. \\

To use I/O redirection, specify ``\mono{>\&}'' as the \mono{MODE}, and \mono{EXPR} is the name of the filehandle
to which output to \mono{FILEHANDLE} is redirected. An example is shown below, which redirects output that are 
sent to \mono{STDERR} to a file that has been previously \mono{open}ed with the filehandle \mono{LOG} instead. \\

\begin{verbatim}
open (STDERR, ">&LOG");
\end{verbatim}

Text that are sent to \mono{STDERR} will be redirected to a file instead, so they are no longer output on screen. \\

%\begin{itemize}
%\item \mono{open()} the file to populate the filehandle \\
%\item 
%\end{itemize}

\section{File Input and Output Functions}

In this section, I will introduce to you various functions you can use to read from or write to a filehandle. 
Note that in functions expecting a filehandle as its argument you can use a lexical filehandle reference instead of a 
typeglob. Simply replace the filehandle with the lexical variable, for example, \mono{\$fhLOG}. \\

\subsection{\mono{readline()} --- Read A Line from Filehandle}

The \mono{readline()} function accepts a typeglob as parameter to read a line from the filehandle contained in
the typeglob. In scalar context, each invocation of \mono{readline()} reads up to the newline character. When
no more lines can be read, \mono{undef} is returned. An example is shown below, which copies a text file 
File1.txt to File2.txt. \\

\begin{verbatimtab}[4]
open FILE1, "<File1.txt" or die "Cannot open File1.txt!";
open FILE2, ">File2.txt" or die "Cannot open File2.txt!";
my $line;
while ($line = readline(FILE1)) {
	print FILE2 $line;
}
close FILE1;
close FILE2;
\end{verbatimtab}

However, customarily \mono{readline()} is not frequently
used because Perl provides an operator \mono{<FILEHANDLE>} which is an interface of \mono{readline()}. We can
replace \mono{readline(FILE1)} above with \mono{<FILE1>}. \\

In list context, both \mono{readline()} and \mono{<FILEHANDLE>} read all the way until end-of-file occurs, 
and split it into lines. The return value is an array with its elements being the lines extracted.
This is seldom used in practice, because if the incoming file is very large, the generated array will
also be very large, consuming a lot of memory space. Therefore, it is a lot safer to set up a loop to read in line by
line as shown above. \\

\subsection{\mono{binmode()} --- Binary Mode Declaration}

Not knowing if you are aware or not, text files actually are stored differently on Unix and MS-DOS/Windows 
systems. The culprit is that the line termination characters used on these platforms are different. That's
why there is an ASCII/Binary transfer mode option in your FTP application. Binary files do not rely on line
termination characters to denote the end-of-line. Therefore, a binary file is represented in an identical 
manner on these platforms. However, because text files uses line termination characters, and they vary from
platform to platform, two files with identical textual content end up being represented differently on different systems. \\

To ensure Perl programs have high levels of portability, in general Perl programmers simply have to treat 
\mono{\textbackslash n} as the line termination character. This is what we have readily assumed from the
very beginning of this tutorial, and the underlying system carries out all necessary conversions for us
automatically. However, this system does not work for MS-DOS/Windows systems because they distinguish between
text files and binary files. Therefore, on these systems Perl needs to introduce a PerlIO layer on top of the native 
file access functions which converts between the underlying line termination characters and \mono{\textbackslash n}.
This does not pose any problems for text files, as this conversion is actually intentional for text files.
However, binary files should never be altered in any way. \mono{binmode()} with just a single parameter of 
\mono{FILEHANDLE} essentially instructs all read/write
through the filehandle to bypass the conversion layer. Because \mono{binmode()} is ignored on other systems,
you should generally use it on all binary files for portability. It should be invoked right after a file 
is \mono{open}ed, i.e. \\

\begin{verbatim}
open BMP, "<logo.bmp";
binmode BMP;
\end{verbatim}

If \mono{binmode()} has two parameters, the second parameter indicates the PerlIO layers to apply which act
as conversion filters. The ``crlf'' layer is the layer we mentioned above for MS-DOS compatible systems which
carries out line termination conversions. Usage of this form is generally not needed as the defaults are generally adequate
for most programming purposes, and so are not described here. \\

\subsection{\mono{read()} --- Read A Specified Number of Characters from Filehandle}

The syntax of \mono{read()} is \\

\mono{%
\textbf{read} FILEHANDLE, SCALAR, LENGTH\textcolor{optionalargs}{[, OFFSET]} \\
}

which reads from \mono{FILEHANDLE} \mono{LENGTH} characters, usually equivalent to bytes and assign it to 
the scalar \mono{SCALAR}. If \mono{OFFSET} is given, it specifies the zero-based offset of \mono{SCALAR}
from which to start writing. \\

This is usually used for binary files, but not necessarily. The file copying program shown above should generally
not be used because binary files are not line oriented, and it does not use \mono{binmode()} which causes file
copying errors on MS-DOS compatible systems. Presumably the correct way is as follows: \\

\begin{verbatimtab}[4]
sub copy ($$) {
	my ($src, $dest) = @_;
	open FILE1, "<$src" or die "Cannot open $src!";
	open FILE2, ">$dest" or die "Cannot open $dest!";
	binmode FILE1;
	binmode FILE2;
	my ($buffer, $numChars); my $bufferSize = 1024;
	my $size = 0;
	while ($numChars = read(FILE1, $buffer, $bufferSize)) {
		$size += $numChars;
		print FILE2 $buffer;
	}
	close FILE1;
	close FILE2;
	return $size;
}
\end{verbatimtab}

\subsection{\mono{print()/printf()} --- Output To A FileHandle}

We have used \mono{print()} quite a lot so it is not worthwhile repeating all the details here again. However,
if a filehandle is specified, it outputs to the filehandle. Otherwise, the filehandle defaults to \mono{STDOUT},
as I mentioned earlier in this chapter. \\

\mono{printf()} is similar to \mono{print()}. However, it is an exceptionally powerful function that lets you
perform varieties of type conversions on the fly. \mono{printf()} is an artifact from the C standard I/O library.
In C, generating a string is not a simple affair, because there is no variable interpolation as in Perl and there is 
not an easy and flexible way of string concatenation. Also, because C is strongly typed and you cannot concatenate
a C-style string with other data types, for example, a number, you have to end up performing a lot of type 
conversions before the desired string can be successfully generated and eventually written to a file descriptor.
Therefore, for convenience purpose C provides a set of functions collectively known as the \mono{printf()} family
of functions which allows generation of many common forms of string to be completed in one function call. \\

While this function is very versatile it is also acclaimed as the most complicated function in C. This function
works by constructing a concise but generally cryptic format string, which consists of the string with some placeholder
fields inserted. These placeholder symbols describe the type conversion operation required for each of the fields 
to be inserted into the string, which are passed as additional arguments to the \mono{printf()} function. 
\mono{printf()} is one of the several few C builtin functions which accept a variable number of arguments. 
The complexity lies completely in constructing the proper format string. While Perl has flexible string interpolation
and automatic type conversions, \mono{printf()} is not as important as in C. Moreover, because \mono{printf()}
involves additional operations, it is less efficient compared with \mono{print()}. Therefore, you should avoid
it if \mono{print()} suffices for the purpose. However, because it is indeed handy for certain kinds of conversion 
operations, I am going to describe it below. In Perl's implementation, another function \mono{sprintf()} exists. 
It returns the generated string instead of directing it to a filehandle. Otherwise, it is identical to \mono{printf()}. 
In fact, \mono{printf()} is internally implemented using \mono{sprintf()}.
This is intended to be an introductory description of most frequently used options only. For more information, please read the 
\link{http://www.perldoc.com/perl5.8.0/pod/func/sprintf.html}{perlfunc/sprintf manpage}. \\

The following \mono{sprintf()} example, however simple, gives you a taste of what it is like: \\

\begin{verbatimtab}[4]
# "Good Morning. The number is 6."
sprintf("%s. The number is %d.", "Good Morning", 6);
\end{verbatimtab}

The format string is just a string with placeholders inserted. Placeholders start with the \mono{\%} character.
Placeholders are replaced by the corresponding arguments given after the format string with the specified type
conversion operations performed. Here, because we have 2 placeholders, we have two additional parameters given
after the format string. \mono{\%s} indicates the placeholder value is a string, while \mono{\%d} indicates a signed
integer. Perl includes an extension to \mono{printf()} functions that you may specify the placeholder values in
an arbitrary order. That is, the arguments may be given in a different order than the order the placeholders 
appear in the format string. However, because this only complicates matters, I shall not discuss this and mandate
all arguments to be given in the same order that the placeholders appear in the format string. s and d here 
indicate the conversion type. Some other conversion types are defined as well. The most frequently used ones
are: \\

\begin{center}
\begin{tabular}{cl}
\hline
\textbf{Conversion Type} & \textbf{Description} \\
\hline
\mono{s} &	String \\
\mono{d} &	Signed integer \\
\mono{u} &	Unsigned integer (decimal) \\
\mono{x} &	Unsigned integer (hexadecimal, lowercase) \\
\mono{X} &	Unsigned integer (hexadecimal, uppercase) \\
\mono{f} & 	Floating point number  \\
\hline 
\end{tabular}
\\[\baselineskip]
\end{center}

Here are some examples: \\

\begin{verbatimtab}[4]
sprintf('%X', 12);	# C
sprintf('Absolute value: %u', -1);		# 4294967295 (see below)
\end{verbatimtab}

When using the unsigned integer conversion type you have to ensure the value passed as parameter is not negative. This concerns
the way an unsigned integer is represented in the underlying C library. All integers are represented with a fixed number of bits.
On my system, this is 32 bits. Therefore, for unsigned integers 32 bits can represent all integers from $0$ up to 
$2^{32}-1$, that is $4294967295$. Signed integers are also represented with 32 bits. The method of representation
of negative integers is known as ``2s complement''. The range of signed integers becomes -2147483648--2147483647. 
Negative integers are represented by inverting the bits of the corresponding positive magnitude and adding 1. 
Therefore, 1 and -1 are represented as follows: \\ 

\begin{tabular}{p{1.5cm}l}
1	&	\mono{00000000 00000000 00000000 00000001} \\
-1	&	\mono{11111111 11111111 11111111 11111111} \\
\end{tabular}
\vspace{\baselineskip}

When you pass -1 as argument while the conversion type is u, the bit sequence of -1 is interpreted as if it is an 
unsigned integer. Note that this bit sequence is identical to 4294967295 in unsigned integer representation. 
Therefore, the string returned is 4294967295. \\

%Note that Perl uses floating point arithmetic by default. 

Between \mono{\%} and the conversion type character additional options may be applied. We will dicuss some of them
below. You may mix and match them, although a few of them are mutually exclusive as otherwise noted. \\

You may stipulate a prepended space before a numeric conversion if the numeric value is not negative by inserting
a space. For example, \\

\begin{verbatimtab}[4]
sprintf('<% d>', 3);	# < 3>
sprintf('<% d>', -3);	# <-3>
\end{verbatimtab}

By replacing the space with a \mono{+} symbol, the \mono{+} sign is prepended instead of a space. Note that the 
space and \mono{+} are mutually exclusive. If both exists, \mono{+} is taken. For example, \\

\begin{verbatimtab}[4]
sprintf('<%+d>', 12);		# <+12>
sprintf('<%+f>', 13.6);		# <+13.600000>
sprintf('<% +f>', 13.6);	# <+13.600000>
\end{verbatimtab}

Once an argument has been converted to the string form, by default the corresponding placeholder is substituted
by the string without introducing any padding, as we have seen in the above examples. However, we may introduce
extra padding by specifying the minimum field width. It specifies the minimum length of the 
placeholder value string. If this number is larger than the length of the string, padding is added. By default 
the string is right-aligned so padding appears on the left. However, if the \mono{-} symbol is prepended to the
minimum field width, the string is left-aligned so padding appears on the right. The padding is in terms of spaces.
If the minimum field width is smaller than the string length required, then the field width is ignored.
For example, \\

\begin{verbatimtab}[4]
sprintf('<%+12f>', 13.6);		# <  +13.600000>
sprintf('<%+-12f>', 13.6);		# <+13.600000  >
sprintf('<%+-5f>', 13.6);		# <+13.600000>
\end{verbatimtab}

If the string is left-aligned (i.e. format string without the \mono{-} symbol) you may place a 0 anywhere before 
the field width to instruct zero padding instead of spaces. For example, \\

\begin{verbatimtab}[4]
sprintf('<%08d>', 12345);		# <00012345>
sprintf('<%0+12f>', 13.6);		# <+0013.600000>
sprintf('<%+012f>', 13.6);		# <+0013.600000>
\end{verbatimtab}

As you have seen, floating point numbers have a certain precision. The precision is influenced by the number of
decimal places. Subject to fixed storage space of floating point numbers, the precision is not
unlimited as in our usual mathematics. You may specify the precision by specifying the number of decimal places,
together with a prepending `.'. For example, \\

\begin{verbatimtab}[4]
sprintf('<%0+12.4f>', 13.6);	# <+000013.6000>
sprintf('<%12.4f>', 13.6);		# <     13.6000>
\end{verbatimtab}

If applied on a string, this specifies the maximum number of characters to fit, somehow like the Perl 
\mono{substr()} function. For example, \\

\begin{verbatimtab}[4]
sprintf('<%.4s>', 'abcde');		# <abcd>
sprintf('<%10.4s>', 'abcde');	# <      abcd>
sprintf('<%10.4s>', 'abc');		# <       abc>
sprintf('<%-10.4s>', 'abcde');	# <abcd      >
\end{verbatimtab}

\subsection{\mono{seek()} --- Set File Pointer Position}

\mono{%
\textbf{seek} FILEHANDLE, POSITION, WHENCE \\
}

You may use the \mono{seek()} function to set the position of file pointer in a file specified by \mono{FILEHANDLE}.
\mono{POSITION} is a signed integer indicating the new position, relative to the position indicated by \mono{WHENCE}.
\mono{WHENCE} is an integer which is either 0, 1 or 2 representing the beginning-of-file, the current file pointer 
position and the end-of-file respectively. However, because hard-coding these integer values is not semantically
obvious, we usually use the names of the corresponding constants available in the C standard I/O library instead.
These constants are defined in the Perl \mono{Fcntl} module. You can import the three constants by \\

\begin{verbatim}
use Fcntl ':seek';
\end{verbatim}

The constants are \mono{SEEK\_SET}, \mono{SEEK\_CUR} and \mono{SEEK\_END} respectively. After you have imported
them, you can use them directly in your programs. \mono{seek()} is frequently used with binary file access. Binary
files usually have their data stored in a certain format that allows efficient access of data. To achieve this, 
some fields are encoded and saved at fixed positions in a binary file, which you may access directly with the 
\mono{seek()} function. The new position is calculated as the sum of \mono{POSITION} and the base position indicated
by \mono{WHENCE}. Here are some examples: \\

\begin{verbatimtab}[4]
seek(FILE, 0, SEEK_SET);	# Jump to beginning of file
seek(FILE, 5, SEEK_CUR);	# Jump 5 bytes forward
seek(FILE, -1, SEEK_END);	# Jump to last byte of file
\end{verbatimtab}

\subsection{\mono{tell()} --- Return File Pointer Position}

The \mono{tell()} function returns the position of the file pointer, in bytes. This function returns meaningful
values only if used on an \mono{open()}ed file. The position is a zero-based offset from the beginning of the file specified
by the filehandle which is passed as the parameter. It is frequently used with \mono{seek()} to move the file pointer
about in a file for reading or writing. This is an example which deduces the size of a file by using \mono{seek()}
and \mono{tell()}: \\

\begin{verbatimtab}[4]
use Fcntl ':seek';

sub getFileSize ($) {
	my $filename = $_[0];
	local *FILE;
	open FILE, "<$filename" or return undef;
	seek(FILE, 0, SEEK_END);
	my $size = tell();		# last filehandle read is FILE
	close FILE;
	return $size;
}

my $filename = $ARGV[0];
print "$filename\t\t", getFileSize($filename), " Bytes \n";
\end{verbatimtab}

However, on a system that supports \mono{stat()} this can be satisfactorily replaced by \\

\begin{verbatim}
(stat($filename))[7]
\end{verbatim}


\subsection{\mono{close()} --- Close An \mono{open}ed File}

At the end, when you have finished working with a filehandle, you should \mono{close()} it. Simply pass the 
filehandle as the parameter to \mono{close()}. \\


\section{Directory Traversal Functions}

In this section, I will introduce to you various functions you can use to traverse the directory 
structure of your system. Note that you ought to be using \mono{File::Find}, which is easier to use
and more flexible. In fact, it uses the functions that we cover below. However, some simple directory 
traversal operations may be more efficient if you implement them directly. \\

Description of the functions is presented first. You will find a full example afterwards which uses
these functions to build a class which performs file search. \\

\subsection{\mono{opendir()} --- Open A Directory}

\mono{%
\textbf{opendir} DIRHANDLE, PATH \\
}

This function prepares the directory \mono{PATH} for subsequent directory traversal functions. If the directory
exists, a true value is returned, and populates \mono{DIRHANDLE} which is a filehandle. \\

\subsection{\mono{readdir()} --- Read Directory Content}

\mono{%
\textbf{readdir} DIRHANDLE \\
}

The \mono{readdir()} function reads the content of the directory referred to by \mono{DIRHANDLE}, which is
populated by the \mono{opendir()} function. In scalar context, each \mono{readdir()} invocation returns an 
entry in the directory. When there is no more entry, \mono{undef} is returned. This is similar to the behaviour
of \mono{read()} saw earlier. You may also get all the entries in one go by calling the function in list 
context, and an array containing all the entries inside will be returned. \\

\subsection{\mono{closedir()} --- Close A Directory}

When you have finished traversal of the directory concerned, you should \mono{closedir()} it. \\

\subsection{Example: File Search}

In this example, we will write a class that allows users to search for a file recursively in a directory tree
whose name matching a pattern specified by the user. The pattern is in the form of a regular expression for
convenience purpose. It is by no means versatile as \mono{File::Find}, but it will give you an idea of how
to use the directory traversal functions. \\

\head{EXAMPLE}{File Search}%
\lstinputlisting[backgroundcolor=\color{CodeBackground}]{src/files/FileSearch.pm}
\addvspace{\baselineskip}

The program below uses this module, getting the root of the search as well as the search pattern from user input,
and uses the module to get a listing of files found. Finally, the list of files together with the corresponding 
file sizes are displayed: \\

\lstinputlisting[backgroundcolor=\color{CodeBackground}]{src/files/search.pl}
\addvspace{\baselineskip}

The transcript of a sample session is shown below. It searches for files whose extension is .pm. Therefore, a list
of Perl modules in the directory tree is shown. \\

\begin{lstlisting}[backgroundcolor=\color{CodeBackground},numbers=none]{}
cbkihong@cbkihong:~/docs/perltut/src/files$ perl search.pl
Please specify the filename pattern by means of a regular expression:
>> ^.*\.pm$
Please specify the root of directory tree to be searched:
>> /home/cbkihong/docs/perltut
/home/cbkihong/docs/perltut/scrap/Number.pm (344 bytes)
/home/cbkihong/docs/perltut/src/ch06/Controller.pm (916 bytes)
/home/cbkihong/docs/perltut/src/ch06/Stats.pm (1264 bytes)
/home/cbkihong/docs/perltut/src/ch06/Stats2.pm (779 bytes)
/home/cbkihong/docs/perltut/src/ch06/Stats3.pm (425 bytes)
/home/cbkihong/docs/perltut/src/ch06/TestObject.pm (198 bytes)
/home/cbkihong/docs/perltut/src/ch06/Timer.pm (1011 bytes)
/home/cbkihong/docs/perltut/src/ch06/TrafficLight.pm (642 bytes)
/home/cbkihong/docs/perltut/src/files/FileSearch.pm (796 bytes)
\end{lstlisting}
\addvspace{\baselineskip}

The \mono{chomp()} function removes the trailing newline character from the variable containing an input 
string if one is present. Note that the argument must be an lvalue because it performs in-place editing
of the string instead of returning the modified string. \\

The \mono{FileSearch::find()} method is a recursive subroutine which takes a single parameter, that is 
the base directory in which to search for files matching the specified pattern, which is passed to the
\mono{FileSearch} object upon instantiation. The search itself is a \term{depth-first search}. That means 
at each level, for each directory entry the content of the directory is searched before the next directory
is searched. Note that we have to avoid . and .. in each directory. In Appendix~\ref{chap:unixprimer} in my 
discussion of hard links I explain what these two entries are. They refer to the current and the
parent directory, respectively. Because they go up instead of go down the directory tree, we should
not delve into them. \\

This program looks fine. However, this search implementation can cause the search to go into an infinite 
loop on Unix systems in a certain case. That happens if two \term{path segments} on a path point to the 
same inode. First, let us agree on the terminology first. A path like /a/b/c/d consists of four path
segments a, b, c and d. Now, let us assume b and d point to the same directory inode (for an exhaustive
explanation of inodes, please read Appendix~\ref{chap:unixprimer}). Now c is a directory inside b. However, c
is also a directory inside d because b and d are actually just two aliases representing the same inode.
Therefore, /a/b/c/d/c is still a valid path, and we can find the directory entry d inside, so we have the
path /a/b/c/d/c/d. It is obvious that this circularity will continue endlessly, and obviously you will go
into an infinite loop as a result. This is exactly the reason why we have to avoid delving into . and .. 
in the program. Therefore, this problem normally should not occur. It only arises when a hard link is created
manually that makes two directories point to the same inode. The core difficulty is that you cannot easily 
identify this circularity in the program because d appears exactly like any other directory entries.
You are hereby asked to think of a way to patch this potential problem. \textit{(hint: use stat())} \\

% solution: record inode # for each path element and reject if already exist

\section{File Test Operators}

Perl provides you with a set of file test operators that you can use to test a file against certain
properties and return a truth value. Table~\ref{tab:filetest} lists the most commonly used operators: \\

\begin{table}[htbp]
\begin{center}
\begin{tabular}{ll}
\hline
\textbf{Operator}	& \textbf{Description} \\
\hline 
	\texttt{-r} &	File is readable by effective user or group. \br \\
	\texttt{-w} &	File is writable by effective user or group. \br \\
	\texttt{-x} &	File is executble by effective user or group. \br \\
	\texttt{-o} &	File is owned by effective user. \br \\
	\texttt{-R} &	File is readable by real user or group. \br \\
	\texttt{-W} &	File is writable by real user or group. \br \\
	\texttt{-X} &	File is executble by real user or group. \br \\
	\texttt{-O} &	File is owned by real user. \br \\
	\texttt{-e} &	File exists. \br \\
	\texttt{-z} &	File is empty (zero size). \br \\
	\texttt{-s} &	File is not empty, size in bytes as return value. \br \\
	\texttt{-f} &	File is a regular file. \br \\
	\texttt{-d} &	File is a directory. \br \\
	\texttt{-l} &	File is a symbolic link. \br \\
	\texttt{-u} &	File has setuid bit set. \br \\
	\texttt{-g} &	File has setgid bit set. \br \\
	\texttt{-k} &	File has sticky bit set. \br \\
\hline 
\hline 
\end{tabular}
\\[\baselineskip]
\end{center}
\caption{File Test Operators}\label{tab:filetest}
\end{table}

Note that many of these options are specific to Unix operating systems. \mono{-e}, \mono{-f}, \mono{-d},
\mono{-z} and \mono{-s} can be used on most operating systems. The only parameter is either a filename
or, in case the file has already been \mono{open}ed, its filehandle. For example, to ensure a certain
regular file (\mono{abc.txt}) exists, you can issue the following statement: \\

\begin{verbatimtab}[4]
(-e "abc.txt" && -f "abc.txt") 
	or die "abc.txt not found or is not regular file!\n";
\end{verbatimtab}

Notice there are two sets of operators to test whether a file is readable/writable or executable.
In general, you should use the lowercase version instead of the uppercase version because the lowercase
version uses the effective user or group identity, which always reflect the identity of the user or group
under which the program is executed.
You can get further information on effective and real user and group in Appendix~\ref{chap:unixprimer}
with respect to setuid or setgid files. \\

\section{File Locking}

%manifestation

File locking, or locking in general, is just one of the various solutions proposed to deal with problems associated 
with \term{resource sharing}. Sharing of resources frequently arises in our everyday life. Driving on a highway,
boarding an elevator etc. are all manifestations of utilization of shared resources. In a computer program, you ---
the developer --- have access to disks, peripheral devices etc. within a program that appears to be self-contained,
which in turn leads you into thinking your program has exclusive access to these resources. No, you don't. 
Resource sharing is equally ubiquitous as, if not more than, in your daily life. In your computer system there
may be a few hundred programs in execution (\term{processes}). Many of them are hidden so you may not know they are running. However, 
with just one Central Processing Unit (CPU) only one of them can be executed at any instant. Therefore, the CPU
actually performs a \term{context switch} at regular, but very short intervals when a currently-running process
is suspended from execution by the CPU with its execution state saved, and another process is resumed. This
context switch is performed so frequently that to a user there is a perception that all processes are executed
concurrently, but they're not. This is already one example of resource sharing, and the solution is to introduce
fine-grained \term{time slicing}. Similarly, all programs that are executing have access to the disks. Therefore,
it is not surprising that when a program is accessing a disk it is possible you can find another one that is also
trying to access the disk at the same time, too. \\

File locks are introduced to set temporary restrictions on certain files to limit how they can be shared among
different processes. Depending on the nature of an operation, we come up with two types of locks. The first type
is a shared lock, and another one is an exclusive lock. With respect to files, read access can be shared by multiple
processes because read access does not result in any changes to the state of the shared resource. Therefore, a
consistent view of the shared resource can be maintained. Write access, however, should by nature be carried out
with exclusive access until the write operation is complete. Another write operation to the file which occurs before
the current one is complete should be queued (certain cases may require it be cancelled instead, depending on the
nature of the operation). A read access cannot be concurrent with a write access either because the write access will 
destroy the consistent view expected by the read access while it is still reading (for example, the write operation 
may be to delete the file altogether, while the read operation is still in its midway). Therefore, the solution is
to introduce shared locks for read access, while exclusive locks for write access. Multiple concurrent shared locks 
are allowed. However, exclusive locks are mutually exclusive. \\

On most Unix systems as well as platforms in the Microsoft Windows NT family (including Windows 2000, XP and Windows
Server 2003 as of this writing) you may use a handy \mono{flock()} function. Please see the ``Function Implementations''
section of the perlport manpage to see any platform-specific issues for your platform. Here is an example demonstrating
file locking: \\

\lstinputlisting[backgroundcolor=\color{CodeBackground}]{src/files/writer.pl}
\lstinputlisting[backgroundcolor=\color{CodeBackground}]{src/files/reader.pl}
\addvspace{\baselineskip}

This example consists of two programs, namely the writer and the reader. The writer program writes some sample
data into a data file, while the reader program reads from the data file previously written data. The writer 
takes two command-line parameters which determine the values to be written to the data file. It first acquires
an exclusive lock. When the lock is acquired, it starts writing the values to the file. When the write operation
is complete, the lock is released. The reader works in a similar manner. It first acquires a shared lock, and
when one is acquired it starts reading from the file. At the end, the shared lock is released. To try this
example, open two or more commnad-line windows on your desktop environment and try different combinations,
such as a writer in a window and a reader in another (writer-reader), reader-reader and writer-writer. Take 
the writer-reader as an example, when you execute the writer program in one window by \\

\begin{verbatim}
perl writer.pl 1 15
\end{verbatim}

That means 15 lines of records will be written into the data file. You will see that there is a 1-second pause
between each write, because of the \mono{sleep()} function. It is inserted after each write or read so that you
can have ample time to switch to another window to execute another writer or reader instance. Now switch to another
window and start the reader before the writer has finished writing. No command-line parameters are needed for the reader. 
You will find that the reader stalls while trying to acquire a shared read lock because an exclusive lock is already
in place. When the writer completes, the reader will be able to get the shared lock and proceed with reading.
If you try the reader-reader combination, you ought to find that concurrent readers are allowed, which agrees to
what I told you earlier. \\

\mono{flock()} accepts two parameters. The first one is the filehandle. The second argument indicates the locking
operation required. Just like the case for \mono{seek()}, it is just an integer whose mnemonic constants can be 
found by importing the \mono{Fcntl} module, by importing the symbol `:flock'. The shared lock is represented by
\mono{LOCK\_SH} and the exclusive lock by \mono{LOCK\_EX}. To release a lock you can specify \mono{LOCK\_UN}. 
You may optionally bitwise-or \mono{LOCK\_NB} with either \mono{LOCK\_EX} or \mono{LOCK\_SH} to indicate 
non-blocking. As demonstrated in the example, an \mono{flock()} call will be blocked until the lock is acquired.
With this optional flag, you may choose to skip if it fails and do something else, or you may retry it later on. 
The predefined variable \mono{\$\$} refers to the current process ID. It is displayed as well as saved to file
to let you easily identify the various processes. \\

\section*{Summary}

\begin{summary}
\item A filehandle is a high-level abstraction in order to provide a uniform interface to I/O involving different media such as file access and socket communication.
\item There are three predefined file handles: \mono{STDIN}, \mono{STDOUT} and \mono{STDERR}.
	\begin{summary}
	\item User input through the keyboard is read from \mono{STDIN}.
	\item Output of a program are usually directed to \mono{STDOUT}.
	\item Error messages are usually directed to \mono{STDERR}. 
	\end{summary}
\item Before performing any operations on a file, you have to \mono{open()} it. You should \mono{close()} it when you are done with it.
\item Output redirection allows you to redirect data sent to a certain filehandle to another filehandle.
\item The \mono{readline()} function reads a sequence of characters from a given filehandle until a newline is encountered. The \mono{<>} operator in Perl serves the same purpose.
\item \mono{binmode()} ensures binary files are not corrupted on MS-DOS based operating systems.
\item \mono{read()} reads a given number of characters from a filehandle.
\item Every open file maintains a file pointer, which you may manipulate with the \mono{seek()} function. The file pointer position may also be obtained by the \mono{tell()} function.
\item The \mono{print()} and \mono{printf()} functions output a string to the given filehandle. \mono{printf()} performs some transformations on the string beforehand.
\item \mono{sprintf()} is identical to \mono{printf()} except the resulting string to be returned instead of written to any filehandles.
\item The \mono{readdir()} function traverses a given directory, whose filehandle as been populated by the \mono{opendir()} function. Upon completion \mono{closedir()} is invoked to close it.
\item File test operator can be used to test if a file possesses a given property.
\item File locking serves to set temporary restrictions on how certain files may be shared among several processes. The \mono{flock()} function is used.
\end{summary}
