\chapter{Regular Expressions\label{chap:regex}}

\section{Introduction}

Now we are marching into probably the most exciting chapter in this tutorial. Regular expressions (regexps, 
or even RE) are what makes Perl an ideal language for ``practical extraction and reporting'', as the name 
implies. To give you an idea, let me first give you an overview on how to perform \term{pattern matching} in Perl. \\

First, you construct the \term{regular expression}, which is essentially a sequence of characters describing the 
\term{pattern} you would like to match. The term ``pattern'' may seem a little bit foreign to you, but you may 
actually have had some experience of it already. For example, in MS-DOS if you would like to list all files 
with the extension \mono{.txt} (presumably text files), you may issue a command like \\

\begin{verbatimtab}[4] 
dir *.txt
\end{verbatimtab} 

On Unix-like operating systems, similarly, you can specify \\

\begin{verbatimtab}[4] 
ls *.txt
\end{verbatimtab} 

The ``\mono{*.txt}'' here can be described as a pattern as it is the specification used by the operating system 
(strictly speaking, the shell, i.e. the program in charge of reading commands from you and displaying output) 
to look for the file entries that have to be displayed. This is a very simple pattern, but Perl provides users with a 
very powerful set of regular expressions that can be used to specify the patterns, so you can make your search 
specification more specific. After constructing the regular expression, you can then bind the data to be 
searched (for example, text files or just a line of user input) to the pattern using the binding operators 
\mono{=$\sim$} or \mono{!$\sim$}. In this process, you have provided the Perl regexp engine with both the data 
to search for and the data to be searched. The return value indicates if pattern matching is successful. If it 
is successful, you may want to store the data temporarily, or in a file, or export the results directly 
to the standard output. \\

Regular expressions are used in Perl in a number of ways: \\

\begin{itemize} 
\item Search for a string that matches a specified pattern, and optionally replacing the pattern found with some other strings
\item Counting the number of occurrences of a pattern in a string
\item Split a formatted string (e.g. a date like 02/06/2001) into respective components (i.e. into day, month and year)
\item Validation of fields received from submitted HTML forms by verifying if a piece of data conforms to a particular format
\item ... and much more
\end{itemize} 

Regular expressions are not exclusive to Perl only. It is a vital component in Unix and other Unix-like operating 
systems like Linux to provide users with powerful text search and replace capabilities. You may find that many 
software on Unix, like \mono{grep} and \mono{awk}, allows users to use regular expression to specify search 
specifications (although their implementations are slightly different from that of Perl). Although this is still 
rare in Windows, many new software with regexp capabilities are emerging because many famous Unix applications 
are now being ported to Windows and other system platforms. Moreover, a large set of Perl regexps are now being 
adopted in JavaScript and JScript implementations in Netscape and Microsoft Internet Explorer respectively. 
This is a piece of good news as sophisticated input validation can now be performed directly by the end users' 
browsers. In this way, invalid data can be detected without sending anything to the server that causes the 
transmission delays. In C/C++ there are also regular expression libraries that programmers can easily use for 
adding regexp capabilities to their programs. That explains how useful regular expressions are in programming
nowadays. \\

As you may know there are books dedicating in their entirety to regular expressions in your local bookstores. 
Therefore, this chapter is by no means a complete coverage of regular expressions in fine detail. However, 
after you have finished this chapter you should appreciate how flexible and powerful regular expressions 
are in Perl and in Unix-like operating systems, and how they can accomplish tricky text manipulating tasks 
on the fly. \\

\section{Building a Pattern}

\subsection{Getting your Foot Wet}

Now you will learn to build a pattern using the regular expressions offered by Perl. To search for a 
pattern match, simply construct the pattern and put it in between the two slashes of the \mono{m//} operator. If you 
don't need the bells and whistles, for example, you just need to know if the characters ``\mono{able}'' 
appear in any given string, the pattern is as simple as: \\

\begin{verbatimtab}[4] 
m/able/
\end{verbatimtab} 

Let's put this to a test. Now, to see if this pattern occurs in the string ``\mono{Capable}'', we bind 
the twos together by using the binding operator \mono{=$\sim$}. Try this script: \\

\begin{lstlisting}[numbers=none] 
if ("Capable" =~ m/able/) {
    print "match!\n";
} else {	# This should NEVER happen
    print "no match!\n";
}
\end{lstlisting}
 
There is not many things special here. Because the pattern ``\mono{able}'' is in the string ``\mono{Capable}'', 
the words ``\mono{match!}'' will be displayed. I intentionally use the literal ``\mono{Capable}'' in the example 
to show that although the symbol looks like an assignment operator, it is not necessary for a valid lvalue on 
the left hand side of the binding operator (remember lvalue is for assignment operators only). You may put any 
piece of scalar data, including scalar variable, in place of the string literal. \\

\subsection{Introduction to \mono{m//} and the Binding Operator}

As you have seen in the above example, the \mono{m//} operator is used for pattern matching. In between the 
forward slashes \mono{//} the pattern to match is placed. Additional options, if any, are placed at the end
after the last slash. If an expression is explicitly bound to the operator using the \mono{=$\sim$} or \mono{!$\sim$}
binding operators, that expression is searched for the pattern specified. If the binding operator is missing,
as you will see in some later examples, \mono{=$\sim$} is assumed and \mono{\$\_} is taken as the expression
to be searched. \\

In scalar context, the binding operator \mono{=$\sim$} returns a true value if the expression matches the 
pattern, an empty string (and hence a false value) if otherwise. \mono{!$\sim$} simply inverts the logic
so that if the expression matches the pattern a false value is returned, a true value otherwise. Therefore,
the following two expressions are equivalent: \\

\begin{verbatimtab}[4]
!($expression =~ m/pattern/)
$expression !~ m/pattern/
\end{verbatimtab}

Similar to double-quoted strings, the pattern may be interpolated. Therefore, you can generate patterns
at runtime and apply them by, for example, \\

\begin{verbatim}
$expression =~ m/$var/
\end{verbatim}

Again, similar to the case of quoted strings, you may use other symbols in place of \mono{//}. However, if you 
use \mono{//}, you may omit the prefix \mono{m}. You may wish to use other symbols in place of \mono{//} if
your pattern is heavily slashed, for example, to match a Unix path name \mono{/var/logs/httpd/error\_log} in
an expression you have to escape the forward slashes (to be covered later) like this: \\

\begin{verbatimtab}[4]
$expression =~ m/\/var\/logs\/httpd\/error_log/
\end{verbatimtab}

In the manual pages, this is described as the \textit{leaning toothpick syndrome} (LTS) where a lot of
forward and backward slashes are present, making the pattern itself difficult to recognize. If you change
the symbol to, for example, $|$, then the entire pattern suddenly becomes clear: \\

\begin{verbatimtab}[4]
$expression =~ m|/var/logs/httpd/error_log|
\end{verbatimtab}

This is just one of the methods to remove the leaning toothpick syndrome. 
We will talk about the \mono{m//} operator in more detail later in this 
chapter, together with the options you may use. I am just giving you a taste of it now anyway. \\

\subsection{Metacharacters}

The list of metacharacters supported in Perl are listed in Table~\ref{tab:metachar}. \\

\begin{table}[htbp]
\begin{center}
\begin{tabular}{ll}
\hline
\textbf{Metacharacter}	& \textbf{Default Behaviour} \\
\hline 
%	\texttt{\textbackslash} &	\multicolumn{3}{l}{Quote next metacharacter} \br \\
	\texttt{\textbackslash} &	Quote next character \br \\
	\texttt{\^} &	Match beginning-of-string \br \\
	\texttt{.} &	Match any character except newline \br \\
	\texttt{\$} &	Match end-of-string \br \\
	\texttt{|} &	Alternation \br \\
	\texttt{()} &	Grouping and save subpattern \br \\
	\texttt{[]} &	Character class \br \\
\hline 
\end{tabular}
\\[\baselineskip]
\end{center}
\caption{Metacharacters in Perl}\label{tab:metachar}
\end{table}

Metacharacters serve specific purposes in a pattern. If any of these metacharacters are to be embedded in the 
pattern literally, you should quote them by prefixing it by \textbackslash, similar to the idea of escaping in 
double-quoted string. In fact, the pattern in between the forward slashes are treated as a double-quoted string.
For example, to match a pair of empty parentheses and execute a code block if they can be found, the code 
should look like \\

\begin{verbatimtab}[4]
if ($string =~ m/\(\)/) {
	# ...
}
\end{verbatimtab}

In the previous section we mentioned the leaning toothpick syndrome. Apart from changing the delimiters of the
\mono{m//} operator, you can suppress the effect of metacharacters by using the \mono{\textbackslash Q ... \textbackslash E} 
escape sequence. This does not suppress interpolation, however. This is demonstrated in the following example: \\

\begin{verbatimtab}[4]
$expression =~ m/\Q/var/logs/httpd/error_log\E/
\end{verbatimtab}

%The use of \mono{quotemeta()} is generally the recommended method to convert a pattern which potentially may contain
%metacharacters to escape all non-word characters in it so that they are interpreted literally. \\

%Another method is to use the \mono{quotemeta()} function. 

%my $pattern = quotemeta('/var/logs/httpd/error_log');
%$expression =~ m/$pattern/

$|$ specifies alternate patterns where matching of either one of them results in a match. These patterns are
tried from left to right. The first one that matches is the one taken. Usually, $|$ are used together with 
parentheses \mono{()} to indicate the groupings preferred. These are some example patterns: \\

\begin{verbatimtab}[4]
m/for|if|while/		# A match if either 'for', 'if' or 'while' found
m/a(a|b|c)a/  	# A match if either 'aaa', 'aba' or 'aca' found
\end{verbatimtab}
%"fortnight" =~ m/for|fort/

Apart from indication of grouping, the use of parentheses also carries another behaviour. If there is a 
pattern match, the expression matched by a grouped pattern is saved. This is called \term{backtracking}.
Backtracking is covered in more detail later in this chapter. \\

% ^ % .
The \mono{.} metacharacter matches any character. By default, it does not match any embedded newline characters in a multi-line string.
However, if the \mono{s} option of \mono{m//} is given, embedded newline characters will be matched. This 
is convenient if you have to match a pattern across multiple lines. \\

\begin{verbatimtab}[4]
"a\nb\nc" =~ m/a.b/		# Not matched, because . does not match \n
"a\nb\nc" =~ m/a.b/s	# Matched with 's' option
\end{verbatimtab}

The \mono{\^} metacharacter matches the beginning of the string, and \mono{\$} matches the end of the string.
However, if the \mono{m} option of \mono{m//} is given, they match the beginning and the end of each line 
respectively. This is used to match individual lines inside a multi-lined string. \\

\begin{verbatimtab}[4]
"a\nb\nc" =~ m/^a$/		# Not matched
"a\nb\nc" =~ m/^a$/m	# Matched 
\end{verbatimtab}

\subsection{Quantifiers}

Quantifiers are used to specify how many times a certain pattern can be matched consecutively. A quantifier
can be specified by putting the range expression inside a pair of curly brackets. The format of which is \\

\mono{
\{m\textcolor{optionalargs}{[,[n]]}\} \\
}

Here are the available variations: \\

\begin{verbatimtab}[4]
{m}		Match exactly m times
{m,}	Match m or more times
{m,n}	Match at least m times but not more than n times
\end{verbatimtab}

This example shows how you can verify if a string is an even number. Note that this example can be further
simplified with the help of character classes, which we will describe next. \\

\begin{verbatimtab}[4]
$string = $ARGV[0];
my $retval = ($string =~ m/^(\+|-){0,1}(0|1|2|3|4|5|6|7|8|9){0,}(0|2|4|6|8)$/); 
printf("$string is%san even integer.\n", $retval?' ':' not ');
\end{verbatimtab}

With different input values, different messages will be printed indicating whether the number is an even
integer. You may split the pattern into three sections. The first part, \mono{(\textbackslash+$|$-)\{0,1\}}
matches the preceding sign symbol if there is one. Note that the minimum number of times is 0. Therefore, 
this part still matches if the sign symbol is absent. Right after the optional sign symbol are the digits.
We establish that an even number has the least significant digit being 0, 2, 4, 6 or 8. Therefore, on the
far right we specify it as the last digit. In between the sign symbols and the least digit there can be
zero or more digits. This is how we arrive at this pattern. \\

Perl defines three special symbols to represent three most commonly used quantifiers. \mono{*} represents
\mono{\{0,\}}; \mono{+} represents \mono{\{1,\}} and \mono{?} represents \mono{\{0,1\}}. Because \mono{+}
is a quantifier as a result, it has to be escaped in the example pattern above. \\

\subsection{Character Classes}

A character class includes a list of characters where matching of any of these characters result in a match of the character class.
It is similar in some sense to alternation, but the way they are interpreted is different. A character
class is constructed by placing the characters inside a pair of square brackets. Here I demonstrate how to 
rewrite the pattern in the above example using character classes. \\

\begin{verbatimtab}[4]
my $retval = ($string =~ m/^[+-]?[0123456789]*[02468]$/); 
\end{verbatimtab}

It's a lot shortened. Isn't it? All characters that appear inside the square brackets belong to one character
class. We have also used the special quantifier symbols described above to further shorten the pattern. But that's
not the end. You can further shorten the character class comprising all digits by specifying in the form of 
a range: \\

\begin{verbatimtab}[4]
my $retval = ($string =~ m/^[+-]?[0-9]*[02468]$/);
\end{verbatimtab}

You may define multiple ranges in a character class, for example, \mono{[a-zA-Z]} matches all lowercase and uppercase
forms of English alphabets. \\

Inside a character class, if you prefix the list of characters with \mono{\^{}}, that means any characters that are 
not listed results in a match. For example, \mono{[\^{}0-9]} matches any character provided it is not numeric. \\

Perl also defines some special character classes that contain lists of common character combinations in pattern
matching. \\

\begin{table}[htbp]
\begin{center}
\begin{tabular}{ll}
\hline
\textbf{Character Class}	& \textbf{Content} \\
\hline 
	\texttt{\textbackslash w} &	Alphanumeric characters and \_ (\mono{[a-zA-Z0-9\_]})\br \\
	\texttt{\textbackslash W} &	Neither alphanumeric characters nor \_ (\mono{[\^{}a-zA-Z0-9\_]})\br \\
	\texttt{\textbackslash s} &	Whitespace characters (\mono{[ \bs t\bs n\bs r\bs f]})\br \\
	\texttt{\textbackslash S} &	Non whitespace characters (\mono{[\^{} \bs t\bs n\bs r\bs f]})\br \\
	\texttt{\textbackslash d} &	Numeric digits (\mono{[0-9]})\br \\
	\texttt{\textbackslash D} &	Non numeric digits (\mono{[\^{}0-9]})\br \\
\hline 
\end{tabular}
\\[\baselineskip]
\end{center}
\caption{Special Character Classes in Perl}\label{tab:charcls}
\end{table}

Finally, our example pattern to match even integers can be simplified as \\

\begin{verbatimtab}[4]
my $retval = ($string =~ m/^[+-]?\d*[02468]$/);
\end{verbatimtab}

which is now the most compact form you can attain. \\

\subsection{Backtracking}

Parenthesised patterns have a useful property. When pattern matching is successful, the matching substrings 
corresponding to the parenthesised parts are saved, which allow you to save them for further operations. For example, \\

\begin{verbatimtab}[4]
$string = 'Telephone:	1234-5678';
if ($string =~ m/^Telephone:\s*(\d{4}-\d{4})$/) {
	print "The telephone number extracted is '$1'.\n";
}
\end{verbatimtab}

In this example, the telephone number extracted is saved as \mono{\$1}. There can be multiple
bracketed patterns in a given pattern. The matched substrings are numbered
in ascending order of position of the opening parentheses. If we change the pattern as follows: \\

\begin{verbatimtab}[4]
$string = 'Telephone:	(852) 1234-5678';
if ($string =~ m/^Telephone:\s*(\((\d+)\)\s*(\d{4}-\d{4}))$/) {
	print "The telephone number extracted is '$1'.\n";
	print "The country code extracted is '$2'.\n";
	print "The local phone number extracted is '$3'.\n";
}
\end{verbatimtab}

\begin{lstlisting}[numbers=none]
The telephone number extracted is '(852) 1234-5678'.
The country code extracted is '852'.
The local phone number extracted is '1234-5678'.
\end{lstlisting}

The pattern looks more complicated then it really is. If you examine it carefully, there are three bracketed
patterns in it. The first one embracing the telephone number in full, including the country code. The second
and third are placed inside this bracket to extract the country code and local phone number separately. The
positions of the opening braces determine the ordering. Therefore, we can observe that in case of nested 
parentheses, the outer one has a smaller number than the inner one. \\

There is a very important property with respect to quantifiers that you have to be aware of. Quantifiers
default to being ``greedy''. That is, quantifiers try to match as many times as possible. For example 
when you match the string ``greediness 2003'' against the pattern \mono{/(.*)(\d+)/}, \mono{\$1} has the
string ``greediness 200'', leaving ``3'' to \mono{\$2}. Therefore, you can see that quantifiers appearing
on the left tries to match as many times as possible while allowing the pattern to match. To work around
this situation, you may append a \mono{?} character to the quantifier to suppress this greedy behaviour.
Therefore, for the sample string the best pattern is \mono{/(.*?)(\bs d+)/}, which separates all numeric 
digits at the end from the rest of the string. \\

\section{Regular Expression Operators}

\subsection{\mono{m//} --- Pattern Matching}

As we have been using so far, the \mono{m//} operator performs pattern matching. It supports a number of options.
We have covered \mono{m} and \mono{s}, and now it's time for a revision.  
The \mono{s} option treats the string being searched as if it consists of a single line only. By doing so, 
\mono{.} will match an intermediate newline character. The \mono{m} option allows matching of individual 
lines in a multi-line string. Here, I will introduce several other commonly used options. \\

The \mono{i} option matches in a case-insensitive manner. Therefore, \\

\begin{verbatim}
'ABCD' =~ m/abc/i
\end{verbatim}

results in a match. By default, pattern matching is case sensitive. Another useful option is \mono{g}, which
attempts to carry out a global pattern matching on the string. In scalar context, a search pointer is maintained.
The search pointer is first initialized to the beginning of the string. In each matching operation, matching starts 
from the search pointer. If a match is found, the search pointer advances to past the end of the matched substring.
If matching fails, the search pointer is reset to the initial position. You can use the \mono{pos()} function
to retrieve the position of the current search pointer. \\

You can use this option to find out the position of occurrences of certain patterns in the string. The following
example illustrates this point: \\

\begin{verbatimtab}[4]
$string = 'Telephone: 1234-5678';
while ($string =~ m/(\d{4})/g) {
	print "'$1' found at position " . (pos($string) - length($1)) . ".\n";
}
\end{verbatimtab}

\begin{lstlisting}[numbers=none]
'1234' found at position 11.
'5678' found at position 16.
\end{lstlisting}

In list context, the \mono{m//g} operator (with \mono{g}) returns a list consisting of all parenthesised 
substrings from the matching. Therefore, \\

\begin{verbatim}
my $string = 'Telephone:	(852) 1234-5678';
my @list = ($string =~ m/^Telephone:\s*(\((\d+)\)\s*(\d{4}-\d{4}))$/g);
# @list = ('(852) 1234-5678', '852', '1234-5678')
\end{verbatim}

results in the list \\

\begin{verbatim}
@list = ('(852) 1234-5678', '852', '1234-5678')
\end{verbatim}

\subsection{\mono{s///} --- Search and Replace}

This operator is a powerful search-and-replace engine that you can use to flexibly search for certain patterns
and replace it with a replacement string. The first argument is the search pattern, just as the case of \mono{m//}.
The second argument is the replacement string. As you will soon see, backtracking is immensely useful in this regard. \\

The options that I mentioned above that applies to \mono{m//} also apply to \mono{s///}. But there is a new one.
The option \mono{e} causes the replacement string to be treated as an expression instead of a double-quoted string.
That is, you can use a combination of operators to generate the desired replacement string at runtime. \\ 

Without the option \mono{g}, only the first occurrence of the pattern is replaced. With the 
option \mono{g}, all occurrences of the pattern are replaced in one go. \\

Here are some examples: \\

\begin{verbatimtab}[4]
$string =~ s/\t/' ' x 4/eg;		# change all tabs to 4 spaces
$string =~ s/^(.*)\n$/$1/s;		# like chomp(), to remove trailing newline
\end{verbatimtab}

\subsection{\mono{tr///} --- Global Character Transliteration}

\mono{tr///} is a convenient and efficient operator that changes a set of characters into another.
The first argument is the character list to search for. The second argument is the character replacement list.
It builds a character translation map at compile time. At runtime, it changes any characters that can be 
found in the string into the corresponding character in the replacement list. For example, \\

\begin{verbatim}
tr/a-z/A-Z/
\end{verbatim}

is just an alternative way to convert characters to uppercase form without using \mono{uc()}. \\

\section{Constructing Complex Regular Expressions}

While the general principles of regular expressions are not complicated, constructing the correct pattern
for your specific purpose is not itself a simple affair, especially if it is a complex one. An
incorrect pattern is one which does not match a string intended to match, or match a string that should 
not be matched. Despite experience develops over time, following a formal, systematic procedure
helps you construct regular expressions with a higher degree of accuracy. \\

The mechanism is known in the academic community as \term{finite-state automaton} (FSA). The definition is 
pretty abstract but the idea is rather easy to understand with the help of examples. Basically, we construct
an FSA diagram. The diagram has a number of states, with each of the states linked together by arrows 
which represent transitions. This diagram is also known as a \term{state diagram}.
At any point in time an FSA system can only be in one of the states. Every input
triggers a transition, which may or may not change the state of the FSA. \\

As an example, we will construct a pattern which matches all relative path expressions. For example,
\mono{images/fsa.gif} and \mono{docs/perltut} are considered valid paths. However, we add in an 
additional requirement to invalidate paths containing ``.'' and ``..'' as a path segment, because
we will reuse the pattern constructed in the next chapter. We also assume each path segment consists
of word characters (\mono{\bs w}) only. \\

The constructed FSA diagram is shown below: \\

\image{!hbt}{0.65\textwidth}{!}{images/fsa.eps}{FSA to Match Valid Relative Paths}{fig:fsa}

How do we interpret this diagram? We regard a path like \mono{images/fsa.gif} as a sequence of characters.
We break this path into its constituent characters, and feed each of them in sequence to the FSA. In this
diagram, each of the states is represented by an oval with the name of the state written at the centre. 
The initial state is marked as ``Initial'', which is the state before any characters are fed to the system.
An arrow represents transition and beside each arrow the character which triggers the transaction is indicated.
We also define that the states marked in blue are valid terminate states, namely ``Initial'' and ``Valid''. 
At the end of the input stream, if the system is at a valid terminate state, then the string matches our criteria.
Now ``i'' is fed to the system. Because ``i'' is a member of \mono{\bs w}, a transition to the ``Valid''
state occurred. Therefore, the state is now ``Valid''. Then, the character ``m'' triggers a loopback to
the ``Valid'' state. Similar are the characters ``a'', ``g'', ``e'' and ``s'' that follow. The next character,
``/'' triggers a transition to the ``Initial'' state. You may follow the remaining characters in a likewise
manner and eventually, after the last character ``f'' you will be at the state ``Valid''. This is the
terminate state, and because it is a valid terminate state, this path matches our criteria. \\

Note that any input character not matching any transitions from the current state shown in the diagram automatically triggers a 
transition to a hidden illegal state, which is not shown in the diagram. This is because we only emphasize
valid transitions, not invalid ones. \\

This diagram was constructed based on the idea that we may break a path into its constituent path segments. 
We start at the initial state. While constructing this diagram, we have to identify that we need to have
special treatment on path segments that start with a dot. We can accept path segments that start with \mono{\bs w}.
Therefore, we can immediately transit to the ``Valid'' state from the initial state if it is a word character. 
If it is a dot, then we follow another path to the ``Critical'' state. If the next character is also a dot,
then we transit to the ``Critical2'' state. Word characters appearing afterwards bring the system back to the ``Valid'' state.
From the diagram, it can be seen that we allow path segments such as ``...'' and ``..somedir'', but not
``..''. Depending on the operating system, you may wish to change this assumption if this is not acceptable
to you (e.g. on Windows three consecutive dots represent two levels up the directory tree). Note that at the 
end of each path segment, a ``/'' brings the system back to the ``Initial'' state. This allows paths consisting
of an arbitrary number of path segments. In order to be more flexible, the path may or may not be terminated
with ``/''. Therefore, the ``Initial'' state has to be made a valid terminate state instead of just the ``Valid''
state. \\

With a correct FSA established, rewriting it into a pattern becomes very easy. We only have to consider paths
from the initial state that lead to the valid states. Note that in this example we may follow three paths to reach
the ``Valid'' state. The first path is with the input of a word character, that is \mono{\bs w}. The second
path passes through the ``Critical'' state only, consisting of a dot followed by \mono{\bs w}, and hence the
path is \mono{.\bs w}. The third path passes both critical states, that is \mono{..[.\bs w]}. Therefore, expressed
in regular expression format these three paths can be represented by \mono{\bs w|\bs .\bs w|\bs .\bs .[.\bs w]}. At the ``Valid''
state, there is a transition back to itself. That means a character of the character class \mono{[.\bs w]} does not trigger any state changes.
This implies you can inject as many characters of this character class without changing the state of the FSA. 
In regular expressions, this is synonymous to \mono{[.\bs w]*}. Finally, please note that the ``Initial''
state is also a valid terminate state. Therefore, ``/'' which transits to the ``Initial'' state is optional.
With all these combined, the regular expression representing a valid path segment is \\

\begin{verbatim}
/(\w|\.\w|\.\.[.\w])[.\w]*\/?/
\end{verbatim}

Because this represents just one path segment, finally we arrive at the following pattern to match full
relative paths: \\

\begin{verbatim}
/^((\w|\.\w|\.\.[.\w])[.\w]*\/?)+$/
\end{verbatim}

%We may
%do an additional optimization step by grouping like terms. 

Construction of an FSA forces you to think about the specifics of the parts that make up the pattern carefully.
For more complicated patterns, this method generally yields more accurate patterns than pure imagination in
your brain because it is easier for you to verify whether the pattern is correct with the FSA. Usually you
may not get the FSA correct at the very first time, but through corrections and refinements you can come
up with a very accurate pattern. When you are highly experienced with regular expressions, it is likely that
you do not have to follow this elaborate procedure anymore. \\


%Although FSA cannot do everything

% TODO: more examples

\section*{Summary}

\begin{summary}
\item Pattern matching refers to the operation of comparing a string against a given pattern to determine if the string matches the pattern.
\item In Perl, a pattern is specified in the form of a regular expression. Regular expression is supported in many programming languages and programs.
\item To perform pattern matching in Perl, a string is bound to the pattern with the binding operator \mono{=$\sim$} or \mono{!$\sim$}.
\item Metacharacters serve special purposes in a pattern.
	\begin{summary}
	\item \bs\ quotes the next metacharacter so that it is interpreted literally.
	\item \mono{\^} refers to the beginning of a string.
	\item \mono{\$} refers to the end of a string.
	\item \mono{.} refers to any character except newline character.
	\item \mono{|} lets you specify alternative subpatterns. Matching is successful if any of the subpatterns match.
	\item \mono{()} are used to group subpatterns. Matched subpatterns are also saved in special variables starting from \mono{\$1} for backtracking.
	\item \mono{[]} establishes a character class. It includes a list of characters where matching of any of these characters result in a match of the character class. If prefixed by \mono{\^}, the character class matches any character not appearing inside the square brackets.
		\begin{summary}
		\item \bs w represents a character class containing alphanumeric characters and \mono{\_}
		\item \bs W represents \mono{[\^{}\bs w]}
		\item \bs s represents a whitespace character 
		\item \bs S represents any non-whitespace character, i.e. \mono{[\^{}\bs s]}
		\item \bs d represents a numeric digit 
		\item \bs D represents any non-numeric character
		\end{summary}
	\item \mono{\{\}} is a quantifier which specifies how many times a certain pattern can be matched consecutively.
		\begin{summary}
		\item \mono{*} implies \mono{\{0,\}}, i.e. zero or more times
		\item \mono{?} implies \mono{\{0,1\}}, i.e. none or single time
		\item \mono{+} implies \mono{\{1,\}}, i.e. one or more times
		\item A \mono{?} character following the above quantifiers disables greedy mode.
		\end{summary}
	\end{summary}
\item The \mono{m//} operator performs pattern matching.
	\begin{summary}
	\item The \mono{m} option allows \mono{m//} to match individual lines in a multi-line string. \mono{\^} and \mono{\$} match the beginning-of-line and end-of-line respectively.
	\item The \mono{s} option causes \mono{.} to match any newline characters.
	\item the \mono{i} option performs case-insensitive pattern matching.
	\item The \mono{g} option enables global pattern matching on the string.
	\end{summary}
\item The \mono{s///} operator performs search and replace, which supports the options \mono{m}, \mono{s}, \mono{i} and \mono{g} with the same semantics as \mono{m//}, the option \mono{e} is supported which causes the replacement string to be treated as an expression. 
\item The \mono{tr///} operator performs global character transliteration on a string.
\end{summary}

\section*{Questions}

\begin{enumerate}
\item {
Modify the FSA diagram to disallow path segments matching the pattern \mono{/\^{}\bs .+\$/} and construct
the corresponding match pattern. \\
}
\item {
In Chapter~\ref{chap:files} we developed a class \mono{FileSearch} to search for a file recursively in a 
directory tree whose name matching a pattern specified by the user. Now, use this class to write an application
which finds out all classes accessible by Perl and their respective locations on the filesystem. For example,
selected lines of output on my system look like this: \\

\begin{verbatim}
Bundle::Apache2 (/usr/local/lib/perl/5.8.0)
Bundle::CPAN (/usr/share/perl/5.8.0)
Bundle::DBD::mysql (/usr/local/lib/perl/5.8.0)
Bundle::DBI (/usr/local/lib/perl/5.8.0)
Bundle::LWP (/usr/share/perl5)
ByteLoader (/usr/lib/perl/5.8.0)
CGI (/usr/share/perl/5.8.0)
CGI::Apache (/usr/share/perl/5.8.0)
CGI::Carp (/usr/share/perl/5.8.0)
CGI::Cookie (/usr/share/perl/5.8.0)
CGI::Pretty (/usr/share/perl/5.8.0)
CGI::Push (/usr/share/perl/5.8.0)
CGI::Switch (/usr/share/perl/5.8.0)
CPAN (/usr/share/perl/5.8.0)
\end{verbatim}

Note that several versions of the same class may appear on the filesystem. The path has to reflect the location
of the version that will be loaded by Perl.
}
\end{enumerate}
