Download logFaces User Manual

Copyright ©2009-2013 Moonlit Software Ltd, All rights reserved
2.8 Working with regular expressions
Regular expressions are used quite extensively in logFaces. You will meet them to process things
like unstructured syslog content or parse log files in arbitrary text format.
Using regular expressions effectively requires some practice and could be frustrating in the
beginning. Working with complex expressions to crunch large amounts of unstructured text may
easily become a daunting task if not applied systematically. So here we come with a solution to make
your life easier even if you are a beginner with regular expressions.
The idea was originally borrowed from logstash project (all the credits go to this community!) and
adopted for our needs with some insignificant polish.
logFaces server is shipped with regular expression patterns library – a text file you will find under
/conf directory on your server. This library initially contains some commonly used expressions and
you can extend and modify them as you go.
Patterns library can be shared amongst clients – they have a built-in capability for parsing log files,
see 'Viewing raw log files' section for more details.
Consider some simple pattern examples:
WORD
HOUR
MINUTE
SECOND
TIME
\b\w+\b
(?:2[0123]|[01]?[0-9])
(?:[0-5][0-9])
(?:(?:[0-5][0-9]|60)(?:[:.,][0-9]+)?)
(?!<[0-9])%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9])
Each line in the library contains exactly one pattern which has a name (left part) and the expression
(right part). Name and expression must be separated by space character.
Note how any expression can reference other patterns by name using %{xxx} notation. Before the
expression is used, server will unfold all the references to produce plain standard regular expression.
In practice this could be very large piece of text, quite often non-readable by humans.
Using this technique makes it very easy to prepare fairly complex expressions to parse unstructured
log data in almost any format.
15