[...]

To make it clear, here's an example:

maxsize = 100000 maxtime = 86400 maxfiles = 5

Kernel messages: facility = "kern" logdir = "/var/log/kernel"

Crond: program = "crond" logdir = "/var/log/crond"

A section defines several things:

• A title (useless for the software, it's just to make your configuration file look better).

• Filters: you can define facilities, program names, urgency levels and regular expressions. An incoming message will pass through all filters. If all conditions are matching, actions defined for the section are performed.

• Actions: they are taken only when all previous conditions are met. Only two actions are currently possible: write the message to a log file, and/or launch an external script.

Here's a list of values that can be independently assigned for each section:

• minimum = : only record a message if its urgency is inferior or equal to . Level '0' is the most critical one, while level '7' is for debugging messages. 'minimum = 5' will strip all non-important messages. The default minimum level is 7 (ie. keep all messages).

Example: record only critical messages to /var/log/important:

Critical messages: facility = "*" minimum = 1 logdir = "/var/log/important"

• maximum = : don't log if the message level exceeds that value. By default, maximum is the largest possible level.

• facility = : only record a message if the application that issued it uses syslog facility .

• break = 0|1 (default=0): if set to 1 and a section is matched, perform action, but don't consider any more possible section matches below this one in the config file. This is useful for creating a config where specific types of log messages are matched and dispatched, while a "catch-all" section at the bottom of the config file handles the default case. Using break=0 would cause messages to be potentially handled by multiple sections, causing message duplication, while using break=1 (default) on the sections above the "catch-all" would avoid duplication.

Facility names are: "auth", "authpriv", "cron", "daemon", "ftp", "kern", "lpr", "mail", "news", "security", "syslog", "user", "uucp", "local0", "local1" ... "local7". All kernel messages are logged with facility "kern".

A section can have several "facility = ..." lines to match more than one facility. If is "*", it'll match all the facilities.

Example: record all authentication messages to /var/log/auth:

Authentication messages: facility = "auth" facility = "authpriv" logdir = "/var/log/auth"

• program = : only record messages signed by a specific daemon or program.

You can use this as a modern alternative to syslog facilities: use "*" as a facility, and set a program name with that directive.

Kernel messages can be caught with 'program = "kernel"'.

Example: record Pure-FTPd messages in a directory, and in.ftpd messages in another directory (although the facility is the same: FTP):

In.FTPd messages: facility = "*" program = "in.ftpd" logdir = "/var/log/in.ftpd"

Pure-FTPd messages: facility = "*" program = "pure-ftpd" logdir = "/var/log/pure-ftpd"

• program_regex = : similar to program, but not just a string equal match, but a regular expression.

Example: this would match program fields with [postfix/smtp], [postfix/smtpd] and [postfix/cleanup] etc.

All Postfix messages: facility = "*" program_regex = "postfix" logdir = /var/log/postfix

• program_neg_regex = : log programs that do not match the regex.

• regex = : a message must match the regular expression to pass that filter. Multiple regexes are allowed for a single section. All Perl extensions are allowed, and matching is case insensitive.

Example: record all authentication failures to /var/log/pwdfail:

Password failures: regex = "(password|login|authentication)\s+(fail|invalid)" regex = "(failed|invalid)\s+(password|login|authentication)" regex = "ILLEGAL ROOT LOGIN" logdir = "/var/log/pwdfail"

• neg_regex = : this is the opposite of the previous directive. Logging will occur if the regex doesn't match. It can be useful to remove the crap, and it can be freely mixed with "regex" directives. "regex" and "neg_regex" directives are scanned in order.

Example: facility = "mail" neg_regex= "starting daemon" logdir = "/var/log/mail"

• socket = : metalog will create a socket file on the given path and listen on this Unix Domain Socket. That way applications in a chroot jail can still send log messages that will be received by metalog. Such a socket could also get bind mounted in LXC containers and overmount /dev/log in there. Within the container then is no longer a own logger necessary. This option can be set in the default part of the config or within a section. When used within a section it can we use as a selector for a specific log file instead of flags like "program", "facility", "regex" and so on.

• maxsize = : automatically rotate log files when their size have exceeded that size. Messages are never truncated and no message can be lost during a rotation. Every section can have a different maxsize value.

• maxfiles = : how many files to keep after rotation. This parameter can be set independently for each section. It defaults to 5.

• maxtime = : automatically rotate log files when they are older than this number of seconds. Every section can have a different maxtime value. maxsize and maxtime can be combined together, so that rotation occurs when any of these conditions occurs.

• compress = 0|1: With this active and when compiled with zlib support the rotated log files will get compress with gzip.

• compress_delay = : This will leave the amount of the latest rotated log files uncompressed.

Example: rotate log files daily or when they are more than 1,000,000 bytes long, keep only 3 history files with the first and second uncompressed and the third one compressed:

maxsize = 1000000 maxtime = 86400 maxfiles = 3 compress = 1 compress_delay = 2

• log_format = : The log entries can be written in different formats: "legacy", "legacy_timestamp", "rfc3164" and "rfc5424". The legacy ones are quite similar to the format defined in obsolete RFC3164. legacy_timestamp is the default format.

• log_severity = : In case a legacy format is used, this will introduce the severity info to a log entry. In both RFC formats this is already included and can't get disabled.

• showrepeats = 0|1: disable printing the "last message repeated N times" summary messages. When showrepeats is set to 1, messages will be printed directly to the log, without summarizing them based on number received (which can cause problems for some log monitoring programs, as well as delaying repeated messages if another unique message isn't received promptyly). Defaults to 0 (i.e. repeated messages will be summarized)

• logdir = : record messages in the specified directory. If the directory doesn't exist, it will be automatically created when the first matching message will be logged (the parent directory has to exist, though).

• perms = permissions for the log directory. Defaults to 0700

Example: Let those in the group with the GID of the process read the log. Don't forget to run metalog as the group.

perms = 0770

• command = <path/to/command>: run a program or a shell-script when all conditions are met. This directive is not incompatible with logdir: a message can be both logged and passed to an external command. When the command is launched, the first argument is filled with the date the message was received, the second argument is the program name, and the last one is the text of the message itself. Environment variables aren't cleared.

Example: send a mail to root when authentication failures occur:

Mail password failures: regex = "(password|login|authentication)\s+(fail|invalid)" regex = "(failed|invalid)\s+(password|login|authentication)" regex = "ILLEGAL ROOT LOGIN" command = "/usr/local/sbin/pwdfail.sh"

"pwdfail.sh" can be a simple shell script like this one:

#! /bin/sh echo "$3" | mail -s "Password failure (program: $2)" root

Don't forget to properly quote arguments to avoid security problems.

• postrotate_cmd = <path/to/script>: run a script after rotating If specified, the postrotate_cmd is run after log-files are rotated. The following parameters are provided, environments are not cleared: date: date of the last message received prog: program name of the last message received file: file name with path of the rotated log-file

Example: compress rotated log files with bzip2

postrotate_cmd = /usr/local/bin/compress.sh

"/usr/local/bin/compress.sh" provides bzip2 compression: #!/bin/sh #* #* Copyright (c) 2006 by Lukas Ruf (lukas.ruf@lpr.ch), #* Computer Engineering and Networks Laboratory (TIK), #* Swiss Federal Institute of Technology (ETH) Zurich #*

DATE="$1" PRG="$2" FILE="$3"

tests to run bzip2

parameter file is provided

file exists

file has a size greater than zero

if [ -n "${FILE}" -a -f "${FILE}" -a -s "${FILE}" ]; then bzip2 -9${FILE} fi

• configdir = : This optional directory can contain more config files that will get interpreted at metalog start. Only file names ending with ".conf" are interpreted. This option is only allowed once, all repetitions will be ignored. The content of the additional config files will be appended to the existing config in alphanumeric order of the file names, to get a determined behaviour of metalog. The config files could contain sections for specific services.

Example: Content of /etc/metalog.d/:

00_kernel.conf
10_metalog.conf
11_crond.conf
20_application_xy.conf
21_apache.conf


     --------- CONFIGURATION: REMOTE SYSLOG SERVERS ------------

metalog can send log messages to syslog server(s) via UDP.

• remote_log = 0|1: Log messages will be sent to a remote syslog server. If defined in the default settings section, all log files will be sent. It can also be defined in the seperate sections, so remote logging can get activated/deactivated for every single section.

• remote_host = : This is the remote syslog server.

• remote_port = : This is the UDP the syslog server listens for new log messages.

• remote_format = : It makes sense to set the format to "rfc3164" or "rfc5424", so the host name of the sender will be included in the message.

• remote_severity_level = : This defines which log messages will be sent to the remote syslog servers. It filters the less important messages and so reduces the amount of messages sent.

Example: send all logs to a remote server:

remote_host = 10.1.0.1 remote_port = 514 remote_format = rfc5424 remote_severity_level = 3

This will send all log messages also to the remote syslog server, that have a severity level of 3 [ERROR] or a higher (that have a lower number).

If several syslog servers should receive the logs, the above remote_* commands can be replaced with remote_*[], where is a alphanumeric value.

Example: This will send the logs to two servers: one called "main" and the other one "backup".

remote_log[main] = 1 remote_host[main] = 10.1.0.1 remote_port[main] = 514 remote_format[main] = rfc5424 remote_severity_level[main] = 3

remote_log[backup] = 1 remote_host[backup] = 192.168.1.10 remote_port[backup] = 514 remote_format[backup] = rfc5424 remote_severity_level[backup] = 3

     ------------------------ LOG FILES ------------------------

With the "logdir" directive, messages from a specific section are recorded in a directory. In that directory, the following files are created:

• "current": this file contains the latest recorded messages. It can be incomplete (ie. data is being written to the file when you are reading it).

• ".timestamp": the creation date of "current". Needed for the "maxtime" directive.

• "log----::": old logs, chronologically sorted.

If you ever delete these files by hand for some obscure reasons, it's not a bad idea to restart the daemon (and even better: to stop it before you mess the directories).

  ------------------------ RUNNING METALOG ------------------------

Kill "klogd" and "syslogd" first. Don't run Metalog while they are running. Multiple programs listening for the same source of data is a silly idea.

Once these programs are killed (and you are sure they are killed), check that /etc/metalog.conf is installed, and simply run:

/usr/local/sbin/metalog &

On GNU/Linux systems, two processes are spawned: one is the "MASTER", doing much of the work. The other one (named "KERNEL") is needed for Linux and replaces the "klogd" daemon. It is responsible for logging kernel messages.

By default, messages are immediately recorded into log files, and the I/O cache is flushed to be sure that you don't loose any message if a fatal crash or a power outage occurs.

But if you prefer speed, Metalog can work asynchronously, using the --async switch. In this mode, to avoid disk I/O, and unlike traditional syslog daemons, Metalog works in memory buffers, then flushes the data to disk by blocks. It improves a lot overall performances.

If you temporarely want to switch to the asynchronous mode, send an USR2 signal to the process. Something like:

kill -USR2 $(cat /var/run/metalog.pid)

should do the trick.

Later, if you want to watch activity in real-time (like a good old "tail -f" on a log file), you can disable buffering. Just send an USR1 signal to the "MASTER" process. You can always re-enable buffering afterwards.

 ------------------------ RUN-TIME OPTIONS ------------------------

Metalog accepts some run-time options:

• '-a' or '--async': improve performance by using buffers (but log files won't get updated in real time).

• '-B' or '--daemonize': have the server start in background (daemonization).

• '-c ' or '--consolelevel=': set the console log level on Linux. Valid values are from 0 to 7. The default is 7.

• '-C ' or '--configfile=': use an alternative configuration file.

• '-g ' or '--group=': change the GID of the metalog process. Created files will be owned by this group.

• '-h' or '--help': show help and version number.

• '-p ' or '--pidfile=': set the name of the file that will hold the PID number. It defaults to /var/run/metalog.pid

• '-S ' or '--socket=': set the path that will name the main syslog socket. It defaults to /dev/log

• '-s' or '--sync': start in synchronous mode, with no bufferization.

• '-t' or '--test-config': Exit after parsing the config file(s). Returns 0 if config is ok.

                    -Frank DENIS "Jedi/Sector One" <j@pureftpd.org> .