
syslog-ng reference manual

Balzs Scheidler

   Copyright  1999 by Balzs Scheidler
   
   This manual is free software; you may redistribute it and/or modify it
   under the terms of the GNU General Public License as published by the
   Free Software Foundation; either version 2, or (at your option) any
   later version.
   
   This is distributed in the hope that it will be useful, but without
   any warranty; without even the implied warranty of merchantability or
   fitness for a particular purpose. See the GNU General Public License
   for more details.
     _________________________________________________________________
   
   Table of Contents
   1. [1]Introduction to syslog-ng
   2. [2]Message paths
          
        [3]Sources
        [4]Filters
        [5]Destinations
        [6]Log paths
        [7]Options
                
   3. [8]Reference
          
        [9]Source drivers
                
              [10]internal()
              [11]unix-stream() and unix-dgram()
              [12]tcp() and udp()
                      
        [13]Destination drivers
        [14]Filter functions
        [15]Options
     _________________________________________________________________
   
Chapter 1. Introduction to syslog-ng

   One of the most neglected area of Unix is handling system events.
   Daily checks for system messages is crucial for the security and
   health conditions of a computer system.
   
   System logs contain much "noise" - messages which have no importance -
   and on the contrary important events, which should not be lost in the
   load of messages. With current tools it's difficult to select which
   messages we are interested in.
   
   A message is sent to different destinations based on the assigned
   facility/priority pair. There are 12+8 (12 real and 8 local)
   predefined facilities (mail, news, auth etc.), and 8 different
   priorities (ranging from alert to debug).
   
   One problem is that there are facilities which are too general
   (daemon), and these facilities are used by many programs, even if they
   do not relate each other. It is difficult to find the interesting bits
   from the enourmous amount of messages.
   
   A second problem is that there are very few programs which allow
   setting their "facility code" to log under. It's at best a compile
   time parameter.
   
   So using facilities as a means of filtering is not the best way. For
   it to be a good solution would require runtime option for all
   applications, which specifies the log facility to log under, and the
   ability to create new facilities in syslogd. Neither of these are
   available, and the first is neither feasible.
   
   One of the design principles of syslog-ng was to make message
   filtering much more finegrained. syslog-ng is able to filter messages
   based on the contents of messages in addition to the priority/facility
   pair. This way only the messages we are really interested in get to a
   specific destination. Another design principle was to make
   logforwarding between firewalled segments easier: long hostname
   format, which makes it easy to find the originating and chain of
   forwarding hosts even if a log message traverses several computers.
   And last principle was a clean and powerful configuration file format.
     _________________________________________________________________
   
Chapter 2. Message paths

   In syslog-ng a message path (or message route) consist of one or more
   sources, one or more filtering rules and one or more destinations
   (sinks). A message is entered to syslog-ng in one of its sources, if
   that message matches the filtering rules it goes out using one of the
   destinations.
     _________________________________________________________________
   
Sources

   A source is a collection of source drivers, which collect messages
   using a given method. For instance there's a source driver for
   AF_UNIX, SOCK_STREAM style sockets, which is used by the Linux
   syslog() call.
   
   To declare a source, you'll need to use the source statement in the
   configuration file with the following syntax:
   
          source <identifier> { source-driver(params); source-driver(params); .
.. };
        
   The identifier has to uniquely identify this given source and of
   course may not clash with any of the reserved words (in case you had a
   nameclash, simply enclose the identifier in quotation marks)
   
   You can control exactly which drivers are used to gather log messages,
   thus you'll have to know how your system and its native syslogd
   communicate. Here's a introduction to the inner workings of syslogd on
   some of the platforms I tested:
   
   Table 2-1. Communication method between syslogd and its clients
   Platform Method
   Linux A SOCK_STREAM unix socket named /dev/log
   BSD flavors A SOCK_DGRAM unix socket named /var/run/log
   Solaris (2.5 or below) An SVR4 style STREAMS device named /dev/log
   Solaris (2.6 or above) In addition to the STREAMS device used in
   versions below 2.6, uses a new multithreaded IPC method called door.
   By default the door used by syslogd is /etc/.syslog_door
   
   Each possible communication mechanism has the corresponding source
   driver in syslog-ng. For instance to open a unix socket with
   SOCK_DGRAM style communication you use the driver unix-dgram, the same
   with SOCK_STREAM style - as used under Linux - is called unix-stream.
   
   Example 2-1. Source statement on a Linux based operating system
          source src { unix-stream("/dev/log"); internal(); udp(ip(0.0.0.0) por
t(514)); };
        
   Each driver may take parameters, some of them required, some of them
   optional. The required parameters are usually positional, which means
   that they have to come first. See the unix-stream driver specification
   above, as it refers to the file /dev/log.
   
   Table 2-2. Available source drivers in syslog-ng
   Name Description
   internal Messages generated internally in syslog-ng
   unix-stream Opens the specified unix socket in SOCK_STREAM mode, and
   listens for messages.
   unix-dgram Opens the specified unix socket in SOCK_DGRAM mode, and
   listens for messages.
   file Opens the specified file, and reads messages.
   pipe, fifo Opens the specified named pipe and reads messages
   udp Listens on the specified UDP port for messages.
   tcp Listens on the specified TCP port for messages.
   sun-stream, sun-streams Opens the specified STREAMS device on Solaris
   systems, and reads messages.
   
   For a complete descriptions on the above drivers, see the reference
   section.
     _________________________________________________________________
   
Filters

   Filters perform log routing inside syslog-ng. You can write a boolean
   expression using internal functions, which has to evaluate to true for
   the message to pass.
   
   Filters have also a uniquely identifying name, so you can refer to
   filters in your log statements. Syntax for the filter statement:
          filter <identifier> { expression; };
        
   An expression may contain the operators "and", "or" and "not", and any
   of the functions listed below.
   
   Example 2-2. A filter statement finding the messages containing the
   word deny coming from the host blurp
          filter f_blurp_deny { host("blurp") and match("deny"); };
        
   Table 2-3. Available filter functions in syslog-ng
   Function Description
   facility() Selects messages based on their facility code
   level() or priority() Selects messages based on their priority
   program() Tries to match a regular expression to the program name
   field of log messages
   host() Tries to match a regular expression to the hostname field of
   log messages
   match() Tries to match a regular expression to the message itself.
   
   For a complete description on the above functions, see the Reference
   chapter.
     _________________________________________________________________
   
Destinations

   A destination is a message sink, where log is sent if filtering rules
   match. Similarly to sources, destinations may include several drivers
   which define how messages are dispatched. To declare a destination in
   the configuration file, you'll need a destination statement, whose
   syntax is as following:
          destination <identifier> { destination-driver(params); destination-dr
iver(params); ... };
        
   Table 2-4. Available destination drivers in syslog-ng
   Name Description>
   file Writes messages to the given file
   fifo, pipe Writes messages to the given named pipe
   unix-stream Sends messages to the given unix socket in SOCK_STREAM
   style (Linux)
   unix-dgram Sends messages to the given unix socket in SOCK_DGRAM style
   (BSD)
   udp Sends messages to specified host and UDP port
   tcp Sends messages to specified host and TCP port
   usertty Sends messages to specified user if logged in
   program Forks and launches given program, and sends messages to its
   standard input.
   For detailed list of the supported drivers, see the Reference chapter.
     _________________________________________________________________
   
Log paths

   In the previous chapters we learnt how to define sources, filters and
   destinations. We'll need to connect those together, which is
   accomplished by the log statement. Any message coming from one of the
   listed sources, matching the filters (each of them) are sent to the
   listed destinations. The needed syntax is here:
          log { source(s1); source(s2); ...
          filter(f1); filter(f2); ...
          destination(d1); destination(d2); ... };
     _________________________________________________________________
   
Options

   There are several options you can specify, which modifies the
   behaviour of syslog-ng. For an exact list of possible options see the
   chapter Reference. The general syntax is here:
          options { option1(params); option2(params); ... };
        
   Each option may have parameters, just like in driver specification.
   
   Table 2-5. List of supported global options in syslog-ng
   Name Accepted values Description
   time_reopen() number The time to wait before a died connection is
   reestablished
   sync_freq() number The number of lines buffered before written to file
   mark_freq() number The number of seconds between two MARK lines. NOTE:
   not implemented yet.
   log_fifo_size() number The number of lines fitting to the output queue
   chain_hostnames() yes or no Enable or disable the chained hostname
   format.
   use_dns() yes or no Enable or disable DNS usage. syslog-ng blocks on
   DNS queries, so enabling DNS may lead to a Denial of Service attack.
   To prevent DoS, protect your syslog-ng network endpoint with firewall
   rules, and make sure that all hosts, which may get to syslog-ng is
   resolvable.
   use_fqdn() yes or no Add Fully Qualified Domain Name instead of short
   hostname.
   gc_idle_threshold() number Sets the threshold value for the garbage
   collector, when syslog-ng is idle. GC phase starts when the number of
   allocated objects reach this number. Default: 30.
   gc_busy_threshold() number Sets the threshold value for the garbage
   collector, when syslog-ng is busy. GC phase starts when the number of
   allocated objects reach this number. Default: 3000.
     _________________________________________________________________
   
Chapter 3. Reference

   This chapter documents the drivers and options you may specify in the
   configuration file.
     _________________________________________________________________
   
Source drivers

   The following drivers may be used in the source statement, as
   described in the previous chapter.
     _________________________________________________________________
   
internal()

   All internally generated messages "come" from this special source. If
   you want warnings, errors and notices from syslog-ng itself, you have
   to include this source in one of your source statement.
   
   This driver has no positional nor optional parameters.
   
   Syslog-ng will print you a warning, if this driver is not referenced.
     _________________________________________________________________
   
unix-stream() and unix-dgram()

   This two drivers behave similarly: they open the given AF_UNIX socket,
   and start listening on them for messages. unix-stream() is primarily
   used on Linux, and uses SOCK_STREAM semantics (connection oriented, no
   messages are lost), unix-dgram() is used on BSDs, and uses SOCK_DGRAM
   semantics, this may result in lost local messages, if the system is
   overloaded.
   
   To avoid denial of service attacks when using connection-oriented
   protocols, the number of simoultaneously accepted connections should
   be limited. This can be achieved using the max-connections()
   parameter.
   
   There's a required positional argument, which specifies the name of
   the socket to open, and other optional parameters may be specified as
   follows:
   
   Table 3-1. Available options for unix-stream
   Name Type Description Default
   owner() string Set the uid of the socket. root
   group() string Set the gid of the socket. Default: root. root
   perm() number Set the permission mask. For octal numbers prefix the
   number with '0', e.g. use 0755 for rwxr-xr-x. 0666
   keep-alive() yes or no Selects whether to keep connections opened when
   syslog-ng is restarted, can be used only with unix-stream(). Default:
   yes. yes
   max-connections() number Limits the number of simoultaneously opened
   connections. Can be used only with unix-stream(). 10
   
   Example 3-1. 
            unix-stream("/dev/log" max-connections(10)); };
        
     _________________________________________________________________
   
tcp() and udp()

   These drivers let you accept messages from the network.
     _________________________________________________________________
   
Destination drivers

   The following drivers may be used in the destination statement, as
   described in the previous chapter.
     _________________________________________________________________
   
Filter functions

   The following functions may be used in the filter statement, as
   described in the previous chapter.
     _________________________________________________________________
   
Options

   The following options can be specified in the options statement, as
   described in the previous chapter.

References

   1. file://localhost/usr/tmp/@18368.2#INTRO
   2. file://localhost/usr/tmp/@18368.2#MSGROUTE
   3. file://localhost/usr/tmp/@18368.2#AEN27
   4. file://localhost/usr/tmp/@18368.2#AEN95
   5. file://localhost/usr/tmp/@18368.2#AEN128
   6. file://localhost/usr/tmp/@18368.2#AEN165
   7. file://localhost/usr/tmp/@18368.2#AEN169
   8. file://localhost/usr/tmp/@18368.2#REFERENCE
   9. file://localhost/usr/tmp/@18368.2#AEN221
  10. file://localhost/usr/tmp/@18368.2#AEN224
  11. file://localhost/usr/tmp/@18368.2#AEN229
  12. file://localhost/usr/tmp/@18368.2#AEN272
  13. file://localhost/usr/tmp/@18368.2#AEN275
  14. file://localhost/usr/tmp/@18368.2#AEN278
  15. file://localhost/usr/tmp/@18368.2#AEN281
