rsyslogd

reliable and extended syslogd

rsyslogd [-c3] [-4][-6][-A][-d][-f config file]
[-i pid file][-l hostlist][-n][-N 1
[-q][-Q][-s domainlist][-u l][-v][-w][-x]

This document severly adapted by Dennis German, assumes an understanding of various syslog utilities. See the html documentation as the man pages only covers the basics rsyslog.com/doc

Includes definition of output formats via templates, precise timestamps and writing to databases

Helpful for debugging conf:
rsyslogd -N 1 |more

rsyslogd -N 1 -d 2>~/0 | egrep "cnf:|ssigned|filter|ACTION|PRIFILT" |egrep -v 'cnf:global:script|END'|more # stderr not helpful

-N 1 CoNfig check. Do NOT run in regular mode. sample
-d debug mode. Does't background, writes on stdout.
‑f config_fileDefault /etc/rsyslog.conf
-n No backgrounding. Needed if started and controlled by init.
-4 listen to IPv4 addresses .only.
-6 listen to IPv6 addresses only.
If neither -4 nor -6 is given, listens to all configured addresses
-i pid_file used if multiple instances are run
-l host [:host …] hostnames that are to be logged with simple hostname and not the FQDN.
-q add hostname if DNS fails during ACL processing when hostnames are resolved to IP addresses. If DNS initally fails hostname is added as wildcard text, which results in proper, but somewhat slower operation once DNS is up again.
-Q do not resolve hostnames to IPs during ACL processing.
-s domain[:domain] domainnames to be stripped.
-u l 1 prevents parsing hostnames and tags inside messages.
2 prevents changing to the root directory. Almost never a good idea
3 both.
-v raspberryPi as of 3/18/18
rsyslogd 8.24.0, compiled with:
    PLATFORM:               arm-unknown-linux-gnueabihf
    PLATFORM (lsb_release -d):      
    FEATURE_REGEXP:             Yes
    GSSAPI Kerberos 5 support:      Yes
    FEATURE_DEBUG (debug build, slow code): No
    32bit Atomic operations supported:  Yes
    64bit Atomic operations supported:  Yes
    memory allocator:           system default
    Runtime Instrumentation (slow code):    No
    uuid support:               Yes
    Number of Bits in RainerScript integers: 6

See http://www.rsyslog.com  
-w warnings suppressed when messages are received from machines in no AllowedSender list.
-x Disable DNS for remote messages.
-A All targets are sent UDP messages. May improve reliability, but causes message duplication.
When sending UDP messages, there are potentially multiple paths to the target destination.
Default: only send to the first target it can successfully send to.
-c version compatibility mode. Must be first.
-c0 compatible to sysklogd Default
-c3: use older v3 native interface,
Issues warning messages if -c3 is not given. Logs compatibility-mode config file directive it has generated which can be copied from logfile and pasted to config.

SIGNALS

To restart if controled by systemd : sudo systemctl restart rsyslog
To send a signal to rsyslogd use: sudo kill -USR1 $(cat /var/run/rsyslogd.pid)

HUP STOP then START:
close/open files, TCP and other connections are torn down,
queues are not running in disk assisted mode or not set to persist data on shutdown, data is lost.

Start, Read changed configuration files.
Extremely expensive operation and should only be done when actually necessary.

TERM,
INT
Well, it TERMinates!
USR1 toggle debugging if started with -d
CHLD Wait for childs if some were born, because of wall messages.

Files

/etc/rsyslog.conf Configuration See filter at rsyslogd.com

Selector extensions: none, comma seperated facilities with same priority, multiple selectors seperated by ; = priority prefix : do not include higher proorities, ! priority prefix

/dev/log Unix domain socket to from where local syslog messages are read.
/var/run/rsyslogd.pid contains the process id of rsyslogd.
prefix/lib/rsyslog Default directory for rsyslogd modules. The prefix is specified during compilation (e.g. /usr/local).

ENVIRONMENT

RSYSLOG_DEBUG Controls runtime debug support. contains an option string of:
LogFuncFlow output the logical flow of functions (entering and exiting them)
FileTrace files to trace LogFuncFlow. Defaults to all files.
May be specified multiple times, one file each (e.g.
export RSYSLOG_DEBUG="LogFuncFlow FileTrace=vm.c FileTrace=expr.c"
PrintFuncDB content of the debug function database whenever debug information is output (e.g. abort case)!
PrintAllDebugInfoOnExit (currently not implemented!)
PrintMutexAction as it happens. Useful for finding deadlocks and such.
NoLogTimeStamp Do not prefix log lines with a timestamp (default is to do that).
NoStdOut Do not emit debug messages to stdout. If RSYSLOG_DEBUGLOG is not set, this means no messages will be displayed at all.
Help Display a very short list of commands
RSYSLOG_DEBUGLOG writes debug messages to the specified log file in addition to stdout.
RSYSLOG_MODDIR directory in which loadable modules reside.

Support for message local and remote logging.


rsyslogd.conf

rules

selectionaction
selection ::= selector [;selector ]
selector ::= facility[,facility].[!][=]priority

Messages of the specified priority and higher are logged according to the given action
unless priority is preceeded by = meaning only this priority .

Preceeding priority with ! ignores this and higher priorities

[-][.]filename[template

A leading - requests suppresses sync on each messsage reducing I/O load.

filename should be a complete path unless preceeded by a . which references files relative to the current directory(usually /).

template a definition of the format of the log message. One defined locally or one of the reserved namesi like: RSYSLOG_TraditionalFileFormat, RSYSLOG_FileFormat, RSYSLOG_TraditionalForwardFormat, RSYSLOG_SysklogdFileFormat, RSYSLOG_ForwardFormat, RSYSLOG_SyslogProtocol23Format, RSYSLOG_DebugFormat, RSYSLOG_WallFmt, RSYSLOG_StdUsrMsgFmt, RSYSLOG_StdDBFmt, RSYSLOG_StdPgSQLFmt, RSYSLOG_spoofadr or RSYSLOG_StdJSONFmt.

facility: cron, ftp, lpr, kern, mail, news, syslog, uucp, daemon, user, local0, 1 … 6, local7,
auth, authpriv
(for security information of a sensitive nature,
*
(i.e. all)
priority emerg, alert, crit, err, notice, info, debug

none
*
(i.e. all)

Templates

Define the format of the log records.

list:: constant(value="vvv") property(name="pr") …

string:: string="specification …"

template(name="tpl3" type="string" string="%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n" )

property replacer

::%pr%
optional range%property:[i]:[j]:options% i,j are origin 1.
j can be $ to specify the end.
regular expresion%property:R:regx-end:options%
"%msg:R:.*Sev:. \(.*\) \[.*-end%"
[BRE|ERE]
field %property:F[,delimiter[+]][,i]:n[,j]:options% delimiter in decimal, default TAB Examples: space:32 or comma:44 or semicolon:59
+ multiple occurances of delimiter. Useful when parsing space filled fields
nth field number; origin 1. The ith through the jth position.

Example: comma seperated values, the 4th field: F44,4

options::

uppercase
lowercase
fixed-widthpads source with spaces up to the value of j
json encode the value so that it can be used inside a JSON field. Example a LineFeed is replaced by \n.
Cannot be used with jsonf or csv .
jsonf[:outname] be expressed as a JSON field. "fieldname"="value" where fieldname is given in the outname property (or the property name if none was assigned) and value is the end result of property replacer operation. value supports all property replacer options, like substrings, case converson, etc. Values are properly JSON-escaped, field names are not,
Cannot be used together with either json or csv . See Rainer's article
csv Comma Seperated Values as specified in RFC 4180. in quotes. Example
$template csvline,"%syslogtag:::csv%,%msg:::csv%" provide commas between the fields
drop-last-lf training LF is dropped. useful for PIX.
date-utc
date-mysql
date-rfc3164
date-rfc3164-buggyday use filled 2 digit day . Avoid when forwarding to remote hosts
date-rfc3339 ccyy-mm-ddTHH:MM:SS
date-unixtimestamp seconds since epoch
date-year ccyy
date-monthmm
date-daydd
date-hour HH 00..23
date-minuteMM
date-secondSS
date-subsecondsmmmmmm milliseconds (0 for a low precision timestamp)
date-tzoffshourzH time-zone offset hours
date-tzoffsmin zM
date-tzoffsdirection+|-
date-ordinal returns the ordinal for the given day, e.g. it is 2 for January, 2nd
date-week returns the week number
date-wday w weekday 0=Sunday, 1=Monday.. 6=Saturday.
date-wdaynameabbreviated english name of the weekday (e.g. Mon, Sat)
escape-cc replace control characters (ASCII 127 and values less then 32) with an escape sequence.
The sequence is #charval where charval is the 3-digit decimal value of the control character.
For example, TAB would be replaced by #009.
Requires $EscapeControlCharactersOnReceive is set to off.
space-cc replace control characters by spaces
Requires $EscapeControlCharactersOnReceive is set to off.
drop-cc drop control characters - the resulting string will neither contain control characters, escape sequences nor any other replacement character like space.
Requires $EscapeControlCharactersOnReceive is off.
compressspace compresses multiple spaces to one, after substring extraction,
sp-if-no-1st-sp A space if and only if the first character of the field is NOT a space. RFC 3164: specifies no delimiter between the tag and message text. Most implementation delimit by a space. Some log parsers to misinterpret what is the tag and what the message. The default template can contain a conditional space, which exists only if the message does not start with one.
secpath-drop Drops slashes (e.g. a/b becomes ab). Useful for secure pathname generation (with dynafiles).
secpath-replace Replace slash by underscore. (e.g. a/b becomes a_b). Useful for secure pathname generation (with dynafiles).

Properties

priority Numeric:PRI| syslogfacility | syslogpriority
Text: pri-text.
Example:
$template TraditionalFormatWithPRI,"%pri-text%: %timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n"

After log file name add a ; and the template name.


Started on RaspberryPi via /etc/init.d/rsyslog

See also

./doc subdirectory From: rsyslog.com

Regarding Memory usage:
On raspberrypi top reports VIRT:27,968 ; RES:1536; SHR: 1096

Debug output

9958.925905116:4007d000: rsyslogd 5.8.11 startup, compatibility mode 0, module path '', cwd:/var/log
9958.930543938:4007d000: caller requested object 'net', not found (iRet -3003)
9958.932890848:4007d000: Requested to load module 'lmnet'
9958.935678741:4007d000: loading module '/usr/lib/rsyslog/lmnet.so'
9958.938752623:4007d000: module of type 2 being loaded.
9958.941091533:4007d000: entry point 'isCompatibleWithFeature' not present in module
9958.942347484:4007d000: source file conf.c requested reference for module 'lmnet', reference count now 1
9958.943439443:4007d000: rsyslog runtime initialized, version 5.8.11, current users 1
9958.945011382:4007d000: source file syslogd.c requested reference for module 'lmnet', reference count now 2

9958.950204184:4007d000: GenerateLocalHostName uses 'raspberrypi'

9958.952694088:4007d000: omfile: using transactional output interface.

9958.955500980:4007d000: module of type 1 being loaded.
9958.957956886:4007d000: module of type 1 being loaded.
9958.959589824:4007d000: entry point 'beginTransaction' not present in module
9958.960286798:4007d000: entry point 'endTransaction' not present in module
9958.960987771:4007d000: source file omfwd.c requested reference for module 'lmnet', reference count now 3

9958.962230723:4007d000: module of type 1 being loaded.
9958.962463714:4007d000: entry point 'doHUP' not present in module
9958.963362679:4007d000: entry point 'beginTransaction' not present in module
9958.964361641:4007d000: entry point 'endTransaction' not present in module

9958.964630631:4007d000: module of type 1 being loaded.
9958.964846622:4007d000: entry point 'doHUP' not present in module
9958.965789586:4007d000: entry point 'beginTransaction' not present in module
9958.965995578:4007d000: entry point 'endTransaction' not present in module

9958.966842546:4007d000: module of type 1 being loaded.
9958.967063537:4007d000: entry point 'doHUP' not present in module
9958.967417524:4007d000: entry point 'beginTransaction' not present in module
9958.968164495:4007d000: entry point 'endTransaction' not present in module

9958.968405486:4007d000: module of type 1 being loaded.
9958.969365449:4007d000: entry point 'doHUP' not present in module
9958.969570441:4007d000: entry point 'beginTransaction' not present in module
9958.969766433:4007d000: entry point 'endTransaction' not present in module
9958.970248415:4007d000: rfc5424 parser init called
9958.971006386:4007d000: GetParserName addr 0x19ff4
9958.971207378:4007d000: module of type 3 being loaded.
9958.971886352:4007d000: Parser 'rsyslog.rfc5424' added to list of available parsers.
9958.972389333:4007d000: rfc3164 parser init called
9958.973297298:4007d000: module of type 3 being loaded.
9958.973529289:4007d000: Parser 'rsyslog.rfc3164' added to list of available parsers.
9958.973937273:4007d000: Parser 'rsyslog.rfc5424' added to default parser set.
9958.974674245:4007d000: Parser 'rsyslog.rfc3164' added to default parser set.
9958.975514213:4007d000: rsyslog standard file format strgen init called, compiled with version 5.8.11
9958.975736204:4007d000: module of type 4 being loaded.
9958.976600171:4007d000: entry point 'isCompatibleWithFeature' not present in module
9958.976844162:4007d000: Strgen 'RSYSLOG_FileFormat' added to list of available strgens.
9958.977224147:4007d000: traditional file format strgen init called, compiled with version 5.8.11
9958.977968118:4007d000: module of type 4 being loaded.
9958.978165111:4007d000: entry point 'isCompatibleWithFeature' not present in module
9958.979068076:4007d000: Strgen 'RSYSLOG_TraditionalFileFormat' added to list of available strgens.
9958.979309067:4007d000: rsyslog standard (network) forward format strgen init called, compiled with version 5.8.11
9958.979511059:4007d000: module of type 4 being loaded.
9958.980409025:4007d000: entry point 'isCompatibleWithFeature' not present in module
9958.980638016:4007d000: Strgen 'RSYSLOG_ForwardFormat' added to list of available strgens.
9958.981503983:4007d000: rsyslog traditional (network) forward format strgen init called, compiled with version 5.8.11
9958.981718974:4007d000: module of type 4 being loaded.
9958.982101960:4007d000: entry point 'isCompatibleWithFeature' not present in module
9958.982850931:4007d000: Strgen 'RSYSLOG_TraditionalForwardFormat' added to list of available strgens.
9958.984849854:4007d000: Called LogError, msg: WARNING: rsyslogd is running in compatibility mode. Automatically generated config directives may interfer with your rsyslog.conf settings. We suggest upgrading your config and adding -c5 as the first rsyslogd option.
9958.987708744:4007d000: Checking pidfile.
9958.993729513:4007d000: Writing pidfile /var/run/rsyslogd.pid.
Can't open or create /var/run/rsyslogd.pid.
Can't write pid.
^C


#  /etc/rsyslog.conf    Configuration file for rsyslog.
# 1/27/13 DGG
#### MODULES :
$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog   # provides kernel logging support
#$ModLoad immark  # provides --MARK-- message capability

# provides UDP syslog reception    DGG enabled
$ModLoad imudp
$UDPServerRun 514

# provides TCP syslog reception   DGG enabled
$ModLoad imtcp
$InputTCPServerRun 514

#### RULES :
# First some standard log files.  Log by facility.
auth,authpriv.*         /var/log/auth.log
*.*;auth,authpriv.none         -/var/log/syslog
#cron.*             /var/log/cron.log
daemon.*               -/var/log/daemon.log
kern.*                 -/var/log/kern.log
mail.*;news.*;lpr.*        -/var/log/unused.log
user.*                 -/var/log/user.log
dhclient.info           -/var/dhcpclient.log

# Some "catch-all" log files.
*.=debug;auth,authpriv.none     -/var/log/07_debug.log
*.=crit             -/var/log/02_crit.log
*.err               -/var/log/03_err.log
*.warn              -/var/log/04_warn.log
*.notice            -/var/log/05_notice.log
*.info              -/var/log/06_info.log
*.info              -/var/log/06_info.log2
*.emerg                         -/var/log/01_crit.log
cron,daemon.none;       -/var/log/messages

# Emergencies are sent to everybody logged in.
*.emerg             :omusrmsg:*

# place spool and state files
$WorkDirectory /var/spool/rsyslog

# Include all config files in /etc/rsyslog.d/ DGG: There aren't any
$IncludeConfig /etc/rsyslog.d/*.conf

#### GLOBAL DIRECTIVES :  Set the default permissions for all log files.
$FileOwner root
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022

# The named pipe /dev/xconsole is for the `xconsole' utility.
# To use it, invoke `xconsole' with the `-file' option:
#
#    $ xconsole -file /dev/xconsole [...]
#
# NOTE: adjust the list below, or you'll go crazy if you have a reasonably   busy site..
daemon.*;*.=debug;*.=info;*.=notice;*.=warn |/dev/xconsole 
Drop in replacment for syslog. Uses traditional syslog.conf and acts like the original syslogd . Used on raspberryPi and mega onion. (tools like phpLogCon can be used to view the log data.)

errors

local7.warning liblogging-stdlog: action 'action 7' suspended, next retry is Wed May 15 12:43:59 2019 [v8.24.0 try http://www.rs
try rsyslogd -N 1 |more

DHCP option 7 specifies the syslog server.