newsyslog

maintain system log files to manageable sizes

anacron even newer version anticapates system not being on when logs are supposed to be rotated.

sudo newsyslog [-CFNnrsv] [-R tagname] [-a directory] [-d directory] [-f config_file] [log]

Example:
sudo newsyslog -vn | grep -v skipping

newsyslog -r -s -v -f insuranceLog.conf -C -d /Volume/PRODUCTION/insuranceLogs

Run periodically by cron ( or launchd in Mac OS) to archive log files
Can be run by nonpriveledge users with their own or group configuration directing archives to specific directories.

Archives generations be moving the current log to log.0̸ has the logfile.1
The archived logs can be compressed.
Initailzates log with correct attributes.

    A log will be archived :
  1. because it's large
  2. because It's a good time   (if time is specified, size is not checked. Consider including 2 entries in config)
  3. it is Forced or Rotated by command options.

-v verbose.
-n Do not cutover logs. With -v display what it would happen(example). For testing config file.
-r run if not root. Don't send HUP to syslogd. Used in testing. ( will not violate file premissions!)
-f config_file default /etc/newsyslog.conf
-d directory files are relative to directory.
Allows archiving of logs outside the root preventing its filesystem from filling up.
With -r premits individual users to specify their own log directory.
-a directory for archive. May be relative to the path of log file
Subdirectorues willbe created as needed. -a directories are unaffected.
-R tagname Rotate a list of files.
tagname used in the messages written to log files which are cutover.
-C create files with C in their config.
syslog may not write to logfiles if they don't exist.
-CC create files if given on the command-line or by Calling process.
-C or -CC only apply to those files.

Intended for programs to trigger a cutover.
Rules such as permissions, number of archive generations, compression … are followed.
When a daemon executes newsyslog -R tag , files must be are closed first and re-opened after newsyslog returns.
The caller should specify -s to prevent newsyslog from signaling itself. Skipping the signal step causes newsyslog return sooner by avoiding a wait.

-N no cutovers. useful with -C or -CC
-F Force trim. Provide fresh logs containng only recent entries.
-s supress signaling to caller. May inhibit compression.
Useful with -R

Specifing logsfiles on the command line supress other log processing. The granularity is dependent on how often it is to run, i.e. if not often enough a log may exceed size

Files

/etc/newsyslog.conf
Initial configuration refers to syslog files located in /var/log but can be used with any files.

N.B. symbolic links (like from /var/log ) will end up in their "logfilename" location and become unlinked on the first rotation. i.e. don't use sym links for /var/log files, however it's OK for subdirectories of /var/log like DiagnosticMessages, asl, Accounts, apache2!


configuration

Each line has five required fields and optional fields, separated with whitespace.
A # causes the rest of the line to be ignored, unless escaped with \ .

Required:
<default>
or
logfile
<default> is used when a logfile is given on the command line which is not in the config.
owner:group Names from /etc/passwd : /etc/group or numeric ID. Default: root:admin.
ugo numeric file mode (permissions) when creating logfile. Example 640 : user read+write, group read, others none.(see chmod).
n number of archive generations in addition to the current log
KB maximun size before cutover occurs.
Rather than using a large size, consider retaining more generations.

If KB is an asterisk (*) the log file will not be cutover based on size.

Optional:
when interval, time, or both.
An optional interval in hours, followed by an @ (at-sign) and a time.

$ day of once a day, once a week, or once a month.

If a time is specified, the log file will only be cutover if newsyslog is run within one hour of the specified time.
If an interval is specified, the log file will be cutover if that many hours have passed since the last rotation.
When both a time and an interval are specified then both conditions must be satisfied for the rotation to take place.

For time, the lead-in character is an @ at-sign.

[[[[[cc]yy]mm]dd][T[hh[mm[ss]]]]].
Optional date fields default to the appropriate component of the current date; Optional time fields default to midnight; (not the best as many other things will trigger then as well)
For example if today is January 22, 1999, these are equivalent:

      19990122T000000
        990122T000000
          0122T000000
            22T000000
              T000000
              T0000
              T00
            22T
              T 
Day, week, and month time format:
The lead-in character $.
$[Dhh], [Ww[Dhh]], and [Mdd[Dhh]] .
Optional time fields default to midnight.
  • hh hours, 0..23
  • w day of week, 0..6, 0 = Sunday
  • dd day of month, 1..31, or L or l the last day of the month.

Examples:

      $D0     daily at midnight (same as @T00)
      $D23    daily at 23:00 (same as @T23)
      $W0D23  weekly on Sunday at 23:00
      $W5D16  weekly on Friday at 16:00
      $M1D0   first day of every month at midnight (i.e., the start of the day; same as @01T00)
      $M5D6   every 5th day of month at 6:00 (same as @05T06)

If when contains an asterisk (*), log rotation will not depend on a time.

B log is a binary or has special format.
An ASCII message is NOT inserted into log indicating when and why the log file was rotated.
C create log if -C was specified on the command line.
D set the UF_NODUMP flag when creating log file which affects how dump treats the log file when making a backup.
G file_name is a shell glob pattern
J bzip2 is used to compress rotated log (15% smaller)
Z gzip is used to compress rotated log (faster)
N no process is signaled when this log file is rotated. Requires that the first line in that file be a negative value to distinguish it from a process ID.
/path_to_pid_file file containing a daemon's process ID or
to find a group process ID if U was specified.
A signal_number is sent the process ID contained in this file.
If not present, then SIGHUP will be sent to syslogd, unless N has been specified.
U file specified by path_to_pid_file will contain the ID for a process group instead of a process.
signal_number sent to the daemon process (or to all processes in a process group, if U was specified). default SIGHUP
- minus sign is a placeholder

See also

bzip2, gzip, syslog, newsyslog.conf, chown, syslogd
rotlogs.sh

Example of config file

#Contents of maillog, messages, and lpd-errs may be confidential.   (not logrotate no longer comes with MacOSX)

#   sudo newsyslog -vn|egrep --invert-match --line-buffered "skip|will"    # show only actions expected to be taken

#                                                       @ interval, $ day...
#                                                                         J:Compress; B:dont add "new" message
# logfilename                       [owner:group] mode count size when  flags [/pid_file] [sig_num]
/var/log/01_alert.log                 root:staff  640  3     100    *      J
/var/log/02_crit.log                  root:staff  640  3     100    *      J
/var/log/03_err.log                   root:staff  640  3     100    *      J
/var/log/04_warn.log                  root:staff  640  3     100    *      J
/var/log/appfirewall.log              root:staff  640  3     100    *      J
/var/log/install.log                  root:staff  640  3     100    *      J
/var/log/mail.log                     root:staff  640  1     100    *      J
/var/log/secure.log                   root:staff  640  5     100    *      J
# seems that if both  size and time are specified, size is ignored! so here's 2 entries
# which confuses newsyslog which rotates logs assigning odd numbers and only compressing sometimes
/var/log/system.log                   root:staff  640  7     200    *      J
/var/log/system.log                   root:staff  640  7     200    @T0007 J
# added 7/31/13 after 165KB
/var/log/kernel.log                   root:staff  640  5     200    *      J
# wtmp doesn;t exist ??!!
/var/log/wtmp                         root:staff  644  3     *      @01T05 B

# added 7/31/13 ; previously listed as secret recently growing FAST trim Previously expected 00:00 8/31/13
/var/log/wifi.log                     root:staff  640  3     100    *      J

#added 6/23/13:
/var/log/opendirectoryd.log           root:staff  644  3     100    *      J

/Volumes/DATA/log/05_notice.log       root:staff  644  3     200    *      J
/Volumes/DATA/log/06_info.log         root:staff  644  3     200    *      J
/Volumes/DATA/log/07_debug.log        root:staff  644  3     200    *      J

/var/log/apache2/access_log           root:staff  644  3     *    $W6D23   J  #Saturday night
/var/log/apache2/error_log            root:staff  644  3     *    $W6D23   J


Examples

sudo newsyslog -vn  # test new configuration
/var/log/01_alert.log <3J>: size (Kb): 4 [100] --> skipping
/var/log/02_crit.log <3J>: size (Kb): 4 [100] --> skipping
/var/log/03_err.log <3J>: size (Kb): 24 [100] --> skipping
/var/log/04_warn.log <3J>: size (Kb): 16 [100] --> skipping
/var/log/appfirewall.log <3J>: size (Kb): 8 [100] --> skipping
/var/log/install.log <3J>: size (Kb): 68 [100] --> skipping
/var/log/mail.log <1J>: size (Kb): 24 [100] --> skipping
/var/log/secure.log <5J>: size (Kb): 80 [100] --> skipping
/var/log/system.log <7J>: size (Kb): 8 [200] --> skipping
/var/log/system.log <7J>: --> will trim at Thu Aug  1 00:07:00 2013
/var/log/kernel.log <5J>: size (Kb): 164 [200] --> skipping
/var/log/wtmp <3>: does not exist, skipped.
/var/log/wifi.log <3J>: size (Kb): 20 [100] --> skipping
/var/log/opendirectoryd.log <3J>: size (Kb): 4 [100] --> skipping
/Volumes/DATA/log/05_notice.log <3J>: size (Kb): 184 [200] --> skipping
/Volumes/DATA/log/06_info.log <3J>: size (Kb): 152 [200] --> skipping
/Volumes/DATA/log/07_debug.log <3J>: size (Kb): 160 [200] --> skipping
/var/log/apache2/access_log <3J>: --> will trim at Sat Aug  3 23:00:00 2013
/var/log/apache2/error_log <3J>: --> will trim at Sat Aug  3 23:00:00 2013
/Library/Logs/slapconfig.log <10J>: does not exist, skipped.
/var/log/kernel.log <5J>: size (Kb): 164 [1000] --> skipping
/Library/Logs/named.log <5J>: does not exist, skipped.
/var/log/wifi.log <3J>: --> will trim at Thu Aug  1 00:00:00 2013
sudo newsyslog -vF # Force in preparation for new activities

/var/log/appfirewall.log <3J>: size (Kb): 20 [100] --> trimming log....
/var/log/ftp.log <1J>: does not exist, skipped.
/var/log/hwmond.log <3J>: does not exist, skipped.
/var/log/install.log <3J>: size (Kb): 12 [100] --> trimming log....
/var/log/ipfw.log <3J>: does not exist, skipped.
/var/log/lookupd.log <1J>: does not exist, skipped.
/var/log/lpr.log <1J>: does not exist, skipped.
/var/log/mail.log <1J>: size (Kb): 4 [100] --> trimming log....
/var/log/ppp.log <1J>: does not exist, skipped.
/var/log/secure.log <5J>: size (Kb): 4 [100] --> trimming log....
/var/log/system.log <7J>: --> trimming log....
/var/log/wtmp <3>: does not exist, skipped.
/Library/Logs/slapconfig.log <10J>: does not exist, skipped.
/var/log/kernel.log <5J>: size (Kb): 512 [1000] --> trimming log....
/Library/Logs/named.log <5J>: does not exist, skipped.
/var/log/wifi.log <3J>: does not exist, skipped.
Signal all daemon process(es)...
Notified daemon pid 20 = /var/run/syslog.pid
Pause 10 seconds to allow daemon(s) to close log file(s)
Compress all rotated log file(s)...  



2/12/18 note: no nn-llll.log files rotated. /var/log/ftp.log <5J>: size (Kb): 4 [1000] --> trimming log.... /var/log/hwmond.log <5J>: size (Kb): 4 [1000] --> trimming log.... /var/log/ipfw.log <5J>: size (Kb): 4 [1000] --> trimming log.... /var/log/lpr.log <5J>: size (Kb): 4 [1000] --> trimming log.... /var/log/ppp.log <5J>: size (Kb): 4 [1000] --> trimming log.... /var/log/wtmp <3>: size (Kb): 0 [1000] --> trimming log.... /Library/Logs/slapconfig.log <10J>: size (Kb): 4 [1000] --> trimming log.... /var/log/slapd.log <10J>: size (Kb): 4 [1000] --> trimming log.... /var/log/xscertd.log <5J>: size (Kb): 4 [10000] --> trimming log.... /Library/Logs/named.log <5J>: --> trimming log.... /Library/Logs/named.log <5J>: --> trimming log.... /var/log/wifi.log <2>: size (Kb): 8 [10] --> trimming log.... /var/log/corecaptured.log <3J>: size (Kb): 4 [10] --> trimming log.... /Volumes/DATA/dgerman/Library/Logs/DiskUtility.log <1J>: size (Kb): 4 [100] --> trimming log.... /Volumes/DATA/dgerman/Library/Logs/GoogleSoftwareUpdateAgent.log <1J>: size (Kb): 4 [50] --> trimming log.... /Volumes/DATA/dgerman/log/acctlg <2J>: size (Kb): 4 [50] --> trimming log.... /Volumes/DATA/dgerman/log/sshgd.sshvvv.log <1J>: size (Kb): 4 [10] --> trimming log.... /var/db/diagnostics/logdata.Persistent.* <1>: size (Kb): 4 [10] --> trimming log.... /var/log/authpriv.log <2J>: size (Kb): 40 [50] --> trimming log.... /Library/Logs/DiagnosticReports <2J>: size (Kb): 0 [50] --> trimming log.... newsyslog: error reading from /Library/Logs/DiagnosticReports: Is a directory log >lt 1484 07:40 04_warn.log 9849 07:40 03_err.log 8169 07:40 02_crit.log 2861 07:40 01_alert.log 2861 07:40 00_em.log 115018 07:41 secure.log 142 07:41 ftp.log.0 79 07:41 ftp.log 142 07:41 hwmond.log.0 79 07:41 hwmond.log 142 07:41 ipfw.log.0 79 07:41 ipfw.log 142 07:41 lpr.log.0 79 07:41 lpr.log 142 07:41 ppp.log.0 79 07:41 ppp.log 0 07:41 wtmp 142 07:41 slapd.log.0 79 07:41 slapd.log 142 07:41 xscertd.log.0 79 07:41 xscertd.log 7919 07:41 wifi.log.0 79 07:41 wifi.log 156 07:41 corecaptured.log.0 79 07:41 corecaptured.log 37166 07:41 authpriv.log.0 79 07:41 authpriv.log 5157 07:41 system.log

Launchctl file

/System/Library/LaunchDaemons/com.apple.newsyslog.plist

rearranged by DGG for clarity

   <key>Label</key> <string>com.apple.newsyslog</string>
    <key>ProgramArguments</key> <array> <string>/usr/sbin/newsyslog</string> </array>
    <key>LowPriorityIO</key> <true/>
    <key>Nice</key> <integer>1</integer>
    <key>StartCalendarInterval</key> <dict> <key>Minute</key> <integer>30</integer> </dict>

Errors

newsyslog: must have root privs or specify -r to run as nonproviledged user.
newsyslog: chmod(/var/log/wifi.log.1.bz2) in change_attrs: No such file or directory or rotation failed perhaps due to insufficient accrss rights.

see also

Apple Syslog Manager
logrotate