atsar

system activity report

atsar [ -flags ] seconds [ n ]

atsar [ -flags ] [ -s time ] [ -e time ] [ -i sec ] [ -f file | -n day# ]

Delivers statistics. Samples cumulative activity counters at n intervals of t seconds.

sends the output for every option specified to standard output.

If only one type of statistical counter is requested, the header is printed once and after every t seconds the statistical counters are shown for that period.
If several options are requested, a header is printed per sample followed by the statistical counters for that period. When no sampling interval specified, data is extracted from a previously recorded file, either the one specified by -f default: the daily activity data file /var/log/atsar/atsadd for the current day dd (day of month).
Alternatively -n can be used to specify the day of the month

The starting and ending times of the report can be defined using -s and -e time of the form hh:mm. The -i option selects records at sec second intervals. Otherwise, all intervals found in the data file are reported.

-S don't Supress times stamps if multiple lines are from sam time period (easier for post-processing).
-A All possible options.
-u CPU utilization (average and per cpu).
-P processes and load-averages.
-d Limited utilization of disks
-D utilization of disks and disk-partitions.
-r memory and swap
-p paging and swapping
-I the number interrupts per second.
-v utilization of kernel-tables.
-y utilization of tty's (serial interfaces).
-l network interfaces.
-L errors for network-interfaces.
-w IP network traffic
-W errors for IP traffic.
-m
-kv6
General ICMP layer activity.
-M
-Kv6
Per-type ICMP layer activity.
-t
-gv6
network traffic
-T
-G
errors for traffic.
-U
-hv6
UDP network traffic.
-jv6 TCPv6 socket-utilization.
-N NFS server- and client-requests.
-E errors for NFS server- and client-requests.
-V behaviour of NFS-server.
-R RPC-calls for NFS-requests. A transfer-rate is given per type of RPC-request (% total num- ber of RPC-requests).
-F Optional the ftp-traffic. This flag only produces relevant output if the script atsaftp is activated with regular intervals (by the script atsa1 ). Note that the names of the FTP-logfiles have to be specified in the /etc/atsar.conf configuration-file.
-H Optional the http-traffic. This flag only produces relevant output if the script atsahttp is activated with regular intervals (by the script atsa1 ). Note that the names of the HTTP-logfiles have to be specified in the /etc/atsar.conf configuration-file.

OUTPUT DESCRIPTION

Depending on the flag, a columns with output-values are produced. The values are mostly presented as a number of events per second.
-u per cpu:
usr% user-mode with a nice-value of zero (default) or negative (i.e. higher priority).
nice% user-mode with a nice-value larger than zero (i.e. lower priority ).
sys% system-mode (kernel-text) for all active processes.
A high percentage indicates a lot of system calls being issued. Interrupt-handling is also part of this percentage.
irq% for interrupt-handling .
softirq% for soft interrupt-handling .
idle% unused because all processes are in a wait-state but not waiting for disk-I/O).
wait% unused At least one of processes in wait-state for disk-I/O
-P
pswtch/s process-switches (aka context-switches) per second on all cpu's. A process-switch occurs at the moment that the active process (i.e. the process using a cpu) enters a wait-state or has used its time-slice completely; another process will then be chosen to use the cpu.
runq processes which are currently waiting for CPU-scheduling (including the processes which are currently active on a CPU).
nrproc Total processes present
lavg1 , lavg5 ,lavg15 Load-average reflecting the average processes in the run-queue during the mm minute(s).
-d per active physical disk:
device Disk-drive number (major-minor).
read/s, rdKb/s
write/s ,wrKb/s
rdwr/s total of read/s + write/s)
-D per used partition of a physical disk:
partition Diskpartition) name.
busy Busy-% the physical disk (i.e. the portion of time that the device was busy handling requests). only for entire physical disk.
read/s ,Kbyt/r
write/s ,Kbyt/w
avque Average number of disk-requests outstanding during the time that the disk is busy. for partitions which represent the entire physical disk.
avserv Average milliseconds needed by a request on this physical disk (seek, latency and data-transfer) only entire physical disk.
-r memory and swap utilization snapshot
memtot, memfree Total, Available
buffers, cached metadata-blocks, cache data-blocks
slabmem dynamically allocated memory by the kernel
swptot, swpfre Total, Available swap
atsar -r -n 11|head

Linux  pi93graf  4.14.98+  #1200 Tue Feb 12 20:11:02 GMT 2019  armv6l  04/11/2019

21:16:16  memtot memfree buffers   cached  slabmem      swptot swpfree  _mem_
21:17:27    433M     91M     21M     128M      13M       2047M   2047M
21:17:32    433M     91M     21M     128M      13M       2047M   2047M
21:20:01     linux restarts
21:30:02    433M    155M     23M     103M      13M       2047M   2047M


-p frequency of paging and swapping:
pagein/s , pageout/s implies normal data-access (so not necessarily indicating lack of memory).
swapin/s , swapout/s
fork/s new processes started
atsar -p -n 11|head

Linux  pi93graf  4.14.98+  #1200 Tue Feb 12 20:11:02 GMT 2019  armv6l  04/11/2019

21:16:16   pagein/s pageout/s     swapin/s swapout/s         fork/s      _page_
21:17:27       0.45     22.48         0.00      0.00           0.99
21:17:32       0.00     32.80         0.00      0.00           0.00
21:20:01     linux restarts
21:30:02     581.67    185.21         0.00      0.00           3.03
21:30:17       0.00     19.73         0.00      0.00           0.13
21:30:32       0.00     25.33         0.00      0.00           0.00


-I interrupt-frequency, reported per irq per cpu.
iq.. interrupts per second for this interrupt-type. /proc/interrupts shows device
-v utilization limited kernel-resources: current and max
superb-sz super-blocks which is maintained by the kernel for mounted filesystems.
inode-sz incore-inodes maintained by the kernel. One entry is needed for every disk-file which is currently open (at least once).
file-sz open-file entries maintained by the kernel. One entry is needed for every open-request for a disk-file. If a file is currently opened by five processes, one incore-inode and five open-file entries.
dquota-sz occupation of the collection of disk-quota entries.
flock-sz file-locks.
-y utilization of tty's:
port port-number
xmit/s , recv/s interrupts
frer/s framing-errors
parer/s parity-errors
ovrun/s overrun-errors
brk/s breaks
-l utilization of network-interfaces:
inpck/s , tpck/s packets
inbyt/s , otbyt/s bytes
incmpr/s otcmpr/s compressed packets (SLIP).
inmcst/s multicast-packets received from this interface
iface Name of the interface.
-L failures for network-interfaces:
inerr/s bad packets received
oterr/s packet-transmit problems
coll/s collisions encountered while transmitting packets.
indrop/s received packets dropped due to lack of buffer-space
otdrop/s transmitted packets dropped due to lack of buffer-space
infram/s frame alignment-errors on received packets.
otcarr/s carrier-errors on transmitted packets.
iface Name
-w utilization of the IPv4-layer (formal snmp-names between brackets):
inrecv/s input datagrams received from interfaces per second, including those received in error (ipInReceives).
outreq/s Number of IP datagrams that local higher-layer protocols supplied to IP in requests for transmission (ipOutRequests).
indeliver/s input datagrams per second that have been succesfully delivered to higher protocol-layers (ipInDelivers).
forward/s input datagrams per second for which this entity was not their final IP destination, as a result of which an attempt was made to forward (ipForwDatagrams).
reasmok/s IP datagrams succesfully reassembled per second (ipReasmOKs).
fragcreat/s IP datagram fragments generated per second at this entity (ipFragCreates).
-W the failures which were detected in the IPv4-layer (formal snmp-names between brackets): indsc/s input IP datagrams per second for which no problems were encountered to prevent their continued processing but that were discarded, e.g. for lack of buffer space (ipInDiscards). hder/s input datagrams per second discarded due to errors in the IP header (ipInHdrErrors). ader/s Number of input datagrams discarded because the IP address in the destination field was not valid to be received by this entity (ipInAddrErrors). unkp/s inbound packets per second that were discarded because of an unknown or unsupported protocol (ipInUnknownPro- tos). ratim/s timeout-situations per second while other fragments were expected for successful reassembly (ipReasmTimeout). rfail/s failures detected per second by the IP reassembly algorithm (ipReasmFails). otdsc/s Number of output IP datagrams per second for which no problems were encountered to prevent their continued processing but that were discarded, e.g. for lack of buffer space (ipOutDiscards). nort/s IP datagrams per second discarded because no route could be found (ipOutNoRoutes).
-t the utilization of the TCP-layer (formal snmp-names between brackets): insegs/s received segments per second, including those received in error (tcpInSegs). otsegs/s transmitted segments per second, excluding those containing only retransmitted octets (tcpOutSegs). actopen/s active opens per second that have been supported by this entity (tcpActiveOpens). pasopen/s passive opens per second that have been supported by this entity (tcpPassiveOpens). nowopen connections currently open (snapshot), for which the state is either ESTABLISHED or CLOSE-WAIT (tcpCur- rEstab). socknow TCPv4-sockets currently open (snapshot). sockmax Maximum parallel TCPv4-sockets ever open. This value is not always supported by the kernel.
-T provides information about the failures which were detected in the TCP-layer (formal snmp-names between brackets): inerr/s received segments per second received in error (tcpInErrs). retrans/s retransmitted segments per second (tcpRetransSegs). attfail/s failed connection attempts per second that have occurred at this entity (tcpAttemptFails). estabreset/s resets per second that have occurred at this entity (tcpEstabResets). outreset/s transmitted segments per second containing the RST flag (tcpOutRsts).
-U the utilization of the UDPv4-layer (formal snmp-names between brackets): indgram/s UDP datagrams per second delivered to UDP users (udpInDatagrams), otdgram/s UDP datagrams transmitted per second from this entity (udpOutDatagrams), inerr/s received UDP datagrams per second that could not be delivered for reasons other than the lack of an applica- tion at the destination port (udpInErrors). noport/s received UDP datagrams per second for which there was no application at the destination port (udpNoPorts). socknow UDP-sockets currently open (snapshot). sockmax Maximum parallel UDP-sockets ever open. This value is not always supported by the kernel.
-m the general utilization of the ICMPv4-layer and some information per type of ICMP-message (formal snmp-names between brackets):
intot/s ICMP messages (any type) received at this entity (icmpInMsgs). outtot/s ICMP messages (any type) transmitted from this entity (icmpOutMsgs). inecho/s ICMP Echo (request) messages received (icmpInEchos). inerep/s ICMP Echo-Reply messages received (icmpInEchoReps). otecho/s ICMP Echo (request) messages transmitted (icmpOutEchos). oterep/s ICMP Echo-Reply messages transmitted (icmpOutEchoReps).
-M other types of ICMPv4-messages (formal snmp-names between brackets): ierr/s ICMP messages received but determined to have ICMP-specific errors (icmpInErrors). isq/s ICMP Source Quench messages received (icmpInSrcQuenchs). ird/s ICMP Redirect messages received (icmpInRedirects). idu/s ICMP Destination Unreachable messages received (icmpInDestUnreachs). ite/s ICMP Time Exceeded messages received (icmpOutTimeExcds). oerr/s ICMP messages transmitted but determined to have ICMP-specific errors (icmpOutErrors). osq/s ICMP Source Quench messages transmitted (icmpOutSrcQuenchs). ord/s ICMP Redirect messages transmitted (icmpOutRedirects). odu/s ICMP Destination Unreachable messages transmitted (icmpOutDestUnreachs). ote/s ICMP Time Exceeded messages transmitted (icmpOutTimeExcds).
-g the utilization of the IPv6-layer (formal snmp-names between brackets): inrecv/s input IPv6-datagrams received from interfaces per second, including those received in error (ipv6IfStatsInRe- ceives). outreq/s Number of IPv6-datagrams that local higher-layer protocols supplied to IP in requests for transmission (ipv6IfStatsOutRequests). This counter does not include any forwarded datagrams. inmc/s multicast packets that have been received by the interface (ipv6IfStatsInMcastPkts). outmc/s multicast packets that have been transmitted to the interface (ipv6IfStatsOutMcastPkts). indeliv/s IP datagrams succesfully delivered to IPv6 user-protocols, including ICMP (ipv6IfStatsInDelivers). reasmok/s IPv6 datagrams succesfully reassembled (ipv6IfStatsReasmOKs). fragcre/s IPv6 datagram fragments generated at this entity (ipv6IfStatsOutFragCreates).
-G the failures which were detected in the IPv4-layer (formal snmp-names between brackets): indsc/s Number of input IPv6 datagrams for which no problems were encountered to prevent their continued processing but that were discarded, e.g. for lack of buffer space (ipv6IfStatsInDiscards). hder/s input datagrams discarded due to errors in the IPv6 header (ipv6IfStatsInHdrErrors). ader/s input datagrams discarded because the IPv6 address in the destination field was not valid to be received by this entity (ipv6IfStatsInAddrErrors). unkp/s Number of locally-addressed datagrams that were discarded because of an unknown or unsupported protocol (ipv6IfStatsInUnknownProtos). ratim/s timeout-situations while other IPv6 fragments were expected for successful reassembly. rfail/s failures detected by the IPv6 reassembly-algorithm (ipv6IfStatsReasmFails). otdsc/s output IPv6 datagrams for which no problems were encountered to prevent their continued processing but that were discarded, e.g. for lack of buffer space (ipv6IfStatsOutDiscards). nort/s IPv6 datagrams discarded because no route could be found (ipv6IfStatsInNoRoutes).
-j the utilization of the TCPv6-sockets: socknow TCPv6-sockets currently open (snapshot). sockmax Maximum parallel TCPv6-sockets ever open. This value is not always supported by the kernel.
-h the utilization of the UDPv6-layer (formal snmp-names between brackets): indgram/s UDPv6 datagrams delivered to UDP users (udpInDatagrams), otdgram/s UDPv6 datagrams transmitted from this entity (udpOutDatagrams), inerr/s Number of received UDPv6 datagrams that could not be delivered for reasons other than the lack of an appli- cation at the destination port (udpInErrors). noport/s received UDPv6 datagrams for which there was no application at the destination port (udpNoPorts). socknow UDPv6 sockets currently open (snapshot). sockmax Maximum parallel UDPv6 sockets ever open. This value is not always supported by the kernel.
-k ICMPv6-layer and some information per type of ICMP-message (formal snmp-names between brackets):
intot/s ,outtot/s messages (any type)
inerr/s messages received which had ICMP-specific errors, such as bad ICMP checksums, bad length, etc
innsol/s,otnsol/s Neighbor Solicit messages received, sent
innadv/s ,otnadv/s Neighbor Advertisement messages
-K other types of ICMPv6-messages (formal snmp-names between brackets):
iecho/s ICMP Echo (request) messages received (ipv6IfIcmpInEchos).
ierep/s ICMP Echo-Reply messages received (ipv6IfIcmpInEchoReplies).
oerep/s ICMP Echo-Reply messages transmitted (ipv6IfIcmpOutEchoReplies).
idu/s ICMP Destination Unreachable messages received (ipv6IfIcmpInDestUnreachs).
odu/s ICMP Destination Unreachable messages transmitted (ipv6IfIcmpOutDestUnreachs).
ird/s ICMP Redirect messages received (ipv6IfIcmpInRedirects).
ord/s ICMP Redirect messages transmitted (ipv6IfIcmpOutRedirect).
ite/s ICMP Time Exceeded messages received (ipv6IfIcmpInTimeExcds).
ote/s ICMP Time Exceeded messages transmitted (ipv6IfIcmpOutTimeExcds).
-N the utilization of the NFS-layer, acting as NFS-client as well as NFS- server:
svrpc/s RPC-requests handled by this NFS-server system per second.
clrpc/s RPC-requests issued by this NFS-client system per second.
sudp/s requests handled via UDP by this NFS-server system per second.
stcp/s requests handled via TCP by this NFS-server system per second.
stcon/s TCP-connections established by this NFS-server system per second.
cudp/s requests issued via UDP by this NFS-client system per second.
ctcp/s requests issued via TCP by this NFS-client system per second.
ctcon/s TCP-connections established by this NFS-client system per second.
-E the failures which were detected in the NFS-layer: svbadfmt/s RPC-requests received by the NFS-server system with a bad format (version-number if RPC-packet not recognized). svbadauth/s RPC-requests received by the NFS-server system with a bad authorization. svbadclnt/s Meaning unknown. clretrans/s RPC-request which have been retransmitted by the NFS-client due to a timeout. clauthrefresh/s times that the credentials have been refreshed by the NFS-client due to rejection by the NFS- server.
-V the behaviour of the NFS-server: rchit/s succesful accesses on the reply-cache per second. rcmiss/s unsuccesful accesses on the reply-cache per second. %hit Hit-ratio for the reply-cache. fhstale/s stale-errors for file-handles. iord/s Number of bytes returned to read-requests per second. This counter is not supported by all NFS-versions (contains zero). iowr/s bytes passed in write-requests per second. This counter is not supported by all NFS-versions (contains zero). thrnow avaliable threads (nfsd-servers). This counter is not supported by all NFS-versions (contains zero). tlast/s Number of times that the last available thread is activated per second. This counter is not supported by all NFS-ver- sions (contains zero).
-R the types of RPC-calls used by the NFS-layer (NFS-client as well as NFS- server). Note that per RPC-type the usage-percentage is shown.
       gat       Get Attribute.  sat       Set Attribute.  
       lku       Lookup.  
       rln       Read Link.     rd        Read.  
       wrc       Write Cache.   wr        Write.  
       cre       Create.        rm        Remove.  
       rnm       Rename.  
       lnk       Link.          sln       Symbolic Link.  
       mkd       Mkdir.         rmd       Rmdir.  
       rdd       Read Directory.  
       fst       Statfs.
-F FTP-traffic. atsaftp should be installed and called by atsa1 with intervals. ; only when viewing long-term counters from a history-file.
xfers/s FTP-transfers per second.
kbytes/s Kbytes transferred during the interval.
avg_time , avg_kbytes Average per transfer.
direct transmitted (output) or received (input)
name Symbolic name (as specified in the /etc/atsar.conf file) of the xfer-log
-H Apache HTTP-traffic. atsahttp by the script atsa1 with regular intervals.
only when viewing long-term counters from a history-file.
head/s get/s put/s post/s delete/s
name Symbolic name (as specified in the /etc/atsar.conf file) of the access-log file from which the usage-info is retrieved.

EXAMPLES To see today's processor-activity so far: atsar To watch TCP activity evolve for ten minutes (10 samples with sixty seconds interval): atsar -t 60 10 FILES /var/log/atsar/atsadd Daily data file, where dd are digits representing the day of the month. SEE ALSO atsadc(1)


atsadc, atsa1, atsaftp, atsahttp

counter-collection

atsadc [ seconds n ] [ ofile ] # Data collector

atsa1 [ seconds n ] #

atsaftp

atsahttp

System activity-data can be gathered on special request of a user [see atsar(1) ] or automatically, on a routine basis, as described here. Usually the kernel maintains statistical counters that are incremented as various system actions occur. These include counters for CPU utilization, disk utilization, memory utilization and various network statistics.

atsadc and the shell-script atsa1 are used to collect, save, and process these counters.

atsadc (the data collector) samples system data and writes in binary format to ofile or (default) to standard output.
If t and n are omitted, a special reset-record is written used when booting to a multi-user state, to mark the time at which the counters restart from zero. For example, the reset-mark can be added to the daily data by the command:

/usr/local/bin/atsadc /var/log/atsar/atsa`date +%d`

This entry is written to /etc/rc.d/init.d/atsar

atsa1 collects and stores data in the binary file /var/log/atsar/atsadd where dd is the current day of the month. The arguments t and n cause records to be written n times at an interval of t seconds, or once if omitted. log-files older than a week are removed once a day. following entries should be added to the /etc/cron.d directory to produce records every 20 minutes during working hours and hourly otherwise: 0 * * * 0-6 root /usr/local/bin/atsa1 20,40 8-17 * * 1-5 root /usr/local/bin/atsa1 atsaftp counts the new transfers registered in the FTP-logfile(s) since the previous time this script was acti- vated; the new counters are stored in the /var/log/atsar/ftpstat file in ASCII-format. The names of the FTP-logfiles to be watched are specified in the /etc/atsar.conf configuration-file. atsahttp counts the new transfers registered in the HTTP-logfile(s) since the previous time this script was activated; the new counters are stored in the /var/log/atsar/httpstat file in ASCII-format. The names of the HTTP-logfiles to be watched are specified in the /etc/atsar.conf configuration-file. Both scripts must be activated just before the program atsadc is started, which also collects these counters.

FILES

/var/log/atsar/atsadd Daily data file, where dd are digits representing the day of the month.