This format does NOT include data for any individual devices such as cpu, disk, network, nfs, lustre, process, slabs or tcp. If you do select any one of them collectl will force --verbose format.
The smaller the value the more accuracy as follows:
#<---Memory--> # Fragments smkj9576040
#<--------CPU--------> #cpu sys inter ctxsw
cpu | Percent of time the cpu was busy during the current interval averaged across all CPUs and is actually the total percentage of time the CPU in one of the following: system, user, nice, irq, soft-irq and steal. Note that this does NOT include time spend in I/O wait. |
sys | Percentage of time the cpu was executing in system mode during the current interval. This includes all those modes as above except user and nice to to determine the amount of time spent as a user you need to subtract these from the total cpu field. |
inter | Total number of interrupts/sec. |
ctxsw | Total number of context switches/sec. |
If you specify filtering with --dskfilt, the disks that match the pattern(s) will either be included or excluded from the the summary data. However, the data will still be collected so if recorded to a file can later be viewed.
#<---------------Disks----------------> #KBRead Reads Size KBWrit Writes Size
KBRead | KB read/sec |
Reads | Number of reads/sec |
Size | Average read size in KB. This field only included if --dskopts i or --iosize specified |
KBWrite | KB written/sec |
Writes | Number of writes/sec |
Size | Average write size in KB. This field only included if --dskopts i or --iosize specified |
#<---------------InfiniBand---------------> # KBIn PktIn Size KBOut PktOut Size Errs
KBIn | KB received/sec. |
PckIn | Packets received/sec. |
Size | Average incoming packet size in KB. This field is only included if --xopts i or --iosize included |
KBOut | KB sent/sec. |
PktOut | Packets sent/sec. |
Size | Average outgoing packet size in KB. This field is only included if --xopts i or --iosize included |
Errs | Count of current errors. Since these are typically infrequent, it is felt that reporting them as a rate would result in either not seeing them OR round-off hiding their values. |
#<----Files---> #Handle Inodes 5100 116110
Handles | Number of allocated file handles |
Inodes | Number of inodes in use |
Lustre data actually falls into one of 3 categories - client, mds and oss. Collectl determines the type of system it is running on (a system can have multiple personalities) and reports on all it finds, unless specifically selected via -L.
Lustre Client, collectl -sl
#<-------------Lustre Client-------------> # KBRead Reads Size KBWrite Writes Size
KBRead | KB/sec delivered to the client. |
Reads | Reads/sec delivered to the client, |
Size | Average read size in KB. This field only included if --iosize specified |
KBWrite | KB Writes/sec delivered to the storage servers. |
Writes | Writes/sec delivered to the storage servers. |
Size | Average write size in KB. This field only included if --iosize specified |
The following format of lustre client data is selected by including -OR and adds readahead statistics to the previous six, noting the Size fields are dependent on --iosize being specificed.
#<--------------------Lustre Client--------------------> # KBRead Reads Size KBWrite Writes Size Hits Misses
KBRead | KB/sec delivered to the client. |
Reads | Reads/sec delivered to the client, not necessarily from the lustre storage servers. |
Size | Average read size in KB. |
KBWrite | KB Writes/sec delivered to the storage servers. |
Writes | Writes/sec delivered to the storage servers. |
Size | Average write size in KB. |
Hits | Number of reads/sec from the lustre prefetch cache. |
Misses | Number of misses/sec from the prefetch cache which must then be satisfied by reading from the storage servers. |
Lustre MDS (Meta-Data Server)
The first format is for lustre versions 1.6.5 and beyond while the second format is for
earlier releases
#<--------Lustre MDS--------> #Gattr+ Sattr+ Sync Unlnk
#<--------Lustre MDS--------> #Gattr+ Sattr+ Sync Reint
Gattr+ | Total number of all getattr operations/sec. See Getattr, GttrLck and Gxattr in the verbose data section |
Sattr+ | Total number of all getattr operations/sec. See Setattr and Sxattr in the verbose data section |
Sync | Number of syncs/sec |
Unlnk | Number of file deletes/sec |
Reint | Number of reints/sec which include unlinks and setattrs. Since older version did not break out setattrs, they are not included in Sattr+. |
Lustre OSS (Object Storage Server), collectl -sc
#<--------------Lustre OST--------------> # KBRead Reads Size KBWrit Writes Size
KBRead | KB/sec read |
Reads | Reads/sec |
Size | Average read size in KB. This field only included if --iosize specified |
KBWrite | KB/sec written |
Writes | Writes/sec |
Size | Average write size in KB. This field only included if --iosize specified |
#<-----------Memory----------> #free buff cach inac slab map
free | Total free memory, which unfortunately is NOT the difference between total memory and the following amounts allocated to used memory. |
buff | Memory used as system buffers. |
cach | This is also commonly known as the file system buffer cache as buffered I/O uses this memory to cache the data. |
inac | Inactive memory. |
slab | Total memory allocated to slabs. |
map | Total mapped memory, which include AnonPages. |
If you specify filtering with --netfilt, the names that match the pattern(s) will either be included or excluded from the the summary data. However, the data will still be collected so if recorded to a file can later be viewed.
Also be sure to keep in mind that like all other data, network counters are always normalized to /sec rates. This means that something like errors, which themselves might be really small, could be reported as 0 if less than 1/2 the sampling interval. To see unnormalized values use -on.
#<------------------Network------------------> # KBIn PktIn Size KBOut PktOut Size Error
KBIn | KB received/sec over all real network interfaces and therefore excludes 'lo' and 'sit'. |
PktIn | Packets received/sec over all real network interfaces. |
Size | Average incoming packet size in bytes. This field is only included if --netopts i or --iosize specified |
KBOut | KB sent/sec over all real network interfaces. |
PktOut | Packets sent/sec over all real network interfaces. |
Size | Average outgoing packet size in bytes. This field is only included if --netopts i or --iosize specified |
Error | Total incoming/outgoing errors/sec. To see individual error counts, use --verbose. This field is only included if --netopts e specified |
#<------NFS Totals------> #<------NFS [s3,c4]-----> # Reads Writes Meta Comm # Reads Writes Meta Comm
Reads | Total nfs reads/sec. |
Writes | Total nfs writes/sec. |
Meta | Total nfs meta data calls/sec, where meta data is considered to be any of: lookup, access, getattr, setattr, readdir and readdirplus, noting that not all types of nfs version report all as V3 clients/servers do. |
Comm | Total nfs commits/sec. |
#<----slab----> # Alloc Bytes
Alloc | Total Number of slabs allocated |
Bytes | Total Number of bytes allocated as slabs |
#<------Sockets-----> # Tcp Udp Raw Frag
Tcp | Total TCP sockets currently in use. |
Udp | Total UDP sockets currently in use. |
Raw | Total RAW sockets currently in use. |
Frag | Total number of IP fragments queues currently in use. |
#<----------TCP----------> # IP Tcp Udp Icmp TcpX 0 0 0 0 0
IPSummary of a number of IP errors from /prod/net/snmp: InHdrErrors+InAddrErrors+InUnknownProtos+InDiscards+OutDiscards+ReasmFails+FragFails |
TcpSummary of a number of Tcp errors from /prod/net/snmp: AttemptFails+InErrs |
UdpSummary of a number of Udp errors, from /prod/net/snmp: NoPorts+InErrors |
IcmpSummary of a number of Icmp errors, from /prod/net/snmp: InErrors+InDestUnreachs+OutErrors |
TcpXSummary of a number of Txp Extended errors, from /prod/net/snmp: TcpLoss+TCPFastRetrans |
updated September 29, 2011 |