nmon for AIX & Linux Performance Monitoring - nmon for AIX5/6 = 12e available now

Briefly - What is nmon?

nmon is short for Nigel's Monitor.
This free tool gives you a huge amount of important performance information in one go.
It can output the data in a number of ways
On screen (console, telnet, VNC, putty or X Windows) using curses for low CPU impact which is updated once every two seconds. You hit single characters on you keyboard to enable/disable the various sorts of data.
Save the data to a comma separated file for analysis and longer term data capture.
- Use this together with nmon Analyser Excel 2000 spreadsheet, which loads the nmon output file and automatically creates dozens of graphs ready for you to study or write performance reports.
- Filter this data, add it to a rrd database (using an excellent freely available utility called rrdtool). This graphs the data to .gif or .png files plus generates the webpage .html file and you can then put the graphs directly on a website automatically on AIX with no need of a Windows based machine.
- Directly put the data into a rrd database for your own analysis
nmon is a single binary for each operating system, so installing is as very easy as getting the file in your $PATH.
Why use five or six tools when one free tool can give you everything you need!!

Warnings, Notices and Licence

No warrantee is given or implied.
Feel free to use these nmon files any way you like but it is at your own risk.
Apart from writing to the output file, nmon does not update or change anything on your machine.

What sort of Performance Data?

CPU utilisation
Memory use
Kernel statistics and run queue
Disks I/O rates, transfers, and read/write ratios
Filesystems size and free spac
Disk adapters
Network I/O rates, transfers, and read/write ratios
Paging space and paging rates
Machine details, CPU and OS specification
Top processors
User defined disk groups
Asynchronous I/O - AIX only
Workload Manager - AIX only
ESS and other disk subsystem - AIX only
Dynamic LPAR changes - AIX and Linux (on POWER hardware)

Sample Screen shots and Output

Take this link nmon Sample Output

Updates

The nmon tool is updated roughly every six months, or when new operating system releases are available.
To place your name on the e-mail list to be informed of updates, contact Nigel Griffiths.

nmon Documentation and Information

Document	URL
nmon Questions and Answers are found in a Community Forum - the nmon developer responds here	Performance Tools Forum
This is part of the AIX and Linux on POWER Community	IBM User Community
nmon Excel Analyser Spread-sheet Download	nmon Analyser Download
The nmon Frequently Asked Questions (FAQ)	nmon FAQ
The nmon Online Manual The definitive document for nmon info	nmon Manual
Release Information for nmon for AIX version 11	README nmon for AIX version 11e
Release Information for nmon for Linux version 11	README nmon for Linux version 11d
Example C source code of how nmon gets its data from AIX Roll Your Own Performance tools	Sample code and AIX 5.3 binaries
nmon Online Workshop = Free training for nmon and the nmon background information	nmon Introduction Workshop ( or see the bottom of this page first)
Introduction Article about nmon - on Developer Works	nmon at IBM DeveloperWorks
nmon Excel Analyser Article	nmon Analyser at IBM DeveloperWorks

Support

Updating to the latest version of nmon fixes ~70% of reported nmon problems!
nmon Support - nmon is the personal project of the developer (Nigel Griffiths), who supports it directly along with the 1000's nmon users via the Manual, FAQ and Forum - see the below "Documentation and Information" section below for the links.
nmon is not for problem determination, it assumes your system is workin correctly and does not include diagnostics information. So do not raise an IBM AIX Support call with nmon data. AIX Support covers tools from AIX Development - this does not include nmon or nmon data.
"AIX Support for a performance issue, what should you do? - If nmon (or any other tool) suggests you have a performance issue and you need AIX Support, do the following:
1. Have your Support Customer Number ready.
2. Collect the configuration data with the snap command, run: snap -a and find the output in /tmp/ibmsupt.
3. Collect the important performance data with the perfPMR tools. Download the latest perfPMR version from the IBM Support Website - this includes an excellent README file. Briefly, unpack perfPMR and then during the problem run: ./perfpmr.sh 600
4. Get ready to FTP this data to the AIX Support FTP site.
5. Prepare a clear description of the symptoms, all changes you have made to the system and the impact this is having.
6. Now call AIX Support and let them know you have snap and perfPMR data ready to send them.
7. Advanced users collect snap and perfPMR regularly (say monthly and before system changes or upgrades), so before & after comparisions can be made.

Sample Online Output

nmon Download

AIX
OS	Hardware	OS Version	Current Release	nmon12 release candidate 12e
AIX	POWER	nmon for 5.1, AIX 5.2, AIX 5.3, AIX 6	nmon4aix_11e.tar.gz Updated July 4th, 2006 Tries to adjust stats like AIX cmds!	WLM fixed, better SPLPAR, MEMPAGE fixed, WPAR fixed, Multiple CPU pools added, Network online more digits + network & disk "=" KB<-->MB - fcstat and real_process/real_user bug fixes and add LPAR/WPAR identification hints. Download nmon12e: nmon4aix12e.tar.gz Now includes AIX 5.1, 5.2, 5.3 & 6.1 Click here for Release Test Instructions - good summary. Click here for nmon12 Line Items - all the details. Includes stats for: WPAR, NFSv4, Donating, Folding, Disk Service Times, Multiple Page Sizes, Mobility support, VIOS SEA, Fibre Channel and more. Updated March 30th 2008
	POWER	nmon for AIX 4.1.5, 4.2.0 and 4.3.3	nmon9f_for_aix4.tar.Z	No more updates for AIX 4
AIX	POWER	nmon for AIX 4.3.2 only	nmon9f_aix432	No more updates for AIX 4
Linux
OS	Hardware	OS Version	Older Release	Current Version
Linux	POWER	SUSE SLES 9, Red Hat EL 3 and 4, and Debian. For Fedora try the RHEL4 version Name = nmon_power_<distro>	nmon4linux_power_11d.zip Virtual Memory, WaitIO, more disks For Fedora/Debian - try RHEL4 Updated July 4th, 2006 See the above table for the README	nmon 11f nmon_power_sles9.zip nmon_power_sles10.zip nmon_power_rhel4.zip Updated June 25th, 2007 Fixes:nmon4linux_v11f.txt
Linux	x86	SUSE SLES 8 and SLES 9, Fedora 5 and RedHat 9, RHEL 2.1, RHEL3 and RHEL4, Knoppix 4 For Debian try the Knoppix 4 name = nmon_x86_<distro>	nmon4linux_x86_11d.zip Virtual Memory, WaitIO, more disks I suspect we really only need 3 versions but I have no time for testing e.g.: RHEL2.1, 2.4 and 2.6 Linux kernel Updated July 5th, 2006 See above table for the README	nmon_x86_11f.zip RHEL2, RHEL3, RHEL4, SLES9, SLES10, Knoppix5, Debian31, Fedora5 I suspect we only need four versions 1) RHEL2 work around duff kernel 2) 2.4 kernel Debain 31/RHEL3 3) 2.6 kernel and 4) 2.6 kernel with improved /proc/stat Updated June 26th, 2007 Fixes:nmon4linux_v11f.txt Beta for RHEL4 nmon_with_external_monitors.zip
Linux	x86_64	Experimental 64 bit version Name = nmon_x86_64_<distro>	nmon4linux_x86_64_b.zip Current Version plus a few fixes Should run on 64 bit 2.6 kernel Linux Complied on Fedora Core 6 and Red Hat 4 update 4 Testing and feedback welcome Updated Jan 15th, 2007	nmon 11f nmon_x86_64_rhel4.zip should be OK for SLES too Updated June 25th, 2007 Fixes:nmon4linux_v11f.txt
Linux	Mainframe	SUSE SLES 8 and 9 name = nmon_mainframe_<distro>	in preparation	nmon 11f nmon_mainframe_rhel4.zip Reported OK for SLES too 32 and 64bit version Updated July 19th, 2007 Fixes:nmon4linux_v11f.txt

Other nmon tools

nmon on Linux Startup Script

To select the right nmon version Linux startup

nmonmerge

To join two nmon output files into one, you need the nmonmerge filter.

Download	Comments
nmonmerge.tar.gz	Binaries nmon merge for AIX and the source code to allow compiling on other platforms and bug fixing.

nmon2rrd

To convert nmon output files to rrdtool format, you need the nmon2rrd filter.

Download	Date	Comments
nmon2rrdv12f.tar.gz	March 6th 2007	Fixes an off by one error in the rrd database start time.
nmon2rrdv12e.tar.gz	March 3rd 2007	Small bug fixes for "%%" in printf's and more accurate time for the rrd databases - Thanks to Ross Druker for these.
nmon2rrdv12d.tar.gz	Jan 4th 2007	Small bug fixes for LPAR, JFS, vpaths. Now includes new graphs for optional data: NFS, WLM, Large Pages, Paging Space Still to do: VG stats - does anybody really use/capture these? Next phase -> add graphs for stats captures but not graphed yet - what do you want? If you have files that fail with nmon2rrd reduce them to 40 captures & attach them to the Performance Tools Forum
nmon2rrdv11.tar.gz	Dec 27th 2006	Improved version 11 and passes my current testset files.
nmon2rrdv10.5.tar.gz	Feb 2006	Fixes thanks to Mr Terry Murray
nmon2rrdv10.6.tar.gz	March 2006	remove IOADAPT graphs if they are not found/supported like on AIX 5.1 and nmon version 11 memuse extra maxclient and numclient percentages
test.nmon.gz	Old	Test file for nmon2rrd. If you get no graphs or problems use this to test the scripts, nmon2rrd, rrdtool etc. Most failures are 1) not capturing enough data snapshots and 2) not checking the log files for errors
nmon2rrdv10.0.tar.gz	Old	Older version that may help people collecting data from AIX 4

Example nmon2rrd usage

Mr Bruce Spencer On the Performance Tools Forum at this link created a couple of scripts that illustrate how the nmon2rrd tool could be automated to convert nmon data files into web pages. All you need to do is to get the nmon file(s) into a directory on a web server, and the script automates everything from there. Thanks Bruce

Part 1 - shows how to automate the collection, conversion, organization and display of daily nmon data from multiple servers into web pages using the nmon2rrd tool.
Part 2 - extends Part 1 to include long term trend charts.
Part 3 - create webpages using nmon2web

External data gatherer for collecting Shared Ethernet Adapter (SEA) Stats.

This is an external data gatherer shell script to collect the Shared Ethernet Adapter states that are normally missing from nmon or any other tool. The stats can only be found via entstat AIX command

Setup is:

export TIMESTAMP=0
export NMON_SNAP=sea.ksh
and have this script in the path - sea.ksh
edit the sea.ksh file to collect data about your adapter - the script by default captures data about en3. Use the VIO Server command: lsdev -virtual to find your SEA name.
newer script that allows multiple SEA adapters to be monitored - you need to have multiple cpoies of the file called from a master one and each have a different SEA number at the top of the script - sea2.ksh

Merge the resulting sea.log with the nmon output file with: cat nmonfile.nmon sea.log >merged.nmon
Then process with the nmon analyser as normal - you will find a SEA section in the spreadsheet.

In testing read and write is nearly identical because I FTPed through the SEA and it reads and writes the data from virtual to physical. It peaked at 12 MB/second = 100 Mbit Ethernet. nmon was doing snapshots once a second to make life easier. One issue has not been solved is the stats are per period rather than per second but it is a good start - this is because the script does not know the number of seconds between invocations.

Shared Ethernet Adapter (SEA) Stats monitoring on the command line

Here is a script (based on sea.ksh) that just outputs to the screen the details once a second, just as the sea.ksh script above will record to a file.
If en8 is yout SEA the just type: seastat en8
Here is the script seastat

Script to find the peak values in many nmon files

Picture the scene, you have hundreds of captured nmon files and some where in them all there is the peak that you want to investigate further - but which file is it in?. Use this script to find the maximum of the statistic you are interested in so you don't have to analyse every single file looking for the peaks. Kenneth Lobb at IBM had this problem, wrote this script to make life simpler and then agreed to share it.
Here is the Perl script find_max_nmon_val.pl

nmon Introductory Workshop - Free and Online

Get the details about nmon from the developer himself. This is the details behind nmon, why it was developered, the principles, some of the ways to use it and the frequently asked questions. this is 64 slides and an Audio commentary by the developer for each slide.
This presentation has been presented at conferences which are internal and external - roughly 90 minutes.

Take this Link - nmon Introduction Workshop

The postings on this site solely reflect the personal views of the authors and do not necessarily represent the views, positions, strategies or opinions of IBM or IBM management.
Back to AIX Home via space
Or goto Linux on Power Home via space