IBM Wikis - AIX 5L Wiki

nmon version 12 release candidate D is ready for Release Testing

Changes for 12d:

Fix MEMPAGE big in the capture file
Added POWER6 Multiple CPU pool stats
- you need Pool Authorisation for some of them
- now called in HMC7 "Allow performance information collection" on the LPAR properties
- note this not in the LPAR profile
Online the Network numbers have more digits for ultra fast networks like Infiniband/10Gb Ethernet
- plus "=" toggles between KB/s and MB/s for network and disks
Fixed WLM online stats
Fixed WPAR stats to file if WPAR inactive

Changes for 12c:

Removed problem raso command
Add AIX 5.1
Move header lines earlier in the output to help some post capture filters programs

Changes for 12b:

WPAR stats improved and handles inactive WPARs now
WLM stats flip to WPAR view, when you use both @ for WPAR stats and W for WLM
WPAR state changes and WPAR create/delete now handled OK
- While collecting to a file there is not much we can do
- You will find a warning in the output file if the number of WPARs changes
More accurate CPU_ALL numbers i.e. adjusted like AIX commands
- Hopefully more consistently too with different online views
Online banner CPU-Utilisation-Long-Term became Physical-CPU-Utilisation-Long-Term
- To highlight on a shared CPU LPAR this is no long Logical CPU average
Additional file output BBBP section stats collected - especially for AIX6 i.e. restricted parameter settings
Typos and wrong units that were reported were fixed
- A big thanks to our eagle eyed beta testers
Work a rounds for various kernel/library bugs
Adapter-I/O became Disk-Adapter-I/O to highlight its just disks
- Note the new Fibre Channel stats can see SAN Tape I/O
Top Process Character I/O now reported in KB (like the others)
- Note when too large a M appears next to the number
- This is then reporting MB (not millions of KBs!)

Now includes AIX 5.2 - note many new features are not available due AIX 5.2 not supporting them but nmon core feature improvements are still worth having.

Note: online nmon will cause issues with Workload Partition Relocation (also called Application Mobility). Development are working on a bug fix. It is related to AIX curses working in nmon type mode but other curses based application have the same issue too.

nmon version 12 release candidate A is ready for Release Testing

This means there are no more functions to be added and internal testing has finished. Next is release testing where every one is invited to give it a try so that the dozens of configurations and environments can be included in the test. If all goes well and no major bugs or issues are found this will become the final version. Otherwise the bugs will be fixed and we move to release candidate B and so on. The beta versions during development have meant a great deal of testing has already been completed.

Most of the new features are for the latest releases of AIX AIX 5.2 ML7 and AIX 6.1 with service pack 2 and some only to POWER6 based machines but some features are applicable to previous versions. So we are concentrating on these releases and then moving back to older ML and AIX releases. As newer AIX releases have new versions of libperfstat and unfortunately these are forward compatible (can't compile nmon for the latest AIX release so it has all the API features and then run it on older AIX releases) it is simplest to have one nmon12 release for each AIX release - some releases with the same libperfstat version can share a nmon release. Later, we may attempted a single nmon12 that for example, runs on all AIX 5.3 releases but that would cause considerable delay and testing complication at this point in time.

Order of nmon 12 Releases for AIX levels

Below is the order of nmon 12 binaries to be made available for testing. We need them all to be tested. This page will be updated to reflect is we have confirmation that a release has either been checked by Basic or Advanced tests (see below). If you complete a successful on a specific AIX level, please send email to nag @ uk . ibm . com (remove the spaces) stating the AIX release, basic/advanced, POWER4/POWE5/POWER6 and any observations. Changes apart from 12a changes to 12b are in bold:

AIX 6.1 with service pack 2 - nmon12b_aix61 ready
AIX 5.3 ML7 - nmon12b_aix537
AIX 5.3 ML6 - nmon12b_aix536
AIX 5.3 ML5 - nmon12b_aix535
AIX 5.3 ML4 - nmon12b_aix534
AIX 5.3 ML3 - use nmon12b_aix530
AIX 5.3 ML2 - use nmon12b_aix530
AIX 5.3 ML1 - use nmon12b_aix530
AIX5.3 GA - nmon12a_aix530
AIX 5.2 ML 5 or above - nmon12b_aix537
AIX 5.2 ML 3 or 4 - no one should be running these
AIX 5.2 ML 1 to 2 - nmon12b_aix522
AIX 5.1 - not available yet.

Testing

We need them all to be tested. This page will be updated to reflect is we have confirmation that a release has either been checked by Basic or Advanced tests (see below).

If you complete a successful on a specific AIX level, please send email to nag @ uk . ibm . com (remove the spaces) stating the AIX release, basic/advanced, POWER4/POWE5/POWER6 and any observations.
If you find a bug/problem please also send email to the above email address and clearly state the issue.
If online please include a "screen scrap" in text form i.e. not large GIF or JPEG images. Please also include a small sample data capture as this includes all the configuration information needed to find a fix.
If it is a file capture problem, please send a copy of the file and some why to locate the problem like a line number, a snapshot number or a string to search for in the file.

Basic function tests

This is the instructions for basic testing of the new versions

Online Basic Tests

Online (i.e. on screen) take a look at the following options and visually check the numbers look about right or compare with other AIX commands:

c for CPU,
If more than one or two CPUs try C for compact CPU layout
l (lowercase L) for longer term CPU graph
m for memory
k for kernel numbers
d for disk graphs, also D more than once for disk numbers
t for Top processes
r for resources
n for network
if in a LPAR, p for partition details
if running WLM, W for WLM
if running NFS, N for NFS (if latest ML the 2nd N shows NFS v4)

File Capture Basic Testing

Capture to a file with with something interesting running as there is no point is capturing a completely idle machine as we just gets lots of zeros.
For Top processes, NFS and WLM try:

nmon -fTNW -s2 -c60

This will capture for just two minutes (2 x 60 seconds) plus configuration capture time.

If the machine or LPAR is idle try generating a little work with the following commands:

yes >/tmp/x
wait 10 seconds and hit control-C
sync
rm /tmp/x
ls -lR / >/dev/null

Please send the resulting .nmon file in the current directory to nag @ uk . ibm . com (with the spaces missing).

Advanced and New Function Testing

	New Feature	Online Activation	File Capture Activation	Comments
1	Disk Service Times	DDD	-d	See DISKSERV & DISKWAIT file sections
2	Multiple Page Sizes (POWER6 only)	M	-M	See MEMPAGEnnnn sections. Note: online first M the numbers are in pages, a 2nd M gives you them in MBs
3	Timestamps with digits - -W 8	-	-w 8	See the T00000001 etc numbers (note lowercase w)
4	Network Packet saved to file	-	default	See NETSIZE section
5	Time Drift	-	-	Save data for 24 hours and compare time shot times in the ZZZZ section
6	Dump all libperfstat available kernel, Hypervisor and LPAR stats	-	-K	See RAWLPAR & RAWCPUTOTAL
7	High Priority nmon	-	-Z -20	Check the ZZZZ section of very busy machines and the nice value for the nmon process
8	Timestamps in CUT/GMT time zone		-G	See ZZZZ sections
9	Shared Ethernet adapter for Virtual I/O Server only	O	-O	See SEA section Note this uses the entstat command behind the covers which takes extra CPU time, recommend not capturing faster than 2 seconds.
	Partitions Mobility (PM) on POWER6 - changes of machine	p	default	See BBBR section
11	Application Mobility (AM) on AIX6 - changes of AIX LPAR/machine	p	default	See BBBR section
12	Donating CPU statistics	p	deault	See DONATE section
13	Multiple Shared CPU pools	p	-	Online see "LPAR ID Group:Pool" field and pool_id in the LPAR section
14	Folded CPU counter	c	default	See LPAR Folded on bottom line on see LPAR section
15	Skip saving of JFS stats as these can be large and lots of mount/umount confused	-	-J
16	Fibre Channel stats	^	-^	This should in theory capture FC disks AND tape drive access. Note this uses the fcstat command behind the covers which takes extra CPU time, recommend not capturing faster than 2 seconds.
17	Workload Partitions - in a WPAR	-	-	Just run nmon. Note: there are no disks, no adapters and no paging spaces
18	Workload Partitions - at global level	@	-@	See WPARCPU and WPARMEM sections
19	File format subtle change			See AAA,time is now in HH:MM:SS format

Of particular interest is the following:

WPAR at the global level
SEA stats for Virtual I/O Servers
Fibre Channel (FC) stats with active tape drives on their own adapter
PM or AM events during a capture
Disk Service Times and compared to AIX iostat command output

Again nmon captures with any of these features - please send to me at nag @ uk . ibm . com (remove spaces) with a note of what you have captured.