Solaris Monitoring

pargs
pflags
pcred
pldd
psig
pstack
pmap
pfiles

pwdx
pstop
prun

pwait
ptree
ptime
preap
pkill

trapstat

mpstat
vmstat
netstat
sar

cputrack cpustat

fsstat

Microstates

• Fine-grained state tracking for processes/threads
• Off by default in Solaris 8 and Solaris 9
• On by default in Solaris 10
• Can be enabled per-process via /proc
• prstat -m reports microstates
• As a percentage of time for the sampling period

1. USR – user mode
2. SYS - kernel mode
3. TRP – trap handling
4. TFL – text page faults
5. DFL – data page faults
6. LCK – user lock wait
7. SLP - sleep
8. LAT – waiting for a processor (sitting on a run queue)

prstat -m

Use prstat -m LAT category, in conjunction with utilization measurements, delivered workload throughput and run queue depth (vmstat “r” column) to determine for CPU capacity planning

prstat – user summary

# prstat -t
NPROC USERNAME SIZE RSS MEMORY TIME CPU
128 root 446M 333M 1.4% 47:14:23 11%
2 measter 6600K 5016K 0.0% 0:00:07 0.2%
1 clamb 9152K 8344K 0.0% 0:02:14 0.1%
2 rmc 7192K 6440K 0.0% 0:00:00 0.1%
1 bricker 5776K 4952K 0.0% 0:00:20 0.1%
2 asd 10M 8696K 0.0% 0:00:01 0.1%
1 fredz 7760K 6944K 0.0% 0:00:05 0.1%
2 jenks 8576K 6904K 0.0% 0:00:01 0.1%
1 muffin 15M 14M 0.1% 0:01:26 0.1%
1 dte 3800K 3016K 0.0% 0:00:04 0.0%
2 adjg 8672K 7040K 0.0% 0:00:03 0.0%
3 msw 14M 10M 0.0% 0:00:00 0.0%
1 welza 4032K 3248K 0.0% 0:00:29 0.0%
2 kimc 7848K 6344K 0.0% 0:00:25 0.0%
4 jcmartin 13M 9904K 0.0% 0:00:03 0.0%
1 rascal 17M 16M 0.1% 0:02:11 0.0%
1 rab 3288K 2632K 0.0% 0:02:11 0.0%
1 gjmurphy 3232K 2392K 0.0% 0:00:00 0.0%
1 ktheisen 15M 14M 0.1% 0:01:16 0.0%
1 nagendra 3232K 2400K 0.0% 0:00:00 0.0%
2 ayong 8320K 6832K 0.0% 0:00:02 0.0%
Total: 711 processes, 902 lwps, load averages: 3.84, 4.30, 4.37

Solaris iostat

● Wait: number of threads queued for I/O ● Actv: number of threads performing I/O ● wsvc_t: Average time spend waiting on queue ● asvc_t: Average time performing I/O ● %w: Only useful if one thread is running on the entire machine – time spent waiting for I/O ● %b: Device utilization – only useful if device can do just 1 I/ O at a time (invalid for arrays etc...)

# iostat -xnz
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
687.8 0.0 38015.3 0.0 0.0 1.9 0.0 2.7 0 100 c0d0
Queue Performing I/O
wait svc

New Formatting flags -C, -l, -m, -r, -s, -z, -T
● -C: report disk statistics by controller
● -l n: Limit the number of disks to n
● -m: Display mount points (most useful with -p)
● -r: Display data n comma separated format
● -s: Suppress state change messages
● -z: Suppress entries with all zero values
● -T d|u Display a timestamp in date (d) or unix time_t (u)