Shell tips

Короткие заметки по shell-программированию

Передать переменные в SED и AWK

 $ echo "unix scripting"
unix scripting

In SED:

This is a general substitution. I am trying to replace "unix" with "BASH", so "unix scripting" will become "BASH scripting"

$ echo "unix scripting" | sed 's/unix/BASH/'
BASH scripting

Suppose, the text "BASH" is assigned to a variable called "var", now if I try to replace "unix" with "$var" in sed single quote notation, its not going to work as SED can't expand external variable in single quotes.

$ var="BASH"; echo "unix scripting" | sed 's/unix/$var/'
$var scripting

Try the same above with double quotes, this will work.

$ var="BASH"; echo "unix scripting" | sed "s/unix/$var/"
BASH scripting

In AWK

General substitution of "unix" with "BASH", will work. "unix scripting" will become "BASH scripting"

$ echo "unix scripting" | awk '{gsub(/unix/,"BASH")}; 1'
BASH scripting

"BASH" is assigned in variable "var". So the following substitution is not going to work.

$ var="BASH"; echo "unix scripting" | awk '{gsub(/unix/,"$var")}; 1'
$var scripting

Method1: See the "'" (double quote-single quote-double quote) before and after the variable var.

$ var="BASH"; echo "unix scripting" | awk '{gsub(/unix/,"'"$var"'")}; 1'
BASH scripting

Method2: Use awk -v flag this way. На солярисе работает только этот способ, как и на HP-UX. Для соляриса надо приментья нормальный awk - /usr/xpg4/bin/awk

$ var="BASH"; echo "unix scripting" | awk -v v="$var" '{sub(/unix/,v)}1'
BASH scripting

Округление

04:45:14-root@wiki:~$ printf "%0.f\n" 4.51
5

while БЕЗ subshell

Способ Ворона

exec 3<$TMP_FILE1                    
# 0 -stdin ... 2 stderr, 3 - first free descriptor
while  read -u 3  NUMBER TYPE
do
    echo NUMBER=$NUMBER TYPE=$TYPE      
done

Считываени нескольких переменных

Пример 1:

cat /tmp/service-perfdata | grep ttiweb  | grep APACHE_HTTPS | grep -v "CRITICAL" | awk '{ print $2" " $16}' | tail -5000 |  while  read DATE  TIMEOUT
do
	DATE1=`date -d @$DATE`
	TIMEOUT1=`echo $TIMEOUT |awk -F"[" '{ print $2 }'`
	echo $DATE1"|"$TIMEOUT1
done

Но для чтения инужно явно делать саб-шелл Пример 2:

for  FILE in  `ls -1 *.log`
do
        echo  $FILE | awk -F"-" '{ print $1" "$2" "$3" "$5 }' | ( read YYYY MM DD TIME
        # Subshell fore read!
                # echo $YYYY $MM $DD
                TIME=`echo $TIME | awk -F"__" '{print $1}'`
                hh=`echo $TIME | cut -b 1-2`
                mm=`echo $TIME | cut -b 3-4`
        D=`date +%s --date="${YYYY}/$MM/$DD $hh:$mm"`
        TMPFILE=`mktemp`
        cat $FILE | grep -v "USER" | head -10 > $TMPFILE
       )
done

if-els

коротко можно желать например, так

function export_data
{
	( 
	$JAVA_HOME/bin/java -Xbootclasspath/a:$CLASSPATH  -jar $BIARENGINE $CONFIG_FILE && ( 
	if test -f $BOXI_EXPORT_FILE
	then
		echo "${DATE_FOR_LOG_FILES}:BOXI Export=SUCCEEDED"  | tee -a $SYNC_STATUS_FILE
		clear_tmp
		exit 0
	else
		echo "${DATE_FOR_LOG_FILES}:BOXI Export=FAILED"  | tee -a $SYNC_STATUS_FILE
	fi  ) 
	) || ( echo "${DATE_FOR_LOG_FILES}:BOXI Export=FAILED"  | tee -a $SYNC_STATUS_FILE )

}

find

For example:

find . -mtime 0   # find files modified between now and 1 day ago
                  # (i.e., within the past 24 hours)
find . -mtime -1  # find files modified less than 1 day ago
                  # (i.e., within the past 24 hours, as before)
find . -mtime 1   # find files modified between 24 and 48 hours ago
find . -mtime +1  # find files modified more than 48 hours ago

find . -mmin +5 -mmin -10 # find files modified between
                          # 6 and 9 minutes ago

Using the "-printf" action instead of the default "-print" is useful to control the output format better than you can with ls or dir. You can use find with -printf to produce output that can easily be parsed by other utilities or imported into spreadsheets or databases. See the man page for the dozens of possibilities with the -printf action. (In fact find with -printf is more versatile than ls and is the preferred tool for forensic examiners even on Windows systems, to list file information.) For example the following displays non-hidden (no leading dot) files in the current directory only (no subdirectories), with an custom output format:

find . -maxdepth 1 -name '[!.]*' -printf 'Name: %16f Size: %6s\n'

"-maxdepth" is a Gnu extension. On a modern, POSIX version of find you could use this:

find . -path './*' -prune ...

On any version of find you can use this more complex (but portable) code:

find . ! -name . -prune ...

which says to "prune" (don't descend into) any directories except ".".

Note that "-maxdepth 1" will include "." unless you also specify "-mindepth 1". A portable way to include "." is:

 find . \( -name . -o -prune \) ...

[This information posted by Stephane Chazelas, on 3/10/09 in newsgroup comp.unix.shell.]

As a system administrator you can use find to locate suspicious files (e.g., world writable files, files with no valid owner and/or group, SetUID files, files with unusual permissions, sizes, names, or dates). Here's a final more complex example (which I saved as a shell script):

find / -noleaf -wholename '/proc' -prune \
     -o -wholename '/sys' -prune \
     -o -wholename '/dev' -prune \
     -o -wholename '/windows-C-Drive' -prune \
     -o -perm -2 ! -type l  ! -type s \
     ! \( -type d -perm -1000 \) -print

This says to seach the whole system, skipping the directories /proc, /sys, /dev, and /windows-C-Drive (presumably a Windows partition on a dual-booted computer). The Gnu -noleaf option tells find not to assume all remaining mounted filesystems are Unix file systems (you might have a mounted CD for instance). The "-o" is the Boolean OR operator, and "!" is the Boolean NOT operator (applies to the following criteria).

So these criteria say to locate files that are world writable ("-perm -2", same as "-o=w") and NOT symlinks ("! -type l") and NOT sockets ("! -type s") and NOT directories with the sticky (or text) bit set ("! \( -type d -perm -1000 \)"). (Symlinks, sockets and directories with the sticky bit set are often world-writable and generally not suspicious.)

A common request is a way to find all the hard links to some file. Using "ls -li file" will tell you how many hard links the file has, and the inode number. You can locate all pathnames to this file with:

  find mount-point -xdev -inum inode-number

Since hard links are restricted to a single filesystem, you need to search that whole filesystem so you start the search at the filesystem's mount point. (This is likely to be either "/home" or "/" for files in your home directory.) The "-xdev" options tells find to not search any other filesystems.

(While most Unix and all Linux systems have a find command that supports the "-inum" criterion, this isn't POSIX standard. Older Unix systems provided the "ncheck" utility instead that could be used for this.)

Solaris-specific

У find на солярисе нет ключа -mmin иногда можно обойтись ключем

find $LOG_DIR/j2ee/FAM/ -newer /tmp/1/logs-fam-indusora-Fri-Aug-20.tar.bz2