Shell Patterns and Regular Expressions in bash
Installfest 2019

Lukáš Bařinka

Table of Contents

  • Shell Patterns
  • Regular Expressions
  • Extended Shell Patterns
  • SP Vs RE
  • Usage

Shell Patterns *

Special Characters

? [] *
  • a?e ~ ace age ale ape are ate awe axe aye
  • [aeiouy]h ~ ah eh oh uh
  • *system ~ ecosystem subsystem system

Set of Characters

abc
List
^abc
Complement
a-c
Range
[:lower:]
Class
[=é=]
Equivalence class
[.ch.]
Collating symbol
             [[:upper:]]????*[^a-i][[=o=]]m

Absalom                Absal   -      o   m
Christendom            Christend      o   m
Chrysostom             Chrysos t      o   m
classroom              classr  o      o   m
Folsom                 Folso   -      -   m
Panmunjom              Padnmun j      o   m
Ångström               Ångst   r      ö   m

Regular Expressions .*

ERE Special Characters

. [] * + ? {,} (|) ^ $
  • hoo?l.rs ~ preschoolers scholars scholarship
  • ^[aeiouy]{3,}$ ~ aye eye yea you
  • .*system ~ ecosystem subsystem system
  • .+system ~ ecosystem subsystem
  • ^(eco|bio).*ing ~ biopsying economizing

Extended Shell Patterns*()

layout_[0-9]*.pdf

  • layout_3.pdf
  • layout_31.pdf
  • layout_32-test.pdf
  • layout_33.pdf
  • layout_3.pdf
  • layout_31.pdf
  • layout_32-test.pdf
  • layout_33.pdf
  • layout_33-final.pdf
  • layout_33-final-final.pdf
  • layout_33-last.pdf

Extended Pattern Matching Operators

?() *() +() @() !()

shopt -s extglob

w=[aeiouy]   p?($w)  p@($w)  p*($w)  p+($w)  p!($w)

             p               p               p
                                             pH
             pa      pa      pa      pa
                                             pace
                             pay     pay
                             payee   payee
                             pea     pea
                             pee     pee
             pi      pi      pi      pi
                             pie     pie
                             poi     poi
                                              ...
                                              pyxes
  • layout_[0-9]*.pdf
  • layout_+([0-9]).pdf
  • layout_+([0-9])?(-final).pdf
  • layout_+([0-9])*(-final).pdf
  • layout_+([0-9])@(*(-final)|?(-last)).pdf
  • layout_+([0-9])*(-final|-last).pdf

Shell Patterns *
Vs

Regular Expressions .*

Shell PatternsRegular Expressions
?.
[ ][ ]
*.*
*( )*
?( )?
+( )+
@( | )( | )
 layout_+([0-9])@(*(-final)|?(-last)).pdf

^layout_[0-9]+((-final)*|(-last)?)\.pdf$

Usage

Pathname Expansion (globbing)

  • shopt -s globasciiranges
    LC_COLLATE, LC_CTYPE, ... LC_ALL

    rm [a-z]*.pdf
  • shopt -s nullglob
    for i in *; mkdir -p "$i"/foo; done
  • shopt -s failglob
    touch *
  • shopt -s dotglob
    rm *
  • shopt -s nocaseglob
    rm *.jpg
  • shopt -s globstar
    rm **/*.jpg
  • set -f
    grep [a-z][0-9]* f1
  • GLOBIGNORE=*.bak:*~

Extended Test

  • [ "$file" = \*.jpg ]
  • [[ $file = *.jpg ]]
  • [[ $file = '*.jpg' ]]
  • [[ $file = a*a ]]
  • [[ $file = a* && $file = *a ]]
  • [[ $file = layout_+([0-9]).pdf ]]
  • [[ $file =~ ^layout_[0-9]+\.pdf$ ]]
  • [[ $file =~ ^layout_([0-9]+)\.pdf$ ]]
    num=${BASH_REMATCH[1]}
  • RE='^layout_([0-9]+)\.pdf$'
    [[ $file =~ $RE ]]
for f in /var/log/**/*.log
do
   [[ $f =~ /var/log/(.*)/(.*)/(.*)/(.*).log ]] \
	|| continue
	service=${BASH_REMATCH[1]}
	Y=${BASH_REMATCH[2]}
	M=${BASH_REMATCH[3]}
	D=${BASH_REMATCH[4]}
	ln -s "$f" "/var/log/$Y$M$D-$service.log"
done
						

Case

  • case $var in
       SP1|SP2) only_this;;
       SP3) test_more;;&
       SP4) do_next;&
       *) default;;
    esac
  • shopt -s nocasematch

Variable Substitution

F=/var/log/access-main_12.log.gz

${F#*/}/var/log/access-main_12.log.gz
${F##*/}/var/log/access-main_12.log.gz
${F%.*}/var/log/access-main_12.log.gz
${F%%.*}/var/log/access-main_12.log.gz

F=/var/log/access-main_12.log.gz

${F/-*_/_}
/var/log/access-main_12.log.gz
/var/log/access_12.log.gz
${F//[-_]/:}
/var/log/access-main_12.log.gz
/var/log/access:main:12.log.gz

for i in !(`date +%Y-%m`*)/
do
   i=${i%/}
   p=${PWD//\//_}
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in !(`date +%Y-%m`*)/
do
   i=${i%/}
   p=${PWD//\//_}
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in !(2019-03*)/
do
   i=${i%/}
   p=${PWD//\//_}
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in !(2019-03*)/
do
   i=${i%/}
   p=${PWD//\//_}
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=${i%/}
   p=${PWD//\//_}
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=${i%/}
   p=${PWD//\//_}
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=${2018-12/%/}
   p=${PWD//\//_}
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=2018-12
   p=${PWD//\//_}
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=2018-12
   p=${PWD//\//_}
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=2018-12
   p=${/var/data//\//_}
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=2018-12
   p=_var_data
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=2018-12
   p=_var_data
   p=${p#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=2018-12
   p=_var_data
   p=${_var_data#_}
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=2018-12
   p=_var_data
   p=var_data
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=2018-12
   p=_var_data
   p=var_data
   tar cvzf "${p}_$i.tgz" "$i" \
   && rm -rf "$i"
done
						

for i in 2018-12/ 2019-01/ 2019-02/
do
   i=2018-12
   p=_var_data
   p=var_data
   tar cvzf "var_data_2018-12.tgz" "2018-12" \
   && rm -rf "2018-12"
done
						

History Control

HISTIGNORE='l[sal]:pwd: *:*\&:&'

Readline

bind -p | grep glob
"\eg": glob-complete-word
"\C-x*": glob-expand-word
"\C-xg": glob-list-expansions

cd /bin
echo b*h█ [ESC-G]
echo bash █
echo *sh█ [CTRL-X][*]
echo bash dash rbash rzsh sh zsh █
echo *sh█ [CTRL-X][G]
echo *sh█
bash dash rbash rzsh sh zsh

Find

-name SP-iname SP
-path SP-ipath SP
-regex RE-iregex RE
-regex-type TYPE‘findutils-default’, ...
‘grep’, ‘egrep’
  • find -name *.pdf
  • find -name '*.pdf'
  • find -regex '.*\.pdf$'
?

Shell Patterns and
Regular Expressions
in bash
Installfest 2019

Lukáš Bařinka