Linux Primer

Assumptions

You are familiar with either:

When something breaks, you:

What do you know about Linux?

by a show of hands:

Filesystems

Filesystem 101

  1. Everything starts at the root
    • the root is a directory
    • /” denotes the root directory
  2. the filesystem has different kinds of objects
    1. files
    2. directories
      • containers for multiple objects
    3. links to objects, which either
      • add a second name for the same object
      • point to a position in the filesystem
  3. objects can be referenced by their path
    • absolute: /dir1/dir2/object
    • relative: dir2/object
  4. special objects in directories:
    • .” is a reference to the directory itself
    • ..” is a reference to the parent directory
  5. the system may consist of multiple filesystems
    • filesystems may be mounted at any (empty) directory

Prompt

This is how the prompt looks by default:

[myname@l4_ ~]$

Ways to get help when you’re stuck:

  • man
    • manual for specified topic
    • find man-pages: apropos
  • whatis
  • info
  • most commands support a -h/--help flag
  • colleagues are often helpful
  • the Internet is often helpful

Execution

To execute a command, we call it:

command
  • Every command that is executed will provide a return-value on exit.
    • A value of 0 means success
    • any other value means failure

History

Your shell keeps a log of all the commands you executed.

Parameters

The default way to apply parameters to a program is to write a space separated list of parameters after the program when calling it.

  1. Single-character (preceded by a single dash)
COMMAND -a -b -c -d -e
COMMAND -abcde
COMMAND -j 2 -a -b
COMMAND -j 2 -ab
  1. Multi-character (preceded by a double dash)
COMMAND --long-parameter
COMMAND --flag-parameter=<FLAGVALUE>
  1. String (e.g.: filenames, URLs)
COMMAND <SRC> <DEST>

Variables

A variable is a temporary store for a piece of information. There are two actions we may perform for variables:
  • Setting a value for a variable.
  • Reading the value for a variable.

But beware, bash variables are untyped!

Setting a variable:

var=value

Accessing a variable:

command $var

Printing & Escapes/Quotes

printing text is done by the echo command
echo "some text"
echo $VARIABLE
  1. Backslash escape:
    • Escapes a character, that would have a special meaning
    • Can be used inside of quotes
  2. Double Quotes:
    • Similar to escaping all whitespace characters
  3. Single Quotes:
    • Additionally prevents expansion of variables
COMMAND The\ value\ of\ \$HOME\ is\ $HOME
COMMAND "The value of \$HOME is $HOME"
COMMAND 'The value of \$HOME is $HOME'

File based utilities

Looking around

looking around is done by the ls (list) command

$ ls      # shows files and directories

$ ls -a   # includes hidden ones

$ ls -l   # detailed view
testdir  test

.  ..  testdir  test

drwxr-xr-x 1 myuser p12345 0 Apr 13 11:55 testdir
-rw-r--r-- 1 myuser p12345 4 Apr 13 11:55 test

command result
ls / bin dev etc home sbin usr
ls /home user1 user2

Patterns

Patterns and expansions define multiple arguments with little overhead:

ls file.???
ls *.*

Moving around

moving around is done by the cd (change directory) command

finding the current position in the filesystem is done by the pwd (print working directory) command

$ cd /bin   # go to an absolute directory

$ cd [~]    # go home

$ cd -      # go to previous directory
$ pwd   # /bin

$ pwd   # /home/user1

$ pwd   # /bin

Copying & moving files around

moving files around is done by the mv (move) command

copying is done by the cp (copy) command

$ mv old   new     # rename old to new

$ mv old   dir/    # move old into dir

$ mv file1 file2   # overwrite file2 with file1
                   # (BEWARE)
$ cp -i input input.bak   # input to input.bak

$ cp -i input backup/     # input into backup

$ cp -a dir1/ dir2        # exact copy of dir1

Copy files from remote machine

scp <source> <destination>
scp <me>@vsc4.vsc.ac.at:~/<myfile.c> .
scp myfile.c <me>@vsc4.vsc.ac.at:~/
rsync [OPTIONs] <source> <destination>
maybe useful: -av, --exclude=/daten/temp/...

Creating and deleting

directories

creating directories is done by the mkdir (make directory) command:

deleting directories is done by the rmdir (remove directory) command:

$ mkdir newdir       # create directory newdir

$ mkdir -p new/dir   # create dir inside of new
$ rmdir emptydir      # removes emptydir

$ rmdir -v emptydir   # removes emptydir (verbose)

files

creating files is done by the touch or echo commands:

deleting files is done by the rm (remove) command:

$ touch file          # create empty file

$ echo "abc" > file   # create/overwrite file
$ rm file      # removes file

$ rm -ri dir   # removes dir recursively

Streams

Redirects

Write output to a file or file-descriptor

Command Redirect Append Description
program > std.log >> std.log redirect stdout to a file
program 2> err.log 2>> err.log redirect stderr to a file
program 2>&1 redirect stderr to stdout

Pipes

Write output into the input-stream of another process

Command Pipe Description
program | grep -i foo pipe stdout into grep
program | tee file1 file2 overwrite files and stdout
program | tee -a file append to files and stdout

Content based utilities

Contents of files

viewing is done by the less command:

concatenating is done by the cat command:

$ less file.txt      # exit with 'q'

$ less -R file.txt   # keep colors

$ cat file1 file2 | less
$ cat file

$ cat -A printable

$ cat -n numbered

searchig for patterns is done by the grep command:

finding files is done by the find command:

$ grep strin file

$ cat file | grep awesome   
$ find . -name file

$ find / -type d -name directory

Space accounting

viewing used space is done by the du (disk usage) command:

viewing free space is done by the df (disk free) command:

$ du -h file1 file2   # human readable output

$ du -s dir           # summarize
$ df -h       # human readable output

$ df -t nfs   # only list filesystems of a type

Sed and awk

sed (stream editor) and awk are powerful tools when working with the command line

$ mycommand | sed "..."
$ mycommand | awk '{...}'

Using sed and awk in action

program command description
sed s/old/new/ replace old with new
sed /from/,/to/ s/old/new/ replace old with new, between from and to
awk 'print $5 $3' print columns 5 and 3 of every line

Example script:

#!/bin/bash

mv space.log space.log.bak
df -h | grep "lv12345" > space.log
cat space.log
#!/bin/bash

mv space.log space.log.bak
df -h | grep "lv12345" > space.log
cat space.log | sed "s|/home/lv12345|ProjectA|" \
              | awk '{print $6, "free:", $4}' \
              | column -t

Aliases

Whenever you have a command

  1. run often and it
    • has a long list of parameters you always use
    • is dangerous
  2. which is an aggregate of many other commands
    • but you don’t want to write a script

you can define an alias for it:

alias ll='ls -alh'
alias rm='rm -i'
alias myProject='cd $ProjectDir; testSuite; compile && testSuite; cd -'

Shell scripting

A shell script is a text file that contains a sequence of commands

to execute a shell script:
bash myscript.sh

BEWARE, your shell script will NOT abort if a command fails

Loops

Loops can be used to loop over files, numeric arguments, until a either the loop condition is false or a break is encountered.
for i in {1..5}; do echo $i; done
for i in *; do echo $i; done

This looks a bit messy, better write a script

for i in {1..5}
do
  echo $i
done
while true
do
  echo "Annoying Hello World"
  sleep 3
done

Conditionals

if [ $VARIABLE1 ]
then
  COMMAND1
elif [ $VARIABLE2 ]
  COMMAND2
else
  COMMAND3
fi
case $VARIABLE in
[0-9])
  COMMAND1
  ;;
*)
  COMMAND2
  ;;
esac

Environment Variables

Setting, getting and unsetting

Set

LANG=en_US.UTF-8 bash
export LANG=en_US.UTF-8

Get

env
echo ${LANG}
echo $PWD

Unset

unset LANG
env -u LANG

Use cases

Some variables that could affect you are:

$EDITOR            # the default editor for the CLI
$PAGER             # utility to read long streams
$PATH              # program paths, in priority order

if you’re aiming for programming, these could be more interesting:

$LIBRARY_PATH      # libraries to link by the compiler
$LD_LIBRARY_PATH   # libraries to link at runtime
$CC                # sometimes used to set default C compiler
$CFLAGS            # default flags for compiling C

if you have a lot of self-compiled binaries:

export PATH="./:$HOME/bin/:$PATH"

Ownership and Permissions

Just to ensure that you are able to run your scripts

chown

Change the owner of files and directories by:

# only works with root privileges

chown user file
chown -R user:group dirs files

chmod

Change the mode of files and directories by:

chmod -R u=rwx,g+w,o-rwx dirs files
chmod 640                files
chmod 750                dirs
chmod 750                executables

Shebang

A little test program, which we mark as executable and hand it over to the corresponding interpreter:

cat << EOF > test.sh

echo "${LANG}"
echo "${PATH}"
EOF
chmod +x test.sh
bash test.sh

Don’t we have an OS, capable of executing everything it recognizes as an executable?

Yes, we do!

cat << EOF > test.sh
#!/bin/bash
echo "${LANG}"
echo "${PATH}"
EOF
chmod +x test.sh
./test.sh

.bashrc

.bashrc

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi

# User specific aliases and functions
alias sq='squeue -u $USER'
alias rm='rm -i'

export PATH="./:$HOME/bin:$PATH"

Q&A

Back to Agenda

AGENDA – VSC-Linux

AGENDA – VSC-Intro