Unix-like Operating Systems

From Colettapedia
Jump to navigation Jump to search

General

  • grub boot, change to run level 3
  • md5sum -c *.md5
    • when you have file_whatever.iso and a file_whatever.iso.md5, and you want to check
  • file = determine file type and output that to STDOUT
  • Here Documents - writing files via command line
    • enclose delimiter in single quotes if you want to turn off backtick, $variable and arithmetic expansion
cat > file_you_want_to_save_text_to.txt <<'EOF'
bunch
of
lines
and
metacharacters ! # ? * "
it's all good
EOF
    • If the redirection operator is `<<-', then all leading tab characters are stripped from input lines and the line containing DELIMITER. This allows here-documents within shell scripts to be indented in a natural fashion.
  • gprof
  • tar -h option dereferences symbolic links and you get the pointed-to file not the symlink
  • Get a list of all files in a tarball:
    • tar -tzf backup.tar.gz (GNU tar)
    • lz backup.tar.gz
  • "tar zcvf - ./ | ssh user@host "cat > " = "tar over ssh"
  • puppetd
  • pgrep <string> - return the PIDs of any process whose name greps with "string"
  • test
  • Unix-like System Startup
  • history | grep <search string>; !<history number>
  • sed
  • gawk - GNU awk
  • diff
  • Unix Run Processes at Startup
  • Unix Network Administration
  • umask
  • install
  • gunzip ... unzip ... uncompress
  • screen
  • SSH and SSH-Keygen
  • PS1="\u@\h \w\n$ " ... user@host pwd newline $prompt
  • PS1="\n\[\e[32m\]\u@\h \[\e[33m\]\w\[\e[0m\]\n\$"
  • date
  • cron
  • echo "hello user2 EOF" | write user2 = send a one-line message to user2 logged in on a machine.
  • wall message => send a message to everybody
  • command line math:
$ echo $[ 34 * 11 ]
374
  • patch
  • xargs --verbose -i placeholder string = '{}'
    • the following command will copy the list of files and directories which start with an uppercase letter in the current directory to destdir: /bin/ls -1d [A-Z]* | xargs -J % cp -rp % destdir
    • implemented differently between gnu and bsd
  • find
  • httrack
  • ctags
  • tidy -im file_to_be_cleaned.html
  • mount
  • nfs
  • ps - process server
  • rsync
  • C-z to stop, jobs to list background processes, fg %1, fg %2 depending on process number
  • indent
  • sudo su <user>
  • chmod a+rwX = make the directories executable
  • grep
  • top
  • df = report file system disk space usage... "macroscopic" view of storage devices
  • du -h --max-depth=1 = How big are all the files and directories in this directory... "microscopic" view of file space usage in a specific directory.
  • Passing the STDERR to the bit bucket 2>/dev/null
    • 0, 1, and 2 = STDIN, STDOUT, and STDERR, respectively
    • By default, if you don’t name or number one explicitly, you’re talking about STDOUT
    • > is output redirection to a file
    • >> Appends to output to a file, rather than clear out the file
    • >& Redirect both regular and error output to a file
  • mail
  • proc directory
  • lsof
  • bash shell keyboard shortcuts
  • basename and dirname
  • sort -u
  • tee - Like a T-shaped pipe joint.

Ghetto Parallelism

Using split

  • split --number=l/6 -d --additional-suffix=.txt filelist.txt filelist
function parallelize {
    set -x
    local input_file=$1 # the file.txt containing input lines to be split
    shift
    local n_splits=$1 # the number of chunks/instances of command to run
    shift
    echo "input: $input_file"
    echo "nsplits: $n_splits"
    local base=`basename $input_file .txt`
    echo $base
    local suffix="_${n_splits}_chunk.txt"
    split --numeric-suffixes --number=l/$n_splits --additional-suffix=$suffix $input_file $base
    # no mid-line splits
    set +x

}

Using xargs

  • find . -name "*.png" | xargs -l --max-procs=30 ./convert_to_tiff.sh
    • that's the letter l not 1
# convert_to_tiff.sh
ORIGNAME=$1
DIRNAME=$(dirname $ORIGNAME)
BASENAME=$(basename $ORIGNAME .png)
NEWNAME="../lanczos_tiff/$DIRNAME/$BASENAME.tif"
echo "$ORIGNAME -> $NEWNAME"
convert -verbose $ORIGNAME $NEWNAME

System Monitoring

  • mpstat -P ALL 5 | tee profile/test2 - givers per core usage stats every 5 seconds
    • %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle