Workshop Syllabus

The Unix Shell

  • Files and directories: pwd, cd, ls, mkdir, ...
  • History and tab completion
  • Pipes and redirection
  • Looping over files
  • Creating and running shell scripts
  • Finding things: grep, find, ...
  • Reference...

Version Control with Git

  • Creating a repository
  • Recording changes to files: add, commit, ...
  • Viewing changes: status, diff, ...
  • Ignoring files
  • Working on the web: clone, pull, push, ...
  • Resolving conflicts
  • Open licenses
  • Where to host work, and why
  • Reference...

Programming in Python

  • Using libraries
  • Working with arrays
  • Reading and plotting data
  • Creating and using functions
  • Loops and conditionals: for, if, else, ...
  • Defensive programming
  • Using Python from the command line
  • Reference...

High Throughput Computing

  • Log in to a remote machine - secured shell ssh
  • Data Transfer scp rsync , globus
  • SSH keys: Access without password
  • Example: Running Python and unix scripts on a remote server
  • Properties of Distributed High-Throughput Computing
  • Scheduling Computing Jobs via HTCondor
  • Properties of Distributed High-Throughput Computing
  • Using software modules and tutorial
  • Examples of running R and Python scripts
  • HTCondor job scripts - condor_submit, condor_q , condor_rm
  • Scaling up the resources: Example Unix and Python scripts
  • stash - the data storage solution
  • Scientfic workflows - DAGMan , Pegasus