VT FDI Summer Track V
Parallel Programming

From 08 June to 10 June 2008, I will be an instructor for a 3 day workshop on parallel computing sponsored by the Faculty Development Initiative. The class will be held in Room 3060, Torgersen Hall, Virgina Tech.

Here is the tentative schedule for the workshop.

Official Description from FDI Website:

Track V - Parallel Programming (Summer Only)

This summer-only track is designed to assist faculty in understanding how to develop parallel computing applications for use on large scale resources like Virginia Tech's supercomputing assets: System X, Inferno, Dante, etc. Parallel programming concepts will be central to this discussion, and participants should have an applications development background. Participants will learn the basics of the Message Passing Interface (MPI) for distributed memory systems as well as code optimization for Symmetric Multiprocessing (SMP) shared memory systems. Emphasis will be placed on numerical research applications in the computational science and engineering disciplines, but other disciplines are always welcome. Academic support staff and graduate teaching assistants may participate with faculty as a 2-person project team.

Lecture Notes:

• fdi_2008_lecture1.pdf "The Prehistory of Parallel Programming";
• fdi_2008_lecture2.pdf "Computer Performance";
• fdi_2008_lecture3.pdf "Parallel Programming Concepts";
• fdi_2008_lecture4.pdf "Some Model Programming Tasks";
• fdi_2008_lecture5.pdf "Introductory OpenMP";
• fdi_2008_lecture6.pdf "Using OpenMP";
• fdi_2008_lecture7.pdf "Introductory MPI";
• fdi_2008_lecture8.pdf "Using MPI";

OpenMP information:

• an introduction to OpenMP
• an OpenMP glossary
• Using OpenMP on the SGI Systems at Virginia Tech
• open_mp_3.0.pdf The OpenMP specification, version 3.0;

MPI information:

• an introduction to MPI
• more information on MPI
• an MPI glossary
• Using MPI on System X at Virginia Tech

Exercises:

There will be three hands-on sessions during the workshop. Here are links to the suggested exercises for these sessions:

Exercise Session #1 - Timing and Performance. Run the "hello" example first, so you are comfortable with how to compile and run programs.

1. heat, a program to solve the time dependent heat equation.
2. hello, says "Hello, world!" and that's all!
3. linpack_bench, the Linpack benchmark;
4. mxm, a matrix-matrix multiplication program;
5. mxv, a matrix-vector multiplication program;
6. prime, sum the prime numbers from 2 to N.
7. sat, satisfiability search for all assignments to a set of variables that make a formula true.

Exercise Session #2 - OpenMP. In most cases, you will be asked to add OpenMP directives to the program, and demonstrate a speedup. Run the "hello2" example first, which should be easy to set up and run.

1. grid2, a grid quadrature program for multidimensional integrals.
2. hello2, says "Hello!" from each OpenMP thread and that's all!
3. mc2, a Monte Carlo quadrature program for multidimensional integrals.
4. mxm2, a matrix-matrix multiplication program;
5. mxv2 a matrix-vector multiplication program.
6. power2 a power method program for eigenvalues.
7. prime2, sum the prime numbers from 2 to N. This exercise suggests 5 steps toward converting the sequential program to an OpenMP program.
8. sat2, redo the satisfiability exercise, letting each processor check a part of the set of inputs.

Exercise Session #3 - MPI. In most cases, you will be asked to modify the given sequential program so that it runs in parallel under MPI. Set up the "hello3" example first and make sure it works.

1. heat3, a program to solve the time dependent heat equation. A pair of MPI_Send and MPI_Recv statements are given, but another pair is missing and you must insert them.
2. hello3, says "Hello!" from each MPI process and that's all!
3. mc3, a Monte Carlo quadrature program for multidimensional integrals. Each process will be generating random numbers. Can you make sure each process generates a different set of random numbers?
4. prime3, sum the prime numbers from 2 to N. If you get that done, can you also "rebalance" the work so that the processes finish at about the same time?
5. sat3, redo the satisfiability exercise, letting each processor check a part of the set of inputs.
6. sort3, an exercise that can be simple or challenging, done on paper or by program.

You can go up one level to the main web page.