Workshop on Parallel Programming with MPI
CSIT, Florida State University
October 2002

Today's toughest problems, such as high speed fluid flow, or the simulated folding of a protein, require the power that parallel computing can provide. Parallel computing has evolved from an obscure bag of tricks into a reliable technology. Now it is possible to write a program in a standard language, and have it execute on 10 or 100 or 10,000 processors. Moreover, it is possible for us to train you how to do this too. The vehicle for this feat is the Message Passing Interface (MPI), which will be the topic of special workshop.

This October, the Ohio Supercomputer Center (OSC) will be broadcasting an MPI workshop over the Access Grid. This workshop will be hosted locally by the School for Computational Science and Information Technology (CSIT) at the Florida State University (FSU). The workshop consists of 4 consecutive sessions held each Thursday afternoon, from October 3 to October 24, 2002.

The workshop is sponsored by the PACS program (Partners for Advanced Computational Services) which includes OSC and FSU. In fact, the same workshop will be accessible from many participating PACS sites around the country, listed in the General Announcement at http://alliance.osc.edu/mpi/.

Description

This workshop presents the MPI library, which is used to design a single program, copies of which run as parallel processes, cooperating to carry out a computation.

MPI allows the programmer to control how the problem data and the individual calculations are assigned to various processes. As the calculations proceed, the processes can communicate with each other through messages, with the overall computation supervised by a master process. The messages are handled by a set of MPI subroutine calls inserted into a standard C/C++ or FORTRAN program.

The workshop explains the ideas behind parallel processing and the philosophy of MPI. The participants are given a detailed look at how the MPI routines are used for processor-to-processor communication, for collective operations, to define high-level processor connection topologies and to create new data types. A complete description of the course coverage is available on the General Announcement.

Where and When and How

The workshop sessions will be held at CSIT, the School for Computational Science and Information Technology, in room 401 of the Dirac Science Library, on the FSU campus, in Tallahassee, Florida. To find the site, you may refer to the FSU Campus Map.

The workshop will be given over four consecutive Thursday afternoons, from October 3, 2002 through October 24, 2002. Each session runs from noon to 3pm, Eastern time.

The workshop will be presented live and interactively via the AccessGrid. The instructor, Dr David Ennis, will be in Columbus, Ohio, but the AccessGrid will make it possible for us to see and hear him, as well as to interrupt with questions.

A note about classes on the AccessGrid: We will try our best to make this a useful workshop for you. However, we want to warn you that this workshop is being offered over the Internet, and to a larger group of sites than usual. Anything that happens between here and Ohio could disrupt the class; we don't expect this to happen, but we want you to be aware of the possibility.

Workshop Schedule

  1. Thursday, October 3, 2002 (noon to 3pm EDT)
    The slides for this lecture are available as a PDF file or as PowerPoint slides.
    • Message Passing Fundamentals
    • MPI Program Structure
    • MPI Messages and Datatypes
    • Point-to-point Communications: Basic send and receive
    • Problem Set.
      You may want to work from a copy of the sample program #1
      in C++, bones.cc, or
      in FORTRAN 77, bones77.f, or
      in FORTRAN 90, bones.f90.
      Send solutions to dje@osc.edu.
    • Worked-out exercises:
      You can take a look at how I did exercise #3
      the Fortran 90 source code, day1_ex3.f90,
      the C++ source code, day1_ex3.cc,
      the SGI command file, day1_ex3.csh, and
      the resulting output file, day1_ex3.out.
  2. Thursday, October 10, 2002 (noon to 3pm EDT)
    The slides for this lecture are available as a PDF file or as PowerPoint slides. There is also a data file you will need for the exercises.)
    • Non-Blocking Communication (IBSEND, IRSEND, ISEND, ISSEND, and IRECV);
    • Derived Datatypes
  3. Thursday, October 17, 2002 (noon to 3pm EDT)
    The slides for this lecture are available as a PDF file. or as PowerPoint slides.
    • Collective Communication
  4. Thursday, October 24, 2002 (noon to 3pm EDT)
    The slides for this lecture are available as a PDF file or as PowerPoint slides.
    • Virtual Topologies

Each session builds on the materials presented in the previous sessions. Participants are encouraged to attend all sessions. Problem sets will be assigned at the end of each session. Participants are encouraged to complete the problems each week and send solutions to the instructor for comments.

*A note about October 3rd: We expect to have some problems with our first session, October 3rd. This is the first day of FSU's fall break, so there will be no classes, the Dirac Science Library building will be locked, and there is a football game scheduled for the evening, so parking may be a real problem! Parking is up to you, but we will try to ensure that someone is downstairs at the main entrance, from 11:30 to noon, to make sure everyone can get in!

Should You Attend?

There is no cost to attend this workshop. In order for you to understand the workshop, you should already be able to program in C, C++, Fortran77 or Fortran90. To get an idea of what MPI is like, it would be a great idea if you could look over the first few chapters of one of the MPI introductions, or go to the MPI web site and look around, or work through some of the material in the online MPI course. Details on these items are given later in this document.

We will not be providing special computer accounts to our attendees. If you want to work on the programming problems given at the end of each session, you'll need to make your own arrangements to get access to a computer system that uses MPI. People who are already CSIT users automatically have such access.

Registration

Our classroom space is limited to about 15 persons and we have reached the limit!. We expect the first class to be completely filled, but there will probably be some spaces in the later classes if you want to try to join in late. If you are interested in attending, we ask that you send a short note to comp-consult@csit.fsu.edu including

  1. Your affiliation (business, university department, other);
  2. Your programming background (languages, machines);
  3. Your area of research or study or interest;
  4. Your reasons for being interested in the course.

Any other questions can be emailed to us at comp-consult@csit.fsu.edu.

Registered Participants

Our current list of registered participants includes:

  1. Vikram Aggarwal, viaggarw@cs.fsu.edu;
  2. Santha Ram Akella, graduate student, FSU mathematics, FORTRAN90 user, santha@csit.fsu.edu, computational fluid dynamics and optimal control;
  3. Michael Allard, graduate student, FSU meteorology, FORTRAN user, michael@bamboo.met.fsu.edu;
  4. Clark Amerault, FSU meteorology, amerault@met.fsu.edu, FORTRAN user, numerical weather prediction;
  5. John Burkardt, CSIT, FSU, burkardt@csit.fsu.edu, centroidal Voronoi algorithms;
  6. Yanzhao Cao, Faculty, FAMU/FSU, FORTRAN user, caoy@csit.fsu.edu, flow control;
  7. Paul Eugenio, Professor, Department of Physics, FSU, eugenio@phy.fsu.edu, nuclear physics;
  8. Xiaoqin Huang, CSIT postdoc, xiaoqin@csit.fsu.edu, computational biophysics;
  9. Kun-Il Jang, graduate student, FSU meteorology, kijang@met.fsu.edu, numerical weather modeling;
  10. Jyoti Kolhe, kolhe@cs.fsu.edu;
  11. Hui Liu, liu@met.fsu.edu;
  12. Charles Mason, mason@csit.fsu.edu;
  13. Lionel Mathelin, CSIT postdoc, mathelin@csit.fsu.edu, FORTRAN 77/90 user, computational fluid dynamics;
  14. Alexander Ostrovidov, Department of Physics, FSU, ostrov@phy.fsu.edu, nuclear physics;
  15. Kyungjeen Park, graduate student, FSU meteorology, kpark@met.fsu.edu;
  16. Shiqin Peng, sqpeng@met.fsu.edu;
  17. Hui Shao, graduate student, FSU meteorology, FORTRAN user, hshao@met.fsu.edu, GPS space-space data assimilation;
  18. Alex Velytsky, CSIT and Physics FSU, velytski@csit.fsu.edu, C/C++ and F77, interested in computational physics;
  19. Jim Wilgenbusch, Professor, CSIT FSU, jwilgenb@csit.fsu.edu, C/Java/perl user, interested in parallelization of existing phylogeny reconstruction code;
  20. Chunmei Xia, PostDoc, CSIT, FSU, C++ and FORTRAN user, xia@csit.fsu.edu, simulation of forest fire propagation;
  21. Jiyang Xia, PostDoc, CSIT, FSU, jiyang@csit.fsu.edu, 3D wind field simulation;
  22. Jinquan Xu, PostDoc, CSIT, FSU, xu@csit.fsu.edu, aero acoustics;
  23. Dekin Zhang, dzhang@met.fsu.edu;
  24. Dr Xiaolei Zou, Professor, FSU meteorology, zou@met.fsu.edu, 4D-Var data assimilation and numerical prediction;

Online MPI Course

There is an online course available for MPI, which is administered using the popular WebCT software. The first time you access the course, you'll need to set up an ID. You can work through as much of the course as you like, and return later at any time to continue your work. This course was developed by PACS partners.

To get to the online MPI course, go to webct.ncsa.uiuc.edu:8900/ .

Readable Introductions to MPI

Extensive MPI References