Syllabus for

Computer Science 668/742

Parallel Computing --- Fall 2007

Time:         TuTh 11:00-12:15 PM
Professor:  Fred Annexstein








... 889 Rhodes Hall


... Department of CS


... College of Engineering


... University of Cincinnati


... Cincinnati, Ohio 45221




Web: Information on this course will be posted on the BLACKBOARD system and will be regularly updated. You should regularly check BLACKBOARD for announcements, updated syllabus, homework assignments, etc. 





Objectives: This class is designed as an introduction to the concepts and practice of Parallel Computing. In this class students will be introduced to some of these tools, techniques, and methods of analysis in parallel computing. We will do a number of programming projects during the term.


Grading Scheme: 

The grading for this class will be based on two lecture exams and approximately 6 homework assignments, including written assignments, programming assignments, a research report, and a final project report. Your final grade will be a weighted average for these two aspects - Homeworks 60% and Lecture Exams 40%.
Late homework or makeup exams are not allowed unless pre-arranged with the instructor.


Academic Dishonesty

  The standard penalty for any form of academic dishonesty in this course is failure in the course. Providing or receiving help during exams, or submitting the work of another as your own both constitute academic dishonesty. There are no "small" offenses to this policy. 


Course Description:

This course is designed as a dual level/ senior undergraduate level course covering the programming and algorithmic design issues arising in parallel computing. The course is designed to meet the competencies for Area 6 for the Minor Program in Computational Science of the Ralph Regula School of Computational Science. The following competencies are addressed in this course:

6.1 Describe the fundamental concepts of parallel programming and related architectures
6.2 Demonstrate parallel programming concepts using MPI
6.3 Demonstrate knowledge of parallel scalability
6.4 Demonstrate knowledge of parallel programming libraries


Parallel Programming in C With Mpi and Openmp
: Michael J. Quinn
Street Price $75.30


Reference Texts:

 Parallel Programming With Mpi: Peter Pacheco
Street Price $25.00
Introduction to Parallel Computing: Design and Analysis of Algorithms: Ananth Grama, Anshul Gupta, George Karpis, Vipin Kumar
Street Price $64.00

Using MPI - 2nd Edition: Portable Parallel Programming with the Message Passing Interface (Scientific and Engineering Computation) by William Gropp, Ewing Lusk, and Anthony Skjellu $34.00

Topics and Schedule:

Lecture Topics


Lecture Notes

Introduction to Parallel Computing

Chapter 1, 1.1-1.8


Parallel Architectures

Chapter 2, 2.3-2.7

cs668-lec1-Parallel Arch.ppt

Introduction to MPI

Chapter 4, 4.1-4.7


More about MPI

Chapter 5, 5.1-5.10


Floyd's Algorithm and Parallelization

Chapter 6, 6.1-6.8


Design and Analysis

Chapter 3, 3.1-3.5


Speedup, Amdahl's Law, Analysis, Iso-Efficiency, Scalability

Chapter 7, 7.1-7.7


Interconnection Networks, Distributed Memory, Distributed Shared Memory

Chapter 2, 2.1-2.2


Manger-Worker Paradigms, Document Classification, Parallel K-means

Chapter 9, 9.1-9.6


Linear Algebra, Matrix-Vector Multiplication

Chapter 8, 8.1-8.7


Monte Carlo Methods, and Parallel Simulation Algorithms

Chapter 10, 10.1-10.4


More on Monte Carlo Methods, Case Studies

Chapter 10, 10.5-10.6


Finite Difference Methods

Chapter 13, 13.1-13.5


Combinatorial Search

Chapter 16, 16.1-16.10


Shared Memory Parallel Programming, Scheduling and Work-stealing

Chapter17 17.1-17.3, Cilk Notes



Profiling Tools, Performance Tools, Parallel Libraries



Bridging Models, Bulk-Synchronous Programming






Programming Assignments


ItemProgramming Assignment #1: Due Tuesday Oct 2
Using the C code in figure 4.8 of Quinn's book (see below), write an MPI parallel version of the code to compute pi using the rectangle rule with 1,000,000 intervals. Benchmark your program on various numbers of processors and produce a graphic such as we saw in class. Extra credit for doing same with Simpson's Rule.

#include <stdio.h>
#define INTERVALS 1000000
int main(int argc, char* argv[])
  double area; /* The final answer */
  double ysum; /* Sum of rectangle heights */
  double xi;   /* Midpoint of interval */
  int i;
  ysum= 0.0;
      xi= ((1.0/INTERVALS)*(i+0.5)); /* Midpoint of interval */
      ysum += 4.0/(1.0 + xi*xi);
      printf("Area is %13.11f\n",area);
  return 0;

ItemProgramming Assignment #2 : Due Oct 15
Complete Exercise 6.13 from Quinn.
The assignment is to implement a well-known cellular automata simulation called the "Game of Life".
Each cell in a 2-D array is either live or dead. During each iteration a dead cell with 3 live neighbors is reborn. A live cell with less than 2 neighbors or more than three neighbors becomes dead. All updates are synchronized. Write a parallel program with command line parameters m,n,j,k where the m by n 2-D array is read from standard input. The simulation runs for j iterations, and every kth iteration the array is printed. You are also to complete a theoretical analysis of a parallel implementation of the problem.

Final Project Resources

Final Project: Parallel Application Performance Study Fall 2007


Web Resources


MPI Information Links

--  from Ryan McGovern



Midterm Study Sheet
CS668-Midterm-Study.doc (21 Kb)






Profiling: Joe and Mike recommend Valgrind.



Scalapack Project

OSC Notes:
Using Parallel Numerical Libraries (focus on ScaLAPACK)



Article on State of Computer Chess

From Adam G.: Here is the article I mentioned to you today that compares modern day computers to Deep Blue. It also goes into some of the heuristics used in the software. It also says there are roughly 10120 possible board positions in chess.,1558,1915527,00.asp 




Final Study Sheet: Final Study Sheet