This course deals with the hardware and software aspects of multiprocessor systems (two or more processors in use simultaneously). The reasons for using such systems, including processor speed limitations and non-uniform capabilities of processors or computers, are examined. Basic hardware concepts such as machine models, parallel programs, networks, and performance and scalability are discussed. Relevant compiler and operating system concepts, programming models, and program development methodologies are introduced. Three lecture hours and one scheduled laboratory hour per week, plus additional laboratory work outside of class.
      Prerequisite: CSC 245.

• CG1: to provide an introduction to the conceptual basis of parallel processing that is independent of any particular technology;
• CG2: to introduce several modern parallel programming enviroments;
• CG3: to illustrate the solution of typical numerical and graphical problems in one or more of these environments.

• CO1: properly defined parallel processing and given examples of problems for which parallelization is appropriate;
• CO2: presented a discussion of several past or present parallel machines;
• CO3: programmed a heat flow problem or other applicable problem on a sequential machine and subsequently implemented that problem as a parallel program;
• CO4: programmed a clustered parallel machine made up of PCs networked together running PVM or another parallel package.

• parallel computer models
  • shared memory
  • distributed memory
  • SIMD and MIMD
  • PRAM, VLSI, and LogP models
• program and network properties
  • dependencies
  • partitioning
  • control flow vs. data flow
  • interconnect methods
• scalable performance
• parallel programming models
  • shared variable model
  • message passing model
  • data flow model
• parallel program design
  • partitioning
  • granularity
  • mapping techniques (indexing, hashing)
  • load balancing techniques (bin packing, randomization, pressure models)
• parallel languages
  • Fortran 90
  • Parallel C/C++
  • Ada95
• program development environments
• parallel operating systems

Case studies and laboratory exercises:
      There will be 4 to 6 design and programming assignments in which students will be asked to implement selected techniques in parallel programming. There will also be periodic written homework assignments.
      Problems will be assigned for solution in some of the following environments:

• PVM (parallel virtual machine) on a PC LAN
• concurrent programming on a single-processor machine
• shared-memory parallel processing on a multiprocessor UNIX system
• use of (part of) a massively parallel processor (MPP) via Internet access

      The examples studied in class and used for programming projects will be drawn from the following list or from similar projects:

• multiplication of large matrices
• parallel solution to the Producer/Consumer Problem
• parallel sorting and searching
• numerical analysis techniques such as Simpson's Rule for numeric integration
• numerical solution to the wave equation

Grading:
      The course grade will be determined using the following approximate weights: laboratory reports — 45% total; written homework — 15%; two one-hour exams — 10% each; final examination — 20%.

Bibliography:
Topic references

• Andrews, Greg.   Concurrent Programming: Principles and Practice. Addison Wesley, 1991.
• Chandy, K. Mani; Taylor, Stephen.   An Introduction to Parallel Programming.   Jones and Bartlett, 1992.
• Foster, Ian.   Designing and Building Parallel Programs: Concepts and Tools for Parallel Software Engineering.   Addison Wesley, 1995.
• Hwang, Kai.   Advanced Computer Architecture: Parallelism, Scalability, Programmability.   McGraw-Hill, 1993.
• Kumar, Vipin; Grama, Anrath; Gupta, Anshul; Karypis, George.   An Introduction to Parallel Computing: Design and Analysis of Algorithms.   Second Edition.   Addison Wesley, 2003.
• Nevison, C. H., et. al.   Laboratories for Parallel Computing.   Jones and Bartlett, 1994.

• Adams, Brainerd, Martin, Smith & Wagener.   Fortran 90 Handbook: Complete ANSI/ISO Reference.   Intertext/McGraw-Hill, 1992.
• Forsythe, Chester.   Contemporary Computing for Engineers and Scientists using Fortran 90.   PWS Publishing Company, 1997.
• Metcalf, Michael; Reid, John.   Fortran 90 Explained.   Oxford University Press, 1993 reprint.
  
• Cohen, Norman H.   Ada as a Second Language.   Second Edition.   McGraw-Hill, 1996.
• Gehani, Narain.   Ada.   An Advanced Introduction including Reference Manual for the Ada Programming Language.   Prentice Hall, 1984.