Exposure to CS330 (Operating Systems), CS422 (Computer Architecture) and CS425 (Computer Networks) is desirable.
Phd, Masters, 3rd and 4th year UG Students
Parallel programming is ubiquitous in today’s multi-core era and solves many real-world scientific problems. Massive parallelism entails significant hardware and software challenges. The course is structured so that the participants understand challenges in efficient execution of large-scale parallel applications. The assignments will be designed to strengthen understanding of parallel programming. The course will also involve a research-based project component.
|S. No.||Broad Title||Topics||No. of Lectures|
|1.||Introduction||Why parallel computing? Shared memory and distributed memory parallelism, Amdahl’s law, speedup and efficiency, supercomputers.||2|
|2.||Message passing||MPI basics, point-to-point communication, collective communication, synchronous/asynchronous send/recv,
algorithms for gather, scatter, broadcast, reduce.
|Network topologies, network evaluation metrics, communication cost, routing ininterconnection networks, static and adaptive
routing, process-to-processor mapping.
|4.||Performance||Scalability, benchmarking, performance modeling, impact of network topologies, parallel code analysis and profiling.||7|
Designing parallel codes
|Domain decomposition, communication-to-computation ratio, load
balancing, adaptivity, case studies: weather and material simulation codes.
|6.||Parallel I/O||MPI I/O algorithms, contemporary large-scale I/O architecture, I/O bottlenecks.||6|
|7.||Additional topics||Job scheduling, RDMA, one-sided communication, NVM, extreme scale computing: issues and trends.||4|
DE Culler, A Gupta and JP Singh, Parallel Computer Architecture: A Hardware/Software Approach Morgan-Kaufmann, 1998.
Marc Snir, Steve W. Otto, Steven Huss-Lederman, David W. Walker and Jack Dongarra, MPI - The Complete Reference, Second Edition, Volume 1, The MPI Core.
William Gropp, Ewing Lusk, Anthony Skjellum, Using MPI : portable parallel programming with the message-passing interface, 3rd Ed., Cambridge MIT Press, 2014.
A Grama, A Gupta, G Karypis, and V Kumar, Introduction to Parallel Computing. 2nd Ed., Addison-Wesley, 2003.
JL Hennessy and DA Patterson, Computer Architecture: A Quantitative Approach, 4th Ed., Morgan Kaufmann/Els India, 2006.
MJ Quinn, Parallel Computing: Theory and Practice, Tata McGraw Hill, 2002.