![[logo] US EPA](http://www.epa.gov/epafiles/images/logo_epaseal.gif){kind=logo}
Computational Science
Research Programs
Model Development
Distributed or so-called massively parallel processing enables the efficient computation of problems with very large problem sizes. However, programming for effective distributed processing requires careful code design and structuring. The basic principle of parallel computing is divide-and-conquer. This principle renders lower overall computational time (CPU) and potentially reduced local memory utilization (storage), and is achieved by decomposing the problem space into many smaller problems that are solved concurrently.
In most applications, a processor must access data that reside in the memory of other processors.
The following is an example to illustrate this.
DO J = 1, N
DO I = 1, M
DATA(I,J) = A(I+2,J) * A(I, J-1)
END DO
END DO
Depending on the platform, there are two main data access paradigms: message passing and shared memory, and in some cases, a combination of both. Each paradigm has its pros and cons. It might be convenient, for example, for programmers to see a global memory model on a distributed processor system. However, interprocessor communication bandwidth might become a serious perfomance issue. For the platforms we envision that will run the Community Multi-scale Air Quality (CMAQ) models, message passing seems the best choice, and consequently we have developed some of the major codes, in particular the CMAQ Chemistry-Transport Model (CCTM), to use this paradigm.
The backbone of the inter-processor communication within CMAQ is the STENEX library which was developed inhouse and based on Message Passing Interface (MPI). STENEX library handles domain decomposition details as well as various types of inter-communication scheme, for example, (a) interior to ghost region, where the ghost region is indicated in light blue, (b) interior to interior, (c) subsection data redistribution, and (d) selective data collection.
( a ) interior to ghost region
( b ) interior to interior
( c ) redistribution of a subsection of data
( d ) selective data collection
Contacts: David Wong, Jeff Young