Parallel Computing Toolbox 4.3

Programming Parallel Applications in MATLAB

Parallel Computing Toolbox provides several high-level programming constructs that let you convert your serial MATLAB code to run in parallel on several workers (MATLAB computational engines that run independently of MATLAB clients). 

Constructs such as parallel for-loops (parfor) and distributed arrays simplify parallel code development by abstracting away the complexity of managing coordination and distribution of computations and data between a MATLAB client and workers, as well as between workers. These constructs function even in the absence of workers, letting you maintain a single version of your code for both serial and parallel execution.

Using parfor to Run Loops in Parallel 2:49
Convert serial applications to parallel using simple annotations, such as parfor and spmd.

You can run an application that uses up to eight workers on your multicore desktop. Using MATLAB Distributed Computing Server, you can run the same application without modification on large resources such as computer clusters or grid and cloud computing services.

Taking Advantage of Built-in Parallel Computing Support in MathWorks Products

Key functions in a growing number of products in the MATLAB and Simulink product families, such as Statistics Toolbox™ and Simulink Design Optimization™, provide built-in support for parallel computing. In the presence of Parallel Computing Toolbox, these functions exploit parallel computing resources by distributing computations across available workers, enabling you to take advantage of parallel computing resources without additional programming effort.

Implementing Task-Parallel Algorithms

You can parallelize Monte Carlo simulations and other coarse-grained or embarrassingly parallel problems by organizing them into independent tasks (units of work). Parallel for-loops in the toolbox offer one way to distribute tasks across multiple MATLAB workers. Using these loops, you can automatically distribute independent loop iterations to multiple MATLAB workers. The parfor construct manages data and code transfer between the MATLAB client session and the workers. It automatically detects the presence of workers and reverts to serial behavior if none are present.

Using parallel for-loops for a task-parallel application. You can use parallel for-loops in MATLAB scripts and functions and execute them both interactively and offline.

You can also explicitly program tasks either as MATLAB functions or as MATLAB scripts. You execute the tasks, when specified as functions, by manipulating task and job objects in the toolbox or, when specified as scripts, by using the batch function.

Shorten Simulation Times with Parallel Computing 5:39
Distribute computations across workers using parfor.

Implementing Data-Parallel Algorithms

For MATLAB algorithms that require large data set processing, Parallel Computing Toolbox provides distributed arrays and parallel functions that can operate on these arrays. Distributed arrays let you process significantly larger data sets than you could in a single MATLAB session by distributing data across multiple workers and using parallel functions to perform operations on the distributed data. The toolbox also provides functions for moving distributed data to and from MAT-files.

You can work with distributed arrays interactively from your MATLAB session as you would with MATLAB arrays. Working with distributed arrays and their associated parallel functions does not require knowledge of low-level programming tools, such as MPI. The toolbox provides more than 150 parallel functions for operating on distributed arrays, including linear algebra routines based on ScaLAPACK. You can use the familiar operators for arrays in MATLAB to perform indexing, matrix multiplication, and transforms directly on distributed arrays.

Programming with distributed arrays. Distributed arrays and parallel algorithms let you create data-parallel MATLAB programs with minimal changes to your code and without programming in MPI.

The toolbox also provides the spmd (single program multiple data) construct, which you can use to designate sections of your code to run concurrently across workers participating in a parallel computation. During program execution, spmd automatically transfers data and code used within its body to the workers and, once the execution is complete, brings results back to the MATLAB client session.

For explicit, fine-grained control over your parallelization scheme, which also requires explicitly managing interworker coordination, Parallel Computing Toolbox functions provide access to message-passing routines based on the MPI standard (MPICH2), including functions for send, receive, broadcast, barrier, and probe operations.

Working Interactively with Distributed Arrays to Process Large Data Sets: Data-Parallel Applications  5:41
Handle problems with large memory requirements with constructs such as spmd and distributed arrays.