Distributed Processing Resource Modeling
This demo uses SystemTest to explore the performance and robustness of a distributed processing network. It does so by using a Simulink model describing the architecture resources for processing a series of jobs using distributed computing resources. This model is also used to explore the resource constraints associated with a selected architecture.
This demo requires the following products to run:
- Statistics Toolbox
The system generates jobs regularly and assumes that the complexity of a particular job forces the system to partition the job, process the partitions independently, and later reassemble the results into one block of data.
The model includes the following components:
- Job Generation - A subsystem that generates jobs at a fixed rate and assigns each job several pieces of data that represent the processing time constants. In this part of the model, each entity represents a job.
- Partitioning & Aggregation Processing - A server that models an initial delay before the first partition is available.
- Generate n Partitioned Tasks - A subsystem that creates a stream of partitions at a given rate. The input entities represent jobs and the output entities represent partitions.
- Task Queue - Stores the partitions that are waiting for processing.
- Task Processing Delay - Processes the partitions.
- Prepare Partitioned Task for Aggregation - A subsystem which copies the Aggregation_Time attribute's value to the Partitioning_Time attribute. As a result, when the partition reaches the server at the top of the model, the service time reflects aggregation time.
- A feedback connection in which the entities representing partitions arrive at the same server that models the initial delay for newly generated jobs.
- Processed Jobs - A sink block that absorbs each partition that departs from the server. A given partition does not need to wait for the other partitions of the same job to complete their processing.
In order to execute a test and explore how the system would respond to a varying number of processors and servers, two test vectors are defined in SystemTest:
- NumberProcessors - A test vector used to increase the number of processors that represent the time to process each partition. Values can be added to pinpoint the exact number of processors resulting in acceptable performance.
- NumberServers - A test vector that indicates the number of processors to use in order to resolve congestion in the partitioning and aggregation process.
The following test vectors use 10 random values, uniformly distributed using the unifrnd function from the Statistics Toolbox, in order to assess the robustness of the system. Since the values for these test vectors should not be permuted, they are defined within SystemTest to be grouped.
- ServiceTime - The time it takes (in seconds) to service jobs.
- JobCreate - The mean rate at which jobs are created (in jobs/second).
Using these test vectors, SystemTest can be used to explore how the system would perform with both high and low traffic networks.
Simulink Element - Simulates the distributed resource model using the specified test vector values.
MATLAB Element - Used to calculate the idle time and maximum number of jobs in the queue.
Limit Check Element - Specifies two conditions that must be met for an iteration to pass:
- The maximum number of jobs in the queue should not be too high.
- The processors should not be idle for too long.
General Plot Element - Used to visualize the pass/fail conditions of the test.
To view the test, use the systemtest function to open the test.
Open the demo in the SystemTest desktop.