Formerly known as Floe
MagicEight is a computer system which supports medium to coarse grain
parallelism, using a dataflow model of execution. The primary data
type supported is a multidimensional stream of relatively small (8 -
1024 bits) scalar data elements. The machine architectures which MagicEight
is intended to support include single general purpose processors, or a
network of heterogeneous processors (using either message passing or
shared memory) of 2 - 500 processors. In case of multiple processors,
some may be specialized -- capable of executing a restricted set of
algorithms much more efficiently than a general purpose processor.
The MagicEight system is being developed, here at the MIT Media Laboratory, to address several
issues encountered in the development of parallel processing
systems :
The current status is "under development." The first stages of
construction are done, and simple programs are being educed, but the
system isn't quite usable at this time.
MagicEight was motivated by the desire to harness the power of specialized
processors for media processing, in a programmable and scalable manner.
We've built and tested an earlier media processing system,
Cheops,
which incorporates specialized processors using a stream mechanism.
In order to provide an early development platform for building
systems incorporating specialized processors, as well as simulating
the effects of different tradeoffs in the system design, MagicEight is
capable of running on a heterogeneous cluster of networked
workstations. The presence of specialized processors is optional.
When running on a commercially available workstation, MagicEight is executed
as a user application on top of a native operating system.
A traditional computer program (regardless of the high level
language used) is written in an imperative manner. One or more
sequences of instructions (with occasional conditional branches) are
defined by the programmers, and executed sequentially by a processor.
Parallelism is currently obtained, at one level, by scheduling
non-interfering consecutive instructions to occur simultaneously on
separate execution units. Control of such instruction level
parallelism may either be static (defined at compile time) ---
referred to as Very Long Instruction Word (VLIW) --- or dynamic
--- referred to as super scalar. The advantage of this
approach is that it works well with existing programming models. Only
the compiler need know that instruction parallelism is being used.
The disadvantage is that achieving more than a small amount (3 - 8) of
parallelism in an instruction sequence is difficult.
Parallelism may be obtained at another level through one of several
mechanisms: parallel threads, or processes; typically targetting a
particular computer system architecture and granularity of
parallelism. The required partitioning of algorithms into a
particular number of threads must currently be done by the
programmers.
We suggest instead that applications be written in an intensional
language, which represents the algorithms using a minimal amount of
programmer imposed dependencies. In effect, writing in an intensional
language is expressing only the basic equations (or intensions) relating
multidimensional input, intermediate, and output data streams, leaving the
details of how to the interpreting system.
Instead of implementing a fine-grained dataflow system to execute
an application written in such a manner, we instead propose the use of
a hybrid dataflow system. In such a system, small sequences of
instructions, with clearly defined inputs and outputs, are scheduled
and executed in a dataflow manner. I extend hybrid dataflow to
include dynamic data parallelism, scheduling the evaluation of a set
of output values (a stream fragment ) based on the number of
available parallel processors at a given time.
A MagicEight user program thus contains two different means of algorithm
specification: dataflow and imperative. The imperative portions are
written using any standard programming language (such as C). The
dataflow portion may be specified using a graphical programming
language, or simply a set of data structures defined using an
imperative initialization routine (the way Cheops is
usually programmed.) It is hoped, however, that MagicEight applications
will be written in a higher level
intensional language, such as
Lucid.
Motivation
Imperative Programming
Intensional Programming & Streams
Additional Documents
An early paper on the MagicEight system, presented at the Workshop on Parallel Processing in Multimedia, held in conjuction with IPPS '97, April 1, 1997
The programming model implemented by MagicEight is named Q. It is specified by the Q Instruction Set Architecture:
A programming language which directly maps into the elements of the Q instruction set architecture is also provided. It is based largely on Multidimensional Lucid, with slightly different primitives and syntax. It is specified by The Q Language:
The stream mechanism used both to facilitate dynamic partitioning of data to match available parallelism, and for synchronization, is described more thoroughly in a paper published in the Proceedings of the 137th SMPTE Technical Conference, September 1995:
John A. Watlington and V. Michael Bove, Jr., "Stream-Based Computing and Future Television",
Jump to the Cheops Homepage
Jump to the Media Lab Homepage