Dataflow Models Revisited: Remembering the MIT Pioneer Who Shaped Computing

The way Jack Dennis spent his career has a subtle lyrical quality. He created graphs. Arrows link boxes, circles contain little operators, and lines indicate how a number could move from one calculation to the next. It could have appeared to anyone passing his MIT office in the late 1970s to be the doodling of a man who was too patient for his own good.

It wasn’t. Even though the majority of engineers who use it today are unable to identify the original creator, those sketches went on to become one of the most important abstractions in contemporary computer science.

When Dennis first started considering dataflow computing, the field was still firmly rooted in the von Neumann model, which was a methodical, nearly bureaucratic approach to processing instructions one after another, line by line, as if a computer were a clerk reading from a ledger. That did not satisfy him. He was right to believe that parallelism would be necessary in the future and that it didn’t fit nicely into a sequential worldview. He therefore put forth an odd idea: a paradigm in which commands were carried out as soon as their data were available. There is no program counter. There is no line. only availability.

One could argue that his concept became a global phenomenon. Really, it didn’t. Many of his students, including Arvind, were in charge of dataflow hardware projects in the 1980s, but they were unable to keep up with the constant advancements in traditional processors. Looking back, it seems like the timing was just off.

Even while dataflow architectures were wonderful on theory, Moore’s Law consistently outperformed them in practice, and engineers creating processors at the time were riding a wave of single-thread performance advances. The concept lingered in the background for almost fifteen years, appearing in scholarly papers that most undergraduates would never read, database query planners, and compilers. The internet then grew beyond anyone’s expectations.

Google programmers were struggling to crawl, index, and search a web that was doubling in size every few months by the early 2000s. The outcome was MapReduce, which anyone familiar with the tradition could immediately understand even if dataflow was hardly discussed in the original work. It was Dennis’s old graph, but it was grown up to ridiculous proportions. The dependencies, the map and reduce steps, and the way intermediate data flowed between nodes. Later, Apache Spark would go one step further and use lineage graphs to recover from failures in a way that previous systems were unable to. It’s difficult to ignore how much of the data infrastructure of today is based on presumptions that Dennis incorporated into his model decades ago.

Observing the praises and obituaries that have been circulating among researchers over the past year, I was struck by how unconcerned he remained. His coworkers characterize him as quiet and hesitant to take credit. He once explained to a pupil that effective abstractions were those that didn’t require thought. That has a humble quality to it. The man whose concepts now transfer trillions of bytes across data centers every second appeared to be primarily concerned with the accuracy of the calculations.

Whether the upcoming generation of distributed systems will resemble his vision is still up in the air. He never had to think about the boundaries being pushed by new architectures, particularly in the area of AI inference. However, the underlying intuition that computing should come after data rather than the other way around seems more pertinent now than it has ever been. He no longer has the whiteboard he used to write. The graphs are still there.