We present the architecture of nreduce, a distributed vir- tual machine which uses parallel graph reduction to run programs across a set of computers. It executes code written in a simple functional language which supports lazy evalu- ation and automatic parallelisation. The execution engine abstracts away details of parallelism and distribution, and uses JIT compilation to produce efficient code. This work is part of a broader project to provide a pro- gramming environment for developing distributed applica- tions which hides low-level details from the application de- veloper. The language we have designed plays the role of an intermediate form into which existing functional languages can be transformed. The runtime system demonstrates how distributed execution can be implemented directly within a virtual machine, instead of a separate piece of middleware that coordinates the execution of external programs.