The HTMT project is an ambitious attempt to combine a variety of emerging technologies into a petaflops-level computing system available many years before an equivalent machine can be built from current technologies. One of the key problems in such architecture is overcoming latencies between the main memory and the high performance CPUs, which can grow to literally tens of thousands of cycles. In HTMT, the approach taken to overcoming this is a multi-level memory system, with most of the levels to be fabricated using Processing-In-Memory (PIM) technologies in architectures, which actively manage the flow of data without centralized CPU control. This paper overviews the current architecture for such chips within the context of the HTMT system, and how this architecture supports the expected execution model.