This paper proposes an integer linear programming (ILP) solution to the combined problem of memory hierarchy design and data allocation in the context of embedded chip multiprocessors. The proposed solution uses compiler analysis to extract data access patterns of parallel processors and employs integer linear programming for determining optimal on-chip memory partitioning across processors and data allocations across the resulting memory components. Our experimental results show that the applicationspecific on-chip memory hierarchies designed using this approach are much more energy efficient than conventional (pure shared or pure private) on-chip memories, conventional caches, and those designed by a prior work that partitions memory space across parallel processors without designing a multi-level hierarchy.