Today, the amount of main memory in mid-range servers is pushing practical limits with as much as 192 GB memory in a 24 processor system. Further, with the onset of multi-threaded, multi-core processor chips, it is likely that the number of memory chips per processor chip will start to increase, making DRAM cost and size an even larger burden. We investigate in this paper an alternative main memory organization - a two-level noninclusive memory hierarchy - where the second level is substantially slower than the first level, with the aim of reducing total system cost and spatial requirements of servers of today and the future. We quantitatively investigate how big and how slow the second level can be. Surprisingly, we find that only 30% of the entire memory resources typically needed must be accessed at DRAM speed whereas the rest can be accessed at a speed that is an order of magnitude slower with a negligible (1.2% on average) performance impact. We also present a cost-effective implementation of how to manage such a hierarchy and how it can bring down memory cost by leveraging memory compression and sharing of memory resources among servers.