The development of Virtual Environment (VE) systems is a challenging endeavor with a complex problem domain. The experience in the past decade has helped contribute significantly to various measures of software quality of the resulting VE systems. However, the resulting solutions remain monolithic in nature without addressing successfully the issue of system interoperability and software aging. This paper argues that the problem resides in the traditional system centric approach and that an alternative approach based on system of systems engineering is necessary. As a result, the paper presents a reference architecture based on layers, where only the core is required for deployment and all others are optional. The paper also presents an evaluation methodology to assess the validity of the resulting architecture, which was applied to the proposed core layer and involving individual sessions with 12 experts in developing VE systems.