IP-over-optical multilayer networks are capable of flexibly dealing with traffic increases and fluctuations, because the networks support both high-speed transmission using end-to-end lightpaths and scalable hop-by-hop IP transmission. This paper introduces a Differentiated Services (DiffServ) architecture for Quality of Service (QoS) control in such networks. The architecture supports both class-based queues and class-based lightpaths to efficiently handle multiple-QoS-class traffic. QoS schemes based on the proposed architecture are categorized into four types according to their traffic-differentiation and transmission mechanisms. Through simulation, the schemes are evaluated in terms of a measure that largely determines network costs.