Instruction layout optimization can efficiently improve the performance of instruction cache by dramatically reducing the instruction fetch miss rate. Traditional instruction layout optimization methods usually do not consider tuning the hardware architecture of the instruction cache in the optimizing process. Therefore, they trend to result in local-optimal solutions. This paper studies the mutual effect of the instruction layout optimization and the instruction memory hierarchy. We built a framework to perform the instruction layout optimizations by profiling the call graph and reordering the instructions at the procedure level. Then, the original procedure and instruction layout optimized one are run on platforms with different cache hierarchy, and the cache miss rates are compared. Experimental results show that the instruction cache configuration greatly influences the benefit of instruction layout optimization, and the performance of the instruction cache could be potentially improved by jointly considering them together.