IPULOC—Exploring Dynamic Program Locality with the Instruction Processing Unit for Filling Memory Gap

Memory gap has become an essential factor influencing the peakperformance of high-speed CPU-based systems. To fill this gap,enlarging cache capacity has been a traditional method based on staticprogram locality principle. However, the order of instructions storedin I-Cache before being sent to Data Processing Unit (DPU) is a kind ofuseful information that has not ever been utilized before. Soan architecture containing an Instruction Processing Unit (IPU) inparallel with the ordinary DPU is proposed. The IPU can prefetch, analyze andpreprocess a large amount of instructions otherwise lying in theI-Cache untouched. It is more efficient than the conventionalprefetch buffer that can only store several instructions forpreviewing. By IPU, Load Instructions can be preprocessed while the DPUis executing on data simultaneously. It is termed as ``InstructionProcessing Unit with Lookahead Cache'' (IPULOC for short) in which theidea of dynamic program locality is presented. This paper describes theprinciple of IPULOC and illustrates the quantitative parameters forevaluation. Tools for simulating the IPULOC have been developed. The simulationresult shows that it can improve program locality during programexecution, and hence can improve the cache hit ratio correspondinglywithout further enlarging the on-chip cache that occupies a largeportion of chip area.