Homomorphic Processing Unit

Fully Homomorphic Encryption (FHE) enables computation over encrypted data without decryption, ensuring data confidentiality throughout the processing pipeline. However, the complexity and heterogeneity of FHE schemes like CKKS, BFV, and TFHE pose challenges for unified hardware design. In this paper, we propose the Homomorphic Processing Unit (HPU), a general-purpose platform supporting multiple FHE schemes. Unlike fixed-function accelerators, the HPU is implemented as a RISC-V extension and introduces a dedicated homomorphic instruction set architecture (H-ISA), comprising micro-instructions for minimal compute kernels and macro-instructions for integration with general-purpose models. Core operations like the number-theoretic transform (NTT) and automorphism are abstracted into a unified instruction layer, mapped onto specialized homomorphic compute units. The HPU integrates a collaborative computation and storage design, utilizing a polynomial-level instruction set and a polynomial-granular memory management unit (MMU) to optimize memory and reduce data movement. By combining a complete ISA with a general-purpose computing core, the HPU can achieve efficient computing, control, and scheduling. We evaluate the HPU prototype using four multi-FHE benchmarks on FPGA and 7nm ASIC, achieving: (1) up to 1596\times/1174\times speedup over CPU for CKKS-alike and TFHE operations; (2) 1.36\times/1.2\times improvement compared to SOTA FPGA solutions for CKKS and TFHE; (3) the ASIC implementation achieves 1.36\times speedup over SOTA TFHE accelerators for LSTM.