SCALaR: A Unified Latent Reasoning Framework for Large Language Models with Multi-Layer Aggregation and Strategic Compression

Large language models have demonstrated remarkable reasoning capabilities when guided by explicit chain-of-thought. However, generating long reasoning traces incurs substantial computational cost due to linguistic and stylistic redundancy. This inefficiency limits the scalability and practical deployment of such methods. To address this issue, recent work has explored latent reasoning, in which models perform reasoning implicitly within internal representations instead of generating explicit intermediate tokens. While promising, existing latent reasoning approaches face key challenges, including representational limitations arising from reliance on final-layer hidden states and optimization difficulties caused by sparse supervision. In this work, we propose SCALaR, a unified framework for latent reasoning that improves reasoning effectiveness through data compression, representation construction, and training strategy. SCALaR compresses training data by retaining the most informative elements, constructs enriched latent representations via multi-layer aggregation, injects calibrated Gaussian noise during training to encourage latent-space exploration, and adopts an alternating compression strategy to balance intermediate supervision and answer-oriented exploration. Extensive experiments on mathematical reasoning and question-answering benchmarks show that SCALaR consistently outperforms existing latent reasoning baselines. Additional analyses examine the individual roles of each component, providing insights into how they contribute to effective latent reasoning.