Abstract: Surgical data science (SDS) aims to model complex surgical workflows, predict clinical outcomes, and enhance surgical efficiency, patient safety, and personalized treatment. This survey reviews SDS with emphasis on surgical scene understanding, including workflow and phase recognition, skill assessment, intra-operative augmented reality, and surgical embodied intelligence, while also outlining representative datasets and recent progress in deep learning and foundation models in this field. As these tasks are mostly intra-operative, they are inherently time-critical. Delayed or inaccurate responses can directly compromise safety and effectiveness. Yet the scaling laws that have fueled recent breakthroughs in AI now reveal critical bottlenecks: ever-larger models impose prohibitive computational and energy demands, making real-time intra-operative deployment increasingly impractical. To address this challenge, we highlight brain-inspired computing, particularly neuromorphic systems and spiking neural networks (SNNs), as a fundamentally different paradigm designed for ultra-low-latency and energy-efficient processing. Recent prototypes such as IBM NorthPole, Tianjic, ActiveN, and SpiNNaker demonstrate the feasibility of scalable neuromorphic hardware, while SNNs are achieving competitive or even superior performance compared with current deep learning models on tasks ranging from image classification to video understanding. By aligning the efficiency and adaptability of biological neural systems with the stringent time-critical demands of surgery, brain-inspired computing offers a sustainable alternative to GPU-based systems and paves the way toward next-generation AI-assisted surgical systems capable of real-time intelligence.