School of Computer and Information Engineering, Beijing Technology and Business University, Beijing 100048, China;Beijing Key Laboratory of Big Data Technology for Food Safety, Beijing Technology and Business University Beijing 100048, China
Abstract This paper presents a novel algorithm for automatically detecting global shakiness in casual videos. Per-frame amplitude is computed by the geometry of motion, based on the kinematic model defined by inter-frame geometric transformations. Inspired by motion perception, we investigate the just-noticeable amplitude of shaky motion perceived by human visual system. Then, we use the thresholding contrast strategy on the statistics of per-frame amplitudes to determine the occurrence of perceived shakiness. For testing the detection accuracy, a dataset of video clips is constructed with manual shakiness label as the ground truth. The experiments demonstrate that our algorithm can obtain good detection accuracy that is in concordance with subjective judgement on the videos in the dataset.
This work was partially supported by the National Natural Science Foundation of China under Grant No. 61602015, the Open Funding Project of State Key Laboratory of Virtual Reality Technology and Systems at Beihang University under Grant No. BUAAVR-16KF-06, Beijing Natural Science Foundation under Grant No. 4162019, and the Research Foundation for Young Scholars of Beijing Technology and Business University.
About author: Xiao-Qun Wu is now a lecturer in the School of Computer and Information Engineering, Beijing Technology and Business University, Beijing. She received her B.S. and M.S. degrees in mathematics from Zhejiang University, Hangzhou, in 2007 and 2009, respectively, and her Ph.D. degree in computer science from Nanyang Technological University, Singapore, in 2014. Her research focuses on computer graphics, multimedia processing and applications.
Cite this article:
Xiao-Qun Wu, Hai-Sheng Li, Jian Cao, Qiang Cai.Geometry of Motion for Video Shakiness Detection[J] Journal of Computer Science and Technology, 2018,V33(3): 475-486
 Abdollahian G, Taskiran C M, Pizlo Z, Delp E J. Camera motion-based analysis of user generated video. IEEE Trans. Multimedia, 2010, 12(1):28-41. Hu S M, Chen T, Xu K, Cheng M M, Martin R R. Internet visual media processing:A survey with graphics and vision applications. The Visual Computer, 2013, 29(5):393-405. Zhang L, Zhou L, Huang H. Bundled kernels for nonuniform blind video deblurring. IEEE Trans. Circuits and Systems for Video Technology, 2017, 27(9):1882-1894. Yan F, Iliyasu A M, Yang H M, Hirota K. Strategy for quantum image stabilization. Science China Information Sciences, 2016, 59(5):052102. Kakar P, Sudha N, Ser W. Exposing digital image forgeries by detecting discrepancies in motion blur. IEEE Trans. Multimedia, 2011, 13(3):443-452. Su B L, Lu S J, Tan C L. Blurred image region detection and classification. In Proc. the 19th ACM Int. Conf. Multimedia, November 2011, pp.1397-1400. Yu X, Xu F, Zhang S L, Zhang L. Efficient patch-wise nonuniform deblurring for a single image. IEEE Trans. Multimedia, 2014, 16(6):1510-1524. Visentini-Scarzanella M, Dragotti P L. Video jitter analysis for automatic bootleg detection. In Proc. the 14th Int. Workshop on Multi-Media Signal Processing, September 2012, pp.101-106. Sibiryakov A. Hand jitter descriptor for mobile video identification. In Proc. Int. Conf. Consumer Electronics, January 2011, pp.77-78. Chen H H, Liang C K, Peng Y C, Chang H A. Integration of digital stabilizer with video codec for digital video cameras. IEEE Trans. Circuits and Systems for Video Technology, 2007, 17(7):801-813. Xue Y Y, Erkin B, Wang Y. A novel no-reference video quality metric for evaluating temporal jerkiness due to frame freezing. IEEE Trans. Multimedia, 2015, 17(1):134-139. Yan B, Yuan B H, Yang B. Effective video retargeting with jittery assessment. IEEE Trans. Multimedia, 2014, 16(1):272-277. Zhang F L, Wang J, Zhao H, Martin R R, Hu S M. Simultaneous camera path optimization and distraction removal for improving amateur video. IEEE Trans. Image Processing, 2015, 24(12):5982-5994. Zhang L, Xu Q K, Huang H. A global approach to fast video stabilization. IEEE Trans. Circuits and Systems for Video Technology, 2017, 27(2):225-235. Huang H Z, Fang X N, Ye Y F, Zhang S H, Rosin P L. Practical automatic background substitution for live video. Computational Visual Media, 2017, 3(3):273-284. Hasegawa K, Saito H. Synthesis of a stroboscopic image from a hand-held camera sequence for a sports analysis. Computational Visual Media, 2016, 2(3):277-289. Joshi N, Kienzle W, Toelle M, Uyttendaele M, Cohen M F. Real-time hyperlapse creation via optimal frame selection. ACM Trans. Graphics, 2015, 34(4):Article No. 63. Wang M, Liang J B, Zhang S H, Lu S P, Shamir A, Hu S M. Hyper-lapse from multiple spatially-overlapping videos. IEEE Trans. Image Processing, 2018, 27(4):1735-1747. Tong H H, Li M J, Zhang H J, Zhang C S. Blur detection for digital images using wavelet transform. In Proc. Int. Conf. Multimedia and Expo., June 2004, pp.17-20. Tico M, Trimeche M, Vehvilainen M. Motion blur identification based on differently exposed images. In Proc. Int. Conf. Image Processing, October 2006, pp.2021-2024. Liu R T, Li Z R, Jia J Y. Image partial blur detection and classification. In Proc. Conf. Computer Vision and Pattern Recognition, June 2008. Yan W Q, Kankanhalli M S. Detection and removal of lighting & shaking artifacts in home videos. In Proc. ACM Int. Conf. Multimedia, December 2002, pp.107-116. Liu F, Gleicher M, Jin H L, Agarwala A. Content-preserving warps for 3D video stabilization. ACM Trans. Graphics, 2009, 28(3):Article No. 44. Zhang L, Chen X Q, Kong X Y, Huang H. Geodesic video stabilization in transformation space. IEEE Trans. Image Processing, 2017, 26(5):2219-2229. Wolpert D M, Ghahramani Z. Computational principles of movement neuroscience. Nature Neuroscience, 2000, 3(Suppl):1212-1217. Murray R M, Li Z X, Sastry S S. A Mathematical Introduction to Robotic Manipulation. CRC Press, 1994. Zacur E, Bossa M, Olmos S. Left-invariant Riemannian geodesics on spatial transformation groups. SIAM Journal on Imaging Sciences, 2014, 7(3):1503-1557. Duan L Y, Jin J S, Tian Q, Xu C S. Nonparametric motion characterization for robust classification of camera motion patterns. IEEE Trans. Multimedia, 2006, 8(2):323-340. Afonso M V, Nascimento J C, Marques J S. Automatic estimation of multiple motion fields from video sequences using a region matching based approach. IEEE Trans. Multimedia, 2013, 16(1):1-14. Nishi K, Onda T. Evaluation system for camera shake and image stabilizers. In Proc. Int. Conf. Multimedia and Expo., July 2010, pp.926-931. Albright T D, Stoner G R. Visual motion perception. Proceedings the National Academy of Sciences of the United States of America, 1995, 92(7):2433-2440. Peli E, García-Pérez M A. Motion perception during involuntary eye vibration. Experimental Brain Research, 2003, 149(4):431-438. Healey C G, Sawant A P. On the limits of resolution and visual angle in visualization. ACM Trans. Applied Perception, 2012, 9(4):Article No. 20. Martins A J, Kowler E, Palmer C. Smooth pursuit of smallamplitude sinusoidal motion. Journal of the Optical Society of America A, 1985, 2(2):234-242. He K M, Chang H W, Sun J. Content-aware rotation. In Proc. Int. Conf. Computer Vision, December 2013, pp.553-560. Shi J B, Tomasi C. Good features to track. In Proc. Computer Society Conf. Computer Vision and Pattern Recognition, June 1994, pp.593-600.
Copyright 2010 by Journal of Computer Science and Technology