›› 2018, Vol. 33 ›› Issue (3): 475-486.doi: 10.1007/s11390-018-1832-5

Special Issue: Artificial Intelligence and Pattern Recognition; Computer Graphics and Multimedia

• Special Section of CVM 2018 • Previous Articles     Next Articles

Geometry of Motion for Video Shakiness Detection

Xiao-Qun Wu, Member, CCF, Hai-Sheng Li, Member, CCF, Member, IEEE, Jian Cao, Member, CCF, Qiang Cai, Senior Member, CCF, Member, IEEE   

  1. School of Computer and Information Engineering, Beijing Technology and Business University, Beijing 100048, China;Beijing Key Laboratory of Big Data Technology for Food Safety, Beijing Technology and Business University Beijing 100048, China
  • Received:2018-01-04 Revised:2018-03-23 Online:2018-05-05 Published:2018-05-05
  • Contact: Xiao-Wu Chen E-mail:chen@buaa.edu.cn
  • About author:Xiao-Qun Wu is now a lecturer in the School of Computer and Information Engineering, Beijing Technology and Business University, Beijing. She received her B.S. and M.S. degrees in mathematics from Zhejiang University, Hangzhou, in 2007 and 2009, respectively, and her Ph.D. degree in computer science from Nanyang Technological University, Singapore, in 2014. Her research focuses on computer graphics, multimedia processing and applications.
  • Supported by:

    This work was partially supported by the National Natural Science Foundation of China under Grant No. 61602015, the Open Funding Project of State Key Laboratory of Virtual Reality Technology and Systems at Beihang University under Grant No. BUAAVR-16KF-06, Beijing Natural Science Foundation under Grant No. 4162019, and the Research Foundation for Young Scholars of Beijing Technology and Business University.

This paper presents a novel algorithm for automatically detecting global shakiness in casual videos. Per-frame amplitude is computed by the geometry of motion, based on the kinematic model defined by inter-frame geometric transformations. Inspired by motion perception, we investigate the just-noticeable amplitude of shaky motion perceived by human visual system. Then, we use the thresholding contrast strategy on the statistics of per-frame amplitudes to determine the occurrence of perceived shakiness. For testing the detection accuracy, a dataset of video clips is constructed with manual shakiness label as the ground truth. The experiments demonstrate that our algorithm can obtain good detection accuracy that is in concordance with subjective judgement on the videos in the dataset.

[1] Abdollahian G, Taskiran C M, Pizlo Z, Delp E J. Camera motion-based analysis of user generated video. IEEE Trans. Multimedia, 2010, 12(1):28-41.

[2] Hu S M, Chen T, Xu K, Cheng M M, Martin R R. Internet visual media processing:A survey with graphics and vision applications. The Visual Computer, 2013, 29(5):393-405.

[3] Zhang L, Zhou L, Huang H. Bundled kernels for nonuniform blind video deblurring. IEEE Trans. Circuits and Systems for Video Technology, 2017, 27(9):1882-1894.

[4] Yan F, Iliyasu A M, Yang H M, Hirota K. Strategy for quantum image stabilization. Science China Information Sciences, 2016, 59(5):052102.

[5] Kakar P, Sudha N, Ser W. Exposing digital image forgeries by detecting discrepancies in motion blur. IEEE Trans. Multimedia, 2011, 13(3):443-452.

[6] Su B L, Lu S J, Tan C L. Blurred image region detection and classification. In Proc. the 19th ACM Int. Conf. Multimedia, November 2011, pp.1397-1400.

[7] Yu X, Xu F, Zhang S L, Zhang L. Efficient patch-wise nonuniform deblurring for a single image. IEEE Trans. Multimedia, 2014, 16(6):1510-1524.

[8] Visentini-Scarzanella M, Dragotti P L. Video jitter analysis for automatic bootleg detection. In Proc. the 14th Int. Workshop on Multi-Media Signal Processing, September 2012, pp.101-106.

[9] Sibiryakov A. Hand jitter descriptor for mobile video identification. In Proc. Int. Conf. Consumer Electronics, January 2011, pp.77-78.

[10] Chen H H, Liang C K, Peng Y C, Chang H A. Integration of digital stabilizer with video codec for digital video cameras. IEEE Trans. Circuits and Systems for Video Technology, 2007, 17(7):801-813.

[11] Xue Y Y, Erkin B, Wang Y. A novel no-reference video quality metric for evaluating temporal jerkiness due to frame freezing. IEEE Trans. Multimedia, 2015, 17(1):134-139.

[12] Yan B, Yuan B H, Yang B. Effective video retargeting with jittery assessment. IEEE Trans. Multimedia, 2014, 16(1):272-277.

[13] Zhang F L, Wang J, Zhao H, Martin R R, Hu S M. Simultaneous camera path optimization and distraction removal for improving amateur video. IEEE Trans. Image Processing, 2015, 24(12):5982-5994.

[14] Zhang L, Xu Q K, Huang H. A global approach to fast video stabilization. IEEE Trans. Circuits and Systems for Video Technology, 2017, 27(2):225-235.

[15] Huang H Z, Fang X N, Ye Y F, Zhang S H, Rosin P L. Practical automatic background substitution for live video. Computational Visual Media, 2017, 3(3):273-284.

[16] Hasegawa K, Saito H. Synthesis of a stroboscopic image from a hand-held camera sequence for a sports analysis. Computational Visual Media, 2016, 2(3):277-289.

[17] Joshi N, Kienzle W, Toelle M, Uyttendaele M, Cohen M F. Real-time hyperlapse creation via optimal frame selection. ACM Trans. Graphics, 2015, 34(4):Article No. 63.

[18] Wang M, Liang J B, Zhang S H, Lu S P, Shamir A, Hu S M. Hyper-lapse from multiple spatially-overlapping videos. IEEE Trans. Image Processing, 2018, 27(4):1735-1747.

[19] Tong H H, Li M J, Zhang H J, Zhang C S. Blur detection for digital images using wavelet transform. In Proc. Int. Conf. Multimedia and Expo., June 2004, pp.17-20.

[20] Tico M, Trimeche M, Vehvilainen M. Motion blur identification based on differently exposed images. In Proc. Int. Conf. Image Processing, October 2006, pp.2021-2024.

[21] Liu R T, Li Z R, Jia J Y. Image partial blur detection and classification. In Proc. Conf. Computer Vision and Pattern Recognition, June 2008.

[22] Yan W Q, Kankanhalli M S. Detection and removal of lighting & shaking artifacts in home videos. In Proc. ACM Int. Conf. Multimedia, December 2002, pp.107-116.

[23] Liu F, Gleicher M, Jin H L, Agarwala A. Content-preserving warps for 3D video stabilization. ACM Trans. Graphics, 2009, 28(3):Article No. 44.

[24] Zhang L, Chen X Q, Kong X Y, Huang H. Geodesic video stabilization in transformation space. IEEE Trans. Image Processing, 2017, 26(5):2219-2229.

[25] Wolpert D M, Ghahramani Z. Computational principles of movement neuroscience. Nature Neuroscience, 2000, 3(Suppl):1212-1217.

[26] Murray R M, Li Z X, Sastry S S. A Mathematical Introduction to Robotic Manipulation. CRC Press, 1994.

[27] Zacur E, Bossa M, Olmos S. Left-invariant Riemannian geodesics on spatial transformation groups. SIAM Journal on Imaging Sciences, 2014, 7(3):1503-1557.

[28] Duan L Y, Jin J S, Tian Q, Xu C S. Nonparametric motion characterization for robust classification of camera motion patterns. IEEE Trans. Multimedia, 2006, 8(2):323-340.

[29] Afonso M V, Nascimento J C, Marques J S. Automatic estimation of multiple motion fields from video sequences using a region matching based approach. IEEE Trans. Multimedia, 2013, 16(1):1-14.

[30] Nishi K, Onda T. Evaluation system for camera shake and image stabilizers. In Proc. Int. Conf. Multimedia and Expo., July 2010, pp.926-931.

[31] Albright T D, Stoner G R. Visual motion perception. Proceedings the National Academy of Sciences of the United States of America, 1995, 92(7):2433-2440.

[32] Peli E, García-Pérez M A. Motion perception during involuntary eye vibration. Experimental Brain Research, 2003, 149(4):431-438.

[33] Healey C G, Sawant A P. On the limits of resolution and visual angle in visualization. ACM Trans. Applied Perception, 2012, 9(4):Article No. 20.

[34] Martins A J, Kowler E, Palmer C. Smooth pursuit of smallamplitude sinusoidal motion. Journal of the Optical Society of America A, 1985, 2(2):234-242.

[35] He K M, Chang H W, Sun J. Content-aware rotation. In Proc. Int. Conf. Computer Vision, December 2013, pp.553-560.

[36] Shi J B, Tomasi C. Good features to track. In Proc. Computer Society Conf. Computer Vision and Pattern Recognition, June 1994, pp.593-600.
No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] Shen Xubang; Ma Guangti; Chen Lan;. An Inference Microprocessor Design[J]. , 1991, 6(3): 209 -213 .
[2] Fei Xianglin; Liao Lei; Wang Hezhen; Wang Chengzao;. Structured Development Environment Based on the Object-Oriented Concepts[J]. , 1992, 7(3): 193 -201 .
[3] Shen Yidong;. Form alizing Incomplete Knowledge in Incomplete Databases[J]. , 1992, 7(4): 295 -304 .
[4] Ying Mingsheng;. Putting Consistent Theories Together in Institutions[J]. , 1995, 10(3): 260 -266 .
[5] Shen Li;. Fuzzy Logic Control ASIC Chip[J]. , 1997, 12(3): 263 -270 .
[6] GU Jing; SHUAI Dianxun;. A New Parallel-by-Cell Approach to Undistorted DataCompression Based on Cellular Automatonand Genetic Algorithm[J]. , 1999, 14(6): 572 -579 .
[7] Ju-Hum Kwon, Chee-Yang Song, Chang-Joo Moon, and Doo-Kwon Baik. Bridging Real World Semantics to Model World Semantics for Taxonomy Based Knowledge Representation System[J]. , 2005, 20(3): 296 -308 .
[8] Shu-Tao Xia. A Note on the Stopping Redundancy of Linear Codes[J]. , 2006, 21(6): 950 -951 .
[9] Xi-Shun Zhao and Yu-Ping Shen. Comparison of Semantics of Disjunctive Logic Programs Based on Model-Equivalent Reduction[J]. , 2007, 22(4): 562 -568 .
[10] Moonki Jung, Hyundeok Cho, Taehwan Roh, and Kunwoo Lee. Integrated Framework for Vehicle Interior Design Using Digital Human Model[J]. , 2009, 24(6): 1149 -1161 .

ISSN 1000-9000(Print)

         1860-4749(Online)
CN 11-2296/TP

Home
Editorial Board
Author Guidelines
Subscription
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
Tel.:86-10-62610746
E-mail: jcst@ict.ac.cn
 
  Copyright ©2015 JCST, All Rights Reserved