计算机科学技术学报 ›› 2019,Vol. 34 ›› Issue (1): 155-169.doi: 10.1007/s11390-019-1904-1

所属专题： Artificial Intelligence and Pattern Recognition； Data Management and Data Mining

• Data Management and Data Mining • 上一篇    下一篇

基于概率图模型自动生成学术海报

Yu-Ting Qiang1, Yan-Wei Fu2, Member, ACM, IEEE, Xiao Yu1, Yan-Wen Guo1,*, Member, IEEE Zhi-Hua Zhou1, Fellow, CCF, ACM, IEEE, and Leonid Sigal3, Member, ACM, IEEE   

  1. 1 National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210046, China;
    2 School of Data Science, Fudan University, Shanghai 200433, China;
    3 Disney Research Pittsburgh, Pittsburgh 15241, U.S.A.
  • 收稿日期:2018-01-14 修回日期:2018-11-12 出版日期:2019-01-05 发布日期:2019-01-12
  • 通讯作者: Yan-Wen Guo E-mail:ywguo.nju@gmail.com
  • 作者简介:Yu-Ting Qiang is a Ph.D. student at the National Key Laboratory for Novel Software Technology, Department of Computer Science and Technology, Nanjing University, Nanjing. She received her B.S. degree in software engineering from Jilin University, Changchun, in 2013. Her research interests include computer vision and machine learning.
  • 基金资助:
    This work was supported by the Natural Science Foundation of Jiangsu Province of China under Grant No. BK20150016, the National Natural Science Foundation of China under Grant Nos. 61772257 and 61672279, and the Fundamental Research Funds for the Central Universities of China under Grant No. 020214380042.

Learning to Generate Posters of Scientific Papers by Probabilistic Graphical Models

Yu-Ting Qiang1, Yan-Wei Fu2, Member, ACM, IEEE, Xiao Yu1, Yan-Wen Guo1,*, Member, IEEE Zhi-Hua Zhou1, Fellow, CCF, ACM, IEEE, and Leonid Sigal3, Member, ACM, IEEE   

  1. 1 National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210046, China;
    2 School of Data Science, Fudan University, Shanghai 200433, China;
    3 Disney Research Pittsburgh, Pittsburgh 15241, U.S.A.
  • Received:2018-01-14 Revised:2018-11-12 Online:2019-01-05 Published:2019-01-12
  • Contact: Yan-Wen Guo E-mail:ywguo.nju@gmail.com
  • About author:Yu-Ting Qiang is a Ph.D. student at the National Key Laboratory for Novel Software Technology, Department of Computer Science and Technology, Nanjing University, Nanjing. She received her B.S. degree in software engineering from Jilin University, Changchun, in 2013. Her research interests include computer vision and machine learning.
  • Supported by:
    This work was supported by the Natural Science Foundation of Jiangsu Province of China under Grant No. BK20150016, the National Natural Science Foundation of China under Grant Nos. 61772257 and 61672279, and the Fundamental Research Funds for the Central Universities of China under Grant No. 020214380042.

PDF (PC)

369

补充材料

1

学术工作者通常利用学术海报来总结展示他们的学术论文的内容，这是因为学术海报能够简洁流畅的表达学术论文的中心思想。然而，学术海报不仅要求良好的可读性和较强的信息表达力，同时还应当是比较美观的，这就使得设计制作学术海报这一工作复杂且耗时。本文中，我们第一次研究利用概率图模型来自动生成学术海报这一挑战性的工作。具体来说，给定需要展示的内容，我们的方案能够自动地从已有数据中学习推断出海报各个面板的大小、形状，能够推断面板的排版以及各个面板内的内容布置。在我们的方案中，最大化后验概率（Maximum a posterior）估计方法被用来引入并建模一些已有的设计原则；而为了串联起面板内部元素排版和整个海报的面板排版，我们提出了一种递归划分海报页面的方案。为了学习以及验证我们的模型，我们收集并公开了NJU-Fudan论文海报数据集，这一数据集包含了详细标注的学术论文海报对。质化和量化的实验结果说明了我们方法的有效性。

关键词: 图形设计, 自动排版, 概率图模型

Abstract: Researchers often summarize their work in the form of scientific posters. Posters provide a coherent and efficient way to convey core ideas expressed in scientific papers. Generating a good scientific poster, however, is a complex and time-consuming cognitive task, since such posters need to be readable, informative, and visually aesthetic. In this paper, for the first time, we study the challenging problem of learning to generate posters from scientific papers. To this end, a data-driven framework, which utilizes graphical models, is proposed. Specifically, given content to display, the key elements of a good poster, including attributes of each panel and arrangements of graphical elements, are learned and inferred from data. During the inference stage, the maximum a posterior (MAP) estimation framework is employed to incorporate some design principles. In order to bridge the gap between panel attributes and the composition within each panel, we also propose a recursive page splitting algorithm to generate the panel layout for a poster. To learn and validate our model, we collect and release a new benchmark dataset, called NJU-Fudan Paper-Poster dataset, which consists of scientific papers and corresponding posters with exhaustively labelled panels and attributes. Qualitative and quantitative results indicate the effectiveness of our approach.

Key words: graphical design, layout automation, probabilistic graphical model

[1] Jahanian A, Liu J, Tretter D R, Lin Q, Damera-Venkata N, O'Brien-Strain E, Lee S, Fan J, Allebach J P. Automatic design of magazine covers. In Proc. IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics, January 2012, Article ID. 83020.
[2] Hunter A, Slatter D, Greig D. Web-based magazine design for self publishers. In Proc. IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics, January 2011, Article ID. 787902.
[3] O'Donovan P, Agarwala A, Hertzmann A. Learning layouts for single-page graphic designs. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(8):1200-1213.
[4] Geigel J, Loui A. Using genetic algorithms for album page layouts. IEEE Multimedia, 2003, 10(4):16-27.
[5] Yu L F, Yeung S K, Tang C K, Terzopoulos D, Chan T F, Osher S J. Make it home:Automatic optimization of furniture arrangement. ACM Transactions on Graphics, 2011, 30(4):Article No. 86.
[6] Merrell P, Schkufza E, Li Z, Agrawala M, Koltun V. Interactive furniture layout using interior design guidelines. ACM Transactions on Graphics, 2011, 30(4):Article No. 87.
[7] Cao Y, Lau R W, Chan A B. Look over here:Attentiondirecting composition of manga elements. ACM Transactions on Graphics, 2014, 33(4):Article No. 94.
[8] Hurst N, Li W, Marriott K. Review of automatic document formatting. In Proc. the 9th ACM Symposium on Document Engineering, September 2009, pp.99-108.
[9] Knuth D E, Plass M F. Breaking paragraphs into lines. Software:Practice and Experience, 1981, 11(11):1119-1184.
[10] Peels A J H, Janssen N J M, Nawijn W. Document architecture and text formatting. ACM Transactions on Information Systems, 1985, 3(4):347-369.
[11] Damera-Venkata N, Bento J, O'Brien-Strain E. Probabilistic document model for automated document composition. In Proc. the 11th ACM Symposium on Document Engineering, September 2011, pp.3-12.
[12] Mihalcea R, Tarau P. TextRank:Bringing order into text. In Proc. the 2004 Conference on Empirical Methods in Natural Language Processing, July 2004, pp.404-411.
[13] Qiang Y T, Fu Y W, Zhou Y W, Zhou Z H, Sigal L. Learning to generate posters of scientific papers. In Proc. the 30th AAAI Conference on Artificial Intelligence, February 2016, pp.51-57.
[14] Holland J H. Adaptation in Natural and Artificial Systems:An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. A Bradford Book, 1992.
[15] Goldberg D E. Genetic Algorithms in Search, Optimization and Machine Learning (1st edition). Addison Wesley, 1989.
[16] Gajos K, Weld D S. Preference elicitation for interface optimization. In Proc. the 18th Annual ACM Symposium on User Interface Software and Technology, October 2005, pp.173-182.
[17] Sarrafzadeh M, Lee D T. Algorithmic Aspects of VLSI Layout. World Scientific Pub. Co. Inc., 1993.
[18] Battista G D, Eades P, Tamassia R, Tollis I G. Graph Drawing:Algorithms for the Visualization of Graphs (1st edition). Pearson, 1998.
[19] Matsui Y, Yamasaki T, Aizawa K. Interactive manga retargeting. In Proc. ACM SIGGRAPH 2011 Posters, August 2011, Article No. 35.
[20] Hoashi K, Ono C, Ishii D, Watanabe H. Automatic preview generation of comic episodes for digitized comic search. In Proc. the 19th ACM International Conference on Multimedia, November 2011, pp.1489-1492.
[21] Qu Y, Pang W M, Wong T T, Heng P N. Richnesspreserving manga screening. ACM Transactions on Graphics, 2008, 27(5):Article No. 155.
[22] Arai K, Herman T. Method for automatic e-comic scene frame extraction for reading comic on mobile devices. In Proc. the 7th International Conference on Information Technology:New Generations, April 2010, pp.370-375.
[23] Pang X, Cao Y, Lau R W H, Chan A B. A robust panel extraction method for manga. In Proc. the 22nd ACM International Conference on Multimedia, November 2014, pp.1125-1128.
[24] Jing G, Hu Y, Guo Y, Yu Y, Wang W. Content-aware video2comics with manga-style layout. IEEE Transactions on Multimedia, 2015, 17(12):2122-2133.
[25] Cao Y, Chan A B, Lau R W H. Automatic stylistic manga layout. ACM Transactions on Graphics, 2012, 31(6):Article No. 141.
[26] Lodi A, Martello S, Monaci M. Two-dimensional packing problems:A survey. European Journal of Operational Research, 2002, 141(2):241-252.
[27] Stockmeyer L. Optimal orientations of cells in slicing floorplan designs. Information and Control, 1983, 57(2/3):91-101.
[28] Kaser O, Lemire D. Tag-cloud drawing:Algorithms for cloud visualization. arXiv:0703109, 2007. https://arxiv.org/abs/cs/0703109, August 2018.
[29] Lok S, Feiner S. A survey of automated layout techniques for information presentations. In Proc. the 1st International Symposium on Smart Graphics, March 2001, pp.61-68.
[30] Jacobs C, Li W, Schrier E, Bargeron D, Salesin D. Adaptive grid-based document layout. ACM Transactions on Graphics, 2003, 22(3):838-847.
[31] Harrington S J, Naveda J F, Jones R P, Roetling P, Thakkar N. Aesthetic measures for automated document layout. In Proc. the 2004 ACM Symposium on Document Engineering, October 2004, pp.109-111.
[32] Purvis L, Harrington S, O'Sullivan B, Freuder E C. Creating personalized documents:An optimization approach. In Proc. the 2003 ACM Symposium on Document Engineering, Nov. 2003, pp.68-77.
[33] Pinto A, Pedrini H, Schwartz W R, Rocha A. Face spoofing detection through visual codebooks of spectral temporal cubes. IEEE Transactions on Image Processing, 2015, 24(12):4726-4740.
[34] Murphy K. The bayes net toolbox for Matlab. Technical Report, Computing Science and Statistics, 2001, http://people.cs.ubc.ca/~/Papers/bnt.pdf, Nov. 2018.
[35] Fung R, Chang K C. Weighing and integrating evidence for stochastic simulation in Bayesian networks. Machine Intelligence and Pattern Recognition, 1990, 10:209-220.
[36] Zhao Y, Zhu S C. Image parsing with stochastic scene grammar. In Proc. the 25th International Conference on Neural Information Processing Systems, December 2011, pp.73-81.
[37] Chen Z, Mukherjee A, Liu B, Hsu M, Castellanos M, Ghosh R. Leveraging multi-domain prior knowledge in topic models. In Proc. the 23rd International Joint Conference on Artificial Intelligence, August 2013, pp.2071-2077.
[1] Cun-Chao Tu, Zhi-Yuan Liu, Mao-Song Sun. 用于用户标签推荐的标签关联模型[J]. , 2015, 30(5): 1063-1072.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] 刘明业; 洪恩宇;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] 陈世华;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] 高庆狮; 张祥; 杨树范; 陈树清;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] 陈肇雄; 高庆狮;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] 黄河燕;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] 闵应骅; 韩智德;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] 唐同诰; 招兆铿;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] 闵应骅;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] 朱鸿;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .
[10] 李明慧;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持：support@magtech.com.cn
总访问量：