We use cookies to improve your experience with our site.

Indexed in:

SCIE, EI, Scopus, INSPEC, DBLP, CSCD, etc.

Submission System
(Author / Reviewer / Editor)
Advanced Search
Volume 34 Issue 1
January  2019
Turn off MathJax
Article Contents
Abstract
References
Supplements
Yu-Ting Qiang, Yan-Wei Fu, Xiao Yu, Yan-Wen Guo, Zhi-Hua Zhou, Leonid Sigal. Learning to Generate Posters of Scientific Papers by Probabilistic Graphical Models[J]. Journal of Computer Science and Technology, 2019, 34(1): 155-169. DOI: 10.1007/s11390-019-1904-1
Citation: Yu-Ting Qiang, Yan-Wei Fu, Xiao Yu, Yan-Wen Guo, Zhi-Hua Zhou, Leonid Sigal. Learning to Generate Posters of Scientific Papers by Probabilistic Graphical Models[J]. Journal of Computer Science and Technology, 2019, 34(1): 155-169. DOI: 10.1007/s11390-019-1904-1
shu

Learning to Generate Posters of Scientific Papers by Probabilistic Graphical Models

  • 1 National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210046, China;
    2 School of Data Science, Fudan University, Shanghai 200433, China;
    3 Disney Research Pittsburgh, Pittsburgh 15241, U.S.A.

Funds: This work was supported by the Natural Science Foundation of Jiangsu Province of China under Grant No. BK20150016, the National Natural Science Foundation of China under Grant Nos. 61772257 and 61672279, and the Fundamental Research Funds for the Central Universities of China under Grant No. 020214380042.
More Information
  • Author Bio:

    Yu-Ting Qiang is a Ph.D. student at the National Key Laboratory for Novel Software Technology, Department of Computer Science and Technology, Nanjing University, Nanjing. She received her B.S. degree in software engineering from Jilin University, Changchun, in 2013. Her research interests include computer vision and machine learning.

  • Corresponding author:

    Yan-Wen Guo E-mail: ywguo.nju@gmail.com

  • Received Date: January 13, 2018
  • Revised Date: November 11, 2018
  • Published Date: January 04, 2019
    Abstract
  • Researchers often summarize their work in the form of scientific posters. Posters provide a coherent and efficient way to convey core ideas expressed in scientific papers. Generating a good scientific poster, however, is a complex and time-consuming cognitive task, since such posters need to be readable, informative, and visually aesthetic. In this paper, for the first time, we study the challenging problem of learning to generate posters from scientific papers. To this end, a data-driven framework, which utilizes graphical models, is proposed. Specifically, given content to display, the key elements of a good poster, including attributes of each panel and arrangements of graphical elements, are learned and inferred from data. During the inference stage, the maximum a posterior (MAP) estimation framework is employed to incorporate some design principles. In order to bridge the gap between panel attributes and the composition within each panel, we also propose a recursive page splitting algorithm to generate the panel layout for a poster. To learn and validate our model, we collect and release a new benchmark dataset, called NJU-Fudan Paper-Poster dataset, which consists of scientific papers and corresponding posters with exhaustively labelled panels and attributes. Qualitative and quantitative results indicate the effectiveness of our approach.
    • FullText(HTML)
    • References (37)
  • [1]
    Jahanian A, Liu J, Tretter D R, Lin Q, Damera-Venkata N, O'Brien-Strain E, Lee S, Fan J, Allebach J P. Automatic design of magazine covers. In Proc. IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics, January 2012, Article ID. 83020.
    [2]
    Hunter A, Slatter D, Greig D. Web-based magazine design for self publishers. In Proc. IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics, January 2011, Article ID. 787902.
    [3]
    O'Donovan P, Agarwala A, Hertzmann A. Learning layouts for single-page graphic designs. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(8):1200-1213.
    [4]
    Geigel J, Loui A. Using genetic algorithms for album page layouts. IEEE Multimedia, 2003, 10(4):16-27.
    [5]
    Yu L F, Yeung S K, Tang C K, Terzopoulos D, Chan T F, Osher S J. Make it home:Automatic optimization of furniture arrangement. ACM Transactions on Graphics, 2011, 30(4):Article No. 86.
    [6]
    Merrell P, Schkufza E, Li Z, Agrawala M, Koltun V. Interactive furniture layout using interior design guidelines. ACM Transactions on Graphics, 2011, 30(4):Article No. 87.
    [7]
    Cao Y, Lau R W, Chan A B. Look over here:Attentiondirecting composition of manga elements. ACM Transactions on Graphics, 2014, 33(4):Article No. 94.
    [8]
    Hurst N, Li W, Marriott K. Review of automatic document formatting. In Proc. the 9th ACM Symposium on Document Engineering, September 2009, pp.99-108.
    [9]
    Knuth D E, Plass M F. Breaking paragraphs into lines. Software:Practice and Experience, 1981, 11(11):1119-1184.
    [10]
    Peels A J H, Janssen N J M, Nawijn W. Document architecture and text formatting. ACM Transactions on Information Systems, 1985, 3(4):347-369.
    [11]
    Damera-Venkata N, Bento J, O'Brien-Strain E. Probabilistic document model for automated document composition. In Proc. the 11th ACM Symposium on Document Engineering, September 2011, pp.3-12.
    [12]
    Mihalcea R, Tarau P. TextRank:Bringing order into text. In Proc. the 2004 Conference on Empirical Methods in Natural Language Processing, July 2004, pp.404-411.
    [13]
    Qiang Y T, Fu Y W, Zhou Y W, Zhou Z H, Sigal L. Learning to generate posters of scientific papers. In Proc. the 30th AAAI Conference on Artificial Intelligence, February 2016, pp.51-57.
    [14]
    Holland J H. Adaptation in Natural and Artificial Systems:An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. A Bradford Book, 1992.
    [15]
    Goldberg D E. Genetic Algorithms in Search, Optimization and Machine Learning (1st edition). Addison Wesley, 1989.
    [16]
    Gajos K, Weld D S. Preference elicitation for interface optimization. In Proc. the 18th Annual ACM Symposium on User Interface Software and Technology, October 2005, pp.173-182.
    [17]
    Sarrafzadeh M, Lee D T. Algorithmic Aspects of VLSI Layout. World Scientific Pub. Co. Inc., 1993.
    [18]
    Battista G D, Eades P, Tamassia R, Tollis I G. Graph Drawing:Algorithms for the Visualization of Graphs (1st edition). Pearson, 1998.
    [19]
    Matsui Y, Yamasaki T, Aizawa K. Interactive manga retargeting. In Proc. ACM SIGGRAPH 2011 Posters, August 2011, Article No. 35.
    [20]
    Hoashi K, Ono C, Ishii D, Watanabe H. Automatic preview generation of comic episodes for digitized comic search. In Proc. the 19th ACM International Conference on Multimedia, November 2011, pp.1489-1492.
    [21]
    Qu Y, Pang W M, Wong T T, Heng P N. Richnesspreserving manga screening. ACM Transactions on Graphics, 2008, 27(5):Article No. 155.
    [22]
    Arai K, Herman T. Method for automatic e-comic scene frame extraction for reading comic on mobile devices. In Proc. the 7th International Conference on Information Technology:New Generations, April 2010, pp.370-375.
    [23]
    Pang X, Cao Y, Lau R W H, Chan A B. A robust panel extraction method for manga. In Proc. the 22nd ACM International Conference on Multimedia, November 2014, pp.1125-1128.
    [24]
    Jing G, Hu Y, Guo Y, Yu Y, Wang W. Content-aware video2comics with manga-style layout. IEEE Transactions on Multimedia, 2015, 17(12):2122-2133.
    [25]
    Cao Y, Chan A B, Lau R W H. Automatic stylistic manga layout. ACM Transactions on Graphics, 2012, 31(6):Article No. 141.
    [26]
    Lodi A, Martello S, Monaci M. Two-dimensional packing problems:A survey. European Journal of Operational Research, 2002, 141(2):241-252.
    [27]
    Stockmeyer L. Optimal orientations of cells in slicing floorplan designs. Information and Control, 1983, 57(2/3):91-101.
    [28]
    Kaser O, Lemire D. Tag-cloud drawing:Algorithms for cloud visualization. arXiv:0703109, 2007. https://arxiv.org/abs/cs/0703109, August 2018.
    [29]
    Lok S, Feiner S. A survey of automated layout techniques for information presentations. In Proc. the 1st International Symposium on Smart Graphics, March 2001, pp.61-68.
    [30]
    Jacobs C, Li W, Schrier E, Bargeron D, Salesin D. Adaptive grid-based document layout. ACM Transactions on Graphics, 2003, 22(3):838-847.
    [31]
    Harrington S J, Naveda J F, Jones R P, Roetling P, Thakkar N. Aesthetic measures for automated document layout. In Proc. the 2004 ACM Symposium on Document Engineering, October 2004, pp.109-111.
    [32]
    Purvis L, Harrington S, O'Sullivan B, Freuder E C. Creating personalized documents:An optimization approach. In Proc. the 2003 ACM Symposium on Document Engineering, Nov. 2003, pp.68-77.
    [33]
    Pinto A, Pedrini H, Schwartz W R, Rocha A. Face spoofing detection through visual codebooks of spectral temporal cubes. IEEE Transactions on Image Processing, 2015, 24(12):4726-4740.
    [34]
    Murphy K. The bayes net toolbox for Matlab. Technical Report, Computing Science and Statistics, 2001, http://people.cs.ubc.ca/~/Papers/bnt.pdf, Nov. 2018.
    [35]
    Fung R, Chang K C. Weighing and integrating evidence for stochastic simulation in Bayesian networks. Machine Intelligence and Pattern Recognition, 1990, 10:209-220.
    [36]
    Zhao Y, Zhu S C. Image parsing with stochastic scene grammar. In Proc. the 25th International Conference on Neural Information Processing Systems, December 2011, pp.73-81.
    [37]
    Chen Z, Mukherjee A, Liu B, Hsu M, Castellanos M, Ghosh R. Leveraging multi-domain prior knowledge in topic models. In Proc. the 23rd International Joint Conference on Artificial Intelligence, August 2013, pp.2071-2077.
    • Relative Articles
  • Supplements (1)

  • Others

  • Cited by

    Periodical cited type(14)

    1. Xinyi Zhong, Zusheng Tan, Jing Li, et al. Scientific poster generation: A new dataset and approach. Pattern Recognition, 2025. DOI:10.1016/j.patcog.2025.111507
    2. Chenchen Xu, Kaixin Han, Weiwei Xu. Image-aware layout generation with user constraints for poster design. The Visual Computer, 2024. DOI:10.1007/s00371-024-03657-z
    3. Mohan Zhang, Fang Liu, Biyao Li, et al. CrePoster: Leveraging multi-level features for cultural relic poster generation via attention-based framework. Expert Systems with Applications, 2024, 245: 123136. DOI:10.1016/j.eswa.2024.123136
    4. Yong Shi, Mengyu Shang, Zhiquan Qi. Intelligent layout generation based on deep generative models: A comprehensive survey. Information Fusion, 2023, 100: 101940. DOI:10.1016/j.inffus.2023.101940
    5. Jia-Yu Liu, Fei Wang, Hai-Ping Ma, et al. A Probabilistic Framework for Temporal Cognitive Diagnosis in Online Learning Systems. Journal of Computer Science and Technology, 2023, 38(6): 1203. DOI:10.1007/s11390-022-1332-5
    6. Dongjie Wang, Yanjie Fu, Kunpeng Liu, et al. Automated Urban Planning for Reimagining City Configuration via Adversarial Learning: Quantification, Generation, and Evaluation. ACM Transactions on Spatial Algorithms and Systems, 2023, 9(1): 1. DOI:10.1145/3524302
    7. Guandong Li, Xian Yang. Smartbanner: intelligent banner design framework that strikes a balance between creative freedom and design rules. Multimedia Tools and Applications, 2023, 82(12): 18653. DOI:10.1007/s11042-022-14138-7
    8. Hao Hu, Chao Zhang, Yanxue Liang. A study on the automatic generation of banner layouts. Computers & Electrical Engineering, 2021, 93: 107269. DOI:10.1016/j.compeleceng.2021.107269
    9. K. Kikuchi, M. Otani, K. Yamaguchi, et al. Modeling Visual Containment for Web Page Layout Optimization. Computer Graphics Forum, 2021, 40(7): 33. DOI:10.1111/cgf.14399
    10. Wei-tao You, Hao Jiang, Zhi-yuan Yang, et al. Automatic synthesis of advertising images according to a specified style. Frontiers of Information Technology & Electronic Engineering, 2020, 21(10): 1455. DOI:10.1631/FITEE.1900367
    11. Shunan Guo, Zhuochen Jin, Fuling Sun, et al. Vinci: An Intelligent Graphic Design System for Generating Advertising Posters. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, DOI:10.1145/3411764.3445117
    12. Xin He, Jiuyang Tang, Zhen Tan, et al. Web and Big Data. Lecture Notes in Computer Science, DOI:10.1007/978-3-030-60290-1_23
    13. Nikita A. Kharitonov, Anatolii G. Maksimov, Alexander L. Tulupyev. Proceedings of the Fourth International Scientific Conference “Intelligent Information Technologies for Industry” (IITI’19). Advances in Intelligent Systems and Computing, DOI:10.1007/978-3-030-50097-9_22
    14. Dongjie Wang, Yanjie Fu, Pengyang Wang, et al. Reimagining City Configuration. Proceedings of the 28th International Conference on Advances in Geographic Information Systems, DOI:10.1145/3397536.3422268

    Other cited types(0)

Catalog

    Get Citation
    PDF
    XML
    Article views (45) PDF downloads (338) Cited by(14)
    Related

    Copyright ©2025 JCST, All Rights Reserved     京ICP备05002829号-1

    Institute of Computing Technology, Chinese Academy of Sciences
    P.O. Box 2704, Beijing 100190, P.R. China

    Tel.: 86-10-62610746 E-mail: jcst@ict.ac.cn

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return