.tabbox {width:400px; margin-top: 15px;margin-bottom: 5px} .tabmenu {width:400px;height:28px;border-left:1px solid #CCC;border-top:1px solid #ccc;} .tabmenu ul {margin:0;padding:0;list-style-type: none;} .tabmenu li { text-align:center; float:left; display:block; width:99px; overflow:hidden; background-color: #f1f1f1; line-height:27px; border-right:#ccc 1px solid; border-bottom:#ccc 1px solid; display:inline;} .tabmenu .cli {text-align:center;float:left;display:block;width:99px;overflow:hidden;background-color: #fff;line-height:27px;border-right:#ccc 1px solid;border-bottom:#fff 1px solid;display:inline; cursor:pointer; color: #810505; font-weight:bold} #tabcontent {width:399px;background-color:#fff;border-left:#CCC 1px solid;border-right:#CCC 1px solid;border-bottom:#CCC 1px solid; height:60px;} #tabcontent ul {margin:0;padding:5px;list-style-type: none;} #tabcontent .hidden {display:none;} Search Browse by Issue Fig/Tab Adv Search
 HOME ABOUT JCST AUTHORS REVIEWERS PUBLISHED PAPERS FORTHCOMING PAPERS

### Multimodal Dependence Attention and Large-Scale Data based Offline Handwritten Formula Recognition

Han-Chao Liu (刘汉超), Lan-Fang Dong (董兰芳), Member, CCF, and Xin-Ming Zhang (张信明), Senior Member, IEEE, CCF

1. School of Computer Science and Technology, University of Science and Technology of China, Hefei 230022, China
• Contact: Xin-Ming Zhang E-mail:xinming@ustc.edu.cn
• About author:
Xin-Ming Zhang received his B.E. and M.E. degrees in electrical engineering from China University of Mining and Technology, Xuzhou, in 1985 and 1988, respectively, and his Ph.D degree in computer science and technology from the University of Science and Technology of China, Hefei, in 2001. Since 2002, he has been with the faculty of the University of Science and Technology of China, where he is currently a professor with the School of Computer Science and Technology. From September 2005 to August 2006, he was a visiting professor with the Department of Electrical Engineering and Computer Science, Korea Advanced Institute of Science and Technology, Daejeon, Korea. His research interest includes wireless networks, big data, smart grid. He has published more than 100 papers. He won the second prize of Science and Technology Award of Anhui Province of China in Natural Sciences in 2017. He is a senior member of CCF and IEEE.

Offline handwritten formula recognition is a challenging task due to the variety of handwritten symbols and two-dimensional formula structures. Recently, the deep neural network recognizers that are based on the encoder-decoder framework achieve great improvements on this task. However, the unsatisfactory recognition performance for formulas with long LaTeX strings is one shortcoming of the existing work. Moreover, lacking sufficient training data also limits the capability of these recognizers. In this paper, we design a multimodal dependence attention (MDA) module to help the model learn visual and semantic dependencies among symbols in the same formula to improve the recognition performance of the formulas with long LaTeX strings. To alleviate overfitting and further improve the recognition performance, we also propose a new dataset, Handwritten Formula Image Dataset (HFID), which contains 25620 handwritten formula images collected from real life. We conducted extensive experiments to demonstrate the effectiveness of our proposed MDA module and HFID dataset and achieved state-of-the-art performances, 63.79% and 65.24% expression accuracy on CROHME 2014 and 2016, respectively.

1、 研究背景（context）

2、 目的（Objective）

3、 方法（Method）

4、 结果（Result & Findings）

5、 结论（Conclusions）

 [1] Inès Mouakher, Fatma Dhaou, and J. Christian Attiogbé. Event-Based Semantics of UML 2.X Concurrent Sequence Diagrams for Formal Verification [J]. Journal of Computer Science and Technology, 2022, 37(1): 4-28. [2] Li-Li Xiao, Hui-Biao Zhu, Qi-Wen Xu. Trace Semantics and Algebraic Laws for Total Store Order Memory Model [J]. Journal of Computer Science and Technology, 2021, 36(6): 1269-1290. [3] Xiao-Li Ren, Kai-Jun Ren, Zi-Chen Xu, Xiao-Yong Li, Ao-Long Zhou, Jun-Qiang Song, Ke-Feng Deng. Improving Ocean Data Services with Semantics and Quick Index [J]. Journal of Computer Science and Technology, 2021, 36(5): 963-984. [4] Jia-Ke Ge, Yan-Feng Chai, Yun-Peng Chai. WATuning: A Workload-Aware Tuning System with Attention-Based Deep Reinforcement Learning [J]. Journal of Computer Science and Technology, 2021, 36(4): 741-761. [5] Chen-Chen Sun, De-Rong Shen. Mixed Hierarchical Networks for Deep Entity Matching [J]. Journal of Computer Science and Technology, 2021, 36(4): 822-838. [6] Yang Liu, Ruili He, Xiaoqian Lv, Wei Wang, Xin Sun, Shengping Zhang. Is It Easy to Recognize Baby's Age and Gender? [J]. Journal of Computer Science and Technology, 2021, 36(3): 508-519. [7] Hui-Xuan Wang, Jing-Liang Peng, Shi-Yi Lu, Xin Cao, Xue-Ying Qin, Chang-He Tu. ReLoc: Indoor Visual Localization with Hierarchical Sitemap and View Synthesis [J]. Journal of Computer Science and Technology, 2021, 36(3): 494-507. [8] Sheng-Luan Hou, Xi-Kun Huang, Chao-Qun Fei, Shu-Han Zhang, Yang-Yang Li, Qi-Lin Sun, Chuan-Qing Wang. A Survey of Text Summarization Approaches Based on Deep Learning [J]. Journal of Computer Science and Technology, 2021, 36(3): 633-663. [9] Hua Chen, Juan Liu, Qing-Man Wen, Zhi-Qun Zuo, Jia-Sheng Liu, Jing Feng, Bao-Chuan Pang, Di Xiao. CytoBrain: Cervical Cancer Screening System Based on Deep Learning Technology [J]. Journal of Computer Science and Technology, 2021, 36(2): 347-360. [10] Xia-An Bi, Zhao-Xu Xing, Rui-Hui Xu, Xi Hu. An Efficient WRF Framework for Discovering Risk Genes and Abnormal Brain Regions in Parkinson's Disease Based on Imaging Genetics Data [J]. Journal of Computer Science and Technology, 2021, 36(2): 361-374. [11] Yuan Huang, Nan Jia, Hao-Jie Zhou, Xiang-Ping Chen, Zi-Bin Zheng, Ming-Dong Tang. Learning Human-Written Commit Messages to Document Code Changes [J]. Journal of Computer Science and Technology, 2020, 35(6): 1258-1277. [12] Yi-Ting Wang, Jie Shen, Zhi-Xu Li, Qiang Yang, An Liu, Peng-Peng Zhao, Jia-Jie Xu, Lei Zhao, Xun-Jie Yang. Enriching Context Information for Entity Linking with Web Data [J]. Journal of Computer Science and Technology, 2020, 35(4): 724-738. [13] Ying Li, Jia-Jie Xu, Peng-Peng Zhao, Jun-Hua Fang, Wei Chen, Lei Zhao. ATLRec: An Attentional Adversarial Transfer Learning Network for Cross-Domain Recommendation [J]. Journal of Computer Science and Technology, 2020, 35(4): 794-808. [14] Huan-Jing Yue, Sheng Shen, Jing-Yu Yang, Hao-Feng Hu, Yan-Fang Chen. Reference Image Guided Super-Resolution via Progressive Channel Attention Networks [J]. Journal of Computer Science and Technology, 2020, 35(3): 551-563. [15] Dun Liang, Yuan-Chen Guo, Shao-Kui Zhang, Tai-Jiang Mu, Xiaolei Huang. Lane Detection: A Survey with New Results [J]. Journal of Computer Science and Technology, 2020, 35(3): 493-505.
Viewed
Full text

Abstract

Cited

Shared
Discussed