›› 2011, Vol. 26 ›› Issue (2): 247-255.doi: 10.1007/s11390-011-1127-6

Special Issue: Artificial Intelligence and Pattern Recognition; Software Systems

• Artificial Intelligence • Previous Articles     Next Articles

Software Agent with Reinforcement Learning Approach for Medical Image Segmentation

Mahsa Chitsaz, and Chaw Seng Woo, Member, IEEE   

  1. Faculty of Computer Science and Information Technology, University of Malaya, 50603 Kuala Lumpur, Malaysia
  • Received:2009-12-03 Revised:2010-12-04 Online:2011-03-05 Published:2011-03-05
  • About author:Mahsa Chitsaz received her B.S. degree in compt. eng. from Shiraz University in 2006. She then obtained her M.Sc. degree from University of Malaya in 2010 under the guidance of Chaw Seng Woo in the area of reinforcement learning in medical image segmentation. Chitsaz’s research interest is in artificial intelligence in real-time systems, mainly in the areas of machine learning and dynamic processes. She also works at the intersection of learning and topics as varied as medical image segmentation, telemedicine, and head-mounted display.
    Chaw Seng Woo is a senior lecturer at the Faculty of Computer Science and Information Technology, University of Malaya. His research interests include image processing and mobile applications.
  • Supported by:

    This research was funded by Peruntukan Penyelidikan Pascasiswazah (PPP) under Grant No. PS349/2008C.

Many image segmentation solutions are problem-based. Medical images have very similar grey level and texture among the interested objects. Therefore, medical image segmentation requires improvements although there have been researches done since the last few decades. We design a self-learning framework to extract several objects of interest simultaneously from Computed Tomography (CT) images. Our segmentation method has a learning phase that is based on reinforcement learning (RL) system. Each RL agent works on a particular sub-image of an input image to find a suitable value for each object in it. The RL system is define by state, action and reward. We defined some actions for each state in the sub-image. A reward function computes reward for each action of the RL agent. Finally, the valuable information, from discovering all states of the interest objects, will be stored in a Q-matrix and the final result can be applied in segmentation of similar images. The experimental results for cranial CT images demonstrated segmentation accuracy above 95%.

[1] Pham D L, Xu C, Prince J L. A survey of current methods in medical image segmentation. Annual Review of Biomedical Engineering, 2000, 2: 315-337.

[2] Jain A K. Fundamentals of Digital Image Processing. Printice Hall, 1989.

[3] Chen P, Pavlidis T. Image segmentation as an estimation problem. Computer Graphics and Image Processing, 1980, 12(20): 153-172.

[4] Liu J. Synergistic hybrid image segmentation: Combining model and image-based Sstartegies

[Ph.D. Dissertation]. Univ. Pennsylvania, 2006.

[5] Kaelbling L P, Littman M L, Moore A W. Reinforcement learning: A survey. Artificial Intelligence Research, 1996, 4: 237-285.

[6] Chitsaz M, Woo C S. Medical image segmentation by using reinforcement learning agent. In Proc. International Conference on Digital Image Processing (ICDIP 2009), Bangkok, Thailand, Mar. 7-10, 2009, pp.216-219.

[7] Watkins C J C H. Learning from delayed rewards

[Ph.D. Dissertation]. Cambridge, 1989.

[8] Watkins C, Dayan P. Q-learning. Machine Learning, 1992, 8(3): 279-292.

[9] Chitsaz M, Woo C S. The rise of multi-agent and R.L. segmentation methods for biomedical images. In Proc. The 4th Malaysian Software Engineering Conference (MySEC2008), Kuala Terengganu, Malaysia, 2008, p.5.

[10] Bhanu B, Peng J. Adaptive integrated image segmentation and object recognition. IEEE Transaction on Systems, Man, and Cybernetics, 2000, 30(4): 427-441.

[11] Peng J, Bhanu B. Delayed reinforcement learning for adaptive image segmentation and feature extraction. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 1998, 28(3): 482-488.

[12] Peng J, Bhanu B. Closed-loop object recognition using reinforcement learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(2): 139-154.

[13] Shokri M, Tizhoosh H R. Using reinforcement learning for image thresholding. In Proc. IEEE Canadian Conference on Electrical and Computer Engineering, May 4-7, 2003, pp.1231-1234.

[14] Shokri M, Tizhoosh H R. A reinforcement agent for threshold fusion. Applied Soft Computing, 2008, 8(1): 174-181.

[15] Sahba F, Tizhoosh H R, Salama M M A. A reinforcement learning framework for medical image segmentation. In Proc. International Joint Conference on Neural Networks, Vancouver, Canada, Jul. 16-21, 2006, pp.511-517.

[16] Sahba F, Tizhoosh H R, Salama M M A. A reinforcement agent for object segmentation in ultrasound images. Expert Systems with Applications, 2008, 35(3): 772-780.

[17] Obaidellah U H B. A finite element approach for the planning and simulation of 3D mandibular osteotomy for orthoganathic surgery

[Master Thesis]. Faculty of Computer Science and Information Technology, University of Malya, 2006.

[18] DICOMsample: DICOM Files. July 2008, http://pubimage.hcuge.ch:8080/.

[19] Zhang Y J. A review of recent evaluation methods for image segmentation. In Proc. The Sixth International Symposium on Signal Processing and Its Applications, Kuala Lumpur, Malaysia, Aug. 13-16, 2001, pp.148-151.

[20] Te-shen Liang, Rodriguez J J. MR cranial image segmentation — A morphological and clustering approach. In Proc. IEEE Southwest Symp. Image Analysis and Interpretation, San Antonio, USA, Apr. 8-19, 1996, pp.184-189.

[21] Pan Z, Lu J. A Bayes-based region-growing algorithm for medical image segmentation. Computing in Science & Engineering, 2007, 9(4): 32-38.

[22] Lu H, Bao S. An extended image force model of snakes for medical image segmentation and smoothing. In Proc. The 8th International Conference on Signal Processing, Beijing, China, Nov. 16-20, 2006.
No related articles found!
Full text



[1] Zhong Renbao; Xing Lin; Ren Zhaoyang;. An Interactive System SDI on Microcomputer[J]. , 1987, 2(1): 64 -71 .
[2] Jin Zhiquan; Liu Chengfei; Sun Zhongxiu; Zhou Xiaofang; Chen Peipei; Gu Jianming;. Design and Implementation of a Heterogeneous Distributed Database System[J]. , 1990, 5(4): 363 -373 .
[3] Shen Yidong;. Form alizing Incomplete Knowledge in Incomplete Databases[J]. , 1992, 7(4): 295 -304 .
[4] Adelino Santos;. Cooperative Hypermedia Editing with CoMEdiA[J]. , 1993, 8(3): 67 -79 .
[5] Fang Zhiyi; Ju Jiubin;. NONH:A New Cache-Based Coherence Protocol for Linked List Structure DSM System and Its Performance Evaluation[J]. , 1996, 11(4): 405 -415 .
[6] WANG Guoping; HUA Xuanji; SUN Jiaguang;. The Differential Equation Algorithm for General Deformed Swept Volumes[J]. , 2000, 15(6): 604 -610 .
[7] Peter M. Haverty, Zhi-Ping Weng, and Ulla Hansen. Transcriptional Regulatory Networks Activated by PI3K and ERK Transduced Growth Signals in Human Glioblastoma Cells[J]. , 2005, 20(4): 439 -445 .
[8] Wei Lu, Xiu-Tao Yang, Tao Lv, and Xiao-Wei Li. An Efficient Evaluation and Vector Generation Method for Observability-Enhanced Statement Coverage[J]. , 2005, 20(6): 875 -884 .
[9] Zhou-Wang Yang, Chun-Lin Wu, Jian-Song Deng,and Fa-Lai Chen. Specification of Initial Shapes for Dynamic Implicit Curve/Surface Reconstruction[J]. , 2006, 21(2): 249 -254 .
[10] Joonghyun Ryu, Rhohun Park, and Deok-Soo Kim. Connolly Surface on an Atomic Structure via Voronoi Diagram of Atoms[J]. , 2006, 21(2): 255 -260 .

ISSN 1000-9000(Print)

CN 11-2296/TP

Editorial Board
Author Guidelines
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
E-mail: jcst@ict.ac.cn
  Copyright ©2015 JCST, All Rights Reserved