›› 2009, Vol. 24 ›› Issue (6): 1010-1017.

Special Issue: Artificial Intelligence and Pattern Recognition

• Special Section on International Partnership Programs Supported by CAS • Previous Articles     Next Articles

Performance Evaluation of Machine Learning Methods in Cultural Modeling

Xiao-Chen Li1 (李晓晨), Wen-Ji Mao1 (毛文吉), Senior Member, CCF, Member, ACM, Daniel Zeng1, 2 (曾大军), Senior Member, IEEE, Peng Su1, 3 (苏鹏), and Fei-Yue Wang1 (王飞跃), Senior Member CCF, Fellow, IEEE   

  1. 1Key Laboratory of Complex Systems and Intelligence Science, Institute of Automation, Chinese Academy of Scienceshspace*3mm}Beijing 100190, China
    2Department of Management Information Systems, University of Arizona, Tucson AZ 85721, U.S.A.
    3School of Management Engineering, Shandong Jianzhu University, Jinan 250013, China
  • Received:2009-03-08 Revised:2009-07-20 Online:2009-11-05 Published:2009-11-05
  • About author:
    Xiao-Chen Li is a Ph.D. candidate at the Institute of Automation, Chinese Academy of Sciences. His research interests mainly focus on social computing and agent-based modeling.
    Wen-Ji Mao received her Ph.D. degree in computer science from the University of Southern California in 2006. She is an associate professor at the Institute of Automation, Chinese Academy of Sciences. Prof. Mao is a member of ACM and AAAI, and a senior member of the China Computer Federation. Her research interests include artificial intelligence, multi-agent systems and social modeling.
    Daniel Zeng received his Ph.D. degree in industrial administration from Carnegie Mellon University in 1998. He is a research professor at the Institute of Automation, Chinese Academy of Sciences. He is also affiliated with the University of Arizona as an associate professor and the director of the Intelligent Systems and Decisions Laboratory. Prof. Zeng is a senior member of IEEE. His research interests include software agents and multi-agent systems, intelligence and security informatics, social computing and recommender systems.
    Peng Su is a Ph.D. candidate at the Institute of Automation, Chinese Academy of Sciences. His research interests mainly focus on machine learning and social computing.
    Fei-Yue Wang received his Ph.D. degree in computer and systems engineering from Rensselaer Polytechnic Institute in 1990. He is the director of the Key Laboratory of Complex Systems and Intelligence Science at the Chinese Academy of Sciences. He is also a professor in the University of Arizona's Systems & Industrial Engineering Department and the director of the university's Program in Advanced Research of Complex Systems. Prof. Wang is a fellow of IEEE, INCOSE, IFAC, ASME, and AAAS. His current research interests include social computing, Web and services science, modeling, analysis and control of complex systems, especially social and cyber-physical systems.
  • Supported by:

    This work is supported in part by the National Natural Science Foundation of China under Grant Nos. 60621001, 60875028, 60875049, and 70890084, the Ministry of Science and Technology of China under Grant No. 2006AA010106, and the Chinese Academy of Sciences under Grant Nos. 2F05N01, 2F08N03 and 2F07C01.

Cultural modeling (CM) is an emergent and promising research area in social computing. It aims to develop behavioral models of human groups and analyze the impact of culture factors on human group behavior using computational methods. Machine learning methods, in particular classification, play a critical role in such applications. Since various cultural-related data sets possess different characteristics, it is important to gain a computational understanding of performance characteristics of various machine learning methods. In this paper, we investigate the performance of seven representative classification algorithms using a benchmark cultural modeling data set and analyze the experimental results as to group behavior forecasting.

[1] Subrahmanian V S. Computer science: Cultural modeling in real time. Science, 2007, 317(5844): 1509–1510.
[2] Subrahmanian V S, Albanese M, Martinez M V, Nau D, Reforgiato D, Simari G I, Sliva A,Wilkenfeld J, Udrea O. CARA: A cultural-reasoning architecture. IEEE Intelligent Systems, 2007, 22(2): 12–16.
[3] Khuller S, Martinez V, Nau D, Simari G, Sliva A, Subrahmanian V S. Finding most probable worlds of logic programs. In Proc. the First International Conference on Scalable Uncertainty Management, Washington DC, USA, October 10–12, 2007, pp.45–59.
[4] Martinez V, Simari G I, Sliva A, Subrahmanian V S. CONVEX: Context vectors as a paradigm for learning group behaviors based on similarity. IEEE Intelligent Systems, 2007, 23(4): 51–57.
[5] Wang F Y. Is culture computable? IEEE Intelligent Systems, 2009, 24(2): 2–3.
[6] Wang F Y, Carley K M, Zeng D, Mao W. Social computing: From social informatics to social intelligence. IEEE Intelligent Systems, 2007, 22(2): 79–83.
[7] Wang F Y. Toward a paradigm shift in social computing: The ACP approach. IEEE Intelligent Systems, 2007, 22(5): 65– 67.
[8] Zeng D, Wang F Y, Carley K M. Social computing. IEEE Intelligent Systems, 2007, 22(5): 20–22.
[9] Minorities at risk organizational behavior dataset. Minorities at Risk Project, University of Maryland, College Park: Center for International Development and Conflict Management, 2008, http://www.cidcm.umd.edu/mar.
[10] Hand D J, Yu K. Idiot’s Bayes — Not so stupid after all? International Statistical Review, 2001, 69(3): 385–398.
[11] Vladimir N Vapnik. Support Vector Estimation of Functions. Statistical Learning Theory, Haykin S (ed.), Springer-Verlag, 1998, pp.375–570.
[12] Jain A K, Mao J, Mohiuddin K M. Artificial neural networks: A tutorial. Computer, 1996, 29(3): 31–44.
[13] Christopher M Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995.
[14] Kotsiantis S, Zaharakis I, Pintelas P. Machine learning: A review of classification and combining techniques. Artificial Intelligence Review, 2006, 26(3): 159–190.
[15] Breiman L. Random forests. Machine Learning, 2001, 45(1): 5–32.
[16] Thabtah F. A review of associative classification mining. The Knowledge Engineering Review, 2007, 22(1): 37–65.
[17] Baralis E, Garza P. A lazy approach to pruning classification rules. In Proc. the Second IEEE International Conference on Data Mining, Maebashi, Japan, December 9–12, 2002, pp.35– 42.
[18] Minorities at Risk Project. College Park, MD: Center for International Development and Conflict Management, 2005, http://www.cidcm.umd.edu/mar/.
[19] Ling C X, Huang J, Zhang H. AUC: A statistically consistent and more discriminating measure than accuracy. In Proc. the Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, Mexico, August 9–15, 2003, pp. 329–341.
[20] Japkowicz N, Stephen S. The class imbalance problem: A systematic study. Intelligent Data Analysis, 2002, 6(5): 429– 450.
[21] Kohavi R, Wolpert D H. Bias plus variance decomposition for zero-one loss functions. In Proc. the Thirteenth International Conference on Machine Learning, Bari, Italy, July 3–6, 1996, pp.275–283.
[22] Govindarajan M. Text mining technique for data mining application. Proceedings of World Academy of Science, Engineering and Technology, 2007, 26(104): 544–549.
[23] Liu B, Hsu W, Ma Y. Integrating classification and association rule mining. In Proc. the Third International Conference on Knowledge Discovery and Data Mining, New York, USA, August 27–31, 1998, pp.80–86.
[24] Li W, Han J, Pei J. CMAR: Accurate and efficient classification based on multiple class-association rules. In Proc. the First IEEE International Conference on Data Mining, San Jose, USA, November 29–December 2, 2001, pp.369–376.
[25] Yin X, Han J. CPAR: Classification based on predictive association rules. In Proc. the Third SIAM International Conference on Data Mining, San Francisco, USA, May 1–3, 2003, pp.369–376.
[26] Grˇcar M, Mladeniˇc D, Fortuna B, Grobelnik M. Data sparsity issues in the collaborative filtering framework. In Proc. the Seventh International Workshop on Knowledge Discovery on the Web, Chicago, USA, August 21, 2005, pp.58–76.
[27] Nathalie J. Class imbalances: Are we focusing on the right issue? In Proc. the ICML 2003 Workshop on Learning from Imbalanced Data Sets, Washington DC, USA, August 21, 2003.
[28] Japkowicz N. The class imbalance problem: Significance and strategies. In Proc. the Second International Conference on Artificial Intelligence, Las Vegas, USA, June 26–29, 2000, pp.111–117.
[29] Zhang J. kNN approach to unbalanced data distributions: A case study involving information extraction. In Proc. the ICML2003 Workshop on Learning from Imbalanced Data Sets, Washington DC, USA, August 21, 2003.
[30] Barandela R, Sanchez J S, Garcia V, Rangel E. Strategies for learning in class imbalance problems. Pattern Recognition, 2003, 36(3): 849–851.
[31] Rish I, Hellerstein J, Thathachar J. An analysis of data characteristics that affect naive Bayes performance. Technical Report, IBM T.J. Watson Research Center, 2001.
[32] Domingos P, Pazzani M. On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning, 1997, 29(2/3): 103–130.
[33] Martinez-Arroyo M, Sucar L E. Learning an optimal naive Bayes classifier. In Proc. the Eighteenth International Conference on Pattern Recognition, Hong Kong, China, August 20–24, 2006, p.958.
[34] Zhang C, Yu P S, Bell D. Domain-driven data mining. IEEE Transactions on Knowledge and Data Engineering, 2009, 21(2): 301.
[35] Cao L, Zhang C. Domain-driven, actionable knowledge discovery. IEEE Intelligent Systems, 2007, 22(4): 78–88.
[36] Nau D, Wilkenfeld J. Computational cultural dynamics. IEEE Intelligent Systems, 2008. 23(4): 18–19.

No related articles found!
Full text



[1] Liu Mingye; Hong Enyu;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] Chen Shihua;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] Gao Qingshi; Zhang Xiang; Yang Shufan; Chen Shuqing;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] Chen Zhaoxiong; Gao Qingshi;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] Huang Heyan;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] Min Yinghua; Han Zhide;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] Tang Tonggao; Zhao Zhaokeng;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] Min Yinghua;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] Zhang Bo; Zhang Ling;. Statistical Heuristic Search[J]. , 1987, 2(1): 1 -11 .
[10] Zhu Hong;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .

ISSN 1000-9000(Print)

CN 11-2296/TP

Editorial Board
Author Guidelines
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
E-mail: jcst@ict.ac.cn
  Copyright ©2015 JCST, All Rights Reserved