Abstract: Face analysis tasks, e.g., estimating gender or age from a face image, have been attracting increasing interest in recent years. However, most existing studies focus mainly on analyzing an adult's face and ignore an interesting question:is it easy to estimate gender and age from a baby's face? In this paper, we explore this interesting problem. We first collect a new face image dataset for our research, named BabyFace, which contains 15 528 images from 5 872 babies younger than two years old. Besides gender, each face image is annotated with age in months from 0 to 24. In addition, we propose new age estimation and gender recognition methods. In particular, based on SSR-Net backbone, we introduce the attention mechanism module to solve the age estimation problem on the BabyFace dataset, named SSR-SE. In the part of gender recognition, inspired by the age estimation method, we also use a two-stream structure, named Two-Steam SE-block with Augment (TSSEAug). We extensively evaluate the performance of the proposed methods against the state-of-the-art methods on BabyFace. Our age estimation model achieves very appealing performance with an estimation error of less than two months. The proposed gender recognition method obtains the best accuracy among all compared methods. To the best of our knowledge, we are the first to study age estimation and gender recognition from a baby's face image, which is complementary to existing adult gender and age estimation methods and can shed some light on exploring baby face analysis.