For several decades, ethologists and comparative psychologists have been using a linguistic terminology to discuss complex communicative abilities in animals, with a particular focus on sound combinatorial rules. One historical example is the possible syntactic ability of songbirds. More recently, context-dependent call combinations have been described in nonhuman primates. This time, the detailed observational and experimental data gathered in this area has even drawn the attention of linguists and has given rise to studies highlighting the relevance of linguistic tools for the study of nonhuman primate communication systems. However, the parallels that can be drawn between humans’, birds’ and nonhuman primates’ verbal/vocal combinations still remain the topic of intense debate possibly because mismatches between the terminologies used have confounding effects. The question is: can we go beyond the traditional dichotomy between phonological and lexical syntax to characterize the diversity of sound combinatorial rules found in animals? Here, we will adopt a two-step approach in order to discuss: (1) what forms sound combination takes in animals, based on structural and functional criteria and when it may or not be appropriate to use linguistic terms; (2) why sound combination may have evolved in some species more than others. We will notably illustrate our arguments with recent findings in some cooperative breeding birds and guenons, where cases of meaningful sound compositionality have been recently described.