Aim Grammomys are mostly arboreal rodents occurring in forests, woodlands and thickets throughout sub-Saharan Africa. We investigated whether the divergence events within the genus follow the existing evolutionary scenario for the development of African forests since the late Miocene. Location Sub-Saharan African forests and woodlands. Methods We inferred the molecular phylogeny of Grammomys using Bayesian and maximum likelihood methods and DNA sequences of 351 specimens collected from across the distribution of the genus. We mapped the genetic diversity, estimated the divergence times by a relaxed clock model and compared evolution of the genus with forest history. Results Phylogenetic analysis confirms the monophyly of Grammomys and reveals five main Grammomys lineages with mainly parapatric distributions: (1) the poensis group in Guineo-Congolese forests; (2) the selousi group with a distribution mainly in coastal forests of southern and eastern Africa; (3) the dolichurus group restricted to the easternmost part of South Africa; (4) the macmillani group in the northern part of eastern and Central Africa with one isolated species in Guinean forests; and (5) the surdaster group, widely distributed in eastern Africa south of the equator. Every group contains well supported sublineages suggesting the existence of undescribed species. The earliest split within the genus (groups 1 vs. 2–5) occurred in the late Miocene and coincides with the formation of the Rift Valley which resulted in the east–west division of the initially pan-African forest. The subsequent separation between groups (2 vs. 3–5) also dates to the end of the Miocene and suggests the split between Grammomys from coastal to upland forests in eastern Africa followed by a single dispersal event into western Africa during the Pleistocene. Conclusions The evolutionary history of the genus Grammomys closely reflects the accepted scenario of major historical changes in the distribution of tropical African forests since the late Miocene.