Despite their close phylogenetic relationship, type A and B influenza viruses exhibit major epidemiological differences in humans, with the latter both less common and less often associated with severe disease. However, it is unclear what processes determine the evolutionary dynamics of influenza B virus, and how influenza viruses A and B interact at the evolutionary scale. To address these questions we inferred the phylogenetic history of human influenza B virus using complete genome sequences for which the date (day) of isolation was available. By comparing the phylogenetic patterns of all eight viral segments we determined the occurrence of segment reassortment over a 30-year sampling period. An analysis of rates of nucleotide substitution and selection pressures revealed sporadic occurrences of adaptive evolution, most notably in the viral hemagglutinin and compatible with the action of antigenic drift, yet lower rates of overall and nonsynonymous nucleotide substitution compared to influenza A virus. Overall, these results led us to propose a model in which evolutionary changes within and between the antigenically distinct 'Yam88' and 'Vic87' lineages of influenza B virus are the result of changes in herd immunity, with reassortment continuously generating novel genetic variation. Additionally, we suggest that the interaction with influenza A virus may be central in shaping the evolutionary dynamics of influenza B virus, facilitating the shift of dominance between the Vic87 and the Yam88 lineages.