B migration in Si and Ge matrices raised a vast attention because of its influence on the production of confined, highly p-doped regions, as required by the miniaturization trend. In this scenario, the diffusion of B atoms can take place under severe conditions, often concomitant, such as very large concentration gradients, non-equilibrium point defect density, amorphous-crystalline transition, extrinsic doping level, co-doping, B clusters formation and dissolution, ultra-short high-temperature annealing. In this paper, we review a large amount of experimental work and present our current understanding of the B diffusion mechanism, disentangling concomitant effects and describing the underlying physics. Whatever the matrix, B migration in amorphous (α-) or crystalline (c-) Si, or c-Ge is revealed to be an indirect process, activated by point defects of the hosting medium. In α-Si in the 450-650 °C range, B diffusivity is 5 orders of magnitude higher than in c-Si, with a transient longer than the typical amorphous relaxation time. A quick B precipitation is also evidenced for concentrations larger than 2 × 1020 B/cm3. B migration in α-Si occurs with the creation of a metastable mobile B, jumping between adjacent sites, stimulated by dangling bonds of α-Si whose density is enhanced by B itself (larger B density causes higher B diffusivity). Similar activation energies for migration of B atoms (3.0 eV) and of dangling bonds (2.6 eV) have been extracted. In c-Si, B diffusion is largely affected by the Fermi level position, occurring through the interaction between the negatively charged substitutional B and a self-interstitial (I) in the neutral or doubly positively charged state, if under intrinsic or extrinsic (p-type doping) conditions, respectively. After charge exchanges, the migrating, uncharged BI pair is formed. Under high n-type doping conditions, B diffusion occurs also through the negatively charged BI pair, even if the migration is depressed by Coulomb pairing with n-type dopants. The interplay between B clustering and migration is also modeled, since B diffusion is greatly affected by precipitation. Small (below 1 nm) and relatively large (5-10 nm in size) BI clusters have been identified with different energy barriers for thermal dissolution (3.6 or 4.8 eV, respectively). In c-Ge, B motion is by far less evident than in c-Si, even if the migration mechanism is revealed to be similarly assisted by Is. If Is density is increased well above the equilibrium (as during ion irradiation), B diffusion occurs up to quite large extents and also at relatively low temperatures, disclosing the underlying mechanism. The lower B diffusivity and the larger activation barrier (4.65 eV, rather than 3.45 eV in c-Si) can be explained by the intrinsic shortage of Is in Ge and by their large formation energy. B diffusion can be strongly enhanced with a proper point defect engineering, as achieved with embedded GeO2 nanoclusters, causing at 650 °C a large Is supersaturation. These aspects of B diffusion are presented and discussed, modeling the key role of point defects in the two different matrices.