In this thesis, the aim is to understand mechanisms behind epitaxial growth, control the crystallinity and properties of grown films, and differentiate between two types of epitaxy: conventional epitaxy, driven by strong chemical bonds between the film and substrate, and van der Waals epitaxy (vdWE), characterized by weaker van der Waals forces, enabling stress-free growth without strict lattice matching. I investigated the epitaxial growth of transition metal oxides and nitrides (NiO, CrN, MoOx (x = 2, 3)) thin films synthesized by reactive magnetron sputtering on various substrates, including silicon, sapphire (c-plane and r-plane) and mica.