In my experience, (I’ve been cycling for 40+ years as an adult and have owned about 20 different bikes.) mixing drivetrain components from different manufacturers frequently causes problems. Keeping all the drivetrain components either all Sram or all Shimano is the safest bet. If you keep the chain, cassette, rear derailleur, and shifter all from the same company you probably would not have this problem. (Back in the old days of 2x and 3x drivetrains, off-brand cranksets and chainrings caused all sorts of problems.)
In addition, you could have a bent derailleur or derailleur hanger. Have that checked, put on a new chain, and have the rear derailleur adjusted before you spend any money on new stuff.
Also, make sure the chain is not too long. I had a bike that was slipping gears and I took it to the bike shop and they adjusted everything but couldn’t really find anything wrong. But, it continued to slip. So I removed 2 links of chain to see if that would help and never had problems with it again. I had asked the mechanic if the chain was too long but he said it wasn’t and it did look right to me also but removing those 2 links saved the day.
If you do switch to all Shimano, consider using the more affordable SLX cassette to save money. SLX cassettes are heavier than XT cassettes but they shift just as well.