Try to go without the pack as much as possible. I have my spare tube strapped under my seat, frame pump, and two bottle cages on my bike (perks of a hardtail). For shorter rides I use the Specialized KEG in one bottle cage with a multi-tool, keys, cash, a bar, and maybe CO2. On the longer rides I just stuff my pockets and bring two water bottles. If I start the ride with an extra layer, I’ll usually tie it around my waist if it gets too hot.
I did a 30 mile ride at the end of the summer and was the only person to not bring a pack. I knew one of the trails we were riding I would be able to refill my water at. I didn’t feel the need to carry water for the whole day when two bottles could be filled and I could keep the weight off my back.