You said “Strangely enough the last real ride I attended was my first ever group ride and I had an absolute blast!” but then you asked for advice. So, here’s my advice. Rather than thinking of the obstacles to overcome, just keep thinking of the way you felt on that ride. Also remember that you’ll be a better husband and father if you’re happy. Sometimes doing things for yourself isn’t selfish. Get back on the horse, and ride on brother.