What is your frame made out of? Clearly you’re not afraid to widen holes in the frame, so maybe you could consider drilling a hole in the seat tube instead? This is where internal dropper ports are usually located, though I think if your solution does work out, it could look a lot cleaner.
I’m sure others will correct me here, and while this isn’t ideal, I will say I’ve run bottom brackets without the shell before (by mistake) and the world did not end. My BB probably wore out a bit prematurely, as the shell exists to keep water out of the frame and the BB itself. While the shell exists to improve durability, it doesn’t affect the performance AFAIK.