Living at about 5,500 ft elevation in Utah, I cannot say that I notice any difference at all due to elevation between sea level and at least 8,000 ft. On the other hand, I was in Colorado last weekend, where my rides (road bike this time) started at 8,500 ft and went to well over 10,000 ft, and I did notice that my heart rate was considerably higher. But I could still climb at much the same rate.
I can imagine it would be different were I not already acclimatized however.
For certain, hydration does become an issue though. Not just due to the elevation, but also because the air in much of the western mountains is extremely dry.