I don't think I'd classify the 3000 mAh battery in the U Ultra as "small" but it's definitely not what you'd originally expect for a phone of this size. For comparison, LG gets a 3200 mAh in the LG V20 (near identical dimensions to the U Ultra) and it's removable. LG gets a 3300 mAh non-removable in the much smaller LG G6, and OnePlus gets 3400 mAh in the OnePlus 3T.
I
never experienced bad battery life, but I think it's an expectation thing — people think if they're going to use a phone this big, it should have a
really big battery. With the optimizations HTC has done, 3000 mAh makes it through a day on the U Ultra, but it definitely isn't the battery life champion some had hoped considering its physical size.