View any smartphone tear-down video and you can see that the insides of your phone are incredibly tightly packed; design decisions must be made, some of which we will agree with and others we will not.
Had the speakers been front-firing, they would have directed sound to the inside of the Daydream headset, but with the way that they are designed there is ample space for sound to channel out, conveniently located by the ears - it would have worked fine.
As the top/bottom bezels on the Pixel XL are actually larger, overall, than they were on the Nexus 6 (and I'm not a bezel-hater... I like being able to hold my phone without concern of accidentally pressing something). Looking at a
teardown, they is actually ample room to have fitted a front-firing speaker within the bottom bezel. The section at the top is pretty jam-packed - the only justification I can see for not running dual front-firing speakers would be if, in the limited space they had, they were unable to effectively seat a speaker that could function at higher output where they seated the existing 'earpiece' speaker.