Screen, processor, RAM&Storage (both speed and capacity). Cameras. Extra hardware, like fingerprint or IR sensors. Speakers.... Waterproof certification...software features that require additional licensing from the HW vendors like Qualcomm. Phones with more features require more R&D to design, built and test... It wouldn't make much business sense for Samsung to pass along the cost to develop the S-Pen to users of the Galaxy Avant for instance.
Sometimes there are market segment concerns.... On top end, flagship-level phones, there is a little bit more room for price wiggling... whereas down in the mid and low-end tiers where competition is brutally tight, OEMs have to be a bit more aggressive in lowering prices.