What phone do you have? I've never had a phone do that. Dragging to homescreen only creates a shortcut to that app. And deleting from there only deletes the shortcut, not the app.
This function differs per launcher. On Nova Launcher, long pressing an app gives me the option to uninstall.