I found out how to manually set the different volume for each and every volume level.
I found the solution I needed in this video: https://www.youtube.com/watch?v=Si3y9MePWZk
You enter Engineer Mode by entering this code in the dialer: *#*#3646633#*#*
When you enter Engineer Mode:
1) Scroll to the right until you reach the tab "Hardware Testing".
2) Select "Audio".
3) Select "LoudSpeaker Mode", if you only want to modify the output from the loudspeaker.
4) Change the "Type" drop-down to "Media", which will affect music playback volume, as well as the volume of your applications.
5) Open the "Level" drop down, which should have 14 levels, note how the values under the drop-down change every time you pick a different level, those are the values that you need to modify.
6) Set appropriate volumes for each level, but never go beyond the volume of level 14, or risk damaging your loudspeaker. I also recommend writing down the original volumes in case you want to restore them. You can have music running in the background while doing this, it will help you set the right volumes while listening.
I am not familiar with the other functions of Engineer Mode and I recommend not touching anything else.