I just found out that the calibration frequency is related to the update frequency you set in the weather app on your Phone. So to improve the altitude calibration you may want to try to increase the frequency with which the weather app on your Phone updates. Below picture shows wat happens with the altidude values if you change the update frequency from 6 to 1 hrs.
View attachment 299872
Also, I found that cleaning the barometer sensor by wiping the back of your watch helps. I had problems with the floor counter (based on the altitude as well), that are now mostly solved.
The barometer sensor might be off a bit (as every sensor can be), but a change in the atmospheric pressure is well detected. So for exact altitude measurement I wouldn't trust it too much (1 hPa difference stands for roughly 10m height difference, and with changing see level pressure as a reference the inaccuracy only increases).