Refer to figure.
The barometric altimeter of an aircraft is a very simple device. It is actually a barometer, measuring the static air pressure around the aircraft. This is then mechanically rigged up inside the instrument and calibrated to display as an altitude.

The barometric reference pressure knob on the altimeter allows us to change the output of the instrument, depending on what we want to measure our altitude against. If we wish to measure our altitude above mean sea level, we change the reference setting to whatever the pressure is at sea level (QNH or regional pressure setting), and the altimeter reads out our height over sea level (which is our altitude).

If we want to know our height above an aerodrome, the tower tells us the QFE which is the pressure at the aerodrome, and we set that on our altimeter, which now tells us the height above that pressure setting. This works also for flight levels, where we wish to fly at a certain height avoe the 1013.2 hPa (standard pressure setting) pressure level.

Therefore, the barometric altimeter is only ever telling us our altitude above a certain pressure level in the atmosphere, and we use that to give us different useful heights and altitudes for navigation, traffic and obstacle avoidance.