There is a chemical reaction between the Carbon Dioxide which is the gas causing the bubbles and the H20 of the water. The gas may have left the water but the chemical change stays and affects the taste.
The chemical change that Ainitatyb mentions is where some of the carbon dioxide has dissolved in the water to form carbonic acid. Though the free gas has gone, the acid remains, and it's what gives the water that taste.