Diving hot tub

That is because the pools you go to do not know how to regulate temperature properly.

Competition pools are typically down around 76 degrees I believe.  These athletes generate a LOT of heat with these efforts and the cooler water temps are better.  Most people swim faster in cooler temps as well (kind of like the different between running a marathon when it's 50 and 70 out...you run faster when it's 50 out).  If properly air conditioned/heated, the air outside a pool will always feel cooler than the pool...because it typically IS.  People like to keep rooms below 80 in the summer and around 70 in the winter.  That is USUALLY cooler than the water temp at most pools...which makes people feel like the outside air is cold.  Plus, you're wet...that doesn't help.

Now, I don't quite understand why the diving well has to be kept really cold.  At SOME aquatics centers is is one big pool and shares water with the lap pool, so it makes sense.  However, that doesn't seem to be the case at the Olympic venue.  They look like two separate bodies of water.  But the divers do a lot of standing around so it's likely that even if the water was warm they would still be pretty cold.