Am I swimming too hard in races?

It's going to be extremely tough to wear yourself out in the swim. You'll *need* / have to back off before you dig yourself into a hole that's going to really hurt the rest of your race.

- edit -

Let me rephrase... I don't think you can go too far out of your fitness level in a swim... but that doesn't mean things like leaving your comfort zone (or just excitement) can't spike your HR or mess you up. Also, being disoriented can hurt.

I find myself seriously gassed getting out of the water as well.  And that's with pretty low perceived effort.  My T1 is usually slow because I'm trying to recover a little, and I spend the first portion of the bike trying to get my HR under control before I feel like I've hit a groove.

Not sure what it is...pool swims don't give me any issue.  More and more I realize that the only thing that replicates an open water tri swim with hundreds of people is....an open water tri swim with hundreds of people.  This doesn't mean to stop putting in the work in the pool, but - for me anyway - it means that the swim is it's own beast and is tough to prepare for.  I guess it's mainly because I'm not the greatest swimmer, but things throw me off - contact, waves, sighting - and that stuff spikes my HR in almost every swim.

I think you've answered your own question here. 

In anything but a sprint (which should, obviously, be at sprint pace), I generally go at "strong" pace but never "hard" pace.  This means that I don't ever breathe hard in these races and am not winded at all exiting the swim.  I breathe every 3 strokes just like in training.  For IM distance, I go at easy pace.  For IM, I treat the swim as a warm up. 

While conventional wisdom suggests that you should go out hard for the first few hundred meters of the swim to clear the pack, I have had much more success at longer distance races by going at a steady pace and clearing the pack by swimming around them.  You don't want to burn matches until you absolutely have to burn them and the 40 seconds gained on the field by sprinting ahead of them at the swim start is easily lost in your first mile of walk/run or just walk if your match-burning compromises your run.

Edited to be more responsive to the OP.

2 Masters sessions per week which range from 3000-4000 yards and one solo sesson which is about 3000 yds