Improving my race OWS - more focus on speed or direction?

I just finished my 2nd Olympic tri yesterday (my 4th overall triathlon) - it was a blast! I specifically had the goal of not finishing DFL (dead ?!?!*# last) on the swim this time around, and although I still had some trouble getting my swim training volume up due to tendinitis and scheduling, I had a good day (for me) out there - made it from bottom 1-2% to 30% in my age group. Progress! 

Here's the deal though - I do intervals 20 x 100 yds in the pool at 1:48 now. If I'm doing 20 x 100meters, it's about 1:58-2:00 on 15 seconds rest. I've measured this in two separate pools.

However, in the OWS, my calculated pace for my 34ish minute swim is 2:35/100yds, which seems horrendously slow for me, even given the human churn around the buoys. I didn't get way off course, but I suspect I zigzagged more than I should have en route, as I felt that I was swimming pretty hard.

For you experienced swimmers out there, does this sound like a OWS sighting problem? Or is it possible for me to be swimming this slowly in OWS despite being on course due to waves, traffic, etc.?
 
(I had a good race - even with a 6 minute mechanical breakdown on the bike in which my rear wheel got jammed into my frame cutout due to the dang limit screws popping in suddenly, I still nearly got ino the top 10% in M30-35 even with my crappy swim!)