Dizziness after a week of loads of swimming normal?

I know this isn't a medical website -- and I know I saw something about dizziness a while back but I coulnd't find the thread... But I figured you guys have more experience with this stuff than anybody!

Last week started with my race swimming in the lake. Followed by the most yards of swimming I've ever done in one week (I'm in the boys vs girls run/swim challenge).

Yesterday, I noticed that I was dizzy off and on throughout the day. And not a hypoglycemia-kind-of dizzy. This was more like verdigo-type of dizzy.

You know how it feels when you have been travelling all day and you finally rest your head on the pillow?  You still feel like you're moving. That was the feeling -- especially if I tilted my head back or to the right side; and when I laid down last night.

It doesn't seem as bad today.  I am noticing little twinges of aching in my left ear - but at this point I'm so hyper-attuned to it, it could be psychosomatic.  I do have allergies and we have had a big shift in the weather (colder/wetter) in the last 3 days.  I wonder if it could be those things and not necessarily an inner-ear / swimming thing? 

Anyone ever experience this?  I'm not one to run to the doctor but need to decide if I should or wait and see -- I'm taking the day off from work tomorrow to take my nieces to the beach and would hate for it to get ruined.