There is an early morning group I have seen there in the summer, although I don't know who organizes it. It's not a good place to swim come summer day time/after work though. Their lifeguards are insistent you don't leave the roped area.
Fergysun, I coach the Heat Tri Team. We will be swimming in Lake Gardner Saturday mornings at 7am. We will also try to get in one weeknight after the lifegaurds leave. Shoot me an email and I will send you the details on the team.