Swim/race question

Ok...Everyone says start to the outside, but what if you are a good swimmer? I started to the outside yesterday and felt a a big disadvantage. I was surrounded by people who either couldn't swim, were trying to run the swim (it was really low tide) and people who were doggie paddling. It took a while to get into a rhythm trying to swim around the walkers and I felt like I would have been better off around faster people who could swim. I'm not the fastest person in the swim, but the swim is probably my best event of the three. Since everyone passes me on the run I need to take every advantage I can get. Thanks in advance for any advice -S