It happened at Montrose Harbor around 5 p.m. Tuesday.
Firefighters say he was walking three dogs when they either jumped or fell into the water. The man jumped in after them and divers had to rescue him.
The dogs were checked out by a veterinarian and are expected to be OK.