Video of the incident was posted on the internet Thursday. It appears to show the officer lifting the dog by its collar and hitting it with a leash. It happened on a busy street with other officers on the scene.
Hammond Mayor Tom McDermott says the officer is now on administrative leave.
His dog has been removed from his control.
In a statement, McDermott says "Anybody who loves dogs as much as I do is always saddened and shocked anytime you hear of a dog's abuse. When you find out it happened with an employee of yours, it makes it that much more shocking and disturbing."