Mayor Rahm Emanuel had hoped the contract offer would serve as a blueprint for a collaborative pension reform.
Only 134 of the 1,100 who voted supported the four-year deal.
The deal would have raised the retirement age for sergeants, increased pension contributions, raised health care contributions and lowered cost of living increases.
But the union says they want a fair collective bargaining agreement that is not tied to any type of pension reform.
"Even if we agree to some type of pension reforms, the message is they don't want it tied together," said Chicago Police Sergeants' Association President James Ade.
Despite the vote, Mayor Emanuel says he will continue to push for pension reform in Springfield.
