The vote was seven to three to remove them.
Students protested earlier this week, calling for their removal.
The Chicago Board of Education voted 4-3 in late June to keep a $33 million contract that pays for Chicago police resource officers in CPS high schools.
But the choice about how many school resource officers will be hired remains in the hands of the local school councils.
The vote took place as several hundred people marched and rallied in the Loop in a demonstration organized by the Chicago Teachers Union.
"We are considering going back into school communities that do not have full-time nurses but we'll have police officers? Misplaced priorities!" said Stacy Davis Gates, vice president of CTU Local 1.
WATCH: School board votes to keep officers in Chicago schools
As the Chicago Board of Education met virtually, demonstrators called for the $33 million contract to be canceled.
"Reforming our SRO program is not enough, that police do not belong in our schools, that this is a systemic issue, that it's a civil rights issue," school board member Elizabeth Todd Breland said.
The rally culminated weeks of public pressure on CPS to remove officers from the schools. But by the thinnest of margins, the motion to cancel the contract was voted down.
But those in favor of keeping police in schools say, they want the same thing -- safety for students.
"Safety is of the utmost concern to all of us," said Chicago Board of Education President Miguel del Valle, who cast the deciding vote.
WATCH: CTU holds rally as board votes on police in schools
Frustrated CPS students like Essence Gatheright demanded their concerns be taken seriously.
"We need more people who are willing to side with youth, who are willing genuinely see us," Gatheright said. "Because it's one thing to see us, it's one thing to hear us, but it's another thing to feel us. It's another thing to support us."
In January 2019, 16-year-old Dnigma Howard was involved in a disturbing altercation with police at Marshall High School after a classroom confrontation with staff over a cell phone.
Her father, Laurentio Howard, said she is still traumatized. He joined those rallying for police to be removed from schools.
"I'm here because my daughter was brutally attacked by two Chicago Police Officers January 29, 2019," Howard said. "She was drug down a flight of stairs, she was Tazed three times."
CPS teacher Danielle Bridges said the decision failed to acknowledge the disparities within the city's Black and Brown communities.
"It doesn't take into account their experiences with police outside of the school and then coming into the school and not feeling safe," Bridges said.
"I wish we had a environment here in Austin where we didn't need school resource officers," said school board member Dwayne Truss.
In the coming weeks, CPS will provide all the local school councils with a tool kit to help them decide whether to have resource officers next year. The councils will have to vote before the school year begins.
A survey done by CPS this spring found that 68% of LSC members who responded favored keeping the officers.
Chicago Mayor Lori Lightfoot has said in the past she was in favor of leaving the decision to the councils, who will vote on if cops will stay in schools in the next few months.