The funeral was for a known gang member on the West Side.
According to police, Roosevelt Yarber was inside the church for about ten minutes.
As Yarber left, officers stopped him and found the weapon.
He is not in a gang, but the young men he was with are confirmed gang members.
This is one of several incidents that have prompted police to closely monitor gang wakes and funerals.
"We do see the potential for violence at these gang wakes and funerals, and I know that it is the hope of the mayor and the superintendent that we can focus our efforts on gang violence at funerals and wakes," said Chicago Police Deputy Chief John Escalante.
Earlier this week, the mayor said police would have a presence at funerals and wakes.
The action comes following the fatal shooting of a man outside a funeral on the South Side last week.
