The security was tight and everyone was checked before entering the Hall.
Dozens of protestors gathered outside.
"We do not accept that Olmert would come to the United States to speak about leadership, let alone anything else, because he is a war criminal," said Awad Hamdan, American Muslims For Palestine.
"We are supporting an occupation and it's bad, dirty business," said Sarah Ross, professor.
Olmert's presence here infuriates those who support Palestinians, especially after a series of attacks in Gaza that started last December.
In fact, the United Nations recently released a report that condemned armed Israelis and Palestinians for that conflict.
Protestors want Olmert to be charged and do not believe he's qualified to speak.
"I think that the students at the University of Chicago and neighboring universities need better role models to speak and visit their universities, better than someone who in the past has killed thousands of people," said Fedaa Balouta, UIC student.
Olmert has been charged with fraud and breach of trust but there are people here who support him and Israel and some hope the peace process will work, eventually.
"I am optimistic, but very skeptical. I'm always optimistic, but I highly doubt it," said Ronald Rutzky, pro-Israel supporter.
"He should be allowed to speak. If he was invited here, obviously someone wanted to hear him speak. There's no reason why he should be disallowed just because of his involvement in Israeli government," said Natasha Naamah, Olmert supporter.
About 30 people disrupted Olmert's speech and had to be pulled out at some point during the speech that lasted about 20 to 30 minutes.
Olmert is facing fraud charges for allegedly taking money from an American businessman. He is facing a trial in February in Israel.