With the population of Illinois state prisons approaching 49,000 inmates, the push is already on locally to follow the attorney general's lead.
"We need to change the way we do business. Mandatory minimums do not work," said John Maki, John Howard Association.
Maki, who leads Illinois' prison watchdog John Howard Association, has been saying it for years: that sending low-level drug users or sellers to prison to serve mandatory sentences has only overcrowded state as well as federal penitentiaries.
"We now have more people under lock and key per capita than any country in the world," said Sen. Dick Durbin, (D), Illinois.
Durbin, a member of his chamber's judiciary committee, is sponsoring a bill to reform federal sentencing guidelines.
"We've had a 30-percent increase in the population of this country and an 800-percent increase in the prison population," Durbin said.
With federal inmate counts soaring and budgets draining, Holder says an end to mandatory minimums coupled with a "compassionate release" policy for older and sick prisoners would make room for more violent criminals.
"It will enable us to use our limited resources to house those who pose the greatest threat," Holder said.
Pardoned ex-felon Gator Bradley says mandatory sentences have led to racial imbalances in American prisons.
"White kid gets caught with drugs, it's a medical problem. African-American kid gets caught with drugs, it's a criminal problem," Bradley said.
John Howard's Maki wants all prosecutors to heed Holder's recommendations.
"For Holder to say our system needs change - we need to change the way we use our prison system. That's a call to prosecutors all across the country, not just on the federal level but on the state level as well," Maki said.
Chicago Mayor Rahm Emanuel, who backs mandatory minimums for gun offenders, said he supports Holder on low-level drug offenders, as does Cook County Board President Toni Preckwinkle, who's on record saying the county can no longer afford to add beds at 26th and California.