While he was in Newton, Conn., the president told the governor of Connecticut that last Friday was the most difficult day of his presidency.
The governor and the president spoke at Sunday night's memorial.
"In winter, each time I see the beginning of a snowfall, I will be thinking of those 27 souls lost," said Connecticut Governor Dan Malloy.
Connecticut's governor introduced President Obama after the president met privately with loved ones of those who were killed. Some families on Sunday night released photos via social media.
The public event was a short, but sobering, call to action after Friday's deadly grade school attack.
"They lost their lives in a school that could have been any school, in a quiet town full of good and decent people that could be any town in America," said Obama.
It could be and it has been in many towns. The president told those here and a national television audience, that this is the fourth time he has attended such an event after a mass murder; hugging survivors and consoling families.
"We can't tolerate this anymore," the president said. "These tragedies must end and to end them we must change."
The president then pledged, in the coming weeks, to bring together experts in law enforcement, mental health, educations and parents to solve the problem.
"We can't accept events like this as routine," he said. "Are we really prepared to say we are powerless in the face of such carnage? That the politics are too hard?"
He ended by reading 20 first names of the children who died.
As the president spoke in the high school once attended by the gunman who took all those lives, about a mile away, mourners surrounded a growing town shrine.
Sunday night, they were joined by a group of comfort dogs from Lutheran Church Charities in the Chicago suburb of Addison. This ministry responds to emergency scenes and the golden retrievers help claim nerves.
"They'll pet the dog, and when you pet a dog, it reduces your blood pressure, and when you're relaxed you're more open to share what's going on. It helps people process what they're going through," said Tim Hetzner of Lutheran Church Charities.
They were invited here by Christ the King Church in Newtown, where funerals will be held this week for two of the first graders.
Sunday morning children in the church and their parents spent time with the comfort dogs and unwound.
"Our handlers are trained listeners, pray with people if they want to and just allow people to talk and process in every way this has affected the families here," Hetzner said.
The dogs and the handlers will be at that church all week for the funerals trying to help parishioners through things. It will be a very difficult week after a very difficult day.
