Smith, 93, says he will never forget the first time he held it.
"I felt a sense of relief, a sense of victory, a sense of togetherness," Smith said Thursday.
So, when his home in Gary was robbed last summer and his Congressional Gold Medal stolen, this proud pioneer didn't want to believe it.
Smith searched his ransacked home up and down until his health landed him in the hospital for months.
"As time went by I became even ashamed to even announce it or ask about it until my police friend came over, and he said, 'Well, let's take a shot at it," said Smith.
"It took us a long time to believe that it was gone," said sister Cleotha McElroy. "He was hurt. It hurt so bad."
Along with the Presidential Medal of Freedom, the Congressional Gold Medal is the nation's highest civilian honor.
In 2007, Smith was too ill to join his fellow Tuskegee Airmen in Washington for an awards ceremony with then-president Bush, so officials held an event just for him in Indiana.
"There were really 15,000 people connected to the Tuskegee Airmen, because with every plane there were 10 people," Smith said. "We get the credit for all the other people and what they did too."
After the war, Smith spent more than 50 years as a teacher and high school principal, often counseling misguided youth.
He had this to say to the person who stole his medal:
"In my heyday I would have given him a chop or two," said Smith. "I'd like to send them on a different kind of path as I did for the high school that I served."
"He would be satisfied with just the medal being returned. He wouldn't even prosecute," said Gary Police Department Sergeant Fred Cook.
For now all Smith can do is hope that this medal forever close to his heart will once again be in his hands.
APRIL 13 UPDATE: Gary officials are working with Congressman Pete Visclosky's office to get Smith a replacement medal.