The actor was 86 years old.
A close friend of the actor first confirmed the death to WITN in North Carolina.
Griffith is known for his role as sheriff in "The Andy Griffith Show," set in fictional Mayberry and ran 1960-1968 and "Matlock."
Griffith died about 7 a.m. at his coastal home, Dare County Sheriff Doug Doughtie said in a statement.
"Mr. Griffith passed away this morning at his home peacefully and has been laid to rest on his beloved Roanoke Island," Doughtie told The Associated Press, reading from a family statement.
The family will release further information, the sheriff said.
He had suffered a heart attack and underwent quadruple bypass surgery in 2000.
Griffith's career spanned more than a half-century on stage, film and television, but he would always be best known as Sheriff Andy Taylor in the television show set in a North Carolina town not too different from Griffith's own hometown of Mount Airy, N.C.
Griffith set the show in the fictional town of Mayberry, N.C., where Sheriff Taylor was the dutiful nephew who ate pickles that tasted like kerosene because they were made by his loving Aunt Bee, played by the late Frances Bavier. He was a widowed father who offered gentle guidance to son Opie, played by Ron Howard, who grew up to become the Oscar-winning director of "A Beautiful Mind."
Don Knotts was the goofy Deputy Barney Fife, while Jim Nabors joined the show as Gomer Pyle, the unworldly, lovable gas pumper.
On "Matlock," which aired from 1986 through 1995, Griffith played a cagey Harvard-educated defense attorney who was Southern-bred and -mannered with a practice in Atlanta.
In his rumpled seersucker suit in a steamy courtroom (air conditioning would have spoiled the mood), Matlock could toy with a witness and tease out a confession like a folksy Perry Mason.
The character -- law-abiding, fatherly and lovable -- was much like Sheriff Andy Taylor with silver hair and a shingle.
In a 2007 interview with The Associated Press, Griffith said "The Andy Griffith Show," which initially aired from 1960 to 1968, was seen somewhere in the world every day. A reunion movie, "Return to Mayberry," was the top-rated TV movie of the 1985-86 season.
"The Andy Griffith Show" was a loving portrait of the town where few grew up but many wished they did -- a place where all foibles are forgiven and friendships are forever. Villains came through town and moved on, usually changed by their stay in Mayberry. That was all a credit to Griffith, said Craig Fincannon, who met Griffith in 1974.
"I see so many TV shows about the South where the creative powers behind it have no life experience in the South," Fincannon said. "What made `The Andy Griffith Show' work was Andy Griffith himself -- the fact that he was of this dirt and had such deep respect for the people and places of his childhood. A character might be broadly eccentric, but the character had an ethical and moral base that allowed us to laugh with them and not at them. And Andy Griffith's the reason for that."
Griffith's career included stints on Broadway, notably "No Time for Sergeants"; movies such as Elia Kazan's "A Face in the Crowd"; and records. He was inducted into the Academy of Television Arts Hall of Fame in 1992 and in 2005, he received the Presidential Medal of Freedom, one of the country's highest civilian honors.
"The Andy Griffith Show" was one of only three series in TV history to bow out at the top of the ratings. (The others were "I Love Lucy" and "Seinfeld.") Griffith said he decided to end it "because I thought it was slipping, and I didn't want it to go down further."
When asked in 2007 to name his favorite episodes, the ones atop Griffith's list were the shows that emphasized Knotts' character. Griffith and Knotts had become friends while performing in "No Time for Sergeants," and remained so until Knotts' death in 2006 at 81.
"The second episode that we shot, I knew Don should be funny and I should play straight for him," Griffith said. "That opened up the whole series because I could play straight for everybody else. And I didn't have to be funny. I just let them be funny."
Letting others get the laughs was something of a role reversal for Griffith, whose career took off after he recorded the comedic monologue "What It Was, Was Football."
That led to his first national television exposure on "The Ed Sullivan Show" in 1954, and the stage and screen versions as the bumbling draftee in "No Time for Sergeants."
In the drama "A Face in the Crowd," he starred as Larry "Lonesome" Rhodes, a local jailbird and amateur singer who becomes a homespun philosopher on national television. As his influence rises, his drinking, womanizing and lust for power are hidden by his handlers.
"Mr. Griffith plays him with thunderous vigor," The New York Times wrote. Said The Washington Post: "He seems to have one of those personalities that sets film blazing."
Griffith said Kazan led him through his role, and it was all a bit overwhelming for someone with, as he put it, just "one little acting course in college."
"He would call me in the morning into his little office there, and he'd tell me all the colors that he wanted to see from my character that day," he recalled in 2007.
"Lonesome Rhodes had wild mood swings. He'd be very happy, he'd be very said, he'd be very angry, very depressed," he said. "And I had to pull all of these emotions out of myself. And it wasn't easy."
His role as Sheriff Taylor seemingly obliterated Hollywood's memory of Griffith as a bad guy. But then, after that show ended, he found roles scarce until he landed a bad-guy role in "Pray for the Wildcats."
Hollywood's memory bank dried up again, he said. "I couldn't get anything but heavies. It's funny how that town is out there. They see you one way."
More recently, Griffith won a Grammy in 1997 for his album of gospel music "I Love to Tell the Story -- 25 Timeless Hymns."
In 2007, he appeared in the independent film "Waitress," playing the boss at the diner. The next year, he appeared in Brad Paisley's awarding-winning music video "Waitin' on a Woman."
Griffith was born in 1926 in Mount Airy and as a child sang and played slide trombone in the band at Grace Moravian Church. He studied at the University of North Carolina at Chapel Hill and for a time contemplated a career in the ministry. But he eventually got a job teaching high school music in Goldsboro.
His acting career began with the role of Sir Walter Raleigh in Paul Green's outdoor pageant, "The Lost Colony," in Manteo. And he remained in the area even after superstardom knocked at his door.
Griffith protected his privacy by building a circle of friends who revealed little to nothing about him. Strangers who asked where Griffith lived in Manteo would receive circular directions that took them to the beach, said William Ivey Long, the Tony Award-winning costume designer whose parents were friends with Griffith and his first wife, Barbara.
Griffith helped Long's father build the house where the family lived in a community of bohemian artists with little money, sharing quart jars of homemade vegetable soup with each other.
Both Long and Fincannon recalled Griffith's sneaky tendency to show up unexpectedly -- sneaking into the choir at "The Lost Colony," or driving the grand marshals of the local Christmas parade incognito in his 1932 roadster convertible.
Fincannon described Griffith as the symbol of North Carolina, a role that "put heavy pressure on him because everyone felt like he was their best friend. With great grace, he handled the constant barrage of people wanting to talk to Andy Taylor."
He and his first wife, Barbara Edwards, had two children, Sam, who died in 1996, and Dixie. His second wife was Solica Cassuto. Both marriages ended in divorce. He married his third wife, Cindi Knight Griffith, in 1983.
"She and I are not only married, we're partners," Griffith said in 2007. "And she helps me very much with everything." When asked if the real Griffith was more wise like Sheriff Taylor or cranky like Joe, the diner owner in "Waitress," Griffith said he was a bit of both, and then some.
"I'm not really wise. But I can be cranky," he said. "I can be a lot like Joe. But I'm lot like Andy Taylor, too. And I'm some Lonesome Rhodes."
The Associated Press contributed to this report. All rights reserved.