Perrone was arraigned early Thursday morning and ordered to be held without bail.
Sources say Perrone made admissions in connection with committing two of the murders. He stopped talking before any discussion of the third.
Investigators found a sawed-off .22-caliber rifle, a bloody knife and women's stockings in his apartment, sources said. Forensics evidence links the gun to one of the murders, they said.
"It's reasonable to assume he was going to keep doing this, and, by arresting him, we saved lives," Kelly said at a news conference.
Kelly called Perrone "talkative" with detectives and said he made incriminating statements. But Kelly refused to reveal what Perrone said.
Police said they believe Perrone carried the murder weapon in the bag and travelled by subway.
Police are still combing for more clues inside the 63-year-old's home in Staten Island. Perrone went without a struggle to the 67th Precinct after nervous residents spotted him in Bay Ridge.
They recognized him from surveillance video showing him walking near the scene of the most recent murder in Flatbush.
"A lot of business owners were scared," one shopkeeper said. "I get out of here at 12, 1, 2 in the morning, and we always worry about someone behind your back."
All three victims were Middle Eastern men who were killed with the same 22-caliber gun as they closed their stores alone, and in each case the suspect covered the victim's head with clothing or boxes.
Perrone has prior arrests in Pennsylvania for burglary, harassment and stalking.
