The New Orleans Saints quarterback said Goodell's ability to serve as "judge, jury and executioner" has been a "black eye" for the NFL.
"I think we would all agree that he definitely has too much power," Brees told SI.com. "He is judge, jury and executioner when it comes to all the discipline. I'm not going to trust any league-led investigation when it comes to anything. It's not transparent."
The subject came up several times Tuesday while Brees was doing an unrelated promotional blitz with several media outlets, and Brees didn't shy away from expressing his opinion.
"Forget the issues at hand here with Deflategate or whatever you want to call it," Brees told The Dan Patrick Show. "I think this was an issue again where the commissioner's authority was challenged and the league is gonna do whatever they can to make sure they know that he is in position to make these types of unilateral decisions, and there's nothing anybody can do about it."
Brees, formerly a longtime member of the NFL Players Association's executive committee, has been passionate about questioning Goodell's autonomy ever since the Saints were severely punished during the 2012 Bountygate case.
"We've been witness to this down in New Orleans firsthand," Brees told "The Rich Eisen Show." "There's just so much mistrust from the players' perspective and the fans' perspective. Nothing's transparent about it. It happens behind closed doors and you always feel like there's an agenda at play."
Goodell defended his authority Tuesday in an interview with Bloomberg TV, saying, "We think it's important that the commissioner protect the integrity of the game, that you can't entrust that to someone who has, really, no understanding of our business."
As for Monday's decision by a three-judge panel, Goodell said, "We're obviously pleased with the court's decision. We think that was the right decision. They were very firm in their decision that that was within our authority and the judgments were based on solid facts."
NFLPA executive George Atallah was equally critical of the NFL's disciplinary process while appearing on ESPN Radio's Russillo & Kanell on Tuesday.
"If we had a reasonable or logical partner, then this thing would have been settled forever ago," Atallah said of the lingering Deflategate battle. "If we have a reasonable and logical and fair disciplinary process, then four games for something as trivial as this, even if you believe that Tom did do it -- which there's still a lot of doubts, if not complete doubt about that -- is ridiculous.
"I'd love to be on this program talking to you guys about some more fun things like whether or not [Oklahoma City Thunder guard Russell] Westbrook is a really superstar or not. Yet here we are still talking about deflated footballs in one half of a playoff game that was a blowout. So what are you going to do?"
Atallah said the NFLPA is still reviewing the decision and weighing appeals options -- including an appeal to the full federal appeals court panel or to the Supreme Court.
The four Saints players who were punished during Bountygate ultimately had their suspensions vacated by former NFL commissioner Paul Tagliabue, who was appointed by Goodell to handle the appeals.
But no such process was available to Saints coach Sean Payton, who served a yearlong suspension. Other coaches and executives were also suspended, and the Saints lost two second-round draft picks.
"At the end of the day all those [players] were vindicated. But unfortunately the damage is done, their reputations took a hit, and the perception is there was something going on just because that's what the league drove down your throat with this," said Brees, who told Dan Patrick that the NFL's handling of the Bountygate investigation was driven by the environment at the time.
"The league was under such pressure about player health and safety," Brees said. "They were getting hammered about their negligence in regards to reporting the information that they had about the long-term effects of head and neck injuries. And this was right in the middle of the concussion lawsuits and a lot of the untimely deaths of so many players because of the effects of those injuries. And so they needed something that would show they were taking a very firm stance in regards to player's health and safety.
"So it was kind of a perfect storm where they had the opportunity to lead this investigation against the Saints, but kind of with a predetermined result."
Stephen A. to Brady: 'You're just a damn quarterback'
Stephen A. Smith shares his thoughts on Saints QB Drew Brees taking aim at the disciplinary power of NFL commissioner Roger Goodell after Patriots QB Tom Brady's four-game suspension was reinstated.
Atallah: Brady appeal should have been settled 'forever ago'
George Atallah of the NFLPA tells Russillo & Kanell that Tom Brady's suspension appeal would have been settled a long time ago if the NFL actually had a fair disciplinary process.
Would Brady benefit from accepting suspension?
ABC legal analyst Ryan Smith says Tom Brady would be taking a big risk by pushing his suspension appeal forward because he could miss more important games later in the season instead of the first four games.