TSA 'Quiet Skies' program watches passengers without their knowledge

Passengers at airports are already used to the routine to take out your electronics so they can be inspected and walk through the body scanner or wait to get patted down.

But now there is word of a TSA program where passengers are secretly being watched, followed on the plane and in the airport without even knowing.

The program is called Quiet Skies and specifically targets U.S. citizens not in the terrorist screening database or suspected of wrongdoing, according to TSA.

"We're seeing examples of the TSA following around people who are in no way significantly suspect. The criteria that they are applying does not point to people who are potential terrorists," ABC News contributor Col. Steve Ganyard, a retired colonel and Marine Corps fighter pilot, said.

As first revealed to the Boston Globe, under the program, thousands of unsuspecting Americans have been watched by armed undercover air marshals.

The marshals are reportedly taking notes on what certain passengers do, who they talk to, and even if they use the bathroom.

Some of the behaviors of passengers they watch closely for include: excessive fidgeting or sweating, using a computer, a jump in their Adam's apple, a cold penetrating stare, or if they boarded last.

But John Casaretti, the Air Marshal Association president, is wary; he released this statement to the Globe:

"The Air Marshal Association believes that missions based on recognized intelligence, or in support of ongoing federal investigations, is the proper criteria for flight scheduling. Currently the Quiet Skies program does not meet the criteria we find acceptable."

TSA tells ABC News they follow individuals who may have suspicious erratic travel patterns and that this is not a surveillance program. However, sources within the agency tell ABC News marshals not only follow passengers in flight, but also once they land.