"I haven't seen evidence that this is a rampant crime," says James O'Neil, the assistant inspector general for investigations.
But Jeff Hinton, a retired Army master sergeant who exposes Green Beret frauds, says that's just the "tip of the iceberg."
"We come across this quite frequently," Hinton says. "It's extremely prevalent."
Hinton says while he supports the efforts of the congressmen in creating new legislation to go after the liars, he's not so sure about the title of the bill.
"They shouldn't call it 'Stolen Valor.' They aren't stealing valor from us. They are defrauding the taxpayers and the American people."
Mary Schantag, chairman of the POW Network, a group that both boasts the accomplishments of heroes and busts of what she calls "fakes and phonies," says the real shame is that even after people are revealed to be liars, often they aren't ever held accountable in a court of law.
"Many of these guys are not getting prosecuted for what they are doing," Schantag says. "They are stealing money from the VA, they are taking jobs from real veterans, they are writing books, and they are leading the parades when a real veteran doesn't."
Schantag says her organization, which started out as an oral history project more than two decades ago, has transformed into a truth-seeking mission.
"It's all we do every day, all day, even on Fourth of July," she says.