NYPD intelligence chief David Cohen, a former top CIA official, was asked about that in September 2005 during a deposition in a lawsuit over the department's policy of randomly searching the bags of subway riders. Civil rights lawyers asked how police knew whether a program deterred terrorism.
"If it works against them, then it works for us," Cohen replied. "That is deterrent to one degree or other."
Cohen was asked, How do you know it works? Is there some police methodology?
"I never bothered to look," Cohen said. "It doesn't exist, as far as I could tell."
At times, police officials themselves have raised concerns about intelligence-gathering programs. In about 2008, for instance, police began monitoring everyone in the city who legally changed names. Anyone who might be a Muslim convert or appeared to be Americanizing his or her name was investigated and personal information was put into police databases.
Current and former officials say it produced no results. Police still receive the list of names of people who change their names, court officials said. But one official said the program is on hold while its effectiveness is evaluated.
Kelly has said the NYPD does not trawl neighborhoods and instead only pursues leads. But those leads can be ambiguous, officials say, and can be used to justify widespread surveillance programs.
For example, the NYPD began the "Moroccan Initiative," a secret program that chronicled Moroccan neighborhoods, after suicide bombings killed 45 people in the Moroccan city of Casablanca in 2003, and after Moroccan terrorists were linked to the 2005 train bombing in Madrid. New York police put people, including U.S. citizens, under surveillance and catalogued where they ate, worked and prayed.
"What we were doing is following leads," Kelly told City Council members during an October hearing when asked about that program. "The Moroccan issue that was mentioned had to do with a specific investigation." [See a collection of political cartoons on WikiLeaks.]
But officials involved in the program said there was no specific threat to New York from Moroccans. The Moroccan Initiative thwarted no plots and led to no arrests, officials said.
Much of the information in the Moroccan Initiative was gathered by a secretive squad known as the Demographics Unit. Using plainclothes officers known as "rakers," the squad infiltrated local businesses and community organizations looking for trouble or "hot spots." Their daily reports helped create searchable databases of life in New York's Muslim neighborhoods.
One NYPD official said that unit identified a Brooklyn bookstore as a hot spot. That led police to open an investigation and send in an informant and undercover detective, ultimately leading to the arrests of two men in the Herald Square case.
The work of that secret unit, the official said, helped the NYPD arrest a Pakistani immigrant named Shahawar Matin Siraj and foiled an attack.
For years, police have said publicly that the Herald Square case began with a tip but have not elaborated. Siraj's lawyer, Martin Stolar, said prosecutors provided no documents related to the Demographics Unit at trial.
Siraj was convicted and sentenced to 30 years in federal prison in 2007. But defense attorneys, and even some inside the NYPD intelligence unit, said police had coaxed the men into making incriminating statements and there was no proof Siraj ever obtained explosives.
The case is arguably the NYPD's greatest counterterrorism success. But there are others.
The NYPD played an important role in the case against Carlos Amonte and Mohammed Alessa, two New Jersey men who pleaded guilty to charges they tried to leave the country in 2010 to join the al-Qaida-linked terrorist group al-Shabaab. The FBI long had been aware of the two men but had been unable to win their trust with an informant or undercover agent, federal officials said. The NYPD, with its deep roster of Muslim officers, provided the undercover officer who ultimately succeeded in winning their confidence.