The imam, father-of-seven Maulama Akonjee (55) and Thara Uddin (64) were shot in the head as they left the Al-Furqan Jame Masjid mosque in the Ozone Park section of Queens shortly before 2pm on Saturday.
Both men were later pronounced dead.
The gunman approached the men from behind and shot both in the head at close range shortly before 2pm local time on a blistering hot afternoon in the Ozone Park neighborhood, police said in a statement, adding that no arrests had been made.
The motive for the shooting was not immediately known and no evidence has been uncovered that the two men were targeted because of their faith, said Tiffany Phillips, a spokeswoman for the New York City police department. Even so, police were not ruling out any possibility, she added.
The victims were both wearing religious garb at the time of shooting, police said.
The Council on American-Islamic Relations, a Muslim civil rights and advocacy group known by the acronym CAIR, said Mr Uddin was an associate of the imam.
“These were two very beloved people,” Afaf Nasher, executive director of the New York chapter of CAIR, told Reuters. “These were community leaders.
“There is a deep sense of mourning and an overwhelming cry for justice to be served,” Nasher said. “There is a very loud cry, too, for the NYPD to investigate fully, with the total amount of their resources, the incident that happened today.”
The organisation held a news conference on Saturday evening in front of the mosque, the Al-Furqan Jame Masjid, where the two men had prayed.
“We are calling for all people, of all faiths, to rally with compassion and with a sense of vigilance so that justice can be served,” Mr Nasher said. “You can’t go up to a person and shoot them in the head and not be motivated by hatred.”
The suspect was seen by witnesses fleeing the scene with a gun in his hand, police said.
“We are currently conducting an extensive canvass of the area for video and additional witnesses,” deputy inspector Henry Sautner said in a statement.
Eric Phillips, a press secretary for New York City Mayor Bill de Blasio, said the mayor was closely monitoring the police investigation into the shootings.
“While it is too early to tell what led to these murders, it is certain that the NYPD will stop at nothing to ensure justice is served,” Phillips said in a statement.
Mr Akonjee was described as a peaceful man who was beloved within Ozone Park’s large Muslim community.
“He would not hurt a fly,” his nephew Rahi Majid (26) told the New York Daily News. “You would watch him come down the street and watch the peace he brings.”
“We feel really insecure and unsafe in a moment like this,” Millat Uddin, an Ozone Park resident told CBS television in New York. “It’s really threatening to us, threatening to our future, threatening to our mobility in our neighborhood, and we’re looking for the justice.”
In June, CAIR issued a statement calling for Muslim community leaders to consider increasing security after the Orlando massacre and incidents that it said had targeted Muslims and Islamic houses of worship.
A gunman who called himself an “Islamic soldier” killed 49 people in an Orlando, Florida, nightclub on June 12th. –Agencies