Reuters quoted a local official in Hudaydah as saying that the boat had come under attack by an Apache helicopter.
Images from the scene showed a boat strewn with bodies.
Millman told a United Nations news briefing in Geneva that he was unable to confirm news reports indicating that an Apache helicopter strike was responsible for the attack, which he said happened late on Thursday.
It is not immediately clear who carried out the attack. The amount of explosives this drone boat carries is not known, but the rebels are believed to have enough to threaten ships that pass through the strategic sea lanes off the Yemeni coast. The alleged incident triggered retaliatory airstrikes by the Saudi-led coalition, including a helicopter gunship assault on a fishing vessel that killed a number of fishermen hours before the strike on the migrant boat. There was no immediate comment from the coalition. The International Organization for Migration, which has operations in Yemen, said 42 bodies had been recovered.
The refugees, who included children, were hit in waters 30 miles off the southern coastline of Shia rebel-held Hodeida province, according to a Yemeni trafficker who survived the attack. The helicopter then stopped firing, he said, but only after dozens had been killed.
He went on to say the IOM is in contact with the hospital, clinics, and detention center in order to provide medical care to the survivors.
Oil prices fall after Opec stocks rise
Saudi Arabian Energy Minister Khalid Al-Falih discusses U.S. -Saudi relations, oil prices and OPEC production cuts . He said, "the market clearly believes that there are cracks in the Saudi commitment to the production cut deal".
Mohammed was not hurt in the attack.
The Saudi-led coalition began its campaign in March 2015 in support of Yemen's internationally recognized government.
Numerous wounded lost arms and legs in the strike, the Yemeni medical official told AP on condition of anonymity.
Most of the immigrants were coming back voluntarily from Yemen and heading towards Sudan, Somalia and other African countries, the security source said.
Among the dead on the boat were Somali refugees carrying identity documents issued by the UNHCR, the UN's refugee agency said. It called on all sides in Yemen's war to protect civilians.