The senior official told Reuters the Taliban were coming in "from all sides" but gave no further details.
A tweet from the Afghan Presidential palace account said firing had been heard at a number of points around Kabul but that security forces, in coordination with international partners, had control of the city.
U.S. officials said the diplomats were being ferried to the airport from the embassy in the fortified Wazir Akbar Khan district. More American troops were being sent to help in the evacuations after the Taliban's lightning advances brought the Islamist group to Kabul in a matter of days.
Just last week, a U.S. intelligence estimate said Kabul could hold out for at least three months.
"Core" U.S. team members were working from the Kabul airport, a U.S. official said, while a NATO official said several EU staff had moved to a safer, undisclosed location in the capital.
A Taliban official told Reuters the group did not want any casualties as it took charge but had not declared a ceasefire.
There was no immediate word on the situation from President Ashraf Ghani, who said on Saturday he was in urgent consultations with local leaders and international partners on the situation.
Earlier on Sunday, the insurgents captured the eastern city of Jalalabad without a fight, giving them control of one of the main highways into landlocked Afghanistan. They also took over the nearby Torkham border post with Pakistan, leaving Kabul airport the only way out of Afghanistan that is still in government hands.
The capture of Jalalabad followed the Taliban's seizure of the northern city of Mazar-i-Sharif late on Saturday, also with little fighting.
"There are no clashes taking place right now in Jalalabad because the governor has surrendered to the Taliban," a Jalalabad-based Afghan official told Reuters. "Allowing passage to the Taliban was the only way to save civilian lives."
A video clip distributed by the Taliban showed people cheering and shout Allahu Akbar - God is greatest - as a convoy of pick-up trucks entered the city with fighters brandishing machine guns and the white Taliban flag.
After U.S.-led forces withdrew the bulk of the their remaining troops in the last month, the Taliban campaign accelerated as the Afghan military's defences appeared to collapse.
