At the weekend, the sign at the ministry was removed, and a sign for the ministry of virtue and vice was put in its place.
Videos on social media showed women employees outside the offices, urging the Taliban to let them return to work.
During the Taliban rule in the 1990s, the ministry forced strict Islamic rules and harsh restrictions on women.
In the last 20 years, Afghan women have fought for and gained a number of basic rights, but there are now fears that the Taliban’s new all-male interim government is upending progress.
The BBC’s chief international correspondent Lyse Doucet, in Kabul, says that while Taliban leaders have made promises that they understand Afghanistan has changed – and that so have they – there seems to be a growing mismatch between promises and policies.
Human rights groups have previously criticised the virtue and vice ministry for silencing dissent, violently enforcing restrictions on citizens – especially women and girls – and spreading fear and distrust throughout communities.
But Taliban members say the institution is important: “The main purpose is to serve Islam. Therefore, it is compulsory to have [a] Ministry of Vice and Virtue,” a Taliban member, Mohammad Yousuf, told the New York Post.
Its full name, according to the new sign on the Kabul compound, is the Ministry for Propagation of Virtue and Prevention of Vice.
It was already in existence before the Taliban first came to power, but it expanded during their rule between 1996 and 2001.
The ministry was responsible for deploying so-called morality police onto the streets to enforce the Taliban’s strict interpretation of Islamic religious law, known as Sharia.
It became known for beating women who it perceived were dressed “immodestly”, or who were outside without a male guardian. Girls were not allowed to be educated past primary school – a measure the group has now reportedly reintroduced.
Entertainment like music and dancing was banned, and activities such as playing chess or flying a kite were prohibited. Prayer times were strictly enforced, men were made to grow beards, and Western-style haircuts were frowned upon.
Anyone found to have violated the rules was harshly punished – flogging, beatings, amputation and public execution were not uncommon.
Two Taliban members in Kabul told The Washington Post newspaper that they did not expect the Taliban to use force in the same way the group had in the past, and that its enforcers would not be police or soldiers.
The ministry was disbanded after the US-led invasion of Afghanistan in 2001, although then-President Hamid Karzai re-established a similar but less powerful department in 2006 following pressure from conservatives.
At the time, Human Rights Watch called the ministry a “notorious symbol of arbitrary abuses”.
Female staff who worked at the women’s affairs ministry compound said they had been trying to come back to work for weeks, but had been locked out of the building.
“For women, nothing will be there anymore,” one worker said. “All of us have responsibilities for our families… We are educated and do not want to confine ourselves to the house.”
Taliban officials have said that under their new government, women will be allowed to study and work in accordance with the group’s interpretation of Islamic religious law.