Conmen agents worry ministry