Residents of a small town in central Japan have overwhelmingly voted to remove the local council's only female member after she accused the mayor of sexual assault.