Four staffers for Justin Fairfax quit amid sexual assault controversy