When it was implemented one argument against affirmative action was?

Many southern conservatives opposed Kennedy's Executive Order 10925, or Affirmative Action, to end discrimination in schools/workplaces. One major point is that it completely abolished the whole idea of meritocracy, a society governed by people with the knowledge and ability to 'rule'. It would destroy the idea of meritocracy by putting race as the superior determining factor in determining hiring and admissions.