During this time in the south, blacks were seen as cursed and looked down upon. They were treated unfairly.
As master of the L'Abri plantation, Armand is a strict master who treats the slaves harshly. The narrator says, that the “negroes had forgotten how to be gay.”
The fact that Désirée asks Armand should she leave also suggests that not only does she feel shame but that she is subservient to him. It is on his decision that the matter is resolved.