It is threatening to tell someone that you think they should be your slave or sent back to Africa.
Do you not think that is intimidating?
Legally? Only if you threaten to take some action (or take some action) to make them your slave or send them back. You can tell somebody anything you want unless you threaten them. You have committed no crime.