9th Amendment: "The enumeration in the Constitution, of certain rights, shall not be construed to deny or disparage others retained by the people."
"Others retained by the people" refers to rights not delineated in 1-8. If there are others retained by the people, what do you think they might include? Travel, having children, privacy? Privacy is not specifically listed but the 4th protects our privacy from unwarranted government searches and seizures of our property, the 1st amendment freedom of association protects your privacy from associating with people you don't like (in your case black people), and the 5th protects you from incriminating yourself.
Although the term "privacy" is not specifically in the Constitution, anyone who doesn't want their privacy protected truly favors an oppressive government and limited personal freedom.
No, those that use the 9th amendment in the manner you support want an oppressive government. When you expect the government to provide you things you should be providing yourself, when that happens, they control you. If you think that's not oppressive and limits freedoms, keep pushing for it. You'll find out the hard way you'll get what you don't want.