Actually, they did swear an oath to the President too. The President is commander-in-chief and as such is an "officer" in the US military.
The Oath of Enlistment for individuals joining the U.S. Armed Forces is as follows:
"I, _____, do solemnly swear (or affirm) that I will support and defend the Constitution of the United States against all enemies, foreign and domestic; that I will bear true faith and allegiance to the same; and that I will obey the orders of the President of the United States and the orders of the officers appointed over me, according to regulations and the Uniform Code of Military Justice".
I highlighted the important part for this discussion.