At the beginning of the war, there were Northerners wearing grey, and Southerners wearing blue... BUT, the Confederacy did not get desperate enough to start thinking about using Blacks until the end of the war.
It is a well known photo of Black UNION troops, who were fighting against the Confederacy, and slavery.
At the end of the war, it was decided to train a small test unit of Black Confederate soldiers. They were never used, as the Confederacy had collapsed, and besides they did not want armed Blacks able to oppose slavery.
Funny you should mention that. After the surrender, the USA provided free transportation home, and supplies along the way to Confederate soldiers. Many have noticed that there were a few Blacks among them. The majority of the Blacks were women. Does this mean that there were a bunch of Black women Confederate soldiers... No, they were slaves that were used as support staff. They cooked for the troops, or similar things. There were no Black POW's during the war, because the Union troops did not consider any of the Blacks to be soldiers of their own choice. It was assumed their families were being threatened, and so the humane thing to do was to just let them go. After the war, they were considered soldiers to get home.