Hello Flash,
Sounds like you're making a good case for secret ballots.
It is partisan because members of Congress support their party's president. They don't support him any less if they are voting in secret. If they don't support him they can vote against him like Romney did. If the founders didn't expect it to be political they wouldn't have given the function to a political body.
A secret ballot might have changed Johnson's conviction but it would not have affected Clinton or Trump.