So if you abbreviate arithmetic then it's legitimate to use arith, not ariths. However it is not grammatically correct to use math for mathematics.
Who made you dictator of the English language? It's just a dialectical difference, that's all. If you are looking for inconsistencies, I promise you, there are far worse ones where the English language is concerned.
Anyway, to examine this further, wouldn't it really depend on the function of s in the word? If it were s, as in the free morpheme that can be attached to words to indicate plurality, then sure, you would usually add s in the abbreviation when that abbreviation also refers to something plural. You cannot, after all, eliminate it without also taking away the explicit plurality. If, on the other hand, the last consonant in the word just happens to end with an s, and it otherwise conveys no special meaning itself, why would you? Seems like you're just hypercorrecting.
So what is the function of the s in mathematics? Well, as your example succintly points out, it appears to be merely the end of the word, as you can't study a mathematic. So, no, you'd no more abbreviate it to "maths" than you'd abbreviate abbreviation to abbrevn - that's not how it works. The s is at best the nonfunctional remnant of s, the free morpheme. Americans would've discarded it because we got so used to thinking abouts mathematics in the singular, which really works just about as well as the plural for subjects (why not call history "the histories?") The British would've held onto the old pronunciation because that's just what they decided to do and there's no reason to make a fuss about it.
Well, that's my two theories anyway - the British are hypercorrecting or Americans came up with a different usage that makes just as much sense anyway.