Correct. Within a closed system, matter can't be destroyed. So, within our universe, there will never be any more or less matter than exists now.
Even in a closed system matter can be destroyed, can it not? It can be converted into energy and vice versa.