For a finite universe, there is only a finite number of combinations of particles. Therefore, there is no way to represent even a single number beyond the set of all the integers up to that number of combinations.1 The axiom of infinity becomes pure myth exactly at the point where the platonic ideal of a number can no longer be reified. I don't believe that platonic ideals really exist.2
Objections: (1) What if the universe is infinite? (2) What if there could be an infinite number of particles in a finite space? And, (3), what if particles are positioned with infinite precision? I might be wrong. I have an extra-scientific feeling that the universe is finite, that there are a finite number of particles and that space is quantized. But I could be wrong, in which case, numbers could be infinite.
However, my second, weaker argument is that no one should ever care about an infinite number of numbers. Over all human history, past and future, only a finite number of numbers will be represented. An even small set will ever be economically useful.
Objection: What if human history is infinite? Not at the rate we're going.
1. The Bekenstein bound might support this idea.
2. In social situations, I tell people that I don't believe in numbers. It's a sharp conversation starter.