Why does it happen?
If the two balloons are almost the same size the larger one will be at a lower pressure.
As one gets smaller the pressure increases so it shrinks until it reaches its unstretched size.
Why is the pressure lower in the larger balloon?
There are two effects which cause this. The first is connected to the shape of the balloon. The small balloon is more curved so more of the tension in the rubber is compressing the gas, so for the same tension there will be a higher pressure in a smaller balloon. This is why blowing up long thin balloons is so difficult, the bit at the end is very curved so the pressure is very high.
If you think of a little bit of rubber and the direction that the tension is pulling in, on a curved piece of rubber the tension is pulling the rubber downwards compressing the gas, but on a less curved piece the rubber is mostly pulling against other rubber and not squashing the gas inside.