The Game Coordinator will be down for maintenance starting around 4:00PM PST tomorrow (0:00 GMT). Any games in progress will complete, but forming new matches will not be possible. Additionally, for a period of time after the Game Coordinator comes up, match results will be significantly delayed, including quest progress. If everything goes according to plan, the Game Coordinator will be down for less than an hour, and the period of time when new match results are not available will also be about an hour. We'll keep you up to date on twitter.
A long time ago, we thought it'd be safe to store Match IDs in 32-bit integers. "It will be forever before Dota players play 4.3 billion matches!" we thought. "We've got plenty of time! Think of all the bits we'll save!" As it turns out, that was optimistic. Our SQL server does not have an unsigned 32-bit data type, only a signed one, which means the Match ID can really only go up to 2^31 before it looks like a negative number to SQL. That cut the Match ID namespace in half. Also, we made some changes that greatly accelerated the rate of exhaustion of the namespace, such as assigning the ID as soon as the match came out of matchmaking, where it would be consumed even if the match was never actually played, and assigning all private lobbies and even offline bot matches a Match ID. So the time when a 32-bit Match ID wasn't big enough came far sooner than we thought.
Actually, that time is right now. Did you know that the current largest Match ID is just over 2.1 billion, which is really close to 2^31?
We've been working on changing all of our infrastructure to use 64-bit Match IDs for a while now. SQL is usually great about changing the size of a column in a table. It can often do it instantly. Old rows store the column in the old data format, and as new data gets written, it gets promoted to the wider size. However, if that column is part of the table's primary key, or there's an index on the table that includes that column, then you cannot do it that way; you have to rebuild the table. We have a lot of tables with a Match ID in the primary key or an index. Fortunately most of them weren't too big, so they could be rebuilt relatively quickly.
However, the tables that store the match data itself are pretty big, about 17 billion rows and 6TB of data. It took about a week just to copy the data from the old tables into the new ones. We also took this opportunity to restructure the tables to make them more efficient.
So during the downtime, what we're doing is copying over the last few hours of matches from the old tables into the shiny, new tables with 64-bit Match IDs, and then we'll start using those new tables. When we bring the Game Coordinator back up, we'll be checking to make sure everything is working properly.
We'll see you again in 2038!