Iteration 2
Push-based architecture: whenever events occur in the system, add them to the user’s "mailbox". When a client asks for updates, return the data that’s already waiting in the mailbox.
Pros: reads are much quicker since the data is already available.
Cons: might waste effort on moving around update data that will never be read. Requires more storage space.
There is still post-processing of updates before returning them to the user. E.g.: collapse 10 updates from a user to 1.
The updates are stored in CLOB’s: 1 CLOB per update-type per user (for a total of 15 CLOB’s per user).
Incoming updates must be added to the CLOB. Use optimistic locking to avoid lock contention.
They had set the CLOB size to 8 kb, which was too large and led to a lot of wasted space.
Design note: instead of CLOB’s, LinkedIn could have created additional tables, one for each type of update. They said that they didn’t do this because of what they would have to do when updates expire: Had they created additional tables then they would have had to delete rows, and that’s very expensive.
They used JMX to monitor and change the configuration in real-time. This was very helpful.