The Flow Collector and the Application Handler
share a common buffer on the hard disk. The Flow Collector uses
a component called the Disk Writer to write flows to the disk and
the Application Handler uses the Disk Reader to read flows from
the disk.
We make a design choice of storing the flows on disk and not
in volatile memory, for capacity reasons. The Disk Writer implements
a simple scheme using circular buffers as the storage mechanism.
The Disk Writer divides flows that it receives from the
Flow Collector into time intervals of 1 minute and writes them
as single files to the disk. The files function as a circular buffer
because Disk Writer overwrites the first file when a file number
limit is exceeded. We use a centralized locking mechanism with
the POSIX pthread’s atomic read and write operations to enforce
synchronization between the Disk Writer and Disk Reader on the
files of the circular buffer