5. Conclusion
Contribution of this work lays in the creation of an
open-source tool that will be usable in the scientic
sphere. We have demonstrated the applicability of
multi-agent approach enabling distributed crawling.
Based on measurements at the laboratory of VSB we
have tested and subsequently performed optimization
of tool for downloading data. The obtained results
show that the tool manages to download large amounts
of data (we have to take into account that the data size
depends on the size and number of tweets on Twitter).
During the testing, we have found that the optimal
number of concurrent crawlers varies between 30 and
40. Using more than 40 crawlers, relative performance
per crawler is decreased. This is due to the fact that
the manager can't handle processing requests for saving
data into the database and simultaneous