By intelligence, most people mean a kind of spying or data theft. However, in today’s age, the gathering of information can best be done from open sources. And this process is called OSINT. This acronym stands for Open Source Intelligence.
OSINT means obtaining information and data that can be accessed without restrictions.
Why is intelligence important?
The purpose of collecting news, data and additional text contents is always to stay up-to-date and always have the latest information. This is the key to gaining or even keeping our market or business advantage and to be able to make the most appropriate decision in every single case.
Areas of intelligence
In terms of how the information obtained is utilized, we distinguish between business (CI – Competitive Intelligence) and government intelligence.
Main users of OSINT
The range of users using OSINT is extremely wide:
- business enterprises
- journalists, investigators
- financial institutions
- state and government agencies
Use cases for OSINT
Open source intelligence can be used in many cases:
- competitor monitoring, social listening and sentiment analysis
- KYC – Know Your Customer, risk analysis, compliance
- investigation, investigative journalism
- civil and military intelligence
- collecting news
- collection of procurement data
How do we get information?
Collecting relevant content can be done in several ways. While document management systems usually provide assistance in managing existing data (files), collecting web content requires data and text mining solutions.
The data mining solution developed by Precognox is TAS Data Collector, which is able to download unstructured data (text contents) found on the Internet. What’s more, it arranges the contents in a structured form, thereby making them available to other data management systems. This makes it suitable for further processing, analysis or visualization.
PAI as the basis of OSINT
All the data meaning the basis of OSINT as a process is called PAI (publicly available information). This information can come from many digital or other sources.
How can we process the collected and thus already available information?
We can use the collected data (text contents) in a number of ways. It happens that the user opens the documents for the sole purpose of reading or editing, but in the vast majority of cases new content is generated by using them (summary, article, presentation or a visualization). This can be achieved by individual work, but the contents are often processed by an entire team. This could be, for example, an expert group consisting of accountants, analysts, statisticians or data scientists. The way of processing is almost endless with the development of technology and the appearance of new methods and softwares. The range is very wide, from data visualization solutions (Business Intelligence tools, Analyst’s Notebook), through presentation apps to knowledge base softwares.
After data collection and cleaning, the important steps towards our business goal are always filtering and searching.
Why is advanced and intelligent search essential?
Exploiting the potential inherent in the collected contents is only possible if the information contained can be available and found. And this can only be achieved with an intelligent search tool that meets all the requirements of the given task.
Advanced search engines, such as the TAS Enterprise Search engine, offer a number of additional services beside the usual functions (advanced search, filtering, and sorting options), enabling a truly sophisticated search process.
The entity recognition and name matching service that is part of the Rosette text analytics platform can also be integrated. These solutions can also be used with great efficiency in the field of security service, where the exact identification of persons (names) is of paramount importance. Such a live project implemented by us has already been proven and is currently operating successfully at an organization in the security field in Hungary.
In addition to the solutions mentioned above, two additional modules that can be connected to the TAS Enterprise Search engine are also available. TAS Thesaurus Manager provides an even more effective search, while the TAS Search Log Analyzer helps users track queries compiled formerly.
Intelligent search is really a complex process, therefore it is important that the solution provides the user the most optimal environment, all with a high degree of integration and customization.
Intelligent search with TAS Enterprise Search engine
https://www.youtube.com/watch?v=XJeLcgz9GUw&t
Is all this necessary for OSINT?
Yes. Open source intelligence is not only about gathering data, news or information, but about the efficient processing of these contents, whether it is about any mentioned data processing process, business or governmental goals of use.
Together, the solutions in this article mentioned provide a comprehensive solution for implementing effective open source intelligence.
