OSINT, pt 2: Global Affairs and Speed & Accuracy

[00:00:00] In our last episode, we looked at the investigation using open source intelligence, OSINT for

[00:00:11] short, into the 2014 shooting down of a passenger jet by pro-Russian rebels in eastern Ukraine.

[00:00:19] We focused on the meticulous efforts of a Dutch based group called Belencat.

[00:00:24] In this episode, we'll hear from Belencat's chief operating officer on what else can

[00:00:30] be learned from open source intelligence.

[00:00:32] It turns out that one rather large organization was quite interested, the US Department of

[00:00:38] Defense.

[00:00:39] We'll also learn more about OSINT and the insidious craft of disinformation.

[00:00:49] I'm Paul Brandes and that's the name of this podcast series, Disinformation, a co-production

[00:00:55] of Evergreen Podcasting and Emergent Risk International, a global risk advisory firm.

[00:01:02] Later in this episode, I'll be joined by one of ERI's intelligence analysts, Noemi

[00:01:07] Masairo.

[00:01:09] Open source intelligence is a critical way of helping not just governments but the

[00:01:14] private sector form judgments about critical issues.

[00:01:18] But the chief operating officer of Belencat would actually prefer to avoid use of the

[00:01:24] word intelligence that COO Desi Demanova explains.

[00:01:29] Investigations, I would like to say here that we would prefer to use the word, we prefer

[00:01:34] to use investigations and not intelligence because it's a little bit different.

[00:01:39] So at the start, the very first days of Belencat actually were discovering things

[00:01:45] which are on social media and basically corroborating the narrative which was visible

[00:01:51] on the traditional media.

[00:01:55] And then we discovered through our work the potential of these open source methods

[00:02:02] that can contribute to traditional journalism.

[00:02:05] So nowadays, we call ourselves a collective that was a pioneer in a way in 2014 for

[00:02:14] open source methods to be used as additional methods for traditional journalism.

[00:02:20] So a lot of media now see us as being the first ones to discover this potential and

[00:02:28] we are also very happy that one of our first employees have now been hired by New

[00:02:33] York Times, by the BBC.

[00:02:35] So basically we have been some sort of an incubator for talent in that area.

[00:02:40] So we consider maybe that to be our modest contribution in the field of journalism

[00:02:47] but also now we apply our methods in a lot of other fields in collaboration with

[00:02:53] other actors. So that is also contributing, we think, to society, we hope.

[00:03:04] Russia's war on Ukraine well into its third year is a prime example of how it is

[00:03:10] contributing. Open source intelligence or in Dmanova's preferred parlance

[00:03:15] information has allowed analysts unaffiliated with any government to track the war by

[00:03:21] studying everything from satellite imagery, phone calls, social media and more.

[00:03:27] In a way you might say that intelligence gathering and analysis has now been

[00:03:32] democratized. But having so much information out in the open for any skilled analyst

[00:03:38] can be a double-edged sword. Dmanova offers this example of how one Bellingcat analyst

[00:03:45] discovered something that theoretically could have placed American troops, even the security

[00:03:51] of nuclear weapons, at risk. It seems that some service members were logging on to

[00:03:57] Quizlet, a California based company that provides online tools for studying and

[00:04:03] learning.

[00:04:04] So one of our researchers who comes from educational backgrounds, he has a very keen

[00:04:14] interest in looking for how applications on internet are used to memorize languages,

[00:04:24] for example Quizlet. So at some point he was looking into Quizlet and discovered

[00:04:33] an interesting pattern that triggered his further curiosity. So he discovered that

[00:04:40] usually Quizlet is used for people to learn new languages to memorize words. But he

[00:04:46] discovered a pattern that he saw a lot of data on Quizlet being numbers. So

[00:04:53] he thought like, to me that doesn't make a lot of sense so I just want to dig further.

[00:04:59] So he started digging further and he saw more instances like this appearing on Quizlet.

[00:05:06] And then he found that the people using that did not have their privacy settings

[00:05:11] on closed but on open. And then he discovered the people behind, and then these were

[00:05:16] American soldiers. He found their profiles on Facebook because they were also open.

[00:05:23] And then what he discovered is that these American soldiers were using Quizlet to

[00:05:27] memorize codes of nuclear volts in the Netherlands where American nuclear weapons are

[00:05:35] based, also guarded. So this was everywhere on internet for everyone to see.

[00:05:42] And this is not misinformation or disinformation. I mean the intent was just to

[00:05:47] use something to memorize very complicated or a lot of data. But so the investigation

[00:05:55] became bigger and bigger, and then what we found out was actually quite dangerous because

[00:06:01] inadvertently the codes but also the places of the security cameras and also the badges and

[00:06:07] a lot of very, very highly sensitive security details were just like in the open internet.

[00:06:13] So we did the investigation and before publishing, because this is like a big scoop,

[00:06:19] so we approached the Dutch Ministry of Defense and also the Pentagon and said look we have

[00:06:24] this information what to do about this. So they were like really shocked that we could

[00:06:30] discover that in such a way. Also took us one investigator and like I don't know a month

[00:06:36] with very limited resources. All of that uncovered in about a month by one analyst

[00:06:43] with limited resources. Imagine what Russia, China or someone else might be able to do.

[00:06:49] Dymanova continues her story. So they had to change the codes. So this story when we

[00:06:55] published it after the codes were changed, generated like an enormous also world coverage.

[00:07:02] It was even published in Kiribati and I don't know translated in so many languages. So

[00:07:06] this is just an illustration of what the potential of open source is. Not only to

[00:07:12] discover wrongdoing but just how real curiosity, a combination of curiosity,

[00:07:21] freedom of our researchers to do, to research what they are interested in.

[00:07:25] And not being limited by time or by any assignment for something to research. Of course,

[00:07:34] it's very simple maybe very simple but there's a combination of things which

[00:07:40] actually describes or defines our methods. So we use open source.

[00:07:47] We use open source to discover all kinds of things.

[00:07:50] All kinds of things indeed. And perhaps something that governments cannot do but

[00:07:55] private analysts can is to crowd source ideas and research which can yield results in some

[00:08:02] very important ways. For us what is important is how we contribute to this public interest

[00:08:08] because you can have different definitions. But then what the difference is how somebody

[00:08:13] contributes to that public interest. So in our goals that's why we have decided actually it's

[00:08:18] a choice not to be a company but to be for non-profit and also a charity. So this

[00:08:25] choice means that our purpose in what we do is we want to use our skills to contribute

[00:08:34] to public good in our area using open source to discover wrongdoing or to discover effects

[00:08:42] which can help further investigation or uncover crimes or shed light on war crimes or human rights

[00:08:52] abuses. Let's take a short break here when we come back more on the war in Ukraine and also

[00:09:01] Israel and how the craft of OSINT can help safeguard employees and business operations

[00:09:07] in potentially dangerous places will be joined by Noemi Masairo of Emergent Risk International.

[00:09:17] This series on disinformation is a co-production of Evergreen Podcasts and Emergent Risk

[00:09:22] International, a global risk advisory firm. Emergent Risk International,

[00:09:26] we build intelligent solutions that find opportunities in a world of risk.

[00:09:30] I'm Allison Holland, host of the Kennedy Dynasty podcast. Equipped with a microphone and a long-term

[00:09:42] fascination of the Kennedy family, I am joined by an incredible cast of experts, friends,

[00:09:47] and guests to take you on a fun, relaxed, yet informative journey through history and

[00:09:51] pop culture. From book references to fashion to philanthropy to our modern expectations of

[00:09:56] the presidency itself, you'll see that there is so much more to Kennedy than just JFK or

[00:10:01] conspiracy theories. Join me for the Kennedy Dynasty podcast.

[00:10:09] Welcome back to Noemi Masairo of Emergent Risk International is a GSOC lead based in Portugal

[00:10:16] who offers an example of how she uses open source intelligence on behalf of a client

[00:10:22] which operates in Ukraine. With my team, because of the duty of care that we owe to our

[00:10:31] client, we have to ensure the security and safety of our employees and travelers in Ukraine.

[00:10:42] As you can imagine, the situation in Ukraine and getting information about incidents in Ukraine

[00:10:50] is not that easy. We have realized that an excellent source is the Telegram account of

[00:10:57] the Kib's mayor Vitaly Kishitko that we use on a daily basis as soon as we have

[00:11:05] information about siren activation. So, we try to always find new sources that are reliable,

[00:11:13] especially for situations that are plagued with myths and disinformation. But Vitaly

[00:11:21] Kishitko has proven to be an excellent source for us. As I was saying, we always try to strive for

[00:11:29] increasing the amount of local sources that we get.

[00:11:35] And he and several other mayors have been excellent sources for us.

[00:11:41] And the main source has been really their Telegram account. It's up to date

[00:11:49] immediately with all the information about the incident development.

[00:11:55] I can really see that he is the best source because he's then also used by other

[00:12:02] international media outlets that I consider as reliable. So, it crystallizes the level of

[00:12:09] reliability of a source, I would say. Just to make sure I understand,

[00:12:13] ERI has clients that have various operations in Ukraine and the mayor of Kyiv,

[00:12:22] his Telegram account is a principal source of information there. What other tools do you find,

[00:12:29] what other OSINT-related tools do you find useful?

[00:12:33] I use a lot Twitter. And for the same reason as the one that I just detailed before,

[00:12:39] I tend to perform the research in local languages because it ends up providing

[00:12:48] that immediate information from local sources. And then hopefully, it's then also verified by

[00:12:56] other international media outlets that we consider as reliable. But oftentimes, you get

[00:13:01] that first input from local sources, I would say. To do so, you have to use some Google dorks. You

[00:13:09] apply them to Twitter. You can decide that you want to make a research just in a specific

[00:13:15] language by using the Google dork lang double dot and then the code of the language.

[00:13:23] Otherwise, you can directly input words in that language. So, for example, if I use,

[00:13:30] for example, for the situation in Israel, sometimes, oftentimes we get information about

[00:13:35] terrorist attack in Israel. And rather than looking at the name written in

[00:13:47] the language we translated in English, I would go and look for it in Hebrew. And if you

[00:13:53] take the word of the name of the city written in Hebrew and then you input it directly into it,

[00:14:01] it's likely that you're going to get way more information than if you perform a research

[00:14:04] in English or in whatever other language. Of course, one problem with Twitter or X,

[00:14:10] as it's officially called, is that safety and verification standards have eroded since the

[00:14:16] company was purchased by Elon Musk. It is easier for false narratives to gain traction,

[00:14:22] obviously complicating the job for any analyst.

[00:14:27] It's a big, big part of open source intelligence, right? Is that it isn't used both together,

[00:14:34] but it has to also be verified because otherwise we are just going to be part of

[00:14:40] disinformation in this case. And we really want to avoid that for our clients and for

[00:14:48] the broader reputation of ERI. Sometimes it can still happen, right? Because we work

[00:14:53] in a very fast-paced environment and sometimes you don't have the luxury to

[00:14:58] ensure that the information is accurate 100%. What I do is that I make sure that we don't

[00:15:07] surf on like, you know, when there is an incident going on, it's really hard to make sure

[00:15:15] that you're accurate reporting on it. So what I do is that I try not to,

[00:15:24] I try to avoid as much as possible sentimentalist language, first of all,

[00:15:29] when I report it. Because for sure there's going to be things that, incidents that afterwards

[00:15:36] are going to be very different than when we initially reported on them, right? Because

[00:15:40] this is the incident as a trajectory. And so afterwards we're going to maybe discover

[00:15:45] that this incident was a terrorist attack, it was done by the ISIS-K, etc., etc. So if

[00:15:51] initially I need to report immediately on the client and I don't have the luxury to ensure

[00:15:56] that this is 100% accurate, so what I'm going to do is that I'm going to really try to

[00:16:01] make it as neutral as possible, avoid saying who did it, if it was terrorism or not. I'm

[00:16:07] going to try to use a language that is as neutral as possible and it ensures that even

[00:16:12] if it's read weeks afterwards, it's still going to be somehow accurate in the sense

[00:16:17] that I didn't say something, that I didn't give responsibility to someone that ended up

[00:16:24] not being true, if that makes sense. Everything to say that unfortunately we don't always have the

[00:16:30] time because we live in this super fast-paced environment. We handle tactical events rather

[00:16:37] than the strategic ones. The strategic ones you have the time to ensure that the information

[00:16:41] you're providing is correct with the tactical ones, with clients and stakeholders that are

[00:16:46] waiting for you to provide a report. Oftentimes you don't have it, so what you can do is that

[00:16:51] you can just ensure that you are providing something that is neutral.

[00:16:57] So there's a bit of a conflict then between speed and accuracy. You want both,

[00:17:05] but sometimes it's not possible to have both. How do you thread that needle? Obviously your

[00:17:11] clients need information quickly, but it has to be accurate and you said that it's impossible

[00:17:19] to have 100% accuracy. I think you have to have a confidence level of a certain degree

[00:17:27] to pass something on to your client. What is that confidence level for you? In other words,

[00:17:32] can you say we have a 95% confidence level in this or something like that? How do you

[00:17:40] thread the needle between speed and accuracy? So one thing that is super important that is

[00:17:47] part of all of this is of course knowing your region, knowing the global risks,

[00:17:55] knowing quite in depth what's happening at a local level to ensure that

[00:18:04] when you are providing an analysis about an incident it fits what you know about that country.

[00:18:10] It's like part of the deal is that you have to know what an anomaly is and that you have to

[00:18:17] have a good understanding of the country you're reporting to ensure that it's not weird that

[00:18:22] I'm reporting a terrorist attack in that specific region. An information that is completely out of

[00:18:29] everything I haven't ever known would be something I would tend to take longer time

[00:18:38] to report if that makes sense. I would take more time to find sources that confirm that.

[00:18:43] Whereas if it fits and it's part of the disinformation, sometimes if it fits something

[00:18:52] you know about the country it can also be a risky slippery slope. But yeah, I would say that

[00:18:58] to handle these two contradictory pressures, the time and the accuracy,

[00:19:11] I tend to spend a lot of time training the team on understanding the global risks

[00:19:18] and the ones that are likely to impact us. And then we take chances. It has never happened

[00:19:27] to us. We tend to maybe keep it shorter rather than providing more information. This is also

[00:19:34] where sometimes it gets tricky if you provide more information about an incident that has just

[00:19:39] happened and then it's just easier to keep it short. If you keep it short and simple,

[00:19:45] the likelihood of the incident to become completely different from what you had described

[00:19:54] is lower. Particularly with say a fast-moving incident where conditions can change rapidly,

[00:20:03] it seems like it's best to be cautious and as you just said provide less is more. And

[00:20:10] you can always add on additional information later as you confirm things. Very interesting.

[00:20:17] And this is something that we do a lot. We have part of our procedure is to provide updates

[00:20:23] about the report. So we send this initial report to the stakeholders, hey guys, we know

[00:20:30] this is happening, we're taking care of it, we're performing, we're being checked,

[00:20:33] we're reaching out to the travelers, we are handling everything. And then after a while,

[00:20:37] when the situation sets and when the incident potentially curves down and is reabsorbed,

[00:20:44] then we provide another either strategic piece or another alert, in any case some sort of

[00:20:50] closure to the business. Thanks to Desi Dmanova, the Chief Operating Officer at Bellingcat,

[00:20:58] who spoke at a disinformation conference at Cambridge University in England. Also thanks

[00:21:04] to Doemi Masairo, a GSOC lead for Emergent Risk International. Our sound designer and editor,

[00:21:11] Noah Fouts. Audio engineer, Nathan Corson. Executive producers, Michael DeAloia and Gerardo

[00:21:18] Orlando. And on behalf of Meredith Wilson, the CEO of Emergent Risk International,

[00:21:25] I'm Paul Brandes. Thanks so much for listening. This is Peter. And this is Tom. We want to tell

[00:21:45] you guys a little bit about our podcast. Tom and I met in college, became best friends,

[00:21:49] and then teachers almost 20 years ago. Sometimes school just does not allow us to elaborate

[00:21:54] on the topics that we find interesting, like the real shark attacks that inspired the movie

[00:21:58] Jaws, or the real historical context to Indiana Jones artifacts. Where does cereal come from?

[00:22:03] Or are zombies real? Does Ben Franklin really deserve to be on a hundred-dollar bill?

[00:22:07] On our podcast, just like in our class, there are no stupid questions.

[00:22:11] Just two brands having lighthearted conversation about history,

[00:22:14] pop culture, and the context of current events.

[00:22:16] Listen to History Teacher's Talking Podcast from Evergreen Network, anywhere you get your

[00:22:19] podcast.