NGA Knows Its Challenges, Now It Needs the Tech to Address Them

In the 1983 movie WarGames, the world is brought to the edge of nuclear destruction when a military computer using artificial intelligence interprets false data as an indication that a Soviet missile strike is imminent. Its human overseers in the Defense Department, unsure if the data is real or not, can’t convince the AI that it may be wrong. A recent finding from the Defense Intelligence Agency, or DIA, suggests that in a real situation where humans and AI were looking at enemy activity, those positions would be reversed. Artificial intelligence can actually be more cautious about its own conclusions in situations when data is limited than are humans. While the results are preliminary, they offer an important glimpse into how humans and AI will compliment one another in critical national security fields.

DIA analyzes activity from militaries around the globe. Terry Busch, the technical director for the agency’s Machine-Assisted Analytic Rapid-Repository System, or MARS, Monday joined a Defense One viewcast to discuss the agency’s efforts to incorporate AI into analysis and decision-making.

Earlier this year, they set up a test between a human and AI. The first part was simple enough, use available data to determine if a particular ship was in U.S. waters. “Four analysts came up with four methodologies; and the machine came up with two different methodologies and that was cool. They all agreed that this particular ship was in the United States,” he said. So far, so good. Humans and machines using available data can reach similar conclusions.

The second phase of the experiment tested something different: conviction. Would humans and machines be equally certain in their conclusions if less data were available? The experimenters removed the feed from the Automatic Identification System, or AIS, which tracks ships worldwide. “It’s pretty easy to find something if you have the AIS feed, because that’s going to tell you exactly where a ship is located in the world. If we took that away, how does that change confidence and do the machine and the humans get to the same end state?”

In theory, with less data, the human analyst should be less certain in their conclusions, like the characters in WarGames. Afterall, humans understand nuance and can conceptualize a wide variety of outcomes. The researchers found the opposite.

“Once we began to take away sources, everyone was left with the same source material — which was numerous reports, generally social media, open source kinds of things, or references to the ship being in the United States — so everyone had access to the same data. The difference was that the machine, and those responsible for doing the machine learning, took far less risk — in confidence — than the humans did,” he said. “The machine actually does a better job of lowering its confidence than the humans do…. There’s a little bit of humor in that because the machine still thinks they’re pretty right.”

The experiment provides a snapshot of how humans and AI will team in the future for important analytical tasks. But it also reveals how human judgement has limits when pride is involved.

Humans, particularly those who are experts in specific fields, have a tendency to overestimate their ability to correctly infer outcomes when given limited data. Nobel-prize winning economist and psychologist Daniel Kahneman has written on the subject extensively. He describes this tendency as the “inside view.” To illustrate it, he refers to a case involving a group of Isreali educators faced with the daunting challenge of writing an entirely new textbook for the Ministry of Education. They anticipated that it would take them a fraction of the amount of time they knew it would take another, similar team. They couldn’t explain why they were overconfident; they just were. Overconfidence is human and a particular trait among highly-functioning expert humans, one that machines don’t necessarily share.

The DIA experiment offers an important insight specifically for military leaders, who are looking for AI to help make faster and better decisions , from inferring enemy positions to predicting possible terror plots. The amount of intelligence data coming online from an ever-wider array of sensors and sources demands algorithmic support, the Pentagon has been saying for years.

DIA’s eventual goal is to have human analysts and machine intelligence compliment each other, since each has a very different approach to analysis, or as Busch describes it, “tradecraft.” On the human side, that means “transitioning the expert into a quantitative workflow,” he says. Take that to mean helping analysts produce insights that are never seen as finished but that can change as rapidly as the data used to draw those insights. That also means teaching analysts to become data literate to understand things like confidence intervals and other statistical terms.

Busch cautioned that the experiment doesn’t imply that defense intelligence work should be handed over to software. The warning from WarGames is still current. “On the machine side, we have experienced confirmation bias in big data. [We’ve] had the machine retrain itself to error… That’s a real concern for us.”

84 comentários em “NGA Knows Its Challenges, Now It Needs the Tech to Address Them

  1. Excellent beat ! I wish to apprentice while you amend your web site, how could
    i subscribe for a blog web site? The account
    aided me a acceptable deal. I had been tiny bit acquainted
    of this your broadcast offered bright clear concept

  2. Awesome blog! Do you have any recommendations for aspiring writers?
    I’m hoping to start my own site soon but I’m a little lost on everything.
    Would you propose starting with a free platform like WordPress
    or go for a paid option? There are so many options out there that I’m completely overwhelmed ..

    Any tips? Cheers!

  3. I’m the business owner of JustCBD Store company (justcbdstore.com) and I’m presently aiming to grow my wholesale side of company. I am hoping someone at targetdomain can help me 🙂 I considered that the most suitable way to do this would be to reach out to vape companies and cbd stores. I was hoping if anyone could suggest a dependable web site where I can buy CBD Shops B2B Database I am presently considering creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. On the fence which one would be the very best option and would appreciate any guidance on this. Or would it be easier for me to scrape my own leads? Ideas?

  4. I am the proprietor of JustCBD company (justcbdstore.com) and I am currently planning to broaden my wholesale side of company. It would be great if anybody at targetdomain share some guidance ! I thought that the best way to do this would be to talk to vape shops and cbd retail stores. I was really hoping if someone could recommend a trusted website where I can purchase Vape Shop Business Sales Leads I am already considering creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. Unsure which one would be the most suitable choice and would appreciate any advice on this. Or would it be easier for me to scrape my own leads? Ideas?

  5. An intriguing discussion is worth comment. There’s no doubt that that you need to publish more on this subject, it may not be a taboo matter but generally folks don’t talk about these topics. To the next! Best wishes!!

  6. Hi, I do believe this is an excellent website. I stumbledupon it 😉 I’m going to come back yet again since I book marked it. Money and freedom is the greatest way to change, may you be rich and continue to guide others.

  7. When I originally commented I seem to have clicked the -Notify me when new comments are added- checkbox and from now on whenever a comment is added I get four emails with the exact same comment. There has to be a means you are able to remove me from that service? Thanks!

  8. When I initially commented I seem to have clicked on the -Notify me when new comments are added- checkbox and from now on every time a comment is added I get 4 emails with the same comment. Perhaps there is an easy method you are able to remove me from that service? Cheers!

  9. I blog quite often and I seriously thank you for your information. This great article has really peaked my interest. I am going to book mark your website and keep checking for new details about once a week. I subscribed to your Feed too.

  10. I seriously love your website.. Pleasant colors & theme. Did you build this site yourself? Please reply back as I’m attempting to create my own personal site and want to learn where you got this from or exactly what the theme is called. Thank you!

  11. I’m impressed, I must say. Rarely do I come across a blog that’s equally educative and amusing, and without a doubt, you have hit the nail on the head. The issue is something which too few men and women are speaking intelligently about. Now i’m very happy I came across this in my search for something relating to this.

  12. After looking over a handful of the blog posts on your web page, I seriously appreciate your technique of writing a blog. I saved as a favorite it to my bookmark website list and will be checking back soon. Take a look at my web site as well and tell me what you think.

  13. After I originally left a comment I seem to have clicked the -Notify me when new comments are added- checkbox and from now on every time a comment is added I get four emails with the same comment. There has to be a way you can remove me from that service? Kudos!

  14. After going over a handful of the blog articles on your web site, I seriously like your technique of blogging. I bookmarked it to my bookmark site list and will be checking back soon. Please visit my website too and let me know what you think.

  15. Can I simply say what a comfort to discover a person that genuinely understands what they’re talking about on the net. You definitely know how to bring a problem to light and make it important. More people should check this out and understand this side of your story. It’s surprising you aren’t more popular since you most certainly possess the gift.

  16. The next time I read a blog, I hope that it does not fail me just as much as this one. After all, I know it was my choice to read through, but I truly believed you would have something helpful to say. All I hear is a bunch of complaining about something you can fix if you weren’t too busy seeking attention.

  17. Hello there, There’s no doubt that your website could possibly be having web browser compatibility issues. Whenever I look at your site in Safari, it looks fine however, when opening in I.E., it’s got some overlapping issues. I merely wanted to give you a quick heads up! Besides that, wonderful blog!

  18. Hello there! This post couldn’t be written any better! Looking at this article reminds me of my previous roommate! He always kept talking about this. I most certainly will forward this information to him. Fairly certain he’ll have a very good read. Thanks for sharing!

  19. Your style is really unique compared to other people I’ve read stuff from. Thank you for posting when you have the opportunity, Guess I will just book mark this page.

  20. After looking at a few of the articles on your blog, I really like your way of blogging. I saved as a favorite it to my bookmark webpage list and will be checking back soon. Please check out my website too and let me know how you feel.

  21. Right here is the right web site for anyone who hopes to understand this topic. You understand a whole lot its almost tough to argue with you (not that I actually would want to…HaHa). You definitely put a new spin on a topic that has been discussed for ages. Great stuff, just great!

  22. You’re so interesting! I do not suppose I’ve truly read anything like that before. So nice to find somebody with some unique thoughts on this subject. Really.. many thanks for starting this up. This site is something that’s needed on the web, someone with some originality!

  23. I’m amazed, I have to admit. Rarely do I encounter a blog that’s both educative and amusing, and without a doubt, you’ve hit the nail on the head. The problem is an issue that not enough men and women are speaking intelligently about. I’m very happy that I came across this in my hunt for something relating to this.

  24. An interesting discussion is definitely worth comment. I do believe that you need to publish more on this subject matter, it might not be a taboo subject but generally people do not talk about these topics. To the next! Many thanks!!

  25. I seriously love your website.. Very nice colors & theme. Did you build this amazing site yourself? Please reply back as I’m hoping to create my own blog and want to find out where you got this from or what the theme is named. Thanks!

  26. It’s appropriate time to make some plans for the long run and it is time to be happy. I have read this submit and if I may I wish to suggest you few fascinating issues or advice. Perhaps you could write subsequent articles referring to this article. I want to read even more things about it!|

  27. You are so interesting! I don’t think I’ve truly read anything like this before. So nice to discover somebody with original thoughts on this subject matter. Seriously.. thanks for starting this up. This site is something that’s needed on the internet, someone with a bit of originality!

  28. Next time I read a blog, I hope that it won’t fail me just as much as this particular one. After all, I know it was my choice to read, nonetheless I genuinely thought you would probably have something interesting to say. All I hear is a bunch of moaning about something that you could possibly fix if you were not too busy looking for attention.

  29. After looking into a number of the blog posts on your site, I truly like your way of writing a blog. I added it to my bookmark site list and will be checking back soon. Take a look at my web site as well and tell me what you think.

  30. Achieving your fitness goals does not need a certified personal trainer or an expensive gym memberships, especially when you have the budget and the space to consider practically every workout machine on the market.

  31. A fascinating discussion is definitely worth comment. I think that you need to write more on this subject matter, it might not be a taboo subject but generally people do not speak about such topics. To the next! Cheers!!

  32. I blog frequently and I seriously thank you for your information. The article has really peaked my interest. I am going to book mark your site and keep checking for new information about once per week. I opted in for your Feed as well.

  33. The very next time I read a blog, I hope that it won’t fail me as much as this particular one. After all, I know it was my choice to read through, nonetheless I really thought you would probably have something useful to say. All I hear is a bunch of crying about something that you could possibly fix if you were not too busy seeking attention.

  34. Oh my goodness! Awesome article dude! Thank you, However I am encountering problems with your RSS. I don’t understand the reason why I cannot join it. Is there anybody getting similar RSS issues? Anybody who knows the answer will you kindly respond? Thanks!!

  35. After I originally commented I seem to have clicked the -Notify me when new comments are added- checkbox and from now on whenever a comment is added I receive four emails with the exact same comment. There has to be a way you can remove me from that service? Appreciate it!

  36. I’m amazed, I have to admit. Rarely do I encounter a blog that’s both educative and amusing, and without a doubt, you’ve hit the nail on the head. The issue is something too few people are speaking intelligently about. I’m very happy that I found this in my search for something relating to this.

  37. Having read this I thought it was rather enlightening. I appreciate you finding the time and effort to put this content together. I once again find myself spending way too much time both reading and posting comments. But so what, it was still worthwhile!

Leave a Reply

Your email address will not be published. Required fields are marked *