Welcome to AI book reviews, a series of posts that explore the latest literature on artificial intelligence.
When discussing the threats of artificial intelligence, the first thing that comes to mind are images of Skynet, The Matrix, and the robot apocalypse. The runner up is technological unemployment, the vision of a foreseeable future in which AI algorithms take over all jobs and push humans into a struggle for meaningless survival in a world where human labor is no longer needed.
Whether any or both of those threats are real is hotly debated among scientists and thought leaders. But AI algorithms also pose more imminent threats that exist today, in ways that are less conspicuous and hardly understood.
In her book, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, mathematician Cathy O’Neil explores how blindly trusting algorithms to make sensitive decisions can harm many people who are on the receiving end of those decisions.
The dangers of AI algorithms can manifest themselves in algorithmic bias and dangerous feedback loops, and they can expand to all sectors of daily life, from the economy to social interactions, to the criminal justice system.
While the use of mathematics and algorithms in decision-making is nothing new, recent advances in deep learning and the proliferation of black-box AI systems amplify their effects, both good and bad. And if we do not understand the present threats of AI, we will not be able to benefit from its advantages.
The characteristics of dangerous AI algorithms
We use algorithms to model to understand and process many things. “A model, after all, is nothing more than an abstract representation of some process, be it a baseball game, an oil company’s supply chain, a foreign government’s actions, or a movie theater’s attendance,” O’Neil writes in Weapons of Math Destruction. “Whether it’s running in a computer program or in our head, the model takes what we know and uses it to predict responses in various situations.”
But more and more of those models are being transferred from our heads to computers, thanks to advances in deep learning and the increased digitization of every aspect of our lives. Thanks to broadband internet, cloud computing, mobile devices, the internet of things (IoT), wearables, and a slew of other emerging technologies, we can collect and process more and more data about anything and everything.
This increased access to data and computing power has helped create AI algorithms that can automate an increasing number of tasks. Deep neural networks, which had previously been limited to research laboratories, have found their way into many areas that were previously challenging for computers, such as computer vision, machine translation, speech and facial recognition.
So far, so good. What can go wrong?
In Weapons of Math Destruction, O’Neil specifies three factors that make AI models dangerous: opacity, scale, and damage.
Algorithmic vs corporate opacity
There are two aspects to the opacity of AI systems: technical and corporate. The technical opacity, also referred to as the black-box problem of artificial intelligence, has received much attention in the past few years.
In a nutshell, the question is, how do we know an AI algorithm is making the right decision? This question is becoming more critical as AI finds its way into loan application processing, credit scoring, teacher rating, recidivism prediction, and many other sensitive fields.
Many media outlets have published articles that depict AI algorithms as mysterious machines whose behavior is unknown even to their developers. But contrary to what the media portrays, not all AI algorithms are opaque.
Traditional software, often referred to as symbolic artificial intelligence in AI jargon, are known for their interpretable and transparent nature. They are composed of hand-coded rules, meticulously put together by software developers and domain experts. They can be probed and audited, and an error can be traced to the line of code where it has occurred.
In contrast, machine learning algorithms, which have become increasingly popular in recent years, develop their behavior by analyzing many training examples and creating statistical inference models. This means that the developers don’t necessarily have the final say on how the AI algorithms behave.
But again, not all machine learning models are opaque. For instance, decision trees and linear regression models, two popular machine learning algorithms, will give clear explanations of the factors that determine their decisions. If you train a decision tree algorithm to process loan applications, it can provide you with a tree-like breakdown (thus the name) of how it decides which loan applications to confirm and which to reject. This provides developers with a chance to discover potentially problematic factors and correct the model.
But deep neural networks, which have become very popular in the past few years, are especially bad at revealing how they work. They are composed of layers upon layers of artificial neurons, small mathematical functions that tune their parameters to the thousands of examples they see during training. In many cases, it’s very hard to probe deep learning models and determine which factors contribute to their decision-making processes.
A loan application processing deep learning algorithm is an end-to-end model where a loan application goes in and a final verdict comes out. There’s not a feature-by-feature breakdown on how the AI algorithm is making decisions. In most cases, a well-trained deep learning model will perform better than its less-sophisticated siblings (decision trees, support vector machines, linear regression, etc.), and it might even spot relevant patterns that will go unnoticed to human experts.
However, even the most accurate deep learning systems make errors every once in a while, and when they do, it will be very hard to determine what went wrong. But a deep learning system doesn’t need to make errors before its opacity turns problematic. Suppose an angry customer wants to know why an AI application has turned down their loan application. When you have an interpretable AI system, you’ll be able to provide a clear explanation of the steps that went into the decision. When you have an opaque system, you can just shrug and say, “The computer said so.”
But while the technical opacity of artificial intelligence algorithms has received a lot of attention in tech media, what’s less discussed is the opaque ways companies use their algorithms, even when the algorithms themselves are trivial and interpretable.
“Even if the participant is aware of being modeled, or what the model is used for, is the model opaque, or even invisible?” O’Neil questions in Weapons of Math Destruction.
Companies that view their AI algorithms as corporate secrets try their best to hide them behind walled gardens to keep the edge over their competitors. We don’t know much about the AI algorithm powering Google Search, the models that define our friend suggestions on Facebook, populate our feed on Twitter, among others.
Some of this secrecy is justified. For instance, if Google published the inner-workings of its search algorithm, then it would become vulnerable to gaming. In fact, even without Google revealing much detail about its search algorithm, there’s an entire industry poised to find shortcuts to Google Search top-ranking positions. Algorithms are, after all, mindless machines that play by their own rules. They don’t use common sense judgment to identify evil actors who twist the rules for devious intentions.
But staying on the same example, without transparency, how can we make sure that Google itself is not manipulating search results to serve its own political goals and economic interests? In 2018, U.S. President Donald Trump accused Google of burying conservative news outlets in its search results and favoring liberal media. The claim put Google on the defensive, and the company’s spokespersons could only promise that they would do no such thing.
This only shows the fine line that organizations walk on when they use AI algorithms. When AI systems are not transparent, they don’t even need to make errors to wreak havoc. Even the shadow of a doubt about the system’s performance can be enough to cause mistrust in the system. On the other hand, too much transparency can also backfire and lead to other disastrous results.
O’Neil wrote Weapons of Math Destruction in 2016, before rules like GDPR and CCPA came into effect. Those regulations require companies to be transparent about the use of AI algorithms and allow users to investigate the decision process behind their automation systems. Other developments, such as the ethical AI rules of the European Commission, also incentivize transparency.
Much progress has been made to address the technical, ethical, and legal issues surrounding AI transparency, a lot more still needs to be done. As regulators pass new laws to regulate corporate secrecy, corporations find new ways to circumvent those rules without finding themselves in hot water, such as very long Terms of Service dialogs that inconspicuously deprive you of your right to algorithmic transparency.
Who bears the damage of AI algorithms?
“Does the model work against the subject’s interest? In short, is it unfair? Does it damage or destroy lives?” O’Neil posits in Weapons of Math Destruction.
There are plenty of examples of AI algorithms making dumb shopping suggestions, misclassifying images, and doing other silly things. But as AI models become more and more ingrained in our lives, their errors are moving from benign to destructive.
In her book, O’Neil explores many cases where algorithms causing damage to people’s lives. Examples include credit scoring systems that wrongfully penalize people, recidivism algorithms that give heavier sentences to defendants based on their race and ethnic backgrounds, teacher-scoring systems that end up firing well-performing teachers and rewarding cheaters, and trade algorithms that make billions of dollars at the expense of low-income classes.
The impact of an algorithm, combined with its lack of transparency, lend to the creation of a dangerous AI system. For example, O’Neil says, “The new recidivism models are complicated and mathematical. But embedded within these models are a host of assumptions, some of them prejudicial,” and adds, “the workings of a recidivism model are tucked away in algorithms, intelligible only to a tiny elite.”
This basically means that an AI algorithm can decide to keep a person in jail based on their race, and the defendant has no way to find out why they were deemed as ineligible for pardon.
There are two more factors that make the damage of dangerous AI algorithms even more harmful.
First, the data. Machine learning algorithms rely on quality data for training and accuracy. If you want an image classifier to accurately detect pictures of cats, you must provide it with many labeled pictures of cats. Likewise, a loan-application algorithm would need lots of historical records of loan applications and their outcome (paid or defaulted).
The problem is, those who are hurt by AI algorithms are often the people on whom there’s not enough quality data. This is why loan application processors provide better services to those who already have adequate access to banking and penalize the unbanked and underprivileged who have been largely deprived of the financial system.
The second problem is the feedback loop. When an AI algorithm starts to make problematic decisions, its behavior generates more erroneous data, which is in turn used to further hone the algorithm, which causes even more prejudice, and the cycle continues endlessly.
On the topic of policing, O’Neil argues that prejudiced crime prediction causes more police presence in impoverished neighborhoods. “This creates a pernicious feedback loop,” she writes. “The policing itself spawns new data, which justifies more policing. And our prisons fill up with hundreds of thousands of people found guilty of victimless crimes.”
When you create a bigger picture of how all these disparate-and-yet-interconnected AI systems feed into each other, you’ll see how the real harm happens. Here’s how O’Neil summarizes the situation: “Poor people are more likely to have bad credit and live in high-crime neighborhoods, surrounded by other poor people. Once the dark universe of WMDs digests that data, it showers them with predatory ads for subprime loans or for-profit schools. It sends more police to arrest them, and when they’re convicted it sentences them to longer terms. This data feeds into other WMDs, which score the same people as high risks or easy targets and proceed to block them from jobs, while jacking up their rates for mortgages, car loans, and every kind of insurance imaginable. This drives their credit rating down further, creating nothing less than a death spiral of modeling. Being poor in a world of WMDs is getting more and more dangerous and expensive.”
The explosive scale of algorithmic harm
“The third question is whether a model has the capacity to grow exponentially. As a statistician would put it, can it scale?” O’Neil writes in Weapons of Math Destruction.
Consider the Google Search example we discussed earlier. Billions of people use Google Search to find answers to important questions about health, politics, and social issues. A tiny mistake in Google’s AI algorithm can have a massive impact on public opinion.
Likewise, Facebook’s ranking algorithms decide the news that hundreds of millions of people see every day. If those algorithms are faulty, they can be gamed to spread fake, sensational news by malicious actors. Even when there’s not a direct malicious intent, they can still cause harm. For instance, news feed algorithms that favor engaging content can amplify biases and create filter bubbles, making users less tolerant of alternative views.
When opaque and faulty Al algorithms determine credit scores for hundreds of millions of people or decide the fate of the country’s education system, then you have all the elements of a weapon of math destruction.
So, what should be done about this? We need to acknowledge the limits of the AI algorithms that we deploy. While having an automated system that relieves you of the duty of making tough decisions might seem tempting, you must understand when humans are on the receiving end of those decisions and how they are affected.
“Big Data processes codify the past. They do not invent the future. Doing that requires moral imagination, and that’s something only humans can provide. We have to explicitly embed better values into our algorithms, creating Big Data models that follow our ethical lead. Sometimes that will mean putting fairness ahead of profit,” O’Neil writes.
Original post: https://bdtechtalks.com/2020/06/10/ai-weapons-of-math-destruction/
I am the proprietor of JustCBD Store company (justcbdstore.com) and am seeking to broaden my wholesale side of business. I am hoping someone at targetdomain give me some advice . I considered that the very best way to do this would be to reach out to vape stores and cbd stores. I was really hoping if anybody could suggest a reputable web-site where I can purchase CBD Shops Business Leads I am currently checking out creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. Unsure which one would be the most ideal selection and would appreciate any support on this. Or would it be simpler for me to scrape my own leads? Ideas?
I’m the owner of JustCBD brand (justcbdstore.com) and I am currently trying to grow my wholesale side of company. I am hoping someone at targetdomain give me some advice 🙂 I considered that the best way to do this would be to connect to vape stores and cbd retail stores. I was hoping if anyone could suggest a trusted web-site where I can get Vape Shop Business Mailing List I am currently checking out creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. Unsure which one would be the best option and would appreciate any guidance on this. Or would it be easier for me to scrape my own leads? Ideas?
You have made some decent points there. I checked on the web for additional information about the issue and found most people will go along with your views on this site.
Hi, I do believe this is a great site. I stumbledupon it 😉 I may come back yet again since i have book marked it. Money and freedom is the greatest way to change, may you be rich and continue to help other people.
I really like it when people come together and share thoughts. Great website, stick with it!
I love looking through an article that can make men and women think. Also, many thanks for allowing for me to comment!
This blog was… how do I say it? Relevant!! Finally I’ve found something that helped me. Many thanks!
I quite like reading an article that can make people think. Also, many thanks for allowing me to comment!
I like it when individuals come together and share ideas. Great website, continue the good work!
Oh my goodness! Awesome article dude! Thank you so much, However I am experiencing issues with your RSS. I don’t understand the reason why I can’t join it. Is there anyone else getting similar RSS problems? Anybody who knows the solution can you kindly respond? Thanks!!
Very good info. Lucky me I recently found your site by accident (stumbleupon). I have book-marked it for later!
I love it when people get together and share ideas. Great website, continue the good work!
bookmarked!!, I love your site!
Very nice blog post. I certainly love this website. Keep it up!
I love it when people come together and share thoughts. Great blog, keep it up!
Aw, this was an incredibly good post. Finding the time and actual effort to make a great article… but what can I say… I procrastinate a lot and never seem to get anything done.
Greetings! Very helpful advice in this particular post! It is the little changes that will make the most important changes. Thanks for sharing!
I seriously love your site.. Great colors & theme. Did you create this amazing site yourself? Please reply back as I’m hoping to create my own website and want to know where you got this from or exactly what the theme is named. Many thanks!
An intriguing discussion is definitely worth comment. I believe that you should write more on this subject matter, it might not be a taboo subject but typically people do not discuss these issues. To the next! Cheers!!
Great article! We will be linking to this great article on our website. Keep up the great writing.
I seriously love your blog.. Excellent colors & theme. Did you create this website yourself? Please reply back as I’m looking to create my very own site and want to find out where you got this from or what the theme is named. Kudos!
This site certainly has all of the information I wanted about this subject and didn’t know who to ask.
Right here is the right webpage for anyone who hopes to understand this topic. You realize a whole lot its almost tough to argue with you (not that I personally would want to…HaHa). You certainly put a brand new spin on a subject which has been written about for a long time. Great stuff, just excellent!
I absolutely love your blog.. Very nice colors & theme. Did you develop this site yourself? Please reply back as I’m wanting to create my own website and would like to find out where you got this from or just what the theme is called. Many thanks!
Great web site you have here.. It’s difficult to find high-quality writing like yours these days. I seriously appreciate individuals like you! Take care!!
Way cool! Some very valid points! I appreciate you penning this post plus the rest of the site is very good.
May I just say what a relief to find a person that really knows what they are discussing on the internet. You definitely understand how to bring a problem to light and make it important. More and more people have to read this and understand this side of the story. I was surprised that you’re not more popular because you definitely have the gift.
The very next time I read a blog, Hopefully it doesn’t disappoint me as much as this particular one. I mean, I know it was my choice to read through, but I really thought you would probably have something interesting to talk about. All I hear is a bunch of complaining about something that you could fix if you weren’t too busy searching for attention.
Good web site you have here.. It’s hard to find high-quality writing like yours these days. I honestly appreciate people like you! Take care!!
It’s hard to come by experienced people for this topic, but you sound like you know what you’re talking about! Thanks
After looking into a number of the blog articles on your website, I honestly appreciate your way of blogging. I saved as a favorite it to my bookmark site list and will be checking back in the near future. Please check out my web site as well and tell me what you think.
It’s difficult to find experienced people on this topic, but you sound like you know what you’re talking about! Thanks
Spot on with this write-up, I seriously believe that this site needs a lot more attention. I’ll probably be returning to read more, thanks for the info!
This page certainly has all the info I wanted about this subject and didn’t know who to ask.
This website was… how do you say it? Relevant!! Finally I have found something which helped me. Cheers!
Having read this I thought it was very enlightening. I appreciate you taking the time and effort to put this article together. I once again find myself personally spending way too much time both reading and posting comments. But so what, it was still worthwhile!
This is a very good tip especially to those fresh to the blogosphere. Short but very accurate info… Thank you for sharing this one. A must read post!
May I simply just say what a relief to discover an individual who really knows what they’re discussing on the net. You certainly realize how to bring an issue to light and make it important. More and more people have to look at this and understand this side of your story. I was surprised you are not more popular because you definitely possess the gift.
I’m impressed, I have to admit. Seldom do I come across a blog that’s both educative and entertaining, and without a doubt, you’ve hit the nail on the head. The issue is something that not enough men and women are speaking intelligently about. I’m very happy that I came across this in my hunt for something relating to this.
I could not refrain from commenting. Well written!
Next time I read a blog, Hopefully it does not fail me as much as this one. I mean, Yes, it was my choice to read through, nonetheless I genuinely thought you’d have something useful to talk about. All I hear is a bunch of crying about something that you could possibly fix if you were not too busy searching for attention.
Hi! I just want to give you a huge thumbs up for your excellent info you’ve got right here on this post. I am returning to your blog for more soon.
This website was… how do you say it? Relevant!! Finally I have found something which helped me. Cheers!
You’ve made some decent points there. I looked on the net for more information about the issue and found most individuals will go along with your views on this web site.
I blog often and I seriously appreciate your content. The article has truly peaked my interest. I will book mark your blog and keep checking for new information about once per week. I opted in for your RSS feed too.
An interesting discussion is definitely worth comment. I think that you should publish more on this subject, it may not be a taboo subject but usually people do not talk about these topics. To the next! All the best!!
Greetings! Very useful advice within this post! It’s the little changes that will make the most important changes. Thanks a lot for sharing!
Your style is so unique compared to other people I have read stuff from. Many thanks for posting when you’ve got the opportunity, Guess I will just book mark this web site.
I have to thank you for the efforts you have put in penning this blog. I really hope to check out the same high-grade content from you later on as well. In fact, your creative writing abilities has motivated me to get my own site now 😉
Pretty! This has been an extremely wonderful post. Many thanks for supplying these details.
Your style is so unique in comparison to other folks I have read stuff from. Thanks for posting when you’ve got the opportunity, Guess I’ll just book mark this web site.
Greetings, There’s no doubt that your web site could be having internet browser compatibility issues. When I look at your blog in Safari, it looks fine however when opening in Internet Explorer, it has some overlapping issues. I merely wanted to give you a quick heads up! Other than that, excellent website!
There’s certainly a lot to learn about this subject. I really like all the points you have made.
I like it when folks get together and share thoughts. Great blog, continue the good work!
Aw, this was a really good post. Taking the time and actual effort to create a great article… but what can I say… I procrastinate a whole lot and don’t manage to get anything done.
After exploring a handful of the blog articles on your web site, I seriously like your technique of writing a blog. I added it to my bookmark site list and will be checking back in the near future. Take a look at my web site too and let me know how you feel.
Achieving your fitness goal doesn’t have to require a certified personal trainer or an expensive gym memberships, it’s not hard to exercise at home. It is easy to go down a training and fitness rabbit hole, however, when you’re looking for the best home exercise equipment to outfit your personal home gym.
Having read this I believed it was extremely enlightening. I appreciate you finding the time and energy to put this information together. I once again find myself spending a lot of time both reading and commenting. But so what, it was still worthwhile!
You’re so awesome! I don’t suppose I’ve read through anything like this before. So good to find someone with some genuine thoughts on this topic. Seriously.. many thanks for starting this up. This website is one thing that’s needed on the internet, someone with a bit of originality!
Next time I read a blog, Hopefully it does not fail me just as much as this particular one. I mean, Yes, it was my choice to read, but I really thought you’d have something useful to talk about. All I hear is a bunch of whining about something you could possibly fix if you were not too busy seeking attention.
Hi, I do believe this is a great blog. I stumbledupon it 😉 I’m going to come back once again since I book-marked it. Money and freedom is the best way to change, may you be rich and continue to help other people.
This is a very good tip particularly to those fresh to the blogosphere. Short but very precise information… Thank you for sharing this one. A must read article!
Greetings! Very useful advice within this post! It’s the little changes that produce the biggest changes. Thanks for sharing!
Way cool! Some extremely valid points! I appreciate you penning this write-up and the rest of the website is really good.
Your style is really unique in comparison to other people I have read stuff from. Thanks for posting when you have the opportunity, Guess I will just book mark this blog.
bookmarked!!, I love your website!
Having read this I believed it was very informative. I appreciate you taking the time and effort to put this article together. I once again find myself personally spending a lot of time both reading and commenting. But so what, it was still worthwhile!
An impressive share! I’ve just forwarded this onto a co-worker who had been doing a little homework on this. And he actually bought me dinner because I discovered it for him… lol. So allow me to reword this…. Thank YOU for the meal!! But yeah, thanx for spending time to talk about this issue here on your internet site.
Nice post. I learn something totally new and challenging on websites I stumbleupon everyday. It’s always helpful to read through content from other writers and use something from other websites.
There is certainly a great deal to learn about this subject. I like all the points you made.
Howdy! I could have sworn I’ve been to this site before but after looking at many of the posts I realized it’s new to me. Nonetheless, I’m certainly delighted I found it and I’ll be book-marking it and checking back regularly!