Symbolic AI algorithms have played an important role in AI’s history, but they face challenges in learning on their own. After IBM Watson used symbolic reasoning to beat Brad Rutter and Ken Jennings at Jeopardy in 2011, the technology has been eclipsed by neural networks trained by deep learning.
The power of neural networks is that they help automate the process of generating models of the world. This has led to several significant milestones in artificial intelligence, giving rise to deep learning models that, for example, could beat humans in progressively complex games, including Go and StarCraft. But it can be challenging to reuse these deep learning models or extend them to new domains.
Now researchers and enterprises are looking for ways to bring neural networks and symbolic AI techniques together.
“Neuro-symbolic modeling is one of the most exciting areas in AI right now,” said Brenden Lake, assistant professor of psychology and data science at New York University. His team has been exploring different ways to bridge the gap between the two AI approaches.
Companies like IBM are also pursuing how to extend these concepts to solve business problems, said David Cox, IBM Director of MIT-IBM Watson AI Lab.
“I would argue that symbolic AI is still waiting, not for data or compute, but deep learning,” Cox said.
His team is working with researchers from MIT CSAIL, Harvard University and Google DeepMind, to develop a new, large-scale video reasoning data set called, “CLEVRER: CoLlision Events for Video REpresentation and Reasoning.” This allows AI to recognize objects and reason about their behaviors in physical events from videos with only a fraction of the data required for traditional deep learning systems.
AI antagonists that have complementary strengths
Deep learning is incredibly adept at large-scale pattern recognition and at capturing complex correlations in massive data sets, NYU’s Lake said. In contrast, deep learning struggles at capturing compositional and causal structure from data, such as understanding how to construct new concepts by composing old ones or understanding the process for generating new data.
Brenden LakeSymbolic models have a complementary strength: They are good at capturing compositional and causal structure. But they struggle to capture complex correlations. The unification of the two approaches would address the shortcomings of each.
“Neuro-symbolic [AI] models will allow us to build AI systems that capture compositionality, causality, and complex correlations,” Lake said.
Limits to learning by correlation
Hadayat Seddiqi, director of machine learning at InCloudCounsel, a legal technology company, said the time is right for developing a neuro-symbolic learning approach. “Deep learning in its present state cannot learn logical rules, since its strength comes from analyzing correlations in the data,” he said.
Hadayat SeddiqiThis attribute makes it effective at tackling problems where logical rules are exceptionally complex, numerous, and ultimately impractical to code, like deciding how a single pixel in an image should be labeled. However, correlation algorithms come with numerous weaknesses.
“This is a prime reason why language is not wholly solved by current deep learning systems,” Seddiqi said.
For almost any type of programming outside of statistical learning algorithms, symbolic processing is used; consequently, it is in some way a necessary part of every AI system. Indeed, Seddiqi said he finds it’s often easier to program a few logical rules to implement some function than to deduce them with machine learning. It is also usually the case that the data needed to train a machine learning model either doesn’t exist or is insufficient. In those cases, rules derived from domain knowledge can help generate training data.
This is important because all AI systems in the real world deal with messy data. Symbolic processing can help filter out irrelevant data. For example, in an application that uses AI to answer questions about legal contracts, simple business logic can filter out data from documents that are not contracts or that are contracts in a different domain such as financial services versus real estate.
System 1 vs. System 2 thinking
Deep learning is better suited for System 1 reasoning, said Debu Chatterjee, head of AI, ML and analytics engineering at ServiceNow, referring to the paradigm developed by the psychologist Daniel Kahneman in his book Thinking Fast and Slow. System 1 thinking is fast, associative, intuitive and automatic.
Debu ChatterjeeDeep learning, in its present state, interprets inputs from the messy, approximate, probabilistic real world Chatterjee said, and it is very powerful: “If you do this on a large enough data set, this can exceed human-level perception.”
When handling a complex input, deep learning can deal with perception problems that attempt to determine whether something is true: for example, whether a picture contains a cat versus a dog. But it is hard for humans to ascertain the properties of these deep learning systems and difficult to test whether they work or under what conditions they work or don’t work. They are opaque to human analysis.
Symbolic AI’s strength lies in its knowledge representation and reasoning through logic, making it more akin to Kahneman’s “System 2” mode of thinking, which is slow, takes work and demands attention. Symbolic reasoning is modular and easier to extend. That is because it is based on relatively simple underlying logic that relies on things being true, and on rules providing a means of inferring new things from things already known to be true. Humans understand how it reached its conclusions.
The weakness of symbolic reasoning is that it does not tolerate ambiguity as seen in the real world. One false assumption can make everything true, effectively rendering the system meaningless.
“There have been many attempts to extend logic to deal with this which have not been successful,” Chatterjee said. Alternatively, in complex perception problems, the set of rules needed may be too large for the AI system to handle.
“Any realistic AI system needs to have both deep learning and symbolic properties,” Chatterjee said. This means it needs to be good at both perception and being able to infer new things from existing facts.
Practical benefits of combining symbolic AI and deep learning
There are many practical benefits to developing neuro-symbolic AI. One of the biggest is to be able to automatically encode better rules for symbolic AI.
David Cox“With symbolic AI there was always a question mark about how to get the symbols,” IBM’s Cox said. The world is presented to applications that use symbolic AI as images, video and natural language, which is not the same as symbols.
“We are finding that neural networks can get you to the symbolic domain and then you can use a wealth of ideas from symbolic AI to understand the world,” Cox said.
Another way the two AI paradigms can be combined is by using neural networks to help prioritize how symbolic programs organize and search through multiple facts related to a question. For example, if an AI is trying to decide if a given statement is true, a symbolic algorithm needs to consider whether thousands of combinations of facts are relevant.
Humans have an intuition about which facts might be relevant to a query. For example, a medical diagnostic expert system would have to weigh a patient’s records and new complaints in making a medical suggestion, whereas an experienced human doctor could see the gestalt of the patient’s state and quickly understand how to investigate the new complaints or what tests to order.
Another benefit of combining the techniques lies in making the AI model easier to understand. Humans reason about the world in symbols, whereas neural networks encode their models using pattern activations. Humans don’t think in terms of patterns of weights in neural networks.
“Our vision is to use neural networks as a bridge to get us to the symbolic domain,” Cox said, referring to work that IBM is exploring with its partners.
The thing symbolic processing can do is provide formal guarantees that a hypothesis is correct. This could prove important when the revenue of the business is on the line and companies need a way of proving the model will behave in a way that can be predicted by humans. In contrast, a neural network may be right most of the time, but when it’s wrong, it’s not always apparent what factors caused it to generate a bad answer.
Pushing the limits of NLP
The deep learning community has made great progress in using new techniques like transformers for natural language understanding tasks. But this is not true understanding — not in the way that symbolic processing works, argued Cox. Transformer models like Google’s BERT and OpenAI’s GPT are really about discovering statistical regularities, he said. While this can be powerful, it is not the same thing as understanding.
AI researchers like Gary Marcus have argued that these systems struggle with answering questions like, “Which direction is a nail going into the floor pointing?” This is not the kind of question that is likely to be written down, since it is common sense.
“As impressive as things like transformers are on our path to natural language understanding, they are not sufficient,” Cox said.
Seddiqi expects many advancements to come from natural language processing. Language is a type of data that relies on statistical pattern matching at the lowest levels but quickly requires logical reasoning at higher levels. Pushing performance for NLP systems will likely be akin to augmenting deep neural networks with logical reasoning capabilities.
The greatest promise here is analogous to experimental particle physics, where large particle accelerators are built to crash atoms together and monitor their behaviors. In natural language processing, researchers have built large models with massive amounts of data using deep neural networks that cost millions of dollars to train. The next step lies in studying the networks to see how this can improve the construction of symbolic representations required for higher order language tasks.
Indeed a lot of work in explainable AI — the effort to highlight the inner workings of AI models relevant to a particular use case — seems to be focused on inferring the underlying concepts and rules, for the reason that rules are easier to explain than weights in a neural network, Chatterjee said.
A key factor in evolution of AI will be dependent on a common programming framework that allows simple integration of both deep learning and symbolic logic. “Without this, these approaches won’t mix, like oil and water,” he said.
Original post: https://searchenterpriseai.techtarget.com/feature/Neuro-symbolic-AI-seen-as-evolution-of-artificial-intelligence
I’m the business owner of JustCBD Store company (justcbdstore.com) and am trying to expand my wholesale side of company. It would be great if someone at targetdomain share some guidance ! I thought that the most suitable way to accomplish this would be to connect to vape shops and cbd retail stores. I was really hoping if anyone could suggest a trustworthy web site where I can buy Vape Shop Business Mailing List I am currently reviewing creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. Not sure which one would be the best selection and would appreciate any assistance on this. Or would it be simpler for me to scrape my own leads? Ideas?
I am the co-founder of JustCBD Store label (justcbdstore.com) and I’m presently trying to expand my wholesale side of company. It would be great if anybody at targetdomain can help me 🙂 I thought that the most effective way to do this would be to connect to vape shops and cbd retail stores. I was hoping if anybody could recommend a dependable web-site where I can buy CBD Shops Business Sales Leads I am currently considering creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. On the fence which one would be the very best solution and would appreciate any support on this. Or would it be simpler for me to scrape my own leads? Suggestions?
There’s definately a lot to know about this issue. I really like all the points you have made.
Very nice article. I definitely love this site. Keep it up!
Good article. I definitely love this site. Thanks!
Hi there! I just would like to give you a big thumbs up for your great information you’ve got right here on this post. I will be returning to your blog for more soon.
Great web site you have here.. It’s difficult to find high quality writing like yours nowadays. I truly appreciate people like you! Take care!!
I like it when people get together and share ideas. Great website, keep it up!
That is a very good tip especially to those new to the blogosphere. Brief but very accurate info… Appreciate your sharing this one. A must read post!
Oh my goodness! Incredible article dude! Thanks, However I am experiencing troubles with your RSS. I don’t understand why I cannot join it. Is there anybody getting the same RSS problems? Anyone who knows the solution will you kindly respond? Thanks!!
I’m not sure where you are getting your information, but great
topic. I needs to spend some time learning more or understanding more.
Thanks for excellent info I was looking for this information for my
mission. adreamoftrains web hosting
You can certainly see your enthusiasm within the article you write.
The sector hopes for even more passionate writers like you who aren’t afraid to say
how they believe. Always follow your heart.
Howdy! This blog post could not be written much better! Going through this article reminds me of my previous roommate! He continually kept preaching about this. I will send this article to him. Pretty sure he’s going to have a good read. Thanks for sharing!
I’m impressed, I must say. Rarely do I come across a blog that’s both educative and interesting, and let me tell you, you have hit the nail on the head. The issue is an issue that too few folks are speaking intelligently about. I’m very happy that I stumbled across this during my hunt for something concerning this.
Excellent write-up. I certainly love this site. Thanks!
This website was… how do I say it? Relevant!! Finally I’ve found something which helped me. Appreciate it!
Achieving your fitness goals does not need a certified personal trainer or an expensive gym membership, it’s not hard to exercise at home. It’s easy to go down a training and fitness rabbit hole, however, when you are looking for the best home exercise equipment to outfit your personal home gym.
An outstanding share! I have just forwarded this onto a friend who was conducting a little research on this. And he in fact ordered me lunch because I discovered it for him… lol. So allow me to reword this…. Thank YOU for the meal!! But yeah, thanx for spending some time to discuss this issue here on your site.
I wanted to thank you for this very good read!! I definitely enjoyed every bit of it. I have got you bookmarked to look at new things you post…
May I simply just say what a relief to discover someone that really knows what they’re discussing on the web. You definitely realize how to bring an issue to light and make it important. More people ought to look at this and understand this side of the story. I was surprised that you’re not more popular because you certainly have the gift.
I blog often and I genuinely appreciate your information. The article has really peaked my interest. I am going to bookmark your blog and keep checking for new details about once per week. I subscribed to your RSS feed as well.
Having read this I believed it was extremely informative. I appreciate you spending some time and effort to put this informative article together. I once again find myself personally spending a significant amount of time both reading and posting comments. But so what, it was still worthwhile!
I was very pleased to uncover this website. I need to to thank you for your time just for this fantastic read!! I definitely appreciated every bit of it and I have you saved as a favorite to check out new things on your blog.
This blog was… how do I say it? Relevant!! Finally I’ve found something that helped me. Kudos!
You’ve made some really good points there. I looked on the internet for more information about the issue and found most individuals will go along with your views on this website.
When I originally left a comment I seem to have clicked the -Notify me when new comments are added- checkbox and now whenever a comment is added I recieve 4 emails with the same comment. Perhaps there is a means you can remove me from that service? Thanks!
I love it when folks get together and share opinions. Great blog, stick with it!
You’re so awesome! I do not suppose I’ve read through something like this before. So good to find somebody with some unique thoughts on this subject matter. Really.. thanks for starting this up. This web site is something that’s needed on the internet, someone with a little originality!
You ought to take part in a contest for one of the most useful websites on the net. I’m going to recommend this site!
A motivating discussion is worth comment. I do think that you should write more on this subject matter, it may not be a taboo subject but typically folks don’t talk about these subjects. To the next! All the best!!
I want to to thank you for this excellent read!! I certainly enjoyed every little bit of it. I’ve got you book marked to look at new things you post…
An impressive share! I’ve just forwarded this onto a co-worker who was conducting a little homework on this. And he actually ordered me lunch due to the fact that I found it for him… lol. So let me reword this…. Thank YOU for the meal!! But yeah, thanks for spending some time to discuss this topic here on your blog.
I would like to thank you for the efforts you have put in penning this site. I really hope to view the same high-grade content from you later on as well. In fact, your creative writing abilities has motivated me to get my own site now 😉
An intriguing discussion is definitely worth comment. I think that you should write more on this subject matter, it might not be a taboo matter but typically people do not speak about these subjects. To the next! Kind regards!!
I blog frequently and I seriously thank you for your content. This article has truly peaked my interest. I will book mark your blog and keep checking for new information about once per week. I opted in for your RSS feed as well.
I want to to thank you for this great read!! I absolutely loved every little bit of it. I have got you saved as a favorite to check out new things you post…
Excellent blog you have got here.. It’s difficult to find good quality writing like yours nowadays. I seriously appreciate individuals like you! Take care!!
Hello! I simply would like to give you a huge thumbs up for the great info you have got here on this post. I am coming back to your website for more soon.
Great information. Lucky me I discovered your blog by chance (stumbleupon). I have saved as a favorite for later!
An interesting discussion is worth comment. I think that you ought to publish more about this issue, it might not be a taboo subject but usually people don’t speak about these topics. To the next! Cheers!!
bookmarked!!, I really like your site!
Good article. I’m facing a few of these issues as well..
Good information. Lucky me I recently found your site by accident (stumbleupon). I’ve saved as a favorite for later!
Saved as a favorite, I love your website!
Everything is very open with a clear description of the issues. It was definitely informative. Your website is very helpful. Many thanks for sharing!
I just want to tell you that I am just new to blogging and site-building and truly liked this web page. Likely I’m want to bookmark your blog post . You amazingly come with superb article content. With thanks for revealing your web-site.
Right here is the perfect webpage for anyone who wishes to find out about this topic. You know a whole lot its almost hard to argue with you (not that I really would want to…HaHa). You definitely put a fresh spin on a subject that has been written about for ages. Great stuff, just excellent!
This site was… how do I say it? Relevant!! Finally I have found something that helped me. Many thanks!
This blog was… how do I say it? Relevant!! Finally I’ve found something which helped me. Appreciate it!
Having read this I thought it was really enlightening. I appreciate you spending some time and energy to put this informative article together. I once again find myself spending a significant amount of time both reading and posting comments. But so what, it was still worthwhile!
Hello there! This article couldn’t be written any better! Looking at this article reminds me of my previous roommate! He always kept preaching about this. I’ll send this information to him. Pretty sure he will have a great read. Thanks for sharing!
Aw, this was an incredibly nice post. Finding the time and actual effort to generate a good article… but what can I say… I hesitate a whole lot and don’t seem to get nearly anything done.
I blog quite often and I seriously thank you for your content. This article has truly peaked my interest. I will book mark your blog and keep checking for new details about once per week. I subscribed to your RSS feed too.
When I originally commented I seem to have clicked the -Notify me when new comments are added- checkbox and now whenever a comment is added I get 4 emails with the same comment. Is there a way you are able to remove me from that service? Thank you!
Hello! I could have sworn I’ve visited this site before but after going through many of the articles I realized it’s new to me. Regardless, I’m definitely delighted I found it and I’ll be book-marking it and checking back regularly!
Good site you’ve got here.. It’s hard to find high quality writing like yours these days. I honestly appreciate people like you! Take care!!
I like reading a post that will make men and women think. Also, thank you for permitting me to comment!
I must thank you for the efforts you’ve put in writing this site. I really hope to see the same high-grade content from you later on as well. In truth, your creative writing abilities has encouraged me to get my own site now 😉
I like it when individuals get together and share thoughts. Great blog, continue the good work!
I couldn’t resist commenting. Well written!
This site was… how do you say it? Relevant!! Finally I have found something that helped me. Thanks!
Good post. I learn something new and challenging on sites I stumbleupon every day. It’s always exciting to read through content from other authors and practice something from their sites.
The next time I read a blog, Hopefully it won’t disappoint me just as much as this particular one. After all, I know it was my choice to read through, nonetheless I truly thought you would probably have something useful to talk about. All I hear is a bunch of complaining about something that you could fix if you were not too busy seeking attention.
I blog frequently and I seriously thank you for your content. Your article has truly peaked my interest. I am going to bookmark your website and keep checking for new information about once a week. I subscribed to your Feed as well.
Very good blog post. I absolutely appreciate this site. Keep writing!
Howdy! I simply wish to offer you a big thumbs up for your excellent information you’ve got right here on this post. I am coming back to your website for more soon.
I’m amazed, I must say. Rarely do I come across a blog that’s both educative and interesting, and let me tell you, you’ve hit the nail on the head. The problem is something that too few people are speaking intelligently about. I’m very happy I found this during my search for something concerning this.