The Four “Pure” Learning Styles in Machine Learning


Back in 2015 when I started picking up ML at Stanford, the concepts and definitions around it were fairly structured and easy to map out. With the rapid growth of Deep Learning in recent years, the variety of terms and concepts used has increased immensely. This can leave newcomers to the field, who wish to learn more about the subject, frustrated and confused.

The trigger for writing this installment of “Structuring Machine Learning Concepts” was the concept confusion that recent breakthroughs in Natural Language Processing (NLP) and Computer Vision have brought to the table. People are starting to realize that some techniques that were previously regarded as Unsupervised Learning should more aptly be named Self-Supervised Learning. Let’s expand on that.

Supervised, Self-Supervised, Unsupervised, and Reinforcement Learning

When people talk about the different forms of Machine Learning, they usually refer to Supervised Learning (SL), Unsupervised Learning (UnSL), and Reinforcement Learning (RL) as the three learning styles. Sometimes, we add Semi-Supervised Learning (SemiSL) to the mix, combining elements of SL and UnSL. In 2018, a new breed of NLP algorithms started to gain popularity, leading to the famous researcher Yann LeCun to coin the concept of Self-Supervised Learning (SelfSL) in 2019.

Lightweight definitions — the original four learning styles

  • SL: We are using a dataset, where we want to predict a specific target, given other features (e.g., a table) or raw input (e.g., an image). This could take the form of classification, (predicting a categorical value — “Is this a cat or a dog?”) or regression (predicting a numerical value — “How cat-like is this image?”)
  • UnSL: Now assume we only have the raw input (e.g., just some user data in tables with no target or a lot of images with no information associated with them). Our goal is to look for previously undetected patterns, e.g., clusters or structures, that give new insights and can guide our decision-making.
  • RL: Normally stands out from the pack, since it is not working on a pre-collected dataset but trying to master a task in an environment by taking actions and observing the reward. Until we explicitly state a reward function to incentivize the agent to behave accordingly, it has no idea what to do in the given environment. Video games are a perfect example: the player (agent) tries to maximize her score (reward) by interacting with the video game (environment).
  • SemiSL: Often added as a fourth learning style, which works on one dataset with specified targets and on another, often larger one, without targets. The goal is to infer the labels of unlabeled data points with the help of knowledge from the labeled data points, thereby building the best possible model to map inputs to the targets. Think about you having access to data points of a manufacturing machine, which can be grouped into two clusters. From the labeled dataset you know, that all the unknown data points that are associated with a failure fall in cluster “A” and all normal states land in cluster “B”; hence the assumption that all points in “A” are critical and those on “B” are not.

Dropping SemiSL for SelfSL

There are two things worth mentioning about these four learning styles:

As already hinted at, most of the UnSL being done in Computer Vision and NLP recently is better described as SelfSL. This new learning style is not supervised using a given ground-truth, but using information contained in the training data itself. However, there are still parts of the “old family” of UnSL algorithms, which are truly unsupervised, that use some metric of closeness or proximity between data points to decide what is a good fit (and guide our loss function).

Also, if you think about it, SemiSL should not be a part of these “pure” learning styles. One, it is rather a mix of two “pure” learning styles and two, its basic setup involves having two different datasets, one labeled and one unlabeled. Therefore, we will save SemiSL for the next post of the “Structuring Machine Learning Concepts” series, where we will talk in more detail about processing unlabeled data.

Lightweight definitions — a new split

  • SelfSL: We only have raw input (e.g., images or lots of text) and we want to capture the implicit information it contains. We do so by using (parts of) the raw input as a target. In an Autoencoder, we are using the “reconstruction loss”, i.e., comparing the decoded image with the original input. In large scale language models, we are “hiding” parts of the sentence to then use it as a prediction target, only using the surrounding words (e.g., Marry ____ her husband → [loves]).
  • UnSL (revisited): In contrast to SelfSL, we are not measuring the “fit” of our model based on some “hidden” ground-truth but the implicit proximity of the “transformed input”, i.e., the distance of data points in the feature space. We can influence how this fit is defined, e.g., if we expect the clusters to be dense or linked continuously, but the proximity objective stays untouched. An example would be clustering users by behavior on a specific platform and using the assigned clusters to guide your marketing strategies.

The 2×2 Matrix

2×2 Matrix for SL, RL, SelfSL, & UnSL. Created by Author.

Iam proposing a simple 2×2 matrix, which maps SL, UnSL, SelfSL, & RL onto two axes, answering the following questions:

Does the ground-truth exist?

For SL and SelfSL, there is a ground-truth we are using to build our loss functions and metrics. Be it the “cat” label on an image for SL, driving the “categorical cross-entropy loss” and the “accuracy”, or the “hidden” word in a sentence (Marry [loves] her husband) for SelfSL, where we use “negative log-likelihood” as a loss and measure “perplexity”.

For UnSL and RL, there is no such ground-truth. We have measures that describe “good fit” or “desired behavior”, but nothing similar to “accuracy”. In “k-means clustering” of UnSL for example, we can measure the “average distance to the cluster mean”, and in RL, we are trying to maximize the “cumulative reward” we are receiving from the environment.

Is the objective explicitly or implicitly given?

In SL and RL, we have an explicit choice of what we want to get out of the data or our agent. For SL, it is our choice to turn a “cats and dogs breeds classification” problem into a simple “cats and dogs classification” problem, by re-assigning the labels. When using RL for mastering multi-player computer games, we can choose to incentivize our agent to act as a team player by rewarding the actions taken for the benefit of the group or to act as an egoist by solely rewarding individual actions.

However, we cannot extrinsically dictate the nature of an image or language in SelfSL. We can surely change some details, but the “reconstruction loss” will always compare two images, and for language models, we will always come up with learning tasks looking at the sentences themselves. With classical UnSL, we are implicitly stuck with finding data points that are close to each other, e.g., two users leaving behind similar behavior data on social media platforms.


Inthis post, we have redefined the “pure” learning styles in ML by separating UnSL and SelfSL and leaving SemiSL out of the equation. This brings us to the four concepts of SL, UnSL, SelfSL, and RL, which we can arrange in a simple framework (full disclosure: I did work in consulting for a while). The 2×2 matrix structures them according to whether a ground-truth exists and whether the objective is explicitly or implicitly given.

If you have any other unconventional mappings, thoughts on the matter, or are interested in our work at Luminovo, I would love to hear from you. You can leave a comment or reach me on Linkedin.

Stay tuned for the next articles.

Original post:

56 comentários em “The Four “Pure” Learning Styles in Machine Learning

  1. Hey there! I simply wish to offer you a big thumbs up for
    your great information you’ve got right here on this
    post. I will be returning to your web site for more soon.

  2. I’m the co-founder of JustCBD Store brand ( and am seeking to broaden my wholesale side of company. I really hope that anybody at targetdomain is able to provide some guidance . I thought that the most suitable way to accomplish this would be to connect to vape shops and cbd retailers. I was hoping if anyone could suggest a reliable site where I can buy CBD Shops Sales Leads I am already reviewing, and Not exactly sure which one would be the best option and would appreciate any advice on this. Or would it be much simpler for me to scrape my own leads? Ideas?

  3. I’m the owner of JustCBD Store brand ( and I am currently looking to develop my wholesale side of company. It would be great if someone at targetdomain share some guidance 🙂 I thought that the very best way to do this would be to talk to vape stores and cbd stores. I was hoping if anybody could suggest a reputable web site where I can get Vape Shop B2B Marketing List I am presently considering, and Not exactly sure which one would be the most suitable option and would appreciate any assistance on this. Or would it be easier for me to scrape my own leads? Suggestions?

  4. Hello there! This post couldn’t be written much better! Looking at this article reminds me of my previous roommate! He continually kept preaching about this. I’ll forward this information to him. Pretty sure he’s going to have a very good read. Thanks for sharing!

  5. After looking into a few of the blog posts on your web site, I honestly appreciate your way of writing a blog. I bookmarked it to my bookmark website list and will be checking back soon. Take a look at my web site too and tell me what you think.

  6. I really love your blog.. Pleasant colors & theme. Did you create this amazing site yourself? Please reply back as I’m planning to create my very own site and would love to know where you got this from or just what the theme is called. Many thanks!

  7. Wow that was odd. I just wrote an very long comment but after I clicked submit my comment didn’t appear.
    Grrrr… well I’m not writing all that over again. Anyway, just wanted to say excellent blog!

  8. An outstanding share! I’ve just forwarded this onto a co-worker who had been doing a little homework on this. And he in fact ordered me dinner because I stumbled upon it for him… lol. So allow me to reword this…. Thanks for the meal!! But yeah, thanx for spending the time to talk about this subject here on your web site.

  9. Nice post. I learn something new and challenging on sites I stumbleupon everyday. It’s always exciting to read content from other authors and use something from other websites.

  10. Hi! I could have sworn I’ve visited this blog before but after browsing through some of the posts I realized it’s new to me. Nonetheless, I’m definitely delighted I stumbled upon it and I’ll be book-marking it and checking back frequently!

  11. Achieving your fitness goal does not have to require a certified personal trainer or an expensive gym membership, it is not hard to exercise at home. It is easy to go down a training and fitness rabbit hole, however, when you’re looking for the best home exercise equipment to outfit your personal home gym.

  12. I must thank you for the efforts you’ve put in writing this site. I’m hoping to view the same high-grade content by you later on as well. In truth, your creative writing abilities has motivated me to get my own site now 😉

  13. Can I simply say what a relief to uncover someone who really understands what they are talking about on the net. You definitely know how to bring an issue to light and make it important. A lot more people need to read this and understand this side of your story. I was surprised you aren’t more popular given that you surely possess the gift.

  14. Hi, I do believe your blog may be having browser compatibility problems. Whenever I look at your web site in Safari, it looks fine however, if opening in I.E., it has some overlapping issues. I simply wanted to provide you with a quick heads up! Aside from that, fantastic site!

  15. Hello, I believe your blog could be having browser compatibility issues. Whenever I look at your website in Safari, it looks fine however when opening in I.E., it’s got some overlapping issues. I simply wanted to provide you with a quick heads up! Apart from that, excellent website!

  16. After I initially commented I seem to have clicked on the -Notify me when new comments are added- checkbox and from now on whenever a comment is added I recieve four emails with the same comment. Is there a way you are able to remove me from that service? Thank you!

  17. I was excited to discover this website. I need to to thank you for your time for this particularly wonderful read!! I definitely enjoyed every part of it and I have you book-marked to check out new information in your site.

  18. I really love your website.. Great colors & theme. Did you build this website yourself? Please reply back as I’m trying to create my own website and would love to find out where you got this from or what the theme is named. Appreciate it!

  19. Hello there! I could have sworn I’ve visited this web site before but after going through some of the posts I realized it’s new to me. Nonetheless, I’m definitely happy I found it and I’ll be book-marking it and checking back regularly!

  20. This is the right web site for everyone who wishes to find out about this topic. You know a whole lot its almost hard to argue with you (not that I actually would want to…HaHa). You definitely put a fresh spin on a topic that’s been written about for years. Great stuff, just wonderful!

  21. I blog frequently and I seriously thank you for your content. This great article has truly peaked my interest. I am going to bookmark your blog and keep checking for new information about once a week. I opted in for your RSS feed too.

  22. After I initially commented I appear to have clicked the -Notify me when new comments are added- checkbox and now whenever a comment is added I receive 4 emails with the exact same comment. There has to be a way you can remove me from that service? Thanks a lot!

  23. After looking over a handful of the blog articles on your site, I really appreciate your technique of writing a blog. I bookmarked it to my bookmark site list and will be checking back in the near future. Take a look at my web site too and tell me how you feel.

Leave a Reply

Your email address will not be published. Required fields are marked *