With the continuous development of network technology and the ever-expanding scale of e-commerce, the number and variety of goods grow rapidly and users need to spend a lot of time to find the goods they want to buy. This is information overload. To solve this problem, the recommendation system came into being.
The recommendation system is a subset of the Information Filtering System, which can be used in a range of areas such as movies, music, e-commerce, and Feed stream recommendations. The recommendation system discovers the user’s personalized needs and interests by analyzing and mining user behaviors and recommends information or products that may be of interest to the user. Unlike search engines, recommendation systems do not require users to accurately describe their needs but model their historical behavior to proactively provide information that meets user interests and needs.
In this article we use PaddlePaddle, a deep learning platform from Baidu, to build a model and combine Milvus, a vector similarity search engine, to build a personalized recommendation system that can quickly and accurately provide users with information that might be of interest to them.
Data Preparation
We take MovieLens Million Dataset (ml-1m) [1] as an example. The ml-1m dataset contains 1,000,000 reviews of 4,000 movies by 6,000 users, collected by the GroupLens Research lab. The original data includes feature data of the movie, user feature, and user rating of the movie, you can refer to ml-1m-README [2].
ml-1m dataset includes 3 .dat articles: movies.dat、users.dat and ratings.dat.movies.dat includes movie’s features, see example below:
MovieID::Title::Genres
This means that the movie id is 1, and the title is 《Toy Story》, which is divided into three categories. These three categories are animation, children, and comedy.
users.dat includes user’s features, see example below:
UserID::Gender::Age::Occupation::Zip-code
1::F::1::10::48067
This means that the user ID is 1, female, and younger than 18 years old. The occupation ID is 10.
ratings.dat includes the feature of movie rating, see example below:
UserID::MovieID::Rating::Timestamp
1::1193::5::978300760
That is, the user 1 evaluates the movie 1193 as 5 points.
Fusion Recommendation Model
In the film personalized recommendation system, we used the Fusion Recommendation Model [3] which PaddlePaddle has implemented. This model is created from its industrial practice.
- First, take user features and movie features as input to the neural network, where:
a. The user features incorporate four attribute information: user ID, gender, occupation, and age.
b. The movie feature incorporates three attribute information: movie ID, movie type ID, and movie name.
- For the user feature, map the user ID to a vector representation with the dimension size of 256, enter the fully connected layer, and do similar processing for the other three attributes. Then the feature representations of the four attributes are fully connected and added separately.
- For movie features, the movie ID is processed like the user ID. The movie type ID is directly input into the fully connected layer in the form of a vector, and the movie name is represented by a fixed-length vector using a text convolutional neural network. The feature representations of the three attributes are then fully connected and added separately.
- After obtaining the vector representation of the user and the movie, calculate the cosine similarity of them as the score of the personalized recommendation system. Finally, the square of the difference between the similarity score and the user’s true score is used as the loss function of the regression model.
System Overview
Combined with PaddlePaddle’s fusion recommendation model, the movie feature vector generated by the model is stored in the Milvus vector similarity search engine, and the user feature is used as the target vector to be searched. A similarity search is performed in Milvus to obtain the query result as the recommended movies for the user.
The inner product (IP) method is provided in Milvus to calculate the vector distance. After normalizing the data, the inner product similarity is consistent with the cosine similarity result in the fusion recommendation model.
Application of Personalized Recommender System
There are three steps in building a movie recommendation system with Milvus, details on how to operate please refer to Milvus Bootcamp [4].
Step 1：Model Training
$ python train.py
Running this command will generate a model recommender_system.inference.model in the directory, which can convert movie data and user data into feature vectors, and generate application data for Milvus to store and retrieve.
Step 2：Data Preprocessing
# Data preprocessing, -f followed by the parameter raw movie data file name
$ python get_movies_data.py -f movies_origin.txt
Running this command will generate test data movies_data.txt in the directory to achieve the pre-processing of movie data.
Step 3：Implementing Personal Recommender System with Milvus
# Implementing personal recommender system based on user conditions
$ python infer_milvus.py -a <age>-g <gender>-j <job>[-i]
Running this command will implement personalized recommendations for specified users.
The main process is:
- Through the load_inference_model, the movie data is processed by the model to generate a movie feature vector.
- Load the movie feature vector into Milvus via milvus.insert.
- According to the user’s age/gender/occupation specified by the parameters, it is converted into a user feature vector, milvus.search_vectors is used for similarity retrieval, and the result with the highest similarity between the user and the movie is returned.
Prediction of the top five movies that the user is interested in:
TopIdsTitleScore
03030Yojimbo2.9444923996925354
13871Shane2.8583481907844543
23467Hud2.849525213241577
31809Hana-bi2.826111316680908
43184Montana2.8119677305221558
Summary
By inputting user information and movie information to the fusion recommendation model we can get matching scores, and then sort the scores of all movies based on the user to recommend movies that may be of interest to the user. This article combines Milvus and PaddlePaddle to build a personalized movie recommendation system. Milvus, a vector search engine, is used to store all movie feature data, and then similarity retrieval is performed on user features in Milvus. The search result is the movie ranking recommended by the system to the user.
Milvus [5] vector similarity search engine is compatible with various deep learning platforms, searching billions of vectors with only millisecond response. You can explore more possibilities of AI applications with Milvus with ease!
References
[1] MovieLens Million Dataset (ml-1m): http://files.grouplens.org/datasets/movielens/ml-1m.zip
[2] ml-1m-README: http://files.grouplens.org/datasets/movielens/ml-1m-README.txt
[3] Fusion Recommendation Model by PaddlePaddle: https://www.paddlepaddle.org.cn/documentation/docs/zh/beginners_guide/basics/recommender_system/index.html#id7
[4] Bootcamp: https://github.com/milvus-io/bootcamp/blob/master/demo/recommender_system
[5] Milvus: https://milvus.io/en/
Original post: https://www.datasciencecentral.com/profiles/blogs/building-a-deep-learning-based-movie-recommender-system
I am the business owner of JustCBD brand (justcbdstore.com) and I’m presently aiming to expand my wholesale side of business. I am hoping someone at targetdomain give me some advice . I considered that the most effective way to accomplish this would be to talk to vape shops and cbd retailers. I was really hoping if anyone could recommend a trusted web site where I can buy Vape Shop B2B Database I am currently taking a look at creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. Not sure which one would be the very best selection and would appreciate any advice on this. Or would it be easier for me to scrape my own leads? Suggestions?
I am the owner of JustCBD Store label (justcbdstore.com) and I am currently seeking to broaden my wholesale side of company. It would be great if someone at targetdomain is able to provide some guidance . I thought that the very best way to accomplish this would be to talk to vape shops and cbd retailers. I was hoping if someone could suggest a reputable site where I can get UK Vape Shop Database I am already taking a look at creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. Not exactly sure which one would be the best choice and would appreciate any assistance on this. Or would it be easier for me to scrape my own leads? Suggestions?
It’s hard to find well-informed people for this topic, but you seem like you know what you’re talking about! Thanks
Great article! We are linking to this great post on our site. Keep up the great writing.
Saved as a favorite, I like your site!
This excellent website certainly has all of the information and facts I needed about this subject and didn’t know who to ask.
This is the perfect site for everyone who wants to understand this topic. You know so much its almost tough to argue with you (not that I personally would want to…HaHa). You certainly put a fresh spin on a subject that’s been discussed for ages. Wonderful stuff, just great!
I need to to thank you for this wonderful read!! I certainly loved every bit of it. I’ve got you book-marked to look at new things you post…
Saved as a favorite, I love your website!
I blog often and I really appreciate your content. The article has really peaked my interest. I’m going to bookmark your site and keep checking for new details about once a week. I subscribed to your Feed too.
I couldn’t resist commenting. Perfectly written!
This is the right web site for anybody who wishes to find out about this topic. You know a whole lot its almost hard to argue with you (not that I personally will need to…HaHa). You definitely put a fresh spin on a subject which has been discussed for a long time. Wonderful stuff, just excellent!
Way cool! Some very valid points! I appreciate you penning this post and also the rest of the site is really good.
Way cool! Some very valid points! I appreciate you penning this post and the rest of the site is very good.
Very good post! We will be linking to this particularly great content on our website. Keep up the good writing.
An interesting discussion is definitely worth comment. I do think that you ought to publish more about this subject, it may not be a taboo matter but usually people don’t speak about such issues. To the next! Best wishes!!
Right here is the perfect blog for anybody who wants to understand this topic. You realize a whole lot its almost hard to argue with you (not that I really would want to…HaHa). You definitely put a fresh spin on a subject that has been written about for ages. Excellent stuff, just great!
Having read this I believed it was extremely enlightening. I appreciate you spending some time and effort to put this informative article together. I once again find myself spending way too much time both reading and posting comments. But so what, it was still worthwhile!
Good site you have got here.. It’s difficult to find high quality writing like yours these days. I seriously appreciate people like you! Take care!!
After I initially left a comment I appear to have clicked the -Notify me when new comments are added- checkbox and now whenever a comment is added I get four emails with the same comment. Perhaps there is a way you can remove me from that service? Kudos!
Hi! I could have sworn I’ve visited this website before but after looking at some of the posts I realized it’s new to me. Regardless, I’m certainly happy I stumbled upon it and I’ll be book-marking it and checking back regularly!
Howdy! This article could not be written much better! Looking through this post reminds me of my previous roommate! He constantly kept preaching about this. I’ll forward this article to him. Fairly certain he’ll have a good read. I appreciate you for sharing!
Saved as a favorite, I really like your site!
This site was… how do I say it? Relevant!! Finally I have found something which helped me. Thanks!
This blog was… how do you say it? Relevant!! Finally I have found something that helped me. Thanks!
You ought to be a part of a contest for one of the finest sites on the net. I most certainly will recommend this web site!
There’s definately a lot to learn about this subject. I like all the points you have made.
You should take part in a contest for one of the highest quality blogs online. I will recommend this blog!
This is a topic that’s close to my heart… Thank you! Where are your contact details though?
You are so awesome! I don’t believe I have read through something like that before. So good to discover another person with some genuine thoughts on this topic. Seriously.. thank you for starting this up. This site is something that’s needed on the internet, someone with a bit of originality!
Spot on with this write-up, I honestly believe that this amazing site needs a great deal more attention. I’ll probably be returning to see more, thanks for the info!
Wonderful article! We are linking to this particularly great content on our site. Keep up the great writing.
Very good info. Lucky me I recently found your website by chance (stumbleupon).
I’ve book-marked it for later!
Hi, I want to subscribe for this webpage to
take newest updates, thus where can i do it please
help out. adreamoftrains best hosting
Great beat ! I would like to apprentice while you amend your website, how could i subscribe for a blog site?
The account aided me a acceptable deal. I had been a little
bit acquainted of this your broadcast offered brilliant transparent concept
Hi, I do believe this is a great blog. I stumbledupon it 😉 I may come back yet again since i have book marked it. Money and freedom is the greatest way to change, may you be rich and continue to guide others.
After going over a few of the blog posts on your blog, I really appreciate your technique of blogging. I bookmarked it to my bookmark webpage list and will be checking back in the near future. Please check out my website too and tell me how you feel.
You have made some good points there. I checked on the web for additional information about the issue and found most people will go along with your views on this website.
Hi, I do believe this is a great website. I stumbledupon it 😉 I’m going to come back yet again since i have book-marked it. Money and freedom is the best way to change, may you be rich and continue to help other people.
Good write-up. I absolutely appreciate this site. Keep writing!
It’s nearly impossible to find well-informed people about this subject, but you seem like you know what you’re talking about! Thanks
This is the right web site for anyone who wants to find out about this topic. You realize a whole lot its almost hard to argue with you (not that I actually will need to…HaHa). You definitely put a brand new spin on a subject that’s been discussed for a long time. Wonderful stuff, just great!
Hi! I could have sworn I’ve been to this blog before but after looking at many of the posts I realized it’s new to me. Regardless, I’m certainly pleased I came across it and I’ll be book-marking it and checking back frequently!
Saved as a favorite, I really like your website!
I need to to thank you for this great read!! I definitely loved every bit of it. I have got you book marked to look at new stuff you post…
Aw, this was an incredibly nice post. Taking a few minutes and actual effort to make a really good article… but what can I say… I put things off a whole lot and don’t manage to get nearly anything done.
May I just say what a relief to uncover an individual who actually understands what they’re talking about online. You definitely understand how to bring an issue to light and make it important. More and more people should read this and understand this side of your story. I can’t believe you aren’t more popular given that you most certainly possess the gift.
You should be a part of a contest for one of the finest blogs on the web. I most certainly will highly recommend this website!
You have made some good points there. I looked on the net to learn more about the issue and found most people will go along with your views on this site.
Howdy! I could have sworn I’ve visited this site before but after browsing through some of the posts I realized it’s new to me. Anyhow, I’m definitely happy I found it and I’ll be book-marking it and checking back often!
I could not resist commenting. Exceptionally well written!
The next time I read a blog, Hopefully it does not fail me just as much as this particular one. I mean, Yes, it was my choice to read through, but I genuinely believed you would probably have something helpful to say. All I hear is a bunch of whining about something you could fix if you weren’t too busy searching for attention.
Your style is unique in comparison to other folks I have read stuff from. Many thanks for posting when you have the opportunity, Guess I will just book mark this blog.
Right here is the right web site for anybody who wants to find out about this topic. You know so much its almost hard to argue with you (not that I really would want to…HaHa). You certainly put a brand new spin on a subject that has been written about for many years. Wonderful stuff, just excellent!
An interesting discussion is worth comment. I think that you should write more about this subject matter, it may not be a taboo subject but typically folks don’t talk about such issues. To the next! Kind regards!!
This is the perfect site for anyone who really wants to find out about this topic. You realize a whole lot its almost hard to argue with you (not that I personally would want to…HaHa). You certainly put a fresh spin on a topic that has been written about for years. Great stuff, just great!
I really love your website.. Excellent colors & theme. Did you create this amazing site yourself? Please reply back as I’m hoping to create my own blog and would like to learn where you got this from or exactly what the theme is called. Many thanks!
You should be a part of a contest for one of the highest quality websites on the internet. I am going to highly recommend this blog!
Hi there! This post couldn’t be written any better! Looking through this post reminds me of my previous roommate! He constantly kept talking about this. I most certainly will forward this article to him. Fairly certain he’s going to have a very good read. Thanks for sharing!
I could not resist commenting. Very well written!|
There’s certainly a lot to find out about this subject. I love all of the points you made.
Aw, this was a really good post. Spending some time and actual effort to produce a top notch article… but what can I say… I put things off a whole lot and never seem to get anything done.
I’m amazed, I have to admit. Rarely do I come across a blog that’s both educative and engaging, and without a doubt, you have hit the nail on the head. The issue is something not enough men and women are speaking intelligently about. I am very happy I came across this during my hunt for something regarding this.
Hi, I do think this is a great website. I stumbledupon it 😉 I may return once again since i have saved as a favorite it. Money and freedom is the best way to change, may you be rich and continue to help others.
Can you tell us more about this? I’d want to find out some additional information.
Having read this I believed it was extremely enlightening. I appreciate you taking the time and effort to put this content together. I once again find myself spending a lot of time both reading and leaving comments. But so what, it was still worthwhile!
Everything is very open with a really clear explanation of the issues. It was definitely informative. Your website is very helpful. Many thanks for sharing!
I’ve been surfing online more than 3 hours today, yet I never found any interesting article like yours. It’s pretty worth enough for me. In my opinion, if all site owners and bloggers made good content as you did, the internet will be a lot more useful than ever before.|
That is a very good tip especially to those fresh to the blogosphere. Brief but very precise info… Thanks for sharing this one. A must read post!
It’s nearly impossible to find knowledgeable people in this particular subject, but you sound like you know what you’re talking about! Thanks
Achieving your fitness goal doesn’t need a certified personal trainer or an expensive gym memberships, it is not hard to exercise at home. It’s easy to go down a training and fitness rabbit hole, however, when you are looking for the best home exercise equipment to outfit your personal home gym.
These are in fact great ideas in about blogging.
You have touched some nice things here. Any way keep up wrinting.
Amazing blog! Is your theme custom made or did
you download it from somewhere? A design like yours with a few simple adjustements
would really make my blog shine. Please let me know where you got your design. With thanks
Very good info. Lucky me I found your site by
chance (stumbleupon). I have bookmarked
it for later!
Awesome things here. I’m very satisfied to look your article.
Thanks a lot and I am taking a look forward to touch you.
Will you please drop me a mail?
Great article. I will be going through a few of these issues as well..
The very next time I read a blog, I hope that it does not fail me as much as this one. I mean, I know it was my choice to read, nonetheless I genuinely thought you’d have something interesting to say. All I hear is a bunch of whining about something you can fix if you were not too busy seeking attention.
I love it when folks come together and share views. Great website, keep it up!
It’s an amazing article for all the online viewers; they will get advantage from it I am sure.|
Does your website have a contact page? I’m having trouble locating it but,
I’d like to shoot you an e-mail. I’ve got some creative ideas for your blog you might be interested in hearing.
Either way, great website and I look forward to seeing it improve
over time.
For most recent information you have to go to see web and
on the web I found this website as a most
excellent web site for hottest updates.
I like it when individuals come together and share views.
Great site, continue the good work!
Heya i’m for the primary time here. I found this board and I in finding It really useful & it helped me out much.
I hope to offer something back and help others like you helped me.
Because the admin of this website is working, no doubt very shortly it will be well-known, due to its quality contents.
Appreciate the recommendation. Will try it out.
Hi there! I could have sworn I’ve been to your blog before but after looking at some of the articles I realized it’s new to me.
Nonetheless, I’m definitely happy I stumbled upon it and
I’ll be bookmarking it and checking back regularly!
I do not know whether it’s just me or if perhaps everybody else
experiencing issues with your site. It appears as if some of the text in your content are running off the screen. Can someone else please provide feedback and
let me know if this is happening to them as well?
This may be a problem with my web browser because I’ve had this
happen previously. Thanks
I like it when people get together and share views. Great site, keep it up!
I’m amazed, I have to admit. Seldom do I come across a blog that’s both educative and engaging, and without a doubt, you’ve hit the nail on the head. The issue is something which not enough men and women are speaking intelligently about. I am very happy that I came across this in my hunt for something regarding this.
I love it when individuals get together and share views. Great website, continue the good work!
Can I simply just say what a relief to discover a person that genuinely knows what they are talking about online. You definitely realize how to bring a problem to light and make it important. More people have to look at this and understand this side of your story. I was surprised that you aren’t more popular since you surely possess the gift.
Good day! I know this is kinda off topic but I was wondering which blog platform are
you using for this site? I’m getting fed up of WordPress because I’ve had issues with hackers and I’m looking at alternatives for
another platform. I would be great if you could point
me in the direction of a good platform.
Oh my goodness! Impressive article dude! Thank you so much, However I am going through problems with your RSS. I don’t understand the reason why I cannot subscribe to it. Is there anyone else having the same RSS issues? Anyone that knows the answer will you kindly respond? Thanks!!
This post is truly a nice one it assists new net viewers, who are wishing for blogging.
Right here is the right webpage for everyone who wants to find out about this topic. You know so much its almost hard to argue with you (not that I really would want to…HaHa). You certainly put a brand new spin on a topic that’s been written about for a long time. Excellent stuff, just great!
I blog quite often and I truly appreciate your information. The article has really peaked my interest. I’m going to bookmark your site and keep checking for new details about once a week. I subscribed to your RSS feed too.
Excellent article. I am going through some of these issues as well..
I must thank you for the efforts you have put in writing this site. I really hope to view the same high-grade content from you later on as well. In truth, your creative writing abilities has inspired me to get my own, personal blog now 😉
Hello there! I could have sworn I’ve been to this site before but after browsing through a few of the posts I realized it’s new to me. Anyways, I’m definitely pleased I discovered it and I’ll be bookmarking it and checking back regularly!
This web site really has all of the information and facts I needed about this subject and didn’t know who to ask.
Very good post! We will be linking to this great content on our site. Keep up the good writing.
Nice post. I learn something new and challenging on blogs I stumbleupon on a daily basis. It’s always interesting to read articles from other writers and practice a little something from other websites.
I want to to thank you for this good read!! I absolutely enjoyed every little bit of it. I’ve got you bookmarked to check out new things you post…
A ‘contributed’ method is primarily based on the proportion contributed to the pot and who has produced the contributions. The emblem was not just sticker, it was really produced into the chip itself.
Hi, I do think this is an excellent site. I stumbledupon it 😉 I am going to come back once again since i have book marked it. Money and freedom is the best way to change, may you be rich and continue to help others.
When I initially left a comment I seem to have clicked the -Notify me when new comments are added- checkbox and from now on every time a comment is added I receive 4 emails with the exact same comment. Is there an easy method you can remove me from that service? Thanks!
Excellent post. I am dealing with many of these issues as well..
There’s certainly a lot to learn about this topic. I really like all the points you made.
An impressive share! I’ve just forwarded this onto a colleague who has been doing a little research on this. And he actually ordered me lunch because I found it for him… lol. So allow me to reword this…. Thanks for the meal!! But yeah, thanks for spending some time to discuss this matter here on your blog.
I just want to mention I’m all new to blogging and site-building and honestly savored your website. More than likely I’m likely to bookmark your blog . You certainly have excellent articles. Thanks a lot for revealing your blog.
Major thanks for the article.Much thanks again. Awesome.
Yesterday, while I was at work, my cousin stole my iPad and tested to see if it can survive a thirty foot drop, just so she can be a youtube sensation. My apple ipad is now broken and she has 83 views.
I know this is totally off topic but I had to share it with
someone!