Breaking Privacy in Federated Learning

Federated learning is a new way of training a machine learning using distributed data that is not centralized in a server. It works by training a generic (shared) model with a given user’s private data, without having direct access to such data.

For a deeper dive into how this works, I’d encourage you to check out my previous blog post, which provides a high-level overview, as well as an in depth look at Google’s research.

Federated learning has the major benefit of building models that are customized based on a user’s private data, which allows for better customization that can enhances the UX. This, as compared to models trained by the data aggregated at a data center that are more generic and may not fit the user quite as well. Federated learning also help save a user’s bandwidth, since they aren’t sending private data to a server.

Despite the benefits of federated learning, there are still ways of breaching a user’s privacy, even without sharing private data. In this article, we’ll review some research papers that discuss how federated learning includes this vulnerability.

The outline of the article is as follows:

  • Introduction

Let’s get started.

Introduction

Federated learning was introduced by Google in 2016 in a paper titled Communication-Efficient Learning of Deep Networks from Decentralized Data. It’s a new machine learning paradigm that allows us to build machine learning models from private data, without sharing such data to a data center.

The summary of the steps we take to do this is as follows:

  • A generic model (i.e. neural network) is created at a server. The model will not be trained on the server but on the users’ devices (the majority are mobile devices).

This way, a model is trained using private data without being moved from the devices. The next figure from a post by Jose Corbacho summarizes the previous steps.

Even though the data isn’t shared with the server, the process is not 100% private, and there’s still a possibility of obtaining information about the data used to train the network and calculate the gradients. The next section discusses how privacy is not entirely preserved using federated learning.


Machine learning is rapidly moving closer to where data is collected — edge devices. Subscribe to the Fritz AI Newsletter to learn more about this transition and how it can help scale your business.


Federated Learning Doesn’t Guarantee Privacy

Federated learning has some privacy advantages as compared to sharing private data with data centers. The benefits also include the ability to build highly-customized machine learning models based on the user data, while avoiding using hits to a user’s bandwidth for transferring the private data to the server.

Undoubtedly, not sharing the data with data centers and keeping it private is an advantage—but there are still some risks. The reason is that there remains a way to extract some private information from the data.

After the generic model is trained at the user’s device, the trained model is sent to the server. Given that the model’s parameters are trained based on the user’s data, there is a chance of getting information about the data from such parameters.

Moreover, joining the user’s data with data from other users has some risks and this is mentioned in the Google research paper:

Holding even an “anonymized” dataset can still put user privacy at risk via joins with other data.

Here, the seminal paper on federated learning makes it clear that there are still some risks, and data privacy is not 100% guaranteed. Even if the data is anonymized, it’s still vulnerable.

The updates transmitted from the device to the server should be minimal. There’s no need to share more than the minimum amount of info required to update the model at the server. Otherwise, there remains the possibility of private data being exposed and intercepted.

The private data is this vulnerable, even without being sent explicitly to the server because it’s possible to restore it based on the parameters trained by such data. In the worst case when an attacker is able to restore the data, it should be anonymous as much as possible without revealing some user’s private information like the name for example.

It’s possible to reveal the words entered by a user based on the gradients for some simple NLP models. In these case, if the private data already contains some information (i.e. words) about the user, then such words could be restored, and thus the privacy would also not be preserved.

The original paper for federated learning didn’t mention a clear example in which the private data could not be deduced, but it mentioned a case in which it would be difficult (which implies still possible) to extract information about a user’s private data by averaging and summing gradients. The example they include involves revealing information about private data from complex networks like CNNs. Here’s what the paper mentioned:

The sum of many gradients for a dense model such as a CNN offers a harder target for attackers seeking information about individual training instances.

In essence, there’s no way to 100% prevent an attacker from getting information about the samples used for calculating the gradients of a neural network. But the key is making things harder for the attacker to get such information. It’s like a cavern puzzle, where you should make it difficult as possible to solve.

This is the case for a convolutional neural network (CNN) because it usually has many layers connecting the input to the output, resulting in a large number of interleaving gradients. These gradients render it difficult (though attacks are still possible) to find a relationship between the inputs and the outputs based on the available gradients.

To summarize the previous discussion—even if the private data itself is not shared with the server, the gradients of the trained network are, which makes it possible to extract information about the training samples. The paper discussed 2 main measures you should take to maximize privacy:

  1. Sharing the minimum of information required to update the generic model at the server.

The next section summarizes a paper that discusses some specific privacy and security issues related to federated learning.


Machine learning models don’t have to live on servers or in the cloud — they can also live on your smartphone. And Fritz AI has the tools to easily teach mobile apps to see, hear, sense, and think.


Privacy and Security Issues of Federated Learning

In a recent paper—Ma, Chuan, et al. “On safeguarding privacy and security in the framework of federated learning.” IEEE Network (2020)—a number of privacy and security issues related to federated learning are discussed.

The paper started by introducing the basic model for federated learning, according to the next figure. This figure shares some similarities to Jose Corbacho’s post.

The paper addresses both the security and privacy issues for federated learning. The difference between security and privacy issues is that security issues refer to unauthorized/malicious access, change or denial to data while privacy issues refer to unintentional disclosure of personal information.

The paper classified the protection methods for the privacy and security issues into 3 categories, which are:

  1. Privacy protection at the client-side

Privacy Protection at the Client-Side

Regarding privacy protection at the client-side, the paper discussed 2 ways which are perturbation and dummy:

  • Perturbation: Adding noise to the shared parameters to the server so that attackers cannot restore the data or at least not able to get the identity of the user.

Privacy Protection at the Server-Side

Privacy protection at the server side is necessary because, as the paper mentioned, when the server broadcasts the aggregated parameters to clients for model synchronizing, this information may leak as there may exist eavesdroppers. The paper mentioned some ways to preserve the privacy at the server side which are aggregation and secure multi-party computation.

  • Aggregation: To make revealing information about the user’s data more complex, the parameters from different users are combined together.

Security Protection for the Federated Learning Framework

After the client trains the model by its private data, the model is sent to the server. At this time, an attacker might make some changes to the model to make it behave for their benefit. For example, the attacker might control the labels assigned to images with certain features.

The paper suggests 2 ways to secure the design of a federated learning pipeline: homomorphic encryption and back-door defender.

  • Homomorphic Encryption: The model parameters are encrypted so that an attacker finds it difficult to interpret; thus, they’re unable to be changed.

The next section provides a quick summary of a paper that is able to reconstruct images by inverting gradients.

Reconstructing Private Data by Inverting Gradients

According to a recent research paper—Geiping, Jonas, et al. “Inverting Gradients — How easy is it to break privacy in federated learning?” arXiv preprint arXiv:2003.14053 (2020)—simply sharing the gradients but not the private data still uncovers private information about the data. Thus, federated learning has not entirely achieved one of its goals, which is keeping user’s data private.

As we mentioned in the previous section, one thing that makes it harder for an attacker to get information about private data is the existence of many gradients, like those available in CNNs. The main proposal of this paper is to reconstruct images based on the gradients of the neural network with high quality. Successfully doing that means the privacy is not guaranteed even if just the parameters, not the data is shared with the server.

The paper proved that the input to a fully connected layer could be reconstructed independently of the network architecture. Even if the gradients are averaged through a number of iterations, this doe not help to protect the user’s privacy.

The paper proves that it is possible to recover much of the information available in the original data. The key findings from this paper are summarized in the following points:

  1. Reconstruction of input data from gradient information is possible for realistic deep architectures with both trained and untrained parameters.

The next figure, taken from the paper shows, an image, and its reconstruction. The original image is reconstructed with high quality (with little degradation) based on the shared gradients to the server.

Conclusion

This article discussed some of the privacy and security issues in federated learning by summarizing 3 papers. It’s clear that it’s immensely challenging to preserve a user’s privacy, even if only sharing gradients returned by training the global model (e.g. neural network). Even though the data used for training local network updates isn’t shared, it is possible to reconstruct that data.


Editor’s Note:Heartbeatis a contributor-driven online publication and community dedicated to exploring the emerging intersection of mobile app development and machine learning. We’re committed to supporting and inspiring developers and engineers from all walks of life.

Editorially independent, Heartbeat is sponsored and published byFritz AI, the machine learning platform that helps developers teach devices to see, hear, sense, and think. We pay our contributors, and we don’t sell ads.

If you’d like to contribute, head on over to ourcall for contributors. You can also sign up to receive our weekly newsletters (Deep Learning Weekly and the Fritz AI Newsletter), join us onSlack, and follow Fritz AI onTwitter for all the latest in mobile machine learning.

Original post: https://heartbeat.fritz.ai/breaking-privacy-in-federated-learning-77fa08ccac9a

78 comentários em “Breaking Privacy in Federated Learning

  1. I’m the proprietor of JustCBD label (justcbdstore.com) and am aiming to expand my wholesale side of business. I am hoping someone at targetdomain is able to provide some guidance ! I considered that the most effective way to do this would be to talk to vape companies and cbd retail stores. I was hoping if anybody at all could suggest a qualified website where I can get Vape Shop B2B Data List I am already reviewing creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. On the fence which one would be the best option and would appreciate any advice on this. Or would it be much simpler for me to scrape my own leads? Ideas?

  2. I am the co-founder of JustCBD Store company (justcbdstore.com) and I am currently looking to develop my wholesale side of company. It would be great if someone at targetdomain share some guidance 🙂 I considered that the very best way to do this would be to connect to vape companies and cbd retailers. I was really hoping if anybody could recommend a reliable web site where I can get Vape Shop B2B Marketing List I am already considering creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. On the fence which one would be the very best solution and would appreciate any guidance on this. Or would it be much simpler for me to scrape my own leads? Suggestions?

  3. Aw, this was a very good post. Taking the time and actual effort to make a very good article… but what can I say… I hesitate a lot and don’t manage to get nearly anything done.

  4. A motivating discussion is definitely worth comment. I believe that you need to publish more about this subject matter, it might not be a taboo subject but usually folks don’t speak about such subjects. To the next! Cheers!!

  5. I seriously love your blog.. Pleasant colors & theme. Did you develop this amazing site yourself? Please reply back as I’m wanting to create my very own blog and would love to know where you got this from or exactly what the theme is named. Cheers!

  6. I blog frequently and I seriously thank you for your content. This article has truly peaked my interest. I’m going to take a note of your blog and keep checking for new details about once a week. I subscribed to your Feed as well.

  7. I’d like to thank you for the efforts you have put in penning this blog. I’m hoping to view the same high-grade blog posts from you in the future as well. In fact, your creative writing abilities has encouraged me to get my own blog now 😉

  8. Hi there! This post couldn’t be written much better! Reading through this article reminds me of my previous roommate! He always kept talking about this. I’ll forward this post to him. Fairly certain he’ll have a good read. Thanks for sharing!

  9. An interesting discussion is definitely worth comment. There’s no doubt that that you ought to publish more about this topic, it might not be a taboo matter but typically people do not talk about such issues. To the next! All the best!!

  10. Hi there! This post couldn’t be written much better! Looking at this post reminds me of my previous roommate! He continually kept talking about this. I will send this information to him. Pretty sure he’ll have a very good read. I appreciate you for sharing!

  11. Hello there! I just wish to give you a big thumbs up for your great information you have got here on this post. I will be returning to your website for more soon.

  12. This is the perfect webpage for anybody who hopes to understand this topic. You know a whole lot its almost tough to argue with you (not that I actually would want to…HaHa). You certainly put a brand new spin on a topic that has been discussed for ages. Great stuff, just excellent!

  13. A motivating discussion is definitely worth comment. I believe that you ought to write more about this issue, it may not be a taboo subject but typically people don’t discuss such topics. To the next! Best wishes!!

  14. I was excited to find this site. I need to to thank you for your time for this wonderful read!! I definitely liked every little bit of it and I have you book-marked to see new things on your web site.

  15. It’s nearly impossible to find well-informed people for this subject, however, you seem like you know what you’re talking about! Thanks

  16. I blog frequently and I genuinely thank you for your content. This great article has truly peaked my interest. I am going to book mark your website and keep checking for new information about once per week. I subscribed to your Feed too.

  17. Hi there! This post could not be written any better! Going through this article reminds me of my previous roommate! He constantly kept preaching about this. I am going to forward this article to him. Pretty sure he will have a great read. Thanks for sharing!

  18. Your style is unique in comparison to other folks I’ve read stuff from. I appreciate you for posting when you have the opportunity, Guess I will just book mark this site.

  19. You’ve made some good points there. I looked on the net to learn more about the issue and found most individuals will go along with your views on this site.

  20. An outstanding share! I’ve just forwarded this onto a friend who had been conducting a little homework on this. And he in fact bought me breakfast because I stumbled upon it for him… lol. So allow me to reword this…. Thanks for the meal!! But yeah, thanks for spending some time to discuss this matter here on your web page.

  21. I’m amazed, I must say. Seldom do I encounter a blog that’s both educative and amusing, and let me tell you, you have hit the nail on the head. The problem is an issue that not enough people are speaking intelligently about. I am very happy that I came across this in my search for something regarding this.

  22. Hello! I could have sworn I’ve visited this website before but after going through some of the articles I realized it’s new to me. Anyways, I’m definitely pleased I came across it and I’ll be book-marking it and checking back regularly!

  23. Good post. I learn something new and challenging on blogs I stumbleupon everyday. It’s always interesting to read through articles from other authors and use a little something from other sites.

  24. Hello! I could have sworn I’ve visited your blog before but after going through many of the posts I realized it’s new to me. Regardless, I’m definitely happy I discovered it and I’ll be bookmarking it and checking back frequently!

  25. I’m more than happy to uncover this web site. I need to to thank you for ones time due to this fantastic read!! I definitely enjoyed every little bit of it and i also have you book-marked to see new stuff in your site.

  26. Having read this I thought it was really enlightening. I appreciate you finding the time and effort to put this article together. I once again find myself personally spending a lot of time both reading and posting comments. But so what, it was still worth it!

  27. Oh my goodness! Incredible article dude! Many thanks, However I am going through difficulties with your RSS. I don’t understand the reason why I am unable to subscribe to it. Is there anyone else having identical RSS issues? Anyone who knows the solution will you kindly respond? Thanks!!

  28. Hey would you mind letting me know which webhost you’re working with? I’ve loaded your blog in 3 completely different web browsers and I must say this blog loads a lot faster then most. Can you recommend a good web hosting provider at a honest price? Thanks a lot, I appreciate it!|

  29. I absolutely love your blog.. Pleasant colors & theme. Did you make this website yourself? Please reply back as I’m wanting to create my very own site and want to find out where you got this from or exactly what the theme is named. Thanks!

  30. Spot on with this write-up, I honestly think this amazing site needs a lot more attention. I’ll probably be returning to read through more, thanks for the information!

  31. I must thank you for the efforts you have put in penning this website. I really hope to view the same high-grade blog posts from you later on as well. In truth, your creative writing abilities has inspired me to get my own website now 😉

  32. I’m amazed, I have to admit. Rarely do I come across a blog that’s both educative and interesting, and let me tell you, you’ve hit the nail on the head. The problem is something which too few folks are speaking intelligently about. I am very happy that I came across this in my hunt for something concerning this.

  33. When I initially commented I appear to have clicked on the -Notify me when new comments are added- checkbox and now every time a comment is added I get 4 emails with the same comment. Is there a way you can remove me from that service? Thank you!

  34. Hello there, I do believe your web site could possibly be having web browser compatibility problems. Whenever I look at your site in Safari, it looks fine however, if opening in I.E., it has some overlapping issues. I simply wanted to give you a quick heads up! Aside from that, fantastic website!

  35. I’m impressed, I have to admit. Seldom do I encounter a blog that’s both educative and interesting, and let me tell you, you have hit the nail on the head. The problem is something which too few folks are speaking intelligently about. I’m very happy that I stumbled across this in my search for something relating to this.

  36. Achieving your fitness goal doesn’t need a certified personal trainer or an expensive gym memberships, it is not hard to exercise at home. It is easy to go down a training and fitness rabbit hole, however, when you are looking for the best home exercise equipment to outfit your personal home gym.

  37. Next time I read a blog, I hope that it does not fail me as much as this one. I mean, I know it was my choice to read, but I actually thought you would have something interesting to talk about. All I hear is a bunch of crying about something you could fix if you weren’t too busy looking for attention.

  38. Hello there! This article could not be written much better! Reading through this article reminds me of my previous roommate! He continually kept talking about this. I most certainly will send this post to him. Fairly certain he’s going to have a very good read. I appreciate you for sharing!

  39. Next time I read a blog, Hopefully it won’t fail me as much as this particular one. After all, Yes, it was my choice to read, however I actually thought you would probably have something interesting to say. All I hear is a bunch of crying about something that you could fix if you were not too busy searching for attention.

  40. Hi, I do believe this is a great web site. I stumbledupon it 😉 I am going to return yet again since I book-marked it. Money and freedom is the greatest way to change, may you be rich and continue to guide other people.

  41. An outstanding share! I’ve just forwarded this onto a co-worker who had been conducting a little homework on this. And he in fact bought me breakfast due to the fact that I found it for him… lol. So let me reword this…. Thanks for the meal!! But yeah, thanx for spending the time to discuss this matter here on your web page.

Leave a Reply

Your email address will not be published. Required fields are marked *