Broadcasters and movie studios alike are starting to explore the huge potential of modern technologies to bring a new generation of filmed entertainment to our TV sets and cinemas. Artificial intelligence, machine learning, and deep learning are the buzzwords that excite video executives with promises of revolutionary new abilities for video creation and editing.
Deep learning, in particular, is the new frontier for the video industry, allowing video professional to do things automatically that would have taken weeks of work in the past, as well as some things that wouldn’t have been possible at all. How is deep learning different from other machine learning algorithms? And what are its practical applications for broadcasting and filmed entertainment? What are the science and its business ramifications?
Artificial Intelligence, Machine Learning, and Deep Learning
Artificial intelligence is any attempt to make a computer appear as though it has intelligence. The computer may be told exactly what to do in any given situation, in which case it hasn’t learned anything. Machine learning seeks to allow the computer to learn how to perform certain tasks. There are a variety of methods to do this, and nearly all of them rely on the computer altering parameters repeatedly through a trial and error process. One of the more complex ways of doing this is by mimicking the neurons in a biological brain. When we make these artificial brains, or neural networks, more complex, we have deep learning.
Deep learning allows a computer to take something complex as input, such as all the pixels in a frame of video, and output something equally complex, such as all the pixels in a new, altered, frame of video. For example, it may be shown frames with unwanted grain as input, and have its output compared to clean frames. By trial and error, it learns how to remove the grain from the input. As more and more images are passed through it, it can learn how to do the same thing for images that it was never shown.
Perhaps the first impressive use of deep learning was when Google trained a neural network to play Go, the famously difficult and complex board game. The game is far too complex for human instructions to create a viable opponent, and a single layer neural network would have never been enough. Deep learning made it possible.
Deep learning is used for a wide variety of other tasks as well. It is used to match generated speech with human speech, so text-to-speech programs sound more natural. In a similar task, it is used by translation companies to teach computers how to translate from one language to another. The self-driving cars that several companies are working on are driven by deep learning. Marketing departments use it to learn the habits of customers and guess how a given customer will behave and what strategies they will best respond to. Digital assistants use it to better understand the requests that we make of them.
Deep learning for TV and Filmed Entertainment
There are many opportunities to apply deep learning techniques in the field of video production, editing, and cataloging. But the technology is not limited to automating repetitive tasks; it can also enhance the creative process, improve video delivery and help preserve the massive video archives that many studios keep.
Video Generation and Editing
Warner Bros. recently had to spend $25M on reshoots for ‘Justice League’ and part of that money went to digitally removing a mustache that star Henry Cavill had grown and could not shave due to an overlapping commitment. It isn’t just ‘Justice League’ – the post-production stage of any movie is time-consuming and expensive. Deep learning will be a game changer for these are types of tasks.
Consumer-grade, easy to use solutions such as Flo allow you to use deep learning to automatically create a video by describing what you want in it. The software will find the relevant videos from your library and edit them together automatically.
Google has a neural network that can automatically separate the foreground and background of a video. What used to require a green screen can now be done with no special equipment.
Deepfakes have hit the news quite a lot recently – when the face of one person is put onto a video of another, likewise, deep portraits which apply motion to still pictures like the Mona Lisa. The potential uses of this technology in special effects are vast.
For example, the mustache problem over at Warner Bros which drew Henry Cavill into a controversy with fans. Cavill needed to grow a mustache for Mission: Impossible – Fallout, and at the same time needed to reshoots for Justice League. Cavill, had a mustache for Fallout, but needed to be clean- shaven for Superman. He opted to keep the mustache, so the Justice League’s editing team had to digitally remove the hairy lip for every scene he’d reshot.
Sadly, this was noticed by fans and it caused a fuss. If hobbyists working at home can put Nicholas Cage into movies that he was never in using deep learning tools, one can only guess how much time and money Warner Bros. could have saved replacing Henry Cavill with older footage of himself.
Video Restoration
According to the UCLA Film & Television Archive, nearly half of all films produced prior to 1950 have disappeared. Worse, 90% of the classic film prints that do exist are in poor condition. The process of restoring these films is long, tedious, and expensive. This is an area in which deep learning is going to make a major difference.
The process of colorizing black and white footage has always been lengthy. There are thousands of frames of footage in a movie and coloring each one takes a long time. Even with advanced tools, the process can only be automated so much. Thanks to Nvidia, deep learning can now speed up the process significantly, with tools that only require an artist to color one frame of a scene. From there, the deep learning network automatically handles the rest.
A previously show-stopping problem was missing or damaged frames from a video. You can’t do reshoots on something that happened years ago.
Restoring that type of movie before meant editing around the missing frames. Now, deep learning networks from Google aim to change that. They have developed a technology that can realistically recreate part of a scene based on start and end frames.
Face/Object Recognition
By detecting the faces of everyone in a video, deep learning can allow you to quickly classify a video collection. You could, for example, search for any clip or movie that has a given performer. Alternatively, you could use the technology to count the exact screen time for every actor in a video. Sky News recently used facial recognition to identify famous faces at the royal wedding.
The technology is not limited to detecting just faces though, sports broadcasts rely on camera people to track the movements of the ball, or to identify other key elements to the game, such as the goal. Using object recognition, AI-powered tools can be used to automate the production of a sports broadcast.
Video Analysis
While Flo can identify what a scene is about and use that data to generate a video about whatever you want, that same technology can be used to sort and classify videos to make it easy to find a particular piece of footage by simply searching for people or actions that appear in it.
This could be used to detect and remove objectionable content from videos to ensure that they remain suitable for a target audience. In a similar vein, it could be used to match new videos up with old videos that a person has shown interest in and provide them with a personalized recommendation list.
Better Streaming
As we move into 4k streaming, and television manufacturers begin the rollout of 8k displays, streaming is using more data than ever before. Anyone with a poor connection knows what a problem this can be. The utility of a shiny 4k display is weakened if your internet connection can’t handle the bandwidth to fully take advantage of it. Thanks to neural networks that can recreate high definition frames from a low definition input, we could soon be streaming low definition streams over our internet connection, while still enjoying the high definition glory that our displays are capable of.
The Future
Deep Learning use in film and broadcast has only begun to nibble at the edges of what it will be used for in the future. I believe its future in the video industry is particularly bright. However, as with all new technologies, deep learning is not without a downside. As with deepfakes or face recognition misuse, there are valid concerns of privacy and trust that arise from the rapid evolution of this technology.
As with any new technology, the industry needs to address a range of issues. The video industry and tech experts must come together to develop the standards of how tomorrow’s new normal might look. However, with the right approach, the benefits of this addition to the toolbox will be bigger than is imaginable now, and, just as the advent of “talkies” and color film did before it, deep learning will take film and television to a whole new level.
Original post: https://www.techradar.com/uk/news/how-deep-learning-could-revolutionize-broadcasting
Good web site you’ve got here.. It’s difficult to find high-quality writing like yours these days.
I really appreciate individuals like you! Take care!!
I visited multiple sites however the audio quality for audio songs
current at this web site is genuinely excellent.
Wow that was odd. I just wrote an very long comment but
after I clicked submit my comment didn’t show up. Grrrr…
well I’m not writing all that over again. Regardless, just wanted to say superb blog!
Hello! I’ve been reading your blog for a long time now
and finally got the courage to go ahead and give you a
shout out from Huffman Tx! Just wanted to tell you keep
up the good work!
I am the proprietor of JustCBD label (justcbdstore.com) and am looking to develop my wholesale side of business. I really hope that someone at targetdomain is able to provide some guidance 🙂 I considered that the most suitable way to accomplish this would be to reach out to vape shops and cbd retailers. I was really hoping if anybody could suggest a reliable site where I can purchase CBD Shops Business Data I am currently checking out creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. Not sure which one would be the most suitable option and would appreciate any assistance on this. Or would it be simpler for me to scrape my own leads? Ideas?
I am the manager of JustCBD brand (justcbdstore.com) and I’m presently seeking to develop my wholesale side of business. It would be great if anybody at targetdomain can help me 🙂 I thought that the most effective way to accomplish this would be to talk to vape shops and cbd retail stores. I was hoping if anybody could recommend a reputable web site where I can purchase CBD Shops Business Mailing List I am already checking out creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. Unsure which one would be the most suitable selection and would appreciate any support on this. Or would it be easier for me to scrape my own leads? Ideas?
I believe that is among the such a lot significant information for me.
And i am satisfied studying your article. However wanna remark on some normal issues, The website style is ideal, the articles is actually excellent : D.
Excellent job, cheers
My coder is trying to convince me to move to .net from PHP.
I have always disliked the idea because of the costs. But he’s tryiong none the less.
I’ve been using WordPress on several websites for about a year and am
worried about switching to another platform. I have heard great things about blogengine.net.
Is there a way I can import all my wordpress content into it?
Any kind of help would be greatly appreciated!
I feel this is among the such a lot important info for me.
And i’m satisfied reading your article. But should observation on few general issues, The
site style is wonderful, the articles is in reality excellent :
D. Good task, cheers
I have been exploring for a little bit for any high quality articles or weblog
posts in this sort of house . Exploring in Yahoo I eventually stumbled upon this web site.
Studying this information So i am satisfied to show that I have a very
good uncanny feeling I discovered exactly what I needed.
I so much for sure will make certain to don?t forget this website
and provides it a glance on a constant basis.
Hi there! This article could not be written any better! Looking at this article reminds me of my previous roommate! He constantly kept preaching about this. I most certainly will send this article to him. Pretty sure he’ll have a good read. I appreciate you for sharing!
There’s certainly a lot to know about this subject. I love all of the points you’ve made.
Hi mates, how is the whole thing, and what you would like to say about
this piece of writing, in my view its actually remarkable
in support of me.
Spot on with this write-up, I actually believe this amazing site needs much more attention. I’ll probably be returning to read through more, thanks for the advice!
Hi there! This post couldn’t be written much better! Looking at this article reminds me of my previous roommate! He continually kept talking about this. I’ll send this post to him. Fairly certain he will have a great read. Many thanks for sharing!
I love looking through an article that can make people think. Also, many thanks for allowing me to comment!
Howdy! I could have sworn I’ve visited this website before but after going through many of the articles I realized it’s new to me. Anyways, I’m certainly delighted I found it and I’ll be bookmarking it and checking back often!
Great info. Lucky me I ran across your blog by chance (stumbleupon). I have saved it for later!
Way cool! Some very valid points! I appreciate you writing this post and the rest of the site is extremely good.
I every time spent my half an hour to read this blog’s articles every day along with a mug of coffee.
Pretty great post. I just stumbled upon your weblog and wished to say that I have
really loved browsing your blog posts. After all I’ll be subscribing on your rss feed and I’m hoping you write again very soon!
I am curious to find out what blog system you’re working with?
I’m having some small security problems with my latest blog
and I would like to find something more risk-free.
Do you have any suggestions?
Aw, this was a very nice post. Taking the time and actual effort to make a superb article… but what can I say… I put things off a lot and never seem to get anything done.
This site certainly has all of the information and facts I wanted concerning this subject and didn’t know who to ask.
I could not resist commenting. Exceptionally well written!
After going over a number of the articles on your site, I seriously appreciate your technique of writing a blog. I book marked it to my bookmark site list and will be checking back in the near future. Please check out my website too and tell me how you feel.
Excellent article. I certainly love this website. Keep writing!
This site was… how do I say it? Relevant!! Finally I’ve found something which helped me. Kudos!
This is the perfect webpage for anyone who would like to understand this topic. You know a whole lot its almost tough to argue with you (not that I personally would want to…HaHa). You certainly put a brand new spin on a topic that has been written about for years. Excellent stuff, just wonderful!
Achieving your fitness goals doesn’t need a certified personal trainer or an expensive gym memberships, especially when you have the budget and the space to consider practically every workout machine on the market.
I was very pleased to find this web site. I wanted to thank you for your time just for this wonderful read!! I definitely liked every little bit of it and I have you saved as a favorite to look at new stuff on your blog.
May I simply say what a comfort to find an individual who truly understands what they are discussing on the internet. You definitely realize how to bring a problem to light and make it important. A lot more people must read this and understand this side of the story. It’s surprising you’re not more popular since you certainly have the gift.
Great web site you have here.. It’s difficult to find high-quality writing like yours nowadays. I seriously appreciate people like you! Take care!!
I think that is among the such a lot significant info for me.
And i am satisfied reading your article. However wanna statement on few common things, The web site taste is great, the articles is truly
great : D. Excellent job, cheers y2yxvvfw cheap flights
You really make it seem so easy with your presentation but I find
this matter to be actually something that I think I would never understand.
It seems too complicated and very broad for me. I am looking forward for your next post,
I’ll try to get the hang of it!
Hi to all, because I am genuinely eager of reading this weblog’s post to be updated on a regular basis.
It includes pleasant material.
An intriguing discussion is worth comment. There’s no doubt that that you ought to publish more on this topic, it might not be a taboo matter but usually people do not speak about these issues. To the next! Kind regards!!
Very nice blog post. I absolutely appreciate this website. Continue the good work!
Oh my goodness! Amazing article dude! Thanks, However I am having difficulties with your RSS. I don’t understand the reason why I am unable to join it. Is there anybody else having the same RSS issues? Anybody who knows the answer will you kindly respond? Thanx!!
This is a topic that’s near to my heart… Thank you! Exactly where are your contact details though?
This site was… how do I say it? Relevant!! Finally I have found something which helped me. Appreciate it!
Hello there! This post could not be written any better! Looking at this post reminds me of my previous roommate! He continually kept talking about this. I’ll forward this article to him. Fairly certain he will have a very good read. I appreciate you for sharing!
Very good info. Lucky me I ran across your website by accident (stumbleupon). I’ve book-marked it for later!
You need to be a part of a contest for one of the finest websites online. I most certainly will highly recommend this website!
Greetings! Very helpful advice within this post! It’s the little changes which will make the most important changes. Thanks a lot for sharing!
Way cool! Some extremely valid points! I appreciate you penning this post and also the rest of the site is really good.
Great article. I’m going through a few of these issues as well..
Spot on with this write-up, I honestly think this web site needs a lot more attention. I’ll probably be back again to read through more, thanks for the information!
Spot on with this write-up, I actually believe that this website needs much more attention. I’ll probably be returning to read through more, thanks for the information!
Good post. I’m experiencing some of these issues as well..
An impressive share! I have just forwarded this onto a coworker who has been doing a little research on this. And he in fact ordered me dinner due to the fact that I discovered it for him… lol. So allow me to reword this…. Thanks for the meal!! But yeah, thanks for spending some time to discuss this subject here on your web page.
I was able to find good advice from your blog articles.
A motivating discussion is worth comment. I do think that you ought to write more about this subject matter, it might not be a taboo subject but usually people don’t speak about these subjects. To the next! All the best!!
An intriguing discussion is definitely worth comment. I do think that you should write more on this topic, it might not be a taboo matter but generally folks don’t discuss such topics. To the next! Best wishes!!
Hi! I just would like to give you a big thumbs up for your excellent information you’ve got right here on this post. I am returning to your blog for more soon.
Great post! We are linking to this particularly great post on our site. Keep up the great writing.
Thank you for your article.Really looking forward to read more. Keep writing.