AI is a common buzz word these days, but most consumers probably aren’t aware how it’s interwoven in their everyday lives. Some of us in the analyst and tech press communities may also scoff at how often the term is used for some technologies that hardly resemble true artificial intelligence. That said, there are a few platforms, beyond just powerful data centers, that are a natural for AI processing and the NNs (Neural Networks) that drive them. One of those is AI inferencing (using the AI to infer information, versus training an NN) at the edge, and in your pocket, with a smartphone.
As you might imagine, smartphone platforms from Android to Apple vary greatly, but there are common workloads like speech-to-text translation, and recommender engines (like Google Assistant and Siri), that make heavy use of common AI NN models, and they do so on-device for speed and latency advantages.
Measuring AI Performance In Mobile Devices
With any new device introduction and with hot, new applications, tech-savvy consumers and tech press members want to be able to measure and glean relative performance of devices across the various mobile ecosystems. Further, AI processing performance in smartphone testing and reviews is trending at the moment, so of course there are multiple tools appearing in the major app stores that claim to be able to measure AI performance of phones and other mobile devices. And — you guessed it — these apps certainly aren’t all created equal.
In an effort to sort through this a bit, Marco and I took a deep dive look over at HotHardware at performance of various flagship Android phones across three popular AI benchmarks that produce vastly different results in some cases.
The key here is to better understand what a specific benchmark metric is actually testing. Does the test represent as close to real-world workloads as possible? An ideal benchmark uses actual applications that a consumer would use, but short of that it could employ the same core software components of popular apps instead, to represent realistic performance expectations. And in this case, that means we need to understand what NNs these benchmark tools are testing against, and what mathematical precision and AI algorithms are being used to process workloads on them.
What’s The Right Yardstick For AI Benchmarks?
LDS MOBILE
What makes for a good AI benchmark for mobile devices is a relatively deep, nuanced subject, but the long and short of it is virtually all mobile NPUs (Neural Processing Units, or dedicated AI engines) employ either INT8 or quantized mathematical precision, or FP16 floating point precision, to make use of popular NNs like ResNet-34 or Google’s DeepLab-v3 for image classification and segmentation in apps, for example. Is that a cat or a dog? What sort of color balance should be applied in this camera shot? These are the kinds of questions the AI is trying to infer answers for from the phone’s environment, in an imaging workload example at least, though there are many others.
Currently, INT8 precision is considered good enough for most consumer mobile applications, and advancements in compression techniques and advanced quantitization continue to elevate INT8 precision on mobile devices, while still reaping the benefits of lower power consumption versus FP16. FP16 offers better precision, but is more costly on critical smartphone power budgets.
As a result, the majority of AI enabled mobile applications out there employ INT8 for its power efficiency. Not all benchmarks currently available weight a mobile platform’s performance the same way, however. Some apps emphasize FP16 precision, even though it’s not practically employed nearly as often as INT8. Further, AI platform SDKs (software development kits) from Qualcomm and others are highly optimized for INT 8. So the question becomes, what do the various test results from some of these benchmarking apps really mean, in terms of real world AI performance in handsets and other mobile devices? As you can see in the in the scores HotHardware collected, some rank leading mobile silicon platforms from Qualcomm and Huawei quite differently. It’s readily apparent, however, that Qualcomm Snapdragon 865 devices seem to have a significant lead when it comes to INT8 NPU processing throughput, and also what is likely closer to current real-world AI performance in mobile apps. It’s also worth pointing out that Qualcomm’s Snapdragon Mobile Platforms also power the vast majority of Android phones in the US currently, so the company’s influence on the ecosystem is deep.
DAVE ALTAVILLA
An Analysts Take From The Very Early Innings Of Mobile AI
Artificial Intelligence and Machine learning at the edge is a fast-changing field that is delivering ever more capable and promising solutions that will enrich our everyday lives in many ways. As a result, benchmark metrics and the apps that are used to measure them are going to have to evolve and change with the times quickly as well. In addition, as with traditional benchmarks for PCs, press, tech enthusiasts, and savvy consumers are going to be looking more closely at AI benchmarks in the days ahead, as AI becomes an even more critical component of the mobile experience and available platform solutions on the market.
As such, it will be incumbent upon these benchmark app developers and the press to sort through the finer points of what makes for a quality mobile AI benchmark, and also what is a truer measure of performance for your own personal pocket AI assistant. Right now, if a benchmark isn’t employing commonly used NNs and realistically representing the importance of INT8 precision, you have to question how valuable that test is for the average consumer. There are no absolutes here, however. The current landscape is shaping up this way but again, AI technologies are moving at a frenzied pace and the rest of the industry will need to keep up.
Follow me on Twitter or LinkedIn. Check out my website or some of my other work here.
Original post: https://www.forbes.com/sites/davealtavilla/2020/06/04/measuring-ai-performance-on-mobile-devices-and-why-it-matters/
An impressive share! I’ve just forwarded this onto a co-worker who has been conducting a little
homework on this. And he in fact ordered me dinner because
I found it for him… lol. So allow me to reword this….
Thanks for the meal!! But yeah, thanks for spending the time to talk about this topic here on your site.
My partner and I stumbled over here coming from a different page and thought I might check things out.
I like what I see so now i’m following you. Look forward to looking into your web page repeatedly.
Hi there friends, pleasant post and nice urging commented here, I
am actually enjoying by these.
If some one desires to be updated with most recent technologies after
that he must be pay a visit this web page and be up to date everyday.
Hello very nice website!! Man .. Beautiful .. Amazing ..
I will bookmark your site and take the feeds also? I’m happy to
search out numerous helpful information right here
in the post, we want work out extra techniques on this regard, thanks for
sharing. . . . . .
I am the manager of JustCBD Store label (justcbdstore.com) and am aiming to expand my wholesale side of business. It would be great if someone at targetdomain can help me 🙂 I considered that the most suitable way to do this would be to connect to vape companies and cbd retailers. I was really hoping if anybody could suggest a trustworthy site where I can buy CBD Shops B2B Email Marketing List I am currently considering creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. Unsure which one would be the most ideal solution and would appreciate any guidance on this. Or would it be much simpler for me to scrape my own leads? Ideas?
I’m the proprietor of JustCBD brand (justcbdstore.com) and am aiming to broaden my wholesale side of company. It would be great if anybody at targetdomain is able to provide some guidance 🙂 I thought that the most suitable way to accomplish this would be to talk to vape shops and cbd retail stores. I was really hoping if anyone could suggest a qualified site where I can buy Global Hemp and CBD Shops Database with Contact Details I am presently reviewing creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. Unsure which one would be the best solution and would appreciate any advice on this. Or would it be easier for me to scrape my own leads? Suggestions?
We are a gaggle of volunteers and starting a brand new scheme in our community.
Your site offered us with useful information to work on. You’ve performed an impressive process and our whole community shall be grateful to
you.
I used to be suggested this website through my cousin. I’m not
positive whether or not this submit is written via him as no one else know such specified
approximately my problem. You are amazing! Thanks!
This is a very good tip particularly to those new to the blogosphere. Simple but very accurate information… Thank you for sharing this one. A must read article!
I absolutely love your website.. Great colors & theme. Did you develop this web site yourself? Please reply back as I’m hoping to create my very own site and would like to find out where you got this from or exactly what the theme is named. Thank you!
After looking into a few of the blog posts on your blog, I truly like your way of writing a blog. I saved as a favorite it to my bookmark site list and will be checking back soon. Please check out my web site too and let me know your opinion.
Great web site you have here.. It’s difficult to find high-quality writing like yours nowadays. I truly appreciate people like you! Take care!!
Hello , all website webmaster need SEO tool to check and find new keyword and analyes websites by using the best seo tool AHREFS now for just 10$ monthly you can use use this golden tool while it costs more than 150$ monthly , you can get trial as well >> https://cutt.ly/ahrefs
You should take part in a contest for one of the finest blogs on the internet. I most certainly will highly recommend this web site!
Right here is the right blog for everyone who hopes to find out about this topic. You realize so much its almost tough to argue with you (not that I really will need to…HaHa). You definitely put a brand new spin on a topic which has been discussed for decades. Wonderful stuff, just wonderful!
Your style is unique in comparison to other folks I have read stuff from. Thank you for posting when you have the opportunity, Guess I will just book mark this page.
Today, while I was at work, my cousin stole my iphone and tested to
see if it can survive a twenty five foot drop, just
so she can be a youtube sensation. My iPad is now broken and she has 83 views.
I know this is entirely off topic but I had to share it with someone!
It’s hard to come by experienced people on this topic, however, you seem like you know what you’re talking about! Thanks
You’ve made some really good points there. I checked on the internet to learn more about the issue and found most individuals will go
along with your views on this website.
After I originally commented I seem to have clicked the -Notify me when new comments are added- checkbox and now each time a comment is added I get four emails with the exact same comment. Is there a way you are able to remove me from that service? Kudos!
Aw, this was an extremely nice post. Taking the time and actual effort to generate a really good article… but what can I say… I hesitate a whole lot and don’t manage to get anything done.
You are so cool! I do not believe I’ve read anything like this before. So wonderful to find someone with some unique thoughts on this topic. Really.. thank you for starting this up. This site is one thing that is needed on the web, someone with some originality!
Good post! We will be linking to this great post on our website. Keep up the great writing.
Good post. I learn something new and challenging on websites I stumbleupon on a daily basis. It’s always interesting to read through articles from other authors and practice a little something from other websites.
Excellent goods from you, man. I have understand your stuff prior to and you’re just too excellent.
I really like what you have acquired right here, really like what you’re
saying and the best way in which you assert it.
You’re making it enjoyable and you still care for to stay
it sensible. I can’t wait to learn far more from you.
That is really a terrific website. adreamoftrains hosting services
An impressive share! I have just forwarded this onto a friend who was conducting a little homework on this. And he in fact ordered me dinner because I discovered it for him… lol. So let me reword this…. Thank YOU for the meal!! But yeah, thanks for spending some time to discuss this issue here on your blog.
Hi there! This post could not be written much better! Reading through this post reminds me of my previous roommate! He constantly kept preaching about this. I am going to forward this information to him. Pretty sure he’s going to have a very good read. I appreciate you for sharing!
I blog frequently and I really thank you for your content. Your article has really peaked my interest. I’m going to book mark your blog and keep checking for new details about once per week. I subscribed to your RSS feed as well.
Everything is very open with a really clear description of the issues. It was really informative. Your website is useful. Thank you for sharing!
Excellent article! We are linking to this great article on our site. Keep up the good writing.
Excellent post. I’m facing some of these issues as well..
I wanted to thank you for this wonderful read!! I certainly enjoyed every bit of it. I have got you book marked to look at new things you post…
Having read this I thought it was really informative. I appreciate you taking the time and energy to put this information together. I once again find myself personally spending a lot of time both reading and leaving comments. But so what, it was still worthwhile!
Having read this I believed it was very informative. I appreciate you spending some time and effort to put this informative article together. I once again find myself personally spending a significant amount of time both reading and commenting. But so what, it was still worthwhile!
Everything is very open with a clear description of the challenges. It was truly informative. Your site is useful. Thank you for sharing!
This is really attention-grabbing, You’re an excessively skilled blogger.
I have joined your rss feed and stay up for seeking more of your fantastic
post. Also, I’ve shared your website in my social networks
Everything is very open with a very clear explanation of the challenges. It was definitely informative. Your site is extremely helpful. Thanks for sharing!
This is a really good tip particularly to those fresh to the blogosphere. Simple but very accurate info… Thanks for sharing this one. A must read post!
I was excited to discover this great site. I need to to thank you for ones time for this particularly wonderful read!! I definitely enjoyed every little bit of it and i also have you saved to fav to see new stuff on your web site.
I was able to find good information from your blog posts.
I really love your blog.. Very nice colors & theme. Did you build this web site yourself? Please reply back as I’m looking to create my very own site and want to know where you got this from or exactly what the theme is named. Thank you!
Good post. I absolutely love this website. Keep it up!
Hey I know this is off topic but I was wondering if you knew of any widgets I
could add to my blog that automatically tweet my
newest twitter updates. I’ve been looking for a plug-in like this for quite some time
and was hoping maybe you would have some experience with something
like this. Please let me know if you run into anything.
I truly enjoy reading your blog and I look forward to your new updates.
This site was… how do you say it? Relevant!! Finally I’ve found something which helped me. Kudos!
An interesting discussion is worth comment. I do believe that you need to write more on this issue, it might not be a taboo matter but generally people don’t discuss such subjects. To the next! Many thanks!!
Pretty! This was an extremely wonderful article. Many thanks for providing these details.
I could not refrain from commenting. Exceptionally well written!
An interesting discussion is definitely worth comment. I think that you ought to publish more about this subject matter, it may not be a taboo matter but typically folks don’t speak about such issues. To the next! Many thanks!!
After looking into a handful of the blog articles on your blog, I really like your technique of blogging. I saved as a favorite it to my bookmark webpage list and will be checking back in the near future. Take a look at my website too and tell me your opinion.
Spot on with this write-up, I really think this website needs a lot more attention. I’ll probably be back again to read more, thanks for the info!
I want to to thank you for this great read!! I certainly enjoyed every bit of it. I have got you book marked to look at new stuff you post…
Excellent write-up. I certainly love this site. Keep writing!
Very nice write-up. I absolutely appreciate this site. Thanks!
Hi, There’s no doubt that your site could be having browser compatibility problems. Whenever I take a look at your web site in Safari, it looks fine however when opening in Internet Explorer, it has some overlapping issues. I just wanted to provide you with a quick heads up! Besides that, wonderful site!
Good information. Lucky me I discovered your site by accident (stumbleupon). I’ve saved it for later!
Your style is really unique in comparison to other people I have read stuff from. Thank you for posting when you have the opportunity, Guess I will just bookmark this blog.
Hello! I just want to give you a big thumbs up for the great info you’ve got right here on this post. I am returning to your blog for more soon.
Hi there! I could have sworn I’ve been to this blog before but after looking at many of the posts I realized it’s new to me. Regardless, I’m definitely pleased I came across it and I’ll be book-marking it and checking back often!
This website was… how do you say it? Relevant!! Finally I’ve found something that helped me. Thanks a lot!
The very next time I read a blog, Hopefully it does not fail me just as much as this one. After all, I know it was my choice to read through, nonetheless I truly believed you would have something helpful to say. All I hear is a bunch of complaining about something you can fix if you weren’t too busy searching for attention.
Great article. I will be going through a few of these issues as well..
Everything is very open with a very clear description of the issues. It was really informative. Your site is very helpful. Thanks for sharing!
An intriguing discussion is worth comment. There’s no doubt that that you should write more on this subject matter, it might not be a taboo subject but generally people do not speak about such topics. To the next! Cheers!!
Pretty! This has been an incredibly wonderful article. Thanks for supplying this info.
After checking out a number of the blog articles on your web site, I honestly like your way of writing a blog. I bookmarked it to my bookmark website list and will be checking back soon. Please check out my website as well and let me know how you feel.
That is a good tip particularly to those new to the blogosphere. Short but very accurate info… Thanks for sharing this one. A must read post!
Aw, this was a very good post. Taking a few minutes and actual effort to make a really good article… but what can I say… I procrastinate a lot and don’t seem to get nearly anything done.
I seriously love your site.. Pleasant colors & theme. Did you make this website yourself? Please reply back as I’m planning to create my own website and would love to learn where you got this from or just what the theme is named. Thank you!
Next time I read a blog, I hope that it won’t fail me just as much as this one. After all, I know it was my choice to read through, but I really thought you’d have something helpful to say. All I hear is a bunch of crying about something that you could possibly fix if you were not too busy looking for attention.
Way cool! Some very valid points! I appreciate you writing this write-up and the rest of the site is extremely good.
Hi! I simply wish to give you a big thumbs up for the excellent info you have right here on this post. I will be coming back to your blog for more soon.
Great article! We will be linking to this great content on our site. Keep up the great writing.
Asking questions are in fact nice thing if you are not understanding something completely, except this article presents nice understanding yet.
Achieving your fitness goal doesn’t have to require a certified personal trainer or an expensive gym memberships, it is not hard to exercise at home. It’s easy to go down a training and fitness rabbit hole, however, when you’re looking for the best home exercise equipment to outfit your personal home gym.
Hi there to all, as I am actually keen of reading this webpage’s post to
be updated on a regular basis. It contains fastidious stuff.
cheap flights y2yxvvfw
Oh my goodness! Amazing article dude! Thank you, However I am going through issues with your RSS. I don’t understand the reason why I cannot subscribe to it. Is there anybody having similar RSS issues? Anyone who knows the solution can you kindly respond? Thanks!!
Hi, I do think this is a great website. I stumbledupon it 😉 I may return yet again since i have bookmarked it. Money and freedom is the greatest way to change, may you be rich and continue to guide others.
Hi, I do believe this is an excellent blog. I stumbledupon it 😉 I will come back yet again since I saved as a favorite it. Money and freedom is the greatest way to change, may you be rich and continue to guide others.
This is a really good tip especially to those new to the blogosphere. Brief but very accurate info… Thanks for sharing this one. A must read post!
Hello there, I think your website may be having browser compatibility issues. When I look at your blog in Safari, it looks fine however, when opening in Internet Explorer, it has some overlapping issues. I simply wanted to provide you with a quick heads up! Other than that, great site!
Good post. I’m dealing with a few of these issues as well..
Your style is really unique compared to other people I’ve read stuff from. Thank you for posting when you’ve got the opportunity, Guess I will just book mark this site.
We stumbled over here coming from a different web page and thought I
may as well check things out. I like what I see
so now i am following you. Look forward to looking over your
web page yet again.
I would like to thank you for the efforts you have put in writing this website. I am hoping to check out the same high-grade content by you later on as well. In fact, your creative writing abilities has inspired me to get my own site now 😉
What’s up to every body, it’s my first pay a
quick visit of this blog; this web site includes amazing
and actually good data designed for readers.
I’m very happy to find this site. I want to to thank you for your time for this particularly fantastic read!! I definitely savored every bit of it and i also have you book-marked to see new information on your website.
I used to be recommended this blog by way of my cousin. I’m now not sure whether
or not this publish is written by means of him as no one else
understand such precise approximately my trouble. You’re incredible!
Thank you!
I absolutely love your website.. Great colors & theme. Did
you develop this web site yourself? Please reply back as I’m looking to create my
own personal website and would love to find out where you got this from or
just what the theme is named. Kudos!
When I originally left a comment I seem to have clicked the -Notify me when new comments are added- checkbox and from now on every time a comment is added I get four emails with the same comment. There has to be a way you can remove me from that service? Thanks a lot!
This is a great tip especially to those fresh to the blogosphere. Brief but very precise information… Thank you for sharing this one. A must read article!
I love it when individuals get together and share opinions. Great blog, keep it up!
This website really has all the information I wanted about this subject and didn’t know who to ask.
This is a topic which is close to my heart… Take care! Exactly where are your contact details though?
This is a topic which is near to my heart… Best wishes! Where are your contact details though?
You’re so cool! I do not think I’ve truly read something like that before. So wonderful to discover somebody with a few genuine thoughts on this issue. Seriously.. thanks for starting this up. This web site is one thing that is required on the web, someone with a bit of originality!
After looking into a number of the blog posts on your blog, I truly like your way of writing a blog. I bookmarked it to my bookmark webpage list and will be checking back in the near future. Please check out my web site as well and tell me your opinion.
It’s nearly impossible to find well-informed people for this subject, however, you seem like you know what you’re talking about! Thanks
Good blog you’ve got here.. It’s hard to find high-quality writing like yours these days. I really appreciate individuals like you! Take care!!
Having read this I thought it was rather enlightening. I appreciate you finding the time and effort to put this informative article together. I once again find myself spending a significant amount of time both reading and posting comments. But so what, it was still worthwhile!
The very next time I read a blog, I hope that it doesn’t fail me just as much as this one. I mean, I know it was my choice to read through, but I really believed you’d have something interesting to talk about. All I hear is a bunch of crying about something you could possibly fix if you weren’t too busy looking for attention.
Having read this I believed it was rather informative. I appreciate you finding the time and effort to put this content together. I once again find myself personally spending way too much time both reading and posting comments. But so what, it was still worthwhile!
Hi, I do think this is an excellent site. I stumbledupon it 😉 I am going to return yet again since i have saved as a favorite it. Money and freedom is the best way to change, may you be rich and continue to guide other people.
Muchas gracias. ?Como puedo iniciar sesion?