From Add Me and Pixel Screenshots to Call Notes, the Google Pixel 9 series has a bunch of AI upgrades, and none of them would be possible without the new Tensor G4. Tom’s Guide sits down with Jesse Seed, group product manager for Google Silicon, and Zach Gleicher, product manager for Google DeepMind, about what the Tensor G4 chip can do and how it stands out from Apple’s A series and Qualcomm’s Snapdragon.
Category
🤖
TechTranscript
00:00The Google Pixel 9 is here, and a lot of people
00:02are going to be talking about new features like the design
00:04as well as the display, battery life.
00:06But what really makes this phone stand out
00:08is the Tensor G4 chip and all the AI experiences
00:11that it enables on this new wave of devices.
00:14And to help us walk us through some of those scenarios
00:16and how it can actually help your life get
00:18a little bit better is Jesse Seed,
00:20who is on the team that is behind the Google Silicon,
00:24as well as Zach Gleicher, who is at DeepMind.
00:26And they have a deep collaboration
00:28with the Tensor G4 team.
00:29So Jesse, what do you think makes the Tensor G4
00:32chip stand out in a sea of smartphones?
00:35I think the biggest innovation that we made this year
00:37was being the first Silicon, the first phone
00:39to run Gemini Nano with multimodality.
00:42And that unlocks some very cool use cases, one of which
00:45is Pixel screenshots.
00:47So that's very handy if you're trying to remember things.
00:52I'm sure you got a chance to play with that.
00:54And another feature, not related to the Gemini Nano model,
00:57but I really love is also the Add Me feature.
00:59And so those of us that are photographers of our family
01:02or our crew definitely appreciate
01:04being able to go back in and dynamically
01:08add the photographer in.
01:09And that's something that we worked a lot
01:12on to tune over 15 different machine learning models
01:15and also using the Google Augmented Reality SDK.
01:19So yeah, I think those are my top two favorite Tensor-enabled
01:23Pixel experiences this year.
01:24So how do you get something like Gemini Nano
01:27to fit on something that's as compact as a phone?
01:30At DeepMind, we collaborate with a whole bunch
01:34of teams across Google.
01:35And we want to make sure that we're
01:37building Gemini models that meet the needs of all Google
01:40products.
01:41So as we were developing Gemini in collaboration
01:46with Android and Pixel, of course,
01:49we realized that there was this need for on-device models.
01:53So we saw this as a challenge because on the server,
02:01everyone was pushing for more capable models that
02:04were potentially bigger.
02:05And we, on the other hand, had all these interesting
02:08constraints that weren't present before on memory constraints,
02:11power consumption constraints.
02:14So in partnership with the Tensor team and Pixel,
02:18we were able to come together and understand
02:20what are the core use cases for these on-device models, what
02:24are the constraints for these on-device models,
02:27and actually co-develop a model together,
02:31which was a really exciting experience,
02:34and made it possible to build something
02:36that was so capable and able to power these use cases.
02:39For someone who hasn't upgraded their phone in,
02:41let's say, like three or four years,
02:42what do you think is going to stand out for them when
02:45it comes to the Tensor G4 chip?
02:46Improving what we call fundamentals,
02:48like power and performance, are very important for us.
02:52So the Tensor G4, which is our fourth generation chip,
02:54is our most efficient and our most performant.
02:58And so we believe that users will
03:00see that in everyday experiences,
03:02like web performance or web browsing,
03:05as well as app launch, and just overall snappiness
03:07of the user interface.
03:09So definitely, I think they'll be able to experience that
03:12in hand.
03:13And what about gaming performance?
03:14Because we know that's really important these days
03:16for people who are buying a new phone.
03:18In our testing, we actually have seen
03:20improved both peak and sustained performance
03:22in gaming and common games that run on the platform.
03:25So yeah.
03:25So I feel like we're almost past the phase where people are
03:28no longer afraid of AI, and they're
03:30more interested in terms of how it's going to help them.
03:32So what are some of the features within Gemini Nano
03:34coming to the phone that you're most excited about?
03:37Some of the main motivations that we
03:39see for the Tensor team and Pixel team
03:42coming to us for on-device use cases is,
03:46one, is better reliability.
03:49So the fact that you don't have to rely
03:51on an internet connection, the experience
03:54can be reliable and work no matter where you are.
03:58Another is, as we think about, potentially, privacy.
04:01If developers don't want the data
04:04to actually leave the device and be fully processed on-device,
04:08that's possible with having an on-device LLM.
04:11I think that some of the features that I'm excited
04:14about is, I think, the Pixel screenshots
04:17is a really great one.
04:18I think that really showcases how
04:21we are able to get these multimodal features that
04:25are working on the device.
04:27As you can see in the demos, it was really snappy,
04:29low latency, going to be reliable in how it works.
04:33But it's also a super capable model.
04:36And then all this information is data
04:38stored locally on your device, can be processed locally.
04:41So we're really excited that it can
04:43enable experiences like that.
04:45I think we're seeing traction for summarization use
04:49cases, and smart reply, and some of these common themes that
04:52are happening.
04:53And those are some of the use cases
04:55that we're really trying to make sure that the model works
04:58especially well for.
05:00So I think the models are continuing
05:02to get better, more capable.
05:04And we're going to just see the possibilities
05:06of what's possible on-device continuing to expand.
05:09So now that the G4 chip is in all of these phones,
05:11how do you balance the higher performance
05:13versus thermals and battery life?
05:15So something like thermal performance, and indeed
05:18even battery life, they're full system design challenges.
05:21It's not just about any one component, like only the chip
05:24or only something else.
05:26It's about the entire system.
05:27So what we're so lucky to have is
05:29the control of the full stack, everything
05:31from the silicon all the way up to the higher level user
05:34application and everything in between.
05:36So that means that we can tweak and refine year over year.
05:39And so, yes, as you mentioned, the addition
05:41of the vapor chamber, that's one concrete thing
05:43that we did in the Pro line this year
05:45to really give a little bit of extra headroom
05:47in those high sustained use cases
05:50where you're burning more power.
05:52But yeah, that's the way we think about it.
05:54It's really like the full system design
05:56and how do we improve that year over year.
05:58I think to a certain degree, sometimes users
06:00get intimidated with AI on phones,
06:02especially since we're still at the early stages.
06:04So how do you make sure with the Pixel 9
06:06in particular that people are excited
06:08and that they actually find these features to begin with?
06:11So I'm sure you've used a Pixel phone
06:13through all this process.
06:14There's this very cool thing called Pixel Tips,
06:16which I love to use when I get my new Pixel
06:19and will actually guide you through some of the new
06:22like applications or new use cases
06:24or new ways that a particular app will work.
06:27So I think that's one way that we can help
06:29communicate to users what's the new cool stuff
06:32to play with on your new Pixel phone.
06:34I think we saw with Microsoft and Recall,
06:36which they had a Recall themselves,
06:38that people are a little bit nervous
06:40about like their phones knowing everything about them.
06:42But I think screenshots is a little bit different
06:44if you guys can go into that,
06:45because I know it's like more manual.
06:47You're deciding what you want your phone to capture,
06:49but at the same time, it can still not know a lot about you.
06:52So how do you make sure that that information
06:54stays private and only on the phone?
06:56So, I mean, one of the ways we do it
06:58is indeed by having a capable on-device model, right?
07:02So that means that the analysis
07:04that's being done on that screenshot,
07:05none of it leaves the device.
07:07So that's one way that we're able
07:08to address that privacy concern.
07:10I think the other thing is just making,
07:11is like empowering users to decide what they want to do,
07:15like how they want to use something like Gemini, right?
07:18And what use cases they feel comfortable
07:21interacting with and what they don't.
07:22So I think it really comes down to user choice.
07:24But in the case of Pixel screenshots in particular,
07:28that is a fully on-device use case, so yeah.
07:32So I don't think third-party benchmarks are going away
07:34because we're going to use them to test these phones
07:37and the Tensor G4 chip.
07:38But at the same time,
07:39I think we have to start thinking about performance
07:41a little bit differently now that the AI era is here.
07:44So from your perspective,
07:45how should we be thinking about performance now
07:47when it comes to this chip?
07:49That's a great question.
07:50I think it really all comes down
07:51to real-world use cases, right?
07:53Like how does this thing actually perform in hand
07:55with the way you're actually going to run it?
07:57So I do think that things like
07:59how fast the web browsing response is,
08:02how fast apps are launching,
08:04the quickness and the responsiveness of the user interface,
08:07those are all sort of for everyday use cases.
08:09Those are good standard things to look at, right?
08:11And then also things like
08:13how fast can you capture a picture?
08:15These are all reasonable things
08:16that people really do through the course of the day.
08:18And I think those are much more representative,
08:20just a few to mention,
08:22those are much more representative
08:23than the sort of oftentimes semi-synthetic benchmarks
08:28that we see out there in the industry.
08:31Yeah, so when it comes to the Gemini Nano model
08:34on Pixel phones,
08:35from your perspective,
08:36when does that phone pass the test in terms of performance?
08:39As we think about benchmarks for LLMs and Gemini
08:44and especially as we think about Gemini Nano,
08:47we've seen like an industry,
08:49a large focus on academic benchmarks.
08:53And academic benchmarks like MMLU
08:56are great as it gives a common metric,
09:00but it could be gamified
09:02and people can optimize for them
09:04and it might not capture what you really care about.
09:08So for example, MMLU is a popular academic benchmark
09:13that is gonna ask like not some of the,
09:15it's a very diverse set of questions
09:17that it asks the LLM to answer,
09:20but maybe some of the questions that it asks
09:22is just information about history, history questions.
09:27For an on-device model,
09:29we don't really care that it knows
09:31and can answer history questions.
09:32We think like that's probably a better use case
09:34for a server-side model
09:36when we care about like use cases like summarization,
09:39where it's not important whether you know
09:42when Rome fell or something like that.
09:45So what we try to do is we work closely
09:49with the partner teams
09:51that are building these on-device experiences
09:53and we really try to gather benchmarks that they care about
09:57and the use cases that they care about
09:59so that we can evaluate against those.
10:02And that's how we think about quality.
10:03But again, this is where what also becomes super important
10:07as we think about Gemini Nano
10:08versus our server-side flash models and Pro
10:11is we also have to think about constraints
10:14like battery consumption and we have to make sure,
10:17so we work that like the model performs well
10:20and doesn't consume too much battery
10:22and that also the latency is good.
10:25So we actually partner with the Tensor team
10:28to profile our models
10:30as we're co-designing these models together
10:33to make sure that we are getting an architecture
10:35that works well and meets their efficiency,
10:38power consumption constraints
10:39and then we collect data for the use cases
10:42that they care about
10:42and make sure that we can hill climb on those use cases
10:45and make the model as good as possible.
10:47Yes, MMLU and other metrics like that are great for us
10:51just to make sure that we have automated metrics
10:54that we could hill climb against
10:55because creating good evals
10:57is often a very difficult task
11:00but we do a lot of that co-development together.
11:02That's great.
11:03And I would also just add,
11:04I think Zach's really on to something here,
11:05it's also something to be said for,
11:07it's not just about traditional
11:08maybe metrics of performance but also quality.
11:10So if you look at things like the quality of responses
11:12coming out of the model
11:13or even things like quality of the photo, right?
11:16Those are more like,
11:17that's what real world users in hand
11:19are gonna care more about than some number
11:22on the side of a box.
11:23Yeah, it's more subjective.
11:25That's true.
11:26And as we think about quality,
11:29sometimes we have human raters who are evaluating quality
11:33but what's I think really exciting
11:35about the development of Gemini
11:38is we could actually use larger Gemini models
11:40to what we call like auto raters
11:42to evaluate the quality of the model.
11:45AI rating AI.
11:46Yeah, self-rating.
11:47That can be a very powerful way
11:50for us to iterate more quickly
11:52and make sure that we are getting the model to perform well.
11:55Of course, these have their mistakes and issues as well
11:58and that's why doing actual sanity checks
12:00with human raters can be helpful too.
12:02All right, so I just wanted to say thank you to you both
12:04in terms of taking the time out
12:05to talk about this new chip
12:07and what's happening with DeepMind behind the scenes
12:09and how this is all coming to life.
12:11We're gonna test out these phones to see how good they are
12:13but now we know a little bit more
12:14about how much smarter these things are getting.