Gemini 实时语音智能体如何工作:Thor Schaeff 演示 Live API 与多模态应用
原题Can Your Agent Hear You Now? Building Live Voice Agents with Gemini — Thor Schaeff
推荐理由想做可打断的语音或实时翻译应用,先理解怎样维持一段连续会话。这些现场案例帮助开发者辨认原生音频、工具调用与有状态接口的分工。
看前速览
Google DeepMind 的 Thor Schaeff 通过故事插画、实时翻译和 AI 广播 DJ 演示 Gemini 的实时语音与多模态 API,并解释其有状态交互和原生音频流机制。
有状态交互让实时对话在保留上下文的同时驱动其他媒体模型。
故事插画演示使用 Interactions API 和上一轮交互 ID 串联对话,并在过程中生成图像。
Live API 通过持续连接的 WebSocket 会话传输多种模态。
讲者介绍了如何在会话中发送文本、音频和视频帧,并让应用持续接收音频片段与文字记录。
Gemini Live 以原生音频处理语音,而非完全依靠串行转录管线。
Schaeff 将这一设计与流畅切换语言、打断和工具调用联系起来,并对比语音转文字、语言模型、文字转语音的级联方式。
演示结合了语音、翻译、图像与音乐生成,也留下生产环境问题。
广播 DJ 会调用音乐生成工具;演讲提到 LiveKit 和 Pipecat 的实时媒体集成,但未提供规模与可靠性的实测结果。
完整摘要
Thor Schaeff 连续演示了 Gemini 最近的音频与生成式媒体能力。故事画布让实时对话驱动图像生成,Interactions API 通过传递上一轮交互 ID 保留上下文;实时翻译和 AI 广播 DJ 则把 Gemini Live 与音乐生成工具结合起来。演讲的技术核心是 Live API:应用通过有状态的 WebSocket 会话持续发送文本、音频和视频帧,并接收音频片段及文字记录。Schaeff 将原生音频理解与「语音转文字、语言模型、文字转语音」的级联管线作对比,并展示打断、工具调用、多语言回复,以及 LiveKit 和 Pipecat 等集成方式。一手现场演示使这些能力更直观,但整体仍以产品功能导览为主,对部署取舍、实测延迟、故障模式和对比证据讨论较少。
章节
文字稿
简要说明Brief Description
你的智能体现在能听见你了吗?使用 Gemini 构建实时语音智能体——Thor SchaeffCan Your Agent Hear You Now? Building Live Voice Agents with Gemini — Thor Schaeff
目录Table of Contents
- 简介Intro
- 猜猜这个模型:Gemini TTSGuess the model: Gemini TTS
- 认识托尔Meet Thor
- 今天发布的内容What shipped today
- 演示:插画故事画布Demo: an illustrated story canvas
- Interactions APIThe Interactions API
- 演示:Gemini 实时翻译Demo: Gemini Live Translate
- Gemini 3.1 Flash Live
- 演示:使用 Lyria 3 的 AI 电台 DJDemo: an AI radio DJ with Lyria 3
- Live API 的工作原理How the Live API works
- 原生音频,而非流水线Native audio, not a pipeline
- Partners: LiveKit and Pipecat
- 在 AI Studio 中试用Try it in AI Studio
- 演示:语音、视觉与普通话Demo: voice, vision and Mandarin
- 总结Wrap-up
简介Intro
大家能听见我吗?太棒了。大家好。嗯,也欢迎几位刚走进来的朋友。很抱歉,日程一直在变。也很抱歉,你们没能找到想要的安静工作区。好。嗯,我们要来看看一些使用 Gemini API 的智能体。正式开始之前,先来听听这段。欢迎各位来到地球上最盛大的舞台。你们正站在史上规模最大的 AI 工程师世界博览会的中心。让我们一起创造未来,好吗?Can you hear me? Fantastic. Hello everyone. Well, hello couple people that wandered in. Um, I'm sorry the schedule has been changing around. I'm sorry you didn't find the quiet working area you were looking for. All right. Um, we are looking at some agents with Gemini API. Now, before we kick off, let's have a little listen to this. Welcome one and all to the grandest stage on Earth. You're standing right at the heart of the largest AI engineer World's Fair in history. Let's build the future together, shall we?
猜猜这个模型:Gemini TTSGuess the model: Gemini TTS
有人知道这是什么模型吗?对。你、你、你不能说。还有别人吗?有人知道是什么模型吗?不知道。猜猜看?它以 G 开头。Gemini。对,它是 Gemini 模型。但具体是哪一个?不,不是 Life。答错了。对,这是 Gemini TTS。你刚才想说的是这个吗?不是。好吧。嗯,也就是 Gemini 文本转语音。呃,它确实是一个非常出色、能生成富有表现力语音的模型。你可以通过内嵌的音频文本来引导它。嗯,你可以流式传输语音。现在它的延迟比以前低了很多。所以,呃,如果你正在寻找语音合成服务,Gemini TTS 是个不错的模型。现在我们来看看一些……Does anyone know what model that is? Yeah. You you you you're not allowed. Anyone else? Anyone know which model? No. Any guesses? It starts with G. Gemini. Yes, it is a Gemini model. But which one? No, it's not life. That's incorrect. Yes. This is Gemini TTS. Is that what you were about to say? No. Okay. Um, so Gemini text to speech. Uh, actually a really incredible model for expressive speech. You have inline audio text you can steer it with. Um, you can stream the speech. It's a lot lower latency now than it used to be. So uh if you are in the market for some speech uh text to speech that is Gemini TTS is a good model. Now let's have a look at uh some of the
认识托尔Meet Thor
我们今天要做的事情。呃,首先自我介绍一下,我叫 Thor,在 Google DeepMind 负责开发者体验。这主要包括 Gemini API 和 Google AI Studio。所以如果你通过 API 使用 Gemini 模型,呃,有什么反馈请告诉我。我同事 Philip 也在这里,他负责 API 上的托管代理。如果你在用这个,也请告诉我们。这算是我们明天要举办的研讨会的一次预览。嗯,明天我们会介绍 Gemini API,重点讲实时代理、对话代理、实时对话代理、多模态实时对话代理——我们把所有内容都塞进去了。我们实际可以看看什么呢?最近发布了很多东西。具体来说,我们想看看 Gemini 的多模态能力和生成媒体模型。呃,其实这张幻灯片现在已经过时了,因为我们今天早上发布了一个新模型,叫 Anyone。things we're doing today. Uh first of all my name is Thor. I work on the developer experience at Google DeepMind. So that mostly includes Gemini API and Google AI Studio. So if you're using the Gemini models through the API, uh you know, let me know if you have any feedback. I have my colleague Philip here as well, uh who's working on managed agents on the API. So if you're using that, uh let us know as well. So this is a bit of a a prelook at the workshop we're doing tomorrow. Um so tomorrow we'll do kind of Gemini API especially focused on um live agents uh conversational agents real time uh conversational agents multimodal real-time conversational agents we're just packing everything in there um and so what we can look at actually there's uh been a lot of things that shipped and so specifically what we want to look at is the multimodality of Gemini and the gen media models Uh so actually this slide is already out to date because this morning we shipped a new model uh that is called anyone
今天发布了什么What shipped today
知道我们发布了哪个模型吗?我们发布了 Omni Flash,这是一款视频生成和编辑模型;还发布了 Nano Banana Light。所以 Nano Banana Light 是一款非常令人兴奋的模型,因为它大幅降低了图像生成和图像编辑的延迟。我接下来会在这里做几个演示。其中一个演示就是这个例子,我把它称为 Gemini……know which model did we ship? We shipped Omni Flash which is uh a video generation and editing model and we shipped Nano Banana Light. So Nano Banana Light is a really exciting model because it cuts down the latency of image generation and image editing quite significantly. And so what I've done, we'll we'll do a couple demos here. Uh and one of the demos is um this example here, which I call the uh Gemini
演示:插图故事画布Demo: an illustrated story canvas
Nano Banana 故事画布。呃,我最近刚用氛围编程把这个做出来,不过我现在鼠标有点不听使唤。嗯,我们可以用 Gemini Live 编一个故事,然后让它用 Nano Banana Lite 为我们把故事画出来。大家知道,这有点像即兴剧场。有人想先来个故事情节吗?我们设定在什么地方?好吧,假设我们在 20 世纪 80 年代的台北。台北夜市的霓虹灯闪烁着近乎魔幻的光芒,倒映在湿漉漉的路面和驶过的摩托车镀铬车身上。雨刚停,一层潮湿的雾气弥漫在空气中。在熙熙攘攘的购物人群和摊贩之间,一个年轻女子把一件奇怪的小物件紧紧藏在外套底下——那物件正发出微光。20 世纪 80 年代的台北,发生了一件非常激动人心的事。Nanobanana story canvas. Uh and I just VIP code this up uh just recently and I'm having some trouble with my mouse here. Um but so what we can do is we can come up with a story with um Gemini Life and then have it use Nano Banana Light to to illustrate that story for us. So anyone you know this is a little bit like improv theater now. Anyone has a story line they want to start with? Where are we somewhere? Okay, let's say um we're in Taipei in the 1980s. The neon lights of Taipe's night market pulsed with an almost magical energy reflecting off the wet pavement and the chrome of passing scooters. Rain had just passed, leaving a humid mist that hung in the air. Amidst the throng of shoppers and vendors, a young woman clutched a small, strangely glowing object under her jacket. Something very exciting happened in 1980s Taipei.
一种新型饮品诞生了。她在一个摊位前停下,只见一名男子正使劲摇晃着一个金属杯,调制出一杯泡沫丰富、奶香浓郁的饮品。他把杯子递给一位顾客,对方睁大双眼,喝了一大口,细细品尝杯底奇特的木薯珍珠。这就是珍珠奶茶的诞生,它甜美又有嚼劲的魅力迅速风靡开来。所以你们在这里看到的是,呃,来自工具调用的内容。Gemini Life 发起了一次工具调用,然后使用 Gemini Interactions API 实时生成图像。现在,实际的图像生成仍然需要大概两到三秒,不过这速度已经相当惊人了。然后,大家可以看到,如果你看一下代码……A new kind of drink emerged. She paused at a stall where a man was vigorously shaking a metal cup, creating a frothy, milky concoction. He handed a cup to a customer who took a long sip with wide eyes, savoring the strange tapioca pearls at the bottom. This was the birth of bubble tea, and its sweet, chewy magic was spreading like wildfire. So what you can see here is uh from the tool call. So Gemini Life is getting a tool call um and then it's using the Gemini interactions API to generate an image on the fly. Now it still takes kind of you know two to three seconds for the actual image generation but that is um pretty impressively fast. And um what you can look at is if you're looking at the code
交互 APIThe Interactions API
这里我们使用的是 interactions API。interactions API 很令人期待,因为它是一个有状态的 API,这意味着我们可以非常顺畅地串联各次交互。你可以看到,我们进行第一次图像生成时没有提供之前的 interaction ID;而在接下来的生成中,我们把这个之前的 interaction ID 传给 interactions API。这样一来,我们就保留了之前交互的上下文,并能在生成过程中延续它。具体到这里的演示,你可以看到,第一次交互生成的图像有着非常鲜明的视觉风格,也包含了角色。我们可以继续这次交互,让这些生成结果之间保持一致性。here uh we're using the interactions API. And so the interactions API is exciting because it is a stateful API which means we can chain our interactions very seamlessly. So you could see here the first image generation that we were doing uh it didn't have a previous interaction ID but then the next generation that we were doing we're passing this previous interaction ID to the interactions API which means that we have the previous context of the previous interaction and we can continue that in the generation. So what that means specifically for our demo here is that you can see is our first you know interaction uh it generated an image with a very specific kind of visual feeling and also with characters right and so we can continue that interaction and so we can keep the same consistency kind of across these generations.
所以,如果你还没有试用过 Interactions API,呃,它是我们用于与所有 Google DeepMind 模型以及智能体交互的统一 API。呃,所以一定要试试。它真的是一种非常非常酷的方式,可以与 Google DeepMind 提供的一切进行交互。好,那么刚才说的是已经发生的一系列事情。呃,这些算是 Gen Media 方面的进展。我们今天早上发布了 Nano Banana Light。现在,我们最近还发布了……So if you haven't tried out the interactions API, uh it is our unified API for interacting with all of the Google DeepMind models as well as agents. Uh and so do try that out. It is a it is a really really cool um way to interact with everything that Google DeepMind has. Okay, so this was you know a bunch of things that happened. Um so this was kind of on the Gen Media side. We shipped uh Nano Banana Light this morning. Now, we also recently shipped
演示:Gemini 实时翻译Demo: Gemini Live Translate
音频这边是我们的实时翻译模型。有人试过 Gemini 3.5 实时翻译吗?试过。好,有几位。呃,太棒了。我们也可以试试看。可以播放一段小演示。接下来我要创建这个实时翻译会话,请用手机扫描这个二维码。现在如果我启用麦克风,我的语音就会通过麦克风传入这个会话。接着,你可以进去选择自己偏好的语言。现在我们看到有人选择了德语,所以有人正在用德语听我讲话。如果你现在正在听,可能需要戴上耳机。如果你有降噪耳机,现在就可以用自己偏好的语言收听这场演讲。on the audio side our live translation model. Has anyone tried out Gemini 3.5 live translate? Yes. Okay, a couple folks. Uh, that's fantastic. So, what we can do is uh we can try that out as well. We can play uh a little demo. And so what I'm going to do here is I'm going to create this live translation session and I want you to scan uh this QR code on your phone. And so what this is doing here is if I now enable um my microphone. So this is getting the the microphone of my speech um here into the session. And now if you go in and you select your preferred language. So we're seeing here we're getting some German. So someone is listening to me now speaking in German. And so if you are uh now listening to this, so you probably need some headphones. So if you have some noiseancelling headphones, you can now listen into this talk in your preferred language.
所以,Gemini 实时翻译确实是一个非常惊人的模型。它几乎可以实时翻译。我说话时,模型一旦识别出我正在谈论的上下文,就会开始实时翻译成你想要的语言。我看到有人在听。你们能听到音频吗?听得见吗?嗯,效果怎么样?太棒了。你看,这真的很令人兴奋,因为你知道,在这样的会议上,听众显然来自世界各地,所以我可以用自己偏好的语言发言。比如,我现在可以切换成德语。Gemini 实时翻译的功能很强大。嗯,它确实是一个非常惊人的模型。它会自动检测发言者所说的语言,然后实时将其翻译成我们偏好的语言。And so Gemini live translate is a really incredible model. It's near realtime translation. So as I'm speaking, as soon as the model is recognizing kind of the context of what I'm speaking about, uh it will then start translating in real time to your desired language. So I see someone is listening in there. Do do you get some audio out? Can you hear it? Yeah. Does it sound good? Fantastic. See, um, so this is this is really exciting because you know in a conference like this obviously we have a very very international audience and so I can speak in my preferred language. So I could switch to German now for example. the powerful Gemini live translates. Um, it is a a really incredible model. Um, it automatically detects the language the speaker is speaking and then translates it in real time uh into our preferred language.
好的,今天现场有很多国际听众。呃,有谁是用中文收听的?你们是用中文收听吗?哦,好的。我没听懂,不过没关系。嗯,这就是 Gemini 实时翻译。如果你想试试,现在就可以把它添加到你的应用中。它已可通过 Gemini API 使用,欢迎试试看。呃,这是一个非常惊艳的模型,也很想听听你们正在用它构建什么。现在,显然我们在谈智能体,对吧?你的智能体能——呃,这场演讲的标题是什么来着?这是 Gemini 想出来的标题。所以,呃,你的智能体现在能听见我说话了吗?我想标题就是这个。那么,我们最新的实时对话……Okay, we have uh a very international crowd here today. Uh, who's listening in in uh Chinese? Are you listening in Chinese? Oh, okay. I didn't understand that, but okay. Um, so that is Gemini live translate. Um, if you want to try it out, you can add it to your um applications now. It is available on the Gemini API. So, do try that out. uh it is a really incredible model and uh would love to hear kind of what you're what you're building with that. Now obviously we're looking at uh agents, right? Can your agents uh what's it the title of this talk? Gemini made up this title. So I uh can your agent hear me now? I think that that is the title. So our most recent real time conversational
Gemini 3.1 Flash Live
这个模型叫 Gemini 3.1 Flash Life。如果你以前在 Gemini 应用里和 Gemini 聊过,那你交流的正是这个模型。我们可以在这里做个小演示。这个演示是 Gemini 在 Google AI Studio 里通过氛围编程写出来的。如果你还记得过去的美好时光——不久以前,人们还会听广播——你可以打电话给电台主持人点一首歌,对吧?你可以点播一首歌。这个演示的构想也是一样的。model is called Gemini 3.1 Flash Life. Now, if you've talked to Gemini before in the Gemini app, that is exactly the model that you're speaking to. And so, we can do a little demo here. So, this demo was vipcoded by Gemini in Google AI Studio. And if you remember the good old times, you know, when you were listening to radio, you know, not too long ago, you could phone up the the the radio host and you could ask for a song, right? You could wish for a song. So, it's the same idea
演示:搭载 Lyria 3 的 AI 电台 DJDemo: an AI radio DJ with Lyria 3
这里,我们让 Gemini Life 担任电台 DJ,然后给 Gemini Life 一个 API,也就是一个调用 Gemini API 的函数,让它接着使用 LIA 3——我们的音乐生成模型——按需为我们创作音乐。好吧,咱们来试试。我们现在就来一场现场演示。来看看。喂,你好!你现在正在直播点唱机节目。今天我们要给你创作什么绝妙的曲子呢,老兄?要把这首曲子安排得妥妥当当,我还需要多了解一点。你想要什么曲风?活力十足的合成器流行乐、舒缓的爵士乐,还是更……嗯,歌词方面想要什么感觉?给我说说具体想法。here. We're using Gemini Life to be our radio DJ and then we're giving Gemini Life an API, a function call to the Gemini API to then use LIA 3, which is our music generation model to then generate some music for us on demand. Okay, let's uh try this out. We're we're just going live demo to live demo here. Let's let's see. Well, hello there. You're live on the live jukebox right now. What absolute banger are we cooking up for you today, mate? To get this track properly sorted, I need a bit more of an idea. What genre are we thinking? Upbeat synth pop, smooth jazz, something with a bit more well. And what's the vibe lyrics-wise? Give us some details.
主持人:好。能不能来一首能量十足的雷鬼顿歌曲?主题是这里在旧金山举办的 AI 工程世界博览会。Host: Yeah. Can we get like a really high energy ragaton, please? About the AI engineering world's fair here in San Francisco.
嘉宾:这听起来可真够劲儿的,老兄。来一首关于 AI 工程界世博会的高能雷鬼顿,太棒了。好吧,要想让它彻底火出圈,你想加进哪些具体的科技元素或大人物吗?还是咱们就奔着那种超大型音乐节的氛围去?Guest: Now that sounds like a proper belter, mate. high energy regaton about the AI engineering world's fair. Brilliant. Right. To make this absolutely top of the pops, any specific bits of tech or big names you want thrown in, or are we just going for that huge festival vibe?
主持人:是啊,给我点惊喜吧。包在我身上,朋友。准备好迎接一场巨大的音乐惊喜吧,我现在就开始生成这首曲子。把雷鬼顿节奏和世博会的科技热潮融合起来。它会震掉你的耳机。世博会。感觉真棒。咱们去舞池吧。神经网络启动了。绝对是开创性的。巧妙的算法翩翩起舞。科技先锋齐聚一堂。这就是创新。低音节拍强劲地脉动。你感受到这股热浪了吗?合成器短促音电力十足。让你的双脚动起来。这就是我们的风格。耶,刚才那段相当酷。你还切换成了多种语言。真不错。谢了。Host: Yeah, just surprise me, please. You got it, my friend. Prepare for a huge musical surprise generating that track right now. Mixing that regaton rhythm with the tech buzz for the World's Fair. It's going to blow your headphones off. Fair. What a feeling. Let's go on the floor. Neural networks firing up. Groundbreaking for sure. Clever algorithms dancing. Tech pioneers mingling. This is innovation. Baseline is pulsing heavy. Can you feel the heat? Sin stabs electrifying. Move in your feet. This is how we are. Yeah, that that was pretty cool. You went multilingual there as well. Nice one. Cheers.
客人:谢了,伙计。很高兴你喜欢这个味道。继续提出那些宏大的想法吧。这台点唱机永不休息。那接下来我们放什么?《Feeling》。Guest: Cheers, mate. Glad you like the flavor. Keep those big ideas coming. This jukebox never sleeps. What are we spinning next then? Feeling.
主持人:好的。嗯,是的,你们可以自己试试看。呃,这是一个在 Google AI Studio 中构建的演示。如果你愿意,可以在 AI Studio 中复刻这个项目并继续完善。我知道这个应用可能不一定对你的业务场景特别有用,如果你正打算把它用于业务的话。不过我认为其中各个独立组件确实很实用。所以,如果你正在构建实时对话代理,嗯,你可能希望它们支持多语言,并具备各种能力,比如调用工具。在这个示例中,我们只是让代理按需调用工具,通过 LIA 3 生成音乐。当然,它也可以调用你的 CRM,获取一些客户信息,或者在后台创建一张新工单。Host: All right. Um, yes, you can try this out yourself. Uh, it is is a a demo that was built in Google AI Studio. Uh so if you want you can fork this in AI studio and continue yourself. Um I know this application might not necessarily be that useful for your business context, you know, if that's something you're looking to do, but I think the individual components are definitely useful. So if you're building real-time conversational agents, uh you know, you want them to be multilingual, you want them to have capabilities, uh tool calls. So in this example, we were just giving the agent a tool call to generate music with LIA 3 uh on demand. Now obviously this could also just call out to your CRM and get some you know customer information or it could create you know a new ticket on the back end.
所以,你知道,我真正希望你们记住的是,你们可以打造这样一种实时多模态对话体验。至于如何利用这个模型来构建,也就是 Gemini 3.1 Flash……Um so you know really what I want you to take away is kind of this real time multimodal conversational uh experience that you can build. And so the way that you can build with this model, so the the Gemini 3.1 flash
Live API 的工作原理How the Live API works
实时对话模型是通过 Live API 实现的。Live API 是一种低延迟的实时语音和视觉 API。借助它,你可以构建多模态的实时交互。它提供了一个有状态的 WebSocket API,你可以创建一个会话,然后通过这个 WebSocket 实时向模型发送文本、音频和视频帧,模型则会以音频片段的形式实时响应。你可以在应用中流式传输这些音频片段。除了音频,你还能获取所说内容的转录文本。如果你之前使用过语音代理,这与传统的级联式语音代理有些不同。real-time conversational model is through the life API. So the life API is a low latency real time voice and vision uh API. So you can build multiodal uh life real-time interactions. And so what this means is you have this stateful websocket API where you create a session and then over this websocket you can send in real time text, audio and video frames to the model and the model will then respond in real time back in audio chunks. And these audio chunks you can then stream uh in your application. Uh and in addition to the audio, you also get the transcript of what was said. So if you've worked with voice agents before, this is a little bit different from kind of your cascading voice agent
原生音频,而非处理流程Native audio, not a pipeline
管线,因为这是一个原生音频模型。因此,模型本身不会经过文本处理,而是将智能直接内置在音频模型中。音频模型可以自然地、原生地理解语音,然后用目前支持的 90 种语言中的任意一种作出回应。这让我们能够非常流畅地在不同语言之间切换。嗯,这里的关键功能包括多语言支持。目前预览版支持 90 种不同的语言。嗯,我们可以插话,也可以打断模型,对吧?如果我们的 DJ 说得有点太长了,我们可以插话、打断它,也可以给它提供工具,而且我们还支持音频转录。如果你正在构建实时对话应用,或者实时多模态应用,那么你知道,实时媒体可以……pipeline because this is a native audio model. So the model itself does not go through text, but rather the intelligence is baked into the audio model. So the audio model can understand the speech naturally and natively and can then respond back in any of the 90 supported languages. So this is what allows us to to switch very fluidly between different languages. Um so kind of the key features here, yeah, multilingual support. So 90 uh different languages currently supported in the preview. Um we can barge in, we can interrupt the model, right? If our DJ is talking a bit too lengthy, uh we we can we can barge in, interrupt it, we can give it uh the tools and we have the audio transcription as well. Now, if you're building real time conversational applications or real-time um multimodal applications, uh you know, real-time media can be
Partners: LiveKit and Pipecat
扩展起来相当有挑战性。所以,我们有一批很棒的合作伙伴。你知道,LiveKit、Pipecat,他们也都在展区里。我建议你去拜访他们。他们做了很多非常酷的东西,能让你快速上手 Gemini Live API。好了,如果你现在想:“好,这听起来很棒,我想要这个。”那么,最简单的尝试方式是……quite challenging to scale. So, we have a great lineup of partners. So, you know, LifeKit, Pipecat, they all actually here in the expo as well. I recommend you go visit them. They do some really really cool stuff to allow you to get up and running quickly with the Gemini Life API. Okay, so if you're now um you know, you're like, "Okay, this sounds great. I want this." Um the easiest way to try
在 AI Studio 中试用Try it in AI Studio
这个功能是通过 Google AI Studio 实现的。我们在 AI.studio 这个域名上花了很多钱,所以希望大家能用一下,这样我们也能获得一些投资回报。所以,去 ai.studio/live 试试模型吧。你可以和它对话,了解它具备哪些能力。现在我们就来试试。我们要访问 AI.dev。我们也有 AI.dev,也请大家使用。那个域名也不便宜。好了,进入 AI.dev/live。在这里,你也可以试试实时翻译模型。我们可以选择 Gemini 3.1 Flash Live 模型,也可以选择其他语音。比如,我喜欢这里的 Puck。我们还可以启用 Google 搜索接地功能。显然,模型是在大量信息上训练出来的,但它无法获取实时信息,对吧?this out is through Google AI Studio. So we spend a lot of money on this domain, AI.studio. So I do appreciate if you use that one. Um so we can get some ROI there. So ai.studioife. You can try out the model. You can talk to it and you can understand the capabilities that are available. So let's do that now. Uh so we're just going to AI.dev. We also have AI.dev. Um yeah, please do use that as well. That one was was also not not so cheap. Uh there we go. AI.devlife. And so here uh we can also try out the live translate model if you want to. Uh but we can here select our Gemini 3.1 flash uh live model. We can select a different voice. Uh for example, I like puck here. And then also we can enable grounding with Google search. So obviously the model is trained on a lot of information but it does not have access to real time information, right?
这样一来,通过让它访问 Google 搜索,它实际上就可以自行搜索,这基本上就像调用一个内置函数。如果我们问它今天的天气,或者你知道,德国为什么在世界杯上表现这么好,呃,我们可以这么做,它可以用 Google 去查……And so by giving it access to Google search, it can actually go off and it's basically like a built-in function call. And if we ask it about today's weather or you know why Germany played so well in the World Cup, uh we can we can do that and it can use Google to to look
演示:语音、视觉与普通话Demo: voice, vision and Mandarin
把那个弄好。好,我们来试试看。that up. Okay, let's give this a try.
主持人:嘿,你能听见我吗?Host: Hey, can you hear me?
主持人:是的,我能听得很清楚。今天有什么可以帮您?Host: Yes, I can hear you perfectly. How can I help you today?
主持人:你能看到我吗?是的,我能清楚地看到你。你穿着黑色衬衫,戴着黑色帽子,坐在蓝色背景前。有什么特别想给我看的吗?Host: Can you see me? Yes, I can see you clearly. You're wearing a black shirt and a black hat sitting in front of a blue background. Anything specific you wanted to show me?
主持人:是啊。我的卫衣上写的是什么?Host: Yeah. What does it say on my sweatshirt?
主持人:我能看到黄色字母印着“Gemini”。还有什么想让我辨认的吗?呃,好吧。我觉得我刚才的普通话说得不太好。呃,你也可以说普通话吗?Host: I can see the word Gemini printed in yellow letters. Anything else you'd like me to identify? Uh, okay. I think my uh my Mandarin didn't work out so well there. Uh, can you also speak Mandarin, please?
主持人:是的,我也会说普通话。你现在想说什么?想聊些什么?Host: Yes, I can speak Mandarin, too. What would you like to say now? What's about?
嘉宾:好吧。如果你看不懂普通话,我很抱歉。我刚才只是在问模型吃了什么,然后我说它不需要吃东西,因为它是个模型。嗯,对,就是这样。好吧,这是你开始使用 Gemini AI Studio 最简单的方式。如果你想进一步了解如何使用 Gemini 和 Gemini Live API 进行开发,Guest: All right. If you don't read Mandarin, I do apologize. I was just asking the model what it has eaten, and I just said it doesn't need to eat because it's it's a model. So, uh, yeah, there we go. All right. So this is the easiest way for you to get started with uh Gemini ai.studiolife. If you want to learn more about building with Gemini and the Gemini live API,
总结Wrap-up
明天早上来2009号房间找我吧,我想是这个房间。查一下日程。日程偶尔会变。呃,除此之外,Dankur,谢谢你。回头见。join me tomorrow morning in uh room 2009, I think. Check the schedule. It it does change every once in a while. Uh but other than that, Dankur, thank you. Cheers.