用 Python 构建把语音变成图片、视频和音频的 AI 智能体

简要说明Brief Description

构建一个能将你的声音转化为图像、视频和音频的 AI（Python）Build an AI That Turns Your Voice Into Images， Video and Audio (Python)

目录Table of Contents

• 概述
  Overview
• 项目演示
  Project Demo
• 设置与安装
  Setup & Install
• OpenRouter
  OpenRouter
• 调用 OpenRouter
  Calling OpenRouter
• 步骤 1：语音转文字
  Step 1: Speech to Text
• 步骤 2：文本生成
  Step 2: Text Generation
• 步骤 3：图像生成
  Step 3: Image Generation
• 步骤 4：语音生成
  Step 4: Voice Generation
• 步骤 5：视频生成
  Step 5: Video Generation
• 最终流水线
  Final Pipeline

概述Overview

在这段视频中，我将向你展示如何构建一个 Python AI 智能体，让它能够生成文本、语音、视频和图像。我们会把所有这些功能整合到一个工作流中，打造一个非常酷的项目：你可以对着电脑说话，告诉它你想创作什么，它就会完成一整套流程，为你制作一段配有完整脚本的视频。真的非常酷。先来看看它的效果，然后我们再进入完整的教程。In this video, I'll show you how to build a Python AI agent that can generate text, voice, video, and images. We'll combine all of this into a pipeline and create a really cool project where you can speak to your computer, tell it what you want to create, and it will go through a full process of making a fully scripted video for you. It's super cool. Let me show you what it looks like, and then we'll get into a full tutorial.

好，我现在就在代码编辑器里，这是我们要构建的流程的一个简化版本。你可以看到这里的各个步骤。接下来，我们会逐一讲解这些步骤。Okay, so I'm in my code editor right here, and this is kind of a simplified version of the pipeline we're going to build. You can see the different steps that we have here. So, really we'll run through these

项目演示Project Demo

不同的步骤。我们先录制声音，也就是把声音录进电脑。然后转录，把音频转换成文本。接着生成脚本，再生成图像，然后生成音频，最后把这一切整合成视频并展示给用户。同时，我们会记录所有费用，这样就能准确知道成本是多少。这里显示的是之前的一次运行。different steps. We'll start by getting our voice, so just recording that into the computer. We'll then transcribe that, so we convert the audio into text. We'll generate a script. We'll then generate an image. We'll then generate audio, and finally put it all together in a video, and then display that to the user. And at the same time, we'll keep track of all of the costs, so we know exactly what the price is. Now, this was a previous run right here.

你可以在我的控制台上看到它，不过我还是再运行一次，让你了解它的作用。好的，首先它会提示我开始说话，我们按一下回车键。“生成一段视频：一位程序员坐在电脑前，喝了很多咖啡，正为一个棘手的 bug 苦苦挣扎。” 好了，然后我按回车键停止录制。You can see it on my console, but let me just run it again, so you get an idea of what it does. Okay, so first it's going to ask me to start talking, so let's press enter. Generate a video of a programmer who's sitting at the computer drinking a lot of coffee and struggling with a difficult bug. Okay, I then press enter to stop the recording.

它现在要做的是开始转录我的语音。你可以看到，它已经用这个 AI 模型完成了转录。接下来它会生成脚本，你马上就能看到。然后它会生成图像，再生成视频，并把所有这些内容都交给我们。整个流程需要一分钟左右，所以我先等它完成，你就能看到最终输出了。好了，现在我们已经完成了语音转录、脚本撰写、图像制作和旁白生成，接下来要做的就是生成视频。What it's going to do right now is start transcribing my voice. You can see it's already done that using this AI model. Then what it's going to do is generate a script, which you'll see in a second. Then it's going to generate the image, then generate the video, and give us all of that content. It takes a minute to run through the process, so let me wait for it to finish, and you'll see the final output. Okay, so we've got to the point where we've done the transcription, we've written the script, we've done the image, we've generated the voiceover, and now what we're doing is generating the video.

这需要等一会儿，因为用 AI 制作视频确实要花更长时间。它会先生成结果，完成后我们就能查看输出。好了，这里的视频刚刚生成完毕，我们可以在下方看到费用摘要，这次花了 3.28 美元。This will take a second because, well, video just takes longer to make with AI. So, it'll kind of pull that result, and then once it's finished, we'll be able to view the output. Okay, so it just finished generating the video here, and we can see we get a cost summary down here, where this cost us $3.28.

其中大部分显然是视频生成的，花了 3.20 美元。生成这个确实挺贵的。不过，我们来看看输出文件夹里的片段，这就是生成的内容，我们来播放一下。对我来说，这看起来相当不错。当然不是世上最好的作品，但这正是我们要求生成的内容。再来看这里的配音，这是我们单独生成的一段配音，只是为了展示音频功能。我们听一小段吧。Most of that obviously coming from in video, which was $3.20. It's quite expensive to create that. But, if we go here to our output folder and we look at the clip, this is what was created and let's run it. And I mean, that looks pretty cool to me. Obviously not the best thing in the world, but that's exactly what we asked for. And if we go here to the voiceover, this is kind of a separate voiceover that we generated just to show the audio capabilities. Let's listen to a bit.

一个程序员，一个顽固的 bug，还有喝得太多的咖啡。One programmer, one stubborn bug, way too much coffee.

你应该明白我的意思了。然后，我们最初采用的封面图是这样的。这样一来，所有内容就都齐了。当然，我们还有脚本等等。我来向你展示如何创建这种类型的代理。让我们开始吧。And you get the idea. And then the cover image that we actually started from was this. So, we have all of that content. We have obviously the script and all that. I'm going to show you how we can create this exact type of agent. Let's dive in.

设置与安装Setup & Install

好，我们将从一个空白的 Python 项目开始。我会复制粘贴大量代码，而所有这些代码都可以通过简介中的链接免费获取。所以，欢迎下载。你只需要加入我的学习社群，完全免费，我就会把所有代码都给你。我们接下来采用这种做法的一个好处是，只需要使用一个 API 密钥。Okay, so we're going to start from a blank Python project. I'll copy in lots of code and all of this code will be available for free from the link in the description. So, feel free to download it. In order to do that, you just have to join my school community, completely free, and I'll give you all of the code. Now, one of the benefits of the ways that we're going to do things here is that we'll only use a single API key.

通常来说，如果你想生成视频、图像等不同类型的媒体，或者想使用各种不同类型的模型，就需要不同的 SDK、不同的模块，还可能需要七个不同的 API 密钥。今天我们会使用 OpenRouter，感谢他们赞助本期视频。稍后你就会看到它是如何运作的。对于这类项目来说，它特别好用，因为你只需要管理一个 API 密钥。Typically, if you want to generate like video, images, different type of media, or you want to use all different types of models, you need different SDKs, you need different modules, you need like seven different API keys. Here, we're going to be using OpenRouter who kindly sponsored today's video. You'll see how it works in a second, but it's very good especially for these types of projects because you just manage a single API key.

那么，要开始的话，我们只需新建一个项目。我刚刚在 Cursor 中打开了一个文件夹。你可以使用任何你喜欢的代码编辑器。接下来，我们打开终端，开始安装需要的一些包。我会输入 `UV init`，然后输入一个点。我会使用 UV 作为 Python 包管理器，也建议你这样做。如果你还没有安装 UV，安装起来非常简单，我会在屏幕上播放一段视频，介绍如何使用它。So, in order to get started, we're just going to open up a new project. So, I've just opened a folder here in Cursor. You can use any code editor that you want. We're going to go into our terminal and we'll just start setting up some of the packages that we need. So, I'm going to type UV init and then dot. I'm going to use UV for my Python package manager and I suggest that you do the same. If you don't have UV installed, very easy to add it and I'll put a video on screen that shows you how to use it.

好，我们现在要执行 UV init dot。从这里开始，我们输入 UV add，后面加一个空格，然后列出所需的 Python 包。这里我们需要 NumPy、python-dotenv（用来加载环境变量）、request 模块、sounddevice，以及 soundfile，后两者用于处理我们对着电脑说话的场景。Okay, so we're going to do UV init dot. From here, we're going to add the Python packages that we need by typing UV add space and then listing them out. Now, what we'll need here is NumPy, python-dotenv just load our environment variables. We'll need the request module. We'll need sounddevice and we'll need soundfile just for handling actually us speaking into the computer.

好，现在把这些都输入进去，然后按回车，应该就可以了。接下来，我们要创建一个 ENV 文件，也就是 .ENV，然后输入 openrouter_api_key=，再粘贴我们的 OpenRouter API 密钥，我稍后会演示如何获取它。Okay, so let's bring in all of those and press enter and then we should be good to go. Now, from here we're going to make a ENV file, so dot ENV and we're going to put openrouter_api_key is equal to and then we're going to paste our openrouter API key which I'll show you how to get in a second.

OpenRouterOpenRouter

基本上，我们要做的是使用 openrouter 处理所有不同的模型请求。当然，如果你愿意，也可以使用其他库，只要在那个位置替换掉就行。不过，这样做非常方便，而且同样能让我们通过一个统一的端点访问所有模型。所以，如果你想使用 openrouter 并跟着我一起操作，可以点击我在简介中提供的链接，然后在那里注册一个账号。Basically, what we're going to do is use openrouter for all of the different model requests. Of course, you can use a different library if you want and just swap it in that place. But this makes it very easy and again allows us to access all of the models from one single endpoint. So, if you want to use openrouter and follow along with me, what you can do is go to the link that I have in the description. You can just make an account there.

从这里，你可以添加任意金额的点数。最低可以只充一美元。当然，如果你想生成完整视频，就需要多充一点，不过你也可以使用一些更便宜的模型。接下来，我们会进入工作区。你也可以按需创建不同的工作区，我就使用默认工作区。然后我会进入 API 密钥页面，生成一个新的 API 密钥，用于这个项目。From there, you can add any amount of credits that you want. You can start with as low as like literally a dollar and then obviously if you want to generate full videos, you'll need a little bit more, but you can use some cheaper models. And from here, what we'll do is we'll just go into our workspace. You can create different ones if you want. I'll just use the default workspace. I'm going to go to API keys and then just generate a new API key that I can use for this project.

创建 API 密钥时，给它起个名字，并设置一个有效期，具体取决于你的需求。然后，如果你想设置信用额度，避免花太多钱，可以直接在这里设置。你可以自定义额度。我现在会再加载一个已经使用过的密钥。你可以看到，最近我用其中一些密钥花了一点钱。使用 OpenRouter 的好处是，你可以通过这一个端点来处理所有模型。And when you make the API key, give it a name, give it an expiry obviously depending on what you want. And then if you want to set a credit limit so you don't spend too much money, feel free to set that right here. You can make it a custom amount. I've got another key that I'll load in right now that's already been used. You can see that I've spent uh a little bit of money here with some of the keys recently. And the benefit of using openrouter is that again, you handle all of the models from this one endpoint.

你可以在这里搜索所有这些模型，查看文本、图像、视频和语音模型。其实我特别喜欢的一点是，点进这些模型后，你可以看到确切的价格、性能、正常运行时间，以及所有正在使用它的应用和活动情况。这里确实挺有意思的；当我想使用大量不同模型，而不是逐一访问各家供应商时，这里就是我的首选。You can search through all of them here, see text, image, video, speech. And actually what I really like is that if you click into these models, you can see the exact pricing, the performance, the uptime, you know, all of the apps that are using it, activity. Really kind of interesting place and this is my go-to choice when I want to be using a ton of different models rather than going to all of the individual providers.

此外，这里还会清晰地列出我的各项开支明细。你可以看到，我最近一直在折腾 Jev。好了，我们需要的就是这些。只要获取 API 密钥即可。接下来，我们只需把它粘贴到环境变量文件中。显然，不要把它分享给任何人，并确保给账户充值一些额度，这样你就可以跟着视频操作了。好了，我已经把 API 密钥放在 .env 文件中了，接下来我们就按视频里展示的步骤来，教你如何转录音频、生成配音、生成文本、生成图像以及生成视频。Plus, it gives me a really nice cost breakdown here of everything that I'm spending money on. You can see I was messing with Jev here recently. Okay, so that's all we need. We just get the API key. From here, what we'll do is just paste it into our environment variable file. Obviously, don't share it with anyone and just make sure you load your account with a few credits so that you can follow along with the video. Okay, so I've got my API key here inside of the.env and now we're going to go through the steps that you saw in the video where I'm going to show you how to transcribe your audio, how you can generate a voiceover, how you can generate text, how you can generate images, and how you can generate videos.

关键在于，一旦你理解了如何做到这一点，就几乎可以将它应用到任何你想做的项目中。The key here is that once you understand how to do this, you can apply it to pretty much any project that you want.

那么，第一步，我们要创建一个 helper.py Python 文件。这是因为接下来我们要进行许多非常相似的操作，我希望把一些会反复用到的函数都放在这个文件里。So, step one, we're going to make a helper.py Python file. That's because we're going to be doing a lot of really similar operations here and I just want to have some functions that we're going to use over and over again in this single file.

调用 OpenRouterCalling OpenRouter

所以，正如我刚才提到的，我只会粘贴一些小段代码。你可以随时暂停下来自己输入，也可以直接从我放在简介里的链接中复制。接下来，我要粘贴完整的 helper.py 文件。我们把它切换到全屏，然后我会逐行讲解，让你明白我们在做什么。现在注意，我们导入了 OS 和 time，还有 requests，然后是 dotenv 中的 load_dotenv。So, like I mentioned, I'll just be pasting small snippets of code. Feel free to pause and type them out yourself or just to copy them from that link that I have in the description. So, I'm going to paste this in which is the full helper.py file. Let's make this full screen and let me go through it line by line so you understand what it is that we're doing. Now, notice that we're importing OS and time as well as requests and then.env load.env.

所以，基本上，这一行，或者说这个导入和函数，只是从 .env 文件中加载我们的 API 密钥。在 Python 中操作时需要用到它。就这样，它只是让密钥在环境中可用。然后我们设置 OpenRouter API URL。注意，它实际上只有一个端点。所有不同的模型都使用同一个端点，只是有些细微差异。So, basically, this line right here or this import and this function just loads our API key from the.env file. You need this when you're doing it in Python. That's it. Just makes it accessible to the environment. Then we set the OpenRouter API URL. Notice it's literally just a single endpoint. It's the same endpoint for all of the different models just with slightly different variations.

你只需要选择想使用的模型，然后发送不同的参数，对吧？接下来，为了使用它，我们需要传入一个授权令牌，也就是我们的 API 密钥。所以，我们写上 authorization，然后空一格，再加上 bearer，接着从环境变量中加载这个 API 密钥。就这么简单。我们只需要这两个值。然后，我们要做的是新建一个目录。这是当前目录下的一个输出目录，用来存放我们将要创建的所有内容，这样就不会弄丢了。You just pick which model you want to use and send different um you know, parameters, right? Then, in order to use this, we need to pass an authorization token which is our API key. So, we say authorization and then we pass bearer with a space and then load in that API key from our environment variable. That's it. We just need these two values. Then, what we do is we create a new directory. This is an output directory in our current directory just to store all of the stuff that we're going to be creating so we don't lose it.

这里看到的其余许多内容都与成本跟踪有关，这样我们就能了解实际运行整个流程时每项操作的成本。而使用 Open Router 再次说明了，这非常简单。我们有一个成本字典，用来跟踪成本。And a lot of the rest of the stuff that you see here is going to be related to tracking the cost so we can see how much every single operation costs when we actually go through this pipeline which again is very easy using Open Router. So we have this cost dictionary which will track the cost.

我们有一个 post 函数。post 要做的就是向 Open Router 发送请求。所以我们调用 request.post，传入 API 基础 URL，再加上路径。路径可以是 /text、/video、/image、/model 之类的，对吧？然后传入请求头。请求头里只有 API 密钥。接着传入负载，也就是参数，用来指明我们实际想生成什么内容。We have a post function. What post is going to do is it's just going to send a request to Open Router. So we say request.post, take our API base URL, add the path. So the path could be like /text, /video, /image, /model, whatever, right? And then we pass our headers. Our headers is just the API key. And then we pass the payload which is going to be the parameters which will indicate kind of what it is we actually want to generate.

我们会检查响应是否正常。如果不正常，就直接抛出错误。如果正常，就返回响应。就这样。这就是我们向 Open Router 发送请求的方式。接下来是这个 add cost 函数。它所做的就是把成本加到这个变量里，这样我们就能跟踪成本。这里非常简单，你会注意到我们会将数值四舍五入到小数点后四位，这样我们就不会跟踪小于这个精度的数值。We check if the response is okay. If it's not, we're just going to raise the error. If it is, then we'll return the response. That's it. That's how we send a request to Open Router. Next we have this uh add cost function. All this does is just add the cost to this variable so we can track it. Pretty straightforward here and you'll notice that we're rounding off to the fourth decimal point just so that we don't track anything that's less than that.

然后我们有一个 call 函数。它的作用只是使用 post 函数发送请求，然后加上该响应的成本。注意，我们在这里获取成本的方式就是这一段。因此，每个 Open Router 响应都会提供用量数据和成本，我们可以直接从中提取。然后，我们可以通过查看这里的这个字段来获取模型。接下来，我们有这个查找成本的函数。Then we have a call function. What call is going to do is just send a request using the post function and then add the cost of that response. Notice that the way that we get the cost here is just this right here. So in every Open Router response it gives you this data usage and then cost where we can just directly pull that out. Then we can get the model by looking at this field right here. Continuing here we have this look up cost function.

现在先不展开讲，不过有几个操作不会直接告诉你费用。如果遇到这种情况，可以通过这里的请求查询费用。同样，把请求发送到 Open Router /generation，然后查询收到的响应 ID，它就会告诉你总费用是多少。辅助函数的内容就到这里。Won't get into it right now but there are a few operations that don't directly tell you the cost. If that's the case, you can look up the cost by using this request right here. Again, sending it to Open Router /generation and then looking up the ID of the response that you got and then it will just tell you what the total cost was. That's it for the helper function.

再说一遍，或者放在辅助文件里，总之确保把它写下来。现在我们要继续编写 step one.py。我会这样编写 step one.py，然后开始实际从用户那里获取音频并进行转录。Again, or helper file, just make sure you have that written. And now we're going to move on to step one.py. So I'm going to do step one.py like this and we're going to start actually getting audio from the user and then transcribing it.

步骤 1：语音转文本Step 1: Speech to Text

好的，第一步也是一样，我会直接把代码复制进来，然后我们一起过一遍，因为现在没人会手动写代码了，大家都在用 AI 生成代码，但你应该了解它实际做了什么。所以，第一步仍然是让用户能够对着电脑说话。要做到这一点，显然我们需要访问麦克风，生成一个音频文件，或者说生成用户所说内容的音频数据，然后使用 AI 模型将其转录成文本。Okay, so same thing for step one, I'm just going to copy in the code and we're going to go through it because at this point no one's manually writing code, we're all using AI to generate it, but you should understand what it actually does. So, the first step again is we just want the user to be able to talk into the computer. Now, in order to do that, we need to access obviously the microphone, we need to generate an audio file or kind of the audio content that they spoke, and then we need to use an AI model to transcribe that into text.

现在，这个过程称为语音转文本。你可以看到，我们在这里要使用的模型是 OpenAI Whisper Large V3 Turbo。你可以使用各种不同的模型，尽管去尝试。这个模型的成本极低，只有一分钱的一小部分，而且速度非常快，所以我们在这里选择了它。Now, this process is known as speech-to-text, and you can see that the model we're going to use here is OpenAI Whisper Large V3 Turbo. Now, there's all kinds of different models you can use, feel free to play around with them. This one is extremely cheap, you'll see it's literally a fraction of a cent, and it's very fast, which is why we've gone with it here.

现在，我们先导入一些内容：base64、NumPy、sounddevice 和 soundfile，以及刚才在辅助文件中编写的 call 函数。再次说明，call 的作用就是向 OpenRouter 发送请求。接下来，我们指定模型和采样率，并定义两个函数：record 和 transcribe，因为这仍然是一个分两步进行的过程。我们先录制音频；音频录制并保存后，就可以对其进行转录并生成文本。Now, we start by bringing in a few imports here, base64, NumPy, sounddevice, and then soundfile, as well as bringing in the call function from the helper file that we just wrote here. So, call again is just going to actually send the request to OpenRouter. From here, we specify the model and the rate, and then we have two functions, record and transcribe, because again, this is a two-step process. We're going to record the audio, once the audio is actually recorded and saved, then we can transcribe it and generate the text.

接下来我们会用这段文本实际发送给 AI 模型，让它生成脚本、生成视频，你明白我的意思。所以这里的代码有点复杂，但我们实际上是在说：“好，准备开始说话时按回车键。”That text we'll then use later to actually send to the AI models to generate a script, to generate the videos, you get the idea. So, the code here is a little bit complicated, but effectively what we're doing is we're saying, "Okay, press enter when you want to start talking."

他们一按回车，我们就会进入这里的循环。接下来我们会使用声音设备，也就是计算机的默认声音设备。我们会创建一个输入流。我们设置采样率，本质上就是每秒要采样多少位。我们将声道数设为 1；你可以有多个声道，但在这个例子中，我们只需要一个。我们还有一个回调函数，我认为它就是每次处理采样时要执行或调用的函数。As soon as they press enter, we're going to go into this with loop here. What will happen is we're going to use the sound device, which is just the default sound device for the computer. We're going to create an input stream. We set the sample rate, which is essentially how many bits we want to sample every single second. We set the channel as one, you can have multiple sound channels, in this case, we only need one. We have a callback function, which I believe is just what's going to be handled or called every single time we process the sample.

然后我们在这里要做的就是追加当前数据块，也就是我们正在读取的所有数据。接着，我们只要在这里输入这个内容，就能停止当前操作。按下回车后，它就会停止这个上下文管理器（也就是这个 with 语句），然后收集这些数据块中的所有音频。What we do here then is we append the current chunk, which is all of the data that we're reading. And then we just stop that what we have by having this input inside of here. So, as soon as we press enter, it's going to stop doing this kind of context manager, which is what this with statement is, and then gather all of the audio inside of these chunks.

基本上，我们所做的就是对音频进行流式处理。每秒，我们都会把刚才说的内容转换成数据，或者转换成计算机能够理解的形式，然后放进这个 chunks 数组或列表里。完成之后，我们会把这些内容写入文件。要写入文件，我们需要一个文件路径，默认情况下我们就把它设为这个值。然后我们使用 NumPy 把所有这些数据块合并起来，再指定保存这个音频文件时要使用的采样率。Basically, all that's happening is we're ex-streaming the audio. So, every second we're going to take whatever was said, convert it into data, or something the computer can understand, put it in this chunks array or this chunks list. And then once this is done, we're going to write that into a file. Now, to write that into a file, we have to have a file path, which by default we just set as this. And then we're using NumPy just to take all these chunks, combine them together, and then we just specify the rate that we want to save this sound file as.

就是这样。我知道这看起来很复杂，但这就是我们处理音频文件的方式。随着视频的推进，你会看到这里有很多文件操作。现在，在我们录下文本并将其保存为文件之后，接下来要做的就是转录它。That's it. I know it looks complicated, but that's how we deal with sound files, and you'll see a lot of file operations here as we go through the video. Now, after we've recorded the text and we've saved it as a file, what we're going to do is transcribe it.

因此，为了转录它，我们先获取文件路径，然后开始提取音频数据。这里我们实际上是把数据转换成了一个文件。现在，我们要把文件再转换回数据，这样就能读取并转录它了。其实你可以把这些步骤合并成两个步骤，但我想保存文件，这样我们就能回放收听，所以这里我把它们分成了两个独立的步骤。So, in order to transcribe it, we take the path to the file, and we start by grabbing the audio data. Now, what we did here is we essentially took the data and converted it into a file. Now, we're going to take the file and convert it back into data so that we can read it and transcribe it. You actually could combine these steps into two, but I wanted to be able to save the file so we could listen back to it, so that's why I have two separate steps right here.

所以，我们使用 base64，也就是我们需要的格式，来对它进行解码。我知道这看起来有点让人困惑，但本质上，我们会说 base64 base64 encode。我们以字节方式打开这个文件，这就是 RB 的含义，然后把全部内容读进来。编码之后，我们再对它进行解码，这样就能得到一种常规格式，可以传递给转录模型。So, we use base64, which is the format that we need this in, to decode this. So, I know it looks a little bit confusing, but essentially we say base64 base64 encode. We open up this file as a byte, that's what RB stands for, and read all of it in. And then once we've encoded it, we decode it, so we get it in kind of the normal format that we can pass to the transcription model.

为了进行转录，我们要发起一个调用。调用时，我们说明想要进行转录，然后传入要发送请求的路径，也就是 /audio/transcriptions。如果你想从 OpenRouter 查找这些路径，可以到这里的模型页面，然后进入转录模型。我随便选一个。我们用的应该是 OpenAI，所以来这里找找看。In order to do a transcription, what we're going to do is do a call. For the call, we say that we want to transcribe, and then we pass the path that we're going to send the request to, which is /audio/transcriptions. If you want to find these paths from open router, you can just go over here to models. From here, you can go to transcription models. Uh I'll just pick one. So, I think we're using OpenAI, so let's look for it here.

没错，就是 Whisper large V3 Turbo。你点到这里，就会看到一个 API。通过 API 页面，它会准确展示如何调用这个模型，我就是从那里获取了发送请求所需的信息。你可以看到，模型就在这里。我们有这个 X-Generation ID。总之，传入你的请求头，再传入数据，也就是 base64 音频，就可以了。Yeah, Whisper large V3 Turbo. And if you go over here, you'll see an API. And from API, it will show you exactly how you can call this model, which is where I got this information in order to send the request. So, you can see that's the model right here. We have this X-Generation ID thing. Whatever, pass your headers, pass your data, which is the base 64 audio, and you are good to go.

就是这样。这就是发送请求的方式。接下来我们会这样说：“好，我们要将请求发送到这个模型。输入音频就是之前看到的 WAV 格式数据。然后我们会返回文本数据，也就是转录结果。”就这样。然后我们会打印文本，再将它保存到文件中，这就是完整的第一步。现在，我们打开它并运行。执行 `UV run step1.py`，它应该会提示我们在这里按 Enter。That's it. That's how you send the request. So, from here, we say, "Okay, this is the model we want to send the request to. The input audio is our data in the format wave, as you saw before. And then we're going to return data text, which will be the transcription." That's it. We then are going to print the text and then save it into a file, and that's the full step one. So, let's open this up and let's run it. UV run step1.py, and it should ask us to press enter here.

那么，我们开始吧。嗨，我只是在做个测试。你能告诉我这个转录功能是否正常吗？看看它能不能用。按下回车，然后过一会儿，转录结果应该就会显示在这里。我们说：“嗨，我只是在做个测试。你能告诉我这个转录功能是否正常吗？看看它能不能用。”然后我们打开输出文件夹，就能看到 transcript.txt 和 idea.wave，我可以播放这个音频。So, let's do it. Hi, I'm just doing a test. Can you tell me if this transcription is working? Let's see if it functions. Press enter, and then it should give us the transcription here in a second. And we say, "Hi, I'm just doing a test. Can you tell me if this transcription is working? Let's see if it functions." And if we go to output, we have this transcript.txt and idea.wave that I can play.

嗨，我只是在做一个测试。你能告诉我这个转录功能是否正常吗？我们来看看它能不能用。你可以看到，我们拿到了那个音频文件。这就是第一步。要获取转录内容，我们实际上只需要做到这里。现在，我们要继续进行第二步，开始发送这些内容并生成一些 AI 内容。Hi, I'm just doing a test. Can you tell me if this transcription is working? Let's see if it functions. And you can see that we get that audio file. So, that is step one. That's really all we need to do in order to get the transcription. And now what we're going to do is move on to step two, where we can actually start sending this in generating some AI content.

步骤 2：文本生成Step 2: Text Generation

好，现在我们进入第二步。我会在这里再次粘贴代码。第二步是生成一段脚本，然后我们可以把它转换成图像和视频。因为我们还是希望有这样一个五到六步的流程。基本上，我们要做的主要是明确希望模型如何撰写脚本：给它一个提示词，再提供一个大致的模式，让它按照这种格式输出。Okay, so we're moving on to step two. I'm going to paste in the code again here. And step two is going to be to generate a script that we can then turn into an image and to a video. Because again, we want to have this kind of five or six-step pipeline. Now, basically, what we're doing here is mostly just specifying how we want the model to write our script by giving it a prompt and giving it kind of a schema to give us the output in.

这个流程相当简单。我们只是向 OpenRouter 发送请求，然后获取返回的脚本。这次你会注意到，我们实际上指定了多个模型。这是因为向 OpenRouter 发送请求时，你可以这样做；如果你使用的某个模型受到速率限制，或者由于某种原因宕机或不可用，就可以使用备用模型。The process is pretty straightforward. We're just sending the request to OpenRouter and then getting the script back. This time you're going to notice that we actually specify multiple models. That's because when you send a request to OpenRouter, you can do that and that will allow you to have fallback models if one of the models you're using is rate-limited or if for some reason it goes down or it's not available.

这真的非常好，尤其是在生产系统中，这样你就可以始终通过一个密钥来路由请求，而不必自己编写回退逻辑。基本上，我会说：“嘿，我想使用 GPT-6.1-Soul，但如果它不可用，就改用 Gemini-3.8-Flash。”然后我们会指定一个提示词和一个 schema。这里的 schema 是 JSON schema，我这么做的原因是想强制模型只以 JSON 格式向我返回响应。This is really good especially in production systems so you can always have it routed through one key and you don't need to write the fallback logic yourself. So basically I say, "Hey, I want to use GPT-6.1-Soul, but if that's not available, then go to Gemini-3.8-Flash." We then are going to specify a prompt and a schema. Now, the schema is a JSON schema and the reason I'm doing this is I want to force the model to send me a response in JSON format only.

这被称为结构化输出。大多数现代模型都支持这种方式，它可以让你强制规定输出的结构，确保得到的不是一些随机文本。基本上，我想让它帮我写一个脚本，而我希望输出采用这里所示的格式：一个包含 script、image prompt 和 video prompt 的对象，它们的类型都为字符串。This is known as structured output. Most of the modern models support it and this allows you to enforce the structure of the output and make sure that you're not just going to get some random text. So basically, I want this to write a script for me and the way that I want to have my output here is in this format right here where I have an object that has script, image prompt, and video prompt which are all type of string.

如果你不知道如何生成这些 schema，也可以让 AI 帮你生成，不过基本上就是这样，对吧？我们说：“嘿，我的输出类型将是一个对象。我的对象中有三个属性，对吧？就是这三个。在这些属性中，我们要指定类型。所以我会说：‘嘿，我希望这是一个字符串。’然后我会描述我希望这个属性是什么样的。”If you don't know how to generate these schemas, you can always have AI create them for you, but basically it looks like this, right? We say, "Hey, the type of my output is going to be an object. The properties in my object are going to be three, right? These three here. In these properties, we're going to have the type. So I'm saying, "Hey, I want this to be a string." Then I put a description of what I want the property to be.

因此，description 是一段 40 到 60 个词的配音稿，并带有 excited 或 whispers 之类的音频标签。对于 image prompt，类型是 string，description 是一段详细的 16:9 封面图片提示词；然后是 video prompt，即一段 8 秒的视频片段，从封面图片开始，再描述画面动作。接着我会说，必填字段就是这些：script、image prompt、video prompt，以确保它包含这些内容。So description, a 40 to 60 word voiceover with audio tags like excited or whispers. For the image prompt, type string, description a detailed prompt for a 16 by 9 cover image, and then video prompt a 8-second clip that starts on the cover image and then describe the motion. Then I say the required fields are this, so script, image prompt, video prompt to make sure that it includes those.

另外，我将 additional properties 设置为 false，这样它就不会返回任何我没有要求的内容。就是这样。接下来，我们有一个名为 write script 的函数。我们实际做的就是在这里向 Open Router 发送请求。我们会说：“嘿，我们想生成一个脚本。”然后我们会访问 {slash} chat {slash} completions 端点，因为这是我们通过 API 实现该操作的方式。接着，我们指定模型。And additional properties I set as false so it doesn't give me anything outside of what I asked for. So that's that. Then what we do is we have a function called write script. Really all we're doing is sending the request here to Open Router. So, we say, "Hey, we want to generate a script." We're going to go to the {slash} chat {slash} completions endpoint because that's how we do that from the API. Then, we specify our models.

注意这里有多个模型，因此必要时我们会使用备用模型。接着我们发送一条消息，因为我们将进行类似于普通 API 对话的常规文本交流。所以，这实际上是一个兼容 OpenAI 的 API 端点，这意味着我们想要生成文本补全时，需要传入消息。我们可以传入多种类型的消息，例如系统消息、用户消息、助手消息或工具调用。Notice there's multiple models, so we'll go to the fallback if we have to. We then send a message because we're going to have kind of a normal text conversation like we would with a normal API. So, this is effectively an OpenAI compatible API endpoint, which means that when we want to generate a text completion, we do that by passing in messages. Now, we can pass multiple types of messages like a system message, a user message, an assistant message, or a tool call.

在我们的例子中，我们只想发送一个提示词，所以就像这样发送。不过，基本上，你可以和模型进行一段完整的对话，把所有上下文都传进去，这样它就知道你之前说过什么，也就是在生成补全内容之前，它会先获得完整的对话链。总之，你可以看到，我们这里的角色是 user，content 只是提示词加上我们的文本，而这段文本就是传入这个函数的内容，是我们之前生成的。In our case, we just want to send a single prompt, so we send it like this. But basically, you could have a full conversation with the model, pass all of that context in so it knows what you said previously, so it kind of has that full chain before it gives you a completion. Anyways, you can see we have role user, content is just the prompt plus our text here, and the text is what's passed into this function that we would have generated previously.

然后我们指定这种响应格式。这是可选的，你不必指定，但在我们的例子中，我们希望输出采用这种结构化的 JSON schema 格式。于是我们说：“嘿，类型是 JSON schema。这就是 schema。名称是 video plan。它是 strict 的，也就是说始终都要给我这个格式。”schema 正是我们在这里指定的内容。然后，我们指定 provider，并将 required parameters 设为 true。We then specify this response format. This is optional. You don't need to specify this, but in our case, we want to have this kind of JSON schema structured output. So, we say, "Hey, it's type JSON schema. This is the schema. The name is video plan. It is strict, meaning always give it to me." And the schema is exactly what we specified here. Then, we specify the provider where we say required parameters equals true.

这意味着只使用能够让这些参数生效并支持此 schema 的模型提供商。这是因为 Open Router 实际上为同一个模型提供了多个提供商，并且可以在它们之间切换。所以，我们在这里指定的就是这个。接下来是 json.loads。我们所做的就是取出针对这个提示返回给我们的响应——它其实藏在这个对象里——并将其加载为普通的 Python 字典，这样我们就可以进一步解析它了。And this means only use a provider of these models where these parameters are going to work and it's going to support this schema. That's because Open Router actually has multiple providers for the same models. It can switch between them. So, that's what we're specifying right here. Then, we have json.loads. So, all we're doing is taking the response that actually gave us for this prompt, which is kind of buried in this object right here, and loading that as a normal Python dictionary so we can then parse it out.

好。然后我们说：“文本是这样的。”接下来，我们基本上要打开上一步得到的提示词。我们把它称为“right script”。只需打印出实际生成的脚本内容，然后将其写入文件，以便在下一步中使用。Cool. We then say, "Text is this." So, we're just going to open essentially the prompt that we had from the previous step. We're going to call this right script. We're just going to print out whatever the script was that was actually generated, and then we're going to dump that into a file so we can use it in the next step.

那么我们运行 UV run 来执行这个，然后这是 step_two.py。看看会得到什么，以及它是否会生成那个脚本。好，我们现在已经拿到了完整脚本。我知道它有点难读，但基本上，脚本的内容是：我正在执行一项任务，你告诉我这段转录是否正常。我们来看看它能不能运行。我正在说几句简单的话，随便说什么。然后它会生成一张用于语音转录测试的简洁现代风格的 60×9 封面图，背景是深海军蓝色，还会给我们视频提示词。So let's run this one by going UV run, and then this is step two. {dot} py, and let's see what we get and if it gives us that script. Okay, so we just got the full script here. I know it's a little bit difficult to read, but basically the script is I'm doing a task and you tell me if this transcription is working. Let us see if it functions. I'm speaking a few simple sentences, whatever. And then it says a clean modern 60 by 9 cover image for a voice transcription test, dark navy background, and then it gives us the video prompt as well.

现在这些东西其实又没什么意义，因为我们只是在测试，不过你明白我的意思：它确实能做到这一点，而且总成本是……呃，是多少来着？48？我都不知道该怎么读。连1美分都不到。算了，我猜那应该是48，也就是一美分的4800分之一。Now these ones don't really make sense again because we're just testing, but you get the point that it was able to do that and the total cost was uh what is it? 48 I don't even know how you would read that. It's not even 1 cent. Whatever 48 before 4800s of a cent I guess is what that is.

这就是第二步。现在我们进入第三步。So that is step two. Now let's move to step three.

步骤 3：图像生成Step 3: Image Generation

好的，第三步就是生成图像。让我先把这里清空，然后继续看这段代码。和之前一样，我们会引入 base 64、JSON、requests，然后再引入那个辅助函数。这次我们传入 API 和 call。要生成图像，只需调用这里的这个函数，非常简单。然后我们会在下面做一些稍微复杂的处理，来了解图像实际接受哪些参数。Okay, so step three is going to be to generate an image. So let me clear this and move on to this code. Now same as before, we're bringing in base 64, JSON, requests, and then we're going to bring in that helper function. This time we take in the API and we take in call. Now in order to make an image, we can just call this function right here, which is very straightforward. And then we're going to do something a little bit fancier down here to understand um what parameters the image actually accepts.

不过，要实现这一点相当简单。首先，我们指定模型。当然，模型有很多种。所以我们回到这里，进入模型页面，再返回。我们可以进入图像模型，然后对它们进行排序，随便浏览一下，看看价格。我们使用 Gemini，只是因为它速度很快，而且相对便宜。But to do this is pretty straightforward. First of all, we specify the model. Of course, there are tons. So if we go back here and we go to models, let's go back. We can go to image models and then we can sort them, you know, whatever, look through them, see the price. So we're using Gemini just because it's quite fast and it's relatively inexpensive.

接下来是生成图像函数。我们只需要说：“好的，我们要生成一张图像。”然后我们会使用 {slash} images。这就是模型。这是生成图像的提示词，然后我们指定宽高比，就完成了。接下来我们需要做的就是将图像实际保存为文件，这需要下面这三行代码。因此，我们需要从图像文本中获取 data 和 B64 JSON。这是因为图像会以这种奇怪的 JSON 格式发送回来。Moving on, we have the make image function. All we do is say, "Okay, we want to make an image." We're going to {slash} images. This is the model. This is the prompt for making the image, and then we specify the aspect ratio, and that's it. Then what we need to do is actually save the image as a file, which requires these three lines. So, from the image text, we have to take the data and the B64 JSON. That's because images are going to be sent back to us in this kind of crazy JSON format.

所以，我们要把它拿过来，将其解码成普通数据，然后保存到一个字节文件中，之后它就会渲染成图像，这样我们就能实际打开它了。就是这么简单。接下来，我还在这里加了几行代码，展示这些图像支持哪些参数，因为可以传入的参数有很多种。So, we're going to take that. We're going to decode that into normal data, and then we're going to save that into a bytes file, which will then render as an image, so we're able to actually open it up. That's literally it. Now, if we keep going, I also just added a few lines here that show us what parameters these images will take because there's a lot of different ones that you can pass.

基本上，我们所做的就是向 images/models.json 发送请求，然后获取数据。接着，我们遍历这些数据，找出实际要向其发送请求的模型，然后打印出所有设置及其允许的取值。这样，我们就能看到一份完整的参数列表，了解使用当前模型生成图像时可以发送哪些参数。Basically, what we've done is we've sent a request here to images/models.json and then gotten the data. We've then looked through all of this to find the model that we actually want to send a request to, and then we're just going to print out all of the settings and the allowed values for those settings. That way, we can see a full list of all of the parameters we could send to generate the image based on the model that we're using.

所以，这是另一个 API 请求，你可以发送这个请求，它同样会返回所有不同的图像模型。另外，要传给它们的内容也各不相同。现在我到这里运行一下，执行 `uv run`。这会运行 `step3.py`。看看会得到什么。你可以看到，它告诉我们：“嘿，对于这张特定的图像，我们可以传入一个分辨率。”So, this is another API request that you can send that again will just give you back all of the different image models, and again, what you send to them because they're all different. So, if I go here now and run this, let's go UV run. This is going to be step3.py. Let's see what we get. And you can see that it shows us, "Hey, for this particular image, we can pass a resolution.

我们可以传入各种不同的宽高比。”然后它在这里显示了输入参考图。我不确定这会是什么，如果有的话。所以，我想，好吧，这里什么也没显示出来。然后我们就在这里得到这张图，花了我们 67 美分。如果我们打开 cover.png，就能看到它按要求为我们生成了这张转录测试图片。现在，为了测试一下，我们实际来试着更改图像生成模型。We can pass various different aspect ratios." And then it shows us input references here. I'm not sure what that's going to be if anything. So, I guess okay, nothing showed up there. And then we get the image right here, which cost us 67 cents. And if we go to cover.png, you can see it created this transcription test image for us like it asked for. Now, just for testing purposes here, let's actually try to change the image uh generation model.

那么，我们也许可以在这里用 Recraft。来看看怎么使用它。首先进入 API。我们要找到它的名称，就是这里这个。看起来我们不需要传入其他太复杂的参数。我觉得这样应该没问题。那我们就到这里替换模型。好，已经替换好了。清除一下，然后用 Recraft 运行看看，因为我很想知道会得到什么结果。So, let's maybe use Recraft here. And let's see how to use this one. So, we want to go to API. We want to find the name of it, which is just this right here. Looks like we don't need to pass anything else that's too crazy. I think this is going to be okay. So, let's just go here and swap the model. Okay, so that's swapped in. Let's clear and now let's just run it with Recraft because I'm interested to see what we're going to get.

注意，这个模型只给出了宽高比。没有其他内容需要发送，而且速度明显快得多，成本也低得多。我们来看看封面图，得到的是这个。好吧，我的意思是，我们也许可以暂时继续使用 Recraft，或者换成另一个模型，因为我想要尽可能高质量的图像，之后我们还要用它来制作视频。那么我们试试……我也不确定。能不能选“最佳”？And notice for this one it just gives us the aspect ratio. There is nothing else to send and this was significantly faster and much cheaper. And if we go to the cover image, we get this. Okay, so I mean we can maybe keep using Recraft for now or let's switch to one other model just cuz I think I want the highest quality image I can get cuz we're then going to make a video from it. So let's go maybe I don't know. Can I do like best?

我们来看看有没有这个。好，我点了“最受欢迎”，看起来 Nano Banana 是其中最受欢迎的模型之一。那我们试着用一下这个，同样进入 API，找一下这个 Google Gemini 的名称。好，就是我们已经在用的那个。不管怎样，把它粘贴到那里，然后就可以了。好，第三步就完成了。现在我们进入第四步。Let's see if they have that. Okay, I've gone most popular and looks like Nano Banana is one of the most popular ones. So let's try to use this and same thing go to API and let's find the name of this Google Gemini. Okay, that's the one we're already using. So anyways, paste that there and then we're good to go. Okay, so that is step three finished up. Now let's move to step four.

好，那么第四步是一个_久久爱Okay, so step four is a

步骤 4：语音生成Step 4: Voice Generation

再额外补充一点内容，因为我们实际上不会在视频里用到这个。不过我想在这里做一段配音，只是想给你演示一下，如果你想这么做的话，如何生成语音。一般来说，你可以用 OpenRouter 为你的任何 AI 智能体生成语音。基本上，我们不会在视频里实际用到这个，只是想生成一段比较长的语音稿。我们可以更换 AI 模型，试着调整语音风格，明白我的意思吧。little bit more of a bonus because we're not actually going to use this for the video, but what I want to have here is a voice over just to show you how you can generate voice um if you want to do that, you know, using OpenRouter just generally speaking for any of your AI agents. So basically we're not actually going to use this in the video, but we're just going to generate kind of like a long voice transcript. We can change the AI model, play around with the voice, you get the idea.

为了做到这一点，我们来看看这个脚本。这里用到了 NumPy、soundfile、JSON 和 helper。注意，我们从 helper 中引入了 lookup cost 函数。如果回到这里看一下，你会发现它只是查询费用，因为有些语音模型不会直接给出费用，所以需要通过查询来获取。In order to do that, we're going to go through well, this script right here. So what we have is NumPy, soundfile, JSON, and then helper. From helper, notice we're bringing in the lookup cost function. If we go back to that here, you can see this just looks up the cost because for some of the voice models it's uh doesn't directly give you the cost and to look it up.

这次我们要做的就是向 /audio/speech 发送一个 post 请求。然后指定模型、我们实际想让它说的文本，以及语音。这里有很多不同的语音。你可以在 Gemini 3.1 Flash TTS Preview Text-to-Speech 的文档中查看。拿到响应后，我们会将其转换成一个对象，这样就能将它保存为音频文件。So what we're going to do this time is just send a post request to {slash} audio {slash} speech. We then are going to specify the model, the script that we actually want it to say, and then the voice. There's a bunch of different voices. see them in the docs here for Gemini 3.1 Flash TTS Preview Text-to-Speech. Once we get the response here, we're going to convert this into an object that we can actually save as a sound file.

我们只要把耗时加上，这样就能看到它到底花了多长时间，然后就可以直接在这个输出文件夹里查看了。再说一次，这算是额外内容。我不确定你们有多少人真的想做这个，只是想演示一下该怎么生成它。接下来我只要输入 `UV run step four.py`。我们来看看这里的输出，看看它会给我们什么结果。We're just going to add the cost so we can see how much that did cost, and then we'll be able to just view it from this output folder. Again, this is kind of just more of a bonus. I'm not sure how many of you guys actually want to do this. Just wanted to show you this is how you can generate it. So, what I'll do is just type UV run step four.py. And let's have a look at the output here and see what it gives us.

同样，这个需要稍微多等一会儿，因为它需要生成语音，数据量比基本脚本大得多。所以，我们点这里的“语音”，稍等一下。好了，生成了 14 秒。我们播放看看效果。Again, this one will take a little bit longer just because it needs to generate voice, which is a lot more data than a basic script. So, if we go here to voice, let's give this a second. Okay, we've got 14 seconds. Let's play it and see what we get.

嗨，我只是做个快速测试。你能告诉我这个转录功能是否正常吗？我们来看看它能不能运行。我正在说几句简单的话，检查这些文字是否能正确显示在屏幕上。只是做个小测试，看看效果如何。Hi, I'm just doing a quick test. Can you tell me if this transcription is working? Let's see if it functions. I'm speaking a few simple sentences and checking whether the words appear correctly on screen. Just a small test to see how it goes.

很好。你可以看到，我们已经拿到了测试脚本，而且一切都在正常运行。现在，让我们进入更有趣的部分：使用第五步生成视频。Nice. And you can see that we get the testing script there and everything is working. Now, let's move on to the more interesting part where we generate a video with step five.

步骤 5：视频生成Step 5: Video Generation

好，这肯定是视频里最复杂的部分，毕竟——无意双关——制作视频比制作图像或其他任何内容都复杂。不过我来给你演示它的工作原理。首先，我们引入所需的模块。注意，这次我们直接引入 add cost、post 和 headers。这是因为我们必须发送几个不同的 API 请求。模型方面，我们将使用 Google VO，呃，3.1。Okay, so this is definitely the most complicated part of the video just because, well, no pun intended, making a video is more complex than making an image or anything else. But let me show you how it works. So, we bring in our imports. Notice this time we're directly bringing in add cost, post, and headers. Uh that's because we have to send a few different API requests. For the model, we're going to use Google VO uh 3.1.

你可以再次使用任何你想用的东西。现在，我们有了这个 make video 函数，这是我们需要创建的主要部分。我们传入一个脚本，然后传入我们创建的原始图像的路径。基本上，我们要做的是从一张图像开始，然后根据它生成视频。你不一定非要这么做，但我发现这样总能得到更好的效果，而且结果也会更加一致。You can use again anything that you want. Now, we have this make video function, which is the bulk of what we need to create. We take in a script and then the path to the original image that we created. Now, basically, what we're going to do here is we're going to start with an image and then generate a video from that. Now, you don't need to do that, but I find this always just gives better results and can be a little bit more consistent.

所以，我所做的就是加载了我们文件夹里的这张图片。我们会说：“好，打开图片后，我们要先对它进行编码，然后再解码。”我知道我们这样写有点奇怪，但必须这样加载，才能把它传递给 Open Router。接着我们说：“嘿，我们要向这个 {slash} videos 端点发送一个 POST 请求。模型就是这里这个。”So, what I've done is I've loaded the image that we had in our folder here. So, we say, "Okay, we're going to encode this image once we open it up and then decode it." I know it's a little bit weird how we're writing this, but that's how we need to load it in order to be able to pass it here to Open Router. We then said, "Hey, we're going to send a post request to this {slash} videos endpoint. The model is going to be this right here.

提示词就是视频提示词。时长为 8 秒。当然，如果你想制作更长的视频，就必须使用其他模型。分辨率为 720p。你可以调高分辨率，但显然成本会更高，耗时也会更长。然后是帧图像。我们就是通过它来指定视频从哪里开始，以及说明视频将从一张图像开始。The prompt is just the video prompt. The duration is 8 seconds. Of course, if you want to make a longer one, you have to use a different model. The resolution is going to be 720p. You can go up, but obviously that's going to be more expensive and take longer. And then we have frame images. Now, this is how we actually specify where we want to start the video from and the fact that we're going to start it from an image.

现在，如果你不想从图像开始，可以直接把这一项完全删除，因为它是可选的。但在我们的例子中，我们要说：“嘿，我们希望有一个图像 URL 类型。”接下来会指定图像 URL。这个 URL 就是这一项加上图像数据。这样，它就知道：“嘿，这就是我实际想要作为起点的图像。”你也可以传入一个它能够加载的真实 URL。不过在我们的例子中，我们只是给它一个本地文件。Now, if you don't want to start it from an image, you can just remove this entirely because it is optional. But in our case, we're saying, "Hey, we want to have type image URL." We're going to specify our image URL. The URL is just simply this plus the image data. So, it knows, "Hey, this is the image I actually want to start from." You also can pass a real URL that it can load. But in our case, we're just giving it like a local file.

然后我们有帧类型，并指定第一帧。你也可以指定最后一帧。我记得这里还有其他几个选项，可以用来指定你希望如何生成视频。接着我们会从这里获取 JSON。视频生成时，基本上会返回一个轮询响应，这意味着我们需要持续轮询，或者不断发送请求，来查看视频是否已经生成完成。Then we have frame type and we say first frame. You also could do like last frame. I believe there's a few other options here to specify how you want to generate the video. Then we get the JSON from this. Now, basically what will happen with the video is it will give us back kind of a polling response, which means that we're going to need to keep polling or keep sending a request to see if the video is finished generating.

这是因为视频可能需要一两分钟才能生成。因此，不像其他端点那样立即返回响应，它会先返回一个状态。状态可能是“等待中”“进行中”或“已完成”。如果状态是“等待中”或“进行中”，我们就等待 15 秒，然后再发送一次请求，询问：“嘿，现在可以用了么？”如果状态变为可用，我们就可以获取视频、下载它，然后保存下来。That's because the video can take like 1 or 2 minutes to be created. So, rather than instantly giving us the response like we would get with the other endpoints, it's going to give us a status. The status will be pending or in progress or completed. If it's pending or in progress, we are just going to sleep for 15 seconds and then send another request and say, "Hey, is this available?" If the status becomes available, then what we can do is actually get the video, download it, and then save it.

不过，我们还是逐步来看这些步骤。基本上，这里会有一个轮询 URL。响应会在这里把这个轮询 URL 提供给我们。通过这个 URL，我们可以持续获取视频是否已创建完成的状态。视频一旦创建完成，我们就进入第三步，向未签名 URL 再发送一个请求。However, let's look at those steps one by one. So, basically there is a polling URL. The polling URL will be provided to us from the response right here. And that's the URL that will continue to give us the status on whether or not the video is created. As soon as the video is actually finished being created, right, we move down to step three, where what we're going to do is send another request to the unsigned URL.

未签名的 URL 就是可以获取视频的 URL。然后我们需要再次传入 API 密钥，接着就能获取内容，也就是整个视频的内容，对吧？比如说字节数据。然后我们要做的是：“好，我们要创建一个新的字节类型文件，把视频内容写进去，然后只要说明费用是多少，再返回 clip.mp4。”如果我们实际触发这个操作，就可以直接加载脚本了，对吧？The unsigned URL is the URL where the video is available. We then again need to pass our API key, and we then get the content, which will be all of the video content, right? Like the bytes. Then what we do is we say, "Okay, we're going to create a new file that is a bytes type, and we're going to write the video content inside of there, and then we can just say how much it cost, and then return clip.mp4." Then if we were to actually trigger this, we can just load the script, right?

所以，从我们的文件中，然后运行 make video。它会完成所有这些步骤，发送请求，等待处理完成，下载视频，然后我们就可以查看它了。我知道我们现在只是在做测试示例，但还是让我们完整地走一遍，看看它是否真的能运行，然后我们就可以创建完整的流水线，多次运行它。So, from our file, and then run make video. It will go through all of this, send the request, wait for it to finish, download the video, and then we can view it. So, I know we're just doing the test example here, but let's just go through and see if this actually works, and then we can create the full pipeline where we can run this multiple times.

那么，我们来运行 uv run，然后这就是第五步，还是我把它叫作什么来着？第四步？啊，第五步放错文件夹了。我们把它移到正确的位置。所以，step five.py，我们等大约一分钟，希望它能生成这段视频。好了，不知道为什么，我刚才收到一条错误消息，说它预期的是 JPEG 或 PNG 之类的格式。我觉得报错的原因是我们使用了 Recraft 模型运行，它生成的图像类型可能和我们预期的不一样。So, let's go UV run, and then this is going to be step five, or what did I call this? Step four? Ah, step five is in the wrong folder. So, let's move that out into the correct place. So, step five.py, and we'll wait about a minute, and hopefully it will generate this video. Okay, now for some reason I just got an error saying that expected like a JPEG or PNG or something. I think the reason why we got that error is because we ran with the Recraft model, and that may have generated different type of image than we were expecting.

所以，我刚才重新运行了步骤三，只是为了重新生成封面图片。现在你可以看到我们得到的是这个，对吧？和之前的结果稍微有些不同。接下来我会重新运行步骤五，希望这次能成功。我还是不确定为什么会出现那个错误，但提示说它预期的是 JPEG 或 PNG，而我能想到的唯一原因是 Recraft 给我们的图片格式稍微有些不同。So, what I've done is I've just rerun step three here just to regenerate the cover image. So, you can see now we get this, right? Again, it's a little bit different than what we had before. So, now I'll rerun step five, and hopefully this is going to work. Again, I'm not sure why we got that error, but it was saying he expected JPEG or PNG, and the only thing I can imagine is because Recraft gave us an image in a slightly different format.

好，这次成功了。还是一样，3.20 美元。现在我们打开 clip.mp4 看看。Okay, so this time it worked. Same thing, $3.20, and now if we go to clip.mp4, let's look at it.

你能告诉我这个转录功能是否正常吗？Can you tell me if this transcription is working?

我们来看看它是否能正常运行。Let's see if it functions.

所以，有点奇怪。显然它并不完美，但你知道，我们得到了一段视频。这就是我们想要的。现在我们可以把所有这些步骤合并到一个流程中，这样我们就能直接对着电脑说话。So, a little bit weird. Obviously, it's not perfect, but you know, we got a video. That's what we were looking for. And now what we can do is combine all of these steps into one pipeline so that we can just speak to the computer.

最终流水线Final Pipeline

一切都会自动处理，然后我们最终得到结果，也就是一段视频。那么，我们新建一个文件吧。就在这个名为 main 的主文件夹中创建。其实，我们已经有 main.py 了。那么，我们打开 main.py。让我先删掉另一个我本来不想创建的文件。把这里的内容都清空，然后粘贴以下内容。好在所有内容都已经写好了。Everything is handled automatically, and then we end up getting the final result, which is a video. So, let's go and make a new file. Just make it in this main folder called main. Actually, we already have main.py. So, let's go inside of main.py. Let me just delete this other one I didn't mean to create. Let's clear everything out of here, and let's paste the following. The beautiful thing is that everything is already written.

我只需要在这里调整一下导入内容。也就是说，我们只需从步骤一、步骤二、步骤三、步骤四和步骤五中导入所有内容，同时从辅助函数或辅助文件中导入内容。然后，我们可以把这些步骤放到一个流水线中，将上一步的结果传递给下一步，再添加这个用于打印所有成本的函数，这样整个流水线就写好了。I just need to adjust the imports here. So, effectively, we're just going to import everything from step one, step two, step three, step four, and step five, as well as from our helper function or our helper file, and then we can just put these in a pipeline where we pass the result of the previous step to the next step, and we just add this one function to print all of the cost, and we have the pipeline fully written.

所以，正如你所看到的，我刚刚导入了所有函数。So, as you can see, I just imported all the functions.

所以，先录制，第二步进行转录，第三步编写脚本，然后制作图像。接着，我们有录制的语音备忘录；有转录语音备忘录得到的文本；有根据提示词或文本编写的脚本；有根据脚本制作的图像，或者说，抱歉，是根据脚本制作图像；还有根据脚本制作的配音音频。So, record, transcribe from step two, write script step three, make image. Then, we have voice note where we're recording. We have our text, which is transcribing the voice note. We have our script, which is writing the script based on the prompt or the text. We have the image, which is making the script or sorry, making the image based on the script. We have the audio, which is making the voice over based on the script.

然后我们有视频这一步，它会根据脚本和我们生成的图像制作视频。就是这样。接着我们打印成本收据，也就是汇总这里从 helper 导入的所有成本，并显示总金额。那么，现在运行这个流水线，试试独特的内容。运行 UV run main.py。然后按回车。一个骑着独角兽的牛仔，在火星上抓住一头牛。And then we have the video, which is making the video based on the script and the image that we generated. That's it, and then we print the cost receipt, which is just going through all of the costs here that we imported from helper and just showing us the full value. So, let's run this pipeline now and try something unique. So, UV run main.py. And let's press enter. A cowboy riding on a unicorn on Mars that catches a cow.

我不知道这能不能给我们带来什么可用的东西，但应该会挺有意思的。好了，转录已经完成。这甚至连一分钱的千分之一都没花，简直不可思议。接下来，它会进入下一步。它并不是把所有内容都打印出来，所以你当然可以把这部分做得更好，然后我们就会生成视频。完成后，我们来看看得到的输出。同时，在视频生成的过程中，我们可以听听旁白。来看看这个是什么。I don't know if that's going to give us anything usable, but that's probably going to be interesting. So, it's done the transcription. That didn't even cost us 1/1000 of a cent, which is just insane. Then, it's going to go on to the next step. It's not really like printing out everything, so you of course could make this better, and then we'll generate the video. Once it's done, let's see the output that we got. And at the meantime, while it generates that video, we can listen to the voice over. Let's see what this one was.

真实故事。嗯，差不多吧。一名牛仔骑着一只闪闪发光的独角兽，穿越火星的红色平原。他的任务是什么？抓住一只逃跑的太空猫。好了，趁着它生成的时候，我们确实遇到了一个我以前从没见过的奇怪错误。提示说，由于第三方内容提供方的利益，无法提交提示词。我觉得这是因为它生成了伍迪，而我当时正试图用他来制作视频，这可能违反了版权之类的规定。True story. Well, almost. A cowboy rides a shimmering unicorn across the red plains of Mars. His mission? Catch one runaway space cat. Okay, and while that's running, we actually got a weird error that I've never seen before. It says the prompt could not be submitted due to the interest of third-party content providers. And I believe that's because it generated Woody here, and then I was trying to create a video with him, which is I guess maybe against like copyright rules or something.

我真是从来没见过这种情况。嗯，不过碰到这种情况倒也挺有意思的。那么，我们试试换个新提示词，看看会不会生成别的东西。一名野蛮人骑着特洛伊木马，在一座古希腊城市里射击一只猎鹰。不要使用任何受版权保护的内容。好，看看这次会生成什么。I've literally never seen that before. Um but I guess that's an interesting one to run into. So, let's try maybe a new prompt and see if we get something else. A barbarian riding a Trojan horse in an ancient Greek city shooting a falcon. Don't use any copyright content. Okay, let's see this time what we get.

我觉得，这张图的效果会很有意思。那我们来看看结果吧。好了，刚刚生成完。我们可以在这里看到费用。生成的封面图看起来挺酷的。接下来看看视频。我还没看过呢。在哪儿？clip.mp4。来看看生成了什么。我的意思是，挺有意思的，虽然算不上特别逼真，但你知道，根据我描述的内容和传入的提示词来看，效果还是挺不错的。That's going to be very interesting, I think, for that image. So, let's see the result. All right, so it just finished. We can see the cost down here. We get the cover image, which looks pretty cool. And let's go to the video. I haven't even seen this yet. Where is it? clip.mp4. Let's see what we get. I mean, interesting, not the most realistic thing in the world, but, you know, pretty cool based on what I described it to do and the prompt that it passed.

好了，大家，代码部分差不多就讲完了。最后，如果你们想查看所有费用，这也是我喜欢 Open Router 的原因之一。回到仪表盘，就能看到全部支出。我觉得目前因为用的是同一个项目，可能已经花了 15 美元左右，基本都花在视频生成模型上了。当然，你完全可以把这项费用降得更低。So, there you go, guys. That pretty much wraps up the code. Last thing, if you do want to see, obviously, all of the costs, this is one of the reasons I like Open Router. If we go back to the dashboard, you can see all of the spend. I think so far cuz I'm using like the same project, maybe I've spent 15 bucks or something, pretty much all in the video generation model, which, of course, you can make significantly cheaper.

你可以看到所有请求和所有令牌。一切都整理得非常清楚，这点我很喜欢，而且只需要一个 API 密钥，我可以根据需要随时切换、删除或替换它。好了，朋友们，今天就到这里。所有代码都可以通过简介中的链接获取。如果你喜欢这个视频，欢迎点赞、订阅，我们下期再见。You can see all of the requests, all of your tokens. It's all sorted out really nicely for you, which I really like with a single API key that I can swap, delete, replace, whatever I need to do. Anyways, guys, I will wrap it up here. All the code will be available from the link in the description. If you enjoyed, leave a like, subscribe, and I will see you in the next one.