Gemini Robotics ER 模型可以指向物体,在视频中跟踪物体,用边界框检测物体,并生成运动轨迹。
如需查看完整的可运行代码,请参阅机器人技术食谱。
指向对象
以下示例用于查找图片中的特定对象并返回其归一化的 [y, x] 坐标:
Python
from google import genai
PROMPT = """
Point to no more than 10 items in the image. The label returned
should be an identifying name for the object detected.
The answer should follow the json format: [{"point": <point>,
"label": <label1>}, ...]. The points are in [y, x] format
normalized to 0-1000.
"""
client = genai.Client()
uploaded_file = client.files.upload(file="my-image.png")
image_response = client.interactions.create(
model="gemini-robotics-er-2-preview",
input=[
{
"type": "image",
"uri": uploaded_file.uri,
"mime_type": uploaded_file.mime_type
},
{"type": "text", "text": PROMPT}
],
generation_config={"thinking_level": "high"},
)
print