空间推理

Gemini Robotics ER 模型可以指向物体,在视频中跟踪物体,用边界框检测物体,并生成运动轨迹。

如需查看完整的可运行代码,请参阅机器人技术食谱

指向对象

以下示例用于查找图片中的特定对象并返回其归一化的 [y, x] 坐标:

Python

from google import genai

PROMPT = """
          Point to no more than 10 items in the image. The label returned
          should be an identifying name for the object detected.
          The answer should follow the json format: [{"point": <point>,
          "label": <label1>}, ...]. The points are in [y, x] format
          normalized to 0-1000.
        """
client = genai.Client()

uploaded_file = client.files.upload(file="my-image.png")

image_response = client.interactions.create(
    model="gemini-robotics-er-2-preview",
    input=[
        {
            "type": "image",
            "uri": uploaded_file.uri,
            "mime_type": uploaded_file.mime_type
        },
        {"type": "text", "text": PROMPT}
    ],
    generation_config={"thinking_level": "high"},
)

print