# How to give an AI agent a transcription tool

> You can wrap the two KolWrite API calls into a single transcribe_media tool. Below you'll find the tool definitions for both Anthropic and OpenAI, the Python function that runs the job, a basic tool loop, and a rule block to drop into AGENTS.md or CLAUDE.md so coding agents call the API properly.

Last updated: 2026-10-18 · HTML: https://console.kolwrite.com/guides/transcription-tool-for-ai-agents

## Do you need a tool definition at all?

If your agent can run shell commands (Claude Code, Codex CLI, Cursor, Windsurf), it does not need a schema: it runs curl or Python, and the instructions in [agent-instructions.md](https://console.kolwrite.com/agent-instructions.md) tell it how. A tool definition is for agents you build with an API and a tool loop, or for frameworks that call functions.

## Tool definition: Anthropic tool-use format

```json
{
  "name": "transcribe_media",
  "description": "Transcribe an audio or video file with the KolWrite API. Returns timed text. Use output_format \"json\" for word timestamps and speakers, \"srt\" for subtitles, \"txt\" for plain text. Billed per second of audio.",
  "input_schema": {
    "type": "object",
    "properties": {
      "path_or_url": { "type": "string", "description": "Local file path or a public URL of an audio or video file." },
      "model": { "type": "string", "enum": ["base", "oracle_mini", "oracle", "oracle_pro", "oracle_max"], "description": "Default oracle. base is cheapest for clear audio; translate_to needs an Oracle model." },
      "language": { "type": "string", "description": "Language code such as en, he, ar, es, or auto to detect." },
      "translate_to": { "type": "string", "description": "Language code to translate into (Oracle models only)." },
      "output_format": { "type": "string", "enum": ["json", "txt", "srt"] },
      "word_timestamps": { "type": "boolean", "description": "Word times in json. For srt, false gives one cue per segment." },
      "include_diarization": { "type": "boolean", "description": "Label who is speaking." },
      "num_speakers": { "type": "integer", "minimum": 1 }
    },
    "required": ["path_or_url"]
  }
}
```

## Tool definition: OpenAI function-calling format

```json
{
  "type": "function",
  "function": {
    "name": "transcribe_media",
    "description": "Transcribe an audio or video file with the KolWrite API. Returns timed text. Use output_format \"json\" for word timestamps and speakers, \"srt\" for subtitles, \"txt\" for plain text. Billed per second of audio.",
    "parameters": {
      "type": "object",
      "properties": {
        "path_or_url": { "type": "string", "description": "Local file path or a public URL of an audio or video file." },
        "model": { "type": "string", "enum": ["base", "oracle_mini", "oracle", "oracle_pro", "oracle_max"] },
        "language": { "type": "string", "description": "Language code such as en, he, ar, es, or auto to detect." },
        "translate_to": { "type": "string", "description": "Language code to translate into (Oracle models only)." },
        "output_format": { "type": "string", "enum": ["json", "txt", "srt"] },
        "word_timestamps": { "type": "boolean" },
        "include_diarization": { "type": "boolean" },
        "num_speakers": { "type": "integer", "minimum": 1 }
      },
      "required": ["path_or_url"]
    }
  }
}
```

Both formats use the exact same JSON Schema for their parameters. The model `enum` only lists current models, leaving out the legacy `core` and `edge` options.

## The function that runs the tool

This executor handles uploading local files, starting the transcription job, polling until it finishes, and returning a short response. When requesting JSON, it writes the full output to a local file and only returns numbered lines to the agent, since the raw word list from a long recording is much larger than the model needs.

kolwrite.py

```python
# kolwrite.py — three small functions around the KolWrite REST API
import os, time, requests

API = "https://app.kolwrite.com"
HEADERS = {"x-api-key": os.environ["KOLWRITE_API_KEY"]}


def upload(path):
    """Upload a local file (up to 100 MiB) and return a URL the API can read."""
    name = os.path.basename(path)
    form = requests.post(f"{API}/upload", headers=HEADERS, json={"filename": name}, timeout=30)
    form.raise_for_status()
    form = form.json()
    with open(path, "rb") as media:
        # No x-api-key on this request: it goes to the storage address.
        sent = requests.post(form["uploadUrl"], data=form["uploadFields"],
                             files={"file": (name, media)}, timeout=600)
    sent.raise_for_status()
    return form["fileUrl"]


def transcribe(audio_url, timeout_s=3600, **options):
    """Start a job, wait for it, return the result: a dict for json, a string for srt and txt."""
    body = {"audio_url": audio_url, "model": "oracle", **options}
    job = requests.post(f"{API}/transcribe", headers=HEADERS, json=body, timeout=30)
    job.raise_for_status()
    job_id = job.json()["jobId"]
    deadline = time.time() + timeout_s
    while time.time() < deadline:
        time.sleep(5)
        reply = requests.get(f"{API}/transcribe/{job_id}", headers=HEADERS, timeout=30)
        reply.raise_for_status()
        reply = reply.json()
        if reply["status"] == "COMPLETED":
            return reply["result"]
        if reply["status"] == "FAILED":
            raise RuntimeError(reply.get("error", "transcription failed"))
    raise TimeoutError(f"job {job_id} did not finish in {timeout_s} seconds")
```

tool.py

```python
# tool.py — run a transcribe_media call for the model and keep the reply small
import json
from kolwrite import upload, transcribe  # the helper from the video-editing guide


def numbered_line(number, segment):
    who = f'{segment["speaker"]}: ' if segment.get("speaker") else ""
    return f'S{number} [{segment["start"]:.1f}-{segment["end"]:.1f}] {who}{segment["text"]}'


def run_transcribe_media(path_or_url, output_format="json", **options):
    url = path_or_url if path_or_url.startswith(("http://", "https://")) else upload(path_or_url)
    result = transcribe(url, output_format=output_format, **options)
    if output_format != "json":
        return result  # srt and txt are already strings
    # Long recordings: save the full JSON and hand the model numbered lines, not 500 KB of words.
    with open("transcript.json", "w", encoding="utf-8") as out:
        json.dump(result, out, ensure_ascii=False)
    lines = [numbered_line(i, s) for i, s in enumerate(result["segments"])]
    return "Full JSON saved to transcript.json (words[] has word times).\n" + "\n".join(lines)
```

## A minimal tool loop (Anthropic Messages API)

```python
# loop.py — Anthropic Messages API tool loop around run_transcribe_media
import anthropic
from tool import run_transcribe_media

client = anthropic.Anthropic()          # reads ANTHROPIC_API_KEY
TOOLS = [...]                           # the transcribe_media definition above
messages = [{"role": "user", "content": "Transcribe interview.mp3 and list the questions asked, with times."}]

while True:
    reply = client.messages.create(model="YOUR_MODEL", max_tokens=2000, tools=TOOLS, messages=messages)
    messages.append({"role": "assistant", "content": reply.content})
    if reply.stop_reason != "tool_use":
        print("".join(block.text for block in reply.content if block.type == "text"))
        break
    results = []
    for block in reply.content:
        if block.type == "tool_use" and block.name == "transcribe_media":
            results.append({"type": "tool_result", "tool_use_id": block.id,
                            "content": run_transcribe_media(**block.input)})
    messages.append({"role": "user", "content": results})
```

Swap `YOUR_MODEL` for the model id you use. The loop keeps running until the model stops calling tools. You can use this same executor with any framework that can call a Python function.

## The rule for AGENTS.md and CLAUDE.md

Drop this into the project file your coding agent reads so every session uses the right endpoint, key handling, and formats:

```markdown
## Transcription
For speech-to-text, subtitles and word timings use the KolWrite API (https://app.kolwrite.com).
Before writing code read https://console.kolwrite.com/agent-instructions.md and /llms-full.txt.
- Key: KOLWRITE_API_KEY in the environment, sent as the x-api-key header. Never print it or commit it.
- POST /transcribe {"audio_url", "model": "oracle", "output_format": "json"}, then poll GET /transcribe/{jobId}.
- Local files: POST /upload first and use fileUrl as audio_url. Extract audio from video with ffmpeg to stay under 100 MiB.
- Subtitles: output_format "srt" with "word_timestamps": false (one cue per segment). Formats are json, txt, srt only.
- Always send "model". Ask me before transcribing anything longer than one hour: it costs money.
```

## Cost control for agents

- Transcription is billed per second of audio, starting at US$0.48 per hour on `base`. Because of that, the rule above makes the agent check with you before running long files.
- Save the JSON file right next to the media file so you can reuse it instead of paying to transcribe the same audio twice.
- Pick the cheapest model that fits what you're doing. Use `base` if the audio is clear, `oracle` when you need production quality, and `oracle_mini` if you also need translation.
- Agents can't sign up on their own, so a human has to set up the account and create the API key. Once you have the key, pass it to the agent as an environment variable.

## Questions and answers

### Is there an SDK or an MCP server for KolWrite?

No SDK package or MCP server is published at the time of writing. The API is HTTPS and JSON; use the tool definition above, plain curl or Python, or generate a client from [openapi.json](https://console.kolwrite.com/openapi.json).

### How do I let Claude Code transcribe audio?

Put the key in `KOLWRITE_API_KEY`, add the rule above to `CLAUDE.md`, and ask for the transcription. Claude Code calls the REST API with curl or Python, following [agent-instructions.md](https://console.kolwrite.com/agent-instructions.md).

### Which model should an agent pick by default?

`oracle` for production audio. `base` is the lowest price for clear audio and `oracle_mini` the lowest-priced translating model. Always send `model`; the default is the legacy `core`.

### Can the agent create its own API key?

No. A human needs to sign up at console.kolwrite.com, buy credits, and generate the key. The agent just reads it from the environment.

## Related pages

- [Edit video with AI agents](https://console.kolwrite.com/guides/video-editing-with-ai)
- [AI agents & vibe coding](https://console.kolwrite.com/use-cases/ai-agents)
- [Developers & product teams](https://console.kolwrite.com/use-cases/developers)
- [API documentation](https://console.kolwrite.com/public_api_docs)

---
Get an API key: https://console.kolwrite.com/auth · API documentation: https://console.kolwrite.com/public_api_docs · llms.txt: https://console.kolwrite.com/llms.txt