
Let Claude watch videos: timestamped transcript, scene-selected keyframes, exact frames and OCR.
Let Claude watch videos: timestamped transcript, scene-selected keyframes, exact frames and OCR. Local, no API keys. MCP server + skill + Claude Code plugin.
·
Let Claude (or any MCP client) watch a video. Give it a local file or a URL (YouTube and 1,500+ other sites via yt-dlp) and it gets a timestamped transcript plus a handful of scene-selected keyframes, then drills into exact frames only when needed. No cloud transcription, no API keys.
> Watch https://www.youtube.com/watch?v=jNQXAC9IVRw and tell me what he says about the elephants.
$ video-cli overview https://www.youtube.com/watch?v=jNQXAC9IVRw --max-frames 8
# Me at the zoo
duration: 00:19
transcript: platform-subs:en, language=en, 6 segments, took 0.4s
keyframes: 3 scene-selected at 00:00, 00:05, 00:11
total time: 6.7s
## Transcript
[00:01] All right, so here we are, in front of the elephants
[00:05] the cool thing about these guys is that they have really... really really long trunks and that's cool
[00:16] and that's pretty much all there is to say
IMAGE: ~/.cache/video_mcp/Youtube_jNQXAC9IVRw/sheets_8/sheet_01.jpg
Claude: He stands in front of the elephants (00:01) and says the cool thing about them is
their really long trunks (00:05). That is the whole video (00:16).| watch-video | Cloud video APIs (e.g. Gemini) | Transcript-only tools | |
|---|---|---|---|
| Sees the screen (code, slides, charts, UI) | Yes: keyframes, hi-res frames, OCR | Yes | No |
| Runs locally, nothing uploaded | Yes | No: the video goes to the provider | Varies |
| API key or account | None | Required, billed per use | Varies |
| Videos without captions | Local Parakeet / Whisper | Yes | Often fails |
| Token cost of the visual pass | ~20 frames per contact sheet (cost of ~1.5 images) | Billed by video length | None (no visuals) |
| Private recordings (meetings, bug repros) | Safe: never leave the machine | Uploaded | Varies |
It ships as two parts:
video MCP server with seven tools (below).watch-video skill that teaches Claude the cheap-first workflow: overview, then frames, then ranges.| Step | What happens |
|---|---|
| Transcript | Platform subtitles if the video has them (seconds), else Parakeet TDT 0.6B v3 (int8, CPU, 25 European languages), else faster-whisper large-v3-turbo (other languages, GPU or language="auto"). |
| Keyframes | Decodes only keyframes at thumbnail size, drops repeats and static slides, then fills the largest time gaps. The selection adapts to how much the video changes (screencast vs. action). |
| Contact sheets | Labelled frames tiled into grids: about 20 frames for the token cost of 1.5 full frames. Black bars are cropped automatically. |
| Drill-down | get_frames seeks to exact timestamps at 1280 px (readable code and charts). For URLs it seeks into the stream without downloading the whole video. |
Everything is cached under ~/.cache/video_mcp, so repeat calls are instant.
Measured on a mid-range CPU (i5-11400, no GPU): 35.7-minute local video overview in about 2.6 minutes; YouTube video with subtitles in about 7 seconds; a drill-down frame in about 0.5 seconds.
Requirements: Python 3.10+ and ffmpeg (including ffprobe) on your PATH.
Installs the MCP server and the skill together. Needs uv for uvx.
/plugin marketplace add marwanm7moud/watch-video
/plugin install watch-video@watch-videopipx install watch-video-mcp # or: uv tool install watch-video-mcp / pip install watch-video-mcp
# optional extras: Whisper (non-European languages / GPU) and OCR (on-screen text as text)
pipx install "watch-video-mcp[whisper,ocr]"
# latest from GitHub instead of PyPI:
pip install git+https://github.com/marwanm7moud/watch-videoNo install needed to try the CLI: uvx --from watch-video-mcp video-cli hw.
Install ffmpeg with winget install Gyan.FFmpeg (Windows), brew install ffmpeg (macOS) or
sudo apt install ffmpeg (Linux). The first transcription downloads the Parakeet model (about 650 MB).
Claude Code:
claude mcp add video --scope user -- video-mcpClaude Desktop (claude_desktop_config.json) or any other MCP client:
{ "mcpServers": { "video": { "command": "video-mcp" } } }If video-mcp is not on your PATH, use the full path to the executable or python -m video_mcp.server.
The skill works with or without the MCP server: it falls back to video-cli and installs it on first use.
Copy the skill folder so your agent knows when and how to use the tools:
# macOS / Linux
mkdir -p ~/.claude/skills && cp -r skills/watch-video ~/.claude/skills/
# Windows (PowerShell)
Copy-Item -Recurse skills\watch-video $HOME\.claude\skills\Then ask Claude: "Watch https://youtu.be/... and summarize it", or "At what point do they explain the stop-loss, and what does the chart show then?"
| Tool | Use |
|---|---|
video_overview(source, max_frames=40, ocr=False) | Start here: metadata, chapters, full transcript, contact sheets. ocr=True adds on-screen text. |
get_frames(source, timestamps, hi_res=True) | Exact frames ("12:30", "1:02:05" or seconds), up to 20. |
scan_range(source, start, end, count=12) | Evenly spaced frames in one sheet, to follow motion or a demo. |
find_cuts(source, start, end, threshold=0.2) | Hard cuts (shot or screen changes) and cuts per minute, for pacing. |
get_transcript(source, start, end) | Transcript slice. |
search_transcript(source, query) | Timestamped hits for any of the words, exact phrase first; plus on-screen text after an ocr=True overview. |
check_hardware() | Detected tools, GPU and the transcription profile in use. |
The same operations work without MCP:
video-cli overview video.mp4 --max-frames 16
video-cli overview demo.mp4 --ocr # also read code/slides on screen
video-cli frames https://youtu.be/... 12:30 13:05
video-cli range video.mp4 10:00 10:30 --count 12
video-cli cuts ad.mp4 # hard cuts + cuts per minute
video-cli search video.mp4 "stop loss"
video-cli hw| Variable | Default | Effect |
|---|---|---|
VIDEO_MCP_CACHE | ~/.cache/video_mcp | Cache directory (delete it to reclaim space). |
VIDEO_MCP_LANG | en | Default transcription language; auto detects it with Whisper. |
VIDEO_MCP_THREADS | 4 (CPU) | Threads for CPU transcription. |
VIDEO_MCP_AVX512 | unset | Set to 1 to let Whisper use AVX-512 (faster, but a very heavy sustained load). |
On CPU the server runs below normal priority, one heavy job at a time, with Whisper held to AVX2, so a long transcription does not freeze your desktop.
language. Parakeet covers 25 European languages on CPU (language="de",
"fr", "es", ...). Other languages (Arabic, Chinese, Japanese, Hindi, ...) need Whisper, which is slow on CPU.pip install -e ".[dev]"
ruff check .
pytestMIT. See LICENSE.
Pick your client and paste the snippet. Each one is the same server, written the way that client expects it.
claude mcp add video -- video-mcp{
"mcpServers": {
"video": {
"command": "video-mcp"
}
}
}code --add-mcp '{"name":"video","command":"video-mcp"}'[mcp_servers.video]
command = "video-mcp"Runs locally on your device. Your client starts the server itself, so nothing has to be hosted.
Paste this prompt into your agent. It reads this page and does the setup for you.
Read https://aiagentslisting.com/mcp/watch-video to learn what the "Watch Video" MCP server does and how to install it. Add it to my coding agent's MCP configuration as documented on that page, then confirm the server connects and list the tools it exposes.Agents can also browse this directory over MCP at https://aiagentslisting.com/api/mcp
claude mcp add video --scope user -- video-mcppipx install watch-video-mcp
pipx install "watch-video-mcp[whisper,ocr]"pip install git+https://github.com/marwanm7moud/watch-videoThis server runs locally, so we can't read its tool list over the web yet.
We could not run this server: it ships no npx package for our sandbox and no public endpoint. The four checks that need a live server are left out, and the grade tops out at B until one exists.
Maintain this server? Claiming proves you own the listing and earns the last 4 points.
Claim this listingPaste this line near the top of your repository's README. The badge always renders the current score.
[](https://aiagentslisting.com/mcp/watch-video)Nothing comparable is listed yet.
v0.3.0watch-video 0.3.0Release notesv0.2.0watch-video 0.2.0Release notesConnect to the AI Agents Listing MCP server at https://aiagentslisting.com/api/mcp using the HTTP transport and complete the sign-in when it asks. Then call the submit_review tool for the listing "watch-video" with a rating from 1 to 5 and a short note on what worked and what did not.No reviews yet
Be the first to share your experience with this tool.
One email a week. New agents, MCP servers and skills, and what is actually getting traction.