Skip to main content

Camera → OCR

Photograph a receipt, a document, a whiteboard, a page of handwriting — and get its text back, filed next to the image, without opening anything.

The flow is image lands in storage → text comes out beside it. How the photo gets into storage is a detail you choose; the OCR half is the same either way.

The text lands as a sibling file, so the image and its contents stay together and both get picked up by whatever indexes that folder next:

receipts/2026-08-14_grocery.jpg
receipts/2026-08-14_grocery.jpg.ocr.txt   ← created automatically

Getting the photo in​

Anything that writes to a watched bucket triggers the flow. Pick whichever you'll actually use — the OCR pipeline does not care which one you chose, and you can run several at once.

RouteGood forSetup
Nextcloud auto-uploadThe default. Your phone's camera roll syncs, and every photo is OCR'd. No bot, no token.Point the Nextcloud mobile app's auto-upload at a folder backed by SeaweedFS
A Telegram botDeliberate capture — you choose what gets sent, rather than everything you photographOption B
rclone / a scriptScanners, batch imports, anything already on diskrclone copyto file.jpg s3:bucket/path.jpg
Any S3 clientOther apps writing to the bucket directlyNothing — the bucket event is the trigger
Start with auto-upload

If you already run Nextcloud, camera auto-upload is the shortest path to this flow working: photograph something, and the text exists a few seconds later. The Telegram route is worth it when you want a deliberate inbox rather than your whole camera roll.

How It Works​

1. The image lands in a watched bucket​

However it got there — camera sync, a bot, rclone, another app — the flow starts the moment the object exists. Nothing polls the image itself.

2. SeaweedFS fires the webhook​

When the image file lands in the bucket, an ObjectCreated event is POSTed to the Decree webhook endpoint. This is the same pipeline used for any file arriving in a watched bucket.

3. s3-router matches image extensions​

s3-router scans the ollama-ocr processor and reads its pattern:

PATTERN='\.(jpg|jpeg|png|webp|gif|heic|heif|tiff?|bmp)$'
IS_PRE_SIGNED=false

A match enqueues a file-processor message with is_pre_signed: false — the file will be downloaded locally.

4. file-processor downloads the image​

file-processor runs rclone copyto to pull the image into a temp path and exports it as FILE_PATH. No pre-signed URL is involved — the full image bytes are needed to encode as base64 for the vision API.

5. ollama-ocr extracts the text​

ollama-ocr.sh invokes ocr.ts, which:

  1. Reads the image from FILE_PATH and encodes it as base64
  2. POSTs to http://ollama:11434/api/generate with the configured vision model and an extraction prompt
  3. Writes the returned text to stdout

The shell processor pipes that output via rclone rcat to Nextcloud at the same path with .ocr.txt appended:

telegram/1745000000_AgADjk.jpg
telegram/1745000000_AgADjk.jpg.ocr.txt   ← created automatically

Prerequisites​

  • SeaweedFS receiving file events and forwarding them to the Decree webhook (see File Processor)
  • Nextcloud + SeaweedFS configured with SeaweedFS as S3 external storage (so images land in both)
  • Ollama running with a vision-capable model pulled (e.g. llava, llava-phi3, moondream)
  • rclone configured with a nextcloud remote

Nothing on that list is specific to how you take the picture.

Setup​

Step 1 — Pull a vision model in Ollama​

docker exec ollama ollama pull llava

Any Ollama-compatible vision model works. llava is a solid general-purpose choice; llava-phi3 is faster on CPU.

Step 2 — object-store webhook and rclone​

Follow the SeaweedFS setup in File Processor to add your image path to path_prefixes in notification.toml (already done for the nextcloud bucket), and ensure your rclone nextcloud remote is configured:

./existential.sh run rclone

Step 3 — Point a camera at it​

That is the whole OCR pipeline. Now pick how images arrive.

Option A — Nextcloud camera auto-upload​

In the Nextcloud mobile app, turn on Auto upload and target a folder that lives on the SeaweedFS-backed external storage. Every photo you take syncs up and gets OCR'd. There is nothing else to configure — the bucket event is the trigger.

Option B — Telegram bot​

Use this when you want a deliberate inbox instead of your whole camera roll: you photograph something and choose to send it.

Create the bot via @BotFather with /newbot and copy the token, then:

mkdir -p automation/secrets/telegram
cat > automation/secrets/telegram/credentials.env << 'EOF'
TELEGRAM_BOT_TOKEN=your-token-here
EOF

The secrets directory is bind-mounted into the decree container at /secrets/telegram/.

Enable telegram-ingest in services/automation/decree/config.yml (the rendered, gitignored config the daemon actually reads — not automation/config.yml, which is an empty Docker-created mountpoint placeholder and has no effect):

shared_routines:
  telegram-ingest:
    enabled: true

Copy and activate the example cron:

cp automation-examples/cron/telegram-poll.md \
   automation/cron/

telegram-ingest polls getUpdates every minute, tracks a cursor in /secrets/telegram/offset.txt so nothing is processed twice, and pipes each new photo through rclone rcat into TELEGRAM_RCLONE_DEST. From there it is an ordinary bucket event and the flow above takes over. Cron frontmatter is read when the daemon starts, so restart it to pick the new schedule up:

docker compose restart automation

Option C — rclone​

For scanners, batch imports, or anything already on disk:

rclone copyto /path/to/scan.jpg s3:documents/scan.jpg \
  --config automation/secrets/rclone/rclone.conf

Customization​

VariableDefaultDescription
TELEGRAM_RCLONE_DESTnextcloud:S3/telegramTelegram route only — where downloaded images are stored (triggers the OCR pipeline)
FILE_SUFFIX.ocr.txtSuffix appended to the image path for the OCR output
OUTPUT_RCLONEnextcloudrclone remote where the OCR result is saved
OCR_MODELEXIST_MODEL_VISIONOllama vision model used for text extraction. Chosen globally in .env.shared, not per-flow.
OLLAMA_URLhttp://ollama:11434Ollama API base URL

Testing​

Drop a test image straight into the bucket to bypass whichever capture route you chose and verify the OCR pipeline on its own:

rclone copyto /path/to/test.jpg nextcloud:S3/telegram/test.jpg \
  --config automation/secrets/rclone/rclone.conf

Then send a synthetic file event:

# automation-webhook publishes no host port — it is reached over the exist
# bridge, so send the event from a container already on it.
docker exec automation curl -X POST http://automation-webhook:8801/s3 \
  -H "Authorization: Bearer <EXIST_DECREE_S3_WEBHOOK_AUTH_TOKEN from .env.shared>" \
  -H "Content-Type: application/json" \
  -d '{"EventName":"s3:ObjectCreated:Put","Key":"telegram/test.jpg","Records":[]}'

Watch the run complete:

docker logs -f automation
docker exec automation decree status
docker exec automation decree log <id-prefix>

Reusing the OCR function​

ocr.ts is a standalone library importable in any other TypeScript routine:

import { ocr } from "/work/.decree/lib/ocr";

const text = await ocr("/tmp/some-file.png", {
  model: "llava-phi3",
  prompt: "What does this receipt total?",
});