# Jetson Nano Image2Text API

Python 3.6-compatible Flask service that accepts an image and returns detected text plus optional EdgeTPU object context.

## Features
- Accepts JPEG/PNG/WebP/BMP/TIFF/GIF (first frame) via multipart or base64 JSON.
- Optional Google Coral EdgeTPU detection for scene context (loaded once at start).
- EAST text detector (OpenCV DNN) when model exists; otherwise OpenCV contour-based text-region fallback.
- OCR via `pytesseract` (default) with per-block confidences; configurable languages.
- Optional `PaddleOCR` backend (`OCR_BACKEND=paddle`) for better text-on-photo quality.
- Auto-selects `rus` or `eng` (via Tesseract OSD script detection) before full OCR when `OCR_LANG=eng+rus`.
- If OCR result is empty, retries with fallback language/full-frame pass to improve recall.
- Hard payload cap (10MB default) and graceful degradation if Coral/EAST unavailable.

## System prerequisites (Ubuntu 18.04 aarch64)
```bash
# EdgeTPU runtime (choose std or max)
sudo apt-get update
sudo apt-get install -y libedgetpu1-std python3-pycoral

# Tesseract OCR + languages
sudo apt-get install -y tesseract-ocr tesseract-ocr-eng tesseract-ocr-rus

# OpenCV (binary from apt is preferred on Jetson)
sudo apt-get install -y python3-opencv

# If you prefer pip OpenCV (may be large on Jetson):
# pip3 install opencv-python==4.5.3.56
```

## Python dependencies
```bash
pip3 install -r requirements.txt
```

Optional PaddleOCR backend (Python 3.8+ only):
```bash
pip3 install -r requirements-paddle.txt
```

## Models
Place your models in `./models` (paths can be overridden via env):
- EdgeTPU SSD: `ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite`
- Labels: `coco_labels.txt`
- EAST text detector (optional): `frozen_east_text_detection.pb`

Auto-download all models (Coral + EAST + Paddle + local `tessdata_best`):
```bash
./scripts/download_models.sh
```

Examples:
```bash
./scripts/download_models.sh --only-paddle
./scripts/download_models.sh --no-paddle
```

### Better OCR models (tessdata_best)
```bash
sudo apt-get install -y wget
sudo mkdir -p /usr/share/tesseract-ocr/4.00/tessdata_best
sudo wget -O /usr/share/tesseract-ocr/4.00/tessdata_best/eng.traineddata https://github.com/tesseract-ocr/tessdata_best/raw/main/eng.traineddata
sudo wget -O /usr/share/tesseract-ocr/4.00/tessdata_best/rus.traineddata https://github.com/tesseract-ocr/tessdata_best/raw/main/rus.traineddata
# then run the service with:
export TESSDATA_PREFIX=/usr/share/tesseract-ocr/4.00/tessdata_best
# if you skip this, the app will auto-try common system paths
```

### OpenCV with CUDA (for faster EAST)
OpenCV from JetPack repos (4.x) usually ships without CUDA dnn. To use CUDA, build OpenCV 4.5+ with `-D WITH_CUDA=ON -D OPENCV_DNN_CUDA=ON`. After installing, export `USE_CUDA_DNN=true` so the service enables CUDA backend/target when available.

## Environment variables
- `MODEL_EDGETPU_PATH` (default `./models/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite`)
- `MODEL_LABELS_PATH` (default `./models/coco_labels.txt`)
- `MODEL_EAST_PATH` (default `./models/frozen_east_text_detection.pb`)
- `OCR_BACKEND` (default `tesseract`; `tesseract|paddle|auto`)
- `OCR_LANG` (default `eng+rus`; request language set for OCR)
- `OCR_AUTO_LANG` (default `true`; auto-pick `eng` or `rus` by script when `OCR_LANG` contains both)
- `OCR_CONFIG` (default `--oem 1 --psm 6`, forwarded to Tesseract)
- `PADDLE_LANG` (default `ru`; used when `OCR_BACKEND=paddle`)
- `PADDLE_USE_GPU` (default `false`)
- `PADDLE_USE_ANGLE_CLS` (default `false`)
- `PADDLE_DET_MODEL_DIR` (default `./models/paddle/PP-OCRv5_mobile_det_infer`)
- `PADDLE_REC_MODEL_DIR` (default `./models/paddle/eslav_PP-OCRv5_mobile_rec_infer`)
- `PADDLE_CLS_MODEL_DIR` (default empty)
- `PADDLE_CPU_THREADS` (default `4`)
- `TESSDATA_PREFIX` (path to tessdata / tessdata_best if you install better models)
- `OCR_UPSCALE` (default `1.5`, upscale factor before OCR; max 3.0)
- `OCR_CLAHE` (default `true`, contrast-limited adaptive histogram)
- `OCR_BILATERAL` (default `false`, light denoise)
- `OCR_BLUR` (default `0`, Gaussian kernel size; 0 disables)
- `OCR_INVERT` (default `false`, invert after threshold)
- `OCR_USE_THRESHOLD` (default `true`, adaptive threshold)
- `MAX_IMAGE_MB` (default `10`)
- `CONF_THRESHOLD` (default `0.4`)
- `RETURN_BOXES` (default `true`)
- `USE_CUDA_DNN` (default `false`, set `true` to attempt CUDA backend for EAST if OpenCV built with CUDA)
- `DEBUG_VISUAL` (default `false`; if true, response includes `debug_image_base64`. Per-request: add `?debug=1` or form field `debug=1`.)
- `SHOW_GUI` (default `false`; if true and OpenCV GUI available, service opens a text-centric window: left = detected text regions, right = OCR block text/conf. Requires X/Wayland/VDI that supports GUI.)

## Run
```bash
python3 app.py --host 0.0.0.0 --port 8000
```
Waitress or gunicorn can also serve the app if desired (not required).

## API
- `GET /health` -> `{ "status": "ok", "coral": true/false, "paddle": true/false, "ocr_backend": "..." }`
- `POST /v1/image2text`
  - Multipart: field `image`
  - JSON: `{ "image_base64": "<base64>" }`

Example curl (multipart):
```bash
curl -X POST http://localhost:8000/v1/image2text \
  -F "image=@sample.jpg" \
  -F "debug=1" \
  | python -m json.tool
```

Example JSON response (fields may vary):
```json
{
  "text": "hello world",
  "blocks": [
    {"bbox": [10, 20, 200, 60], "text": "hello", "conf": 87.3},
    {"bbox": [10, 70, 200, 120], "text": "world", "conf": 90.1}
  ],
  "objects": [
    {"label": "person", "score": 0.63, "bbox": [5, 12, 180, 210]}
  ],
  "scene_summary": "person x1",
  "meta": {
    "latency_ms": 123,
    "coral_used": true,
    "ocr_backend": "tesseract",
    "east_used": true,
    "text_locator": "east",
    "ocr_lang": "eng",
    "ocr_lang_requested": "eng+rus",
    "ocr_lang_strategy": "auto_script",
    "ocr_passes": 1,
    "request_id": "..."
  }
}
```

## Notes
- If Coral is absent or fails, the service still returns OCR with `coral_used=false`.
- If `OCR_BACKEND=paddle` but PaddleOCR package/models are unavailable, service falls back to Tesseract.
- If EAST model is missing, service first tries OpenCV contour text regions and only then full-image OCR.
- Keep images under `MAX_IMAGE_MB` (413 returned otherwise).
- Optimize Tesseract languages to those you need to speed up OCR.