Jetson Nano Image2Text API

Python 3.6-compatible Flask service that accepts an image and returns detected text plus optional EdgeTPU object context.

Features

• Accepts JPEG/PNG/WebP/BMP/TIFF/GIF (first frame) via multipart or base64 JSON.
• Optional Google Coral EdgeTPU detection for scene context (loaded once at start).
• EAST text detector (OpenCV DNN) when model exists; otherwise OpenCV contour-based text-region fallback.
• OCR via pytesseract (default) with per-block confidences; configurable languages.
• Optional PaddleOCR backend (OCR_BACKEND=paddle) for better text-on-photo quality.
• Auto-selects rus or eng (via Tesseract OSD script detection) before full OCR when OCR_LANG=eng+rus.
• If OCR result is empty, retries with fallback language/full-frame pass to improve recall.
• Hard payload cap (10MB default) and graceful degradation if Coral/EAST unavailable.

System prerequisites (Ubuntu 18.04 aarch64)

# EdgeTPU runtime (choose std or max)
sudo apt-get update
sudo apt-get install -y libedgetpu1-std python3-pycoral

# Tesseract OCR + languages
sudo apt-get install -y tesseract-ocr tesseract-ocr-eng tesseract-ocr-rus

# OpenCV (binary from apt is preferred on Jetson)
sudo apt-get install -y python3-opencv

# If you prefer pip OpenCV (may be large on Jetson):
# pip3 install opencv-python==4.5.3.56

Python dependencies

pip3 install -r requirements.txt

Optional PaddleOCR backend for Python 3.6:

pip3 install -r requirements-paddle.txt

If you see No matching distribution found for paddleocr>=3.0.0, use the pinned 2.7.x requirement above (already set in this repo).

Models

Place your models in ./models (paths can be overridden via env):

• EdgeTPU SSD: ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite
• Labels: coco_labels.txt
• EAST text detector (optional): frozen_east_text_detection.pb

Auto-download all models (Coral + EAST + Paddle + local tessdata_best):

./scripts/download_models.sh

Examples:

./scripts/download_models.sh --only-paddle
./scripts/download_models.sh --no-paddle

Better OCR models (tessdata_best)

sudo apt-get install -y wget
sudo mkdir -p /usr/share/tesseract-ocr/4.00/tessdata_best
sudo wget -O /usr/share/tesseract-ocr/4.00/tessdata_best/eng.traineddata https://github.com/tesseract-ocr/tessdata_best/raw/main/eng.traineddata
sudo wget -O /usr/share/tesseract-ocr/4.00/tessdata_best/rus.traineddata https://github.com/tesseract-ocr/tessdata_best/raw/main/rus.traineddata
# then run the service with:
export TESSDATA_PREFIX=/usr/share/tesseract-ocr/4.00/tessdata_best
# if you skip this, the app will auto-try common system paths

OpenCV with CUDA (for faster EAST)

OpenCV from JetPack repos (4.x) usually ships without CUDA dnn. To use CUDA, build OpenCV 4.5+ with -D WITH_CUDA=ON -D OPENCV_DNN_CUDA=ON. After installing, export USE_CUDA_DNN=true so the service enables CUDA backend/target when available.

Environment variables

• MODEL_EDGETPU_PATH (default ./models/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite)
• MODEL_LABELS_PATH (default ./models/coco_labels.txt)
• MODEL_EAST_PATH (default ./models/frozen_east_text_detection.pb)
• OCR_BACKEND (default tesseract; tesseract|paddle|auto)
• OCR_LANG (default eng+rus; request language set for OCR)
• OCR_AUTO_LANG (default true; auto-pick eng or rus by script when OCR_LANG contains both)
• OCR_CONFIG (default --oem 1 --psm 6, forwarded to Tesseract)
• PADDLE_LANG (default ru; used when OCR_BACKEND=paddle)
• PADDLE_USE_GPU (default false)
• PADDLE_USE_ANGLE_CLS (default false)
• PADDLE_DET_MODEL_DIR (default ./models/paddle/PP-OCRv3_mobile_det_infer)
• PADDLE_REC_MODEL_DIR (default ./models/paddle/cyrillic_PP-OCRv3_mobile_rec_infer)
• PADDLE_CLS_MODEL_DIR (default empty)
• PADDLE_CPU_THREADS (default 4)
• TESSDATA_PREFIX (path to tessdata / tessdata_best if you install better models)
• OCR_UPSCALE (default 1.5, upscale factor before OCR; max 3.0)
• OCR_CLAHE (default true, contrast-limited adaptive histogram)
• OCR_BILATERAL (default false, light denoise)
• OCR_BLUR (default 0, Gaussian kernel size; 0 disables)
• OCR_INVERT (default false, invert after threshold)
• OCR_USE_THRESHOLD (default true, adaptive threshold)
• MAX_IMAGE_MB (default 10)
• CONF_THRESHOLD (default 0.4)
• RETURN_BOXES (default true)
• USE_CUDA_DNN (default false, set true to attempt CUDA backend for EAST if OpenCV built with CUDA)
• DEBUG_VISUAL (default false; if true, response includes debug_image_base64. Per-request: add ?debug=1 or form field debug=1.)
• SHOW_GUI (default false; if true and OpenCV GUI available, service opens a text-centric window: left = detected text regions, right = OCR block text/conf. Requires X/Wayland/VDI that supports GUI.)

Run

python3 app.py --host 0.0.0.0 --port 8000

Waitress or gunicorn can also serve the app if desired (not required).

API

• GET /health -> { "status": "ok", "coral": true/false, "paddle": true/false, "ocr_backend": "..." }
• POST /v1/image2text
  • Multipart: field image
  • JSON: { "image_base64": "<base64>" }

Example curl (multipart):

curl -X POST http://localhost:8000/v1/image2text \
  -F "image=@sample.jpg" \
  -F "debug=1" \
  | python -m json.tool

Example JSON response (fields may vary):

{
  "text": "hello world",
  "blocks": [
    {"bbox": [10, 20, 200, 60], "text": "hello", "conf": 87.3},
    {"bbox": [10, 70, 200, 120], "text": "world", "conf": 90.1}
  ],
  "objects": [
    {"label": "person", "score": 0.63, "bbox": [5, 12, 180, 210]}
  ],
  "scene_summary": "person x1",
  "meta": {
    "latency_ms": 123,
    "coral_used": true,
    "ocr_backend": "tesseract",
    "east_used": true,
    "text_locator": "east",
    "ocr_lang": "eng",
    "ocr_lang_requested": "eng+rus",
    "ocr_lang_strategy": "auto_script",
    "ocr_passes": 1,
    "request_id": "..."
  }
}

Notes

• If Coral is absent or fails, the service still returns OCR with coral_used=false.
• If OCR_BACKEND=paddle but PaddleOCR package/models are unavailable, service falls back to Tesseract.
• If EAST model is missing, service first tries OpenCV contour text regions and only then full-image OCR.
• Keep images under MAX_IMAGE_MB (413 returned otherwise).
• Optimize Tesseract languages to those you need to speed up OCR.