Jetson Nano Image2Text API

Python 3.6-compatible Flask service that accepts an image and returns detected text plus optional EdgeTPU object context.

Features

• Accepts JPEG/PNG/WebP/BMP/TIFF/GIF (first frame) via multipart or base64 JSON.
• Optional Google Coral EdgeTPU detection for scene context (loaded once at start).
• EAST text detector (OpenCV DNN) when model exists; otherwise whole-image fallback.
• OCR via pytesseract with per-block confidences; configurable languages.
• Auto-selects rus or eng (via Tesseract OSD script detection) before full OCR when OCR_LANG=eng+rus.
• If OCR result is empty, retries with fallback language/full-frame pass to improve recall.
• Hard payload cap (10MB default) and graceful degradation if Coral/EAST unavailable.

System prerequisites (Ubuntu 18.04 aarch64)

# EdgeTPU runtime (choose std or max)
sudo apt-get update
sudo apt-get install -y libedgetpu1-std python3-pycoral

# Tesseract OCR + languages
sudo apt-get install -y tesseract-ocr tesseract-ocr-eng tesseract-ocr-rus

# OpenCV (binary from apt is preferred on Jetson)
sudo apt-get install -y python3-opencv

# If you prefer pip OpenCV (may be large on Jetson):
# pip3 install opencv-python==4.5.3.56

Python dependencies

pip3 install -r requirements.txt

Models

Place your models in ./models (paths can be overridden via env):

• EdgeTPU SSD: ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite
• Labels: coco_labels.txt
• EAST text detector (optional): frozen_east_text_detection.pb

Better OCR models (tessdata_best)

sudo apt-get install -y wget
sudo mkdir -p /usr/share/tesseract-ocr/4.00/tessdata_best
sudo wget -O /usr/share/tesseract-ocr/4.00/tessdata_best/eng.traineddata https://github.com/tesseract-ocr/tessdata_best/raw/main/eng.traineddata
sudo wget -O /usr/share/tesseract-ocr/4.00/tessdata_best/rus.traineddata https://github.com/tesseract-ocr/tessdata_best/raw/main/rus.traineddata
# then run the service with:
export TESSDATA_PREFIX=/usr/share/tesseract-ocr/4.00/tessdata_best
# if you skip this, the app will auto-try common system paths

OpenCV with CUDA (for faster EAST)

OpenCV from JetPack repos (4.x) usually ships without CUDA dnn. To use CUDA, build OpenCV 4.5+ with -D WITH_CUDA=ON -D OPENCV_DNN_CUDA=ON. After installing, export USE_CUDA_DNN=true so the service enables CUDA backend/target when available.

Environment variables

• MODEL_EDGETPU_PATH (default ./models/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite)
• MODEL_LABELS_PATH (default ./models/coco_labels.txt)
• MODEL_EAST_PATH (default ./models/frozen_east_text_detection.pb)
• OCR_LANG (default eng+rus; request language set for OCR)
• OCR_AUTO_LANG (default true; auto-pick eng or rus by script when OCR_LANG contains both)
• OCR_CONFIG (default --oem 1 --psm 6, forwarded to Tesseract)
• TESSDATA_PREFIX (path to tessdata / tessdata_best if you install better models)
• OCR_UPSCALE (default 1.5, upscale factor before OCR; max 3.0)
• OCR_CLAHE (default true, contrast-limited adaptive histogram)
• OCR_BILATERAL (default false, light denoise)
• OCR_BLUR (default 0, Gaussian kernel size; 0 disables)
• OCR_INVERT (default false, invert after threshold)
• OCR_USE_THRESHOLD (default true, adaptive threshold)
• MAX_IMAGE_MB (default 10)
• CONF_THRESHOLD (default 0.4)
• RETURN_BOXES (default true)
• USE_CUDA_DNN (default false, set true to attempt CUDA backend for EAST if OpenCV built with CUDA)
• DEBUG_VISUAL (default false; if true, response includes debug_image_base64. Per-request: add ?debug=1 or form field debug=1.)
• SHOW_GUI (default false; if true and OpenCV GUI available, service opens a window with overlays. Requires X/Wayland/VDI that supports GUI.)

Run

python3 app.py --host 0.0.0.0 --port 8000

Waitress or gunicorn can also serve the app if desired (not required).

API

• GET /health -> { "status": "ok", "coral": true/false }
• POST /v1/image2text
  • Multipart: field image
  • JSON: { "image_base64": "<base64>" }

Example curl (multipart):

curl -X POST http://localhost:8000/v1/image2text \
  -F "image=@sample.jpg" \
  -F "debug=1" \
  | python -m json.tool

Example JSON response (fields may vary):

{
  "text": "hello world",
  "blocks": [
    {"bbox": [10, 20, 200, 60], "text": "hello", "conf": 87.3},
    {"bbox": [10, 70, 200, 120], "text": "world", "conf": 90.1}
  ],
  "objects": [
    {"label": "person", "score": 0.63, "bbox": [5, 12, 180, 210]}
  ],
  "scene_summary": "person x1",
  "meta": {
    "latency_ms": 123,
    "coral_used": true,
    "east_used": true,
    "ocr_lang": "eng",
    "ocr_lang_requested": "eng+rus",
    "ocr_lang_strategy": "auto_script",
    "ocr_passes": 1,
    "request_id": "..."
  }
}

Notes

• If Coral is absent or fails, the service still returns OCR with coral_used=false.
• If EAST model is missing, the whole image is passed to OCR to avoid empty results.
• Keep images under MAX_IMAGE_MB (413 returned otherwise).
• Optimize Tesseract languages to those you need to speed up OCR.