126 lines
5.2 KiB
Markdown
126 lines
5.2 KiB
Markdown
# Jetson Nano Image2Text API
|
|
|
|
Python 3.6-compatible Flask service that accepts an image and returns detected text plus optional EdgeTPU object context.
|
|
|
|
## Features
|
|
- Accepts JPEG/PNG/WebP/BMP/TIFF/GIF (first frame) via multipart or base64 JSON.
|
|
- Optional Google Coral EdgeTPU detection for scene context (loaded once at start).
|
|
- EAST text detector (OpenCV DNN) when model exists; otherwise OpenCV contour-based text-region fallback.
|
|
- OCR via `pytesseract` with per-block confidences; configurable languages.
|
|
- Auto-selects `rus` or `eng` (via Tesseract OSD script detection) before full OCR when `OCR_LANG=eng+rus`.
|
|
- If OCR result is empty, retries with fallback language/full-frame pass to improve recall.
|
|
- Hard payload cap (10MB default) and graceful degradation if Coral/EAST unavailable.
|
|
|
|
## System prerequisites (Ubuntu 18.04 aarch64)
|
|
```bash
|
|
# EdgeTPU runtime (choose std or max)
|
|
sudo apt-get update
|
|
sudo apt-get install -y libedgetpu1-std python3-pycoral
|
|
|
|
# Tesseract OCR + languages
|
|
sudo apt-get install -y tesseract-ocr tesseract-ocr-eng tesseract-ocr-rus
|
|
|
|
# OpenCV (binary from apt is preferred on Jetson)
|
|
sudo apt-get install -y python3-opencv
|
|
|
|
# If you prefer pip OpenCV (may be large on Jetson):
|
|
# pip3 install opencv-python==4.5.3.56
|
|
```
|
|
|
|
## Python dependencies
|
|
```bash
|
|
pip3 install -r requirements.txt
|
|
```
|
|
|
|
## Models
|
|
Place your models in `./models` (paths can be overridden via env):
|
|
- EdgeTPU SSD: `ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite`
|
|
- Labels: `coco_labels.txt`
|
|
- EAST text detector (optional): `frozen_east_text_detection.pb`
|
|
|
|
### Better OCR models (tessdata_best)
|
|
```bash
|
|
sudo apt-get install -y wget
|
|
sudo mkdir -p /usr/share/tesseract-ocr/4.00/tessdata_best
|
|
sudo wget -O /usr/share/tesseract-ocr/4.00/tessdata_best/eng.traineddata https://github.com/tesseract-ocr/tessdata_best/raw/main/eng.traineddata
|
|
sudo wget -O /usr/share/tesseract-ocr/4.00/tessdata_best/rus.traineddata https://github.com/tesseract-ocr/tessdata_best/raw/main/rus.traineddata
|
|
# then run the service with:
|
|
export TESSDATA_PREFIX=/usr/share/tesseract-ocr/4.00/tessdata_best
|
|
# if you skip this, the app will auto-try common system paths
|
|
```
|
|
|
|
### OpenCV with CUDA (for faster EAST)
|
|
OpenCV from JetPack repos (4.x) usually ships without CUDA dnn. To use CUDA, build OpenCV 4.5+ with `-D WITH_CUDA=ON -D OPENCV_DNN_CUDA=ON`. After installing, export `USE_CUDA_DNN=true` so the service enables CUDA backend/target when available.
|
|
|
|
## Environment variables
|
|
- `MODEL_EDGETPU_PATH` (default `./models/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite`)
|
|
- `MODEL_LABELS_PATH` (default `./models/coco_labels.txt`)
|
|
- `MODEL_EAST_PATH` (default `./models/frozen_east_text_detection.pb`)
|
|
- `OCR_LANG` (default `eng+rus`; request language set for OCR)
|
|
- `OCR_AUTO_LANG` (default `true`; auto-pick `eng` or `rus` by script when `OCR_LANG` contains both)
|
|
- `OCR_CONFIG` (default `--oem 1 --psm 6`, forwarded to Tesseract)
|
|
- `TESSDATA_PREFIX` (path to tessdata / tessdata_best if you install better models)
|
|
- `OCR_UPSCALE` (default `1.5`, upscale factor before OCR; max 3.0)
|
|
- `OCR_CLAHE` (default `true`, contrast-limited adaptive histogram)
|
|
- `OCR_BILATERAL` (default `false`, light denoise)
|
|
- `OCR_BLUR` (default `0`, Gaussian kernel size; 0 disables)
|
|
- `OCR_INVERT` (default `false`, invert after threshold)
|
|
- `OCR_USE_THRESHOLD` (default `true`, adaptive threshold)
|
|
- `MAX_IMAGE_MB` (default `10`)
|
|
- `CONF_THRESHOLD` (default `0.4`)
|
|
- `RETURN_BOXES` (default `true`)
|
|
- `USE_CUDA_DNN` (default `false`, set `true` to attempt CUDA backend for EAST if OpenCV built with CUDA)
|
|
- `DEBUG_VISUAL` (default `false`; if true, response includes `debug_image_base64`. Per-request: add `?debug=1` or form field `debug=1`.)
|
|
- `SHOW_GUI` (default `false`; if true and OpenCV GUI available, service opens a text-centric window: left = detected text regions, right = OCR block text/conf. Requires X/Wayland/VDI that supports GUI.)
|
|
|
|
## Run
|
|
```bash
|
|
python3 app.py --host 0.0.0.0 --port 8000
|
|
```
|
|
Waitress or gunicorn can also serve the app if desired (not required).
|
|
|
|
## API
|
|
- `GET /health` -> `{ "status": "ok", "coral": true/false }`
|
|
- `POST /v1/image2text`
|
|
- Multipart: field `image`
|
|
- JSON: `{ "image_base64": "<base64>" }`
|
|
|
|
Example curl (multipart):
|
|
```bash
|
|
curl -X POST http://localhost:8000/v1/image2text \
|
|
-F "image=@sample.jpg" \
|
|
-F "debug=1" \
|
|
| python -m json.tool
|
|
```
|
|
|
|
Example JSON response (fields may vary):
|
|
```json
|
|
{
|
|
"text": "hello world",
|
|
"blocks": [
|
|
{"bbox": [10, 20, 200, 60], "text": "hello", "conf": 87.3},
|
|
{"bbox": [10, 70, 200, 120], "text": "world", "conf": 90.1}
|
|
],
|
|
"objects": [
|
|
{"label": "person", "score": 0.63, "bbox": [5, 12, 180, 210]}
|
|
],
|
|
"scene_summary": "person x1",
|
|
"meta": {
|
|
"latency_ms": 123,
|
|
"coral_used": true,
|
|
"east_used": true,
|
|
"text_locator": "east",
|
|
"ocr_lang": "eng",
|
|
"ocr_lang_requested": "eng+rus",
|
|
"ocr_lang_strategy": "auto_script",
|
|
"ocr_passes": 1,
|
|
"request_id": "..."
|
|
}
|
|
}
|
|
```
|
|
|
|
## Notes
|
|
- If Coral is absent or fails, the service still returns OCR with `coral_used=false`.
|
|
- If EAST model is missing, service first tries OpenCV contour text regions and only then full-image OCR.
|
|
- Keep images under `MAX_IMAGE_MB` (413 returned otherwise).
|
|
- Optimize Tesseract languages to those you need to speed up OCR.
|