About ADA-IG

Browser │ HTTP :18080 ▼ ┌─────────────────────────────┐ │ api │ FastAPI + Uvicorn │ (FastAPI + Jinja2 UI) │ Serves the dashboard UI and REST API └──────────────┬──────────────┘ │ SQLAlchemy ▼ ┌─────────────────────────────┐ │ db │ PostgreSQL 16 + pgvector extension │ (PostgreSQL 16 + pgvector) │ Stores all normalized entities and vectors └─────────────────────────────┘ ▲ │ SQLAlchemy ┌──────────────┴──────────────┐ │ worker │ Background ingestion loop │ (background ingestor) │ Periodically fetches and processes data └─────────────────────────────┘ ┌─────────────────────────────┐ │ migrate │ One-shot Alembic runner │ (alembic upgrade head) │ Runs on startup, exits when done └─────────────────────────────┘ ┌─────────────────────────────┐ │ playwright │ Optional public capture (off by default) │ (--profile public-capture)│ Requires PUBLIC_CAPTURE_ENABLED=true └─────────────────────────────┘

openclaw-instagram-data-platform/ ├── app/ │ ├── main.py # FastAPI app factory; mounts all routers and UI routes │ ├── api/routes/ # One file per REST resource │ │ ├── health.py # /healthz and /readyz probes │ │ ├── runs.py # Ingestion run listing and detail │ │ ├── profiles.py # Profile listing │ │ ├── posts.py # Post listing │ │ ├── comments.py # Comment listing │ │ ├── changes.py # Field-level diff listing │ │ ├── search.py # Vector similarity search │ │ ├── replay.py # Re-normalize an existing run's artifacts │ │ ├── hooks.py # Inbound OpenClaw automation webhook │ │ ├── settings.py # Settings form save + JSON read │ │ └── operator.py # Seed, wipe, summary, connection status │ ├── core/ # App infrastructure, no business logic │ │ ├── config.py # Pydantic Settings; reads .env │ │ ├── db.py # SQLAlchemy engine and session factory │ │ ├── logging.py # Structured logging setup │ │ ├── security.py # Guardrail checks (public capture, playwright) │ │ ├── storage.py # Artifact file read/write abstraction │ │ ├── env_validation.py # Startup .env sanity checks │ │ ├── metrics.py # Internal counters │ │ └── queue.py # In-process task queue │ ├── models/ # SQLAlchemy ORM models (one per table) │ │ ├── source_target.py # Monitored account (profile_key, status) │ │ ├── source_run.py # One ingestion run (status, trigger, timestamps) │ │ ├── source_artifact.py # Raw JSON blob record (hash, size, path) │ │ ├── profile.py # Canonical Instagram account │ │ ├── profile_snapshot.py # Point-in-time profile state │ │ ├── post.py # Canonical Instagram post │ │ ├── post_snapshot.py # Point-in-time post state (caption, likes) │ │ ├── comment.py # Individual comment │ │ ├── entity_diff.py # Field-level change (old_value, new_value) │ │ ├── embedding_chunk.py # Text chunk + pgvector embedding │ │ ├── run_event.py # Timestamped event within a run │ │ ├── dead_letter.py # Records that failed processing │ │ ├── enums.py # Shared enums (EntityType, RunStatus, etc.) │ │ └── common.py # Shared base model and UUID/timestamp mixins │ ├── services/ingestion/ # The full ingestion pipeline │ │ ├── pipeline.py # Orchestrates a complete ingestion run end-to-end │ │ ├── normalizer.py # Maps raw API payload to ORM models + snapshots │ │ ├── diff_engine.py # Compares snapshots, writes EntityDiff records │ │ ├── artifact_writer.py # Writes raw JSON to disk and SourceArtifact to DB │ │ ├── official_ingestor.py # Meta Graph API fetcher │ │ └── public_capture_ingestor.py # Playwright-based public capture (opt-in) │ ├── services/retrieval/ # Search pipeline │ │ ├── search.py # Query entry point and result assembly │ │ ├── search_service.py # pgvector cosine similarity query │ │ ├── chunker.py # Splits text into overlapping embedding chunks │ │ └── embedder.py # Generates vectors (dev: hash stub, prod: model) │ ├── services/dashboard/ │ │ ├── queries.py # Raw ORM queries for dashboard data │ │ └── query_service.py # Aggregation and summary logic │ ├── settings_service.py # Reads and writes .env settings from the browser │ ├── operator_service.py # Seed demo data, wipe platform data, count summary │ └── connection_service.py # Evaluates Instagram API credential health │ ├── integrations/ │ │ ├── instagram/client.py # Meta Graph API HTTP client (stub, incomplete) │ │ └── playwright/capture.py # Playwright browser automation (stub) │ ├── workers/ │ │ ├── main.py # Worker container entrypoint │ │ ├── runner.py # Task execution loop │ │ └── tasks/ # Background task definitions │ ├── schemas/ # Pydantic request/response schemas │ │ ├── common.py # Shared types (pagination, provenance) │ │ ├── runs.py # Run request/response shapes │ │ └── search.py # Search result schema │ └── ui/templates/ # Jinja2 HTML templates │ ├── base.html # Navigation, global CSS, layout shell │ ├── index.html # Dashboard (KPIs, profiles, posts, runs) │ ├── profile.html # Profile drilldown (posts, comments, diffs) │ ├── settings.html # Platform settings (tabbed form + operator controls) │ └── about.html # This page ├── migrations/versions/ # Alembic migration scripts ├── docker/ │ ├── api.Dockerfile # Image for api and migrate services │ ├── worker.Dockerfile # Image for the background worker │ └── playwright.Dockerfile # Image for optional public capture ├── scripts/ │ ├── dev_seed.py # CLI script to seed demo data into the DB │ ├── demo_payloads.py # Sample Instagram API payloads for testing │ └── rebuild.sh # Full image rebuild + migration + restart ├── compose.yaml # Docker Compose service definitions ├── pyproject.toml # Python dependencies and tool config (ruff, pytest) └── alembic.ini # Alembic migration configuration