ThingsBoard Integration Epic Design (Issue #104)

Context

This epic defines the integration foundation between ThingsBoard and the existing IoT platform before building the Monitoring screen under Dashboard. The goal is to avoid local, one-off UI implementation and instead establish stable platform boundaries for realtime telemetry and future operations workflows.

Confirmed Decisions

  • Prioritize telemetry-to-UI latency in the MVP.
  • Use a full backend proxy model for ThingsBoard access.
  • Use a shared ThingsBoard tenant model with strict backend-enforced scoping.
  • Keep browser clients isolated from direct ThingsBoard auth and APIs.
  • Keep alert source-of-truth evaluation in ThingsBoard, with business orchestration in backend.

Architecture

Boundary Ownership

  • ThingsBoard owns telemetry ingestion/storage, device-side platform features, and alert rule evaluation.
  • Backend owns product API surface, tenant boundary enforcement, RBAC enforcement, data normalization, workflow orchestration, and observability across systems.
  • Frontend owns user experience only, consuming backend APIs and streams without direct ThingsBoard dependencies.

Core Backend Components

  1. ThingsBoard Integration Adapter

    • Encapsulates ThingsBoard API and stream protocols.
    • Converts vendor payloads to internal normalized DTOs.

  2. Monitoring Realtime Gateway

    • Provides /api/v1/monitoring/stream for UI clients.
    • Maintains persistent upstream subscriptions and per-client scoped fanout.

  3. Monitoring Query Service

    • Provides snapshot/history APIs for dashboard and detail pages.
    • Applies tenant and permission filters before any data leaves backend.

  4. Tenant + RBAC Scope Resolver

    • Resolves user -> organization -> authorized devices/resources.
    • Produces subscription/query scopes consumed by stream and query paths.

  5. Alert Orchestration Service

    • Consumes alert state from ThingsBoard.
    • Applies platform workflow logic (acknowledgement, escalation, audit integration).

  6. Integration Observability Layer

    • Emits latency, stream health, error, and reconnect metrics.
    • Correlates traces/logs across frontend request, backend service, and ThingsBoard calls.

Contracts and Data Flow

Realtime Telemetry

  1. Frontend opens backend stream: wss /api/v1/monitoring/stream with platform JWT.
  2. Backend authenticates user and computes authorized device scope.
  3. Backend reuses or creates upstream ThingsBoard subscriptions for required devices.
  4. Backend normalizes telemetry events and emits authorized events to client sessions.

Normalized event contract (logical shape):

  • deviceId
  • metric
  • value
  • ts
  • quality
  • source

Snapshot and History

  • GET /api/v1/monitoring/snapshot?scope=...
  • GET /api/v1/monitoring/history?deviceId=...&from=...&to=...

Backend responsibilities:

  • validate org and permission scope;
  • fetch from ThingsBoard (or backend cache when applicable);
  • return normalized DTOs for consistent frontend rendering.

Alert Contract

  • ThingsBoard performs alert evaluation and state transitions at source level.
  • Backend ingests alert states, enriches them with business context, and exposes product-safe APIs.

Proposed API surface:

  • GET /api/v1/alerts
  • POST /api/v1/alerts/:id/ack

RPC Bridge (Post-MVP)

  • Frontend sends command intent to backend endpoint.
  • Backend validates RBAC/scope, dispatches RPC via ThingsBoard, and returns correlation state for async status tracking.

Tenant and Identity Mapping

Because ThingsBoard tenant is shared, tenant isolation is enforced in backend policy and mapping layers:

  • Every user session resolves to organization-scoped accessible resources.
  • Device ownership/visibility mapping is maintained in backend canonical domain model.
  • Backend never emits an event or query result for devices outside resolved org scope.

Hard requirement:

  • Zero cross-tenant leakage tolerance by design and by runtime safeguards.

Reliability and Performance Design

Latency Strategy

  • Persistent upstream connections to ThingsBoard.
  • Subscription pooling to avoid per-client upstream duplication.
  • Lightweight normalization in hot path.
  • Optional latest-value in-memory cache for first-paint snapshot acceleration.

Reliability Strategy

  • Bounded exponential reconnect for upstream stream disruption.
  • Heartbeat and stale-stream detection.
  • Idempotent subscription lifecycle management.
  • Backpressure policy per client connection.
  • Degraded mode response path when upstream instability is detected.

Observability Baseline

Track at minimum:

  • end-to-end stream latency;
  • stream throughput and drop count;
  • reconnect frequency;
  • ThingsBoard call error rates;
  • per-organization stream health.

Logs and traces must include correlation IDs to connect UI actions with backend and ThingsBoard operations.

Security Boundary

  • Frontend has no direct ThingsBoard token or endpoint dependency.
  • Backend credentials for ThingsBoard stay server-side only.
  • All access paths enforce JWT auth + RBAC + organization scope before data emission.
  • Shared-tenant ThingsBoard usage must be guarded by explicit backend filters and deny-by-default policy.

Epic-to-Issue Roadmap

Phase 1: Integration Architecture and Contracts

  • #105 Integration architecture and contracts
  • Define DTOs, stream contracts, endpoint boundaries, and error semantics.

Phase 2: Tenant and Identity Mapping

  • #106 Tenant and identity mapping
  • Implement authoritative scope resolver and mapping verification.

Phase 3: Realtime Telemetry Pipeline MVP

  • #107 Telemetry realtime pipeline
  • Implement backend proxy stream, upstream integration adapter, and scoped fanout.

Phase 4: Monitoring UI MVP

  • #109 MVP Monitoring screen in Dashboard
  • Use backend contracts only; no direct ThingsBoard frontend integration.

Phase 5: Hardening and Operations

  • #108 Observability and reliability
  • Add robust retries, alerting, metrics dashboards, and failure playbooks.

Phase 6: RPC Bridge

  • #110 RPC command bridge
  • Add command routing, correlation tracking, and operational safeguards.

Acceptance Criteria Alignment

This design satisfies epic acceptance by providing:

  • one agreed integration architecture;
  • clear telemetry and identity/tenant contracts;
  • explicit alert ownership split;
  • phased backlog structure from MVP to hardening;
  • measurable initial success criteria around latency, stability, and isolation.

Initial Success Metrics

  • P95 realtime telemetry latency from ingestion path to UI under agreed threshold.
  • Integration error rate below agreed alert threshold.
  • Zero confirmed multi-tenant boundary violations.
  • Stable stream session behavior under expected concurrent dashboard usage.

Out of Scope

  • Full completion of Monitoring UI in this epic.
  • Replacing all existing business backend logic with ThingsBoard.
  • Full historical data migration in phase one.