2025-09-22-server-computed-analytics

ADR: Server-computed Weekly Analytics and Dashboard Endpoints

  • Status: Proposed
  • Date: 2025-09-22
  • Owners: Frontend + Backend teams

Context

The frontend currently derives several analytics at runtime from weekly snapshots and project lists (e.g., Starts/Finishes, Throughput, Active Ratio helpers, moving averages, week-over-week deltas, and month bucketing). This duplication creates:

  • Inconsistent definitions across clients and time (ordering, timezone, rounding).
  • Fragile logic (e.g., most-recent-first vs ascending sort, partial-week shading heuristics).
  • Extra client CPU and multiple requests for basic dashboard KPIs.

Recent live OpenAPI additions already provide avg_wip, max_wip, and sample_count in weekly rollups. We propose extending server output to cover the remaining derived metrics and offer summary endpoints to simplify the frontend.

Decision

Shift key derived analytics from client-side computation to server-side responses. Concretely:

  1. Enrich GET /stats/weekly items (ascending order) with:
  • Identity and ordering:
    • week_id (e.g., 2025-W38), period_start, period_end, index_asc (0-based), is_partial_week (true if the trailing week isn’t complete at generation time).
  • Deltas and derived counts:
    • starts, finishes, net (starts - finishes), delta_active, delta_backlog (vs previous week end snapshot).
  • Throughput helpers:
    • throughput (alias of finishes), throughput_ma4 (4-week moving average), throughput_wow (last/prev relative delta).
  • Active ratio helpers:
    • active_ratio_end, active_ratio_avg, plus active_ratio_ma4, active_ratio_wow.
  • WIP helpers (building on existing fields):
    • avg_wip, max_wip, plus wip_ma4, wip_wow.
  • Progress distribution:
    • progress_bins (0–100% histogram), progress_weighted_mean.
  • Samples:
    • sample_count.
  1. Add summary and aligned-series endpoints for dashboards:
  • GET /dashboard/summary — one-hop KPI payload (projects counts, customers, people, shared %, average duration, upcoming/ending soon, missing dates) with generated_at and tz.
  • GET /dashboard/sparklines — aligned 12-week labels and series for active_ratio, throughput, wip_avg, wip_max (with MA(4) and WoW), configurable window and tz.
  • GET /counts/active-by-month — backend month bucketing for last N months with tz and generated_at.

All arrays are ordered ascending in time. Default timezone is UTC; clients may request an IANA timezone via tz.

Alternatives Considered

  • Keep analytics client-side: simplest for backend, but perpetuates inconsistencies and UI bugs (ordering, partial-week) and increases client complexity.
  • GraphQL computed fields: flexible but requires new infra; REST with additive fields meets current needs with lower effort.
  • Batch-only endpoints (no helpers): reduces hops but still requires re-implementing helpers client-side.

Consequences

  • Pros
    • Consistent, audited definitions across clients (one source of truth).
    • Less client massaging and fewer round-trips; simpler UI code paths.
    • Fewer timezone/order bugs; easier testing via golden payloads.
  • Cons
    • Backend complexity and compute cost (moving averages, WoW, deltas).
    • Requires caching/invalidation strategy and light versioning.

API Changes (summary)

Additive changes to existing WeeklyRollup and three new endpoints. Sketches below (OpenAPI excerpts):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

components:
  schemas:
    WeeklyRollup:  # augment existing
      type: object
      properties:
        week_id: { type: string, description: Canonical week key, e.g. 2025-W38 }
        period_start: { type: string, format: date }
        period_end: { type: string, format: date }
        index_asc: { type: integer }
        is_partial_week: { type: boolean }
        starts: { type: integer, minimum: 0 }
        finishes: { type: integer, minimum: 0 }
        net: { type: integer }
        delta_active: { type: integer }
        delta_backlog: { type: integer }
        throughput: { type: integer }
        throughput_ma4: { type: number, format: float, nullable: true }
        throughput_wow: { type: number, format: float, nullable: true }
        active_ratio_end: { type: number, format: float, minimum: 0, maximum: 1 }
        active_ratio_avg: { type: number, format: float, minimum: 0, maximum: 1 }
        active_ratio_ma4: { type: number, format: float, nullable: true }
        active_ratio_wow: { type: number, format: float, nullable: true }
        avg_wip: { type: number, format: float, nullable: true }
        max_wip: { type: number, format: float, nullable: true }
        wip_ma4: { type: number, format: float, nullable: true }
        wip_wow: { type: number, format: float, nullable: true }
        progress_weighted_mean: { type: number, format: float, minimum: 0, maximum: 1 }
        progress_bins:
          type: array
          items: { $ref: '#/components/schemas/ProgressBin' }
        sample_count: { type: integer, minimum: 0 }

    ProgressBin:
      type: object
      properties:
        pct: { type: integer, minimum: 0, maximum: 100 }
        count: { type: integer, minimum: 0 }
      required: [pct, count]

    DashboardSummary:
      type: object
      properties:
        projects_total: { type: integer, minimum: 0 }
        projects_active: { type: integer, minimum: 0 }
        projects_delayed: { type: integer, minimum: 0 }
        projects_archived: { type: integer, minimum: 0 }
        projects_backlog: { type: integer, minimum: 0 }
        customers_count: { type: integer, minimum: 0 }
        unique_people_count: { type: integer, minimum: 0 }
        shared_projects_pct: { type: number, format: float, minimum: 0, maximum: 1 }
        avg_duration_days: { type: number, format: float, minimum: 0 }
        upcoming_starts_count: { type: integer, minimum: 0 }
        ending_soon_count: { type: integer, minimum: 0 }
        missing_dates_count: { type: integer, minimum: 0 }
        generated_at: { type: string, format: date-time }
        tz: { type: string }

    DashboardSparklines:
      type: object
      properties:
        labels: { type: array, items: { type: string } }
        series:
          type: array
          items:
            type: object
            properties:
              label: { type: string, enum: [active_ratio, throughput, wip_avg, wip_max] }
              unit: { type: string, enum: [ratio, count] }
              data: { type: array, items: { type: number } }
              ma4: { type: array, items: { type: number } }
              wow: { type: number, nullable: true }
        tz: { type: string }
        generated_at: { type: string, format: date-time }

    ActiveByMonth:
      type: object
      properties:
        labels: { type: array, items: { type: string, pattern: '^[0-9]{4}-[0-9]{2}$' } }
        active_counts: { type: array, items: { type: integer, minimum: 0 } }
        tz: { type: string }
        generated_at: { type: string, format: date-time }
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77

paths:
  /stats/weekly:
    get:
      summary: Weekly rollups (enriched)
      parameters:
        - in: query
          name: from
          schema: { type: string, format: date }
        - in: query
          name: to
          schema: { type: string, format: date }
        - in: query
          name: limit
          schema: { type: integer, minimum: 1, maximum: 104, default: 52 }
        - in: query
          name: tz
          schema: { type: string }
      responses:
        '200':
          content:
            application/json:
              schema:
                type: array
                items: { $ref: '#/components/schemas/WeeklyRollup' }

  /dashboard/summary:
    get:
      summary: Dashboard KPI summary
      parameters:
        - in: query
          name: tz
          schema: { type: string }
      responses:
        '200':
          content:
            application/json:
              schema: { $ref: '#/components/schemas/DashboardSummary' }

  /dashboard/sparklines:
    get:
      summary: Aligned 12-week dashboard sparklines
      parameters:
        - in: query
          name: series
          schema:
            type: array
            items: { type: string, enum: [active_ratio, throughput, wip_avg, wip_max] }
            default: [active_ratio, throughput, wip_avg, wip_max]
          style: form
          explode: true
        - in: query
          name: window
          schema: { type: integer, minimum: 4, maximum: 52, default: 12 }
        - in: query
          name: tz
          schema: { type: string }
      responses:
        '200':
          content:
            application/json:
              schema: { $ref: '#/components/schemas/DashboardSparklines' }

  /counts/active-by-month:
    get:
      summary: Active projects by month
      parameters:
        - in: query
          name: months
          schema: { type: integer, minimum: 1, maximum: 60, default: 12 }
        - in: query
          name: tz
          schema: { type: string }
      responses:
        '200':
          content:
            application/json:
              schema: { $ref: '#/components/schemas/ActiveByMonth' }

Backward Compatibility

  • Changes to WeeklyRollup are additive; existing consumers remain valid.
  • New endpoints are optional. Frontend will initially implement a dual-path: use server-provided helpers when available and fall back to current client computation when absent.
  • Consider adding a lightweight response header (e.g., X-Stats-Version: 2025-09-22) or a version property for debugging/multi-client environments.

Rollout Plan

  1. Backend implements enriched WeeklyRollup fields (behind a feature flag if needed) and new endpoints.
  2. Frontend consumes new fields with graceful fallback; add metrics logging when falling back.
  3. Bake in staging for at least one full week of data to validate rolling metrics.
  4. Flip default to server-computed in production; remove client computation after 2–4 weeks.

Performance & Caching

  • Precompute weekly rollups on schedule (e.g., hourly/daily) and cache responses (ETag/Last-Modified + CDN/local cache with short TTL).
  • Moving averages are cheap to compute incrementally; optionally persist them to avoid per-request scanning.
  • Summary endpoints are light; cache per tz.

Security & Privacy

  • Payloads contain aggregate counts only; no PII beyond existing entities.
  • Rate-limit externally exposed endpoints consistent with current API policy.

Testing Strategy

  • Backend unit tests for delta/throughput/ratio helpers and partial-week flag.
  • Contract tests (e.g., Schemathesis) against OpenAPI.
  • Golden fixtures of 12–16 weeks for reproducible expectations (ascending order required).
  • Frontend Vitest toggles: simulate presence/absence of helpers to ensure fallback behavior.

Acceptance Criteria

  • GET /stats/weekly returns ascending weeks with is_partial_week and the helper fields listed above.
  • GET /dashboard/summary returns stable KPIs with generated_at and tz.
  • GET /dashboard/sparklines returns aligned labels and series arrays for requested metrics.
  • GET /counts/active-by-month returns month-bucketed active counts with the requested months and tz.
  • Frontend renders throughput and WIP KPIs/sparklines without local recomputation when helpers are present.

Notes

  • Keep numeric fields as numbers; clients will format units. Return both ratio and count forms where helpful for reuse.
  • Prefer UTC default with explicit tz parameter for deterministic server behavior.