Infrastructure Resources
This page is the ops-level inventory of every Cloudflare and PlanetScale resource the platform depends on. It complements the Physical Architecture (which maps logical components to infrastructure) and the Architecture Decision Records (which explain why each choice was made). Use this page when provisioning a new environment, auditing costs, or onboarding someone to the infrastructure.
Resource connectivity
Section titled “Resource connectivity”graph TB
subgraph "Cloudflare Edge"
Pages["Pages<br/>(React SPA + Docs)"]
Access["CF Access<br/>(Google SSO)"]
subgraph "Workers (Compute)"
DashAPI["dashboard-api<br/>(Hono, staff API)"]
Ingest["ingest<br/>(Workflows, queue consumer)"]
PlatformAPI["platform-api<br/>(public API)"]
end
subgraph "Durable Execution"
Workflows["CF Workflows<br/>(6 workflows)"]
DO["Durable Objects<br/>(watermark state)"]
end
Queue["Queue: ingest-queue"]
KV["KV Namespace<br/>(config cache)"]
subgraph "Analytics"
Pipelines["Pipelines<br/>(3 telemetry streams)"]
WAE["Workers Analytics Engine<br/>(real-time counters)"]
end
subgraph "Storage"
R2Assets["R2: Design Assets"]
R2Archive["R2: JSONL Archive"]
R2Catalog["R2 Data Catalog<br/>(Iceberg tables)"]
end
end
subgraph "External"
PS["PlanetScale<br/>(PostgreSQL)"]
Shopify["Shopify Admin API"]
Dropbox["Dropbox"]
HD["Hyperdrive<br/>(connection pool)"]
end
Access -->|JWT gate| Pages
Access -->|JWT gate| DashAPI
Pages -->|API proxy| DashAPI
DashAPI -->|Service Binding| Ingest
DashAPI -->|send| Queue
Queue -->|consume| Ingest
Ingest --> Workflows
Workflows --> DO
Ingest -->|stream push| Pipelines
PlatformAPI -->|stream push| Pipelines
Pipelines --> R2Catalog
Ingest --> R2Assets
Ingest --> R2Archive
DashAPI --> HD
Ingest --> HD
PlatformAPI --> HD
HD --> PS
Ingest --> Shopify
Ingest --> Dropbox
PlatformAPI -->|write| WAE
Resource summary
Section titled “Resource summary”| Resource Type | Name | Purpose | Used By (scenarios) |
|---|---|---|---|
| Worker | dashboard-api | Staff-facing HTTP API | All staff scenarios |
| Worker | platform-api | Public API (publishers, telemetry) | Publisher telemetry, public endpoints |
| Worker | ingest | Queue consumer, Workflow executor, archival | Bulk Sync, Asset Ingest, CSV Export, Combo Suggestions |
| CF Workflow | Shopify Bulk Sync | Multi-step order/product sync | Bulk Sync |
| CF Workflow | Measurement Refresh | DuckDB aggregation over Iceberg | Performance Metrics, Tag Analytics |
| CF Workflow | Asset Ingest | Dropbox to R2 to Shopify metafield | Asset Ingest |
| CF Workflow | CSV Export | Query, format, upload to R2 | CSV Export |
| CF Workflow | Suggestion Generation | Generate combo suggestions from metrics | Combo Suggestions |
| CF Workflow | Combo Import | Parse CSV, upsert combo reports | Combo CRUD |
| Queue | ingest-queue | Job dispatch (typed discriminator) | All background jobs |
| R2 Bucket | Design assets | Product design images | Asset Ingest |
| R2 Bucket | JSONL archive | Bulk sync audit trail, CSV exports | Bulk Sync, CSV Export |
| R2 Data Catalog | Iceberg tables | Analytics retention (3 telemetry + 3 measurement) | Performance Metrics, Pipeline |
| Pipeline | COMMERCE_TELEMETRY | Order lifecycle events | Bulk Sync |
| Pipeline | PUBLICATION_TELEMETRY | Clicks, impressions, views | Publisher telemetry |
| Pipeline | SYNC_TELEMETRY | Workflow audit trail | All Workflows |
| Durable Object | Watermark singleton | Cursor state for incremental sync | Bulk Sync |
| KV Namespace | Config cache | Feature flags, classification thresholds | Optional — all scenarios |
| Analytics Engine | Real-time counters | Clicks, impressions, page views | Publisher telemetry |
| Pages | Dashboard SPA | React frontend | All staff scenarios |
| Pages | Docs site | This documentation | N/A |
| CF Access | Application policy | Google SSO gate | All staff scenarios |
| Hyperdrive | Connection pool | PlanetScale connection optimization | All Workers |
| PlanetScale | PostgreSQL database | Domain/transactional store (153 tables) | All scenarios |
Workers (Compute)
Section titled “Workers (Compute)”The platform runs three Workers, each with a distinct trust boundary and binding set.
dashboard-api — staff API
Section titled “dashboard-api — staff API”The primary HTTP surface for the React dashboard. Protected by Cloudflare Access (Google SSO). Built with Hono.
Bindings:
| Binding | Type | Purpose |
|---|---|---|
HYPERDRIVE | Hyperdrive | PlanetScale connection pooling |
R2_BUCKET | R2 Bucket | Asset reads, CSV export downloads |
INGEST_QUEUE | Queue (producer) | Dispatch background jobs |
INGEST | Service Binding | Direct calls to the ingest Worker (health, status) |
KV | KV Namespace | Optional config/feature flag cache |
SHOPIFY_ACCESS_TOKEN | Secret | Shopify API auth |
CF_ACCESS_AUD | Variable | Access application audience tag for JWT validation |
Routes overview:
| Route group | Endpoints | Module |
|---|---|---|
/api/performance-metrics | CRUD, sync trigger, classification settings | Performance |
/api/combo-logs | CRUD, import, export | Combos |
/api/combo-templates | CRUD | Combos |
/api/combo-suggestions | Generate, list, accept | Combos |
/api/products/tags | Tag sync, classification, groups | Tags |
/api/tag-analytics | Performance, distribution, refresh | Tags |
/api/tag-groups | CRUD | Tags |
/api/assets | Ingest trigger, status | Assets |
/api/csv | Generate, list, download | CSV Exports |
/api/orders | Lookup | Orders |
/api/designers | List, assign, unassign | Designer Assignment |
/api/fulfillment-daily-summary | Read-only | Fulfillment |
/api/health | Service + Workflow status | Health |
See the full endpoint inventory in the Scenarios Overview.
platform-api — public API
Section titled “platform-api — public API”The public-facing API for publishers, affiliate tracking, and engagement telemetry. Authenticated via API keys or OAuth — not behind CF Access.
Bindings:
| Binding | Type | Purpose |
|---|---|---|
HYPERDRIVE | Hyperdrive | PlanetScale connection pooling |
PUBLICATION_TELEMETRY | Pipeline | Stream engagement events |
ANALYTICS_ENGINE | Analytics Engine | Real-time click/impression counters |
KV | KV Namespace | Optional config cache |
Responsibilities:
- Serve public product/publication data
- Capture engagement telemetry (clicks, impressions, views)
- Handle publisher authentication and commission lookups
ingest — queue consumer and Workflow executor
Section titled “ingest — queue consumer and Workflow executor”The background processing Worker. Consumes from the ingest-queue, executes all 6 CF Workflows, handles Shopify webhooks, and pushes enriched events to Pipeline streams.
Bindings:
| Binding | Type | Purpose |
|---|---|---|
HYPERDRIVE | Hyperdrive | PlanetScale connection pooling |
R2_BUCKET | R2 Bucket | Asset uploads, JSONL archival, CSV writes |
INGEST_QUEUE | Queue (consumer) | Receive dispatched jobs |
COMMERCE_TELEMETRY | Pipeline | Stream commerce events |
PUBLICATION_TELEMETRY | Pipeline | Stream publication events |
SYNC_TELEMETRY | Pipeline | Stream Workflow audit events |
WATERMARK_DO | Durable Object | Read/write sync cursor state |
SHOPIFY_ACCESS_TOKEN | Secret | Shopify API auth |
Queue consumer config:
[[queues.consumers]]queue = "ingest-queue"max_batch_size = 10max_batch_timeout = 5Cron Triggers (defined in wrangler.toml):
| Cron | Schedule | Workflow started |
|---|---|---|
0 6 * * * | Daily 06:00 UTC | Shopify Bulk Sync |
0 * * * * | Hourly | Measurement Refresh |
0 3 * * 0 | Weekly, Sunday 03:00 UTC | Asset Health Check |
Cloudflare Workflows
Section titled “Cloudflare Workflows”All six Workflows run inside the ingest Worker. Each is a durable multi-step execution with automatic retries and replay. Persistent failures emit error events to SYNC_TELEMETRY for long-term audit retention.
Shopify Bulk Sync
Section titled “Shopify Bulk Sync”Synchronizes orders, products, and related data from Shopify via the GraphQL Bulk Operations API. See Daily Shopify Bulk Sync.
| Property | Value |
|---|---|
| Trigger | Cron (0 6 * * *) or manual via POST /api/performance-metrics/sync |
| Steps | Initiate -> Request bulk query -> Poll until complete -> Download JSONL -> Parse + upsert to PG -> Enrich + stream to COMMERCE_TELEMETRY -> Advance watermark in DO |
| Bindings | HYPERDRIVE, R2_BUCKET (archive), COMMERCE_TELEMETRY, SYNC_TELEMETRY, WATERMARK_DO, SHOPIFY_ACCESS_TOKEN |
Measurement Refresh
Section titled “Measurement Refresh”Replaces the old PG-based materialized view refresh. Runs DuckDB over R2 Iceberg tables to compute sliding-window aggregates and nominal classifications. See Pipeline.
| Property | Value |
|---|---|
| Trigger | Cron (0 * * * * — hourly) |
| Steps | Query commerce.telemetry Iceberg table -> Compute 7/14/30-day windows -> Classify against objective spec thresholds -> Write measurement.sales + measurement.performance Iceberg tables |
| Bindings | R2_BUCKET (Iceberg read/write), SYNC_TELEMETRY |
Asset Ingest
Section titled “Asset Ingest”Downloads design assets from Dropbox, uploads to R2, links to products in PG, and writes the Shopify metafield. See Asset Ingest.
| Property | Value |
|---|---|
| Trigger | API (POST /api/assets/ingest) -> Queue -> Workflow |
| Steps | Download from Dropbox -> Content hash + dedup -> Upload to R2 -> Link product in PG -> Write Shopify metafield |
| Bindings | HYPERDRIVE, R2_BUCKET (asset upload), SYNC_TELEMETRY, SHOPIFY_ACCESS_TOKEN |
CSV Export
Section titled “CSV Export”Generates downloadable CSV files from domain queries and stores them on R2. See CSV Export.
| Property | Value |
|---|---|
| Trigger | API (POST /api/csv/generate) -> Queue -> Workflow |
| Steps | Query PG domain tables -> Format CSV -> Upload to R2 -> Record artifact in PG |
| Bindings | HYPERDRIVE, R2_BUCKET (CSV upload), SYNC_TELEMETRY |
Suggestion Generation
Section titled “Suggestion Generation”Generates combo suggestions from measurement data and performance metrics. See Combo Suggestions.
| Property | Value |
|---|---|
| Trigger | API (POST /api/combo-suggestions/generate) -> Queue -> Workflow |
| Steps | Query measurement.performance Iceberg table -> Generate suggestion artifacts -> Write macrodata_artifact records to PG |
| Bindings | HYPERDRIVE, R2_BUCKET (Iceberg read), SYNC_TELEMETRY |
Combo Import
Section titled “Combo Import”Parses uploaded CSV files and upserts combo report artifacts. See Combo CRUD.
| Property | Value |
|---|---|
| Trigger | API (POST /api/combo-logs/import) -> Queue -> Workflow |
| Steps | Parse CSV payload -> Validate rows against schema -> Upsert macrodata_artifact (kind: combo_report) records in PG |
| Bindings | HYPERDRIVE, SYNC_TELEMETRY |
Queues
Section titled “Queues”ingest-queue
Section titled “ingest-queue”A single Cloudflare Queue with a job_type discriminated union (ADR-005). The dashboard-api Worker produces messages; the ingest Worker consumes them.
Configuration:
| Setting | Value |
|---|---|
| Queue name | ingest-queue |
| Max batch size | 10 |
| Max batch timeout | 5 seconds |
| Retry limit | 3 (default) |
| Dead Letter Queue | Recommended for poison message handling |
Job types (Zod discriminated union):
job_type | Dispatched by | Workflow started |
|---|---|---|
SHOPIFY_BULK_SYNC_START | Cron / API | Shopify Bulk Sync |
ASSET_INGEST | POST /api/assets/ingest | Asset Ingest |
DOCUMENT_ARTIFACT_GENERATE | POST /api/csv/generate | CSV Export |
MEASUREMENT_DATASET_REFRESH | Cron | Measurement Refresh |
SUGGESTION_GENERATE | POST /api/combo-suggestions/generate | Suggestion Generation |
COMBO_IMPORT | POST /api/combo-logs/import | Combo Import |
All messages flow through the same queue and share retry/DLQ configuration. The trade-off is noisy-neighbor risk if one job type floods the queue, mitigated by batch size configuration.
R2 Buckets
Section titled “R2 Buckets”Three logical R2 buckets (may be consolidated into fewer physical buckets with prefix-based separation).
Design assets bucket
Section titled “Design assets bucket”Stores product design images. Served via a public R2 custom domain.
| Property | Value |
|---|---|
| Path pattern | assets/{hash_prefix}/{filename} |
| Public URL | https://assets.bluntcases.com/{key} |
| Access | Worker binding (env.R2_BUCKET) + public custom domain |
| Used by | Asset Ingest |
| Content types | PNG, JPG, SVG, PDF, WebP |
JSONL archive bucket
Section titled “JSONL archive bucket”Stores raw Shopify bulk operation results for audit and replay, plus generated CSV exports.
| Property | Value |
|---|---|
| Path pattern (archives) | archives/ingest/{run_id}.jsonl |
| Path pattern (exports) | exports/csv/{export_id}.csv |
| Access | Worker binding only (not public) |
| Used by | Bulk Sync, CSV Export |
Iceberg data catalog (R2 Data Catalog)
Section titled “Iceberg data catalog (R2 Data Catalog)”Managed Apache Iceberg tables on R2, written by Cloudflare Pipelines and the Measurement Refresh Workflow. Queryable via the Iceberg REST catalog interface by DuckDB, Spark, Trino, or Cloudflare R2 SQL.
| Iceberg table | Source | Partitioning | Description |
|---|---|---|---|
commerce.telemetry | COMMERCE_TELEMETRY Pipeline | Daily | Order lifecycle events |
publication.telemetry | PUBLICATION_TELEMETRY Pipeline | Hourly | Engagement events (clicks, impressions, views) |
sync.telemetry | SYNC_TELEMETRY Pipeline | Daily | Workflow execution audit trail |
measurement.sales | Measurement Refresh Workflow | Daily | Daily sales aggregates by product x day |
measurement.performance | Measurement Refresh Workflow | Daily | Aggregated metrics with sliding windows + nominal classification |
configuration.objective_spec | Replicated from PG | Unpartitioned | Classification thresholds (co-located with analytics for query performance) |
See Pipeline for details on each table.
Cloudflare Pipelines
Section titled “Cloudflare Pipelines”Three dedicated streaming ingestion pipelines (ADR-008). Workers push enriched events via env.STREAM.send(); Pipelines applies stateless SQL transforms and lands Iceberg/Parquet tables on R2 Data Catalog.
COMMERCE_TELEMETRY
Section titled “COMMERCE_TELEMETRY”| Property | Value |
|---|---|
| Events | order_created, order_completed, payment_captured, payment_refunded, fulfillment_shipped, return_requested |
| Volume | Medium (order throughput) |
| Iceberg table | commerce.telemetry |
| Partitioning | Daily |
| Producer | ingest Worker (after PG domain write + enrichment) |
PUBLICATION_TELEMETRY
Section titled “PUBLICATION_TELEMETRY”| Property | Value |
|---|---|
| Events | click, impression, view, engagement, conversion — enriched with publisher/item/program context |
| Volume | High (every visitor interaction) |
| Iceberg table | publication.telemetry |
| Partitioning | Hourly |
| Producer | platform-api Worker (may skip PG for raw engagement events) |
SYNC_TELEMETRY
Section titled “SYNC_TELEMETRY”| Property | Value |
|---|---|
| Events | Workflow completions with record counts, watermark positions, durations |
| Volume | Low (operational audit trail) |
| Iceberg table | sync.telemetry |
| Partitioning | Daily |
| Producer | ingest Worker (at Workflow step boundaries) |
Beta constraints (as of Pipelines open beta): 20 streams per account, 5 MB/s per stream, 1 MB max payload per request. The platform uses 3 of the 20 available streams.
Durable Objects
Section titled “Durable Objects”Watermark/cursor state
Section titled “Watermark/cursor state”A Durable Object singleton manages incremental sync state for the Shopify Bulk Sync Workflow (ADR-009). This keeps pipeline orchestration state out of PlanetScale entirely.
Stored state:
{ "key": "orders_last_watermark", "value_json": { "ts": "2026-02-19T06:00:00Z" }}Behavior:
- Read at Workflow start to determine the
updated_atfilter for Shopify bulk queries - Advanced only after successful Workflow step completion
- Hard floor: 128 days back — watermarks older than this reset to
NOW() - 128 days - Historical loads beyond 128 days use a separate full backfill Workflow
Binding:
[durable_objects]bindings = [ { name = "WATERMARK_DO", class_name = "WatermarkDO" }]KV Namespaces
Section titled “KV Namespaces”Config cache (optional)
Section titled “Config cache (optional)”A KV namespace for caching configuration data and feature flags. This is optional — the system works without it, falling back to direct PG reads.
| Property | Value |
|---|---|
| Binding | KV |
| Use cases | Classification thresholds, feature flags, cached lookup data |
| TTL | Application-defined per key |
| Consumers | dashboard-api, platform-api |
[[kv_namespaces]]binding = "KV"id = "your-kv-namespace-id"Workers Analytics Engine
Section titled “Workers Analytics Engine”Real-time counters for high-frequency events that need sub-second reads but not long-term Iceberg retention. Writes happen in parallel with Pipeline stream pushes — not sequentially.
| Property | Value |
|---|---|
| Binding | ANALYTICS_ENGINE |
| Data points | 20 blobs + 20 doubles per data point |
| Retention | 3 months (automatic) |
| Sampling | Automatic at high volume |
| Counters | Clicks, impressions, page views |
| Query | SQL API or Grafana integration |
| Consumers | platform-api Worker |
The Analytics Engine handles the real-time read path (dashboards, live counters), while Pipelines handles the long-term retention path (Iceberg tables for historical analysis). See Pipeline: Real-time counters.
Cloudflare Pages
Section titled “Cloudflare Pages”Dashboard SPA
Section titled “Dashboard SPA”The React frontend, built with Vite and deployed to Cloudflare Pages.
| Property | Value |
|---|---|
| Build | cd client && npm run build (Vite) |
| Routing | SPA fallback — all paths resolve to index.html |
| API proxy | Pages Functions or custom domain routing to dashboard-api Worker |
| Auth | CF Access gate sits in front of Pages (user authenticates before the app loads) |
Docs site
Section titled “Docs site”This Astro Starlight documentation site, also deployed to Cloudflare Pages.
Cloudflare Access
Section titled “Cloudflare Access”Cloudflare Access is the single authentication layer (ADR-001). There is no auth UI in the application itself. See Authentication for the full details.
Access application
Section titled “Access application”| Property | Value |
|---|---|
| Identity provider | Google Workspace SSO |
| Protected resources | Dashboard SPA (Pages), dashboard-api Worker |
| Token delivery | Cf-Access-Jwt-Assertion header |
| RBAC | Not in v1 — all authenticated staff have equal access |
Access policies
Section titled “Access policies”| Policy | Scope | Rule |
|---|---|---|
| Staff application | Pages + dashboard-api routes | Google SSO, organization email domain |
| Service token | Cron, queue, admin automation | Access service token (machine-to-machine) |
| Webhook bypass | Shopify webhook routes on ingest Worker | Bypasses CF Access (external services cannot do Google SSO) |
Key bindings
Section titled “Key bindings”| Binding | Type | Purpose |
|---|---|---|
CF_ACCESS_AUD | Environment variable | Access application audience tag for JWT validation |
PlanetScale
Section titled “PlanetScale”Database
Section titled “Database”A single PlanetScale PostgreSQL database stores all domain and transactional data. Workers connect via Hyperdrive for connection pooling and edge proximity.
| Property | Value |
|---|---|
| Engine | PlanetScale PostgreSQL |
| Connection | Cloudflare Hyperdrive (env.HYPERDRIVE.connectionString) |
| ORM | Drizzle (PostgreSQL dialect) |
| Tables | 153 across 25 modules (including 24 link tables) |
| Primary keys | TEXT with prefixed UIDs (ord_xxx, prd_xxx, mda_xxx) |
| Migrations | dbmate (planetscale/migrations/YYYYMMDDHHMMSS_description.sql) |
| FK enforcement | None at DB level (ADR-004) — documented in Drizzle schema only |
| Stored procedures | None (ADR-003) — all logic in application code |
Module breakdown:
| Pass | Modules | Tables | Focus |
|---|---|---|---|
| Commerce Core | 01—13, 15 | 77 | Products, orders, payments, fulfillment, promotions, regions |
| Creator Commerce | 16—22 | 44 | Publishers, ad servers, memberships, ledger, invoices, disputes, leads |
| Platform | 23 | 2 | Macrodata catalog + artifact registry |
| Links | 14 | 21 | Cross-module join tables |
See Schema Design for the full table definitions and PlanetScale Constraints for system limits that shape all pipeline operations.
Hyperdrive
Section titled “Hyperdrive”| Property | Value |
|---|---|
| Binding | HYPERDRIVE |
| Features | Connection pooling, optional query caching, edge-routed connections |
| Consumers | All three Workers (dashboard-api, platform-api, ingest) |
[[hyperdrive]]binding = "HYPERDRIVE"id = "your-hyperdrive-id"Safety limits
Section titled “Safety limits”These PlanetScale constraints shape all background job design. See PlanetScale Constraints for detailed guidance.
| Limit | Value | Design response |
|---|---|---|
| Rows per query | 100k | Cursor-based pagination |
| Rows per statement | 100k | Chunked batch writes |
| Result size | 64 MiB | Selective column queries |
| Transaction timeout | 20s | Short, independent transactions |
| Autocommit timeout | 900s | Bounded single statements |
Cron Triggers
Section titled “Cron Triggers”All cron triggers execute in the ingest Worker’s scheduled() handler. Each trigger starts a CF Workflow for the actual processing.
| Schedule | Cron expression | Workflow | Scenario | Notes |
|---|---|---|---|---|
| Daily 06:00 UTC | 0 6 * * * | Shopify Bulk Sync | Daily Sync | Incremental sync from last watermark |
| Hourly | 0 * * * * | Measurement Refresh | Pipeline | DuckDB over R2 Iceberg tables |
| Weekly Sun 03:00 UTC | 0 3 * * 0 | Asset Health Check | Asset Ingest | Verify R2 assets match DB records |
# wrangler.toml for ingest Worker[triggers]crons = ["0 6 * * *", "0 * * * *", "0 3 * * 0"]Cron triggers use UTC time. The scheduled() handler dispatches to the appropriate Workflow based on the event.cron expression.