Asset Ingest

Overview

Backup and normalize creative assets: a workflow (ProcedureExecution) that ingests from Dropbox, creates an artifact record stored as an R2 object (Information Bearing Entity), deduplicates by content hash (SHA-256), and establishes a product-artifact link to the target product.

Goal: Reliable asset redundancy and a canonical bearer URL per product.

API endpoints

Method	Path	Spec	Description
POST	/creative-ops/design-assets/upload	creative-ops.yaml	Upload a single design asset to R2
POST	/creative-ops/design-assets/bulk-upload	creative-ops.yaml	Batch upload design assets
GET	/creative-ops/design-assets	creative-ops.yaml	List design assets
GET	/creative-ops/design-assets/{asset_id}	creative-ops.yaml	Get asset detail with signed R2 download URL
POST	/creative-ops/design-assets/{asset_id}/link-products	creative-ops.yaml	Link products to a design asset
DELETE	/creative-ops/design-assets/{asset_id}/link-products/{product_id}	creative-ops.yaml	Unlink a product from an asset

Schema tables

Table	Module	Role
macrodata_artifact (kind: design_asset)	Macrodata	Write: design asset record with hash, bearer URL, metadata
link_product_macrodata_artifact	Macrodata	Write: product-to-asset link

Infrastructure

Resource	Type	Purpose
internal	Worker	API validation, job enqueue
ingest	Worker	Workflow executor: download, hash, upload, record
CF Queue	Queue	ICE_INGEST job delivery
R2 bucket	Object storage	Stores design asset files

Participants

Actor	Role
Staff user	Submits artifact_identifier + Dropbox URL
Dashboard API Worker	Validates, enqueues job
Ingest Worker	Workflow: downloads, uploads, updates
Dropbox public link	Source of design asset file
R2	Object storage for design assets
Shopify Admin API	Metafield update target
PlanetScale	Database for artifact records + product links

Main success scenario

1. Staff calls POST /api/assets/ingest
2. API validates Access JWT and enqueues ICE_INGEST, returns 202 Accepted
3. Step: Download — Ingest Worker downloads asset from Dropbox link (converts dl=0 → dl=1)
4. Step: Hash — Worker computes content hash (SHA-256)
5. Step: Deduplicate — Worker checks macrodata_artifact.content_hash_identifier (kind: design_asset) for existing match — if found, reuses existing record
6. Step: Store — Worker uploads to R2 via binding API (env.R2_BUCKET.put()) — creates the stored file (R2 object)
7. Step: Record — Worker upserts:
   • macrodata_artifact(content_hash_identifier, bearer_entity_key, bearer_entity_url, file_size, content_type, dropbox_url) (kind: design_asset) — the design asset record
   • link_product_macrodata_artifact(product_id, macrodata_artifact_id) — product-artifact link
8. Step: Metafield — Worker updates Shopify metafield custom.production_asset_url with bearer_entity_url

sequenceDiagram
  participant User as Staff
  participant API as Dashboard API
  participant Q as CF Queue
  participant Ingest as Ingest Worker
  participant Dropbox as Dropbox
  participant R2 as R2
  participant DB as PlanetScale
  participant Shopify as Shopify API

  User->>API: POST /api/assets/ingest
  API->>Q: enqueue ICE_INGEST
  API-->>User: 202 Accepted + job_id
  Q->>Ingest: deliver
  Ingest->>Dropbox: fetch asset stream
  Ingest->>Ingest: compute content hash (SHA-256)
  Ingest->>R2: put(R2 object)
  Ingest->>DB: upsert macrodata_artifact + link_product_macrodata_artifact
  Ingest->>Shopify: update metafield (bearer_entity_url)

Race condition handling

If two concurrent workflow executions produce the same content hash:

1. First write succeeds (UNIQUE constraint on content_hash_identifier)
2. Second write gets a duplicate key error
3. Recovery: handler catches the error, fetches existing macrodata_artifact by content hash
4. Reuses existing macrodata_artifact id for the product-artifact link

Exception flows

Error condition	Recovery
Dropbox link invalid/expired	Mark workflow failed; no DB writes beyond failure record
R2 upload fails	Retry with backoff; Worker CPU limit allows up to 5 minutes
Shopify metafield update fails	Retry; idempotent (same URL set = safe)
Duplicate content hash (race)	Fetch existing macrodata_artifact, reuse ID for product-artifact link

Queue contract

{
  "job_type": "ICE_INGEST",
  "job_id": "uuid",
  "artifact_identifier": "string",
  "dropbox_url": "string",
  "original_filename": "string",
  "requested_at": "ISO8601"
}

Acceptance criteria

• PASS: Given product_id + dropboxUrl, system produces stable bearer_entity_url; product-artifact link exists; Shopify metafield equals the URL
• FAIL: Duplicate macrodata_artifacts created for identical content hash; metafield not updated after retries