Onboard a client workflow¶

A tenant workflow is the complete, versioned logical graph from sources to governed actions. The YAML references reusable catalog implementations and explicit client bindings; it does not contain credentials, provider payloads, recommendation instances or execution history.

Live demo tenants¶

Three published accounts show the workflows we sell:

• faf5b57c-57f8-41f0-b373-e5906e55b7fb — Shopify + Klaviyo + Google Ads. Identity and graph, LTV and propensity, one Customer Match audience. North star is revenue. Growth is the lead. Media is media_strategist@1.3.0.
• 8724ac3d-e12f-4db5-abfa-d3c16a1656e0 — GTM only. Container snapshot plus live collection events roll up error_rate (cap 10%), event_volume, and error_events. Analytics (analytics_specialist@1.3.0) is the lead and writes a gtm_tag_change. No identity, graph, scores, audiences, or Ads. Unset Insights pages say that those pieces were not configured, not that the pipeline failed.
• a855e4c0-308f-4dac-a9f7-70ec20dead7a — 31-day organic + checkout bench. Shopify + GA4 + Search Console + page inspect. SEO + CRO + Forecast + Growth. No Ads, CRM, or GTM. Site fixes are recommend-only.

The ecommerce catalog example used by tests and bootstrap lives in optimization/templates/ecommerce.yaml.

flowchart LR
  gtmSnap[GTM_container_snapshot] --> configChecks[Config_checks]
  gtmEvents[GTM_collection_events] --> eventHealth[Event_health]
  configChecks --> breaking[tracking.breaking]
  eventHealth --> breaking
  breaking --> analytics[Analytics_specialist_lead]
  analytics --> reco[Finding_with_GTM_diff]
  reco --> gtmFix[Approved_gtm_tag_change]

Current catalog example¶

flowchart LR
  GA4[GA4_BigQuery] --> Graph[Identity_and_graph]
  Ads[Google_Ads_BigQuery] --> Graph
  Graph --> Propensity[Purchase_propensity]
  Graph --> LTV[LTV]
  Propensity --> Media[Media_specialist]
  LTV --> Media
  AdsTool[Google_Ads_account_snapshot] --> Media
  Media --> Growth[Growth_lead]
  Growth --> Audience[Approved_Google_Ads_audience]
  GTM[GTM_snapshot_and_checks] --> Analytics[Analytics_specialist]
  Analytics --> Growth
  Growth --> TagChange[Approved_GTM_workspace_change]

The Media path uses model outputs:

• propensity@1.1.0 provides likelihood.purchase;
• ltv@2.0.0 provides value.lifetime;
• the resolver enables high_value_high_propensity;
• the Media specialist proposes a typed audience action;
• the user approves the exact account/resource target;
• a worker executes it and stores the provider receipt and rollback mode.

The Analytics path is independent of customer models:

• the GTM snapshot runs deterministic checks;
• a breaking finding publishes tracking.breaking;
• the resolver enables repair_broken_tracking;
• the Analytics specialist proposes an exact workspace/tag diff;
• the user approves before the workspace mutation;
• publication follows account policy and a container version supports rollback.

Projected impact is not measured incrementality. Causal measurement is designed separately.

Tenant YAML logic¶

tenant_id: client_123

goals:
  north_star: total_revenue

connections:
  - id: warehouse
    kind: bigquery
    roles: [source]
  - id: ads_account
    kind: google_ads
    roles: [source]
  - id: gtm_container
    kind: gtm
    roles: [source]

actions:
  - id: deliver_high_value_audience
    ref: google_ads_audience_delivery@2.0.0
    connection: ads_account
    approval: required
    rollback: unsupported
  - id: repair_gtm_tag
    ref: gtm_tag_change@2.0.0
    connection: gtm_container
    approval: required
    rollback: version_restore

use_cases:
  - id: activate_high_value
    ref: high_value_high_propensity@1.0.0
    specialist: media
    actions: [deliver_high_value_audience]
  - id: repair_tracking
    ref: repair_broken_tracking@1.0.0
    specialist: analytics
    actions: [repair_gtm_tag]

nodes:
  - id: ga4
    type: source
    ref: ga4@1.0.0
  - id: google_ads
    type: source
    ref: google_ads@1.0.0
  - id: propensity
    type: algo
    ref: client_purchase_propensity@2.0.0
    runner: external_table
    depends_on: [graph]
  - id: ltv
    type: algo
    ref: client_ltv@1.0.0
    runner: package
    depends_on: [graph]
  - id: media
    type: agent
    ref: media_strategist@1.3.0
    depends_on: [propensity, ltv, google_ads]
  - id: analytics
    type: agent
    ref: analytics_specialist@1.3.0
  - id: growth
    type: agent
    ref: growth_agent@1.3.0
    depends_on: [media, analytics]

Important distinctions:

• ref selects an immutable catalog implementation/version.
• connection selects a tenant-owned logical connection; secrets remain encrypted in the database.
• the action worker resolves the Ads account or GTM container currently selected on that connection and validates resource paths against it; agents cannot redirect execution by supplying another account ID.
• generated sections come from onboarding and connection capabilities;
• explicit tenant overrides are preserved during recompilation;
• validation rejects unknown refs, missing signals/actions, unsupported rollback and unavailable connections;
• every active workflow and run has an immutable configuration hash.

Add a new client¶

1. Record the business objective, primary KPI, units and guardrails.
2. Connect warehouse and tool accounts; select explicit Ads accounts, Analytics properties and GTM containers.
3. Verify source grain, history, identity, schema, freshness and permissions.
4. Register custom algorithms and semantic outputs.
5. Select Media, CRM and Analytics use cases.
6. Select permitted typed actions, approval policy and supported rollback.
7. Compile a tenant YAML draft from catalog refs and live capabilities.
8. Validate schema, dependencies, refs, accounts, permissions and quality gates.
9. Run data/models/checks and recommendations in shadow mode.
10. Dry-run provider diffs, then validate in a sandbox or GTM workspace.
11. Obtain human approval and activate the immutable workflow version.
12. Monitor manifests, recommendations, executions, receipts and rollback.

Where it runs¶

• Laptop or a separate process (ADC, OPTIMIZATION_ALLOW_PIPELINE=1): ingest → identity → graph → algos/score → traits → audiences → agents. All make client-*, make optimization*, python -m optimization --step …, --worker-once for pipeline jobs, --rebuild-ready-index.
• VM: API reads the store; action_worker executes approved Ads/GTM actions. Tool snapshots refresh on Cloud Run (--mode refresh_tools), not on the VM. No pipeline consume. make vm-up does not start the optimization worker. Do not pass --local if the UI must see the run (the UI reads GCS).

Client commands always require TENANT=<uuid> and AS_OF=YYYY-MM-DD (UTC day the run is scored as). SINCE is optional and bounds ingest.

How to refresh each part¶

The chain is:

ingest → identity → graph → score|algos → traits → audiences → agents

score predicts from a frozen model release (no LightGBM). algos fits and publishes a new release. Closed history is immutable; later modes recompute only the window that could have changed.

opt() { OPTIMIZATION_ALLOW_PIPELINE=1 uv run --extra optimization python -m optimization "$@"; }

• First load (train + first cards). Raw data already landed on the tenant connections. make client-onboard TENANT=$TENANT AS_OF=$AS_OF → ingest, identity, graph, algos, traits, audiences, agents. Required before client-daily will plan, because daily scores against those releases.
• Normal day (no refit). make client-daily TENANT=$TENANT AS_OF=$AS_OF SINCE=YYYY-MM-DD → ingest with a 3-day overlap (or SINCE), delta identity, graph append, score, traits, audiences, agents. Agents skip the LLM when their input digest is unchanged. Without SINCE, a warehouse stream with no watermark backfills ~24 months.
• New hive days, dashboard actuals only. opt --tenant $TENANT --mode refresh_kpis --as-of $AS_OF → identity + graph. No LightGBM, no inbox rewrite.
• Ingest only. opt --tenant $TENANT --mode refresh_data --as-of $AS_OF. Then refresh_kpis, client-daily, or client-retrain depending on whether models must move.
• Refit models (weekly, coverage loss, algo/config change). make client-retrain TENANT=$TENANT AS_OF=$AS_OF → identity, graph, algos, traits, audiences, agents. Reuses raw extracts; does not ingest.
• Inbox / audiences only, models still valid. opt --tenant $TENANT --mode recommend --as-of $AS_OF → traits, audiences, agents. Rejected if source or workflow hash moved.
• One stage while debugging. opt --tenant $TENANT --step ingest|identity|graph|algos|score|traits|audiences|agents --as-of $AS_OF. Do not pass --mode. --from identity continues from mid-graph. --resume-run-id retries a partial run with a fresh stage budget.
• Tool snapshots (GTM / Ads / GA / Sheets). Not a pipeline stage. Cloud Run --mode refresh_tools on optimization-daily, or make tool-snapshot once on a laptop. Unblocks Media/Analytics cards that read those snapshots; rerunning recommend cannot fetch OAuth.
• Approved Ads / GTM mutation. VM action_worker (lists ready/kind=action/ only). Laptop: make action-worker-once. Dismiss stays inbox-only.
• Draft before the live workflow. make workspace-create WORKSPACE=… then workspace-run MODE=retrain|recommend then workspace-promote. Provider execution stays off until promote.
• Queue inspection (pipeline jobs, laptop). opt --tenant $TENANT --jobs / --expire-stale-jobs / --worker-once. --rebuild-ready-index walks historical job.json once; do not run it on the VM.

Do not guess the mode. recommend after a source or workflow change is rejected. OAuth, account selection, or missing scopes are not repaired by any optimization rerun.

Exploration, draft, and promotion¶

Demos and customers use the same lifecycle. Synthetic generation is only another source adapter.

1. Fetch or append source data. Exploration publishes privacy-safe source profiles (entities, schema, grain, date range, row counts, null/uniqueness, identity/consent coverage, candidate KPI semantics). Profiles never include raw customer samples.
2. Compile a draft workspace from those profiles and connection capabilities. Compilation does not activate the live workflow.
3. Queue a selected run mode against the isolated workspace. Draft recommendations are previews; provider action execution stays disabled until promotion.
4. Inspect artifacts and recommendations in the workspace, then promote only after a successful validation run whose workflow hash matches the draft.

make workspace-create WORKSPACE=client-draft
make workspace-run WORKSPACE=client-draft MODE=retrain
make workspace-run WORKSPACE=client-draft MODE=recommend
make workspace-promote WORKSPACE=client-draft

GTM monitoring tenant (8724ac3d-e12f-4db5-abfa-d3c16a1656e0)¶

A repo edit of tenants/8724….yaml does not change production until it is published. The store copy is what workers load.

From a laptop with ADC (gcloud auth application-default login), never on the VM:

TENANT=8724ac3d-e12f-4db5-abfa-d3c16a1656e0
AS_OF=$(date -u +%F)
opt() { OPTIMIZATION_ALLOW_PIPELINE=1 uv run --extra optimization python -m optimization "$@"; }

# 1. Replace the live workflow (drops Shopify audiences / Ads sync).
make tenant-config-publish TENANT=$TENANT

# 2. Refresh GTM container + check snapshots (Cloud Run refresh_tools, or once here).
make tool-snapshot

# 3. After a workflow hash change, daily is the full GTM path:
#    score (collection KPI rollup, including error_rate) + Analytics agents.
#    ingest / identity / graph / traits / audiences are not pinned and are skipped.
opt --tenant $TENANT --mode daily --as-of $AS_OF

End-to-end recommendation test:

1. Connect GTM OAuth on that site (Settings → Integrations). tool-snapshot then writes container checks.
2. Collection events must be in the events hive (source=gtm). Quality errors (missing event_name, purchase without value/currency) raise error_rate. Over the 10% cap the rollup publishes tracking.breaking.
3. repair_broken_tracking becomes eligible only while that check is active. Analytics writes a finding with a gtm_tag_change intent.
4. Approve the card in the inbox. make action-worker-once (laptop) or the VM action_worker applies the workspace edit. You publish in GTM.

Do not run make optimization / demo-refresh on this tenant — those generate the ecommerce template, not GTM.

Organic + checkout tenant (a855e4c0-308f-4dac-a9f7-70ec20dead7a)¶

31 days of Shopify + GA4 + Search Console + one page-inspect snapshot. SEO and CRO write memos (catalog plays or observation: cards). Forecast calibrates catalog plays only. Growth is the lead. No Ads or GTM.

TENANT=a855e4c0-308f-4dac-a9f7-70ec20dead7a
AS_OF=$(date -u +%F)
opt() { OPTIMIZATION_ALLOW_PIPELINE=1 uv run --extra optimization python -m optimization "$@"; }

make tenant-config-publish TENANT=$TENANT
opt --tenant $TENANT --reset-synthetic   # required if a prior generate wrote a 730-day shop spine
opt --tenant $TENANT --step generate --as-of $AS_OF --customers 250
opt --tenant $TENANT --mode daily --as-of $AS_OF

Grant synthetic_ingest in OPTIMIZATION_TENANT_CAPABILITIES. Inbox cards are recommend-only (query-to-landing mismatch, checkout leak, plus digest observations). Do not run make optimization on this tenant.

Test tenant only¶

The ecommerce generate/refresh commands default to 8724ac3d-e12f-4db5-abfa-d3c16a1656e0 but that live account is now the GTM workflow. They read optimization/templates/ecommerce.yaml and must not be used on a client.

make optimization          # generate + full workflow (only path that creates data)
make demo-refresh          # full rebuild over data already generated
make demo-reset            # clear synthetic state
make optimization-refresh-data
make optimization-refresh-kpis
make optimization-retrain
make optimization-recommend

Seed or population changes need make demo-reset first. FROM=generate and FROM=activate remain compatibility aliases. --local writes ./data/events for worker tests; the UI will not see that run.

Prompt template for a workflow draft¶

Copy this prompt and replace every bracketed value. Do not include secrets.

Draft a tenant workflow YAML for Adwize from this onboarding brief.

Tenant:
- tenant_id: [TENANT_ID]
- objective: [BUSINESS_OBJECTIVE]
- primary KPI, unit and definition: [KPI]
- guardrails: [GUARDRAILS]

Available source connections:
[CONNECTION_IDS, KINDS, DATASET/SOURCE BINDINGS, FRESHNESS]

Available tool bindings:
[CONNECTION_ID, PROVIDER, EXPLICIT ACCOUNT/PROPERTY/CONTAINER ID, READ/WRITE CAPABILITIES]

Algorithms:
[VERSIONED REF, EXECUTION MODE, ENTITY GRAIN, OBJECTIVE, SEMANTIC OUTPUTS,
HORIZON, REQUIRED SOURCES, QUALITY/FRESHNESS GATES]

Desired use cases:
[USE CASES FOR MEDIA, CRM, ANALYTICS]

Allowed actions:
[VERSIONED ACTION REF, CONNECTION, EXPLICIT PROVIDER ACCOUNT/RESOURCE,
APPROVAL POLICY, SUPPORTED ROLLBACK MODE]

Requirements:
1. Use only version-pinned catalog refs confirmed in the supplied catalog.
2. Define the complete logical graph: sources, foundations, checks, algorithms,
   agents, use cases, actions, dependencies, KPI and policies.
3. Never include credentials, tokens, secret values, provider API payloads,
   recommendation instances or execution state.
4. Do not invent sources, semantics, permissions, accounts, actions or rollback.
5. Mark every missing input as MISSING and do not make that path executable.
6. Explain generated values versus explicit tenant overrides.
7. Validate the draft against the pipeline schema, catalog capabilities,
   dependency graph, connection IDs and provider account/resource ownership.
8. Return the YAML, validation result, unresolved inputs and disabled use cases.

Generated YAML is always a draft. Automated schema/capability validation and human approval are mandatory before publishing an active immutable version.