Running Connectors

A connector runs anywhere it can reach the workspace over HTTPS — a CI job, a Kubernetes CronJob, a developer laptop. The only thing it needs from the workspace is a token.

1. Mint the tokens

Two tokens, for two different purposes:

Token	Created under	Used for
API token	Settings → API integrations	Writing data into the graph from a connector.
Endpoint token	Settings → Tokens	Calling custom endpoints from external systems.

For a vanilla ingestion connector you only need the API token. Scope it to ingestion only — there's no reason a connector should be able to manage tokens or read user data.

2. Set environment variables

The Curiosity starter templates expect:

Variable	Example	Meaning
`CURIOSITY_ENDPOINT`	`https://workspace.example.com`	Workspace URL.
`CURIOSITY_TOKEN`	`eyJ...`	API token from step 1.
`CURIOSITY_ENDPOINTS_TOKEN`	`eyJ...`	Endpoint token (only if calling endpoints from the connector).

Inject from a secret manager in production (Vault, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager). Never commit tokens to git.

3. Run

CURIOSITY_ENDPOINT="https://workspace.example.com" \
CURIOSITY_TOKEN="eyJ..." \
dotnet run --project MyConnector.csproj

The connector should print its progress to stdout; pipe to your aggregator in production.

4. Schedule

A connector that runs once is a backfill. For ongoing sync, run it on a schedule. Two common shapes:

Outside the workspace. GitHub Actions, GitLab CI, Kubernetes CronJob, plain cron. The connector reads the cursor from durable storage (S3, a database, an on-disk volume mounted into the container), runs once, persists the new cursor.

# Kubernetes CronJob — runs every 15 minutes.
apiVersion: batch/v1
kind: CronJob
metadata: { name: news-connector }
spec:
  schedule: "*/15 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
            - name: connector
              image: my-org/news-connector:1.4.0
              envFrom:
                - secretRef: { name: curiosity-token }

Inside the workspace. Package the same code as a Scheduled task. Runs in-process, logs flow through the workspace's normal logging surface, and the cursor lives as a graph node.

5. Validate the run

Inside the workspace shell:

return new
{
    Counts    = Q().EmitSummary(),
    LatestCase= Q().StartAt("SupportCase")
                   .SortByTimestamp(oldestFirst: false)
                   .Take(1)
                   .Emit("N"),
};

For automated validation, return the same shape from a scheduled task and write it to a log node — operators can dashboard freshness without poking around.

Cross-links

Custom connector from scratch — full reference loop.
Schemas, Ingestion, Access control, Idempotency.
Scheduled tasks — running connectors inside the workspace.
Production deployment checklist — token rotation, secret discipline.

Referenced by

index