Data Connectors
A data connector is the code that pulls data out of a source system and into a Curiosity workspace's graph. This section is the reference for Curiosity.Library — the .NET driver you use to define schemas, map records to nodes, mirror ACLs, and run idempotently in C#. Python bindings track the same surface.
For the hands-on walkthrough, see Custom connector from scratch. For starter templates per source system, see Connector templates. For copy-paste connectors against common file formats and databases, see Examples.
Shape of a connector
flowchart LR
Source[(Source system)] -->|paginated fetch| Fetcher
Fetcher --> Mapper[Map record → node + edges + ACL]
Mapper -->|AddOrUpdate / TryAdd / Link| Graph[(Curiosity graph)]
Mapper -->|RestrictAccessTo*| Graph
Loop[Cursor loop] -->|since: lastSync| Fetcher
Graph -->|max UpdatedAt| Checkpoint[(Checkpoint store)]
Checkpoint --> Loop
The four pieces — schema bootstrap, pure mapper, cursor loop, checkpoint store — are the same in every connector. The pages below cover each piece in detail.
In this section
- Defining schemas —
[Node],[Key],[Property],[Timestamp], edge constants. - Ingesting data —
TryAdd,AddOrUpdate,Link, batching. - Access control —
RestrictAccessToTeam,RestrictAccessToUser, users and teams. - Querying — looking up existing nodes during ingestion.
- Running connectors — tokens, env vars, execution.
- Idempotency — stable keys and re-runs.
- Performance — auto-commit, pause indexing, batch size.
- Troubleshooting — common failure modes.
- Examples — ready-made connector starters for JSON, CSV, Parquet, Excel, XML, and SQL databases.
When to use what
| Surface | Use when |
|---|---|
| Connector (this section) | Ingesting external data into the graph; runs outside the workspace. |
| Scheduled tasks | Periodic work that lives inside the workspace process. |
| Custom endpoints | Request/response logic over the graph for user-facing or service callers. |
| AI tools | Letting the chat assistant fetch data or perform actions. |