CSV Connector

Stream a CSV file into the graph using CsvHelper — the de-facto .NET CSV parser. CsvHelper handles quoting, escaping, culture-aware parsing, and direct mapping to your POCOs.

Packages

Curiosity.Library on NuGet CsvHelper on NuGet

dotnet add package Curiosity.Library
dotnet add package CsvHelper

Expected source shape

A header row plus comma-separated values, in invariant culture (dates as ISO-8601, decimals with .):

employee_id,name,department,hired_at
E-001,Alice Anders,Engineering,2022-03-14
E-002,Bob Boniface,Marketing,2021-07-09
E-003,Carla Costa,Engineering,2023-11-22

Connector code

Program.cs

using System.Globalization;
using Curiosity.Library;
using CsvHelper;
using CsvHelper.Configuration;
using CsvHelper.Configuration.Attributes;

[Node]
public class Employee
{
    [Key]       public string         EmployeeId { get; set; }
    [Property]  public string         Name       { get; set; }
    [Property]  public string         Department { get; set; }
    [Timestamp] public DateTimeOffset HiredAt    { get; set; }
}

// DTO matches the CSV header names exactly.
class EmployeeRow
{
    [Name("employee_id")] public string         EmployeeId { get; set; }
    [Name("name")]        public string         Name       { get; set; }
    [Name("department")]  public string         Department { get; set; }
    [Name("hired_at")]    public DateTimeOffset HiredAt    { get; set; }
}

using var graph = Graph.Connect(
    endpoint:      Environment.GetEnvironmentVariable("CURIOSITY_ENDPOINT")!,
    token:         Environment.GetEnvironmentVariable("CURIOSITY_TOKEN")!,
    connectorName: "csv-employees");

await graph.CreateNodeSchemaAsync<Employee>();
graph.SetAutoCommitCost(everyNodes: 10_000);

var path = args.Length > 0 ? args[0] : "employees.csv";

var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
    HasHeaderRecord    = true,
    MissingFieldFound  = null,   // tolerate extra columns
    TrimOptions        = TrimOptions.Trim,
};

using var reader = new StreamReader(path);
using var csv    = new CsvReader(reader, config);

var ingested = 0;
foreach (var row in csv.GetRecords<EmployeeRow>())
{
    graph.AddOrUpdate(new Employee
    {
        EmployeeId = row.EmployeeId,
        Name       = row.Name,
        Department = row.Department,
        HiredAt    = row.HiredAt,
    });
    ingested++;
}

await graph.CommitPendingAsync();
Console.WriteLine($"Ingested {ingested} employees from {path}");

How it works

csv.GetRecords<EmployeeRow>() returns a lazy IEnumerable<EmployeeRow> — the parser reads one row at a time, materializes the DTO, yields it, and discards it before reading the next. Memory stays flat. [Name("employee_id")] tells CsvHelper which CSV column to map to each property, decoupling the wire format from C# naming.

AddOrUpdate keyed by EmployeeId makes the run idempotent — re-running the connector after the CSV updates simply rewrites changed properties.

Common variants

Tab-separated values (TSV):

var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
    Delimiter = "\t",
    HasHeaderRecord = true,
};

No header row, positional mapping:

csv.Context.RegisterClassMap<EmployeeMap>();

class EmployeeMap : ClassMap<EmployeeRow>
{
    public EmployeeMap()
    {
        Map(r => r.EmployeeId).Index(0);
        Map(r => r.Name).Index(1);
        Map(r => r.Department).Index(2);
        Map(r => r.HiredAt).Index(3);
    }
}

Plus HasHeaderRecord = false in the config.

European decimals (, instead of .):

var config = new CsvConfiguration(CultureInfo.GetCultureInfo("de-DE"))
{
    Delimiter = ";",
};

Notes & pitfalls

Quoting and escaping. CsvHelper handles "quoted, values" and "" (escaped quote) per RFC 4180. If your source uses a non-standard scheme, configure Quote and Escape in CsvConfiguration.
Culture matters. Mixing en-US and de-DE CSVs without setting CultureInfo is the most common production bug. Pick one explicitly; don't rely on CultureInfo.CurrentCulture.
Bad rows. By default, a malformed row throws. Set csv.Context.RegisterClassMap<> with Optional() on flexible fields, or wrap the loop in a try/catch and log the offending line number (csv.Context.Parser.Row).
Stable key. If the CSV doesn't include an ID column, hash the immutable fields (see Idempotency) — don't use the line number, which changes when rows are reordered.
Encoding. Pass an explicit Encoding to StreamReader if the file isn't UTF-8. Excel-exported CSVs are often UTF-16 LE with a BOM.