Part 1 · Import · 🟢 Easy · ~25 min

📖 Theory

Every dataset starts life as a file somebody exported from somewhere else. The usual next step is to write a parser: open the file, split the lines, coerce the columns, build objects, handle the one row that is different. Six months later there are eleven of those parsers, each slightly different, and nobody knows which one is right.

The MeshWeaver Data Import module takes the opposite position: you declare what the file is; the pipeline does the rest. A declaration has three parts — where the file is, which type its rows become, and how it should be merged into the store — and none of them is code that walks lines.

Here is how the Northwind Analytics app loads its reference data at startup. This is the real code from MeshWeaver.Northwind.Model, not a simplification:

public static MessageHubConfiguration AddNorthwindReferenceData(this MessageHubConfiguration configuration)
    => configuration
        .AddImport()
        .AddData(data => data
            .FromEmbeddedResource(new EmbeddedResource(MyAssembly, "Files.categories.csv"),
                                  config => config.WithType<Category>())
            .FromEmbeddedResource(new EmbeddedResource(MyAssembly, "Files.regions.csv"),
                                  config => config.WithType<Region>())
            .FromEmbeddedResource(new EmbeddedResource(MyAssembly, "Files.territories.csv"),
                                  config => config.WithType<Territory>()));

Read it as a sentence: add the import capability; the data comes from these three files; each file's rows are this type. There is no foreach, no Split, no DateTime.Parse. The reader is chosen by the file's extension (.csv, .xls, .xlsx), the columns are matched to the type's properties by name, and the resulting instances land in the workspace as ordinary typed data — the same collections the layout areas query.

The sectioned format

One import can carry several types in one file. The pipeline's own text format marks each block with a line starting with @@ and the type name, followed by a header row and the data:

@@<Type>
<header row: one column name per field>
<data row>
<more data rows…>

with the marker being the two characters @ @ immediately followed by the type name. A concrete file with a Category block and a Region block looks like this — the same one the cell below reads:

That file is exactly what an ImportRequest carries when you import from a string:

var request = new ImportRequest(fileText);          // MIME type inferred; Format defaults to Default
var response = client.AwaitResponse(request, o => o.WithTarget(new ImportAddress(2024)));
// response.Message.Log.Status == ActivityStatus.Succeeded, with one activity log per import

Every import is an activity: it succeeds or fails as a whole, and its log says which rows landed and which were rejected. That is the second thing a hand-written parser rarely gives you.

⚡ Experience it

The cell below is a tiny reader for the sectioned format — deliberately small, so you can see the shape the pipeline produces before any type is involved: a list of sections, each with a type name, a header and rows. The real reader adds quoting, both Excel formats and type coercion; the shape is the same. The cell opens with the file and the delimiter — change them and press Run.

Write a small reader for the type-marker sectioned CSV format: split the text into sections, take the first line after each marker as the header and the rest as rows, and show a grid with one line per section — type, columns, row count and the first row.

Notice what the reader never needed to know: what a Category is. The declaration config.WithType<Category>() is the only place that says so, and it is one line.

Where the decisions really live

If the parser is gone, what is left to decide? Three things, and the next lessons take one each:

1. How a string becomes a value — 2024-01-04 is a date, 32.38 is a decimal, and in a German export 32,38 is too. That is Lesson 2.
2. What a worksheet means — which row the data starts on, which rows are totals to skip, and how a lump sum spreads over its lines. That is Lesson 3.
3. How the result leaves the mesh again — as a document somebody can read without an account. That is Lesson 4.

Further reading, straight from the platform docs: Excel and CSV import is a module and the Data Import package page.

✍️ Your turn

Difficulty: 🟢 Easy — 15 minutes

The workbench asks you to write the section splitter yourself — the marker line, the header, the rows. Six checks are waiting; the stub returns nothing, so the battery ships red.

Open the workbench — Split the sections » (your editable copy is installed with the course; the worked solution is one click away).

Check yourself

Course home: Data Import & Export · Next: Lesson 2 — CSV to typed rows »