Motherload

Motherload is a set of TypeScript loaders for data formats: import a TOML, YAML, JSON5, JSONC, CSV, TSV, Markdown, text or .env file, or a JSON Schema, and get a typed module.

It's powered by TypeScript's content mappers (introduced typescript@7.1). These are like plugins for TypeScript, and it lets Motherload provide full static types for the imported data, in a way that natively integrates with tsc.

import plans from "./plans.toml";
import team from "./team.yaml";
import strings from "./strings.json5";
import posts from "./posts.csv";
import notes from "./notes.md";
import "./.env";
import user from "./user.schema.jsonc";

Hover over an import to see its type. Each type comes from the file, in the type check and in the editor, and the same imports run under Node.js, Nub and esbuild.

How it works

The type check and the editor run Motherload as a content mapper: TypeScript hands it each data file a project imports, and it returns the module's types. At run time, a preload or a bundler plugin turns the same file into a module.

Setup

Install Motherload, add it to tsconfig.json, then add it to each tool that loads your code.

Install

npm install -D motherload
# content mappers are in the TypeScript 7.1 dev builds
npm install -D typescript@next

The package needs Node 24 or later. An application that runs under the preload instead of a bundle loads Motherload at run time, so it lists motherload in dependencies.

TypeScript

Add Motherload to tsconfig.json, with the extensions it should own. The types entry types ?raw imports.

{
  "compilerOptions": {
    "types": ["motherload/client"] 
  },
  "contentMappers": [ 
    { 
      "package": "motherload", 
      "extensions": [ 
        ".toml", ".yaml", ".yml", ".json5", ".jsonc", ".env", 
        ".csv", ".tsv", ".txt", ".md", 
        ".schema.jsonc", ".schema.json5"
      ] 
    } 
  ] 
}

Type-check with --runExternalCode. Without the flag, tsc ignores content mappers.

{
  "scripts": {
    "typecheck": "tsc --noEmit --runExternalCode"
  }
}

Editors

The editor needs the TypeScript 7.1 language server, with content mappers on.

Node

Node loads Motherload with --import, which turns each data file into a module as it loads:

node --import motherload/register main.ts

Plugins

Everywhere else, a plugin turns each data file into a module: Nub loads Motherload's preload from its config file, and a bundler builds each module at build time, so a bundled app needs no --import and no preload, and Motherload stays a development dependency.

Loaders

A loader turns a file into a module. Each file is parsed when the module is built, so a bundle holds the data as an object literal and no parser.

The default export is the file's data, typed as as const would type it:

import plans from "./plans.toml";

const pro = plans.pro;
     
const pro: {
    readonly price: 12;
    readonly seats: 10;
}

Add ?raw to an import for the file's text:

import text from "./plans.toml?raw";
       
const text: string

.toml

Motherload reads TOML 1.1 with smol-toml. TOML has four kinds of date. Only an offset date-time names an instant, so only that one becomes a Date:

import release from "./release.toml";
        
const release: {
    readonly published: Date;
    readonly day: "1979-05-27";
    readonly at: "07:32:00.000";
    readonly downloads: 9007199254740993n;
    readonly ratio: number;
}

A local date-time, date or time is its ISO text. An integer that a number cannot hold exactly is a bigint.

.yaml

Motherload reads YAML 1.2 with yaml, so yes is a string. Merge keys (<<) are read, and a duplicate key is an error. A .yml file is read the same way.

import app from "./app.yaml";

const production = app.production;
         
const production: {
    readonly retries: 3;
    readonly timeout: 1.5;
    readonly host: "example.com";
}
const enabled = app.enabled;
       
const enabled: "yes"

A file of several documents (---) is the list of them:

import deploy from "./deploy.yaml";
        
const deploy: readonly [{
    readonly kind: "Service";
}, {
    readonly kind: "Deployment";
    readonly replicas: 2;
}]

.json5

Motherload reads JSON5 with json5, the JSON5 project's own parser.

import data from "./data.json5";
       
const data: {
    readonly unquoted: "single quotes";
    readonly hex: 16;
    readonly limit: number;
    readonly list: readonly [1, 2, 3];
}

.jsonc

Motherload reads JSON with comments with Microsoft's jsonc-parser. Comments and trailing commas are allowed.

import settings from "./settings.jsonc";
         
const settings: {
    readonly "editor.fontSize": 14;
    readonly "files.exclude": readonly ["dist", "node_modules"];
}

.env

Importing a .env file loads it: its variables are set on process.env when the module runs, with no dotenv and no --env-file. The import also adds each of them to the type of process.env, across the whole project, so the editor completes their names and each one is a string:

import "./.env";

process.env.D
DATABASE_URL;

const url = new URL(process.env.DATABASE_URL);
                                    
NodeJS.ProcessEnv.DATABASE_URL: string
const typo = new URL(process.env.DATABSE_URL);
Argument of type 'string | undefined' is not assignable to parameter of type 'string | URL'.
  Type 'undefined' is not assignable to type 'string | URL'.

A name the file does not set keeps the type @types/node gives every variable, string | undefined, so a misspelled one is an error wherever a string is needed.

A variable the environment already sets keeps its value. No value from the file is in the module or in a bundle. A bundle reads the file from the working directory when it runs, and loads nothing if the file is absent.

The .env entry reaches a file named .env or ending in .env, such as prod.env. A file such as .env.local ends in .local, which the entry does not reach.

.csv

Motherload reads CSV with csv-parse. The first row names the columns, and each later row is an object of strings keyed by them:

import users from "./users.csv";
       
const users: readonly {
    readonly name: string;
    readonly email: string;
    readonly zip: string;
}[]

Values stay as written, so a zip code such as 02134 keeps its leading zero. A quoted field can hold commas, doubled quotes and line breaks. Blank lines are skipped. A row whose field count differs from the header's is an error, as is an empty or repeated column name.

.tsv

A .tsv file is read the same way, with a tab between fields. Quotes are ordinary characters, so a field is exactly the text between two tabs.

import scores from "./scores.tsv";
        
const scores: readonly {
    readonly player: string;
    readonly score: string;
}[]

.txt

Importing a .txt file gives its text as a string.

import notes from "./notes.txt";
       
const notes: string

.md

Importing a .md file gives an object of two parts: frontmatter, read and typed as a .yaml file is, and md, the Markdown body without the frontmatter:

import post from "./post.md";
       
const post: {
    readonly md: string;
    readonly frontmatter: {
        readonly title: "Hello, world";
        readonly draft: false;
        readonly tags: readonly ["data", "types"];
    };
}

Each part is also a named export, as in import { frontmatter } from "./post.md", and a bundle that imports only the frontmatter leaves the body out. The frontmatter runs from a --- first line to the next --- line; a file without one has {}. Pass md to a Markdown renderer for HTML, and add ?raw to the import for the whole file.

.schema.jsonc

A .schema.jsonc file is a schema, not data. Its module is the type the schema describes and a validator for it.

import user, { type Type as User } from "./user.schema.jsonc";

const ada = user.parse(input);       // or a TypeError listing every issue
     
const ada: User

user.is(input);                      // input is User
user["~standard"].validate(input);   // Standard Schema v1
user.schema;                         // the schema, as a literal type

The schema's type is always exported as Type. The validator is compiled by ajv when the module is built, so the module imports nothing at run time, and the default export works with any library that accepts a Standard Schema. Each entry of the schema's $defs is exported as a type of its own, named in PascalCase.

• Drafts: 06, 07, 2019-09 and 2020-12, chosen by $schema.
• Formats: checked, with ajv-formats.
• References: a $ref within the file is followed; a $ref to another file or a URL is an error.
• The suffix: .schema.jsonc, or .schema.json5 for a schema written in JSON5, because TypeScript refuses a content mapper extension that ends in .json. A .schema.jsonc file is read as JSON with comments, so comments and trailing commas are allowed, and VS Code checks it against the draft its $schema names.

.schema.json5

A .schema.json5 file is a schema written in JSON5, read as a .json5 file is. Its module is the same as a .schema.jsonc file's.

import point, { type Type as Point } from "./point.schema.json5";

const origin = point.parse(input);   // or a TypeError listing every issue
       
const origin: Point

.json

Plain .json stays TypeScript's own: a content mapper cannot claim it. Turn on resolveJsonModule to import it:

import pkg from "./package.json";

.sql

.sql files are Reylo's: a content mapper that types each query from your live Postgres, MySQL or SQLite database.

Errors

Invalid data in a file is a type error. Each loader hands its parser's error to the TypeScript compiler with the file, line, column, span and message, so tsc and the editor report it where it is:

$ tsc --noEmit --runExternalCode
config.toml:3:1 - error motherload1: Invalid TOML document: trying to redefine an already defined table or value

3 port = 9090
  ~

users.csv:3:1 - error motherload1: This row has 3 fields; the header has 2.

3 Grace,grace@example.com,extra
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Found 2 errors in 2 files.

Errors  Files
     1  config.toml:3
     1  users.csv:3