CSV Formatting, Parsing, and Customization

Overview

By default, the CSV files generated by our reporting data feeds comply with RFC 4180, the standard specification that defines the CSV file format — including field delimiting, quoting, and escaping conventions. This includes standard doubled-quote escaping ("") for a literal quote character inside a quoted field — not backslash-escaping.

Our CSV writer produces RFC 4180-compliant output by default. Because of this, the CSV parser you use to read our data feeds should also be RFC 4180-compliant so that it correctly handles the quoting, escaping, and other formatting conventions described later in this guide.

Customization Options

We also support customizing several aspects of the CSV format to match your system's requirements, including:

• Delimiter — the character (or characters) used to separate fields. Comma (,) is the default, but other delimiters (for example a pipe |, a double pipe ||, or a tab) can be configured instead.

• Quote Mode — controls which fields get quote-encapsulated. Supported modes are:

  Quote Mode	Behavior
  MINIMAL	Quotes a field only if it contains a delimiter, quote character, or line break. (Default per RFC 4180.)
  ALL	Quotes every field.
  NON_NUMERIC	Quotes every non-numeric field.
  ALL_NON_NULL	Quotes every field that is not null.
  NONE	Never quotes fields — special characters are escaped using the escape character instead.

• Quote Character — the character used to encapsulate a field. Double quote (") is the default, but this can be changed to another character (for example, a backtick).

• Escape Character — the character used to escape a literal quote character inside a field. By default this matches the quote character itself ("), which is what produces RFC 4180's standard doubled-quote ("") escaping. If your downstream system expects a different escaping convention — for example, backslash-escaping (\") — this can be configured instead, but note that doing so moves the feed away from strict RFC 4180 compliance for quote-escaping specifically.

• Null String — the string used to represent a null value. It defaults to a blank string.

• Record Separator — the line-ending sequence between rows (for example \r\n or \n). CRLF (\r\n) is the default.

• Header Row — whether a header row is included, and whether it should be unquoted even when the quote mode would otherwise quote it. The header row is present by default.

• File Type — the output can also be generated as CSV, TSV, or another delimited plain-text format, in addition to standard CSV.

note

If you've requested any of these customizations for your feed, make sure your CSV parser is configured to match. A parser that assumes RFC 4180 defaults (comma delimiter, double-quote encapsulation, doubled-quote escaping) may not correctly parse a feed that has been customized away from those defaults.

File Format Basics

Unless a feed has been customized (see Customization Options above), you can expect the following from our standard CSV output:

• Delimiter — comma (,).
• Encoding — UTF-8, with no byte-order-mark (BOM).
• Line endings — consistent within a file (\r\n by default). Most modern parsers handle both LF and CRLF automatically.
• Header row — the first line of the file, containing the column names.

Quoting and Escaping

Per RFC 4180, a field only needs to be quoted if it contains the delimiter, the quote character, or a line break — this is what our default MINIMAL quote mode does. Other quote modes (ALL, NON_NUMERIC, ALL_NON_NULL) quote more fields than strictly necessary, but the underlying quoting and escaping rules are the same.

When a quoted field needs to contain a literal quote character, our default behavior is to double it (""), per RFC 4180. For example, a field containing 5" hose would be written as "5"" hose". If your feed has been configured with a different escape character (see Customization Options above), literal quotes will instead be represented using that character, and your parser needs to be configured to match.

Multi-line quoted fields: RFC 4180 explicitly allows a quoted field to contain literal line breaks — for example, free-text fields like notes or comments. This means a single logical CSV record can span multiple physical lines in the file. This is expected, valid CSV, not a formatting error. The important implication is for your parser: it needs to track whether it's currently inside a quoted field and only treat a line break as the end of a record when it's outside of one. A naive parser that simply splits the file by newline and then splits each line by comma will incorrectly break these records apart. Any standard, RFC 4180-compliant CSV parsing library handles this correctly by default — this is really a requirement on the parser, not something we can eliminate from the file, since the line breaks are part of the underlying data.

Parser Recommendations

• Use a standard, RFC 4180-compliant CSV parsing library rather than a hand-rolled "split by newline, then split by comma" implementation. A compliant library will correctly handle quoted fields, embedded delimiters, embedded quote characters, and multi-line quoted fields for you.
• Match your parser's configuration to your feed's actual settings. If you've requested any customization (a different delimiter, quote character, escape character, quote mode, etc.), configure your parser with the same settings. A parser left on its library's defaults may not correctly read a feed that's been customized away from ours.
• Check that your library supports the combination of settings you need. Not every CSV library lets you configure quote character and escape character independently, for example. If you're planning to request a customization, it's worth confirming your parser can support it before you do.
• Parse the raw JSON payload column as a nested step, not as part of CSV parsing. The Raw Event Payload Column is a JSON object that has been serialized to a string and placed inside a single CSV field. Let your CSV parser extract that field as a plain string first, then run a JSON parser (for example JSON.parse()) on that string to get the actual event object. Don't try to extract fields from it with string manipulation or regular expressions — the embedded JSON may itself contain commas, quotes, and line breaks.

Code samples

Python v3.8+Java v1.8+Node.js v20+

"""
Reading a myKaarma CSV data feed in Python.

The standard-library csv module is RFC 4180-compliant out of the box -- comma
delimiter, double-quote encapsulation, "" for a literal quote, and line breaks
inside quoted fields. There is nothing to configure.
"""

import csv

# newline="" is required: it stops Python translating line endings, so the line
# breaks inside quoted fields survive intact.

with open("feed.csv", newline="", encoding="utf-8") as fh:
    for record in csv.DictReader(fh):
        # add your fields handling here

FAQ

Why does the raw JSON payload column look like an escaped string instead of a JSON object?

Because it's stored as a single CSV field, the JSON has to be serialized to a string (with its own internal quotes escaped) so it fits inside that field. Once your CSV parser has extracted the field's raw string value, run it through a JSON parser to get the actual object — see Parser Recommendations above.

Can a single row in the CSV file span multiple physical lines?

Yes, if a field contains an embedded line break, RFC 4180 requires that field to be quoted, and the line break becomes part of the quoted value. See Multi-line quoted fields above.

My parser works fine on some feeds but throws errors on others — why?

This usually means the feeds have different customizations (delimiter, quote character, escape character, or quote mode) and your parser is only configured for one of them. Check the specific settings for each feed and make sure your parser configuration matches.