Export communications

Turn the search you are looking at into a CSV file, built server-side and delivered when it is ready.

Concept

Export takes your current view - the query, the time range and the whisperers you already selected - and runs it as a background job. You do not wait on the request: a large export can run for tens of minutes, and Spider mails you when the file is ready.

Available on HTTP, stream (SSE and WebSocket), Redis, Kafka, gRPC, PostgreSQL, MySQL, DNS, MQTT and MongoDB searches, plus the correlated multi-protocol view.

Naming the export

• Name and Description are optional. The name labels the run in My exports, names the file you download, and is repeated in the mail subject so a mailbox full of notifications stays scannable.
• Notify me by email is on by default. Turn it off and the run still happens - you just follow it in My exports instead.

The column matrix

The panel opens seeded with the columns you already have on screen. Each column is one row of a matrix, reorderable by dragging its grip - the order of the rows is the order of the columns in the file.

Field	What it is
Name	The column header written into the CSV. Required.
Source	Which part of the document the path is read from - see below.
Path	Where the value lives inside that part. Required.

Expanding a row (the chevron on the right) reveals two more settings, Format and Resolve. A small dot on the row means one of them is set, so you can see at a glance which columns carry extra behaviour without expanding every row.

Source

Source	Reads from
com	Fields on the communication document itself - timestamps, whisperer, URI, status.
req	The request side.
res	The response side.
payload	The value that produced this row - the body, the Redis reply value, the Kafka record, the gRPC message.

Not every protocol offers all four:

• HTTP, Redis, Kafka offer all four.
• Stream messages (SSE and WebSocket) offer com and payload only. A frame or an event belongs to one side of the exchange, so pinning a column to req or res addresses a side the message does not have.
• gRPC offers all four, but for a reason worth knowing. A streaming gRPC communication is a single message, one side only. A unary one carries both the request and the response message on one document - and there, payload resolves to the response. If you are exporting fields out of a protobuf request body, req is the source you need.
• PostgreSQL offers com and payload only, and there is no res to offer even hidden behind a toggle: a query's DataRow values are a positional array a flat CSV row cannot carry, so the export explodes on the request side alone. payload here is the resolved SQL statement text, not a captured body.
• MySQL behaves like PostgreSQL: com and payload only, one row per communication that has SQL text, and no response payload, since result rows have per-query columns. payload is the full statement - the untruncated copy from the content store when there is one - and the bound values of a prepared statement are offered as meta.params, one JSON array with NULL as null.
• DNS explodes on the response side: one row per answer record, with payload the record data and meta.name, meta.type and meta.ttl its name, type and TTL. A communication with no answer gives a single row. Authority and additional records are not exported.
• MQTT gives one row per communication, except SUBSCRIBE and UNSUBSCRIBE: one row per topic filter, with meta.filter, meta.shareGroup, meta.requestedQos and meta.grantedQos. payload is the PUBLISH payload, read on the side that carried it (the request of a publish, the response of a delivery), with meta.size, meta.truncated and meta.encoding (utf8, or base64 for a binary payload). A CONNECT password is never exported.
• MongoDB gives one row per communication. payload is the command document on the request side and the reply documents, as one JSON array, on the response side, with meta.size and meta.truncated. Authentication payloads are never exported.

Path

Paths address the stored document, using gjson syntax rather than JSONPath - req.uri for a com column, value.items.#.sku for a payload one, where # walks into an array.

caution

A path that matches nothing produces an empty column, not an error. Nothing downstream can tell a mistyped path from a field that genuinely was not captured, so check a new path against a real communication before launching a long run.

Format

Rewrites the raw stored value into something readable in a spreadsheet. One of:

none (the default, the value as stored) · date · number · bytes · duration

Dates are written in your own timezone, not UTC.

Readable for a human, harder for a machine

Formatting is a trade-off, and which side you want depends on who opens the file.

A formatted cell is written for a reader: a size becomes 1.4 MB, a duration becomes 128 ms, a date becomes 2026-09-09 14:02:31 in your timezone, and a number carries your thousands and decimal separators. That is what you want in a report someone will actually read.

But the unit is now part of the text. 1.4 MB is a string, not a number - a spreadsheet will not sum it, a script has to strip the suffix and multiply back, and the same column can mix suffixes (980 B next to 1.4 MB) from one row to the next. A formatted date has lost its timezone offset and its sub-second precision, and a decimal comma will be read as a thousands separator by anything expecting a point.

So:

• Exporting for a person to read - a report, a mail attachment, a review - use the formats.
• Exporting to feed something else - a script, a pandas or R load, a BI tool, a re-import - leave the column on none. You get the value exactly as Spider stored it: sizes in bytes, durations in seconds, dates in ISO 8601 UTC, numbers with a plain . decimal point. Convert on your side, once, knowing the unit.

Format is per column, so a single file can do both - keep the machine-readable column and add a second, formatted one beside it under a different name.

note

An unformatted duration is written in seconds, rounded to the microsecond - 0.128437, not 0.12843699999999999. The rounding only trims float noise; it never changes a value you would notice.

Resolve

Turns an opaque id into a name. One of:

• none - the default.
• whisperer - looks each id up and writes the whisperer's name.
• controller - the same, for controllers.
• tag plugins - decodes a tag column's values through the tag plugins you have loaded, so the CSV carries the same labels the grid shows.

The tag plugins option only appears on a tag column whose values a loaded plugin was actually seen to decode. Offering it everywhere would put a control on every column that then silently resolves nothing.

Whichever you pick, the panel answers in a line under the picker, because the outcome is not otherwise visible until the file arrives:

• 143 values resolved.
• No plugin decoded these values; this column exports raw ids.
• Scope changed; this column's resolved values were dropped. Resolve again.
• When a scope holds more values than the resolver's cap, the line says how many resolved and that the rest export as raw ids.

note

A resolve built from a custom id-to-name map is not stored with the export. Reopening such an export with Export like this seeds the column with its resolve dropped, and says so - resolve it again, or the ids export unresolved.

Adding columns

The Add a column block offers the fields Spider indexes, grouped by where they come from - communication, request content, response content, message content. Pick one and it becomes a new row, with its name, source and path filled in.

Custom path... adds an empty row instead, for a field you know is there and want to address yourself.

Adding fields from the payloads

Fields inside request and response bodies are not indexed, so they are not in that list to begin with.

Scan 5 | 10 | 20 coms for body fields reads the bodies of that many real communications from your current search and adds whatever it finds to the picker above. It does not add columns - you still choose which of the discovered fields you want.

PostgreSQL, MySQL and DNS have no scan

PostgreSQL, MySQL and DNS do not offer this control: its columns already come entirely from the indexed catalog above, not from an arbitrary user-shaped body - there is nothing a scan could find that is not already listed.

The summary line reports what happened, for example:

Sampled 5 coms, 10 bodies: 6 JSON, 4 not JSON. 14 body fields are now in the list above.

A body that is not JSON contributes no fields; that is normal for a binary payload with no schema bound.

tip

For Kafka and Redis the discovered fields are labelled seen in a request / seen in a response rather than pinned to a side. The column follows whichever side the communication actually used, so the label tells you where a value like this was observed, not which side the column will read.

Where the file goes, and for how long

• The job paginates your search, joins each communication's content, and writes the CSV server-side. Nothing is assembled in your browser.
• The finished file is stored in Spider's object storage, not on the Spider nodes. Your administrator provisions that bucket at install time - see S3 bucket for com export.
• It is kept for 7 days by default. My exports shows each run's status, row count, size and its own expiry date.
• When it completes, Spider mails you a link - and mails you too if the run fails, with the reason. The mail repeats the export's name, protocol, date range, filter and whisperers, so it stands on its own.

The link is to Spider, not to the file

The mail links back into Network-View, not to the stored object. Opening it takes you to the export, and the actual download URL is minted at that moment and expires in five minutes. A forwarded mail therefore does not hand anyone a lasting URL to your data, and the link is useless to someone who cannot log in to Spider.

Your access filters - query scope, forbidden payloads, forbidden headers - apply to the file exactly as they do on screen. An export can never contain something you could not see in the grid.

Limits

Limit	Value
Communications per export	1,000,000
File size	5 GiB
Exports per user per day	50
Exports running at once, per install	2

Bulk export requires the EXPORT licence entitlement, available from the TEAM tier up. Writing decoded protobuf or Avro bodies additionally requires TRANSCODE; without it the export still runs, it just writes those bodies as raw bytes.

The feature can be turned off for an install with global.flips.comExport: false.