Skip to main content
API docs by Redocly

Spider API (2.14.0)

Download OpenAPI specification:Download

E-mail: contact@floocus.com
Spider Analyzer Website

Description

These are the services behind the GUI. In fact, all Spider processing is done using these APIs.

Description

Spider API allows you to:

  • configure your system (teams, users, agents)
  • search, browse and compute statistics on the resources created while parsing the network communications
  • upload new communications
  • and even write a new agent!

You may search through Packets, Tcp Sessions, Http Communications, download them, analyse them with ElasticSearch power, rebuild the content...

These are the services behind the GUI. In fact, all Spider processing is done using these APIs.

How to start?

  1. Create a Service Account on the UI
  2. Start by authenticating with POST /customer/v1/service-accounts/sessions to get your JWT token to use in all further calls.
  3. Click 'Authorize' button to register your token.
  4. Then use any customers authorized API. Ex:
    • POST /web-read/v1/http-com/_search to search for HTTP communications
    • GET /web-read/v1/http-com/{id}/res/body/ to get the response body of a communication
  5. When using a team, you may have to get the team token to access its whisperers
    • GET /teams/v1/teams/{id}/user-token/

Common

Common API for all services.

Get micro service info

Description

Give basic info, can be used for healthcheck of service

Access

  • No identification
path Parameters
service
required
string (ServiceNames)
Enum: "alert" "capture-status-poller" "ciphers" "ciphers-status" "ciphers-status-poller" "ciphers-raw-status-poller" "ciphers-status-agg" "controls" "customer" "gui-logs" "gui-settings" "hosts" "hosts-agg" "hosts-poller" "job" "link" "mail-sender" "maintenance" "pack-poller" "pack-read" "pack-write" "pack-update" "parsing-status-tcpsession-poller" "parsing-status-httppers-poller" "parsing-status-pgparsing-poller" "pg-com-poller" "pg-com-content-poller" "pg-parser" "pg-parsing-poller" "pg-read" "plugins" "session" "stats-collector" "status-poller" "tcp-poller" "tcp-read" "tcp-write" "tcp-update" "teams" "tls-keys" "tls-keys-linker" "web-httpcom-poller" "web-httpcom-content-poller" "web-httppers-poller" "web-read" "web-write" "whisp" "whisp-status-poller" "whisps-status" "whisps-status-agg"

Service name

Responses

Response samples

Content type
text/plain
Spider Customer

Get Api stats

Description

Give metrics informations for HTTP Api of this service

Access

  • Admin
  • Application
  • User
Authorizations:
Bearer
path Parameters
service
required
string (ServiceNames)
Enum: "alert" "capture-status-poller" "ciphers" "ciphers-status" "ciphers-status-poller" "ciphers-raw-status-poller" "ciphers-status-agg" "controls" "customer" "gui-logs" "gui-settings" "hosts" "hosts-agg" "hosts-poller" "job" "link" "mail-sender" "maintenance" "pack-poller" "pack-read" "pack-write" "pack-update" "parsing-status-tcpsession-poller" "parsing-status-httppers-poller" "parsing-status-pgparsing-poller" "pg-com-poller" "pg-com-content-poller" "pg-parser" "pg-parsing-poller" "pg-read" "plugins" "session" "stats-collector" "status-poller" "tcp-poller" "tcp-read" "tcp-write" "tcp-update" "teams" "tls-keys" "tls-keys-linker" "web-httpcom-poller" "web-httpcom-content-poller" "web-httppers-poller" "web-read" "web-write" "whisp" "whisp-status-poller" "whisps-status" "whisps-status-agg"

Micro service name

Responses

Response samples

Content type
application/json
{
  • "application": "tcp-streams-update",
  • "hostname": "traballand-Latitude-E7240",
  • "instanceId": "c1033254c68f",
  • "requests": 886,
  • "successes": 886,
  • "errors": 0,
  • "errors4xx": 0,
  • "errors5xx": 0,
  • "duration": 2311,
  • "api": {
    }
}

Get circuit breakers stats

Description

Give circuit breakers informations for downstream connections of this service

Access

  • Admin
  • Application
  • User
Authorizations:
Bearer
path Parameters
service
required
string (ServiceNames)
Enum: "alert" "capture-status-poller" "ciphers" "ciphers-status" "ciphers-status-poller" "ciphers-raw-status-poller" "ciphers-status-agg" "controls" "customer" "gui-logs" "gui-settings" "hosts" "hosts-agg" "hosts-poller" "job" "link" "mail-sender" "maintenance" "pack-poller" "pack-read" "pack-write" "pack-update" "parsing-status-tcpsession-poller" "parsing-status-httppers-poller" "parsing-status-pgparsing-poller" "pg-com-poller" "pg-com-content-poller" "pg-parser" "pg-parsing-poller" "pg-read" "plugins" "session" "stats-collector" "status-poller" "tcp-poller" "tcp-read" "tcp-write" "tcp-update" "teams" "tls-keys" "tls-keys-linker" "web-httpcom-poller" "web-httpcom-content-poller" "web-httppers-poller" "web-read" "web-write" "whisp" "whisp-status-poller" "whisps-status" "whisps-status-agg"

Micro service name

Responses

Response samples

Content type
application/json
{
  • "application": "tcp-streams-write",
  • "hostname": "spider4",
  • "instanceId": "9300ef2cec06",
  • "circuitBreakers": {
    }
}

Get process stats

Description

Give process metrics for this service

Access

  • Admin
  • Application
  • User
Authorizations:
Bearer
path Parameters
service
required
string (ServiceNames)
Enum: "alert" "capture-status-poller" "ciphers" "ciphers-status" "ciphers-status-poller" "ciphers-raw-status-poller" "ciphers-status-agg" "controls" "customer" "gui-logs" "gui-settings" "hosts" "hosts-agg" "hosts-poller" "job" "link" "mail-sender" "maintenance" "pack-poller" "pack-read" "pack-write" "pack-update" "parsing-status-tcpsession-poller" "parsing-status-httppers-poller" "parsing-status-pgparsing-poller" "pg-com-poller" "pg-com-content-poller" "pg-parser" "pg-parsing-poller" "pg-read" "plugins" "session" "stats-collector" "status-poller" "tcp-poller" "tcp-read" "tcp-write" "tcp-update" "teams" "tls-keys" "tls-keys-linker" "web-httpcom-poller" "web-httpcom-content-poller" "web-httppers-poller" "web-read" "web-write" "whisp" "whisp-status-poller" "whisps-status" "whisps-status-agg"

Micro service name

Responses

Response samples

Content type
application/json
{
  • "application": "tcp-streams-write",
  • "hostname": "spider7",
  • "instanceId": "dd8ff8ac5565",
  • "startTime": "2019-01-16T21:56:26.109Z",
  • "upTime": 1209512.309,
  • "cpu": {
    },
  • "memory": {
    }
}

Get parsing stats

Description

Give metrics informations for parsing

Access

  • Admin
  • Application
  • User
Authorizations:
Bearer
path Parameters
parser
required
string (ParserNames)
Enum: "web-write" "tls-keys-linker"

Micro service name

Responses

Response samples

Content type
application/json
{
  • "application": "web-streams-write",
  • "hostname": "spider4",
  • "instanceId": "cce9207215ae",
  • "parsed": 13259,
  • "created": 5005,
  • "errors": 0,
  • "completed": 856,
  • "duration": 131623.869581,
  • "started": 856,
  • "delay": 8747139,
  • "durationPercentiles": {
    },
  • "delayPercentiles": {
    }
}

Alert

Alert service.

Get metrics

Description

Give metrics information collected by alerting probes in Prometheus format.

Access

  • Free

Responses

Get health

Description

Give summary of probes status

Access

  • Free
  • Exposed

Responses

Response samples

Content type
applications/json
{
  • "license": {
    },
  • "endpoint": "https://spider.mycompany.com",
  • "probes": {
    }
}

Controllers

Controllers are able to spawn Whisperers in a remote cluster.

Create a new Controller

Description

Create a new controller, and associate it to the owner customer.

Access

  • User, with controllers creation rights
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

Controller creation request

customer
required
string

System Id of the customer.

name
required
string

Name of the controller to create.

Responses

Request samples

Content type
application/json
{
  • "customer": "YOD66VZ54Jih",
  • "name": "Upload"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Controllers

Description

Search for controllers

Rules

If client is not admin, the search will be limited to the Controllers owned by this customer or shared with him, directly or by the team.

Access

  • Admin
  • User
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get default Configuration for Controllers

Description

Get default configuration.

Access

  • User
  • Admin
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a Controller

Description

Get a controller's details.

Access

  • User owning the controller
  • The own controller
  • User being shared access to this controller
  • User of the team being shared access to this controller
  • Customer application
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Controller

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "Controller",
  • "version": "string",
  • "name": "string",
  • "customer": "string",
  • "apikey": "string",
  • "config": {
    },
  • "users": [
    ],
  • "teams": [
    ],
  • "status": {
    },
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "editor": "string",
  • "dateModified": "2019-08-24"
}

Change controller's name or shared users

Description

Updates a customer. You can:

  • Update controller's name
  • Change sharing settings: teams, users and users rights on this controller

Rules

  • Can do any change:
    • Admin
    • User owning the controller
  • User being shared access to this controller with share rights can:
    • Add or remove users from the sharing settings
  • User being shared access to this controller with rights change rights can:
    • Change rights of users in the sharing settings

Access

  • User wning the controller
  • User of a team having shared access to the controller
  • User being shared access to this controller with config, share or rights modification rights
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Controller

header Parameters
If-Match
required
string

eTag of previous state of the controller

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Delete a Controller

Description

Delete a controller.

  • Put it to technicalStatus DELETED.

Access

  • User wning the controller
  • User being shared access to this controller and with delete right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Controller

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create/Replace the controller API key

Description

Create/Replace the controller API key used for controller connection to Spider.

The API key is a public/private key pair.

  • The public key is stored in controller's settings.
  • The private key is sent in response to this call.

Any call to this API (if authorized) will overwrite the previous API key, and the controller will not be able to use the previous one. A connected controller will be disconnected when its current JWT token will expire. The API key is taken as a configuration AT START of controllers, and need a restart to be changed.

Output

The API can supports two outputs:

  • application/json:
    • Provides a file with the private key, Spider's URL, and the controller's id This file is the only expected configuration file at controller start.
  • application/x-pem-file
    • Provides only the private key as a PEM file

Access

  • User wning the controller
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Responses

Response samples

Content type
{}

Generate an API key signature with the private key of the controller

Description

This endpoint is for testing purposes only:

  • It allows testing the API key or the controllers API without a controller
  • It generates a valid signature for the controller to call the configuration endpoint
  • However, IT REQUIRES YOUR PRIVATE KEY IN INPUT
  • After testing, please, reset your API key

Output

  • The signature for this controller, timestamp and private API key
  • A validation of this signature with the controller registered public API key

Access

  • User owning the controller
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

query Parameters
timeStamp
required
string <date-time>

Timestamp to use in signature

instanceId
required
string

InstanceId to use in signature

Request Body schema: application/x-pem-file
string

The controller RSA private key, as a PEM

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get Controller configuration

Description

Get the configuration of this Controller Usages:

  • Called by controller at start (or before token expiration) with their API key
  • Called by controllers regularly to check for configuration change
  • Called by UI to get configuration

API key

The first call from controllers is made by:

  • building a json payload with current time and controller id,
  • then signing it with SHA256 algorithm using controller's private key
const timeStamp = moment().toISOString();
const info = {
    timeStamp,
    controllerId
};
const privKey = new NodeRSA(privatePem);
const signature = privKey.sign(Buffer.from(JSON.stringify(info)), 'base64');
  • and then calling this API with specific headers:
  Spider-TimeStamp: timeStamp
  Spider-Signature: signature //base 64 encoded

Output

  • The configuration
  • A JWT token to use on further calls in Spider-Token header
    • If no token provided, or if called from a Customer
    • A Customer may call to get the configuration of one of its controllers and use the generated token to upload data (as on the UI)

Access

  • The own controller
  • User owning the controller
  • User being shared access to this controller
  • User of the team being shared access to this controller
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

header Parameters
Spider-Signature
string <base64>

Signature of the call by the Controller, with its API key

Spider-Timestamp
string <date-time>

Provided with API key in first Controller call to get its JWT token with the config

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Change Controller's config

Description

Updates a Controller configuration.

Rules

  • Can do any change:
    • Admin
    • User owning the Controller
  • User being shared access to this Controller with config rights can:
    • Change configuration settings
  • User being shared access to this Controller with share rights can:
    • Change sharing settings

Access

  • User owning the Controller
  • User being shared access to this Controller with config or share rights
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

header Parameters
If-Match
required
string

eTag of previous state of the Controller's config

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get namespaces of the cluster where the Controller is

Description

Get the list of namespaces names from the cluster.

Output

  • An array of string

Returns 404 if the controller is not connected.

Access

  • User owning the controller
  • User being shared access to this controller with attach right
  • User of the team being shared access to this controller with attach right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Responses

Response samples

Content type
application/json
[
  • "string"
]

Get list of objects of the requested namespace and collection from the cluster

Description

Get the list of objects from the cluster.

Output

  • An array of items

Returns 404 if the controller is not connected.

Access

  • User owning the controller
  • User being shared access to this controller with attach right
  • User of the team being shared access to this controller with attach right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

namespace
required
string

Name of the namespace we want the object from

collection
required
string
Enum: "pods" "statefulsets" "deployments" "cronjobs"

Collection we are interested in

Responses

Response samples

Content type
application/json
[
  • {
    },
  • {
    }
]

Create a new attachment

Description

Create a new attachment, asking, by it, to spawn a Whisperer to each linked Pod

  • The attachment is first saved in DB, and it will be fetched by the Controller next time with its configuration.
  • Then, an attachment request is sent to the Controller for an immediate attachment (if Controller is connected)

Access

  • User owning the controller
  • User being shared access to this controller with attach right
  • User of the team being shared access to this controller with attach right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Request Body schema: application/json
required

Attachment creation request

whisperer
required
string

Whisperer Id to attach.

namespace
required
string

Namespace where the workload is.

collection
required
string
Enum: "pods" "statefulsets" "daemonsets" "deployments" "cronjobs"

Collection of the workload.

item
required
string

Name of the workload.

agent
string
Enum: "whisperer" "gossiper"

Agent to attach.

Responses

Request samples

Content type
application/json
{
  • "whisperer": "string",
  • "namespace": "string",
  • "collection": "pods",
  • "item": "string",
  • "agent": "whisperer"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get last logs of the attached Whisperer

Description

Get last logs of the attached Whisperer.

Access

  • User owning the controller
  • User being shared access to this controller with attach or monitor right
  • User of the team being shared access to this controller with attach or monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

attachment
required
string

System id of the Attachment

container
required
string

Container id of the Whisperer

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Detach the attachment

Description

Ask for the attachment to be terminated.

The Whisperers connected to the worload linked to this attachment will terminate the next time they check their status.

Access

  • User owning the controller
  • User being shared access to this controller with attach right
  • User of the team being shared access to this controller with attach right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

attachment
required
string

System id of the Attachment

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Attachments

Description

Search for attachments

Rules

If client has limited access to namespaces, the search will be limited to the Attachments the user can access, directly or by the team.

Access

  • Admin
  • User owning this Controller
  • User being shared access to this Controller
  • User of the team being shared access to this Controller
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get the Whisperers managed by this Controller

Description

Returns the list of Whisperers that the Controller knows and manages.

Access

  • User owning the controller
  • User being shared access to this controller with attach or monitor right
  • User of the team being shared access to this controller with attach or monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get the Whisperers deployed as Sidecars seen by this Controller

Description

Returns the list of Sidecar Whisperers that the Controller knows.

Access

  • User owning the controller
  • User being shared access to this controller with monitor right
  • User of the team being shared access to this controller with monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get the last logs as Sidecar Whisperer

Description

Returns the last logs of the Sidecar Whisperer

Access

  • User owning the controller
  • User being shared access to this controller with monitor right
  • User of the team being shared access to this controller with monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

key
required
string

@id of the sidecar whisperer container as returned by GET /whisperers/sidecars

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get the Gociphers known by this Controller

Description

Returns the list of Gociphers that the Controller knows.

Access

  • User owning the controller
  • User being shared access to this controller with monitor right
  • User of the team being shared access to this controller with monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get the last logs of the Gociphers

Description

Returns the last Logs of the Gocipher.

Access

  • User owning the controller
  • User being shared access to this controller with monitor right
  • User of the team being shared access to this controller with monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

key
required
string

@id of the Gocipher container as returned by GET /gociphers

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Save Network Usage

Description

Save Network Usage to make it accessible for analysis

Access

  • The Controller
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Request Body schema: application/json
required

Network Usage

controller
required
string
minute
required
string <date-time>
nodes
required
integer >= 0
required
Array of objects

Responses

Request samples

Content type
application/json
{
  • "controller": "string",
  • "minute": "2019-08-24T14:15:22Z",
  • "nodes": 0,
  • "items": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Network Usage

Description

Search for Network Usage

Rules

If client has limited access to namespaces, the search will be limited to the Attachments the user can access, directly or by the team.

Access

  • Admin
  • User owning this Controller
  • User being shared access to this Controller
  • User of the team being shared access to this Controller
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get the Attachments associated to this Whisperer, from all know attachments

Description

Returns the list of Attachments linked to a Whisperer.

The service will calls all referenced and connected Controllers to get the status of the Whisperers.

Access

  • User owning the controller
  • User being shared access to this controller with attach or monitor right
  • User of the team being shared access to this controller with attach or monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of a Whisperer

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get the last logs of the Controller

Description

Returns the last logs of the Controller.

Access

  • User owning the controller
  • User being shared access to this controller with monitor right
  • User of the team being shared access to this controller with monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of a Controller

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Customers

Users accounts.

Connect a customer

Description

Connects a customer and returns a JWT token

When successful, a cookie is set in the answer, containing the refresh_token. The cookie is http only, secure, with strict domain and path.

Rules

  • After many unsuccessful attempts, the email will be blocked for some time.

Access

  • No identification required
Request Body schema: application/json
required

The email/password for connection

email
required
string

Email

password
required
string <password>

Password

Responses

Request samples

Content type
application/json
{
  • "email": "string",
  • "password": "pa$$word"
}

Response samples

Content type
application/json
{
  • "customer": "string",
  • "token": "string"
}

Logs out a customer

Description

Logs out a customer and clears the refresh token cookie

Access

  • The cookie with the refresh token is expected in input

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Connect a customer using OIDC code flow

Description

Connects a customer using OIDC and returns a JWT token.
Takes in input the code provided by the Identity Provider and the name of the IP, as set in configuration.

Access

  • No identification required
Request Body schema: application/json
required

The code and IP to get the tokens from

code
required
string

Code received from the authorization_endpoint

provider
string

Identity provider name set in configuration

Responses

Request samples

Content type
application/json
{
  • "code": "string",
  • "provider": "string"
}

Response samples

Content type
application/json
{
  • "customer": "string",
  • "token": "string"
}

Connect a service account using OAuth2 client-credentials flow

Description

Connects a service account and returns a JWT token.
Takes in input the client_id and client_secret of the service account.

  • grant_type must be equals to "client_credentials"
  • when using application/x-www-form-urlencoded content type, client_id and client_secret may also be provided as a Basic Authorization header

Access

  • No identification required
Request Body schema:
required

The service account credentials

client_id
required
string
client_secret
required
string
grant_type
required
string
Value: "client_credentials"

Responses

Request samples

Content type
{
  • "client_id": "string",
  • "client_secret": "string",
  • "grant_type": "client_credentials"
}

Response samples

Content type
application/json
{
  • "access_token": "string",
  • "token_type": "Bearer",
  • "expires_in": 0
}

Generate a new access token and refreshes the refresh token

Description

Rotate the access token and issue a new refres token for a user

Responses

Response samples

Content type
application/json
{
  • "customer": "string",
  • "token": "string"
}

Create a new customer

Description

Create a new customer.

Rules

  • If a customer with same email already exists, creation is cancelled.
  • Depending on system settings, a self created customer (no token) will be:
    • Created as Draft but checked for all fields when admin activation is required
    • Created as Active when no activation is required

Draft customer

All fields are optional at start, but checked for correctness.

  • It then creates a DRAFT resource.
  • To update it to ACTIVE
    • Some fields are required.
    • Use PATCH.

Active customer

To create an ACTIVE customer, all mandatory fields must be set. Status must be set as ACTIVE.

Access

Either:

  • Self creation and no identification is required
  • Creation by someone else:
    • An admin user or user with users creation rights
    • Only a user with rights modification rights can create a user with rights
    • Only an admin can create an admin
  • Creation by a userTrainer, with trainee rights set.
Authorizations:
Bearer
Request Body schema: application/json
required

The customers details

email
required
string

Customer's email.

_password
required
string >= 6 characters

Customer's password.

_admin
boolean

True if user is an administrator.

required
object
birthDate
string <date>

Date of birth.

honorificPrefix
string

An honorific prefix preceding a name such as Dr/Mrs/Mr.

givenName
required
string

The given name, the first name.

familyName
required
string

The family name, the last name.

nationality
required
string

Nationality.

jobTitle
string

The job title (for example, Financial Manager).

worksFor
string

Organizations'name that the person works for.

Responses

Request samples

Content type
application/json
{
  • "email": "example@gmail.com",
  • "_password": "YGIUHIdzzf!/85F",
  • "_admin": true,
  • "givenName": "John",
  • "familyName": "Doe",
  • "nationality": "American",
  • "address": {
    },
  • "technicalStatus": "DRAFT"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create a new service account

Description

Create a new service account.

Draft service account

All fields are optional at start, but checked for correctness.

  • It then creates a DRAFT resource.
  • To update it to ACTIVE
    • Some fields are required.
    • Use PATCH.

Active customer

To create an ACTIVE service account, all mandatory fields must be set. Status must be set as ACTIVE.

Access

  • Creation by someone else:
    • An admin user or user with service accounts creation rights
    • Only a user with rights modification rights can create a service account with rights
    • Only an admin can create an admin
Authorizations:
Bearer
Request Body schema: application/json
required

The service account details

name
required
string
description
string
_password
required
string >= 6 characters

client_secret

_admin
boolean

True if service account is an administrator.

worksFor
string

Organizations'name that the person works for.

Responses

Request samples

Content type
application/json
{
  • "name": "Spider bot",
  • "_password": "YGIUHOIUHIOEUKBEZUICUYEGIIdzzf!/85F",
  • "technicalStatus": "ACTIVE"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Customers or Service Accounts

Description

Search for customers.

Also available by GET method on the collection

Access

  • Admin
  • User with user or service account management rights
  • userTrainer (but may only search on HIS trainees)
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a customer's or service account details

Description

Get a customer's details.

  • _password field is always stripped out.
  • Deleted customers are only visible by admins.
  • Depending on access rights, returns diverse representations.

Clients getting full details

  • The own client
  • Admins
  • User s with users administration rights

Response is:

  • Full content
  • Minus _* fields (password, admin...)

Clients getting system details

  • Whisp application (to update linked whisperers)

Response is limited to:

  • @id
  • email / name
  • rights
  • whisperers

Clients getting shortened info

  • Any other client

Response is limited to:

  • @id
  • email

Access

  • Customer, expect when using public links
  • Admin
  • User with users management right
  • User impersonating the user to open
  • User with userTrainer rights opening one of its trainee
  • Whisp application (to update linked whisperers)
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

Responses

Response samples

Content type
application/json
{
  • "@id": "Jz6lxOiuQYaIZNNouiyt6w",
  • "@type": "Person",
  • "version": "0.1",
  • "technicalStatus": "ACTIVE",
  • "creator": "AWUb00luIXCLtCIFlzoO",
  • "dateCreated": "2018-12-21T10:00:00.837Z",
  • "editor": "Jz6lxOiuQYaIZNNouiyt6w",
  • "dateModified": "2018-12-21T16:15:18.978Z",
  • "givenName": "John",
  • "familyName": "Doe",
  • "nationality": "American",
  • "email": "example@gmail.com",
  • "address": {
    },
  • "rights": {
    },
  • "whisperers": [
    ],
  • "_eTag": "\"d7-+6EM6pRmKNKDSTtAgeIK5g\""
}

Update customer's or service account details

Description

Updates a customer. You can:

  • Update customer details
  • Change password
  • Set new rights (and admin flag)
  • Change Whisperer names (for synchro)
  • Change its state from DRAFT to ACTIVE

Rules

  • Patch must be done with resource previous eTag

    • eTag must be given in ifMatch header (* not authorized)
    • eTag must match current eTag

  • Technical fields are protected (@id, creator, dateCreated, @type)

  • For a customer to change its password or email, patch operations must include previous password value inside a test operation

    • { "op":"test", "path":"_password", "value":"xxx" }

  • When customer changes email:

    • If a customer with same email already exists, operation is cancelled.
    • A confirmation email challenge is sent. The account is blocked at connection until the mail is confirmed.
    • A mail is sent to old email

  • When customer changes password:

    • A confirmation mail is sent

  • When a customer changes from Draft to Active

    • A information mail is sent

  • User cannot change its own Whisperers list

  • Admins can:

    • Update a customer's details ONLY when DRAFT
      • Also clients with creation right
    • Update rights
      • Also clients with rights admin right
    • Reinit password without providing old one
      • Also clients with password admin right
    • Set a customer as admin
    • Change a DELETED user back to ACTIVE

  • User details, once in ACTIVE state, can only be modified by own customer

  • Whisp & Maintenance services can:

    • Update associated whisperers

  • _password and _admin field cannot be removed, copied or moved

  • email field cannot be removed or moved

Access

  • Admin
  • Own customer
  • Whisp service (to update linked whisperers)
  • User with rights administration right
  • User with user creation right
  • User with password change/reinit right
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

header Parameters
If-Match
required
string

eTag of previous state of the customer

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Delete a customer or a service account

Description

Set a customer to DELETED status. When the customer has one to many own Whisperers of UPLOAD type, they are also deleted.

Access

  • Admin
  • User s with delete customer right
  • User with userTrainer right deleting on of its trainee account
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a customer info by its email

Description

Same as GET /customer/v1/customers/{id}

Authorizations:
Bearer
path Parameters
email
required
string

Customer email

Responses

Response samples

Content type
application/json
{
  • "@id": "Jz6lxOiuQYaIZNNouiyt6w",
  • "@type": "Person",
  • "version": "0.1",
  • "technicalStatus": "ACTIVE",
  • "creator": "AWUb00luIXCLtCIFlzoO",
  • "dateCreated": "2018-12-21T10:00:00.837Z",
  • "editor": "Jz6lxOiuQYaIZNNouiyt6w",
  • "dateModified": "2018-12-21T16:15:18.978Z",
  • "givenName": "John",
  • "familyName": "Doe",
  • "nationality": "American",
  • "email": "example@gmail.com",
  • "address": {
    },
  • "rights": {
    },
  • "whisperers": [
    ],
  • "_eTag": "\"d7-+6EM6pRmKNKDSTtAgeIK5g\""
}

Get user token to impersonate this customer

Description

Generates a token for a user to be able to use another user whisperers and rights.

Rules

  • User must not be deleted
  • Only an administrator may impersonate another administrator

Output

  • Generate a new token with
    • The requested customer id in impersonated field.
    • The customer's whisperers
    • The customers rights (if useUserRights is 'true')
  • The token can be used to call any API
  • The services will behave as if the customer was calling, except that all traces and audit fields will be valued with the original caller's id.

Access

  • User with impersonate right
  • User with userTrainer right impersonating one of its trainee
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

query Parameters
useUserRights
boolean

Ask to use user's right in the token. Keep caller's right if false.

Responses

Response samples

Content type
application/json
{
  • "token": "string"
}

Create a password challenge

Description

Create a password challenge to reinitialize a password. A token is created and sent by mail with a redirection link to the user. The redirection link:

  • launches an UI presenting a form to enter a new password
  • contains a unique token, valid once and a limited time

Rules

  • Account with this email must exist and not be deleted

Access

  • No identification required
Request Body schema: application/json
required

The email of the account

email
required
string

Email

redirectUrl
required
string

Base Url to construct the redirection link. Expected: Login UI endpoint.

Responses

Request samples

Content type
application/json
{
  • "email": "string",
  • "redirectUrl": "string"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Set a password after a password challenge

Description

Set a new password to a user with:

  • The challenge token
  • The user email
  • The password (plain)

An email is sent to the user to inform him of password change. Connections error count is reset ;)

Rules

  • Account with this email must exist and not be deleted
  • Token must still exists (not used, not too old)
  • Token must be associated to right email and user

Access

  • No identification required
Request Body schema: application/json
required

The new password of the account

email
required
string

Email

token
required
string

Token sent in challenge

password
required
string

Password choosen by the user

Responses

Request samples

Content type
application/json
{
  • "email": "string",
  • "token": "string",
  • "password": "string"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Confirm user email

Description

Used in mails sent to users to confirm their email addresses. This is why it is a GET, not a POST.

Rules

  • User must exist and not be deleted
  • The account confirmation token must exist
  • The token is linked to the right email and account
  • If user is in Draft status, an email is sent to admins for account activation
  • Confirmation page is shown

Access

  • No identification required
Authorizations:
Bearer
query Parameters
email
required
string

Email

token
required
string

Mail confirmation token

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

OAuth

OAuth 2.1 authorization server for human-delegated agent connections (RFC 7591 dynamic client registration, RFC 6749 authorization code + PKCE, RFC 7662 introspection, RFC 7009 revocation, per-user connected-app grants). Machine-to-machine integrations should keep using service accounts with the client_credentials flow instead. Requires the MCP licence capability (TEAM tier and above); ships off by default (customers.oauth.enabled).

Dynamically register an OAuth client (RFC 7591)

Description

Registers a new OAuth client (an agent such as claude.ai or ChatGPT) and returns its client_id (and, unless token_endpoint_auth_method is none, a client_secret shown exactly once, in this response, and never retrievable again).

Unauthenticated by design - this is the front door an agent calls before any Spider identity exists yet.

Rules

  • Reachable whenever customers.oauth.enabled is true and customers.oauth.dynamicRegistration is open or gated. Only disabled (or oauth.enabled: false) always answers 404 not_found - never a distinguishing error - so a scanner cannot tell "registration is closed here" from "this Spider has no OAuth support".
  • In gated mode, every redirect_uri in the request must match an entry on the server's callback catalogue (GET /customer/v1/oauth/callback-catalogue lists which vendors an admin has authorised - see that path). A non-matching redirect_uri is refused with the exact same 400 invalid_redirect_uri body as a structurally malformed one - the caller learns its URI was unacceptable, never whether a catalogue exists or what is on it.
  • verified on the created client is true only when every redirect_uri matched an enabled catalogue entry and none of them is a loopback address (localhost, 127.0.0.1, [::1]) - a loopback callback can be admitted (a native client has nowhere else to redirect) but is never verified, since any local process can bind that port. This is computed once, at registration; disabling the catalogue entry afterwards never revokes an already-issued verified: true.
  • Rate limited to customers.oauth.registrationRateLimit (default 10 per hour) - a platform-wide ceiling behind the gateway, not a per-caller one: this service never trusts X-Forwarded-For, so every external caller is counted against the same counter.
  • Only application/json request bodies are accepted.
  • RFC 8707's resource parameter is accepted and silently ignored: the token this server mints is opaque and carries no audience.
  • Reads the callback catalogue in every mode, not only gated (M6, final review) - the catalogue match that decides gated's admission also feeds verified in open mode. A fault reading the catalogue's own ES store therefore surfaces as this endpoint's generic 500 server_error in open mode too, not only gated.

Access

  • No identification required
  • Requires the MCP licence capability (TEAM tier and above)
Request Body schema: application/json
required

The client metadata to register

client_name
required
string [ 1 .. 200 ] characters

Human-readable name shown on the consent screen. Client-supplied and NOT verified - never trust it as proof of the caller's identity.

redirect_uris
required
Array of strings <uri> [ 1 .. 20 ] items [ items <uri > ]

Registerable redirect URIs. Matched byte-exact (no normalisation) at /authorize.

token_endpoint_auth_method
string
Default: "none"
Enum: "none" "client_secret_post" "client_secret_basic"

How this client authenticates at /token and /revoke. none mints no client_secret (public client, e.g. a native/desktop agent using PKCE).

grant_types
Array of strings
Default: ["authorization_code","refresh_token"]
Items Enum: "authorization_code" "refresh_token"
response_types
Array of strings
Default: ["code"]
Items Value: "code"
client_uri
string <uri>

Must be https.

logo_uri
string <uri>

Must be https.

scope
string
Value: "spider:full"

The only scope this authorization server issues.

Responses

Request samples

Content type
application/json
{
  • "client_name": "string",
  • "redirect_uris": [],
  • "token_endpoint_auth_method": "none",
  • "grant_types": [
    ],
  • "response_types": [
    ],
  • "client_uri": "http://example.com",
  • "logo_uri": "http://example.com",
  • "scope": "spider:full"
}

Response samples

Content type
application/json
{
  • "client_id": "string",
  • "client_name": "string",
  • "redirect_uris": [],
  • "grant_types": [
    ],
  • "response_types": [
    ],
  • "token_endpoint_auth_method": "none",
  • "client_id_issued_at": 0,
  • "client_secret": "string"
}

Authorization endpoint (RFC 6749 authorization code + PKCE)

Description

The front door of the authorization-code flow. The browser lands here with no Spider session. client_id and redirect_uri are validated FIRST, before anything that could redirect anywhere (open-redirect defence): a failure on either is a direct 400, never a redirect. Once redirect_uri is confirmed to match one of the client's registered URIs, every later failure (bad scope, missing/invalid PKCE) is instead delivered as a 302 redirect to that redirect_uri carrying error (+ state).

On success, redirects (302) to the Login GUI's consent screen, carrying an opaque request_id the GUI uses to fetch and decide the pending request (see GET/POST /customer/v1/oauth/authorize/request/{requestId} below).

Unauthenticated by design (D11): this route never reads the Spider session cookie - by both its path and sameSite: strict attributes, the cookie is never presented here, which is deliberate defense against CSRF on the refresh token, not an oversight. Consent is obtained afterwards, from a logged-in user, at the consent-screen endpoints below.

client_id may be an https Client ID Metadata Document URL (see client_id_metadata_document_supported). Error redirects to the client carry iss.

Rules

  • code_challenge_method must be S256 (RFC 7636); plain is rejected as invalid_request.
  • scope, if present, must be exactly spider:full - the only scope this server issues.
  • RFC 8707's resource parameter is accepted and silently ignored.

Access

  • No identification required
  • Requires the MCP licence capability (TEAM tier and above)
query Parameters
client_id
required
string

The registered client_id.

redirect_uri
required
string <uri>

Must byte-exact match one of the client's registered redirect_uris.

response_type
required
string
Value: "code"

Only the authorization code flow is implemented.

code_challenge
required
string

RFC 7636 PKCE code challenge.

code_challenge_method
required
string
Value: "S256"

Only S256 is accepted.

scope
string
Value: "spider:full"

Defaults to spider:full, the only scope issued.

state
string

Opaque value echoed back verbatim on the eventual redirect.

resource
string

RFC 8707 - accepted and ignored.

Responses

Response samples

Content type
application/json
{
  • "error": "string",
  • "error_description": "string"
}

Read a pending consent request

Description

Non-consuming read of a pending authorization request, for the Login GUI's consent screen. May be called any number of times (refresh, back navigation) before the user decides - only approve/deny below consume it.

client_name is supplied by the registering client and is NOT verified. redirect_host is the only field on this screen an operator or user can actually trust - and, since Plan B, it is also the evidence verified is derived from (a positive badge is shown only when redirect_host matched an admin-authorised callback catalogue entry at registration time), which makes that sentence more true, not less.

AUTHENTICATED, deliberately not exempt from the JWT middleware like /authorize above: consent is granted by a logged-in Spider user, so the identity reading/approving this request must already be known.

Access

  • Any authenticated Spider user
  • Requires the MCP licence capability (TEAM tier and above)
Authorizations:
Bearer
path Parameters
requestId
required
string

Opaque id minted by GET /customer/v1/oauth/authorize.

Responses

Response samples

Content type
application/json
{
  • "client_name": "string",
  • "verified": true,
  • "redirect_host": "string",
  • "redirect_is_loopback": true,
  • "client_id_host": "string",
  • "scopes": [
    ],
  • "grants_modify": true
}

Approve a pending consent request

Description

Consumes the pending request (atomically, exactly once) and mints a single-use authorization code, redeemable at POST /customer/v1/oauth/token. No token is minted here.

The identity the resulting grant is bound to is the caller's own verified JWT (ctx.state.jwt.customer) - never a value supplied in the request - so a caller cannot approve a pending request on someone else's behalf.

Called as a same-origin XHR from the Login GUI, which then performs the actual browser navigation to the URL in redirect_to - this endpoint itself never issues a 302.

Access

  • Any authenticated Spider user
  • Requires the MCP licence capability (TEAM tier and above)
Authorizations:
Bearer
path Parameters
requestId
required
string

Responses

Response samples

Content type
application/json
{}

Deny a pending consent request

Description

Consumes the pending request (atomically, exactly once, same as approve) with no token or code minted. Returns the URL the Login GUI navigates to, carrying error=access_denied (and state, if supplied).

Access

  • Any authenticated Spider user
  • Requires the MCP licence capability (TEAM tier and above)
Authorizations:
Bearer
path Parameters
requestId
required
string

Responses

Response samples

Content type
application/json
{}

Token endpoint (RFC 6749 authorization_code and refresh_token grants)

Description

Exchanges an authorization code (with its PKCE code_verifier) or a refresh token for a fresh access/refresh token pair. This endpoint authenticates the OAuth CLIENT, not a Spider user, so it is unauthenticated by koa-jwt design - client authentication is one of none (public client), client_secret_post (client_id/client_secret in the body), client_secret_basic (Authorization: Basic base64(client_id:client_secret)), or private_key_jwt (client_assertion_type/client_assertion, for a CIMD client whose document declares it); only one method may be used per request.

Both application/json and application/x-www-form-urlencoded bodies are accepted.

Every response, success or error, carries Cache-Control: no-store (RFC 6749 §5.1).

access_token is opaque, not a JWT - it is meaningless to any Spider service other than this one's own /introspect. Refresh tokens are single-use: presenting an already-rotated refresh token revokes the whole grant (reuse detection), and RFC 6749 §6 scope narrowing does not apply here, so scope is omitted from refresh_token responses.

Rules

  • grant_type=authorization_code requires code, redirect_uri and code_verifier; the code, once redeemed, cannot be redeemed again, and its client_id/redirect_uri bindings from consent time must match this request exactly.
  • grant_type=refresh_token requires exactly refresh_token, and rejects it if it was issued to a different client than the one authenticating this request.
  • RFC 8707's resource parameter is accepted and silently ignored.

Access

  • No identification required (the OAuth client authenticates itself, see above)
  • Requires the MCP licence capability (TEAM tier and above)
Request Body schema:
required

The token request, per grant_type

grant_type
required
string
Enum: "authorization_code" "refresh_token"
code
string

Required for authorization_code; forbidden for refresh_token.

redirect_uri
string <uri>

Required for authorization_code; forbidden for refresh_token.

code_verifier
string

Required for authorization_code; forbidden for refresh_token.

refresh_token
string

Required for refresh_token; forbidden for authorization_code.

client_id
string

Omit when authenticating via HTTP Basic instead.

client_secret
string

Omit when authenticating via HTTP Basic instead, or for a none-auth public client.

client_assertion_type
string
Value: "urn:ietf:params:oauth:client-assertion-type:jwt-bearer"

RFC 7523 private_key_jwt, for a CIMD client whose document declares it. Signed RS256 or ES256 by a key in the document's jwks_uri; iss and sub equal the client_id; aud contains the token endpoint URL or the issuer; exp at most 5 minutes ahead; jti single-use. Cannot be combined with client_secret or HTTP Basic.

client_assertion
string

The signed JWT assertion - see client_assertion_type.

resource
string

RFC 8707 - accepted and ignored.

Responses

Request samples

Content type
No sample

Response samples

Content type
application/json
{
  • "access_token": "string",
  • "token_type": "Bearer",
  • "expires_in": 0,
  • "refresh_token": "string",
  • "scope": "string"
}

Introspect an opaque access/refresh token (RFC 7662)

Description

Resolves an opaque OAuth token back to a live Spider identity. This is the interface an external resource server (e.g. an MCP server) calls to authorize a tool call. The response field names are a fixed contract for that caller - active: false is rendered alone for any unknown, expired or revoked token, never as an error.

access_token in the response is a freshly-minted Spider JWT for the grant's customer (refreshed transparently, single-flight, when the cached one has expired) - never the OAuth access token itself, and never the underlying Spider refresh token.

Every response carries Cache-Control: no-store.

This endpoint's error responses (403, 404, 422) use the same {error, error_description} OAuth error shape as every other endpoint on this surface (see #/components/schemas/OAuthError).

Access

  • Caller must be an authenticated Spider service account (not an ordinary user session)
  • Requires the MCP licence capability (TEAM tier and above)
Authorizations:
Bearer
Request Body schema: application/json
required

The token to introspect

token
required
string
token_type_hint
string

Accepted per RFC 7662 §2.1 but never narrows the search - both access and refresh token stores are always tried.

Responses

Request samples

Content type
application/json
{
  • "token": "string",
  • "token_type_hint": "string"
}

Response samples

Content type
application/json
{
  • "active": true,
  • "customer": "string",
  • "email": "string",
  • "isAdmin": true,
  • "whisperers": [
    ],
  • "scopes": [
    ],
  • "access_token": "string",
  • "expires_in": 0
}

Revoke a grant by one of its tokens (RFC 7009)

Description

Revokes the whole grant reachable from the presented access or refresh token value (revocation is per-grant, not per-token). This endpoint authenticates the OAuth CLIENT, not a Spider user, using the same methods as /token (none, client_secret_post, client_secret_basic, private_key_jwt); it is unauthenticated by koa-jwt design for the same reason /token is.

A tombstoned (already-rotated) refresh token still resolves to its grant and is honoured here - a client revoking a credential it once legitimately held gets the same whole-grant revoke a live token would produce.

Both application/json and application/x-www-form-urlencoded bodies are accepted, both token and token_type_hint and the client credential fields.

RFC 7009 §2.2 requires this endpoint to answer 200 regardless of outcome - an unknown token, a token belonging to a different client than the one authenticating this request (refused, not revoked), and an actual revocation are all rendered identically, to avoid turning this endpoint into an oracle for which token values are real. Every response carries Cache-Control: no-store.

Access

  • No identification required (the OAuth client authenticates itself, see above)
  • Requires the MCP licence capability (TEAM tier and above)
Request Body schema:
required

The token to revoke, and the client's own credentials

token
required
string
token_type_hint
string

Accepted per RFC 7009 §2.1 but never narrows the search.

client_id
string

Omit when authenticating via HTTP Basic instead.

client_secret
string

Omit when authenticating via HTTP Basic instead, or for a none-auth public client.

client_assertion_type
string
Value: "urn:ietf:params:oauth:client-assertion-type:jwt-bearer"

RFC 7523 private_key_jwt, for a CIMD client whose document declares it. Signed RS256 or ES256 by a key in the document's jwks_uri; iss and sub equal the client_id; aud contains the token endpoint URL or the issuer; exp at most 5 minutes ahead; jti single-use. Cannot be combined with client_secret or HTTP Basic.

client_assertion
string

The signed JWT assertion - see client_assertion_type.

Responses

Request samples

Content type
{
  • "token": "string",
  • "token_type_hint": "string",
  • "client_id": "string",
  • "client_secret": "string",
  • "client_assertion_type": "urn:ietf:params:oauth:client-assertion-type:jwt-bearer",
  • "client_assertion": "string"
}

Response samples

Content type
application/json
{
  • "error": "string",
  • "error_description": "string"
}

JSON Web Key Set for this authorization server (RFC 7517)

Description

Publishes the public key an artefact issued by this authorization server could be verified with. In practice nothing on this surface issues a JWT to a caller outside this process (access/refresh tokens are opaque - see OAuthTokenResponse), so this endpoint exists for completeness/future use rather than a verifier that exists today.

Unauthenticated by design - a JWKS document is meant to be publicly fetchable.

Access

  • No identification required
  • Requires the MCP licence capability (TEAM tier and above) - gated on licence alone, independent of customers.oauth.enabled

Responses

Response samples

Content type
application/json
{
  • "keys": [
    ]
}

List the caller's own connected-app grants

Description

Lists every live OAuth grant belonging to the calling Spider customer - the "Connected apps" view. This is one of only two OAuth paths that take an ordinary Spider user JWT rather than OAuth client credentials (the other is the DELETE below).

Signing out of Spider does NOT revoke a connected agent's grant - a grant has its own, independent Spider session by design, so it survives a browser logout. This listing (and the DELETE below) is the only in-product way to end a grant short of its refresh token's own TTL.

lastUsedAt/expiresAt are always null today - nothing on this surface populates them yet; the keys are reserved for a future task.

Access

  • Any authenticated Spider user (lists only that user's own grants)
  • Requires the MCP licence capability (TEAM tier and above) - gated on licence alone, independent of customers.oauth.enabled (same exception as jwks above)
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Revoke one of the caller's own connected-app grants

Description

Ends a single grant - the "Connected apps" disconnect action. The other of the two OAuth paths taking an ordinary Spider user JWT (see GET /customer/v1/oauth/grants above).

grant.customer must match the caller; a caller can never revoke another customer's grant.

Rules

  • Returns 404 - never 403 - for BOTH "no such grant" and "exists, but belongs to someone else". A 403 would confirm the grantId is real, letting a caller enumerate other customers' grant ids by probing which ones come back 403 vs 404. The two cases are told apart only in the server log, never in the response.
  • Idempotent: revoking an already-revoked grantId answers the same 404.

Access

  • Any authenticated Spider user (only over their own grants, see Rules)
  • Requires the MCP licence capability (TEAM tier and above) - gated on licence alone, independent of customers.oauth.enabled (same exception as jwks above)
Authorizations:
Bearer
path Parameters
grantId
required
string

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

List/search registered OAuth clients

Description

Lists every registered OAuth client (self-registered via POST /customer/v1/oauth/register

  • there is no admin client-creation route; an admin authorises callbacks, never creates clients, see /customer/v1/oauth/callback-catalogue below), for the client-administration view.

Rules

  • query, if present, is an Elasticsearch query_string matched against the client's _quick_search field (client_id, client_name).
  • Sorted by createdAt descending, newest-registered first (M2, final review). Capped at 100 results with no pagination cursor - a deployment with more than 100 registered clients cannot page past the first 100 through this endpoint today.
  • An unauthorised caller gets 404, never 403 - see Access below.

Access

  • Admin
  • User with the oauthClients.create right
Authorizations:
Bearer
query Parameters
query
string

Elasticsearch query_string against the client's _quick_search field.

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get one registered OAuth client

Description

Returns one registered client, including its current count of still-live grants.

Access

  • Admin
  • User with the oauthClients.create right
Authorizations:
Bearer
path Parameters
client_id
required
string

32 hex characters for a registered client, or the URL-encoded metadata document URL of a CIMD client.

Responses

Response samples

Content type
application/json
{
  • "client_id": "string",
  • "client_name": "string",
  • "verified": true,
  • "disabled": true,
  • "registration_type": "dynamic",
  • "redirect_uris": [],
  • "token_endpoint_auth_method": "none",
  • "jwks_uri": "http://example.com",
  • "cimdExpiresAt": "2019-08-24T14:15:22Z",
  • "createdAt": "2019-08-24T14:15:22Z",
  • "lastUsedAt": "2019-08-24T14:15:22Z",
  • "createdBy": "string",
  • "liveGrants": 0
}

Pause or resume an OAuth client

Description

Toggles disabled, the only field this endpoint may change. Disabling stops new authorizations, token issuance and refreshes for this client at /authorize, /token and /revoke - it deliberately leaves every grant the client already holds untouched, so an operator can pause a suspicious client and investigate before destroying evidence. DELETE below is the separate, destructive step that also revokes live grants.

Access

  • Admin
  • User with the oauthClients.create right
Authorizations:
Bearer
path Parameters
client_id
required
string

32 hex characters for a registered client, or the URL-encoded metadata document URL of a CIMD client.

Request Body schema: application/json
required
disabled
required
boolean

Responses

Request samples

Content type
application/json
{
  • "disabled": true
}

Response samples

Content type
application/json
{
  • "client_id": "string",
  • "client_name": "string",
  • "verified": true,
  • "disabled": true,
  • "registration_type": "dynamic",
  • "redirect_uris": [],
  • "token_endpoint_auth_method": "none",
  • "jwks_uri": "http://example.com",
  • "cimdExpiresAt": "2019-08-24T14:15:22Z",
  • "createdAt": "2019-08-24T14:15:22Z",
  • "lastUsedAt": "2019-08-24T14:15:22Z",
  • "createdBy": "string",
  • "liveGrants": 0
}

Delete an OAuth client and revoke its live grants

Description

Deletes the client registration and cascades: revokes every grant this client still holds live first, then deletes the client record. This is the destructive counterpart to PATCH's disabled toggle above, which touches no grant.

Rules

  • Idempotent: deleting an already-deleted client_id answers the same 404 as one that never existed.
  • Requires a second, independent right beyond the other seven routes on this surface - see Access - because this is the one action that destroys live credentials, not just a future authorisation.

Access

  • Admin
  • User holding both the oauthClients.create and oauthClients.delete rights (holding only one of the two is refused identically to holding neither)
Authorizations:
Bearer
path Parameters
client_id
required
string

32 hex characters for a registered client, or the URL-encoded metadata document URL of a CIMD client.

Responses

Response samples

Content type
application/json
{
  • "deleted": true,
  • "grantsRevoked": 0
}

List the callback catalogue

Description

Lists every callback-catalogue entry: the shipped vendor definitions (from the Config service defaults, each with its live enabled state) plus any custom entries an admin has added. This is the surface an admin uses to authorise which agent platforms may self-register - D18: an admin authorises callbacks, never creates clients.

Access

  • Admin
  • User with the oauthClients.create right
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Add a custom callback-catalogue entry

Description

Adds a custom entry for a self-hosted or bespoke client no vendor catalogue could know about - the admin authorises its callback URL(s), and the client then self-registers unattended like any other, once the entry is enabled (see PATCH below).

Rules

  • callback_urls must be https:// - no fragment, no userinfo - loopback URLs are never accepted for a custom entry (loopback is reserved for the match: loopback kind, which only a shipped entry uses today).
  • Created disabled - enabling is always a separate, deliberate action (PATCH below), same as a shipped vendor entry.
  • match is always exact for a custom entry.

Access

  • Admin
  • User with the oauthClients.create right
Authorizations:
Bearer
Request Body schema: application/json
required
vendor_name
required
string [ 1 .. 120 ] characters
callback_urls
required
Array of strings <uri> [ 1 .. 20 ] items [ items <uri > ]

https only - no fragment, no userinfo. Loopback URLs are not accepted for a custom entry.

client_ids
Array of strings <uri> <= 20 items [ items <uri > ]

Optional. Exact Client ID Metadata Document URLs this entry vouches for.

Responses

Request samples

Content type
application/json
{}

Response samples

Content type
application/json
{
  • "entry_id": "string",
  • "vendor_name": "string",
  • "callback_urls": [],
  • "client_ids": [],
  • "match": "exact",
  • "enabled": true,
  • "custom": true,
  • "createdBy": "string",
  • "createdAt": "2019-08-24T14:15:22Z"
}

Enable or disable a callback-catalogue entry, or set its client_ids

Description

Toggles enabled (shipped vendor entry or custom one alike) and/or sets client_ids (custom entries only) - the only fields this endpoint may change.

Rules

  • Disabling refuses future registrations against this entry; it never touches a client already registered under it, its verified flag, or any of its grants (D16 applied one level up from the client PATCH above).
  • client_ids can only be set on a custom entry - 400 on a shipped one.

Access

  • Admin
  • User with the oauthClients.create right
Authorizations:
Bearer
path Parameters
entry_id
required
string^[a-z0-9-]{1,64}$
Request Body schema: application/json
required
enabled
boolean
client_ids
Array of strings <uri> <= 20 items [ items <uri > ]

Custom entries only - 400 on a shipped entry.

Responses

Request samples

Content type
application/json
{}

Response samples

Content type
application/json
{
  • "entry_id": "string",
  • "vendor_name": "string",
  • "callback_urls": [],
  • "client_ids": [],
  • "match": "exact",
  • "enabled": true,
  • "custom": true,
  • "createdBy": "string",
  • "createdAt": "2019-08-24T14:15:22Z"
}

Delete a custom callback-catalogue entry

Description

Removes a custom entry. A shipped vendor entry can only be disabled (PATCH above), never removed - it ships again, unchanged, on the next deploy regardless of this call.

Rules

  • A shipped entry's entry_id answers the same 404 as an unknown one - this endpoint never confirms whether an id names a shipped entry.
  • Unlike DELETE /customer/v1/oauth/clients/{client_id}, no second right is required: removing a catalogue entry revokes a future authorisation, never a live credential.

Access

  • Admin
  • User with the oauthClients.create right
Authorizations:
Bearer
path Parameters
entry_id
required
string^[a-z0-9-]{1,64}$

Responses

Response samples

Content type
application/json
{
  • "deleted": true
}

OAuth authorization server metadata (RFC 8414)

Description

Served at the deployment ROOT, not under /customer like every other OAuth endpoint: RFC 8414 derives this document's own URL by inserting /.well-known/oauth-authorization-server between the host and the issuer's own path, so an issuer of https://host/customer would have to be discoverable at https://host/.well-known/oauth-authorization-server/customer - not a path this service can register. The issuer field is therefore the bare deployment root, while every endpoint it advertises is still /customer-prefixed.

registration_endpoint is present whenever POST /customer/v1/oauth/register actually accepts registrations - customers.oauth.dynamicRegistration is open or gated (gated restricts which redirect_uris are accepted, it does not close the endpoint - see that path's own docs). Only dynamicRegistration: disabled omits the key - advertising an endpoint that would always 404 is worse than omitting it.

Unauthenticated by design - discovery must work before any client has credentials.

Access

  • No identification required
  • Requires the MCP licence capability (TEAM tier and above) - gating discovery itself means a client without the licence learns why immediately, rather than completing discovery and failing later at /authorize with no explanation.

Responses

Response samples

Content type
application/json
{
  • "issuer": "http://example.com",
  • "authorization_endpoint": "http://example.com",
  • "token_endpoint": "http://example.com",
  • "jwks_uri": "http://example.com",
  • "revocation_endpoint": "http://example.com",
  • "introspection_endpoint": "http://example.com",
  • "registration_endpoint": "http://example.com",
  • "scopes_supported": [
    ],
  • "response_types_supported": [
    ],
  • "grant_types_supported": [
    ],
  • "code_challenge_methods_supported": [
    ],
  • "token_endpoint_auth_methods_supported": [
    ],
  • "client_id_metadata_document_supported": true,
  • "authorization_response_iss_parameter_supported": true,
  • "token_endpoint_auth_signing_alg_values_supported": [
    ]
}

Teams

Teams of users.

Create a new Team

Description

Creates a new team and add the owner as the first full rights customer.

Rules

  • A team with the same name must not exist

Access

  • User with team create right
  • User with userTrainer right creating a training team
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

The team to create

name
required
string

Team's name.

description
string

Team's description.

required
object

Owner of the team.

Responses

Request samples

Content type
application/json
{
  • "name": "string",
  • "description": "string",
  • "owner": {
    }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a team's details

Description

Get a team's details.

  • Depending on access rights, returns diverse representations.

Clients getting full details

  • Admins
  • Customers from the team
  • Customers using this team

Response is:

  • Full content

Clients getting system details

  • Whisp application (to update linked whisperers)

Response is limited to:

  • @id
  • name
  • whisperers

Clients getting shortened info

  • Any other client

Response is limited to:

  • @id
  • name

Access

  • Customer
  • Admin
  • Whisp application (to update linked whisperers)
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

Responses

Response samples

Content type
application/json
{
  • "@id": "AnaRheQISOmLu8bKiQv11w",
  • "@type": "Team",
  • "version": "0.1",
  • "name": "First team",
  • "description": "Short team description.",
  • "dateCreated": "2021-03-07T14:43:35.152Z",
  • "creator": "hfQ1rsfcRKWslHwWAPJ9bg",
  • "customers": [
    ],
  • "whisperers": [
    ],
  • "settings": {
    },
  • "technicalStatus": "ACTIVE",
  • "dateModified": "2021-03-18T22:15:39.345Z",
  • "token": "7p8i66dCQsynmFNULKdnjA",
  • "editor": "hfQ1rsfcRKWslHwWAPJ9bg"
}

Update team's details

Description

Updates a team. You can:

  • Update team's details
  • Add/remove customers
  • Set new customers rights
  • Add/remove whisperer names (for synchro)
  • Change team's settings

Rules

  • Patch must be done with resource previous eTag

    • eTag must be given in ifMatch header (* not authorized)
    • eTag must match current eTag

  • Patch cannot be done on DELETED team

  • Technical fields are protected (@id, creator, dateCreated, @type)

  • Customers with share right can update customers list and access filters

  • Customers with settings right can update team settings

  • Customers with update right can update name, description

  • Whisp and Maintenance services can:

    • Update associated whisperers

  • Token cannot be changed with patch

  • A mail is sent to team administrators with changes made

  • After update,

    • Whisperers list for customers and access filters are cleaned from any removed whisperer from the team
    • User s list for access filters are cleaned from any removed user from the team

Access

  • Admin
  • Whisp service (to update linked whisperers)
  • User with rights on team: share, whisperers, settings, update, rights
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

header Parameters
If-Match
required
string

eTag of previous state of the team

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Delete a team

Description

Set a team to DELETED status.

If the team is a training team, its own whisperers are deleted.

Access

  • Admin
  • User s with update team right
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Join a team

Description

Join a user to a team with:

  • The join token
  • The user @id
  • The user email

An email is sent to the team admin to tell them about new user.

Rules

  • The join token must belong to a team
  • Team must still be active
  • User must not be part of the team

Access

  • Admin
  • User whose @id is in body
Authorizations:
Bearer
Request Body schema: application/json
required

The new password of the account

email
required
string

Email

token
required
string

Token sent in challenge

password
required
string

Password choosen by the user

Responses

Request samples

Content type
application/json
{
  • "email": "string",
  • "token": "string",
  • "password": "string"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Add a share token to the team

Description

Set a share token to the team. The token is randomly generated.

Rules

  • Team must not be deleted
  • A notification mail is sent to team admins

Access

  • User being part of the team, with team update right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Remove team share token

Description

Delete the team share token.

Rules

  • Team must not be deleted
  • A notification mail is sent to team admins

Access

  • User being part of the team, with team update right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get user token for this team

Description

Generates a token for the user to be able to use team's whisperers and rights.

Rules

  • Team must not be deleted

Access

  • User being part of the team
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

Responses

Response samples

Content type
application/json
{
  • "token": "string"
}

Get a team info by its name

Description

Same as GET /teams/v1/teams/{id}

Authorizations:
Bearer
path Parameters
name
required
string

Team name

Responses

Response samples

Content type
application/json
{
  • "@id": "AnaRheQISOmLu8bKiQv11w",
  • "@type": "Team",
  • "version": "0.1",
  • "name": "First team",
  • "description": "Short team description.",
  • "dateCreated": "2021-03-07T14:43:35.152Z",
  • "creator": "hfQ1rsfcRKWslHwWAPJ9bg",
  • "customers": [
    ],
  • "whisperers": [
    ],
  • "settings": {
    },
  • "technicalStatus": "ACTIVE",
  • "dateModified": "2021-03-18T22:15:39.345Z",
  • "token": "7p8i66dCQsynmFNULKdnjA",
  • "editor": "hfQ1rsfcRKWslHwWAPJ9bg"
}

Search for Teams

Description

Search for teams.

Also available by GET method on the collection

Rules

  • When not called by admin, will limit to teams of which the user belongs.

Access

  • Admin
  • User of the team
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Otel export

OTLP export of RED metrics: admin-managed collector targets, and per-team exporters selecting which captured traffic is aggregated and pushed.

Create an OTLP collector target

Description

Creates an OtelTarget: an OTLP/HTTP collector endpoint an administrator makes available to one or more teams. Teams then reference it from their own OtelExporters.

Rules

  • credential is write-only. It is accepted here and on PATCH, sealed at rest, and never returned by any read: every response carries the literal "***" and cannot carry anything else. Generated clients must not round-trip that placeholder back on an update - omit the field instead.
  • auth: none is the only mode that may omit credential; every other mode requires one.
  • A target is only selectable by a team listed in its teams.

Access

  • Admin
  • User with the otelTargets.create right
Authorizations:
Bearer
Request Body schema: application/json
required

The target to create

name
required
string

Human-readable name, unique enough for an operator to pick from a list.

description
string
endpoint
required
string

The collector's base URL, e.g. https://collector.example. Spider appends /v1/metrics when it pushes, so an endpoint that already ends in that path posts to /v1/metrics/v1/metrics and is answered 404 with no other symptom.

auth
required
string
Enum: "none" "bearer" "basic" "headers"

How credential is applied to each push.

credential
string

The secret, in the shape auth implies: the bearer token, user:password for basic, or one Name: value per line for headers.

Write-only. It is sealed at rest and never returned by any read - omit it on a PATCH to keep the stored one. Sending an empty string is not "unchanged"; it is an invalid credential for every mode but none.

tlsInsecureSkipVerify
boolean

Accept an untrusted collector certificate.

Array of objects

Teams whose exporters may reference this target.

disabled
boolean

When true, no exporter pushes to it and the run loop skips it.

Responses

Request samples

Content type
application/json
{
  • "name": "string",
  • "description": "string",
  • "endpoint": "string",
  • "auth": "none",
  • "credential": "string",
  • "tlsInsecureSkipVerify": true,
  • "teams": [
    ],
  • "disabled": true
}

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "credential": "***",
  • "name": "string",
  • "description": "string",
  • "endpoint": "string",
  • "auth": "none",
  • "credentialKeyId": "string",
  • "tlsInsecureSkipVerify": true,
  • "teams": [
    ],
  • "disabled": true,
  • "lastUsedAt": "2019-08-24T14:15:22Z",
  • "lastTestAt": "2019-08-24T14:15:22Z",
  • "lastTestResult": "string",
  • "createdBy": "string",
  • "dateCreated": "2019-08-24T14:15:22Z",
  • "dateModified": "2019-08-24T14:15:22Z",
  • "technicalStatus": "string",
  • "_eTag": "string"
}

List OTLP collector targets

Description

Lists the targets the caller may see. Every item's credential reads as "***".

Rules

  • An admin or an otelTargets.create holder sees every target.
  • A plain team member sees only the targets shared with their own team.

Access

  • Admin
  • User with the otelTargets.create right
  • Member of a team a target is shared with
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "items": [
    ],
  • "lastSort": [
    ]
}

Get a target's details

Description

Returns the target. credential reads as the literal "***" here and in every other response; the real value never leaves the server.

Access

  • Admin
  • User with the otelTargets.create right
  • Member of a team this target is shared with
Authorizations:
Bearer
path Parameters
id
required
string

Target internal @id

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "credential": "***",
  • "name": "string",
  • "description": "string",
  • "endpoint": "string",
  • "auth": "none",
  • "credentialKeyId": "string",
  • "tlsInsecureSkipVerify": true,
  • "teams": [
    ],
  • "disabled": true,
  • "lastUsedAt": "2019-08-24T14:15:22Z",
  • "lastTestAt": "2019-08-24T14:15:22Z",
  • "lastTestResult": "string",
  • "createdBy": "string",
  • "dateCreated": "2019-08-24T14:15:22Z",
  • "dateModified": "2019-08-24T14:15:22Z",
  • "technicalStatus": "string",
  • "_eTag": "string"
}

Update a target

Description

Merges the supplied fields into the stored target.

Rules

  • credential is write-only: supply it to replace the sealed credential, omit it to keep the existing one. Sending an empty string is not "unchanged" - it is an invalid credential for any mode other than none.
  • Changing endpoint clears the stored credential, so a request that repoints a target must supply a fresh credential in the same body (or set auth to none); otherwise it is refused with 400 credential is required when auth is "…". A credential is scoped to the collector it was issued for: without this rule a caller holding only otelTargets.create could repoint a target at a listener they control and have the connection test hand that listener the real secret.
  • Server-owned fields (@id, _eTag, technicalStatus, dateCreated, dateModified, createdBy, lastUsedAt, lastTestAt, lastTestResult, credentialKeyId) are stripped from the body rather than applied.

Access

  • Admin
  • User with the otelTargets.create right
Authorizations:
Bearer
path Parameters
id
required
string

Target internal @id

Request Body schema: application/json
required

The fields to change

name
required
string

Human-readable name, unique enough for an operator to pick from a list.

description
string
endpoint
required
string

The collector's base URL, e.g. https://collector.example. Spider appends /v1/metrics when it pushes, so an endpoint that already ends in that path posts to /v1/metrics/v1/metrics and is answered 404 with no other symptom.

auth
required
string
Enum: "none" "bearer" "basic" "headers"

How credential is applied to each push.

credential
string

The secret, in the shape auth implies: the bearer token, user:password for basic, or one Name: value per line for headers.

Write-only. It is sealed at rest and never returned by any read - omit it on a PATCH to keep the stored one. Sending an empty string is not "unchanged"; it is an invalid credential for every mode but none.

tlsInsecureSkipVerify
boolean

Accept an untrusted collector certificate.

Array of objects

Teams whose exporters may reference this target.

disabled
boolean

When true, no exporter pushes to it and the run loop skips it.

Responses

Request samples

Content type
application/json
{
  • "name": "string",
  • "description": "string",
  • "endpoint": "string",
  • "auth": "none",
  • "credential": "string",
  • "tlsInsecureSkipVerify": true,
  • "teams": [
    ],
  • "disabled": true
}

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "credential": "***",
  • "name": "string",
  • "description": "string",
  • "endpoint": "string",
  • "auth": "none",
  • "credentialKeyId": "string",
  • "tlsInsecureSkipVerify": true,
  • "teams": [
    ],
  • "disabled": true,
  • "lastUsedAt": "2019-08-24T14:15:22Z",
  • "lastTestAt": "2019-08-24T14:15:22Z",
  • "lastTestResult": "string",
  • "createdBy": "string",
  • "dateCreated": "2019-08-24T14:15:22Z",
  • "dateModified": "2019-08-24T14:15:22Z",
  • "technicalStatus": "string",
  • "_eTag": "string"
}

Delete a target

Description

Soft-deletes the target (technicalStatus: DELETED). The Maintenance purge job removes the document for good after its retention window.

Access

  • Admin
  • User with the otelTargets.create right
Authorizations:
Bearer
path Parameters
id
required
string

Target internal @id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Test the connection to a target

Description

Performs the OTLP handshake against the target's endpoint, server-side, using the sealed credential, and reports whether it worked. This is the only reason the operation exists: the credential never has to reach a browser to be tested.

Rules

  • The response carries no part of the credential and no request headers - only reachable, status, latencyMs and, on failure, a coarse error.
  • It is a write: lastTestAt and lastTestResult are stamped on the target, so it is gated like PATCH, not like GET.

Access

  • Admin
  • User with the otelTargets.create right
Authorizations:
Bearer
path Parameters
id
required
string

Target internal @id

Responses

Response samples

Content type
application/json
{
  • "reachable": true,
  • "status": 0,
  • "latencyMs": 0,
  • "error": "string"
}

Create a team's metrics exporter

Description

Creates an OtelExporter: the team-owned selection of captured traffic that is rolled up into RED metrics and pushed to the referenced targets, once a minute, two minutes behind real time.

Rules

  • The effective scope is whisperers ∩ the team's whisperers, recomputed on every run. A request naming a whisperer the team does not own is rejected with 403, and a whisperer later removed from the team stops being exported without the exporter being edited.
  • A non-admin author may additionally only select whisperers within their own token's whisperers: an exporter must not emit metrics for traffic its author cannot read.
  • Every selected whisperer must have capture mode INTERFACE; an UPLOAD whisperer produces no live traffic to roll up.
  • whisperers, protocols and targets must each be non-empty. An empty list has no "means everything" meaning here.
  • Every referenced target must exist, be enabled, and list this exporter's team.
  • An exporter carries no credential: those live only on the target it references.

Access

  • Member of the team holding its settings right
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

The exporter to create

name
required
string
enabled
boolean

When false the exporter is kept but nothing is pushed.

team
required
string

@id of the owning team. Set at creation and immutable: a PATCH is authorised against the current team, so the field is stripped from patch bodies.

targets
Array of strings

@ids of the OtelTargets to fan out to. Each must exist, be enabled, and list this exporter's team.

whisperers
required
Array of strings

@ids of the whisperers whose traffic is exported. The effective scope is this list intersected with the team's whisperers, recomputed on every run - a stale exporter can only ever narrow, never widen. Every entry must be an INTERFACE-mode whisperer.

protocols
required
Array of strings
Items Enum: "http" "postgresql" "redis" "grpc" "kafka"

Must be non-empty; an empty list does not mean "all".

filter
string

Author-supplied Lucene fragment narrowing what is aggregated. No access filter is injected into it implicitly.

endpointLabel
boolean

Add spider.endpoint to every series. For HTTP and PostgreSQL its value depends on the whisperer's configured request templates: without them, high-cardinality paths become high-cardinality series.

object

Per-protocol latency histogram bounds, overriding the service defaults. Keys are the protocol names above. Each list must be non-empty and strictly increasing; a protocol absent here falls back to the default.

Responses

Request samples

Content type
application/json
{
  • "name": "string",
  • "enabled": true,
  • "team": "string",
  • "targets": [
    ],
  • "whisperers": [
    ],
  • "protocols": [
    ],
  • "filter": "string",
  • "endpointLabel": true,
  • "bucketEdges": {
    }
}

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "name": "string",
  • "enabled": true,
  • "team": "string",
  • "targets": [
    ],
  • "whisperers": [
    ],
  • "protocols": [
    ],
  • "filter": "string",
  • "accessFilters": {
    },
  • "endpointLabel": true,
  • "bucketEdges": {
    },
  • "status": {
    },
  • "createdBy": "string",
  • "dateCreated": "2019-08-24T14:15:22Z",
  • "dateModified": "2019-08-24T14:15:22Z",
  • "technicalStatus": "string",
  • "_eTag": "string"
}

List metrics exporters

Description

Lists exporters, with their durable per-target push status.

Rules

  • A team member sees their own team's exporters; the query is filtered server-side.
  • An admin, or a service (application) token, sees every team's. The latter is what the Alert service's otelExportStale and otelExportDropping probes read.

Access

  • Any member of a team
  • Admin
  • Service account (application token)
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "items": [
    ],
  • "lastSort": [
    ]
}

Preview what an unsaved exporter would emit

Description

Runs the real rollup aggregation for an unsaved exporter body over the single window the run loop would process next, and returns a sample of the series plus the estimated series count. Nothing is stored and nothing is pushed.

Rules

  • Every rule POST /otel-export/v1/otel/exporters enforces is enforced here too - the whisperer scope in particular. Preview is not a way around it. The single relaxation is that targets may be empty: there is nothing to push to.
  • truncated is the field that matters: true means paging stopped because the series budget ran out, so the saved exporter would drop the overflow (and log OTEX-RUN-008). When it is true, estimatedSeries is a lower bound, not a count. Do not infer truncation from estimatedSeries >= cappedAt - an exact count landing on the cap is not truncation.

Access

  • Member of the team holding its settings right
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

The unsaved exporter body to preview

name
required
string
enabled
boolean

When false the exporter is kept but nothing is pushed.

team
required
string

@id of the owning team. Set at creation and immutable: a PATCH is authorised against the current team, so the field is stripped from patch bodies.

targets
Array of strings

@ids of the OtelTargets to fan out to. Each must exist, be enabled, and list this exporter's team.

whisperers
required
Array of strings

@ids of the whisperers whose traffic is exported. The effective scope is this list intersected with the team's whisperers, recomputed on every run - a stale exporter can only ever narrow, never widen. Every entry must be an INTERFACE-mode whisperer.

protocols
required
Array of strings
Items Enum: "http" "postgresql" "redis" "grpc" "kafka"

Must be non-empty; an empty list does not mean "all".

filter
string

Author-supplied Lucene fragment narrowing what is aggregated. No access filter is injected into it implicitly.

endpointLabel
boolean

Add spider.endpoint to every series. For HTTP and PostgreSQL its value depends on the whisperer's configured request templates: without them, high-cardinality paths become high-cardinality series.

object

Per-protocol latency histogram bounds, overriding the service defaults. Keys are the protocol names above. Each list must be non-empty and strictly increasing; a protocol absent here falls back to the default.

Responses

Request samples

Content type
application/json
{
  • "name": "string",
  • "enabled": true,
  • "team": "string",
  • "targets": [
    ],
  • "whisperers": [
    ],
  • "protocols": [
    ],
  • "filter": "string",
  • "endpointLabel": true,
  • "bucketEdges": {
    }
}

Response samples

Content type
application/json
{
  • "sampleSeries": [
    ],
  • "estimatedSeries": 0,
  • "cappedAt": 0,
  • "truncated": true,
  • "window": {
    }
}

Get an exporter's details

Description

Returns the exporter, including its durable per-target push status.

Access

  • Any member of the owning team
  • Admin
  • Service account (application token)
Authorizations:
Bearer
path Parameters
id
required
string

Exporter internal @id

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "name": "string",
  • "enabled": true,
  • "team": "string",
  • "targets": [
    ],
  • "whisperers": [
    ],
  • "protocols": [
    ],
  • "filter": "string",
  • "accessFilters": {
    },
  • "endpointLabel": true,
  • "bucketEdges": {
    },
  • "status": {
    },
  • "createdBy": "string",
  • "dateCreated": "2019-08-24T14:15:22Z",
  • "dateModified": "2019-08-24T14:15:22Z",
  • "technicalStatus": "string",
  • "_eTag": "string"
}

Update an exporter

Description

Merges the supplied fields into the stored exporter and re-validates the whole result.

Rules

  • team cannot be changed. The request is authorised against the exporter's current team, so allowing the field to change would spend that authorisation on a different resource than the one it was checked against; the field is stripped from the body.
  • status is stripped too: the run loop owns it, and a stale copy echoed back would overwrite live push watermarks.
  • Every rule POST enforces is re-checked against the merged result.

Access

  • Member of the owning team holding its settings right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Exporter internal @id

Request Body schema: application/json
required

The fields to change

name
required
string
enabled
boolean

When false the exporter is kept but nothing is pushed.

team
required
string

@id of the owning team. Set at creation and immutable: a PATCH is authorised against the current team, so the field is stripped from patch bodies.

targets
Array of strings

@ids of the OtelTargets to fan out to. Each must exist, be enabled, and list this exporter's team.

whisperers
required
Array of strings

@ids of the whisperers whose traffic is exported. The effective scope is this list intersected with the team's whisperers, recomputed on every run - a stale exporter can only ever narrow, never widen. Every entry must be an INTERFACE-mode whisperer.

protocols
required
Array of strings
Items Enum: "http" "postgresql" "redis" "grpc" "kafka"

Must be non-empty; an empty list does not mean "all".

filter
string

Author-supplied Lucene fragment narrowing what is aggregated. No access filter is injected into it implicitly.

endpointLabel
boolean

Add spider.endpoint to every series. For HTTP and PostgreSQL its value depends on the whisperer's configured request templates: without them, high-cardinality paths become high-cardinality series.

object

Per-protocol latency histogram bounds, overriding the service defaults. Keys are the protocol names above. Each list must be non-empty and strictly increasing; a protocol absent here falls back to the default.

Responses

Request samples

Content type
application/json
{
  • "name": "string",
  • "enabled": true,
  • "team": "string",
  • "targets": [
    ],
  • "whisperers": [
    ],
  • "protocols": [
    ],
  • "filter": "string",
  • "endpointLabel": true,
  • "bucketEdges": {
    }
}

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "name": "string",
  • "enabled": true,
  • "team": "string",
  • "targets": [
    ],
  • "whisperers": [
    ],
  • "protocols": [
    ],
  • "filter": "string",
  • "accessFilters": {
    },
  • "endpointLabel": true,
  • "bucketEdges": {
    },
  • "status": {
    },
  • "createdBy": "string",
  • "dateCreated": "2019-08-24T14:15:22Z",
  • "dateModified": "2019-08-24T14:15:22Z",
  • "technicalStatus": "string",
  • "_eTag": "string"
}

Delete an exporter

Description

Soft-deletes the exporter (technicalStatus: DELETED); it stops being picked up by the run loop immediately. The Maintenance purge job removes the document after its retention window.

Access

  • Member of the owning team holding its settings right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Exporter internal @id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Export

Bulk export of captured communications to CSV: create a background job over a search, list your own jobs, fetch a protocol's column catalog, and download a completed export.

Create a com export job

Description

Exports a search over one protocol's communications to CSV, run as a background job: the server pages the search, joins each com's content, explodes it into rows and writes the CSV to object storage. GET /export/v1/exports lists progress; GET /export/v1/exports/{jobId}/download hands back the finished file.

Rows are per-payload, not per-com: http and webmessagestream export one row per com; redis exports one row per reply value (req.values[] / res.values[], carrying meta.index); kafka exports one row per record (meta.topic, meta.partition, meta.offset, meta.headers); grpc exports one row per stream message (meta.compressed, meta.size). explode picks which side's payloads become rows for a protocol with more than one payload per com - req or res forces a side; anything else follows the com's own direction, then prefers res, then req.

Each column names a source: com, req, res or payload. For grpc and webmessagestream, a stream message belongs to one side of the exchange, so req/res is empty for every message travelling the other way; the API only warns about this - it does not reject the column. The Network-View export dialog offers just com and payload for these two collections as a convention. A unary gRPC com is the exception: it carries both request and response messages on one document, so req and res are meaningful there and the API accepts them.

When the licence carries TRANSCODE, a schema-bound payload body decodes the same way the protocol's own transcoded content view does (http, redis, kafka, grpc - webmessagestream has no schema binding surface and always stays raw). A value with no bound schema, or one that fails to decode, exports raw rather than being dropped, so meta.index never drifts out of sync with the reply it belongs to.

Rules

  • Exactly one entry in protocols per export.
  • A non-admin caller must name at least one whisperer they own; an admin may leave whisperers empty to export across every whisperer.
  • The exact row count is computed synchronously (a size:0 search) and returned as coms; the request is refused if it exceeds the install's configured cap.
  • A per-user daily export quota applies; exceeding it is a 429.

Access

  • Any authenticated user, scoped to their own whisperers
  • Admin
  • Requires the EXPORT licence assess (TEAM tier and above)
Authorizations:
Bearer
Request Body schema: application/json
required

The search, columns and format to export

protocols
required
Array of strings = 1 items
Items Enum: "http" "webmessagestream" "redis" "kafka" "grpc"

Exactly one protocol per export.

query
string

Lucene query narrowing the search, same syntax as the grid.

object
whisperers
Array of strings

Whisperer @ids to export. A non-admin caller must name at least one whisperer they own; an admin may leave this empty to export across every whisperer.

explode
string

Which side's payloads become rows for a protocol with more than one payload per com. req or res forces a side; any other value (including omitted) follows the com's own direction, then prefers res, then req. Has no effect on http/webmessagestream, which are always one row per com.

required
Array of objects (ExportColumnSpec) non-empty
object
object
name
string

Optional label for this export. Max 120 characters.

description
string

Optional longer description. Max 500 characters.

Responses

Request samples

Content type
application/json
{
  • "protocols": [
    ],
  • "query": "string",
  • "timeRange": {
    },
  • "whisperers": [
    ],
  • "explode": "string",
  • "columns": [
    ],
  • "format": {
    },
  • "notify": {
    },
  • "name": "string",
  • "description": "string"
}

Response samples

Content type
application/json
{
  • "jobId": "string",
  • "coms": 0,
  • "warnings": [
    ]
}

List the caller's com export jobs

Description

Lists export jobs, newest first. available and expiresAt are computed against object storage at read time; they are not part of the stored job document.

Rules

  • Scoped to the caller's own jobs, always - a non-admin caller can never widen this, and an admin only sees every caller's jobs by passing all=true explicitly.

Access

  • Any authenticated user, their own jobs
  • Admin, all=true for every caller's jobs
Authorizations:
Bearer
query Parameters
status
string

Filter by actionStatus (e.g. ACTIVE, COMPLETED, FAILED)

whisperers
string

Comma-separated whisperer @ids to filter on

size
integer <= 99

Page size, capped at 99

next
string

JSON-encoded search_after cursor from a previous page's nextPage

from
integer

Lower bound, epoch seconds. Defaults to 0.

to
integer

Upper bound, epoch seconds. Defaults to now plus 24 hours.

all
boolean

Admin only. true lists every caller's jobs instead of just the caller's own.

Responses

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "nextPage": {
    }
}

Get one protocol's addressable export columns

Description

Returns the Elasticsearch-mapping-derived column paths for one protocol's com document, plus its fixed meta.* keys. Tag keys (tags.<key>) are not included - they are data-dependent and vary per protocol; a tags.<key> or value.<path> column is still accepted by POST /export/v1/exports even though it is not listed here.

Access

  • Any authenticated user
Authorizations:
Bearer
query Parameters
protocol
required
string
Enum: "http" "webmessagestream" "redis" "kafka" "grpc"

The protocol to fetch columns for

Responses

Response samples

Content type
application/json
{
  • "protocol": "string",
  • "columns": [
    ]
}

Download a completed export

Description

Redirects to a presigned, time-limited object-storage URL for the finished CSV (or zipped CSV). The URL forces a download with the export's own name, rather than an in-browser render.

A caller sending Accept: application/json gets the same presigned URL as a JSON body instead of a redirect, since a cross-origin 302 cannot be followed by a browser fetch().

Rules

  • Authorised on the job's own stored creator and whisperer set, never on the request - a caller cannot widen access by any query parameter.
  • The presigned URL expires after a short, fixed TTL; re-request this endpoint for a fresh one.

Access

  • The job's own creator (or, during impersonation, the impersonated account)
  • Admin
Authorizations:
Bearer
path Parameters
jobId
required
string

Export job @id

Responses

Response samples

Content type
application/json
{
  • "url": "string",
  • "filename": "string"
}

Packets

Network packets, as captured on the wire(less). ANSI layer 3.

Push packets

Description

Process a json payload of packets and:

  • Make them ready for parsing.
  • Save them (optional).

Checks

  • Spider packets structure

Access

  • Whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

Json payload with packets to analyse by Spider.

Array
@id
string

Unique id of the packet in the system.

@type
string
Value: "Packet"
version
string
Value: "2.0"

Version of the schema.

name
string

Name of the packet (for display).

whisperer
string

Whisperer that captured the packet

instanceId
string

Instance id of the whisperer

tcpSession
string

Id of the Tcp session the packet is in

timestamp
number <double>

Unix timestamp of capture, with microseconds

length
integer

Size of the packet (size of rawPacket.buf buffer)

object

List of protocols used by this packet, keys are protocols name: TCP, UDP, IPv4...

object

Responses

Request samples

Content type
application/json
{
  • "@id": "ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5",
  • "@type": "Packet",
  • "version": "2.0",
  • "commonId": "ROlrqlFhTY2ayXIxTV2uZA.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5",
  • "name": "456285.172.16.102.72-43118-172.16.102.125-8080#5",
  • "whisperer": "ROlrqlFhTY2ayXIxTV2uZA",
  • "instanceId": "rd-srv508-bes",
  • "tcpSession": "ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931",
  • "timestamp": 1642625100.188968,
  • "length": 229,
  • "protocols": {
    },
  • "rawPacket": {
    }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for packets

Description

Searches or aggregation analysis on packets

Also available by GET method on the collection

Rules

Admin may search without specifying a whisperer

Access

  • Admin or client
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a packet

Description

Get a packet

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of packet

Responses

Response samples

Content type
application/json
{
  • "@id": "ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5",
  • "@type": "Packet",
  • "version": "2.0",
  • "commonId": "ROlrqlFhTY2ayXIxTV2uZA.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5",
  • "name": "456285.172.16.102.72-43118-172.16.102.125-8080#5",
  • "whisperer": "ROlrqlFhTY2ayXIxTV2uZA",
  • "instanceId": "rd-srv508-bes",
  • "tcpSession": "ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931",
  • "timestamp": 1642625100.188968,
  • "length": 229,
  • "rawPacket": {
    },
  • "protocols": {
    },
  • "date": "2022-01-19T20:45:00.188Z",
  • "minute": "2022-01-19T20:45:00.000Z",
  • "protocolsList": [
    ]
}

Aggregate TCP payload for provided packets (fallback)

Description

Build the tcp payload of the packets listed in input.

Also available by GET method

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

List of packets identifiers

Array
string

System id of packet

Responses

Request samples

Content type
application/json
[
  • "string"
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a collection of packets (fallback)

Description

Get the packets listed in input (by id).

Also available by GET method

Access

  • Admin
  • Application (another service)
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

List of packets identifiers

Array
string

System id of packet

Responses

Request samples

Content type
application/json
[
  • "string"
]

Response samples

Content type
application/json
[
  • { }
]

Get a packets of a Tcp Session between two indices (or from a index to the end)

Description

Get the packets of the {tcpSession} in input, from {indexStart} to {indexEnd} (opt).

Access

  • Admin
  • Application (another service)
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Range of packets to receive

tcpSession
string

Id of the Tcp session the packet is in

indexStart
integer

Index above which the first packet to send must be (exclusive)

indexEnd
string

Index before which the last packet to send must be (inclusive), optional

Responses

Request samples

Content type
application/json
{
  • "tcpSession": "string",
  • "indexStart": 0,
  • "indexEnd": "string"
}

Response samples

Content type
application/json
[
  • { }
]

Aggregate TCP payload for several group of packets

Description

Build the tcp payload of the list of packets groups listed in input.

Also available by GET method

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

List of packets groups

Array
requestId
string

Id of group of packets, internal to client

packetIds
Array of strings

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
[
  • {
    }
]

Set packets as parsed

Description

Set the packets of {tcpSession} before {maxIndex} as parsed. When parsed, they are removed from working memory in Redis. Depending of whisperer settings to save Packets:

  • Poller may synchronize them to ES, then remove them from Redis.
  • Poller may plainly remove them from Redis.
  • Pack Update may directly remove them from Redis, if already processed by Poller.

This API is used in real time processing of packets to optimise Redis usage and speed of processing.

Access

  • Admin
  • Application (another service)
Authorizations:
Bearer
Request Body schema: application/json
required
Array
tcpSession
string

System id of the TCP session owning the packets

maxIndex
integer

Maximum index that has been parsed (inclusive)

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Purge packets

Description

Create a asynchronous purging job of packets.

Access

  • Admin
  • User asking to purge only on whisperers it owns of with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

List of packets identifiers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge packets progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Tcp sessions

TCP sessions: consistent stateful communications of packets between 2 hosts. Can contain any kind of exchanges. ANSI layer 4.

Push Tcp sessions

Description

Stores TCP sessions and trigger parsing of payload according to Whisperers parsing configuration.

Access

  • Whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

Tcp Session with packets id to analyse by Spider.

Array
@id
string

System id of TCP session

name
string
object

Client host

object

Server host

state
string
Enum: "SYN_SENT" "SYN_RECEIVED" "ESTABLISHED" "CLOSE_WAIT" "LAST_ACK" "CLOSED"

State of TCP session lifecycle

packetsCount
integer

Count of packets in the sessions

synTimestamp
number <double>

Timestamp of SYN packet

missedSyn
boolean

If whisperer missed SYN

connectTimestamp
number <double>

Timestamp when connection was established

firstTimestamp
number <double>

Timestamp of first packet (different from SYN when missedSyn)

lastTimestamp
number <double>

Timestamp of last packet

object

Out packets (responses from server)

object

In packets (responses from server)

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a TCP session

Description

Get a TCP session

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of TCP session

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "Tcp session",
  • "version": "string",
  • "name": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "src": {
    },
  • "dst": {
    },
  • "state": "SYN_SENT",
  • "packetsCount": 0,
  • "syn": 0.1,
  • "missedSyn": true,
  • "connect": 0.1,
  • "first": 0.1,
  • "firstDate": "2019-08-24",
  • "last": 0.1,
  • "lastDate": "2019-08-24",
  • "duration": 0.1,
  • "timespan": {
    },
  • "latency": 0.1,
  • "out": {
    },
  • "in": {
    },
  • "parsers": {
    },
  • "dateModified": "2019-08-24"
}

Search for TCP sessions (fallback)

Description

Searches or aggregation analysis on TCP sessions

Also available by GET method on the collection

Rules

May ask for an aggregation, with size:0 and no whisperers defined:

  • Admin
  • User with admin monitoring right

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

type
string
Enum: "TcpSession" "UdpFlow" "UdpConversation"

Filter results by session type. TcpSession excludes UDP kinds via a must_not clause (pre-migration documents have no indexed @type, so a positive match would wrongly exclude them); UdpFlow/UdpConversation match on the indexed @type. Omit to search every type (unchanged default behaviour).

Responses

Request samples

Content type
application/json
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startDate": "2019-01-18T05:28:18.676123",
  • "stopDate": "2019-01-18T10:01:57.362456",
  • "type": "UdpConversation"
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Preaggregated search for parsing status histogram

Description

Histogram aggregation analysis on TCP sessions

Rules

Size:0, no next, no sort.

  • Admin
  • User with admin monitoring right

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Modify the parsing result of a TCP session

Description

Update the Tcp session once the parsing is done.

Rules

  • Only /parsers block can be updated

Access

  • Admin
  • Application
  • Whisperer
    • Limited access to the whisperer linked to the Tcp session
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of TCP session

header Parameters
If-Match
required
string

eTag of previous state of the TCP session

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    },
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Modify the parsing result of TCP sessions, in bulk

Description

Update Tcp sessions once the parsing is done.

Rules

  • Only /parsers block can be updated

Access

  • Admin
  • Application
Authorizations:
Bearer
Request Body schema: application/json
required

Array of Patches to TcpSessions

Array
@id
string

System Id of the Tcp session to update

_eTag
string

eTag of the Tcp session to update

Array of objects (JsonPatch)

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "successes": [
    ],
  • "failures": [
    ]
}

Get a set of TCP sessions with Tls by their client random

Description

Get a set of TCP session from a list of client ranndoms.

Access

  • Admin
  • Tls keys linker application
Authorizations:
Bearer
Request Body schema: application/json
required

Array of Whisperer+ClientRandom

Array
whisperer
string

Whisperer Id of Whisperer having captured the TcpSession

clientRandom
string

Client random found in the TLS handshake

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
[
  • { }
]

Ask for Tcp sessions to parse by {type} parser

Description

Get Tcp sessions that needs parsing by {type} parser

  • Returns a collection of Tcp sessions to parse

Access

  • Admin
  • Application
Authorizations:
Bearer
path Parameters
type
required
string
Value: "HTTP"

Type of parsing

Request Body schema: application/json
required

Polling parameters

before
string <date-time>

Date before which to get sessions to parse. Safety delay.

Responses

Request samples

Content type
application/json
{
  • "before": "2019-08-24T14:15:22Z"
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ]
}

Ask for Tcp sessions in Warning status to parse by {type} parser

Description

Get Tcp sessions that have already been parsed by {type} parser, but that ended in WARNING parsing status. Indeed, most often the WARNING status means that the TCP session could not be parsed do to missing packets. We retry a bit ater once packets should be there.

  • Returns a collection of Tcp sessions to parse

Access

  • Admin
  • Application
Authorizations:
Bearer
path Parameters
type
required
string
Value: "HTTP"

Type of parsing

Request Body schema: application/json
required

Polling parameters

before
string <date-time>

Date before which to get sessions to parse. Safety delay.

Responses

Request samples

Content type
application/json
{
  • "before": "2019-08-24T14:15:22Z"
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ]
}

Purge TCP sessions

Description

Create a asynchronous purging job of TCP sessions.

Access

  • Admin
  • User asking to purge only on whisperers it owns of with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

List of TCP sessions identifiers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge TCP sessions progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Get parsing queue status

Description

Get parsing queue status for TCP sessions in WAITING parsing status

Access

  • Admin
  • Application
  • Customer
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "totalInWaiting": 2,
  • "ageOfFirst": 2345,
  • "ageOfLast": 2456
}

Http communications

HTTP communications: unitary communications used on the Web. ANSI layer 5-7.

Purge HTTP communications

Description

Create a asynchronous purging job of HTTP communications.

Access

  • Admin
  • User asking to purge only on whisperers it owns of with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

List of HTTP communications identifiers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge Http communications progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Purge HTTP parsing logs

Description

Create a asynchronous purging job of HTTP parsing logs.

Access

  • Admin
  • User asking to purge only on whisperers it owns of with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

List of HTTP parsing logs identifiers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge Http parsing logs progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Test HTTP tag & template rules against the parsing engine

Description

Renamed from /web-parser/v1/playground - the path now distinguishes it from the sibling /web-parser/v1/wms-playground (SSE/WebSocket message tags & templates), served by the same pod.

Runs draft tag and template rules through the actual Go (RE2) HTTP parsing engine against a set of already-parsed HTTP communications. Used by the Whisperer parsing-config "Playground". Because it uses the real engine, it reflects exactly what the parser would extract - and reports regexes the parser cannot use (RE2 rejects backreferences / lookaround).

When contentSchemas is supplied, each per-com result also carries the transcoded request/response bodies and the per-side schema resolution (which binding fired, or an error explaining the miss/decode failure). Invalid URI templates in contentSchemas are surfaced as rule errors with scope "contentSchema".

Access

  • Admin
  • User with whisperer-config rights on every whisperer referenced by the supplied communications
  • Not available to access-filter-restricted users
Authorizations:
Bearer
Request Body schema: application/json
required

Communications to test against, plus the draft tag/template rules

ids
required
Array of strings

@ids of already-parsed HttpCom to test against (1–50)

object
templates
Array of objects

Request template rules

Array of objects

Draft content-schema bindings to test. Each binding declares which HTTP method, URI template, and content-type should trigger transcoding, and the schema (format + schemaId + messageType) to use for the request and/or response side. When supplied, the playground transcodes matching binary bodies before tag/template extraction - mirroring the live parser hot path - and attaches transcoded and schemaResolution to each per-com result.

Responses

Request samples

Content type
application/json
{
  • "ids": [
    ],
  • "tags": {
    },
  • "templates": [
    ],
  • "contentSchemas": [
    ]
}

Response samples

Content type
application/json
{
  • "results": {
    },
  • "ruleErrors": [
    ]
}

Test SSE/WebSocket message tag & template rules against the parsing engine

Description

Served by the web parser (web-parser), alongside the sibling /web-parser/v1/http-playground.

Runs draft per-message tag and template rules through the actual Go SSE parsing engine against a set of already-parsed WebMessageStreamCom messages. Used by the Whisperer parsing-config "Playground" for the stream's server.sse rules. Because it uses the real engine, it reflects exactly what the parser would extract.

Each result carries both the merged outcome AND the two halves that produced it: template/tags is what this message ends up with (inherited value, overridden/unioned by this message's own rules - see addMatchesToTag's union semantics), while inheritedTemplate/inheritedTags is what the opening request's handshake already established, on its own, before this message's rules ran. A merge that changed nothing is otherwise indistinguishable from a rule that never fired; comparing the two tells them apart. An SSE message has no request side of its own - identity (URI) lives on req for both directions, but every extracted/inherited tag or template value lands under the res side of tags/inheritedTags.

Access

  • Admin
  • User with whisperer-config rights on every whisperer referenced by the supplied communications
  • Not available to access-filter-restricted users
Authorizations:
Bearer
Request Body schema: application/json
required

Communications to test against, plus the draft per-message tag/template rules

ids
required
Array of strings

@ids of already-parsed WebMessageStreamCom to test against (1–50)

object
templates
Array of objects

Message template rules

Responses

Request samples

Content type
application/json
{
  • "ids": [
    ],
  • "tags": {
    },
  • "templates": [
    ]
}

Response samples

Content type
application/json
{
  • "results": {
    },
  • "ruleErrors": [
    ]
}

Import Http communications

Description

Import a collection of Http communications

Access

  • Admin
  • Whisperer
    • Only Httpcoms on this Whisperer can be created
Authorizations:
Bearer
Request Body schema: application/json
@type
string
Value: "HttpComDownload"
user
string
totalItems
integer

Count of Http communications to import

Array of objects (HttpCommunication)

Array of Http communications

Responses

Request samples

Content type
application/json
{
  • "@type": "HttpComDownload",
  • "user": "string",
  • "totalItems": 0,
  • "items": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get an HTTP communication

Description

Get an Http communication by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of HTTP Communication

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "name": "string",
  • "tcpSession": "string",
  • "httpPers": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "uploaded": true,
  • "stats": {
    },
  • "req": {
    },
  • "res": {
    }
}

Get the headers part of an HTTP communication (request or response)

Description

Get the HTTP headers of an Http communication by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of HTTP Communication

part
required
string
Enum: "req" "res"

Request or Response

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get the body part of an HTTP communication (request or response)

Description

Get the payload body of an Http communication by its Id

Output

The output can be in 5 types, based on view parameter:

  • [not defined]: Get the payload inline with original content type and content encoding, and with filename {id}.{extension based on mime type}
  • raw: Get the payload as text with original encoding, inline, and with filename {id}.{extension based on mime type}
  • download: Get the payload with original content-type and encoding, in an attachement file {id}.{extension based on mime type}
  • octet-stream: Get the payload as captured on the network, in a binary attachement {id}.a
  • transcoded: Get the transcoded/decoded JSON body when a content-schema binding matched, as application/json inline

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of HTTP Communication

part
required
string
Enum: "req" "res"

Request or Response

query Parameters
view
required
string
Enum: "raw" "download" "octet-stream" "transcoded"

Format of output:

  • raw: body as text with original encoding, inline
  • download: body with original content-type/encoding, as attachment
  • octet-stream: raw captured bytes as binary attachment
  • transcoded: server-side transcode to JSON using the whisperer's content-schema binding. Returns application/json on success; 422 LD-Error when no binding matches or transcoding fails; 502 when the whisperer config is unavailable. Access checks are identical to raw.

Responses

Response samples

Content type
No sample

Get an HTTP parsing log

Description

Get an Http parsing log by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of HTTP Persistent Connection

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "name": "string",
  • "tcpSession": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "first": 0.1,
  • "last": 0.1,
  • "lastPacketLotComplete": 0,
  • "lastPacketParsedIndex": 0,
  • "parsingCount": 0,
  • "itemsFound": 0,
  • "packetLots": [
    ]
}

Search for Http communications

Description

Searches or aggregation analysis on Http communications

Also available by GET method on the collection

Rules

May ask for an aggregation, with size:0 and no whisperers defined:

  • Admin
  • User with admin monitoring right

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Preaggregated search for parsing status histogram

Description

Histogram aggregation analysis on Http communications

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

PostgreSQL communications

PostgreSQL communications: unitary communications using PostgreSQL protocol. ANSI layer 5-7.

Import PostgreSQL communications

Description

Import a collection of PostgreSQL communications

Access

  • Admin
  • Whisperer
    • Only PgComs on this Whisperer can be created
Authorizations:
Bearer
Request Body schema: application/json
@type
string
Value: "PgComDownload"
user
string
totalItems
integer

Count of PostgreSQL communications to import

Array of objects (PgCommunication)

Array of PostgreSQL communications

Responses

Request samples

Content type
application/json
{
  • "@type": "PgComDownload",
  • "user": "string",
  • "totalItems": 0,
  • "items": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Test PostgreSQL tag & template rules against the parsing engine

Description

Runs draft tag and template rules through the actual Go (RE2) PostgreSQL parsing engine against a set of already-parsed PostgreSQL communications. Used by the Whisperer parsing-config "Playground". Because it uses the real engine, it reflects exactly what the parser would extract - and reports regexes the parser cannot use (RE2 rejects backreferences / lookaround).

Access

  • Admin
  • User with whisperer-config rights on every whisperer referenced by the supplied communications
  • Not available to access-filter-restricted users
Authorizations:
Bearer
Request Body schema: application/json
required

Communications to test against, plus the draft tag/template rules

ids
Array of strings

@ids of already-parsed PgCom to test against (1–50)

object
templates
Array of objects

Request template rules

Responses

Request samples

Content type
application/json
{
  • "ids": [
    ],
  • "tags": {
    },
  • "templates": [
    ]
}

Response samples

Content type
application/json
{
  • "results": {
    },
  • "ruleErrors": [
    ]
}

Get an PostgreSQL communication

Description

Get an PostgreSQL communication by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of PostgreSQL Communication

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "name": "string",
  • "tcpSession": "string",
  • "pgParsing": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "uploaded": true,
  • "stats": {
    },
  • "req": {
    },
  • "res": {
    },
  • "connectionMetadata": {
    }
}

Get an PostgreSQL parsing log

Description

Get an PostgreSQL parsing log by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of PostgreSQL parsing log

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "name": "string",
  • "tcpSession": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "first": 0.1,
  • "last": 0.1,
  • "lastPacketLotComplete": 0,
  • "lastPacketParsedIndex": 0,
  • "parsingCount": 0,
  • "itemsFound": 0,
  • "packetLots": [
    ]
}

Search for PostgreSQL communications

Description

Searches or aggregation analysis on PostgreSQL communications

Also available by GET method on the collection

Rules

May ask for an aggregation, with size:0 and no whisperers defined:

  • Admin
  • User with admin monitoring right

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Preaggregated search for parsing status histogram

Description

Histogram aggregation analysis on PostgreSQL communications

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Redis communications

Redis communications: unitary command/reply exchanges decoded from the Redis RESP protocol (RESP2/RESP3). ANSI layer 5-7.

Import Redis communications

Description

Import a collection of Redis communications

Access

  • Admin
  • Whisperer
    • Only RedisComs on this Whisperer can be created
Authorizations:
Bearer
Request Body schema: application/json
@type
string
Value: "RedisComDownload"
user
string
totalItems
integer

Count of Redis communications to import

Array of objects (RedisCommunication)

Array of Redis communications

Responses

Request samples

Content type
application/json
{
  • "@type": "RedisComDownload",
  • "user": "string",
  • "totalItems": 0,
  • "items": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Test Redis tag & template rules against the parsing engine

Description

Runs draft tag and template rules through the actual Go (RE2) Redis parsing engine against a set of already-parsed Redis communications. Used by the Whisperer parsing-config "Playground". Because it uses the real engine, it reflects exactly what the parser would extract - and reports regexes the parser cannot use (RE2 rejects backreferences / lookaround).

Access

  • Admin
  • User with whisperer-config rights on every whisperer referenced by the supplied communications
  • Not available to access-filter-restricted users
Authorizations:
Bearer
Request Body schema: application/json
required

Communications to test against, plus the draft tag/template rules

ids
Array of strings

@ids of already-parsed RedisCom to test against (1–50)

object
templates
Array of objects

Request template rules

Responses

Request samples

Content type
application/json
{
  • "ids": [
    ],
  • "tags": {
    },
  • "templates": [
    ]
}

Response samples

Content type
application/json
{
  • "results": {
    },
  • "ruleErrors": [
    ]
}

Purge Redis communications

Description

Create an asynchronous purging job of Redis communications.

Access

  • Admin
  • User asking to purge only on whisperers it owns or with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

Purge window and target whisperers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge Redis communications progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Purge Redis parsing logs

Description

Create an asynchronous purging job of Redis parsing logs.

Access

  • Admin
  • User asking to purge only on whisperers it owns or with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

Purge window and target whisperers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge Redis parsing logs progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Get a Redis communication

Description

Get a Redis communication by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of Redis Communication

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "dateModified": "2019-08-24",
  • "name": "string",
  • "tcpSession": "string",
  • "redisParsing": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "uploaded": true,
  • "kind": "simple",
  • "transactionId": "string",
  • "stats": {
    },
  • "req": {
    },
  • "res": {
    },
  • "connectionMetadata": {
    }
}

Get the request or response values of a Redis communication

Description

Returns the value(s) carried by the request (req) or the reply (res) of a Redis communication, fetched on demand from the dedicated content store. Multi-value commands and replies (MSET, MGET, HSET, HGETALL, LRANGE…) return one entry per value.

The view query parameter controls the representation:

  • default / raw - the RESP values as a per-value JSON array (one element per value)
  • transcoded - when a content schema is bound to the (database, key glob), the values decoded from their binary form (Protobuf / MessagePack) to JSON. Falls back to raw when no binding matches or decoding fails.
  • download - the byte-exact raw bytes, as an attachment

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of Redis Communication

part
required
string
Enum: "req" "res"

Which side to fetch: request or response values

query Parameters
view
string
Enum: "raw" "transcoded" "download"

Representation of the values (defaults to raw)

Responses

Response samples

Content type
[
  • { }
]

Get a Redis parsing log

Description

Get a Redis parsing log by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of Redis parsing log

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "name": "string",
  • "tcpSession": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "first": 0.1,
  • "last": 0.1,
  • "lastPacketLotComplete": 0,
  • "lastPacketParsedIndex": 0,
  • "parsingCount": 0,
  • "itemsFound": 0,
  • "packetLots": [
    ]
}

Search for Redis communications

Description

Searches or aggregation analysis on Redis communications

Also available by GET method on the collection

Rules

May ask for an aggregation, with size:0 and no whisperers defined:

  • Admin
  • User with admin monitoring right

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Preaggregated search for parsing status histogram

Description

Histogram aggregation analysis on Redis communications

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

gRPC communications

gRPC communications: unitary Remote Procedure Calls decoded from gRPC-over-HTTP/2. ANSI layer 5-7.

Import gRPC communications

Description

Re-import a collection of already-parsed gRPC communications (the Network-View download → upload round-trip). Each communication carries its per-message content inline on req.messages / res.messages, which the parser turns back into the per-message content store.

Served by the web parser (web-parser); the standalone gRPC parser service was retired in the M2a absorption and the /grpc-parser path prefix is kept for compatibility with existing clients.

Access

  • Admin
  • Whisperer
    • Only GrpcComs on this Whisperer can be created
Authorizations:
Bearer
Request Body schema: application/json
@type
string
Value: "GrpcComDownload"
user
string
totalItems
integer

Count of gRPC communications to import

Array of objects (GrpcCommunication)

Array of gRPC communications

Responses

Request samples

Content type
application/json
{
  • "@type": "GrpcComDownload",
  • "user": "string",
  • "totalItems": 0,
  • "items": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Test gRPC tag & template rules against the parsing engine

Description

Served by the web parser (web-parser). Renamed from /grpc-parser/v1/playground in the M2a absorption -- the HTTP and gRPC playgrounds now share a service and cannot share a path (the gateway strips the first path segment, so both prefixes arrive at the pod identically).

Runs draft tag and template rules through the actual Go gRPC parsing engine against a set of already-parsed gRPC communications. Used by the Whisperer parsing-config "Playground". gRPC tags are field paths (into the request/response metadata or the decoded protobuf message), while templates are regular expressions over a selected fullMethod / metadata / message concatenation, defaulting to the full method. Because it uses the real engine, it reflects exactly what the parser would extract - and reports invalid rules (e.g. a content schema binding the transcoder cannot use).

Access

  • Admin
  • User with whisperer-config rights on every whisperer referenced by the supplied communications
  • Not available to access-filter-restricted users
Authorizations:
Bearer
Request Body schema: application/json
required

Communications to test against, plus the draft tag/template rules

ids
Array of strings

@ids of already-parsed GrpcCom to test against (1–50)

object
templates
Array of objects

Request template rules

Responses

Request samples

Content type
application/json
{
  • "ids": [
    ],
  • "tags": {
    },
  • "templates": [
    ]
}

Response samples

Content type
application/json
{
  • "results": {
    },
  • "ruleErrors": [
    ]
}

Purge gRPC communications

Description

Create an asynchronous purging job of gRPC communications. The associated per-message content is purged in the same cascade.

Served by the web parser (web-parser); the standalone gRPC parser service was retired in the M2a absorption and the /grpc-parser path prefix is kept for compatibility with existing clients.

Access

  • Admin
  • User asking to purge only on whisperers it owns or with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

Purge window and target whisperers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge gRPC communications progress

Description

Get purge progress

Served by the web parser (web-parser); the standalone gRPC parser service was retired in the M2a absorption and the /grpc-parser path prefix is kept for compatibility with existing clients.

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Purge gRPC parsing logs

Description

Create an asynchronous purging job of gRPC parsing logs.

Served by the web parser (web-parser); the standalone gRPC parser service was retired in the M2a absorption and the /grpc-parser path prefix is kept for compatibility with existing clients.

Access

  • Admin
  • User asking to purge only on whisperers it owns or with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

Purge window and target whisperers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge gRPC parsing logs progress

Description

Get purge progress

Served by the web parser (web-parser); the standalone gRPC parser service was retired in the M2a absorption and the /grpc-parser path prefix is kept for compatibility with existing clients.

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Get a gRPC communication

Description

Get a gRPC communication by its Id. The response also carries a transcode hint reporting, independently for the request and the response side, whether a content schema is bound and would decode the protobuf message(s).

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of gRPC Communication

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "dateModified": "2019-08-24",
  • "name": "string",
  • "tcpSession": "string",
  • "grpcParsing": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "uploaded": true,
  • "kind": "unary",
  • "transactionId": "string",
  • "direction": "req",
  • "index": 0,
  • "terminal": true,
  • "stats": {
    },
  • "req": {
    },
  • "connectionMetadata": { },
  • "res": {
    }
}

Get the request or response messages of a gRPC communication

Description

Returns the protobuf message(s) carried by the request (req) or the response (res) of a gRPC communication, fetched on demand from the per-message content store, as a JSON array with one entry per message (index, tsStart, tsEnd, size, compressed, message). For a streaming-message communication only its own side carries content.

The view query parameter controls the representation:

  • default / raw - the raw protobuf message bytes (base64) as a per-message JSON array
  • transcoded - when a content schema is bound to the (service, method), each message decoded from its binary protobuf form to JSON. Falls back to raw per message when no binding matches or decoding fails.
  • download / octet-stream - the side's message bytes concatenated in index order, as an attachment

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of gRPC Communication

part
required
string
Enum: "req" "res"

Which side to fetch: request or response messages

query Parameters
view
string
Enum: "raw" "transcoded" "download" "octet-stream"

Representation of the messages (defaults to raw)

Responses

Response samples

Content type
[
  • { }
]

Reconstruct gRPC calls (transactions)

Description

Reconstructs each gRPC call from its per-message communications, grouped by transactionId. Returns one entry per call with its start / stop, request and response message counts, and the denormalized fullMethod / kind / final gRPC status. Used to present a streaming call as a single row.

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Reconstruct a single gRPC call (transaction)

Description

Reconstructs a single gRPC call by its transactionId.

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
transactionId
required
string

Transaction (call) id grouping the per-message communications

Responses

Response samples

Content type
application/json
{ }

Get a gRPC parsing log

Description

Get a gRPC parsing log by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of gRPC parsing log

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "name": "string",
  • "state": "string",
  • "tcpSession": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "first": 0.1,
  • "firstDate": "2019-08-24",
  • "last": 0.1,
  • "lastDate": "2019-08-24",
  • "dateModified": "2019-08-24",
  • "packetLots": [
    ]
}

Search for gRPC communications

Description

Searches or aggregation analysis on gRPC communications

Also available by GET method on the collection

Rules

May ask for an aggregation, with size:0 and no whisperers defined:

  • Admin
  • User with admin monitoring right

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Kafka communications

Kafka communications: unitary request/response exchanges decoded from the Kafka wire protocol (ApiKey-keyed). ANSI layer 5-7.

Import Kafka communications

Description

Re-import a collection of already-parsed Kafka communications (the Network-View download → upload round-trip). Each communication carries its per-record content inline on req.records / res.records, which the parser turns back into the per-record content store.

Access

  • Admin
  • Whisperer
    • Only KafkaComs on this Whisperer can be created
Authorizations:
Bearer
Request Body schema: application/json
@type
string
Value: "KafkaComDownload"
user
string
totalItems
integer

Count of Kafka communications to import

Array of objects (KafkaCom)

Array of Kafka communications

Responses

Request samples

Content type
application/json
{
  • "@type": "KafkaComDownload",
  • "user": "string",
  • "totalItems": 0,
  • "items": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Test Kafka tag & template rules against the parsing engine

Description

Runs draft tag and template rules through the actual Go Kafka parsing engine against a set of already-parsed Kafka communications. Used by the Whisperer parsing-config "Playground". Kafka tags are field paths (into the decoded record key or value JSON), while templates are regular expressions over a selected command / topic / clientId / decoded message concatenation, defaulting to command + topics. Because it uses the real engine, it reflects exactly what the parser would extract - and reports invalid rules (e.g. a content schema binding the transcoder cannot use).

Access

  • Admin
  • User with whisperer-config rights on every whisperer referenced by the supplied communications
  • Not available to access-filter-restricted users
Authorizations:
Bearer
Request Body schema: application/json
required

Communications to test against, plus the draft tag/template rules

ids
Array of strings

@ids of already-parsed KafkaCom to test against (1–50)

object
templates
Array of objects

Request template rules

Responses

Request samples

Content type
application/json
{
  • "ids": [
    ],
  • "tags": {
    },
  • "templates": [
    ]
}

Response samples

Content type
application/json
{
  • "results": {
    },
  • "ruleErrors": [
    ]
}

Purge Kafka communications

Description

Create an asynchronous purging job of Kafka communications. The associated per-record content is purged in the same cascade.

Access

  • Admin
  • User asking to purge only on whisperers it owns or with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

Purge window and target whisperers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge Kafka communications progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Purge Kafka parsing logs

Description

Create an asynchronous purging job of Kafka parsing logs.

Access

  • Admin
  • User asking to purge only on whisperers it owns or with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

Purge window and target whisperers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge Kafka parsing logs progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Get a Kafka communication

Description

Get a Kafka communication by its Id.

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of Kafka Communication

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "KafkaCom",
  • "kind": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "tcpSession": "string",
  • "kafkaParsing": "string",
  • "status": "COMPLETE",
  • "_update": 0,
  • "dateCreated": "2019-08-24",
  • "dateModified": "2019-08-24",
  • "stats": {
    },
  • "req": {
    },
  • "res": {
    }
}

Get the request or response records of a Kafka communication

Description

Returns the record key/value bytes carried by the request (req) or the response (res) of a Kafka communication, fetched on demand from the dedicated per-record content store. Each entry carries the record topic, partition, offset, raw key and value bytes (base64), and a filtered flag for suppressed records.

The view query parameter controls the representation:

  • default / raw - the raw record bytes (base64) as a per-record JSON array
  • transcoded - when a content schema binding matches the (topic, side), each record's key and value are decoded from their Avro or Confluent-framed binary form to JSON. Falls back to raw per record when no binding matches or decoding fails.

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of Kafka Communication

part
required
string
Enum: "req" "res"

Which side to fetch: request or response records

query Parameters
view
string
Enum: "raw" "transcoded" "download"

Representation of the records (defaults to raw)

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "KafkaComContent",
  • "whisperer": "string",
  • "dateModified": "2019-08-24",
  • "req": {
    },
  • "res": {
    }
}

Get a Kafka parsing log

Description

Get a Kafka parsing log by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of Kafka parsing log

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "name": "string",
  • "state": "string",
  • "status": "string",
  • "tcpSession": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "first": 0.1,
  • "firstDate": "2019-08-24",
  • "last": 0.1,
  • "lastDate": "2019-08-24",
  • "dateModified": "2019-08-24",
  • "packetLots": [
    ]
}

Search for Kafka communications

Description

Searches or aggregation analysis on Kafka communications

Also available by GET method on the collection

Rules

May ask for an aggregation, with size:0 and no whisperers defined:

  • Admin
  • User with admin monitoring right

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Preaggregated search for parsing status histogram

Description

Histogram aggregation analysis on Kafka communications

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Tls keys

TLS keys used to encrypt TLS sessions. ANSI layer 5.

Push Tls keys

Description

Stores Tls keys and trigger parsing of to link them to TCP sessions.

Access

  • Gociphers
  • Whisperers
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

Tls Keys to store.

Array
object

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "message": "3 Tls keys saved successfully.",
  • "saved": 3,
  • "rejected": 0
}

Ask for new Tls keys to link to TCP sessions

Description

Get Tls keys that needs linking

  • Returns a collection of Tls keys to link

Access

  • Admin
  • Application
Authorizations:
Bearer
Request Body schema: application/json
required

Polling parameters

before
string <date-time>

Date before which to get tls keys to parse. Safety delay.

Responses

Request samples

Content type
application/json
{
  • "before": "2019-08-24T14:15:22Z"
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ]
}

Ask for existing Tls keys to be removed from cache

Description

Remove Tls keys from cache

Access

  • Admin
  • Application: tls-leys-linker
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Whisp

To control Whisperers, the probes sending sniffed communications to Spider

Ciphers

To control Gociphers, the probes sending TLS keys to Spider

Create a new Gocipher

Description

Create a new Gocipher, and associate it to the owner customer.

Access

  • Customer, with Gociphers creation rights
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

Gocipher creation request

customer
required
string

System Id of the customer.

name
required
string

Name of the Gocipher to create.

Responses

Request samples

Content type
application/json
{
  • "customer": "YOD66VZ54Jih",
  • "name": "Dev cluster"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get default Configuration for Gociphers

Description

Get default configuration.

Access

  • User
  • Admin
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Gociphers

Description

Search for Gociphers

Also available by GET method on the collection

Rules

If client is not admin, the search will be limited to the Gociphers owned by this customer or shared with him.

Access

  • Admin
  • Customer
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a Gocipher

Description

Get a Gocipher's details.

Rules

Only admins can see DELETED Gociphers.

Access

  • User owning the Gocipher
  • The own Gocipher
  • User being shared access to this Gocipher
  • User of a Team having access to this Gocipher
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Gocipher

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "Gocipher",
  • "version": "string",
  • "name": "string",
  • "customer": "string",
  • "apikey": "string",
  • "config": {
    },
  • "users": [
    ],
  • "teams": [
    ],
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "editor": "string",
  • "dateModified": "2019-08-24"
}

Change Gocipher's name or shared users/teams

Description

Updates a customer. You can:

  • Update Gocipher's name
  • Change sharing settings: users and users rights on this Gocipher

Rules

  • Can do any change:
    • Admin
    • User owning the Gocipher
    • User of the team owning the Gocipher with Gociphers right
  • User being shared access to this Gocipher with config rights can:
    • Change the name
  • User being shared access to this Gocipher with share rights can:
    • Add or remove users from the sharing settings
  • User being shared access to this Gocipher with rights change rights can:
    • Change rights of users in the sharing settings

Access

  • User owning the Gocipher
  • User being shared access to this Gocipher with config, share or rights modification rights
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Gocipher

header Parameters
If-Match
required
string

eTag of previous state of the Gocipher

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Delete a Gocipher

Description

Delete a Gocipher.

  • Put it to technicalStatus DELETED.

Access

  • User owning the Gocipher
  • User being shared access to this Gocipher and with delete right
  • Customer application
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Gocipher

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create/Replace the Gocipher API key

Description

Create/Replace the Gocipher API key used for Gocipher connection to Spider.

The API key is a public/private key pair.

  • The public key is stored in Gocipher's settings.
  • The private key is sent in response to this call.

Any call to this API (if authorized) will overwrite the previous API key, and the Gocipher will not be able to use the previous one. A connected Gocipher will be disconnected when its current JWT token will expire. The API key is taken as a configuration AT START of Gociphers, and need a restart to be changed.

Output

The API can supports two outputs:

  • application/json:
    • Provides a file with the private key, Spider's URL, and the Gocipher's id This file is the only expected configuration file at Gocipher start.
  • application/x-pem-file
    • Provides only the private key as a PEM file

Access

  • User owning the Gocipher
  • User of the team owning the Gocipher with Gociphers right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Gocipher

Responses

Response samples

Content type
{}

Generate an API key signature with the private key of the Gocipher

Description

This endpoint is for testing purposes only:

  • It allows testing the API key or the Gociphers API without a Gocipher
  • It generates a valid signature for the Gocipher to call the configuration endpoint
  • However, IT REQUIRES YOUR PRIVATE KEY IN INPUT
  • After testing, please, reset your API key

Output

  • The signature for this Gocipher, timestamp and private API key
  • A validation of this signature with the Gocipher registered public API key

Access

  • User owning the Gocipher
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Gocipher

query Parameters
timeStamp
required
string <date-time>

Timestamp to use in signature

instanceId
required
string

InstanceId to use in signature

Request Body schema: application/x-pem-file
string

The Gocipher RSA private key, as a PEM

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get Gocipher configuration

Description

Get the configuration of this Gocipher Usages:

  • Called by Gocipher at start (or before token expiration) with their API key
  • Called by ephemeral Gocipher at start with the token given by the Controller
  • Called by Gociphers regularly to check for configuration change
  • Called by UI to get a Gocipher token to perform upload
  • Called by UI to get configuration
  • Called by services to get Gocipher configuration when parsing

API key

The first call from Gociphers is made by:

  • building a json payload with current time and Gocipher id,
  • then signing it with SHA256 algorithm using Gocipher's private key
const timeStamp = moment().toISOString();
const info = {
    timeStamp,
    GocipherId,
    instanceId
};
const privKey = new NodeRSA(privatePem);
const signature = privKey.sign(Buffer.from(JSON.stringify(info)), 'base64');
  • and then calling this API with specific headers:
  Spider-TimeStamp: timeStamp
  Spider-InstanceId: instanceId,
  Spider-Signature: signature //base 64 encoded

Parameters

The view parameter allows to modulate the output:

server

  • Only server part (parsing) is sent
  • Current configuration is merged with default configuration

client

  • Only client part (parsing) is sent
  • Current configuration is merged with default configuration

full

  • Both client & server are sent
  • Current configuration is merged with default configuration

null

  • The raw configuration is sent (without merging with defaults)
  • This view is used by UI to know what settings are specific, and what settings are defaults

Output

  • The configuration
  • A JWT token to use on further calls in Spider-Token header
    • If no token provided

Access

  • The own Gocipher
  • User owning the Gocipher
  • User being shared access to this Gocipher
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Gocipher

query Parameters
view
string
Enum: "server" "client" "full"

Type of configuration to get

header Parameters
Spider-Signature
string <base64>

Signature of the call by the Gocipher, with its API key

Spider-Timestamp
string <date-time>

Provided with API key in first Gocipher call to get its JWT token with the config

Spider-InstanceId
string

Provided with API key in first Gocipher call to get its JWT token with the config

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Change Gocipher's config

Description

Updates a Gocipher configuration.

Rules

  • Can do any change:
    • Admin
    • User owning the Gocipher
  • User being shared access to this Gocipher with config rights can:
    • Change configuration settings
  • User being shared access to this Gocipher with share rights can:
    • Change sharing settings

Access

  • User owning the Gocipher
  • User being shared access to this Gocipher with config or share rights
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Gocipher

header Parameters
If-Match
required
string

eTag of previous state of the Gocipher's config

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Hosts

The list of Hosts detected by Whisperers

Send an hosts update for a whisperer

Description

An hosts list contains all hosts the whisperer has seen during its capture.

When processing the list, Spider can create a new entry of hosts for this Whisperer or update the nearest one.

Hosts lists can be sent in any order (to support upload).

Access

  • Own whisperer
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

Request Body schema: application/json
Array (non-empty)
ip
required
string <ipv4>

IP address of host

name
required
string

FQDN of host as given by DNS

type
required
string
Enum: "SERVER" "CLIENT" null

Type of host

firstSeen
required
string <date-time>

Date of first packet seen for this host

lastSeen
required
string <date-time>

Date of last packet seen for this host

lastUpdate
required
string <date-time>

Last time the DNS was queried to update the host name

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Change the custom name of a Host.

Description

Update an Host:

  • identified by an IP
  • between two dates
  • on one to many whisperers

The update:

  • May change the customName of the host
  • May update several hosts lists of the same whisperer
  • May update several hosts lists of many whisperers

Access

  • Admin
  • User asking to update only on its whisperers (owned or shared)
Authorizations:
Bearer
Request Body schema: application/json
required
whisperers
required
Array of strings non-empty

List of whisperers on which to do the update

ip
required
string <ipv4>

IP of the host for which to set custom name

startTime
required
number <double>

Timestamp of period start for which to change

stopTime
required
number <double>

Timestamp of period end for which to change

customName
string

The new name / optional - can be null

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "ip": "10.0.0.236",
  • "startTime": 1548595820.545,
  • "stopTime": 1548595882.677,
  • "customName": "test-service"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Hosts detected by a whisperer

Description

Search for hosts Returns a collection of HostsList

Rules

May ask searching without specifying a whisperer:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Whisp status

Runtime metrics from Whisperers (status, speed, cpu...)

Send status information from a whisperer

Description

Whisperers are sending status every x seconds when started

Access

  • Own whisperer
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
time
required
string <date-time>

Time of status

whisperer
required
string

System Id of whisperer

instanceId
required
string

Instance id of the whisperer

whispererName
string

Name of whisperer - enriched by service

hostname
required
string

FQDN of the host

startTime
required
string <date-time>

Start time of the whisperer

sessionStartTime
string

Recording session start time

upTime
required
number

Uptime of whisperer

state
required
string
Enum: "STARTING" "RECORDING" "STOPPED" "SERVER_DOWN" "BREAK" "INTERNAL_ERROR" "INVALID_CONFIG"

State of the whisperer

object

Statistics of REST API calls to the server, by API

required
object

Total of metrics values since start of session

object

Metrics values since last send of status

Array of objects

List of network interfaces of the host

Responses

Request samples

Content type
application/json
{
  • "@id": "Y2GaZbDXRLq3cj-m8Zjviw.1548595914874",
  • "time": "2019-01-27T13:31:54.874Z",
  • "whisperer": "Y2GaZbDXRLq3cj-m8Zjviw",
  • "hostname": "node-3.streetsmart.sit3",
  • "instanceId": "1a2aebcf734f",
  • "startTime": "2019-01-16T14:40:19.843Z",
  • "sessionStartTime": "2019-01-25T11:17:40.234Z",
  • "upTime": 946295.88,
  • "state": "RECORDING",
  • "circuitBreakers": {
    },
  • "total": {
    },
  • "new": {
    },
  • "interfaces": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a collection of current whisps status

Description

Get the statuses listed in input (by id).

Also available by GET method

Access

  • Admin
  • Customer
    • Only the statuses of whisperers it has access to are returned; other identifiers are silently dropped
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

List of whisperers identifiers (or whisperer instance identifiers)

Array
string

System id of whisperer

Responses

Request samples

Content type
application/json
[
  • "string"
]

Response samples

Content type
application/json
[
  • { }
]

Search for raw status sent by a whisperer

Description

Search for raw status

Rules

May ask searching without specifying a whisperer:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer, limited on search on its whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Preaggregated search for parsing status histogram

Description

Histogram aggregation analysis on Whisperers status

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Search for current status of whisperers

Description

Search for current status of whisperers

Rules

May ask searching without specifying a whisperer:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer, limited on search on its whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resources fetched to get next ones.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

Responses

Request samples

Content type
application/json
{
  • "size": 20,
  • "whisperers": [
    ],
  • "query": "",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Ciphers status

Runtime metrics from Gociphers (status, speed, cpu...)

Send status information from a Gocipher

Description

Gociphers are sending status every x seconds when started

Access

  • Own whisperer
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
time
required
string <date-time>

Time of status

whisperer
required
string

System Id of whisperer

instanceId
required
string

Instance id of the whisperer

whispererName
string

Name of whisperer - enriched by service

hostname
required
string

FQDN of the host

startTime
required
string <date-time>

Start time of the whisperer

sessionStartTime
string

Recording session start time

upTime
required
number

Uptime of whisperer

state
required
string
Enum: "STARTING" "RECORDING" "STOPPED" "SERVER_DOWN" "BREAK" "INTERNAL_ERROR" "INVALID_CONFIG"

State of the whisperer

object

Statistics of REST API calls to the server, by API

required
object

Total of metrics values since start of session

object

Metrics values since last send of status

Array of objects

List of network interfaces of the host

Responses

Request samples

Content type
application/json
{
  • "@id": "Y2GaZbDXRLq3cj-m8Zjviw.1548595914874",
  • "time": "2019-01-27T13:31:54.874Z",
  • "whisperer": "Y2GaZbDXRLq3cj-m8Zjviw",
  • "hostname": "node-3.streetsmart.sit3",
  • "instanceId": "1a2aebcf734f",
  • "startTime": "2019-01-16T14:40:19.843Z",
  • "sessionStartTime": "2019-01-25T11:17:40.234Z",
  • "upTime": 946295.88,
  • "state": "RECORDING",
  • "circuitBreakers": {
    },
  • "total": {
    },
  • "new": {
    },
  • "interfaces": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a collection of current Gociphers status

Description

Get the statuses listed in input (by id).

Also available by GET method

Access

  • Admin
  • Customer
    • Only the statuses of Gociphers it has access to are returned; other identifiers are silently dropped
    • At most 199 distinct Gociphers per request
Authorizations:
Bearer
Request Body schema: application/json
required

List of Gociphers identifiers (or Gocipher instance identifiers)

Array
string

System id of Gocipher

Responses

Request samples

Content type
application/json
[
  • "string"
]

Response samples

Content type
application/json
[
  • { }
]

Search for raw status sent by a Gocipher

Description

Search for raw status

Rules

May ask searching without specifying a Gocipher:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer, limited on search on its Gociphers
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Search for current status of Gociphers

Description

Search for current status of Gociphers

Rules

May ask searching without specifying a Gocipher:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer, limited on search on its Gociphers
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resources fetched to get next ones.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

Responses

Request samples

Content type
application/json
{
  • "size": 20,
  • "whisperers": [
    ],
  • "query": "",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get current targets for this Whisperer across all Gociphers

Description

Get the targets being watched for this Whisperer.

Also available by GET method

Access

  • Admin
  • Customer
    • Limited access to the Whisperers it has access to
Authorizations:
Bearer
path Parameters
whisperer
required
string

Internal id of Whisperer

Responses

Response samples

Content type
application/json
[
  • { }
]

Create a new Link

Description

Stores a UI state and gives a link in returns.

Access

  • Customer
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

The state to link

required
object
object
main
required
object
required
object
search
required
object

Responses

Request samples

Content type
application/json
{
  • "userInfo": {
    },
  • "time": {
    },
  • "main": { },
  • "networkMap": {
    },
  • "search": { }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Links

Description

Search for links.

Also available by GET method on the collection

Rules

May ask searching without specifying a whisperer:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a link

Description

Get a link
The link may be public or not

  • Customer
  • Admin

Rules

  • If Public link, it is not DELETED or deprecated
  • User who created it
  • Admin
  • When using public link token
    • FreeAccess is true
    • User email is in recipients list
    • User email domain is in domains list
Authorizations:
Bearer
path Parameters
id
required
string

Links system Id

Responses

Response samples

Content type
application/json
{
  • "@id": "fH5R3ZjhR7WPHBn2GDg1cA",
  • "@type": "sp:link",
  • "dateCreated": "2019-01-27T20:58:30.421Z",
  • "creator": "VBqPbgjYRsK00O76DVeroQ==",
  • "version": "0.1",
  • "content": { }
}

Create a new Public Link

Description

Stores a UI state and gives a public link in return.

Access

  • User with publishing permission on the whisperers present in the state, or on the team
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required
required
Array of objects

A collection of filters defining access control.

freeAccess
boolean

Flag indicating if the link provides free access without restrictions.

sendEmail
boolean

Indicates whether an email notification should be sent.

dateDeprecated
string <date>

The ISO date when the public link becomes deprecated.

domains
required
Array of strings

List of domains whose emails have access to the public link. Starting with '@'.

recipients
required
Array of strings

Email addresses of the recipients with access.

stateToShare
required
object

Object representing the UI state to be shared.

Responses

Request samples

Content type
application/json
{
  • "accessFilters": [
    ],
  • "freeAccess": true,
  • "sendEmail": true,
  • "dateDeprecated": "2019-08-24",
  • "domains": [
    ],
  • "recipients": [
    ],
  • "stateToShare": { }
}

Response samples

Content type
application/json
{
  • "urlToShare": "string"
}

Search for Public Links

Description

Search for public links.

Also available by GET method on the collection

Rules

  • content is removed
  • urlToShare field is added with the Url of the link

Access

  • Admin
  • User to the public links he created
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Delete a public link

Description

Set a public link to DELETED status, making it invisible to searches and get.

Access

  • User that has created the public link
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Public link system Id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create and send a One Time Password for a user trying to connect to a Public link

Description

Create a One Time Password for a user trying to connect to a Public link. Send this OTP to the email of the user.

Rules

  • Link is not DELETED or deprecated

Access

  • FreeAccess is true
  • User email is in recipients list
  • User email domain is in domains list
path Parameters
id
required
string

Public link system Id

Request Body schema: application/json
required
email
Array of strings

Email address of the user trying to access.

Responses

Request samples

Content type
application/json
{
  • "email": [
    ]
}

Response samples

Content type
application/json
{
  • "status": "string"
}

Connects a public user and generates a public link JWT

Description

Connects a public user by generating a JWT specific to this public link, with dedicated access filters. Issues also a cookie with refresh token scoped to the token renewal api.

Rules

  • Link is not DELETED or deprecated
  • OTP is not too old
  • OTP is associated to this email and link

Processing

  • Email and connection date are added to public link tracked sessions

Access

  • FreeAccess is true
  • User email is in recipients list
  • User email domain is in domains list
path Parameters
id
required
string

Public link system Id

Request Body schema: application/json
required
email
Array of strings

Email address of the user trying to access.

otp
Array of strings

One Time Password associated to this email.

Responses

Request samples

Content type
application/json
{
  • "email": [
    ],
  • "otp": [
    ]
}

Response samples

Content type
application/json
{
  • "token": "string"
}

Logs out a user from its public link sessions

Description

Logs out a user and clears the refresh token cookie

Access

  • The cookie with the refresh token is expected in input
path Parameters
id
required
string

Public link system Id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Generate a new access token and refreshes the refresh token

Description

Rotate the access token and issue a new refresh token for a user

Responses

Response samples

Content type
application/json
{
  • "token": "string"
}

Sessions

Statistics from UI sessions

Search for Sessions

Description

Search for sessions. Sessions are confidential, searching them is much restricted.

Also available by GET method on the collection

Access

  • Admin
  • User with admin monitoring right asking for an aggregation with size:0
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a session

Description

Get a session

Access

  • User whose session it is
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Sessions system Id

Responses

Response samples

Content type
application/json
{
  • "@id": "06cb72b6-5a87-4baa-a3ca-620806a6aa7b",
  • "dateCreated": "2019-01-27T21:03:38.750Z",
  • "creator": "VBqPbgjYRsK00dhzdVeroQ",
  • "version": "0.1",
  • "dateModified": "2019-01-27T21:18:41.746Z",
  • "updater": "VBqPbgjYRsK00dhzdVeroQ",
  • "user": {
    },
  • "main": {
    },
  • "views": {
    },
  • "subViews": {
    },
  • "options": {
    },
  • "whisperers": {
    },
  • "actions": [
    ],
  • "actionsTotalCount": 11
}

Save a session

Description

Save a session

Access

  • User whose session it is
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Sessions system Id

Request Body schema: application/json
required

The session

@id
required
string

System id

required
object

Connected user

required
object
views
required
object

Statistics on the view used

subViews
object

Statistics on the subview used

options
object

Options selected

whisperers
object

List of selected whisperers

Array of objects

Statistics on the actions triggered by the user

Responses

Request samples

Content type
application/json
{
  • "@id": "06cb72b6-5a87-4baa-a3ca-620806a6aa7b",
  • "dateCreated": "2019-01-27T21:03:38.750Z",
  • "creator": "VBqPbgjYRsK00dhzdVeroQ",
  • "version": "0.1",
  • "dateModified": "2019-01-27T21:18:41.746Z",
  • "updater": "VBqPbgjYRsK00dhzdVeroQ",
  • "user": {
    },
  • "main": {
    },
  • "views": {
    },
  • "subViews": {
    },
  • "options": {
    },
  • "whisperers": {
    },
  • "actions": [
    ],
  • "actionsTotalCount": 11
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Jobs

Traces of upload, download and purge Jobs

Create a new Job

Description

Create a job

Process

  • If type is PurgeJob
    • Launches the purge on all selected resources
    • Check progress and update the job regularly until success or failure

Access

  • Admin
  • User creating a Job on its whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

The Job details

jobType
required
string
Enum: "PurgeJob" "DownloadJob" "UploadJob" "..."

Type of job

jobParameters
required
object

Input of the job

progress
required
integer

Progress in %

whisperer
required
Array of strings

List of whisperers

startTime
string <date-time>

Start of the job

endTime
string <date-time>

End of the job

updateTime
string <date-time>

Last update of the job

actionStatus
required
string
Enum: "ActiveActionStatus" "CompletedActionStatus" "FailedActionStatus" "PotentialActionStatus"

Status of the job

result
object

Result of the job

error
object

Error of the job, if any

Responses

Request samples

Content type
application/json
{
  • "jobType": "PurgeJob",
  • "jobParameters": { },
  • "progress": 0,
  • "whisperer": [
    ],
  • "startTime": "2019-08-24T14:15:22Z",
  • "endTime": "2019-08-24T14:15:22Z",
  • "updateTime": "2019-08-24T14:15:22Z",
  • "actionStatus": "ActiveActionStatus",
  • "result": { },
  • "error": { }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Jobs

Description

Search for jobs.

Also available by GET method on the collection

Rules

Admin and monitoring admin may search without specifying a whisperer

Access

  • Admin
  • Customer, limited on its whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a jobs details

Description

Get a jobs details.

Access

  • User linked to the whisperers in this job
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

Responses

Response samples

Content type
application/json
{
  • "@id": "Ju_4O7N9QvWLC8x7PswfnQ",
  • "@type": "Job",
  • "jobType": "DownloadJsonJob",
  • "creator": "wB0wdZgkTJqxcSZYv8yZ8g",
  • "creationTime": "2019-01-18T15:20:09.158Z",
  • "updateTime": "2019-01-18T15:20:09.159Z",
  • "endTime": "2019-01-18T15:20:09.106Z",
  • "actionStatus": "CompletedActionStatus",
  • "progress": 100,
  • "jobParameters": {
    },
  • "result": {
    },
  • "whisperer": [
    ]
}

Update job details

Description

Update a Job

Rules

Impossible to update the following fields:

  • jobType
  • creationTime
  • creator

Access

  • Admin
  • User that created the job and linked to all whisperers of the job
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

header Parameters
If-Match
required
string

eTag of previous state of the job

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

MailSender

Service to send emails

Send an email

Description

Sends an email from the configured account. Can be configured to be connected by OAuth (gmail for instance) or login/password.

Can send to one or several recipients, a text message or html. Txt message being mandatory.

As it is not intended for mass mailing, it makes a connection to the SMTP server for each email.

Access

  • Can only be call by applications or admin
Request Body schema: application/json
required

The email to send

to
required
Array of strings

List of recipients

subject
required
string

Subject of the mail. Accepts unicodes characters (for icons)

text
required
string

The plain message

html
string

The message in html

Responses

Request samples

Content type
application/json
{
  • "to": [
    ],
  • "subject": "string",
  • "text": "string",
  • "html": "string"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

GuiLogs

Logs of errors generated on GUI

Create a new Gui Log

Description

Create a Gui Log to trace an error happened on GUI

Access

  • No identification required, but if identified, it will be saved
Authorizations:
Bearer
Request Body schema: application/json
required

The Gui Log

app
required
string
Enum: "Login" "NetworkView" "SelfMonitoring"

Name of application

time
required
string <date-time>

Time of log

type
required
string
Enum: "BAD REQUEST" "SERVER ERROR" "UNEXPECTED" "TIMEOUT" "UNKNOWN" "XXX RENDER"

Type of error

name
required
string
Enum: "Saga Error" "React Error"

Name of the error

level
required
string
Enum: "ERROR" "WARNING"

Level of Log

message
required
string

Message of the error (from JS error.message)

stack
required
string

Stack of the error

details
string

Details of the error. Component stack in React.

whisperer
Array of strings

List of whisperers

customer
string

System id of the customer

timeout
integer

Timeout value in case of timeouts

object

Request that failed

object

Response that failed

Responses

Request samples

Content type
application/json
{
  • "app": "Login",
  • "time": "2019-08-24T14:15:22Z",
  • "type": "BAD REQUEST",
  • "name": "Saga Error",
  • "level": "ERROR",
  • "message": "string",
  • "stack": "string",
  • "details": "string",
  • "whisperer": [
    ],
  • "customer": "string",
  • "timeout": 0,
  • "req": {
    },
  • "res": {
    }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Gui Logs

Description

Search for Gui Logs.

Also available by GET method on the collection

Rules

Admin and monitoring admin may search without specifying a whisperer

Access

  • Admin
  • User with admin monitoring right asking for an aggregation with size:0
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

GuiSettings

User settings for GUI

Create / update GUI settings

Description

Stores the user settings. Mostly used by the UI itself

Access

  • Customer
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

The settings to save

@id
string

Id of the settings, and of the customer

required
object

GUI settings

Responses

Request samples

Content type
application/json
{
  • "@id": "string",
  • "settings": {
    }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Settings

Description

Search for settings.

Also available by GET method on the collection

Access

  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get GUI settings for a user

Description

Get settings

Access

  • Customer
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Settings Id == Id of user

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "GuiSettings",
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "version": "string",
  • "settings": {
    }
}

Plugins

Plugins store backend or proxy

Create / update a plugin

Description

Uploads a plugin.
Depending on global setting, the plugin is stored locally in ES or remotely in Floocus managed S3 service.

Access

  • User with plugins.upload permission
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

The plugin to store

@id
required
string

Id of the plugin, unique for the organisation

@type
required
string

Type of the plugin

@version
required
string

Version of the plugin

name
required
string

Human name of the plugin

description
string

Explains what the plugins does

source
required
string

Base64 encoded javascript source code of the plugin

Responses

Request samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "@version": "string",
  • "name": "string",
  • "description": "string",
  • "source": "string"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Plugins

Description

When global, searches for both own organisation plugins and 'Floocus' ones.
When local, searches for any plugin.

Also available by GET method on the collection

Access

  • Customer
Authorizations:
Bearer
Request Body schema: application/json
required

The plugin to store

type
string

Type of the plugin

Array of objects

Sorting option.

next
Array of strings

Ids of the last plugin fetched.

size
required
integer

Page size.

Responses

Request samples

Content type
application/json
{
  • "type": "string",
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get the manifest of a specific plugin

Description

Get plugin manifest

Access

  • Customer
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Plugin @id

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "GuiSettings",
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "version": "string",
  • "settings": {
    }
}

Get the code of a specific plugin

Description

Get the code

Access

  • Open to everybody
path Parameters
id
required
string

Plugin @id

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "GuiSettings",
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "version": "string",
  • "settings": {
    }
}

Monitor

Access monitoring metrics and logs from Spider

Search for monitoring information

Description

Searches or aggregation analysis on monitored metrics:

  • Processes stats
  • Circuit breakers stats
  • Pollers stats
  • Parsing queues stats
  • Elasticsearch indices stats
  • Elasticsearch nodes stats
  • Redis stats
  • Applicative logs stats

Also available by GET method on the collections

Access

  • Admin
  • User with admin monitoring right
Authorizations:
Bearer
path Parameters
collection
required
string
Enum: "pollers" "parsers" "parsingqueues" "circuitbreakers" "redis" "elasticsearch" "elasticsearchNodes" "processes" "api" "logs"

Collection of metrics

Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Pollers

Pollers statistics

Get poller queue status

Description

Give the queue status of this poller

Access

  • Admin
  • Application
  • Customer
Authorizations:
Bearer
path Parameters
service
required
string (PollerNames)
Enum: "pack-poller" "tcp-poller" "pg-com-poller" "pg-com-content-poller" "pg-parsing-poller" "web-httpcom-poller" "web-httpcom-content-poller" "web-httppers-poller" "hosts-poller" "hosts-agg" "whisp-status-poller" "whisps-status-agg" "status-poller" "capture-status-poller" "ciphers-status-poller" "ciphers-raw-status-poller" "ciphers-status-agg" "parsing-status-tcpsession-poller" "parsing-status-httppers-poller" "parsing-status-pgparsing-poller"

Pollers service name

Responses

Response samples

Content type
application/json
{
  • "count": 182,
  • "first": "2019-01-30T22:35:51.254Z",
  • "last": "2019-01-30T22:36:01.641Z"
}

Schemas

Content-addressed schema store used for binary payload transcoding (protobuf, MessagePack). Schemas are immutable: the @id is the SHA-256 hash of the file bundle.

Upload a schema bundle

Description

Upload a content-addressed schema bundle used for binary payload transcoding (protobuf, MessagePack, JSON). The bundle is validated on upload (protobuf is compiled, JSON is parsed). If the same file content has already been stored, the existing document is returned with HTTP 200 instead of 201.

For protobuf bundles (contentType: application/x-protobuf or application/protobuf), the list of fully-qualified message type names is extracted at upload time and stored alongside the files.

Maximum total upload size: 16 MB.

Access

  • Admin
  • User with whisperer-config rights (any)
Authorizations:
Bearer
Request Body schema: multipart/form-data
required

Multipart form data: one contentType field and one or more files file parts.

contentType
required
string

MIME type of the schema files. Supported values: "application/x-protobuf", "application/protobuf", "application/json", "application/x-msgpack".

files
required
Array of strings <binary> [ items <binary > ]

One or more schema files (e.g. .proto files for protobuf bundles).

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "Schema",
  • "creator": "string",
  • "dateCreated": "2019-08-24T14:15:22Z",
  • "contentType": "string",
  • "files": [
    ],
  • "size": 0,
  • "messageTypes": [
    ]
}

List schemas by content type

Description

Returns a paginated list of schema summaries filtered by content type. File contents are not included - use GET /schemas/v1/schemas/{id} to retrieve a full schema with file bytes.

Access

  • Admin
  • User with whisperer-config rights (any)
Authorizations:
Bearer
query Parameters
contentType
required
string

Filter by MIME type (e.g. "application/x-protobuf")

from
integer
Default: 0

Pagination offset (number of items to skip)

size
integer <= 200
Default: 50

Number of items to return (max 200)

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get a schema by id

Description

Retrieves the full schema document including all file contents (base64-encoded bytes). Used by parsers and the Web-Read-Go transcoder at runtime to fetch schema bundles by their content-addressed id.

Access

  • Admin
  • User with whisperer-config rights (any)
  • Internal services (parsers, web-read-go) via their service JWT
Authorizations:
Bearer
path Parameters
id
required
string

Content-addressed @id of the schema (SHA-256 hex hash of the file bundle)

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "Schema",
  • "creator": "string",
  • "dateCreated": "2019-08-24T14:15:22Z",
  • "contentType": "string",
  • "files": [
    ],
  • "size": 0,
  • "messageTypes": [
    ]
}

Whisperers

Create a new Whisperer

Description

Create a new whisperer, and associate it to the owner customer.

Access

  • Customer, with whisperers creation rights
  • User of the team owning the whisperer with whisperers right
  • User with userTrainer rights impersonating a trainee creating a whisperer for the trainee
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

Whisperer creation request

customer
required
string

System Id of the customer.

name
required
string

Name of the whisperer to create.

Responses

Request samples

Content type
application/json
{
  • "customer": "YOD66VZ54Jih",
  • "name": "Upload"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get default Configuration for Whisperers

Description

Get default configuration.

Access

  • User
  • Admin
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Whisperers

Description

Search for whisperers

Also available by GET method on the collection

Rules

If client is not admin, the search will be limited to the Whisperers owned by this customer or shared with him.

If client is using public link,

  • Search is done on the whisperers from its token, not based on whisperers date.
  • users and teams are omitted in response

Access

  • Admin
  • Customer
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a Whisperer

Description

Get a whisperer's details.

Rules

Only admins can see DELETED whisperers.

Access

  • User owning the whisperer
  • The own whisperer
  • User being shared access to this whisperer
  • User of the team owning the whisperer with whisperers right
  • customers or links applications
  • A public link user having this Whisperer in the token
  • Admin

Response

Response varies depending on client:

  • Full resource for
    • User owning the whisperer
    • The own whisperer
    • User being shared access to this whisperer
    • User of the team owning the whisperer with whisperers right
    • User or Links applications
  • Full minus users and teams fields for public links users
  • Only id and name for all Customers
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Whisperer

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "Whisperer",
  • "version": "string",
  • "name": "string",
  • "customer": "string",
  • "team": "string",
  • "dns": "string",
  • "apikey": "string",
  • "config": {
    },
  • "users": [
    ],
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "editor": "string",
  • "dateModified": "2019-08-24"
}

Change whisperer's name or shared users

Description

Updates a customer. You can:

  • Update whisperer's name
  • Change sharing settings: users and users rights on this whisperer

Rules

  • Can do any change:
    • Admin
    • User owning the whisperer
    • User of the team owning the whisperer with whisperers right
  • User being shared access to this whisperer with config rights can:
    • Change the name
  • User being shared access to this whisperer with share rights can:
    • Add or remove users from the sharing settings
  • User being shared access to this whisperer with rights change rights can:
    • Change rights of users in the sharing settings

Side effects

  • When adding or removing customers of shared list, the service updates the whisperers list in the customer resource (by calling Customer service).
  • When adding or removing teams of shared list, the service updates the whisperers list in the team resource (by calling Customer service).

Access

  • User owning the whisperer
  • User of the team owning the whisperer with whisperers right
  • User being shared access to this whisperer with config, share or rights modification rights
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Whisperer

header Parameters
If-Match
required
string

eTag of previous state of the whisperer

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Delete a Whisperer

Description

Delete a whisperer.

  • Put it to technicalStatus DELETED.

Access

  • User owning the whisperer
  • User being shared access to this whisperer and with delete right
  • User of the team owning the whisperer with whisperers right
  • Customer application
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Whisperer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create/Replace the whisperer API key

Description

Create/Replace the whisperer API key used for whisperer connection to Spider.

The API key is a public/private key pair.

  • The public key is stored in whisperer's settings.
  • The private key is sent in response to this call.

Any call to this API (if authorized) will overwrite the previous API key, and the whisperer will not be able to use the previous one. A connected whisperer will be disconnected when its current JWT token will expire. The API key is taken as a configuration AT START of whisperers, and need a restart to be changed.

Output

The API can supports two outputs:

  • application/json:
    • Provides a file with the private key, Spider's URL, and the whisperer's id This file is the only expected configuration file at whisperer start.
  • application/x-pem-file
    • Provides only the private key as a PEM file

Access

  • User owning the whisperer
  • User of the team owning the whisperer with whisperers right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

Responses

Response samples

Content type
{}

Generate an API key signature with the private key of the whisperer

Description

This endpoint is for testing purposes only:

  • It allows testing the API key or the whisperers API without a whisperer
  • It generates a valid signature for the whisperer to call the configuration endpoint
  • However, IT REQUIRES YOUR PRIVATE KEY IN INPUT
  • After testing, please, reset your API key

Output

  • The signature for this whisperer, timestamp and private API key
  • A validation of this signature with the whisperer registered public API key

Access

  • User owning the whisperer
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

query Parameters
timeStamp
required
string <date-time>

Timestamp to use in signature

instanceId
required
string

InstanceId to use in signature

Request Body schema: application/x-pem-file
string

The whisperer RSA private key, as a PEM

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create the whisperer configuration

Description

Initialize the configuration of the whisperer

Access

  • User owning the whisperer and whisperer creation rights
  • User of the team owning the whisperer with whisperers right
  • User with userTrainer rights impersonating a trainee creating a whisperer for the trainee
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

Request Body schema: application/json
required
version
required
string
required
object

Settings used client side - the Sniffer

required
object

Settings used server side, for parsing

Responses

Request samples

Content type
application/json
{
  • "version": "0.1",
  • "client": {
    },
  • "server": {
    }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get Whisperer configuration

Description

Get the configuration of this Whisperer Usages:

  • Called by whisperer at start (or before token expiration) with their API key
  • Called by ephemeral whisperer at start with the token given by the Controller
  • Called by whisperers regularly to check for configuration change
  • Called by UI to get a whisperer token to perform upload
  • Called by UI to get configuration
  • Called by services to get whisperer configuration when parsing

API key

The first call from whisperers is made by:

  • building a json payload with current time and whisperer id,
  • then signing it with SHA256 algorithm using whisperer's private key
const timeStamp = moment().toISOString();
const info = {
    timeStamp,
    whispererId,
    instanceId
};
const privKey = new NodeRSA(privatePem);
const signature = privKey.sign(Buffer.from(JSON.stringify(info)), 'base64');
  • and then calling this API with specific headers:
  Spider-TimeStamp: timeStamp
  Spider-InstanceId: instanceId,
  Spider-Signature: signature //base 64 encoded

Parameters

The view parameter allows to modulate the output:

server

  • Only server part (parsing) is sent
  • Current configuration is merged with default configuration

client

  • Only client part (parsing) is sent
  • Current configuration is merged with default configuration

full

  • Both client & server are sent
  • Current configuration is merged with default configuration

null

  • The raw configuration is sent (without merging with defaults)
  • This view is used by UI to know what settings are specific, and what settings are defaults

Output

  • The configuration
  • A JWT token to use on further calls in Spider-Token header
    • If no token provided, or if called from a Customer
    • A Customer may call to get the configuration of one of its whisperers and use the generated token to upload data (as on the UI)
    • If the Whisperer has a TTL associated to attachments, the token will have this as expiration instead of the default duration

Access

  • The own whisperer
  • User owning the whisperer
  • User being shared access to this whisperer
  • User of the team owning the whisperer with whisperers right
  • Controller asking for a token for a Whisperer to spawn on an Pod (InstanceId)
  • Admin
  • Application
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

query Parameters
view
string
Enum: "server" "client" "full"

Type of configuration to get

header Parameters
Spider-Signature
string <base64>

Signature of the call by the Whisperer, with its API key

Spider-Timestamp
string <date-time>

Provided with API key in first Whisperer call to get its JWT token with the config

Spider-InstanceId
string

Provided with API key in first Whisperer call to get its JWT token with the config

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "dateCreated": "2019-08-24",
  • "dateModified": "2019-08-24",
  • "editor": "string",
  • "creator": "string",
  • "client": {
    },
  • "server": {
    }
}

Change whisperer's config

Description

Updates a Whisperer configuration.

Rules

  • Can do any change:
    • Admin
    • User owning the whisperer
  • User being shared access to this whisperer with config rights can:
    • Change configuration settings
  • User being shared access to this whisperer with share rights can:
    • Change sharing settings
  • User being shared access to this whisperer with record rights can:
    • Start or stop capture

Access

  • User owning the whisperer
  • User being shared access to this whisperer with config, share, or record rights
  • User of the team owning the whisperer with whisperers right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

header Parameters
If-Match
required
string

eTag of previous state of the whisperer's config

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get TLS configurations from a list of Whisperers

Description

From a list of Whisperers, provide their TLS configuration

Access

  • Gocipher
  • Admin requestBody: content: 'application/json': schema: description: 'The whisperer RSA private key, as a PEM' type: array items: type: string description: Internal Id of Whisperers minItems: 1
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "whisperer": "string",
  • "requiredState": "string",
  • "tlsKeys": { }
}

Renew an ephemeral Whisperer token

Description

Called by ephemeral Whisperers (only) to renew their token (they do not have an API key).

If the Whisperer has a Time to Live associated to attachment, it won't renew.

Output

  • A JWT token to use on further calls in Spider-Token header
    • If no token provided, or if called from a Customer
    • A Customer may call to get the configuration of one of its whisperers and use the generated token to upload data (as on the UI)

Access

  • The own whisperer (only ephemerals can)
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}