Skip to main content
API docs by Redocly

Spider API (2.13.0)

Download OpenAPI specification:Download

E-mail: contact@floocus.com
Spider Analyzer Website

Description

These are the services behind the GUI. In fact, all Spider processing is done using these APIs.

Description

Spider API allows you to:

  • configure your system (teams, users, agents)
  • search, browse and compute statistics on the resources created while parsing the network communications
  • upload new communications
  • and even write a new agent!

You may search through Packets, Tcp Sessions, Http Communications, download them, analyse them with ElasticSearch power, rebuild the content...

These are the services behind the GUI. In fact, all Spider processing is done using these APIs.

How to start?

  1. Create a Service Account on the UI
  2. Start by authenticating with POST /customer/v1/service-accounts/sessions to get your JWT token to use in all further calls.
  3. Click 'Authorize' button to register your token.
  4. Then use any customers authorized API. Ex:
    • POST /web-read/v1/http-com/_search to search for HTTP communications
    • GET /web-read/v1/http-com/{id}/res/body/ to get the response body of a communication
  5. When using a team, you may have to get the team token to access its whisperers
    • GET /teams/v1/teams/{id}/user-token/

Common

Common API for all services.

Get micro service info

Description

Give basic info, can be used for healthcheck of service

Access

  • No identification
path Parameters
service
required
string (ServiceNames)
Enum: "alert" "capture-status-poller" "ciphers" "ciphers-status" "ciphers-status-poller" "ciphers-raw-status-poller" "ciphers-status-agg" "controls" "customer" "gui-logs" "gui-settings" "hosts" "hosts-agg" "hosts-poller" "job" "link" "mail-sender" "maintenance" "pack-poller" "pack-read" "pack-write" "pack-update" "parsing-status-tcpsession-poller" "parsing-status-httppers-poller" "plugins" "session" "stats-collector" "status-poller" "tcp-poller" "tcp-read" "tcp-write" "tcp-update" "teams" "tls-keys" "tls-keys-linker" "web-httpcom-poller" "web-httpcom-content-poller" "web-httppers-poller" "web-read" "web-upload" "web-write" "whisp" "whisp-status-poller" "whisps-status" "whisps-status-agg"

Service name

Responses

Response samples

Content type
text/plain
Spider Customer

Get Api stats

Description

Give metrics informations for HTTP Api of this service

Access

  • Admin
  • Application
  • User
Authorizations:
Bearer
path Parameters
service
required
string (ServiceNames)
Enum: "alert" "capture-status-poller" "ciphers" "ciphers-status" "ciphers-status-poller" "ciphers-raw-status-poller" "ciphers-status-agg" "controls" "customer" "gui-logs" "gui-settings" "hosts" "hosts-agg" "hosts-poller" "job" "link" "mail-sender" "maintenance" "pack-poller" "pack-read" "pack-write" "pack-update" "parsing-status-tcpsession-poller" "parsing-status-httppers-poller" "plugins" "session" "stats-collector" "status-poller" "tcp-poller" "tcp-read" "tcp-write" "tcp-update" "teams" "tls-keys" "tls-keys-linker" "web-httpcom-poller" "web-httpcom-content-poller" "web-httppers-poller" "web-read" "web-upload" "web-write" "whisp" "whisp-status-poller" "whisps-status" "whisps-status-agg"

Micro service name

Responses

Response samples

Content type
application/json
{
  • "application": "tcp-streams-update",
  • "hostname": "traballand-Latitude-E7240",
  • "instanceId": "c1033254c68f",
  • "requests": 886,
  • "successes": 886,
  • "errors": 0,
  • "errors4xx": 0,
  • "errors5xx": 0,
  • "duration": 2311,
  • "api": {
    }
}

Get circuit breakers stats

Description

Give circuit breakers informations for downstream connections of this service

Access

  • Admin
  • Application
  • User
Authorizations:
Bearer
path Parameters
service
required
string (ServiceNames)
Enum: "alert" "capture-status-poller" "ciphers" "ciphers-status" "ciphers-status-poller" "ciphers-raw-status-poller" "ciphers-status-agg" "controls" "customer" "gui-logs" "gui-settings" "hosts" "hosts-agg" "hosts-poller" "job" "link" "mail-sender" "maintenance" "pack-poller" "pack-read" "pack-write" "pack-update" "parsing-status-tcpsession-poller" "parsing-status-httppers-poller" "plugins" "session" "stats-collector" "status-poller" "tcp-poller" "tcp-read" "tcp-write" "tcp-update" "teams" "tls-keys" "tls-keys-linker" "web-httpcom-poller" "web-httpcom-content-poller" "web-httppers-poller" "web-read" "web-upload" "web-write" "whisp" "whisp-status-poller" "whisps-status" "whisps-status-agg"

Micro service name

Responses

Response samples

Content type
application/json
{
  • "application": "tcp-streams-write",
  • "hostname": "spider4",
  • "instanceId": "9300ef2cec06",
  • "circuitBreakers": {
    }
}

Get process stats

Description

Give process metrics for this service

Access

  • Admin
  • Application
  • User
Authorizations:
Bearer
path Parameters
service
required
string (ServiceNames)
Enum: "alert" "capture-status-poller" "ciphers" "ciphers-status" "ciphers-status-poller" "ciphers-raw-status-poller" "ciphers-status-agg" "controls" "customer" "gui-logs" "gui-settings" "hosts" "hosts-agg" "hosts-poller" "job" "link" "mail-sender" "maintenance" "pack-poller" "pack-read" "pack-write" "pack-update" "parsing-status-tcpsession-poller" "parsing-status-httppers-poller" "plugins" "session" "stats-collector" "status-poller" "tcp-poller" "tcp-read" "tcp-write" "tcp-update" "teams" "tls-keys" "tls-keys-linker" "web-httpcom-poller" "web-httpcom-content-poller" "web-httppers-poller" "web-read" "web-upload" "web-write" "whisp" "whisp-status-poller" "whisps-status" "whisps-status-agg"

Micro service name

Responses

Response samples

Content type
application/json
{
  • "application": "tcp-streams-write",
  • "hostname": "spider7",
  • "instanceId": "dd8ff8ac5565",
  • "startTime": "2019-01-16T21:56:26.109Z",
  • "upTime": 1209512.309,
  • "cpu": {
    },
  • "memory": {
    }
}

Get parsing stats

Description

Give metrics informations for parsing

Access

  • Admin
  • Application
  • User
Authorizations:
Bearer
path Parameters
parser
required
string (ParserNames)
Enum: "web-write" "tls-keys-linker"

Micro service name

Responses

Response samples

Content type
application/json
{
  • "application": "web-streams-write",
  • "hostname": "spider4",
  • "instanceId": "cce9207215ae",
  • "parsed": 13259,
  • "created": 5005,
  • "errors": 0,
  • "completed": 856,
  • "duration": 131623.869581,
  • "started": 856,
  • "delay": 8747139,
  • "durationPercentiles": {
    },
  • "delayPercentiles": {
    }
}

Alert

Alert service.

Get metrics

Description

Give metrics information collected by alerting probes in Prometheus format.

Access

  • Free

Responses

Get health

Description

Give summary of probes status

Access

  • Free
  • Exposed

Responses

Response samples

Content type
applications/json
{
  • "license": {
    },
  • "endpoint": "https://spider.mycompany.com",
  • "probes": {
    }
}

Controllers

Controllers are able to spawn Whisperers in a remote cluster.

Create a new Controller

Description

Create a new controller, and associate it to the owner customer.

Access

  • User, with controllers creation rights
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

Controller creation request

customer
required
string

System Id of the customer.

name
required
string

Name of the controller to create.

Responses

Request samples

Content type
application/json
{
  • "customer": "YOD66VZ54Jih",
  • "name": "Upload"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Controllers

Description

Search for controllers

Rules

If client is not admin, the search will be limited to the Controllers owned by this customer or shared with him, directly or by the team.

Access

  • Admin
  • User
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get default Configuration for Controllers

Description

Get default configuration.

Access

  • User
  • Admin
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a Controller

Description

Get a controller's details.

Access

  • User owning the controller
  • The own controller
  • User being shared access to this controller
  • User of the team being shared access to this controller
  • Customer application
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Controller

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "Controller",
  • "version": "string",
  • "name": "string",
  • "customer": "string",
  • "apikey": "string",
  • "config": {
    },
  • "users": [
    ],
  • "teams": [
    ],
  • "status": {
    },
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "editor": "string",
  • "dateModified": "2019-08-24"
}

Change controller's name or shared users

Description

Updates a customer. You can:

  • Update controller's name
  • Change sharing settings: teams, users and users rights on this controller

Rules

  • Can do any change:
    • Admin
    • User owning the controller
  • User being shared access to this controller with share rights can:
    • Add or remove users from the sharing settings
  • User being shared access to this controller with rights change rights can:
    • Change rights of users in the sharing settings

Access

  • User wning the controller
  • User of a team having shared access to the controller
  • User being shared access to this controller with config, share or rights modification rights
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Controller

header Parameters
If-Match
required
string

eTag of previous state of the controller

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Delete a Controller

Description

Delete a controller.

  • Put it to technicalStatus DELETED.

Access

  • User wning the controller
  • User being shared access to this controller and with delete right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Controller

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create/Replace the controller API key

Description

Create/Replace the controller API key used for controller connection to Spider.

The API key is a public/private key pair.

  • The public key is stored in controller's settings.
  • The private key is sent in response to this call.

Any call to this API (if authorized) will overwrite the previous API key, and the controller will not be able to use the previous one. A connected controller will be disconnected when its current JWT token will expire. The API key is taken as a configuration AT START of controllers, and need a restart to be changed.

Output

The API can supports two outputs:

  • application/json:
    • Provides a file with the private key, Spider's URL, and the controller's id This file is the only expected configuration file at controller start.
  • application/x-pem-file
    • Provides only the private key as a PEM file

Access

  • User wning the controller
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Responses

Response samples

Content type
{}

Generate an API key signature with the private key of the controller

Description

This endpoint is for testing purposes only:

  • It allows testing the API key or the controllers API without a controller
  • It generates a valid signature for the controller to call the configuration endpoint
  • However, IT REQUIRES YOUR PRIVATE KEY IN INPUT
  • After testing, please, reset your API key

Output

  • The signature for this controller, timestamp and private API key
  • A validation of this signature with the controller registered public API key

Access

  • User owning the controller
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

query Parameters
timeStamp
required
string <date-time>

Timestamp to use in signature

instanceId
required
string

InstanceId to use in signature

Request Body schema: application/x-pem-file
string

The controller RSA private key, as a PEM

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get Controller configuration

Description

Get the configuration of this Controller Usages:

  • Called by controller at start (or before token expiration) with their API key
  • Called by controllers regularly to check for configuration change
  • Called by UI to get configuration

API key

The first call from controllers is made by:

  • building a json payload with current time and controller id,
  • then signing it with SHA256 algorithm using controller's private key
const timeStamp = moment().toISOString();
const info = {
    timeStamp,
    controllerId
};
const privKey = new NodeRSA(privatePem);
const signature = privKey.sign(Buffer.from(JSON.stringify(info)), 'base64');
  • and then calling this API with specific headers:
  Spider-TimeStamp: timeStamp
  Spider-Signature: signature //base 64 encoded

Output

  • The configuration
  • A JWT token to use on further calls in Spider-Token header
    • If no token provided, or if called from a Customer
    • A Customer may call to get the configuration of one of its controllers and use the generated token to upload data (as on the UI)

Access

  • The own controller
  • User owning the controller
  • User being shared access to this controller
  • User of the team being shared access to this controller
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

header Parameters
Spider-Signature
string <base64>

Signature of the call by the Controller, with its API key

Spider-Timestamp
string <date-time>

Provided with API key in first Controller call to get its JWT token with the config

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Change Controller's config

Description

Updates a Controller configuration.

Rules

  • Can do any change:
    • Admin
    • User owning the Controller
  • User being shared access to this Controller with config rights can:
    • Change configuration settings
  • User being shared access to this Controller with share rights can:
    • Change sharing settings

Access

  • User owning the Controller
  • User being shared access to this Controller with config or share rights
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

header Parameters
If-Match
required
string

eTag of previous state of the Controller's config

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get namespaces of the cluster where the Controller is

Description

Get the list of namespaces names from the cluster.

Output

  • An array of string

Returns 404 if the controller is not connected.

Access

  • User owning the controller
  • User being shared access to this controller with attach right
  • User of the team being shared access to this controller with attach right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Responses

Response samples

Content type
application/json
[
  • "string"
]

Get list of objects of the requested namespace and collection from the cluster

Description

Get the list of objects from the cluster.

Output

  • An array of items

Returns 404 if the controller is not connected.

Access

  • User owning the controller
  • User being shared access to this controller with attach right
  • User of the team being shared access to this controller with attach right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

namespace
required
string

Name of the namespace we want the object from

collection
required
string
Enum: "pods" "statefulsets" "deployments" "cronjobs"

Collection we are interested in

Responses

Response samples

Content type
application/json
[
  • {
    },
  • {
    }
]

Create a new attachment

Description

Create a new attachment, asking, by it, to spawn a Whisperer to each linked Pod

  • The attachment is first saved in DB, and it will be fetched by the Controller next time with its configuration.
  • Then, an attachment request is sent to the Controller for an immediate attachment (if Controller is connected)

Access

  • User owning the controller
  • User being shared access to this controller with attach right
  • User of the team being shared access to this controller with attach right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Request Body schema: application/json
required

Attachment creation request

whisperer
required
string

Whisperer Id to attach.

namespace
required
string

Namespace where the workload is.

collection
required
string
Enum: "pods" "statefulsets" "daemonsets" "deployments" "cronjobs"

Collection of the workload.

item
required
string

Name of the workload.

agent
string
Enum: "whisperer" "gossiper"

Agent to attach.

Responses

Request samples

Content type
application/json
{
  • "whisperer": "string",
  • "namespace": "string",
  • "collection": "pods",
  • "item": "string",
  • "agent": "whisperer"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get last logs of the attached Whisperer

Description

Get last logs of the attached Whisperer.

Access

  • User owning the controller
  • User being shared access to this controller with attach or monitor right
  • User of the team being shared access to this controller with attach or monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

attachment
required
string

System id of the Attachment

container
required
string

Container id of the Whisperer

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Detach the attachment

Description

Ask for the attachment to be terminated.

The Whisperers connected to the worload linked to this attachment will terminate the next time they check their status.

Access

  • User owning the controller
  • User being shared access to this controller with attach right
  • User of the team being shared access to this controller with attach right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

attachment
required
string

System id of the Attachment

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Attachments

Description

Search for attachments

Rules

If client has limited access to namespaces, the search will be limited to the Attachments the user can access, directly or by the team.

Access

  • Admin
  • User owning this Controller
  • User being shared access to this Controller
  • User of the team being shared access to this Controller
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get the Whisperers managed by this Controller

Description

Returns the list of Whisperers that the Controller knows and manages.

Access

  • User owning the controller
  • User being shared access to this controller with attach or monitor right
  • User of the team being shared access to this controller with attach or monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get the Whisperers deployed as Sidecars seen by this Controller

Description

Returns the list of Sidecar Whisperers that the Controller knows.

Access

  • User owning the controller
  • User being shared access to this controller with monitor right
  • User of the team being shared access to this controller with monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get the last logs as Sidecar Whisperer

Description

Returns the last logs of the Sidecar Whisperer

Access

  • User owning the controller
  • User being shared access to this controller with monitor right
  • User of the team being shared access to this controller with monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

key
required
string

@id of the sidecar whisperer container as returned by GET /whisperers/sidecars

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get the Gociphers known by this Controller

Description

Returns the list of Gociphers that the Controller knows.

Access

  • User owning the controller
  • User being shared access to this controller with monitor right
  • User of the team being shared access to this controller with monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get the last logs of the Gociphers

Description

Returns the last Logs of the Gocipher.

Access

  • User owning the controller
  • User being shared access to this controller with monitor right
  • User of the team being shared access to this controller with monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

key
required
string

@id of the Gocipher container as returned by GET /gociphers

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Save Network Usage

Description

Save Network Usage to make it accessible for analysis

Access

  • The Controller
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Request Body schema: application/json
required

Network Usage

controller
required
string
minute
required
string <date-time>
nodes
required
integer >= 0
required
Array of objects

Responses

Request samples

Content type
application/json
{
  • "controller": "string",
  • "minute": "2019-08-24T14:15:22Z",
  • "nodes": 0,
  • "items": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Network Usage

Description

Search for Network Usage

Rules

If client has limited access to namespaces, the search will be limited to the Attachments the user can access, directly or by the team.

Access

  • Admin
  • User owning this Controller
  • User being shared access to this Controller
  • User of the team being shared access to this Controller
Authorizations:
Bearer
path Parameters
id
required
string

System id of Controller

Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get the Attachments associated to this Whisperer, from all know attachments

Description

Returns the list of Attachments linked to a Whisperer.

The service will calls all referenced and connected Controllers to get the status of the Whisperers.

Access

  • User owning the controller
  • User being shared access to this controller with attach or monitor right
  • User of the team being shared access to this controller with attach or monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of a Whisperer

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get the last logs of the Controller

Description

Returns the last logs of the Controller.

Access

  • User owning the controller
  • User being shared access to this controller with monitor right
  • User of the team being shared access to this controller with monitor right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of a Controller

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Customers

Users accounts.

Connect a customer

Description

Connects a customer and returns a JWT token

When successful, a cookie is set in the answer, containing the refresh_token. The cookie is http only, secure, with strict domain and path.

Rules

  • After many unsuccessful attempts, the email will be blocked for some time.

Access

  • No identification required
Request Body schema: application/json
required

The email/password for connection

email
required
string

Email

password
required
string <password>

Password

Responses

Request samples

Content type
application/json
{
  • "email": "string",
  • "password": "pa$$word"
}

Response samples

Content type
application/json
{
  • "customer": "string",
  • "token": "string"
}

Logs out a customer

Description

Logs out a customer and clears the refresh token cookie

Access

  • The cookie with the refresh token is expected in input

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Connect a customer using OIDC code flow

Description

Connects a customer using OIDC and returns a JWT token.
Takes in input the code provided by the Identity Provider and the name of the IP, as set in configuration.

Access

  • No identification required
Request Body schema: application/json
required

The code and IP to get the tokens from

code
required
string

Code received from the authorization_endpoint

provider
string

Identity provider name set in configuration

Responses

Request samples

Content type
application/json
{
  • "code": "string",
  • "provider": "string"
}

Response samples

Content type
application/json
{
  • "customer": "string",
  • "token": "string"
}

Connect a service account using OAuth2 client-credentials flow

Description

Connects a service account and returns a JWT token.
Takes in input the client_id and client_secret of the service account.

  • grant_type must be equals to "client_credentials"
  • when using application/x-www-form-urlencoded content type, client_id and client_secret may also be provided as a Basic Authorization header

Access

  • No identification required
Request Body schema:
required

The service account credentials

client_id
required
string
client_secret
required
string
grant_type
required
string
Value: "client_credentials"

Responses

Request samples

Content type
{
  • "client_id": "string",
  • "client_secret": "string",
  • "grant_type": "client_credentials"
}

Response samples

Content type
application/json
{
  • "access_token": "string",
  • "token_type": "Bearer",
  • "expires_in": 0
}

Generate a new access token and refreshes the refresh token

Description

Rotate the access token and issue a new refres token for a user

Responses

Response samples

Content type
application/json
{
  • "customer": "string",
  • "token": "string"
}

Create a new customer

Description

Create a new customer.

Rules

  • If a customer with same email already exists, creation is cancelled.
  • Depending on system settings, a self created customer (no token) will be:
    • Created as Draft but checked for all fields when admin activation is required
    • Created as Active when no activation is required

Draft customer

All fields are optional at start, but checked for correctness.

  • It then creates a DRAFT resource.
  • To update it to ACTIVE
    • Some fields are required.
    • Use PATCH.

Active customer

To create an ACTIVE customer, all mandatory fields must be set. Status must be set as ACTIVE.

Access

Either:

  • Self creation and no identification is required
  • Creation by someone else:
    • An admin user or user with users creation rights
    • Only a user with rights modification rights can create a user with rights
    • Only an admin can create an admin
  • Creation by a userTrainer, with trainee rights set.
Authorizations:
Bearer
Request Body schema: application/json
required

The customers details

email
required
string

Customer's email.

_password
required
string >= 6 characters

Customer's password.

_admin
boolean

True if user is an administrator.

required
object
birthDate
string <date>

Date of birth.

honorificPrefix
string

An honorific prefix preceding a name such as Dr/Mrs/Mr.

givenName
required
string

The given name, the first name.

familyName
required
string

The family name, the last name.

nationality
required
string

Nationality.

jobTitle
string

The job title (for example, Financial Manager).

worksFor
string

Organizations'name that the person works for.

Responses

Request samples

Content type
application/json
{
  • "email": "example@gmail.com",
  • "_password": "YGIUHIdzzf!/85F",
  • "_admin": true,
  • "givenName": "John",
  • "familyName": "Doe",
  • "nationality": "American",
  • "address": {
    },
  • "technicalStatus": "DRAFT"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create a new service account

Description

Create a new service account.

Draft service account

All fields are optional at start, but checked for correctness.

  • It then creates a DRAFT resource.
  • To update it to ACTIVE
    • Some fields are required.
    • Use PATCH.

Active customer

To create an ACTIVE service account, all mandatory fields must be set. Status must be set as ACTIVE.

Access

  • Creation by someone else:
    • An admin user or user with service accounts creation rights
    • Only a user with rights modification rights can create a service account with rights
    • Only an admin can create an admin
Authorizations:
Bearer
Request Body schema: application/json
required

The service account details

name
required
string
description
string
_password
required
string >= 6 characters

client_secret

_admin
boolean

True if service account is an administrator.

worksFor
string

Organizations'name that the person works for.

Responses

Request samples

Content type
application/json
{
  • "name": "Spider bot",
  • "_password": "YGIUHOIUHIOEUKBEZUICUYEGIIdzzf!/85F",
  • "technicalStatus": "ACTIVE"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Customers or Service Accounts

Description

Search for customers.

Also available by GET method on the collection

Access

  • Admin
  • User with user or service account management rights
  • userTrainer (but may only search on HIS trainees)
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a customer's or service account details

Description

Get a customer's details.

  • _password field is always stripped out.
  • Deleted customers are only visible by admins.
  • Depending on access rights, returns diverse representations.

Clients getting full details

  • The own client
  • Admins
  • User s with users administration rights

Response is:

  • Full content
  • Minus _* fields (password, admin...)

Clients getting system details

  • Whisp application (to update linked whisperers)

Response is limited to:

  • @id
  • email / name
  • rights
  • whisperers

Clients getting shortened info

  • Any other client

Response is limited to:

  • @id
  • email

Access

  • Customer, expect when using public links
  • Admin
  • User with users management right
  • User impersonating the user to open
  • User with userTrainer rights opening one of its trainee
  • Whisp application (to update linked whisperers)
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

Responses

Response samples

Content type
application/json
{
  • "@id": "Jz6lxOiuQYaIZNNouiyt6w",
  • "@type": "Person",
  • "version": "0.1",
  • "technicalStatus": "ACTIVE",
  • "creator": "AWUb00luIXCLtCIFlzoO",
  • "dateCreated": "2018-12-21T10:00:00.837Z",
  • "editor": "Jz6lxOiuQYaIZNNouiyt6w",
  • "dateModified": "2018-12-21T16:15:18.978Z",
  • "givenName": "John",
  • "familyName": "Doe",
  • "nationality": "American",
  • "email": "example@gmail.com",
  • "address": {
    },
  • "rights": {
    },
  • "whisperers": [
    ],
  • "_eTag": "\"d7-+6EM6pRmKNKDSTtAgeIK5g\""
}

Update customer's or service account details

Description

Updates a customer. You can:

  • Update customer details
  • Change password
  • Set new rights (and admin flag)
  • Change Whisperer names (for synchro)
  • Change its state from DRAFT to ACTIVE

Rules

  • Patch must be done with resource previous eTag

    • eTag must be given in ifMatch header (* not authorized)
    • eTag must match current eTag

  • Technical fields are protected (@id, creator, dateCreated, @type)

  • For a customer to change its password or email, patch operations must include previous password value inside a test operation

    • { "op":"test", "path":"_password", "value":"xxx" }

  • When customer changes email:

    • If a customer with same email already exists, operation is cancelled.
    • A confirmation email challenge is sent. The account is blocked at connection until the mail is confirmed.
    • A mail is sent to old email

  • When customer changes password:

    • A confirmation mail is sent

  • When a customer changes from Draft to Active

    • A information mail is sent

  • User cannot change its own Whisperers list

  • Admins can:

    • Update a customer's details ONLY when DRAFT
      • Also clients with creation right
    • Update rights
      • Also clients with rights admin right
    • Reinit password without providing old one
      • Also clients with password admin right
    • Set a customer as admin
    • Change a DELETED user back to ACTIVE

  • User details, once in ACTIVE state, can only be modified by own customer

  • Whisp & Maintenance services can:

    • Update associated whisperers

  • _password and _admin field cannot be removed, copied or moved

  • email field cannot be removed or moved

Access

  • Admin
  • Own customer
  • Whisp service (to update linked whisperers)
  • User with rights administration right
  • User with user creation right
  • User with password change/reinit right
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

header Parameters
If-Match
required
string

eTag of previous state of the customer

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Delete a customer or a service account

Description

Set a customer to DELETED status. When the customer has one to many own Whisperers of UPLOAD type, they are also deleted.

Access

  • Admin
  • User s with delete customer right
  • User with userTrainer right deleting on of its trainee account
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a customer info by its email

Description

Same as GET /customer/v1/customers/{id}

Authorizations:
Bearer
path Parameters
email
required
string

Customer email

Responses

Response samples

Content type
application/json
{
  • "@id": "Jz6lxOiuQYaIZNNouiyt6w",
  • "@type": "Person",
  • "version": "0.1",
  • "technicalStatus": "ACTIVE",
  • "creator": "AWUb00luIXCLtCIFlzoO",
  • "dateCreated": "2018-12-21T10:00:00.837Z",
  • "editor": "Jz6lxOiuQYaIZNNouiyt6w",
  • "dateModified": "2018-12-21T16:15:18.978Z",
  • "givenName": "John",
  • "familyName": "Doe",
  • "nationality": "American",
  • "email": "example@gmail.com",
  • "address": {
    },
  • "rights": {
    },
  • "whisperers": [
    ],
  • "_eTag": "\"d7-+6EM6pRmKNKDSTtAgeIK5g\""
}

Get user token to impersonate this customer

Description

Generates a token for a user to be able to use another user whisperers and rights.

Rules

  • User must not be deleted
  • Only an administrator may impersonate another administrator

Output

  • Generate a new token with
    • The requested customer id in impersonated field.
    • The customer's whisperers
    • The customers rights (if useUserRights is 'true')
  • The token can be used to call any API
  • The services will behave as if the customer was calling, except that all traces and audit fields will be valued with the original caller's id.

Access

  • User with impersonate right
  • User with userTrainer right impersonating one of its trainee
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

query Parameters
useUserRights
boolean

Ask to use user's right in the token. Keep caller's right if false.

Responses

Response samples

Content type
application/json
{
  • "token": "string"
}

Create a password challenge

Description

Create a password challenge to reinitialize a password. A token is created and sent by mail with a redirection link to the user. The redirection link:

  • launches an UI presenting a form to enter a new password
  • contains a unique token, valid once and a limited time

Rules

  • Account with this email must exist and not be deleted

Access

  • No identification required
Request Body schema: application/json
required

The email of the account

email
required
string

Email

redirectUrl
required
string

Base Url to construct the redirection link. Expected: Login UI endpoint.

Responses

Request samples

Content type
application/json
{
  • "email": "string",
  • "redirectUrl": "string"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Set a password after a password challenge

Description

Set a new password to a user with:

  • The challenge token
  • The user email
  • The password (plain)

An email is sent to the user to inform him of password change. Connections error count is reset ;)

Rules

  • Account with this email must exist and not be deleted
  • Token must still exists (not used, not too old)
  • Token must be associated to right email and user

Access

  • No identification required
Request Body schema: application/json
required

The new password of the account

email
required
string

Email

token
required
string

Token sent in challenge

password
required
string

Password choosen by the user

Responses

Request samples

Content type
application/json
{
  • "email": "string",
  • "token": "string",
  • "password": "string"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Confirm user email

Description

Used in mails sent to users to confirm their email addresses. This is why it is a GET, not a POST.

Rules

  • User must exist and not be deleted
  • The account confirmation token must exist
  • The token is linked to the right email and account
  • If user is in Draft status, an email is sent to admins for account activation
  • Confirmation page is shown

Access

  • No identification required
Authorizations:
Bearer
query Parameters
email
required
string

Email

token
required
string

Mail confirmation token

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Teams

Teams of users.

Create a new Team

Description

Creates a new team and add the owner as the first full rights customer.

Rules

  • A team with the same name must not exist

Access

  • User with team create right
  • User with userTrainer right creating a training team
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

The team to create

name
required
string

Team's name.

description
string

Team's description.

required
object

Owner of the team.

Responses

Request samples

Content type
application/json
{
  • "name": "string",
  • "description": "string",
  • "owner": {
    }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a team's details

Description

Get a team's details.

  • Depending on access rights, returns diverse representations.

Clients getting full details

  • Admins
  • Customers from the team
  • Customers using this team

Response is:

  • Full content

Clients getting system details

  • Whisp application (to update linked whisperers)

Response is limited to:

  • @id
  • name
  • whisperers

Clients getting shortened info

  • Any other client

Response is limited to:

  • @id
  • name

Access

  • Customer
  • Admin
  • Whisp application (to update linked whisperers)
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

Responses

Response samples

Content type
application/json
{
  • "@id": "AnaRheQISOmLu8bKiQv11w",
  • "@type": "Team",
  • "version": "0.1",
  • "name": "First team",
  • "description": "Short team description.",
  • "dateCreated": "2021-03-07T14:43:35.152Z",
  • "creator": "hfQ1rsfcRKWslHwWAPJ9bg",
  • "customers": [
    ],
  • "whisperers": [
    ],
  • "settings": {
    },
  • "technicalStatus": "ACTIVE",
  • "dateModified": "2021-03-18T22:15:39.345Z",
  • "token": "7p8i66dCQsynmFNULKdnjA",
  • "editor": "hfQ1rsfcRKWslHwWAPJ9bg"
}

Update team's details

Description

Updates a team. You can:

  • Update team's details
  • Add/remove customers
  • Set new customers rights
  • Add/remove whisperer names (for synchro)
  • Change team's settings

Rules

  • Patch must be done with resource previous eTag

    • eTag must be given in ifMatch header (* not authorized)
    • eTag must match current eTag

  • Patch cannot be done on DELETED team

  • Technical fields are protected (@id, creator, dateCreated, @type)

  • Customers with share right can update customers list and access filters

  • Customers with settings right can update team settings

  • Customers with update right can update name, description

  • Whisp and Maintenance services can:

    • Update associated whisperers

  • Token cannot be changed with patch

  • A mail is sent to team administrators with changes made

  • After update,

    • Whisperers list for customers and access filters are cleaned from any removed whisperer from the team
    • User s list for access filters are cleaned from any removed user from the team

Access

  • Admin
  • Whisp service (to update linked whisperers)
  • User with rights on team: share, whisperers, settings, update, rights
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

header Parameters
If-Match
required
string

eTag of previous state of the team

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Delete a team

Description

Set a team to DELETED status.

If the team is a training team, its own whisperers are deleted.

Access

  • Admin
  • User s with update team right
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Join a team

Description

Join a user to a team with:

  • The join token
  • The user @id
  • The user email

An email is sent to the team admin to tell them about new user.

Rules

  • The join token must belong to a team
  • Team must still be active
  • User must not be part of the team

Access

  • Admin
  • User whose @id is in body
Authorizations:
Bearer
Request Body schema: application/json
required

The new password of the account

email
required
string

Email

token
required
string

Token sent in challenge

password
required
string

Password choosen by the user

Responses

Request samples

Content type
application/json
{
  • "email": "string",
  • "token": "string",
  • "password": "string"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Add a share token to the team

Description

Set a share token to the team. The token is randomly generated.

Rules

  • Team must not be deleted
  • A notification mail is sent to team admins

Access

  • User being part of the team, with team update right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Remove team share token

Description

Delete the team share token.

Rules

  • Team must not be deleted
  • A notification mail is sent to team admins

Access

  • User being part of the team, with team update right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get user token for this team

Description

Generates a token for the user to be able to use team's whisperers and rights.

Rules

  • Team must not be deleted

Access

  • User being part of the team
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Team internal @id

Responses

Response samples

Content type
application/json
{
  • "token": "string"
}

Get a team info by its name

Description

Same as GET /teams/v1/teams/{id}

Authorizations:
Bearer
path Parameters
name
required
string

Team name

Responses

Response samples

Content type
application/json
{
  • "@id": "AnaRheQISOmLu8bKiQv11w",
  • "@type": "Team",
  • "version": "0.1",
  • "name": "First team",
  • "description": "Short team description.",
  • "dateCreated": "2021-03-07T14:43:35.152Z",
  • "creator": "hfQ1rsfcRKWslHwWAPJ9bg",
  • "customers": [
    ],
  • "whisperers": [
    ],
  • "settings": {
    },
  • "technicalStatus": "ACTIVE",
  • "dateModified": "2021-03-18T22:15:39.345Z",
  • "token": "7p8i66dCQsynmFNULKdnjA",
  • "editor": "hfQ1rsfcRKWslHwWAPJ9bg"
}

Search for Teams

Description

Search for teams.

Also available by GET method on the collection

Rules

  • When not called by admin, will limit to teams of which the user belongs.

Access

  • Admin
  • User of the team
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Packets

Network packets, as captured on the wire(less). ANSI layer 3.

Push packets

Description

Process a json payload of packets and:

  • Make them ready for parsing.
  • Save them (optional).

Checks

  • Spider packets structure

Access

  • Whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

Json payload with packets to analyse by Spider.

Array
@id
string

Unique id of the packet in the system.

@type
string
Value: "Packet"
version
string
Value: "2.0"

Version of the schema.

name
string

Name of the packet (for display).

whisperer
string

Whisperer that captured the packet

instanceId
string

Instance id of the whisperer

tcpSession
string

Id of the Tcp session the packet is in

timestamp
number <double>

Unix timestamp of capture, with microseconds

length
integer

Size of the packet (size of rawPacket.buf buffer)

object

List of protocols used by this packet, keys are protocols name: TCP, UDP, IPv4...

object

Responses

Request samples

Content type
application/json
{
  • "@id": "ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5",
  • "@type": "Packet",
  • "version": "2.0",
  • "commonId": "ROlrqlFhTY2ayXIxTV2uZA.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5",
  • "name": "456285.172.16.102.72-43118-172.16.102.125-8080#5",
  • "whisperer": "ROlrqlFhTY2ayXIxTV2uZA",
  • "instanceId": "rd-srv508-bes",
  • "tcpSession": "ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931",
  • "timestamp": 1642625100.188968,
  • "length": 229,
  • "protocols": {
    },
  • "rawPacket": {
    }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for packets

Description

Searches or aggregation analysis on packets

Also available by GET method on the collection

Rules

Admin may search without specifying a whisperer

Access

  • Admin or client
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a packet

Description

Get a packet

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of packet

Responses

Response samples

Content type
application/json
{
  • "@id": "ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5",
  • "@type": "Packet",
  • "version": "2.0",
  • "commonId": "ROlrqlFhTY2ayXIxTV2uZA.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931-5",
  • "name": "456285.172.16.102.72-43118-172.16.102.125-8080#5",
  • "whisperer": "ROlrqlFhTY2ayXIxTV2uZA",
  • "instanceId": "rd-srv508-bes",
  • "tcpSession": "ROlrqlFhTY2ayXIxTV2uZA.rd-srv508-bes.456285.172.16.102.72-43118-172.16.102.125-8080.1682747931",
  • "timestamp": 1642625100.188968,
  • "length": 229,
  • "rawPacket": {
    },
  • "protocols": {
    },
  • "date": "2022-01-19T20:45:00.188Z",
  • "minute": "2022-01-19T20:45:00.000Z",
  • "protocolsList": [
    ]
}

Aggregate TCP payload for provided packets (fallback)

Description

Build the tcp payload of the packets listed in input.

Also available by GET method

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

List of packets identifiers

Array
string

System id of packet

Responses

Request samples

Content type
application/json
[
  • "string"
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a collection of packets (fallback)

Description

Get the packets listed in input (by id).

Also available by GET method

Access

  • Admin
  • Application (another service)
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

List of packets identifiers

Array
string

System id of packet

Responses

Request samples

Content type
application/json
[
  • "string"
]

Response samples

Content type
application/json
[
  • { }
]

Get a packets of a Tcp Session between two indices (or from a index to the end)

Description

Get the packets of the {tcpSession} in input, from {indexStart} to {indexEnd} (opt).

Access

  • Admin
  • Application (another service)
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Range of packets to receive

tcpSession
string

Id of the Tcp session the packet is in

indexStart
integer

Index above which the first packet to send must be (exclusive)

indexEnd
string

Index before which the last packet to send must be (inclusive), optional

Responses

Request samples

Content type
application/json
{
  • "tcpSession": "string",
  • "indexStart": 0,
  • "indexEnd": "string"
}

Response samples

Content type
application/json
[
  • { }
]

Aggregate TCP payload for several group of packets

Description

Build the tcp payload of the list of packets groups listed in input.

Also available by GET method

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

List of packets groups

Array
requestId
string

Id of group of packets, internal to client

packetIds
Array of strings

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
[
  • {
    }
]

Set packets as parsed

Description

Set the packets of {tcpSession} before {maxIndex} as parsed. When parsed, they are removed from working memory in Redis. Depending of whisperer settings to save Packets:

  • Poller may synchronize them to ES, then remove them from Redis.
  • Poller may plainly remove them from Redis.
  • Pack Update may directly remove them from Redis, if already processed by Poller.

This API is used in real time processing of packets to optimise Redis usage and speed of processing.

Access

  • Admin
  • Application (another service)
Authorizations:
Bearer
Request Body schema: application/json
required
Array
tcpSession
string

System id of the TCP session owning the packets

maxIndex
integer

Maximum index that has been parsed (inclusive)

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Purge packets

Description

Create a asynchronous purging job of packets.

Access

  • Admin
  • User asking to purge only on whisperers it owns of with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

List of packets identifiers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge packets progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Tcp sessions

TCP sessions: consistent stateful communication of packets between 2 hosts. Can contain any kind of exchanges. ANSI layer 4.

Push Tcp sessions

Description

Stores TCP sessions and trigger parsing of payload according to Whisperers parsing configuration.

Access

  • Whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

Tcp Session with packets id to analyse by Spider.

Array
@id
string

System id of TCP session

name
string
object

Client host

object

Server host

state
string
Enum: "SYN_SENT" "SYN_RECEIVED" "ESTABLISHED" "CLOSE_WAIT" "LAST_ACK" "CLOSED"

State of TCP session lifecycle

packetsCount
integer

Count of packets in the sessions

synTimestamp
number <double>

Timestamp of SYN packet

missedSyn
boolean

If whisperer missed SYN

connectTimestamp
number <double>

Timestamp when connection was established

firstTimestamp
number <double>

Timestamp of first packet (different from SYN when missedSyn)

lastTimestamp
number <double>

Timestamp of last packet

object

Out packets (responses from server)

object

In packets (responses from server)

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a TCP session

Description

Get a TCP session

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of TCP session

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "Tcp session",
  • "version": "string",
  • "name": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "src": {
    },
  • "dst": {
    },
  • "state": "SYN_SENT",
  • "packetsCount": 0,
  • "syn": 0.1,
  • "missedSyn": true,
  • "connect": 0.1,
  • "first": 0.1,
  • "firstDate": "2019-08-24",
  • "last": 0.1,
  • "lastDate": "2019-08-24",
  • "duration": 0.1,
  • "timespan": {
    },
  • "latency": 0.1,
  • "out": {
    },
  • "in": {
    },
  • "parsers": {
    },
  • "dateModified": "2019-08-24"
}

Search for TCP sessions (fallback)

Description

Searches or aggregation analysis on TCP sessions

Also available by GET method on the collection

Rules

May ask for an aggregation, with size:0 and no whisperers defined:

  • Admin
  • User with admin monitoring right

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Preaggregated search for parsing status histogram

Description

Histogram aggregation analysis on TCP sessions

Rules

Size:0, no next, no sort.

  • Admin
  • User with admin monitoring right

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Modify the parsing result of a TCP session

Description

Update the Tcp session once the parsing is done.

Rules

  • Only /parsers block can be updated

Access

  • Admin
  • Application
  • Whisperer
    • Limited access to the whisperer linked to the Tcp session
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of TCP session

header Parameters
If-Match
required
string

eTag of previous state of the TCP session

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    },
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Modify the parsing result of TCP sessions, in bulk

Description

Update Tcp sessions once the parsing is done.

Rules

  • Only /parsers block can be updated

Access

  • Admin
  • Application
Authorizations:
Bearer
Request Body schema: application/json
required

Array of Patches to TcpSessions

Array
@id
string

System Id of the Tcp session to update

_eTag
string

eTag of the Tcp session to update

Array of objects (JsonPatch)

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "successes": [
    ],
  • "failures": [
    ]
}

Get a set of TCP sessions with Tls by their client random

Description

Get a set of TCP session from a list of client ranndoms.

Access

  • Admin
  • Tls keys linker application
Authorizations:
Bearer
Request Body schema: application/json
required

Array of Whisperer+ClientRandom

Array
whisperer
string

Whisperer Id of Whisperer having captured the TcpSession

clientRandom
string

Client random found in the TLS handshake

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
[
  • { }
]

Ask for Tcp sessions to parse by {type} parser

Description

Get Tcp sessions that needs parsing by {type} parser

  • Returns a collection of Tcp sessions to parse

Access

  • Admin
  • Application
Authorizations:
Bearer
path Parameters
type
required
string
Value: "HTTP"

Type of parsing

Request Body schema: application/json
required

Polling parameters

before
string <date-time>

Date before which to get sessions to parse. Safety delay.

Responses

Request samples

Content type
application/json
{
  • "before": "2019-08-24T14:15:22Z"
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ]
}

Ask for Tcp sessions in Warning status to parse by {type} parser

Description

Get Tcp sessions that have already been parsed by {type} parser, but that ended in WARNING parsing status. Indeed, most often the WARNING status means that the TCP session could not be parsed do to missing packets. We retry a bit ater once packets should be there.

  • Returns a collection of Tcp sessions to parse

Access

  • Admin
  • Application
Authorizations:
Bearer
path Parameters
type
required
string
Value: "HTTP"

Type of parsing

Request Body schema: application/json
required

Polling parameters

before
string <date-time>

Date before which to get sessions to parse. Safety delay.

Responses

Request samples

Content type
application/json
{
  • "before": "2019-08-24T14:15:22Z"
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ]
}

Purge TCP sessions

Description

Create a asynchronous purging job of TCP sessions.

Access

  • Admin
  • User asking to purge only on whisperers it owns of with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

List of TCP sessions identifiers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge TCP sessions progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Get parsing queue status

Description

Get parsing queue status for TCP sessions in WAITING parsing status

Access

  • Admin
  • Application
  • Customer
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "totalInWaiting": 2,
  • "ageOfFirst": 2345,
  • "ageOfLast": 2456
}

Http communications

HTTP communications: unitary communication used on the Web. ANSI layer 5-7.

Purge HTTP communications

Description

Create a asynchronous purging job of HTTP communications.

Access

  • Admin
  • User asking to purge only on whisperers it owns of with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

List of HTTP communications identifiers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge Http communications progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Purge HTTP parsing logs

Description

Create a asynchronous purging job of HTTP parsing logs.

Access

  • Admin
  • User asking to purge only on whisperers it owns of with purge right
Authorizations:
Bearer
Request Body schema: application/json
required

List of HTTP parsing logs identifiers

whisperers
Array of strings
from
number <double>

Start unix timestamp of purge window. Up to 6 decimals for microseconds.

to
number <double>

Stop unix timestamp of purge window. Up to 6 decimals for microseconds.

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "from": 0.1,
  • "to": 0.1
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get purge Http parsing logs progress

Description

Get purge progress

Access

  • Admin
  • Customer
Authorizations:
Bearer
path Parameters
job
required
string

Internal id of job

Responses

Response samples

Content type
application/json
{
  • "completed": true,
  • "total": 0,
  • "deleted": 0,
  • "failures": [
    ],
  • "durationMs": 0
}

Import Http communications

Description

Import a collection of Http communications

Access

  • Admin
  • Whisperer
    • Only Httpcoms on this Whisperer can be created
Authorizations:
Bearer
Request Body schema: application/json
@type
string
Value: "HttpComDownload"
user
string
totalItems
integer

Count of Http communications to import

Array of objects (HttpCommunication)

Array of Http communications

Responses

Request samples

Content type
application/json
{
  • "@type": "HttpComDownload",
  • "user": "string",
  • "totalItems": 0,
  • "items": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get an HTTP communication

Description

Get an Http communication by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of HTTP Communication

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "name": "string",
  • "tcpSession": "string",
  • "httpPers": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "uploaded": true,
  • "stats": {
    },
  • "req": {
    },
  • "res": {
    }
}

Get the headers part of an HTTP communication (request or response)

Description

Get the HTTP headers of an Http communication by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of HTTP Communication

part
required
string
Enum: "req" "res"

Request or Response

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get the body part of an HTTP communication (request or response)

Description

Get the payload body of an Http communication by its Id

Output

The output can be in 4 types, based on view parameter:

  • [not defined]: Get the payload inline with original content type and content encoding, and with filename {id}.{extension based on mime type}
  • raw: Get the payload as text with original encoding, inline, and with filename {id}.{extension based on mime type}
  • download: Get the payload with original content-type and encoding, in an attachement file {id}.{extension based on mime type}
  • octet-stream: Get the payload as captured on the network, in a binary attachement {id}.a

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of HTTP Communication

part
required
string
Enum: "req" "res"

Request or Response

query Parameters
view
required
string
Enum: "raw" "download" "octet-stream"

Format of output

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get an HTTP parsing log

Description

Get an Http parsing log by its Id

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
path Parameters
id
required
string

System id of HTTP Persistent Connection

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "name": "string",
  • "tcpSession": "string",
  • "whisperer": "string",
  • "instanceId": "string",
  • "first": 0.1,
  • "last": 0.1,
  • "lastPacketLotComplete": 0,
  • "lastPacketParsedIndex": 0,
  • "parsingCount": 0,
  • "itemsFound": 0,
  • "packetLots": [
    ]
}

Search for Http communications

Description

Searches or aggregation analysis on Http communications

Also available by GET method on the collection

Rules

May ask for an aggregation, with size:0 and no whisperers defined:

  • Admin
  • User with admin monitoring right

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account access filters
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Preaggregated search for parsing status histogram

Description

Histogram aggregation analysis on Http communications

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Tls keys

TLS keys used to encrypt TLS sessions. ANSI layer 5.

Push Tls keys

Description

Stores Tls keys and trigger parsing of to link them to TCP sessions.

Access

  • Gociphers
  • Whisperers
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

Tls Keys to store.

Array
object

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Ask for new Tls keys to link to TCP sessions

Description

Get Tls keys that needs linking

  • Returns a collection of Tls keys to link

Access

  • Admin
  • Application
Authorizations:
Bearer
Request Body schema: application/json
required

Polling parameters

before
string <date-time>

Date before which to get tls keys to parse. Safety delay.

Responses

Request samples

Content type
application/json
{
  • "before": "2019-08-24T14:15:22Z"
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ]
}

Ask for existing Tls keys to be removed from cache

Description

Remove Tls keys from cache

Access

  • Admin
  • Application: tls-leys-linker
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Whisp

To control Whisperers, the probes sending sniffed communications to Spider

Ciphers

To control Gociphers, the probes sending TLS keys to Spider

Create a new Gocipher

Description

Create a new Gocipher, and associate it to the owner customer.

Access

  • Customer, with Gociphers creation rights
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

Gocipher creation request

customer
required
string

System Id of the customer.

name
required
string

Name of the Gocipher to create.

Responses

Request samples

Content type
application/json
{
  • "customer": "YOD66VZ54Jih",
  • "name": "Dev cluster"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get default Configuration for Gociphers

Description

Get default configuration.

Access

  • User
  • Admin
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Gociphers

Description

Search for Gociphers

Also available by GET method on the collection

Rules

If client is not admin, the search will be limited to the Gociphers owned by this customer or shared with him.

Access

  • Admin
  • Customer
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a Gocipher

Description

Get a Gocipher's details.

Rules

Only admins can see DELETED Gociphers.

Access

  • User owning the Gocipher
  • The own Gocipher
  • User being shared access to this Gocipher
  • User of a Team having access to this Gocipher
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Gocipher

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "Gocipher",
  • "version": "string",
  • "name": "string",
  • "customer": "string",
  • "apikey": "string",
  • "config": {
    },
  • "users": [
    ],
  • "teams": [
    ],
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "editor": "string",
  • "dateModified": "2019-08-24"
}

Change Gocipher's name or shared users/teams

Description

Updates a customer. You can:

  • Update Gocipher's name
  • Change sharing settings: users and users rights on this Gocipher

Rules

  • Can do any change:
    • Admin
    • User owning the Gocipher
    • User of the team owning the Gocipher with Gociphers right
  • User being shared access to this Gocipher with config rights can:
    • Change the name
  • User being shared access to this Gocipher with share rights can:
    • Add or remove users from the sharing settings
  • User being shared access to this Gocipher with rights change rights can:
    • Change rights of users in the sharing settings

Access

  • User owning the Gocipher
  • User being shared access to this Gocipher with config, share or rights modification rights
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Gocipher

header Parameters
If-Match
required
string

eTag of previous state of the Gocipher

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Delete a Gocipher

Description

Delete a Gocipher.

  • Put it to technicalStatus DELETED.

Access

  • User owning the Gocipher
  • User being shared access to this Gocipher and with delete right
  • Customer application
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Gocipher

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create/Replace the Gocipher API key

Description

Create/Replace the Gocipher API key used for Gocipher connection to Spider.

The API key is a public/private key pair.

  • The public key is stored in Gocipher's settings.
  • The private key is sent in response to this call.

Any call to this API (if authorized) will overwrite the previous API key, and the Gocipher will not be able to use the previous one. A connected Gocipher will be disconnected when its current JWT token will expire. The API key is taken as a configuration AT START of Gociphers, and need a restart to be changed.

Output

The API can supports two outputs:

  • application/json:
    • Provides a file with the private key, Spider's URL, and the Gocipher's id This file is the only expected configuration file at Gocipher start.
  • application/x-pem-file
    • Provides only the private key as a PEM file

Access

  • User owning the Gocipher
  • User of the team owning the Gocipher with Gociphers right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Gocipher

Responses

Response samples

Content type
{}

Generate an API key signature with the private key of the Gocipher

Description

This endpoint is for testing purposes only:

  • It allows testing the API key or the Gociphers API without a Gocipher
  • It generates a valid signature for the Gocipher to call the configuration endpoint
  • However, IT REQUIRES YOUR PRIVATE KEY IN INPUT
  • After testing, please, reset your API key

Output

  • The signature for this Gocipher, timestamp and private API key
  • A validation of this signature with the Gocipher registered public API key

Access

  • User owning the Gocipher
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Gocipher

query Parameters
timeStamp
required
string <date-time>

Timestamp to use in signature

instanceId
required
string

InstanceId to use in signature

Request Body schema: application/x-pem-file
string

The Gocipher RSA private key, as a PEM

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get Gocipher configuration

Description

Get the configuration of this Gocipher Usages:

  • Called by Gocipher at start (or before token expiration) with their API key
  • Called by ephemeral Gocipher at start with the token given by the Controller
  • Called by Gociphers regularly to check for configuration change
  • Called by UI to get a Gocipher token to perform upload
  • Called by UI to get configuration
  • Called by services to get Gocipher configuration when parsing

API key

The first call from Gociphers is made by:

  • building a json payload with current time and Gocipher id,
  • then signing it with SHA256 algorithm using Gocipher's private key
const timeStamp = moment().toISOString();
const info = {
    timeStamp,
    GocipherId,
    instanceId
};
const privKey = new NodeRSA(privatePem);
const signature = privKey.sign(Buffer.from(JSON.stringify(info)), 'base64');
  • and then calling this API with specific headers:
  Spider-TimeStamp: timeStamp
  Spider-InstanceId: instanceId,
  Spider-Signature: signature //base 64 encoded

Parameters

The view parameter allows to modulate the output:

server

  • Only server part (parsing) is sent
  • Current configuration is merged with default configuration

client

  • Only client part (parsing) is sent
  • Current configuration is merged with default configuration

full

  • Both client & server are sent
  • Current configuration is merged with default configuration

null

  • The raw configuration is sent (without merging with defaults)
  • This view is used by UI to know what settings are specific, and what settings are defaults

Output

  • The configuration
  • A JWT token to use on further calls in Spider-Token header
    • If no token provided

Access

  • The own Gocipher
  • User owning the Gocipher
  • User being shared access to this Gocipher
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Gocipher

query Parameters
view
string
Enum: "server" "client" "full"

Type of configuration to get

header Parameters
Spider-Signature
string <base64>

Signature of the call by the Gocipher, with its API key

Spider-Timestamp
string <date-time>

Provided with API key in first Gocipher call to get its JWT token with the config

Spider-InstanceId
string

Provided with API key in first Gocipher call to get its JWT token with the config

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Change Gocipher's config

Description

Updates a Gocipher configuration.

Rules

  • Can do any change:
    • Admin
    • User owning the Gocipher
  • User being shared access to this Gocipher with config rights can:
    • Change configuration settings
  • User being shared access to this Gocipher with share rights can:
    • Change sharing settings

Access

  • User owning the Gocipher
  • User being shared access to this Gocipher with config or share rights
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Gocipher

header Parameters
If-Match
required
string

eTag of previous state of the Gocipher's config

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Hosts

The list of Hosts detected by Whisperers

Send an hosts update for a whisperer

Description

An hosts list contains all hosts the whisperer has seen during its capture.

When processing the list, Spider can create a new entry of hosts for this Whisperer or update the nearest one.

Hosts lists can be sent in any order (to support upload).

Access

  • Own whisperer
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

Request Body schema: application/json
Array (non-empty)
ip
required
string <ipv4>

IP address of host

name
required
string

FQDN of host as given by DNS

type
required
string
Enum: "SERVER" "CLIENT" null

Type of host

firstSeen
required
string <date-time>

Date of first packet seen for this host

lastSeen
required
string <date-time>

Date of last packet seen for this host

lastUpdate
required
string <date-time>

Last time the DNS was queried to update the host name

Responses

Request samples

Content type
application/json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Change the custom name of a Host.

Description

Update an Host:

  • identified by an IP
  • between two dates
  • on one to many whisperers

The update:

  • May change the customName of the host
  • May update several hosts lists of the same whisperer
  • May update several hosts lists of many whisperers

Access

  • Admin
  • User asking to update only on its whisperers (owned or shared)
Authorizations:
Bearer
Request Body schema: application/json
required
whisperers
required
Array of strings non-empty

List of whisperers on which to do the update

ip
required
string <ipv4>

IP of the host for which to set custom name

startTime
required
number <double>

Timestamp of period start for which to change

stopTime
required
number <double>

Timestamp of period end for which to change

customName
string

The new name / optional - can be null

Responses

Request samples

Content type
application/json
{
  • "whisperers": [
    ],
  • "ip": "10.0.0.236",
  • "startTime": 1548595820.545,
  • "stopTime": 1548595882.677,
  • "customName": "test-service"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Hosts detected by a whisperer

Description

Search for hosts Returns a collection of HostsList

Rules

May ask searching without specifying a whisperer:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Whisp status

Runtime metrics from Whisperers (status, speed, cpu...)

Send status information from a whisperer

Description

Whisperers are sending status every x seconds when started

Access

  • Own whisperer
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
time
required
string <date-time>

Time of status

whisperer
required
string

System Id of whisperer

instanceId
required
string

Instance id of the whisperer

whispererName
string

Name of whisperer - enriched by service

hostname
required
string

FQDN of the host

startTime
required
string <date-time>

Start time of the whisperer

sessionStartTime
string

Recording session start time

upTime
required
number

Uptime of whisperer

state
required
string
Enum: "STARTING" "RECORDING" "STOPPED" "SERVER_DOWN" "BREAK" "INTERNAL_ERROR" "INVALID_CONFIG"

State of the whisperer

object

Statistics of REST API calls to the server, by API

required
object

Total of metrics values since start of session

object

Metrics values since last send of status

Array of objects

List of network interfaces of the host

Responses

Request samples

Content type
application/json
{
  • "@id": "Y2GaZbDXRLq3cj-m8Zjviw.1548595914874",
  • "time": "2019-01-27T13:31:54.874Z",
  • "whisperer": "Y2GaZbDXRLq3cj-m8Zjviw",
  • "hostname": "node-3.streetsmart.sit3",
  • "instanceId": "1a2aebcf734f",
  • "startTime": "2019-01-16T14:40:19.843Z",
  • "sessionStartTime": "2019-01-25T11:17:40.234Z",
  • "upTime": 946295.88,
  • "state": "RECORDING",
  • "circuitBreakers": {
    },
  • "total": {
    },
  • "new": {
    },
  • "interfaces": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a collection of current whisps status

Description

Get the statuses listed in input (by id).

Also available by GET method

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

List of whisperers identifiers

Array
string

System id of whisperer

Responses

Request samples

Content type
application/json
[
  • "string"
]

Response samples

Content type
application/json
[
  • { }
]

Search for raw status sent by a whisperer

Description

Search for raw status

Rules

May ask searching without specifying a whisperer:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer, limited on search on its whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Preaggregated search for parsing status histogram

Description

Histogram aggregation analysis on Whisperers status

Access

  • Admin
  • Customer
    • Limited access to the whisperers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Search for current status of whisperers

Description

Search for current status of whisperers

Rules

May ask searching without specifying a whisperer:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer, limited on search on its whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

Responses

Request samples

Content type
application/json
{
  • "size": 20,
  • "whisperers": [
    ],
  • "query": "",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Ciphers status

Runtime metrics from Gociphers (status, speed, cpu...)

Send status information from a Gocipher

Description

Gociphers are sending status every x seconds when started

Access

  • Own whisperer
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
time
required
string <date-time>

Time of status

whisperer
required
string

System Id of whisperer

instanceId
required
string

Instance id of the whisperer

whispererName
string

Name of whisperer - enriched by service

hostname
required
string

FQDN of the host

startTime
required
string <date-time>

Start time of the whisperer

sessionStartTime
string

Recording session start time

upTime
required
number

Uptime of whisperer

state
required
string
Enum: "STARTING" "RECORDING" "STOPPED" "SERVER_DOWN" "BREAK" "INTERNAL_ERROR" "INVALID_CONFIG"

State of the whisperer

object

Statistics of REST API calls to the server, by API

required
object

Total of metrics values since start of session

object

Metrics values since last send of status

Array of objects

List of network interfaces of the host

Responses

Request samples

Content type
application/json
{
  • "@id": "Y2GaZbDXRLq3cj-m8Zjviw.1548595914874",
  • "time": "2019-01-27T13:31:54.874Z",
  • "whisperer": "Y2GaZbDXRLq3cj-m8Zjviw",
  • "hostname": "node-3.streetsmart.sit3",
  • "instanceId": "1a2aebcf734f",
  • "startTime": "2019-01-16T14:40:19.843Z",
  • "sessionStartTime": "2019-01-25T11:17:40.234Z",
  • "upTime": 946295.88,
  • "state": "RECORDING",
  • "circuitBreakers": {
    },
  • "total": {
    },
  • "new": {
    },
  • "interfaces": [
    ]
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get a collection of current Gociphers status

Description

Get the statuses listed in input (by id).

Also available by GET method

Access

  • Admin
  • Customer
    • Limited access to the Gociphers it has access to
    • Taking into account time range for public links
Authorizations:
Bearer
Request Body schema: application/json
required

List of Gociphers identifiers

Array
string

System id of Gocipher

Responses

Request samples

Content type
application/json
[
  • "string"
]

Response samples

Content type
application/json
[
  • { }
]

Search for raw status sent by a Gocipher

Description

Search for raw status

Rules

May ask searching without specifying a Gocipher:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer, limited on search on its Gociphers
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Search for current status of Gociphers

Description

Search for current status of Gociphers

Rules

May ask searching without specifying a Gocipher:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer, limited on search on its Gociphers
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

Responses

Request samples

Content type
application/json
{
  • "size": 20,
  • "whisperers": [
    ],
  • "query": "",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get current targets for this Whisperer across all Gociphers

Description

Get the targets being watched for this Whisperer.

Also available by GET method

Access

  • Admin
  • Customer
    • Limited access to the Whisperers it has access to
Authorizations:
Bearer
path Parameters
whisperer
required
string

Internal id of Whisperer

Responses

Response samples

Content type
application/json
[
  • { }
]

Create a new Link

Description

Stores a UI state and gives a link in returns.

Access

  • Customer
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

The state to link

required
object
object
main
required
object
required
object
search
required
object

Responses

Request samples

Content type
application/json
{
  • "userInfo": {
    },
  • "time": {
    },
  • "main": { },
  • "networkMap": {
    },
  • "search": { }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Links

Description

Search for links.

Also available by GET method on the collection

Rules

May ask searching without specifying a whisperer:

  • Admin
  • User with admin monitoring rights

Access

  • Admin
  • Customer
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a link

Description

Get a link
The link may be public or not

  • Customer
  • Admin

Rules

  • If Public link, it is not DELETED or deprecated
  • User who created it
  • Admin
  • When using public link token
    • FreeAccess is true
    • User email is in recipients list
    • User email domain is in domains list
Authorizations:
Bearer
path Parameters
id
required
string

Links system Id

Responses

Response samples

Content type
application/json
{
  • "@id": "fH5R3ZjhR7WPHBn2GDg1cA",
  • "@type": "sp:link",
  • "dateCreated": "2019-01-27T20:58:30.421Z",
  • "creator": "VBqPbgjYRsK00O76DVeroQ==",
  • "version": "0.1",
  • "content": { }
}

Create a new Public Link

Description

Stores a UI state and gives a public link in return.

Access

  • User with publishing permission on the whisperers present in the state, or on the team
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required
required
Array of objects

A collection of filters defining access control.

freeAccess
boolean

Flag indicating if the link provides free access without restrictions.

sendEmail
boolean

Indicates whether an email notification should be sent.

dateDeprecated
string <date>

The ISO date when the public link becomes deprecated.

domains
required
Array of strings

List of domains whose emails have access to the public link. Starting with '@'.

recipients
required
Array of strings

Email addresses of the recipients with access.

stateToShare
required
object

Object representing the UI state to be shared.

Responses

Request samples

Content type
application/json
{
  • "accessFilters": [
    ],
  • "freeAccess": true,
  • "sendEmail": true,
  • "dateDeprecated": "2019-08-24",
  • "domains": [
    ],
  • "recipients": [
    ],
  • "stateToShare": { }
}

Response samples

Content type
application/json
{
  • "urlToShare": "string"
}

Search for Public Links

Description

Search for public links.

Also available by GET method on the collection

Rules

  • content is removed
  • urlToShare field is added with the Url of the link

Access

  • Admin
  • User to the public links he created
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Delete a public link

Description

Set a public link to DELETED status, making it invisible to searches and get.

Access

  • User that has created the public link
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Public link system Id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create and send a One Time Password for a user trying to connect to a Public link

Description

Create a One Time Password for a user trying to connect to a Public link. Send this OTP to the email of the user.

Rules

  • Link is not DELETED or deprecated

Access

  • FreeAccess is true
  • User email is in recipients list
  • User email domain is in domains list
path Parameters
id
required
string

Public link system Id

Request Body schema: application/json
required
email
Array of strings

Email address of the user trying to access.

Responses

Request samples

Content type
application/json
{
  • "email": [
    ]
}

Response samples

Content type
application/json
{
  • "status": "string"
}

Connects a public user and generates a public link JWT

Description

Connects a public user by generating a JWT specific to this public link, with dedicated access filters. Issues also a cookie with refresh token scoped to the token renewal api.

Rules

  • Link is not DELETED or deprecated
  • OTP is not too old
  • OTP is associated to this email and link

Processing

  • Email and connection date are added to public link tracked sessions

Access

  • FreeAccess is true
  • User email is in recipients list
  • User email domain is in domains list
path Parameters
id
required
string

Public link system Id

Request Body schema: application/json
required
email
Array of strings

Email address of the user trying to access.

otp
Array of strings

One Time Password associated to this email.

Responses

Request samples

Content type
application/json
{
  • "email": [
    ],
  • "otp": [
    ]
}

Response samples

Content type
application/json
{
  • "token": "string"
}

Logs out a user from its public link sessions

Description

Logs out a user and clears the refresh token cookie

Access

  • The cookie with the refresh token is expected in input
path Parameters
id
required
string

Public link system Id

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Generate a new access token and refreshes the refresh token

Description

Rotate the access token and issue a new refresh token for a user

Responses

Response samples

Content type
application/json
{
  • "token": "string"
}

Sessions

Statistics from UI sessions

Search for Sessions

Description

Search for sessions. Sessions are confidential, searching them is much restricted.

Also available by GET method on the collection

Access

  • Admin
  • User with admin monitoring right asking for an aggregation with size:0
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a session

Description

Get a session

Access

  • User whose session it is
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Sessions system Id

Responses

Response samples

Content type
application/json
{
  • "@id": "06cb72b6-5a87-4baa-a3ca-620806a6aa7b",
  • "dateCreated": "2019-01-27T21:03:38.750Z",
  • "creator": "VBqPbgjYRsK00dhzdVeroQ",
  • "version": "0.1",
  • "dateModified": "2019-01-27T21:18:41.746Z",
  • "updater": "VBqPbgjYRsK00dhzdVeroQ",
  • "user": {
    },
  • "main": {
    },
  • "views": {
    },
  • "subViews": {
    },
  • "options": {
    },
  • "whisperers": {
    },
  • "actions": [
    ],
  • "actionsTotalCount": 11
}

Save a session

Description

Save a session

Access

  • User whose session it is
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Sessions system Id

Request Body schema: application/json
required

The session

@id
required
string

System id

required
object

Connected user

required
object
views
required
object

Statistics on the view used

subViews
object

Statistics on the subview used

options
object

Options selected

whisperers
object

List of selected whisperers

Array of objects

Statistics on the actions triggered by the user

Responses

Request samples

Content type
application/json
{
  • "@id": "06cb72b6-5a87-4baa-a3ca-620806a6aa7b",
  • "dateCreated": "2019-01-27T21:03:38.750Z",
  • "creator": "VBqPbgjYRsK00dhzdVeroQ",
  • "version": "0.1",
  • "dateModified": "2019-01-27T21:18:41.746Z",
  • "updater": "VBqPbgjYRsK00dhzdVeroQ",
  • "user": {
    },
  • "main": {
    },
  • "views": {
    },
  • "subViews": {
    },
  • "options": {
    },
  • "whisperers": {
    },
  • "actions": [
    ],
  • "actionsTotalCount": 11
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Jobs

Traces of upload, download and purge Jobs

Create a new Job

Description

Create a job

Process

  • If type is PurgeJob
    • Launches the purge on all selected resources
    • Check progress and update the job regularly until success or failure

Access

  • Admin
  • User creating a Job on its whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

The Job details

jobType
required
string
Enum: "PurgeJob" "DownloadJob" "UploadJob" "..."

Type of job

jobParameters
required
object

Input of the job

progress
required
integer

Progress in %

whisperer
required
Array of strings

List of whisperers

startTime
string <date-time>

Start of the job

endTime
string <date-time>

End of the job

updateTime
string <date-time>

Last update of the job

actionStatus
required
string
Enum: "ActiveActionStatus" "CompletedActionStatus" "FailedActionStatus" "PotentialActionStatus"

Status of the job

result
object

Result of the job

error
object

Error of the job, if any

Responses

Request samples

Content type
application/json
{
  • "jobType": "PurgeJob",
  • "jobParameters": { },
  • "progress": 0,
  • "whisperer": [
    ],
  • "startTime": "2019-08-24T14:15:22Z",
  • "endTime": "2019-08-24T14:15:22Z",
  • "updateTime": "2019-08-24T14:15:22Z",
  • "actionStatus": "ActiveActionStatus",
  • "result": { },
  • "error": { }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Jobs

Description

Search for jobs.

Also available by GET method on the collection

Rules

Admin and monitoring admin may search without specifying a whisperer

Access

  • Admin
  • Customer, limited on its whisperers
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a jobs details

Description

Get a jobs details.

Access

  • User linked to the whisperers in this job
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

Responses

Response samples

Content type
application/json
{
  • "@id": "Ju_4O7N9QvWLC8x7PswfnQ",
  • "@type": "Job",
  • "jobType": "DownloadJsonJob",
  • "creator": "wB0wdZgkTJqxcSZYv8yZ8g",
  • "creationTime": "2019-01-18T15:20:09.158Z",
  • "updateTime": "2019-01-18T15:20:09.159Z",
  • "endTime": "2019-01-18T15:20:09.106Z",
  • "actionStatus": "CompletedActionStatus",
  • "progress": 100,
  • "jobParameters": {
    },
  • "result": {
    },
  • "whisperer": [
    ]
}

Update job details

Description

Update a Job

Rules

Impossible to update the following fields:

  • jobType
  • creationTime
  • creator

Access

  • Admin
  • User that created the job and linked to all whisperers of the job
Authorizations:
Bearer
path Parameters
id
required
string

Customer internal @id

header Parameters
If-Match
required
string

eTag of previous state of the job

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

MailSender

Service to send emails

Send an email

Description

Sends an email from the configured account. Can be configured to be connected by OAuth (gmail for instance) or login/password.

Can send to one or several recipients, a text message or html. Txt message being mandatory.

As it is not intended for mass mailing, it makes a connection to the SMTP server for each email.

Access

  • Can only be call by applications or admin
Request Body schema: application/json
required

The email to send

to
required
Array of strings

List of recipients

subject
required
string

Subject of the mail. Accepts unicodes characters (for icons)

text
required
string

The plain message

html
string

The message in html

Responses

Request samples

Content type
application/json
{
  • "to": [
    ],
  • "subject": "string",
  • "text": "string",
  • "html": "string"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

GuiLogs

Logs of errors generated on GUI

Create a new Gui Log

Description

Create a Gui Log to trace an error happened on GUI

Access

  • No identification required, but if identified, it will be saved
Authorizations:
Bearer
Request Body schema: application/json
required

The Gui Log

app
required
string
Enum: "Login" "NetworkView" "SelfMonitoring"

Name of application

time
required
string <date-time>

Time of log

type
required
string
Enum: "BAD REQUEST" "SERVER ERROR" "UNEXPECTED" "TIMEOUT" "UNKNOWN" "XXX RENDER"

Type of error

name
required
string
Enum: "Saga Error" "React Error"

Name of the error

level
required
string
Enum: "ERROR" "WARNING"

Level of Log

message
required
string

Message of the error (from JS error.message)

stack
required
string

Stack of the error

details
string

Details of the error. Component stack in React.

whisperer
Array of strings

List of whisperers

customer
string

System id of the customer

timeout
integer

Timeout value in case of timeouts

object

Request that failed

object

Response that failed

Responses

Request samples

Content type
application/json
{
  • "app": "Login",
  • "time": "2019-08-24T14:15:22Z",
  • "type": "BAD REQUEST",
  • "name": "Saga Error",
  • "level": "ERROR",
  • "message": "string",
  • "stack": "string",
  • "details": "string",
  • "whisperer": [
    ],
  • "customer": "string",
  • "timeout": 0,
  • "req": {
    },
  • "res": {
    }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Gui Logs

Description

Search for Gui Logs.

Also available by GET method on the collection

Rules

Admin and monitoring admin may search without specifying a whisperer

Access

  • Admin
  • User with admin monitoring right asking for an aggregation with size:0
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

GuiSettings

User settings for GUI

Create / update GUI settings

Description

Stores the user settings. Mostly used by the UI itself

Access

  • Customer
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

The settings to save

@id
string

Id of the settings, and of the customer

required
object

GUI settings

Responses

Request samples

Content type
application/json
{
  • "@id": "string",
  • "settings": {
    }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Settings

Description

Search for settings.

Also available by GET method on the collection

Access

  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get GUI settings for a user

Description

Get settings

Access

  • Customer
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Settings Id == Id of user

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "GuiSettings",
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "version": "string",
  • "settings": {
    }
}

Plugins

Plugins store backend or proxy

Create / update a plugin

Description

Uploads a plugin.
Depending on global setting, the plugin is stored locally in ES or remotely in Floocus managed S3 service.

Access

  • User with plugins.upload permission
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

The plugin to store

@id
required
string

Id of the plugin, unique for the organisation

@type
required
string

Type of the plugin

@version
required
string

Version of the plugin

name
required
string

Human name of the plugin

description
string

Explains what the plugins does

source
required
string

Base64 encoded javascript source code of the plugin

Responses

Request samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "@version": "string",
  • "name": "string",
  • "description": "string",
  • "source": "string"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Plugins

Description

When global, searches for both own organisation plugins and 'Floocus' ones.
When local, searches for any plugin.

Also available by GET method on the collection

Access

  • Customer
Authorizations:
Bearer
Request Body schema: application/json
required

The plugin to store

type
string

Type of the plugin

Array of objects

Sorting option.

next
Array of strings

Ids of the last plugin fetched.

size
required
integer

Page size.

Responses

Request samples

Content type
application/json
{
  • "type": "string",
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get the manifest of a specific plugin

Description

Get plugin manifest

Access

  • Customer
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Plugin @id

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "GuiSettings",
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "version": "string",
  • "settings": {
    }
}

Get the code of a specific plugin

Description

Get the code

Access

  • Open to everybody
path Parameters
id
required
string

Plugin @id

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "GuiSettings",
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "version": "string",
  • "settings": {
    }
}

Monitor

Access monitoring metrics and logs from Spider

Search for monitoring information

Description

Searches or aggregation analysis on monitored metrics:

  • Processes stats
  • Circuit breakers stats
  • Pollers stats
  • Parsing queues stats
  • Elasticsearch indices stats
  • Elasticsearch nodes stats
  • Redis stats
  • Applicative logs stats

Also available by GET method on the collections

Access

  • Admin
  • User with admin monitoring right
Authorizations:
Bearer
path Parameters
collection
required
string
Enum: "processes" "circuitbreakers" "pollers" "parsing" "elasticsearch" "elasticsearchNodes" "redis" "logs"

Collection of metrics

Request Body schema: application/json
required

Search parameters

whisperers
Array of strings

List of whisperers to search on.

startTime
number <double>

Start unix timestamp of search window. Up to 6 decimals for microseconds.

stopTime
number <double>

Stop unix timestamp of search window. Up to 6 decimals for microseconds.

startDate
string <date-time>

Start date of search window (can replace startTime).

stopDate
string <date-time>

Stop date of search window (can replace stopTime).

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Ids of the last resource already fetched to get next ones.

size
integer

Page size.

withContent
boolean

True if you want to embed content in the result items (only for HTTP coms)

avoidTotalHits
boolean

Tells if response should include total hits count.

async
boolean

Ask for an async search (when supported).

asyncDelayMs
number <integer>

How long (in milliseconds) server should wait for an answer before answering with a partial answer (when async).

asyncId
string

Id of previous async answer from server (to get follow up). Included in hypermedia answer from server when async answer.

asyncKeepAliveS
number <integer>

How long (in seconds) the async answer is allowed to search before being killed.

Responses

Request samples

Content type
application/json
Example
{
  • "size": 20,
  • "whisperers": [
    ],
  • "startTime": 1547789298.676,
  • "stopTime": 1547805717.362,
  • "query": "!req.uri:contexts AND !req.uri:version AND !req.query:afterUpdate*",
  • "sort": [
    ]
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Pollers

Pollers statistics

Get poller queue status

Description

Give the queue status of this poller

Access

  • Admin
  • Application
  • Customer
Authorizations:
Bearer
path Parameters
service
required
string (PollerNames)
Enum: "pack-poller" "tcp-poller" "web-httpcom-poller" "web-httpcom-content-poller" "web-httppers-poller" "hosts-poller" "hosts-agg" "whisp-status-poller" "whisps-status-agg" "status-poller" "capture-status-poller" "ciphers-status-poller" "ciphers-raw-status-poller" "ciphers-status-agg" "parsing-status-tcpsession-poller" "parsing-status-httppers-poller"

Pollers service name

Responses

Response samples

Content type
application/json
{
  • "count": 182,
  • "first": "2019-01-30T22:35:51.254Z",
  • "last": "2019-01-30T22:36:01.641Z"
}

Whisperers

Create a new Whisperer

Description

Create a new whisperer, and associate it to the owner customer.

Access

  • Customer, with whisperers creation rights
  • User of the team owning the whisperer with whisperers right
  • User with userTrainer rights impersonating a trainee creating a whisperer for the trainee
  • Admin
Authorizations:
Bearer
Request Body schema: application/json
required

Whisperer creation request

customer
required
string

System Id of the customer.

name
required
string

Name of the whisperer to create.

Responses

Request samples

Content type
application/json
{
  • "customer": "YOD66VZ54Jih",
  • "name": "Upload"
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get default Configuration for Whisperers

Description

Get default configuration.

Access

  • User
  • Admin
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Search for Whisperers

Description

Search for whisperers

Also available by GET method on the collection

Rules

If client is not admin, the search will be limited to the Whisperers owned by this customer or shared with him.

If client is using public link,

  • Search is done on the whisperers from its token, not based on whisperers date.
  • users and teams are omitted in response

Access

  • Admin
  • Customer
Authorizations:
Bearer
Request Body schema: application/json
required

Search parameters

query
string

Free query, using Elasticsearch query string DSL.

aggs
object

Aggregation request, using Elasticsearch aggregation DSL.

Array of objects

Sorting option. Sorting by @id is automaticaly added.

next
Array of strings

Aggregation request, using Elasticsearch aggregation DSL.

size
required
integer

Page size.

avoidTotalHits
boolean

Tells if response should include total hits count.

includeETags
boolean

Tells if response should include _eTags fields for update.

Responses

Request samples

Content type
application/json
{
  • "query": "string",
  • "aggs": { },
  • "sort": [
    ],
  • "next": [
    ],
  • "size": 0,
  • "avoidTotalHits": true,
  • "includeETags": true
}

Response samples

Content type
application/json
{
  • "total": 0,
  • "items": [
    ],
  • "aggs": { },
  • "nextPage": {
    },
  • "asyncResults": {
    }
}

Get a Whisperer

Description

Get a whisperer's details.

Rules

Only admins can see DELETED whisperers.

Access

  • User owning the whisperer
  • The own whisperer
  • User being shared access to this whisperer
  • User of the team owning the whisperer with whisperers right
  • customers or links applications
  • A public link user having this Whisperer in the token
  • Admin

Response

Response varies depending on client:

  • Full resource for
    • User owning the whisperer
    • The own whisperer
    • User being shared access to this whisperer
    • User of the team owning the whisperer with whisperers right
    • User or Links applications
  • Full minus users and teams fields for public links users
  • Only id and name for all Customers
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Whisperer

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "Whisperer",
  • "version": "string",
  • "name": "string",
  • "customer": "string",
  • "team": "string",
  • "dns": "string",
  • "apikey": "string",
  • "config": {
    },
  • "users": [
    ],
  • "creator": "string",
  • "dateCreated": "2019-08-24",
  • "editor": "string",
  • "dateModified": "2019-08-24"
}

Change whisperer's name or shared users

Description

Updates a customer. You can:

  • Update whisperer's name
  • Change sharing settings: users and users rights on this whisperer

Rules

  • Can do any change:
    • Admin
    • User owning the whisperer
    • User of the team owning the whisperer with whisperers right
  • User being shared access to this whisperer with config rights can:
    • Change the name
  • User being shared access to this whisperer with share rights can:
    • Add or remove users from the sharing settings
  • User being shared access to this whisperer with rights change rights can:
    • Change rights of users in the sharing settings

Side effects

  • When adding or removing customers of shared list, the service updates the whisperers list in the customer resource (by calling Customer service).
  • When adding or removing teams of shared list, the service updates the whisperers list in the team resource (by calling Customer service).

Access

  • User owning the whisperer
  • User of the team owning the whisperer with whisperers right
  • User being shared access to this whisperer with config, share or rights modification rights
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Whisperer

header Parameters
If-Match
required
string

eTag of previous state of the whisperer

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Delete a Whisperer

Description

Delete a whisperer.

  • Put it to technicalStatus DELETED.

Access

  • User owning the whisperer
  • User being shared access to this whisperer and with delete right
  • User of the team owning the whisperer with whisperers right
  • Customer application
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

Internal id of Whisperer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create/Replace the whisperer API key

Description

Create/Replace the whisperer API key used for whisperer connection to Spider.

The API key is a public/private key pair.

  • The public key is stored in whisperer's settings.
  • The private key is sent in response to this call.

Any call to this API (if authorized) will overwrite the previous API key, and the whisperer will not be able to use the previous one. A connected whisperer will be disconnected when its current JWT token will expire. The API key is taken as a configuration AT START of whisperers, and need a restart to be changed.

Output

The API can supports two outputs:

  • application/json:
    • Provides a file with the private key, Spider's URL, and the whisperer's id This file is the only expected configuration file at whisperer start.
  • application/x-pem-file
    • Provides only the private key as a PEM file

Access

  • User owning the whisperer
  • User of the team owning the whisperer with whisperers right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

Responses

Response samples

Content type
{}

Generate an API key signature with the private key of the whisperer

Description

This endpoint is for testing purposes only:

  • It allows testing the API key or the whisperers API without a whisperer
  • It generates a valid signature for the whisperer to call the configuration endpoint
  • However, IT REQUIRES YOUR PRIVATE KEY IN INPUT
  • After testing, please, reset your API key

Output

  • The signature for this whisperer, timestamp and private API key
  • A validation of this signature with the whisperer registered public API key

Access

  • User owning the whisperer
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

query Parameters
timeStamp
required
string <date-time>

Timestamp to use in signature

instanceId
required
string

InstanceId to use in signature

Request Body schema: application/x-pem-file
string

The whisperer RSA private key, as a PEM

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Create the whisperer configuration

Description

Initialize the configuration of the whisperer

Access

  • User owning the whisperer and whisperer creation rights
  • User of the team owning the whisperer with whisperers right
  • User with userTrainer rights impersonating a trainee creating a whisperer for the trainee
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

Request Body schema: application/json
required
version
required
string
required
object

Settings used client side - the Sniffer

required
object

Settings used server side, for parsing

Responses

Request samples

Content type
application/json
{
  • "version": "0.1",
  • "client": {
    },
  • "server": {
    }
}

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get Whisperer configuration

Description

Get the configuration of this Whisperer Usages:

  • Called by whisperer at start (or before token expiration) with their API key
  • Called by ephemeral whisperer at start with the token given by the Controller
  • Called by whisperers regularly to check for configuration change
  • Called by UI to get a whisperer token to perform upload
  • Called by UI to get configuration
  • Called by services to get whisperer configuration when parsing

API key

The first call from whisperers is made by:

  • building a json payload with current time and whisperer id,
  • then signing it with SHA256 algorithm using whisperer's private key
const timeStamp = moment().toISOString();
const info = {
    timeStamp,
    whispererId,
    instanceId
};
const privKey = new NodeRSA(privatePem);
const signature = privKey.sign(Buffer.from(JSON.stringify(info)), 'base64');
  • and then calling this API with specific headers:
  Spider-TimeStamp: timeStamp
  Spider-InstanceId: instanceId,
  Spider-Signature: signature //base 64 encoded

Parameters

The view parameter allows to modulate the output:

server

  • Only server part (parsing) is sent
  • Current configuration is merged with default configuration

client

  • Only client part (parsing) is sent
  • Current configuration is merged with default configuration

full

  • Both client & server are sent
  • Current configuration is merged with default configuration

null

  • The raw configuration is sent (without merging with defaults)
  • This view is used by UI to know what settings are specific, and what settings are defaults

Output

  • The configuration
  • A JWT token to use on further calls in Spider-Token header
    • If no token provided, or if called from a Customer
    • A Customer may call to get the configuration of one of its whisperers and use the generated token to upload data (as on the UI)
    • If the Whisperer has a TTL associated to attachments, the token will have this as expiration instead of the default duration

Access

  • The own whisperer
  • User owning the whisperer
  • User being shared access to this whisperer
  • User of the team owning the whisperer with whisperers right
  • Controller asking for a token for a Whisperer to spawn on an Pod (InstanceId)
  • Admin
  • Application
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

query Parameters
view
string
Enum: "server" "client" "full"

Type of configuration to get

header Parameters
Spider-Signature
string <base64>

Signature of the call by the Whisperer, with its API key

Spider-Timestamp
string <date-time>

Provided with API key in first Whisperer call to get its JWT token with the config

Spider-InstanceId
string

Provided with API key in first Whisperer call to get its JWT token with the config

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "dateCreated": "2019-08-24",
  • "dateModified": "2019-08-24",
  • "editor": "string",
  • "creator": "string",
  • "client": {
    },
  • "server": {
    }
}

Change whisperer's config

Description

Updates a Whisperer configuration.

Rules

  • Can do any change:
    • Admin
    • User owning the whisperer
  • User being shared access to this whisperer with config rights can:
    • Change configuration settings
  • User being shared access to this whisperer with share rights can:
    • Change sharing settings
  • User being shared access to this whisperer with record rights can:
    • Start or stop capture

Access

  • User owning the whisperer
  • User being shared access to this whisperer with config, share, or record rights
  • User of the team owning the whisperer with whisperers right
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

header Parameters
If-Match
required
string

eTag of previous state of the whisperer's config

Request Body schema: application/json-patch+json
required

Json patch with the changes

Array
op
required
string
Enum: "test" "remove" "add" "replace" "move" "copy"
path
required
string
value
string
from
string

Responses

Request samples

Content type
application/json-patch+json
[
  • {
    }
]

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}

Get TLS configurations from a list of Whisperers

Description

From a list of Whisperers, provide their TLS configuration

Access

  • Gocipher
  • Admin requestBody: content: 'application/json': schema: description: 'The whisperer RSA private key, as a PEM' type: array items: type: string description: Internal Id of Whisperers minItems: 1
Authorizations:
Bearer

Responses

Response samples

Content type
application/json
{
  • "@id": "string",
  • "@type": "string",
  • "version": "string",
  • "whisperer": "string",
  • "requiredState": "string",
  • "tlsKeys": { }
}

Renew an ephemeral Whisperer token

Description

Called by ephemeral Whisperers (only) to renew their token (they do not have an API key).

If the Whisperer has a Time to Live associated to attachment, it won't renew.

Output

  • A JWT token to use on further calls in Spider-Token header
    • If no token provided, or if called from a Customer
    • A Customer may call to get the configuration of one of its whisperers and use the generated token to upload data (as on the UI)

Access

  • The own whisperer (only ephemerals can)
  • Admin
Authorizations:
Bearer
path Parameters
id
required
string

System id of Whisperer

Responses

Response samples

Content type
application/json
{
  • "@type": "Error",
  • "title": "Error title",
  • "message": "Error details"
}