Services

Services are specified as a collection named services in the config file. They can be one of the built in services, or provided by a third party plugin in.

A service can be runnable, observable, and/or possible to interact with different tasks. Each service has a set of capabilities implemented as interfaces.

services:
  myscheduler:
    type: scheduler
    interval: "00:01:00"
  ...

inmemoryqueue - Non durable in-memory message queue

Message queues are fundamental for Foopipes to operate. A default in-memory queue service with name queue is automatically created at startup if not specified in the configuration file.

services:
  queue:
    type: inmemoryqueue

Capabilities

IQueue, IObservableService

scheduler - Triggers events with intervals

Triggers an event with interval.

services:
  scheduler:
    type: scheduler
    interval: "00:01:00"

interval - Event interval in hh:mm:ss format

Example

services:
  scheduler:
    type: scheduler
    interval: "00:01:00"

pipelines:
  - when:
      - scheduler
    do:
      - { select, body: "Boing!!" }
      - { http: "http://myurl/tweet", method: post }

Capabilities

IObservableService

httplistener - Listen and react to http requests

Service for receiving http requests.

services:
  myListener: 
    type: httplistener
    path: mylistener

path - Url path for the listener. In this case http://localhost:5000/mylistener

responseBody - (optional) Format of the response sent to the client. auto, none, jsonObject, json, binary

contentType - (optional) Response ContentType

Route templates

The path argument may contain route templates which are accessible in the metadata collection and resulting document under the key route. {index=MyIndex}/{dataType=MyDataType}/{id?} will will match a URL path like /Products/Details/17 and extract the route values { index = Products, dataType = Details, id = 17 }. The route values are determined by splitting the URL path into segments, and matching each segment with the route parameter name in the route template. Route parameters are named. They are defined by enclosing the parameter name in braces { }.

Read more about route templates here: https://docs.microsoft.com/en-us/aspnet/core/fundamentals/routing#route-template-reference

    path: mylistener/{argname}

Data

The resulting document from the httplistener service looks like this:

 {
  "contentType": null,
  "query": {},
  "headers": {
    "Cache-Control": "max-age=0",
    "Connection": "keep-alive",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, sdch, br",
    "Accept-Language": "sv-SE,sv;q=0.8,en-US;q=0.6,en;q=0.4",
    "Host": "localhost:5000",
    "User-Agent": "Mozilla/5.0",
    "Upgrade-Insecure-Requests": "1"
  },
  "isHttps": false,
  "method": "GET",
  "path": {
    "Value": "/mylistener/hello",
    "HasValue": true
  },
  "body": null,
  "route": {
    "argname": "hello"
  }
}

Metadata

headers - http headers

query - The query string

route - The route template arguments

Example

services:
  web:
    type: httplistener
    path: mypath/{name}
    responseBody: jsonObject

pipelines:
  - when: 
      - web
    do:
      - log
      - { select, greeting: "Hello #{route.name}" }

Capabilities

IObservableService

udplistener - Listen and react to UDP packages

Service for receiving UDP packages.

services:
  myListener: 
    type: udplistener
    port: 8080

port - Port to listen to

format - Format of received UDP packages. json or binary (default)

Data

The received object is the data. If format is binary the data is the received binary data.

Metadata

remoteEndPoint - set to the origin IP address of the received UDP package.

Example

services: 
  udp: 
    type: udplistener
    port: 8080
    format: json

pipelines: 
  - 
    when: 
      - udp
    do:
      - { select, v: 1, t: event, tid: UA-xxxx-y, cid: "#{NodeId}", ec: "got_udp", ea: "#{EventType}", ev: "#{Count}", uip: "#{metadata:remoteEndPoint}" }
      - { http: "https://www.google-analytics.com/collect", method: post, body: formUrlEncoded }

Capabilities

IObservableService

file - Service for file storage

Service for file storage persistence. Automatically created with the name file if not specified in the configuration file. The file service can also be used as a simple key value store.

services:
  mycontent:
    type: file
    path: "/mycontent"
    createMissingFolders: true

path - Path to the folder relative to current directory

createMissingFolders - True if missing folder should be created automatically. Defaults to false.

Example

services:
  jsonstore:
     type: file
     path: .
     createMissingFolders: true
pipelines:
  - 
    when:
     - queue: started
    from:
      - { http: "https://jsonplaceholder.typicode.com/posts", method: get }
    to:
      - { store: jsonstore, index: jsonplaceholder, dataType: post, key: "#{id}" }

Capabilities

IFileBinding, IDataStorage, IKeyValueStorage, IEnumerableDataStorage

http - Http service for sending http requests

Service for sending Http requests. Automatically created with the name http if not specified in the configuration file.

  http:
    type: http
    timeout: "00:00:05"

timeout - Request timeout hh:mm:ss

Capabilities

IHttpBinding

Plugin service types

elasticsearch - Load/store/search Elasticsearch database

Elasticsearch is a service provided by the Elasticsearch plugin.

services:
  elasticsearch: 
    type: elasticsearch
    url: "http://${elasticsearch|localhost}:9200"
    termMappings:
        - { index: myindex, dataType: mytype, field: "fields.url" }
        - { index: myindex, dataType: mytype, field: "fields.parent.id" }
        - { index: myindex, dataType: mytype, field: "contentType.id" }

url - The url to Elasticsearch

termMappings - An optional array of term mappings which are created on startup.

Capabilities

IKeyValueStorage, ISearchableDataStorage, IDataStorage

ravendb - Load/store RavenDB database

RavenDB is a service provided by the RavenDB plugin.

services:
  ravendb: 
    type: ravendb 
    connectionString: "Url=http://${ravendb|localhost}:8080/;Database=mydb"
    changesForDocumentsInCollection:
      - product
      - stock

connectionString - Connection string to RavenDB server

changesForDocumentsInCollection - (optional) Subscribe for changes for documents in collection (array)

changesForDocumentsOfType - (optional) Subscribe for changes for documents of type (array)

changesForDocumentsStartingWith - (optional) Subscribe for changes for documents starting with (array)

changesForDocument - (optional) Subscribe for changes for document with id (array)

changesForIndex - (optional) Subscribe for changes for index with name (array)

changesForAllDocuments - (optional) Subscribe for changes for all documents true/false.

See also https://ravendb.net/docs/article-page/3.5/csharp/glossary/document-change-notification

Capabilities

IKeyValueStorage, IDataStorage, IObservableService

Data

{
  "CollectionName": "post",
  "Etag": {
    "Restarts": 72057594037927942,
    "Changes": 5
  },
  "Id": "98",
  "Message": null,
  "Type": 1
}

Example

services:
  ravendb:
    type: ravendb
    connectionString: "Url=http://localhost:8080/;Database=mydb"
    changesForDocumentsInCollection:
      - post

pipelines: 
  - 
    when: 
      - scheduler
    from:
      - { http: "https://jsonplaceholder.typicode.com/posts", method: get }
    to:
      - { store: ravendb, dataType: post, key: "#{id}" }
  -
    when:

   - ravendb: Put
    from:
      - { load: ravendb, dataType: "#{CollectionName}", key: "#{Id}" }
    to:
      - log
  -
    when:
      - ravendb: Delete
    to:
      - log: "Document #{Id} deleted"

rabbitmq - External RabbitMQ message queue

Todo