12. FIWARE - Evaluation

Starting in october 2015 the SOSPilot platform was extended to use and deploy FIWARE.

12.1. The Plan

The architecture used for this setup is depicted below. This architecture emerged dynamically as described below.

This setup is similar as in this FIWARE presentation.

_images/sospilot-fiware-arch1.jpg

Architecture for FIWARE within SOSPilot platform

From bottom to top the setup is as follows:

  • Sensors or manual input use the UltraLight 2.0 (UL2.0) and MQTT protocols for managing services and devices and sending observations
  • The FIWARE IoTAgentCpp device API is used
  • Client libraries (Python, Arduino) are used to facilitate using the UL2.0 and MQTT protocols
  • The client may reside anywhere on the Internet
  • the server sensors.geonovum.nl hosts the FIWARE components Orion Context Broker (Orion CB or OCB), IoTAgentCpp and Short Term History (STH)
  • all persistence for these components is done in MongoDB
  • IoTAgentCpp translates requests from the UL2.0/MQTT clients to Orion CB NGSI requests
  • OCB persists the latest values as Entities in MongoDB
  • Short Term History (STH aka STH Comet) subscribes to Orion CB NGSI Entity attribute change events and stores Timeseries in MongoDB
  • WireCloud (WC), the Mashup environment runs in the FIWARE Lab server at lab.fiware.org
  • WC communicates to (any) OCB using the NGSI10 protocol
  • within WC mashups are produced in order to view and interact with the sensor data
  • by developing an OpenLayers NGSI10 Vector Layer, viewers like HeronViewer can show (real-time) sensor data

NGSI10 is a specification from the Open Mobile Alliance (OMA). The FI-WARE version of the OMA NGSI 10 interface is a RESTful API via HTTP.

STH provides Timeseries storage and an API to request Timeseries data. See https://github.com/telefonicaid/fiware-sth-comet.

The above setup provides APIs for a complete IoT platform:

For data producers (sensors):

  • APIs for sensor-provisioning (via UL2.0)
  • APIs (UL2.0, MQTT) for sensors to send observations

For data consumers:

  • API to request latest values (NGSI10 Query API)
  • API to subscribe to real-time updates (NGSI10 Publish-Subscribe API)
  • API to request historic timeseries data (STH API)

12.2. FIWARE Docs

This document provides most of the details of the above components: http://fiware-iot-stack.readthedocs.org/en/latest/index.html

12.3. Via lab.fiware.org

Initially it was attempted to realize the above architecture via the public FIWARE Lab platform but we found some isssues. In a later stage the above setup was realized within the Geonovum (Ubuntu/Linux) server VPS. Some info follows:

Registering at the international FIWARE Lab worked ok, but could not get the Orion CB with the IDAS IoTAgent working. The interaction between the two seemed to be broken. This was reported, see specifics here on StackOverflow:

And appearantly this issue was found by others as well.

After an unsuccessful attempt to compile and run the OCB and IoT Agent on Ubuntu 14.04-3 it was decided to use Docker on the Geonovum SOSPilot VPS (Ubuntu/Linux server VPS). Via Docker seems the best/recommended option anyway as CentOS is the primary target platform for FIWARE. See next sections.

12.4. Installing FIWARE - with Docker

See http://www.slideshare.net/dmoranj/iot-agents-introduction

12.4.1. Install Docker

See https://docs.docker.com/installation/ubuntulinux. Install via Docker APT repo.

Steps.

# Kernel version OK for Docker
$ uname -r
3.13.0-66-generic

# Add key
apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D

# Add to repo by putting this line in /etc/apt/sources.list.d/docker.list
add deb https://apt.dockerproject.org/repo ubuntu-trusty main to

$ apt-get update
$ apt-cache policy docker-engine

# install docker engine
apt-get update
apt-get install docker-engine

# test
docker run hello-world
docker run -it ubuntu bash

# cleanup non-running images
docker rm -v $(docker ps -a -q -f status=exited)
docker rmi $(docker images -f "dangling=true" -q)

Docker-compose. https://docs.docker.com/compose/install. Easiest via pip.

$ pip install docker-compose

See also CLI utils for docker-compose: https://docs.docker.com/v1.5/compose/cli/

Docker utils.

docker ps -a

# go into docker image named docker_iotacpp_1 to bash prompt
docker exec -it docker_iotacpp_1 bash

12.4.2. Install FIWARE

Installing the FIWARE Docker components to realize IoT setup: IoT Agent, Orion CB, Short term History (STH) with MongoDB persistence. Intro: http://www.slideshare.net/dmoranj/iot-agents-introduction

Take docker-compose for fiware-IoTAgent-Cplusplus as starting point: https://github.com/telefonicaid/fiware-IoTAgent-Cplusplus/tree/develop/docker. All our local development related to Docker can be found here: https://github.com/Geonovum/sospilot/tree/master/src/fiware/docker .

Steps. Follow: https://github.com/telefonicaid/fiware-IoTAgent-Cplusplus/blob/develop/docker/readme.md

mkdir -p /opt/fiware/iotagent
cd /opt/fiware/iotagent
git clone https://github.com/telefonicaid/fiware-IoTAgent-Cplusplus iotacpp

#
cd /opt/fiware/iotagent/iotacpp/docker

Networking from outside to docker containers. See http://blog.oddbit.com/2014/08/11/four-ways-to-connect-a-docker. Make two utilities, docker-pid and docker-ip in /opt/bin.

#!/bin/sh

exec docker inspect --format '{{ .State.Pid }}' "$@"

#!/bin/sh

exec docker inspect --format '{{ .NetworkSettings.IPAddress }}' "$@"

But simpler is to follow: https://docs.docker.com/userguide/dockerlinks/ and even easier via docker-compose iota.yaml: https://docs.docker.com/compose/yml. Use the ports property: “Expose ports. Either specify both ports (HOST:CONTAINER), or just the container port (a random host port will be chosen).” So our iotarush.yml (derived from iota.yaml) becomes:

#mongodbdata:
#    image: mongo:2.6
#    volumes:
#        - /data/db
#    restart: "no"
#    command: /bin/echo "Data-only container for mongodb."

#mongodb:
#    image: mongo:2.6
#    volumes_from:
#        - mongodbdata
#    expose:
#        - "27017"
#    command: --smallfiles

mongodb:
    image: mongo:2.6

    # Port mapping: allow only access on local docker host
    ports:
      - "127.0.0.1:27017:27017"
    expose:
      - "27017"

    # Use storage on host file system
    volumes:
      - /var/lib/mongodb_docker:/data/db
    command: --smallfiles

    # restart: always

orion:
    image: geonovum/orionrush
    links:
        - mongodb
    ports:
        - "1026:1026"

sthcomet:
    image: geonovum/sthcomet
    log_driver: "syslog"
    links:
        - mongodb
        - orion
    ports:
        - "8666:8666"

iotacpp:
    image: telefonicaiot/iotacpp
    links:
        - mongodb
        - orion
    ports:
        - "185.21.189.59:8000:8080"
        - "185.21.189.59:8081:8081"
        - "185.21.189.59:1883:1883"

Now start.

# Start containers (-d iotacpp option not required?)
$ docker-compose -f iotarush.yaml up

# Stopping
$ docker-compose -f iotarush.yaml stop

# check
$ docker images
REPOSITORY              TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
geonovum/orionrush      latest              d097edbfaa4d        2 days ago          535.8 MB
geonovum/sthcomet       latest              778b09dbecc9        3 days ago          686.4 MB
fiware/orion            latest              a5f228ae72c3        5 weeks ago         277.1 MB
telefonicaiot/iotacpp   latest              583fbe68b08e        5 weeks ago         2.092 GB
mongo                   2.6                 dd4b3c1d1e51        5 weeks ago         392.8 MB

$ docker ps
CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                                                                                      NAMES
d3e35f677462        telefonicaiot/iotacpp   "/docker-entrypoint.s"   47 hours ago        Up About an hour    185.21.189.59:1883->1883/tcp, 185.21.189.59:8081->8081/tcp, 185.21.189.59:8000->8080/tcp   docker_iotacpp_1
d60d467d4e18        geonovum/sthcomet       "npm start"              47 hours ago        Up About an hour    0.0.0.0:8666->8666/tcp                                                                     docker_sthcomet_1
425dd1179b71        geonovum/orionrush      "/docker-entrypoint.s"   47 hours ago        Up About an hour    0.0.0.0:1026->1026/tcp                                                                     docker_orion_1
ad755ed4d28f        mongo:2.6               "/entrypoint.sh --sma"   47 hours ago        Up About an hour    27017/tcp                                                                                  docker_mongodb_1

# get into a container with bash
docker exec -it docker_orion_1 bash
[root@1c9dceec8ec8 /]# ps -elf
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN  STIME TTY          TIME CMD
4 S root         1     0  0  80   0 - 50812 hrtime 15:19 ?        00:00:00 /usr/bin/contextBroker -fg -multiservice
4 S root       976     0  0  80   0 -  3374 wait   15:35 ?        00:00:00 bash
0 R root      1021   976  0  80   0 -  3846 -      15:36 ?        00:00:00 ps -elf

[root@1c9dceec8ec8 /]# cat /etc/hosts
172.17.0.41     1c9dceec8ec8
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.40     mongodb 41028cff906b docker_mongodb_1
172.17.0.40     mongodb_1 41028cff906b docker_mongodb_1
172.17.0.40     docker_mongodb_1 41028cff906b
172.17.0.40     docker_mongodb_1.bridge
172.17.0.41     docker_orion_1.bridge
172.17.0.41     docker_orion_1
172.17.0.40     docker_mongodb_1
172.17.0.42     docker_iotacpp_1
172.17.0.42     docker_iotacpp_1.bridge

# Check Orion
$ curl 172.17.0.41:1026/statistics
<orion>
        <versionRequests>0</versionRequests>
        <statisticsRequests>1</statisticsRequests>
        <uptime_in_secs>1472</uptime_in_secs>
        <measuring_interval_in_secs>1472</measuring_interval_in_secs>
        <subCacheRefreshs>3</subCacheRefreshs>
        <subCacheInserts>0</subCacheInserts>
        <subCacheRemoves>0</subCacheRemoves>
        <subCacheUpdates>0</subCacheUpdates>
        <subCacheItems>0</subCacheItems>
</orion>

# Check iot agent iotacpp
$ docker exec -it docker_iotacpp_1 bash
[root@8e317c6b9405 /]# ps -elf
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN  STIME TTY          TIME CMD
4 S root         1     0  0  80   0 -  4821 poll_s 15:49 ?        00:00:00 /sbin/init
1 S root        74     1  0  80   0 -  2673 poll_s 15:49 ?        00:00:00 /sbin/udevd -d
5 S iotagent   292     1  0  80   0 - 12087 poll_s 15:49 ?        00:00:00 /usr/sbin/mosquitto -d -c /etc/iot/mosquitto.conf
0 S iotagent   312     1  0  80   0 - 186499 futex_ 15:49 ?       00:00:00 /usr/local/iot/bin/iotagent -n IoTPlatform -v ERROR -i 0.0.0.0 -p 8080 -d /usr/local/iot/lib -c /etc/iot/config.json
0 S root       365     1  0  80   0 -  1028 hrtime 15:50 ?        00:00:00 /sbin/mingetty /dev/tty[1-6]
4 S root       366     0  1  80   0 - 27087 wait   15:50 ?        00:00:00 bash
0 R root       378   366  0  80   0 - 27557 -      15:50 ?        00:00:00 ps -elf
[root@8e317c6b9405 /]# curl -g -X GET http://127.0.0.1:8080/iot/about
Welcome to IoTAgents  identifier:IoTPlatform:8080  1.3.0 commit 128.g14ee433 in Oct 28 2015
[root@8e317c6b9405 /]#

# and from outside
curl -g -X GET http://sensors.geonovum.nl:8000/iot/about
Welcome to IoTAgents  identifier:IoTPlatform:8080  1.3.0 commit 128.g14ee433 in Oct 28 2015

# Get log output
# See https://docs.docker.com/v1.5/compose/cli
$ docker-compose -f iotarush.yaml logs

Finally a custom Dockerfile was created for OCB to include Rush (and Redis) as an extension of the fiware/orion Docker file to support HTTPS notifications for OCB subscriptions (see WireCloud below). This is the content of the geonovum/orionrush Dockerfile:

FROM fiware/orion
MAINTAINER Just van den Broecke for Geonovum  www.geonovum.nl

ENV MONGODB_HOST mongodb
ENV REDIS_VERSION 2.8.3
ENV RUSH_VERSION 1.8.3
ENV RUSH_LISTEN_PORT 5001

WORKDIR /opt

RUN \
	# Dependencies
	yum -y install npm  && \
	# yum -y install redis   (gave errors see below)
	yum -y install wget  && \
	wget http://download.redis.io/releases/redis-${REDIS_VERSION}.tar.gz  && \
	tar xzvf redis-${REDIS_VERSION}.tar.gz  && \
	cd redis-${REDIS_VERSION} && \
	make && \
	make install && \
	# /etc/init.d/redis start  (no daemon installed)
	# Rush
	cd /opt && \
	curl https://codeload.github.com/telefonicaid/Rush/tar.gz/${RUSH_VERSION} > Rush-${RUSH_VERSION}.tar.gz  && \
	tar xzvf Rush-${RUSH_VERSION}.tar.gz  && \
	cd Rush-${RUSH_VERSION} && \
	npm install --production
	# set mongodb host as linked in docker-compose iota.yml
	# export RUSH_GEN_MONGO=${MONGODB_HOST}
	# bin/listener
	# bin/consumer

WORKDIR /

COPY docker-entrypoint.sh /
COPY server.key /
COPY server.crt /
COPY server.csr /

RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]

EXPOSE 1026

The entry point was changed to :

#!/bin/bash

REDIS_HOME=/opt/redis-${REDIS_VERSION}
RUSH_HOME=/opt/Rush-${RUSH_VERSION}

cd ${REDIS_HOME}
redis-server redis.conf > /var/log/redis.log 2>&1 &

cd ${RUSH_HOME}
export RUSH_GEN_MONGO=${MONGODB_HOST}
mv /server.key /server.crt /server.csr ${RUSH_HOME}/utils

bin/listener > /var/log/rush-listener.log 2>&1 &
bin/consumer > /var/log/rush-consumer.log 2>&1 &

/usr/bin/contextBroker -dbhost ${MONGODB_HOST} -fg -multiservice -rush localhost:5001 -logDir /var/log/contextBroker

To run the entire Fiware suite with IoTAgentCpp, OrionRush, STH and MongoDB, a new docker-compose file was created, iotarush.yaml :

#mongodbdata:
#    image: mongo:2.6
#    volumes:
#        - /data/db
#    restart: "no"
#    command: /bin/echo "Data-only container for mongodb."

#mongodb:
#    image: mongo:2.6
#    volumes_from:
#        - mongodbdata
#    expose:
#        - "27017"
#    command: --smallfiles

mongodb:
    image: mongo:2.6

    # Port mapping: allow only access on local docker host
    ports:
      - "127.0.0.1:27017:27017"
    expose:
      - "27017"

    # Use storage on host file system
    volumes:
      - /var/lib/mongodb_docker:/data/db
    command: --smallfiles

    # restart: always

orion:
    image: geonovum/orionrush
    links:
        - mongodb
    ports:
        - "1026:1026"

sthcomet:
    image: geonovum/sthcomet
    log_driver: "syslog"
    links:
        - mongodb
        - orion
    ports:
        - "8666:8666"

iotacpp:
    image: telefonicaiot/iotacpp
    links:
        - mongodb
        - orion
    ports:
        - "185.21.189.59:8000:8080"
        - "185.21.189.59:8081:8081"
        - "185.21.189.59:1883:1883"

To start/stop all fresh the following init.d script was created:

#!/bin/sh
#
# init.d script with LSB support for entire FIWARE stack via Docker.
#
# Copyright (c) 2015 Just van den Broecke for Geonovum - <just@justobjects.nl>
#
# This is free software; you may redistribute it and/or modify
# it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2,
# or (at your option) any later version.
#
# This is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License with
# the Debian operating system, in /usr/share/common-licenses/GPL;  if
# not, write to the Free Software Foundation, Inc., 59 Temple Place,
# Suite 330, Boston, MA 02111-1307 USA
#
### BEGIN INIT INFO
# Provides:          iotarush
# Required-Start:    $network $local_fs $remote_fs
# Required-Stop:     $network $local_fs $remote_fs
# Should-Start:      $named
# Should-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: IoT stack for FIWARE: IotAgent, Orion with MongoDB and Rush/Redis via Docker(-compose)
# Description:       Manages run cycle for a set of Docker containers together providing a FIWARE IoT stack.
### END INIT INFO

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

NAME=iotarush
DESC=docker_composition
# Docker-compose file
CONF=/opt/geonovum/sospilot/git/src/fiware/docker/iotarush.yaml
SERVICE_CONTAINERS="docker_iotacpp_1 docker_orion_1 docker_sthcomet_1 docker_mongodb_1"

. /lib/lsb/init-functions

set -e

purge_services() {
	docker rm -v ${SERVICE_CONTAINERS}
    return 0
}

purge_all() {
	# Removes all containers including volume-data for mongodb!
	docker rm -v $(docker ps -a -q -f status=exited)
    return 0
}

start_containers() {
	docker-compose -f ${CONF} up -d
	docker ps
    return 0
}

stop_containers() {
	docker-compose -f ${CONF} stop
	docker ps
    return 0
}

case "$1" in
  start)
	log_daemon_msg "Starting $DESC" "$NAME"
	start_containers
	;;

  stop)
	 log_daemon_msg "Stopping $DESC" "$NAME"
	 stop_containers
	 purge_all
	 ;;
	 
  restart)
        log_daemon_msg "Restarting $DESC" "$NAME"
		$0 stop
		$0 start
	;;
	
  status)
        log_daemon_msg "Checking status of $DESC" "$NAME"
        docker ps
        ;;
        
  reinit)
        log_warning_msg "Reinit of $NAME ; all mongodb data will be cleared!!!"
		stop_containers
		purge_all
		/bin/rm -rf /var/lib/mongodb_docker/*
		$0 start
        ;;

  *)
	N=/etc/init.d/$NAME
	echo "Usage: $N {start|stop|restart|reinit|status}" >&2
	exit 1
	;;
esac

exit 0

Activating our iotarush init.d script and showing commands

$ cp iotarush /etc/init.d
$ update-rc.d iotarush  defaults

# Start
service iotarush start

# Status
service iotarush status

# Stop without loosing data
service iotarush stop

# Restart without loosing data
service iotarush restart

# Restart with purging all (mongodb) data
service iotarush reinit

12.5. Testing the FIWARE Installation

Using test clients at See test clients at https://github.com/Geonovum/sospilot/tree/master/src/fiware/client and the WireCloud Mashup.

12.5.1. Testing IoTAgent-Orion

Simple scenario, using the UltraLight (UL2.0) IoT protocol.:

  1. create IoT service via IoTAgent (IDAS)
  2. create IoT device via IoTAgent (IDAS)
  3. observe new Entity in Orion
  4. send temperature via IDAS
  5. observe changed Entity via Orion

See test clients at https://github.com/Geonovum/sospilot/tree/master/src/fiware/client/python/UL20

See tutorial at: http://www.slideshare.net/FI-WARE/fiware-iotidasintroul20v2. Documentation https://fiware-orion.readthedocs.org/en/develop (OCB).

Sending commands from local system using FIWARE FIGWAY (python-IDAS4): https://github.com/Geonovum/sospilot/tree/master/src/fiware/client/python/UL20. These are a set of Python commands for most common REST services for both the OCB and IoTAgent/Manager.

Prepare the right config.ini used by all Python commands:

[user]
# Please, configure here your username at FIWARE Cloud and a valid Oauth2.0 TOKEN for your user (you can use get_token.py to obtain a valid TOKEN). 
username=<your username>
token=<your token>

[contextbroker]
# host=130.206.80.40
host=sensors.geonovum.nl
port=1026
OAuth=no
# Here you need to specify the ContextBroker database you are querying. 
# Leave it blank if you want the general database or the IDAS service if you are looking for IoT devices connected by you. 
fiware_service=fiwareiot
# fiware_service=bus_auto
fiware-service-path=/

[idas]
host=sensors.geonovum.nl
# host=130.206.80.40
#adminport=5371
#ul20port=5371
adminport=443
ul20port=443
OAuth=no
# Here you need to configure the IDAS service your devices will be sending data to. 
# By default the OpenIoT service is provided. 
#
fiware-service=fiwareiot
# fiware-service=workshop
# fiware-service=bus_auto
fiware-service-path=/
#apikey=4jggokgpepnvsb2uv4s40d59ov
apikey=4jggokgpepnvsb2uv4s40d59ov

[local]
#Choose here your System type. Examples: RaspberryPI, MACOSX, Linux, ...
host_type=MACOSX
# Here please add a unique identifier for you. Suggestion: the 3 lower hexa bytes of your Ethernet MAC. E.g. 79:ed:af
# Also you may use your e-mail address.
host_id=a0:11:00

Create device template (called GEONOVUM_TEMP) under https://github.com/telefonicaid/fiware-figway/tree/master/python-IDAS4/Sensors_UL20/devices :

	{
	 "devices": [
	    { "device_id": "DEV_ID",
	      "entity_name": "ENTITY_ID",
	      "entity_type": "thing",
	      "protocol": "PDI-IoTA-UltraLight",
	      "timezone": "Europe/Amsterdam",
	"attributes": [
	        {
	          "object_id": "temp",
	          "name": "temperature",
	          "type": "int"
	        },
            {
	        "object_id": "pos",
            "name": "position",
            "type": "coords",
            "metadatas": [
                {
                    "name": "location",
                    "type": "string",
                    "value": "WGS84"
                }
              ]
            }
	      ],
	 "static_attributes": [
	        { "name": "organization",
	          "type": "string",
	          "value": "Geonovum"
	        }
	       ]
	      }
	     ]
	    }

Create service, then a device and send an observation using Python code under https://github.com/Geonovum/sospilot/tree/master/src/fiware/client/python/UL20 (IoTAgent with UL protocol) and https://github.com/Geonovum/sospilot/tree/master/src/fiware/client/python/ContextBroker (OCB).

Watch that the related Entity is created in the OCB and that it gets an attribute value when sending an Observation to the IoTAgent.

    # IoTAgent: List devices (none present)
    python ListDevices.py
    * Asking to http://sensors.geonovum.nl:8000/iot/devices
    * Headers: {'Fiware-Service': 'fiwareiot', 'content-type': 'application/json', 'Fiware-ServicePath': '/', 'X-Auth-Token': 'NULL'}
    ...

    * Status Code: 200
    * Response:
    { "count": 0,"devices": []}

    # OCB: Show Entities ot OCB (none present)
    $ python GetEntities.py ALL
    * Asking to http://sensors.geonovum.nl:1026/ngsi10/queryContext
    * Headers: {'Fiware-Service': 'fiwareiot', 'content-type': 'application/json', 'accept': 'application/json', 'X-Auth-Token': 'NULL'}
    * Sending PAYLOAD:
    {
        "entities": [
            {
                "type": "",
                "id": ".*",
                "isPattern": "true"
            }
        ],
        "attributes": []
    }

    # IoTAgent: Create an IoT service
    $ python CreateService.py fiwareiot 4jggokgpepnvsb2uv4s40d59ov 185.21.189.59 1026
    * Asking to http://sensors.geonovum.nl:8000/iot/services
    * Headers: {'Fiware-Service': 'fiwareiot', 'content-type': 'application/json', 'Fiware-ServicePath': '/', 'X-Auth-Token': 'NULL'}
    * Sending PAYLOAD:
    {
        "services": [
            {
                "token": "token2",
                "apikey": "4jggokgpepnvsb2uv4s40d59ov",
                "resource": "/iot/d",
                "entity_type": "thing",
                "cbroker": "http://185.21.189.59:1026"
            }
        ]
    }

    # IoTAgent: Register a Device passing related Entity name
    $ python RegisterDevice.py OTTERLO_TEMP NexusProDev WeatherOtterloEnt
    * opening: ./devices/OTTERLO_TEMP
    * Asking to http://sensors.geonovum.nl:8000/iot/devices
    * Headers: {'Fiware-Service': 'fiwareiot', 'content-type': 'application/json', 'Fiware-ServicePath': '/', 'X-Auth-Token': 'NULL'}
    * Sending PAYLOAD:
    {
        "devices": [
            {
                "protocol": "PDI-IoTA-UltraLight",
                "entity_name": "WeatherOtterloEnt",
                "entity_type": "thing",
                "static_attributes": [
                    {
                        "type": "string",
                        "name": "location",
                        "value": "BosHut"
                    }
                ],
                "timezone": "Europe/Amsterdam",
                "attributes": [
                    {
                        "type": "int",
                        "name": "temperature",
                        "object_id": "ot"
                    }
                ],
                "device_id": "NexusProDev"
            }
        ]
    }

    ...

    * Status Code: 201

# IoTAgent: List the newly added device
    $ python ListDevices.py
    * Asking to http://sensors.geonovum.nl:8000/iot/devices
    * Headers: {'Fiware-Service': 'fiwareiot', 'content-type': 'application/json', 'Fiware-ServicePath': '/', 'X-Auth-Token': 'NULL'}
    ...

    * Status Code: 200
    * Response:
    { "count": 1,"devices": [{ "device_id" : "NexusProDev" }]}

    # OCB: Show related Entity in OCB
    $ python GetEntities.py ALL
    * Asking to http://sensors.geonovum.nl:1026/ngsi10/queryContext
    * Headers: {'Fiware-Service': 'fiwareiot', 'content-type': 'application/json', 'accept': 'application/json', 'X-Auth-Token': 'NULL'}
    * Sending PAYLOAD:
    {
        "entities": [
            {
                "type": "",
                "id": ".*",
                "isPattern": "true"
            }
        ],
        "attributes": []
    }

    ...

    * Status Code: 200
    ***** Number of Entity Types: 1

    ***** List of Entity Types
    <entityId type="thing" isPattern="false"> : 1

    **** Number of Entity IDs: 1

    **** List of Entity IDs
    <id>WeatherOtterloEnt< : 1

    Do you want me to print all Entities? (yes/no)yes
    <queryContextResponse>
      <contextResponseList>
        <contextElementResponse>
          <contextElement>
            <entityId type="thing" isPattern="false">
              <id>WeatherOtterloEnt</id>
            </entityId>
            <contextAttributeList>
              <contextAttribute>
                <name>TimeInstant</name>
                <type>ISO8601</type>
                <contextValue>2015-10-30T20:21:17.557970</contextValue>
              </contextAttribute>
              <contextAttribute>
                <name>location</name>
                <type>string</type>
                <contextValue>BosHut</contextValue>
                <metadata>
                  <contextMetadata>
                    <name>TimeInstant</name>
                    <type>ISO8601</type>
                    <value>2015-10-30T20:21:17.558093</value>
                  </contextMetadata>
                </metadata>
              </contextAttribute>
            </contextAttributeList>
          </contextElement>
          <statusCode>
            <code>200</code>
            <reasonPhrase>OK</reasonPhrase>
          </statusCode>
        </contextElementResponse>
      </contextResponseList>
    </queryContextResponse>

    # IoTAgent: Send an Observation to device
    $ python SendObservation.py NexusProDev 'ot|16'
    * Asking to http://sensors.geonovum.nl:8000/iot/d?k=4jggokgpepnvsb2uv4s40d59ov&i=NexusProDev
    * Headers: {'Fiware-Service': 'fiwareiot', 'content-type': 'application/json', 'Fiware-ServicePath': '/', 'X-Auth-Token': 'NULL'}
    * Sending PAYLOAD:
    ot|16

    ...

    * Status Code: 200
    * Response:

    # OCB: See value in Entity
python GetEntities.py ALL
* Asking to http://sensors.geonovum.nl:1026/ngsi10/queryContext
* Headers: {'Fiware-Service': 'fiwareiot', 'content-type': 'application/json', 'accept': 'application/json', 'X-Auth-Token': 'NULL'}
* Sending PAYLOAD:
{
    "entities": [
        {
            "type": "",
            "id": ".*",
            "isPattern": "true"
        }
    ],
    "attributes": []
}

...

* Status Code: 200
***** Number of Entity Types: 1

***** List of Entity Types
<entityId type="thing" isPattern="false"> : 1

**** Number of Entity IDs: 1

**** List of Entity IDs
<id>WeatherOtterloEnt< : 1

Do you want me to print all Entities? (yes/no)yes
<queryContextResponse>
  <contextResponseList>
    <contextElementResponse>
      <contextElement>
        <entityId type="thing" isPattern="false">
          <id>WeatherOtterloEnt</id>
        </entityId>
        <contextAttributeList>
          <contextAttribute>
            <name>TimeInstant</name>
            <type>ISO8601</type>
            <contextValue>2015-10-30T21:04:13.770532</contextValue>
          </contextAttribute>
          <contextAttribute>
            <name>location</name>
            <type>string</type>
            <contextValue>BosHut</contextValue>
            <metadata>
              <contextMetadata>
                <name>TimeInstant</name>
                <type>ISO8601</type>
                <value>2015-10-30T21:04:13.770563</value>
              </contextMetadata>
            </metadata>
          </contextAttribute>
          <contextAttribute>
            <name>temperature</name>
            <type>int</type>
            <contextValue>16</contextValue>
            <metadata>
              <contextMetadata>
                <name>TimeInstant</name>
                <type>ISO8601</type>
                <value>2015-10-30T21:04:13.770532</value>
              </contextMetadata>
            </metadata>
          </contextAttribute>
        </contextAttributeList>
      </contextElement>
      <statusCode>
        <code>200</code>
        <reasonPhrase>OK</reasonPhrase>
      </statusCode>
    </contextElementResponse>
  </contextResponseList>
</queryContextResponse>

# Get Context Types, note: --header 'Fiware-Service: fiwareiot' needs to be present!!
    $ curl sensors.geonovum.nl:1026/v1/contextTypes -S --header 'Accept: application/json' --header 'Fiware-Service: fiwareiot'
    {
      "types" : [
        {
          "name" : "thing",
          "attributes" : [
            "temperature",
            "location",
            "TimeInstant"
          ]
        }
      ],
      "statusCode" : {
        "code" : "200",
        "reasonPhrase" : "OK"
      }
    }

    # Get Context Entities, note: --header 'Fiware-Service: fiwareiot' needs to be present!!
    $ curl sensors.geonovum.nl:1026/v1/contextEntities -S --header 'Accept: application/json' --header 'Fiware-Service: fiwareiot'
    {
      "contextResponses" : [
        {
          "contextElement" : {
            "type" : "thing",
            "isPattern" : "false",
            "id" : "WeatherOtterloEnt",
            "attributes" : [
              {
                "name" : "TimeInstant",
                "type" : "ISO8601",
                "value" : "2015-10-31T12:55:28.157330"
              },
              {
                "name" : "location",
                "type" : "string",
                "value" : "BosHut",
                "metadatas" : [
                  {
                    "name" : "TimeInstant",
                    "type" : "ISO8601",
                    "value" : "2015-10-31T12:55:28.157371"
                  }
                ]
              },
              {
                "name" : "temperature",
                "type" : "int",
                "value" : "11",
                "metadatas" : [
                  {
                    "name" : "TimeInstant",
                    "type" : "ISO8601",
                    "value" : "2015-10-31T12:55:28.157330"
                  }
                ]
              }
            ]
          },
          "statusCode" : {
            "code" : "200",
            "reasonPhrase" : "OK"
          }
        }
      ]
    }

12.5.2. Testing with MQTT Client

The IoTAgent also supports the MQTT protocol: http://mqtt.org

MQTT is a machine-to-machine (M2M)/”Internet of Things” connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport.

We will use Mosquitto as MQTT-client first:

Mosquitto is an open source (BSD licensed) message broker that implements the MQ Telemetry Transport protocol versions 3.1 and 3.1.1. MQTT provides a lightweight method of carrying out messaging using a publish/subscribe model.

See test clients at https://github.com/Geonovum/sospilot/tree/master/src/fiware/client/python/MQTT

Before observations can be sent a Service needs to be created and a Device(s) registered.

On Mac OSX install Mosquitto via HomeBrew:

$ brew install mosquitto
==> Installing dependencies for mosquitto: c-ares, libwebsockets
==> Installing mosquitto dependency: c-ares
==> Downloading https://homebrew.bintray.com/bottles/c-ares-1.10.0.mavericks.bottle.tar.gz

Use mosquitto_pub as a commandline client http://mosquitto.org/man/mosquitto_pub-1.html for initial tests.

$ mosquitto_pub -d -h sensors.geonovum.nl -p 1883 -t sensors/temperature -m "1266193804 32"
Client mosqpub/18773-sunda sending CONNECT
Client mosqpub/18773-sunda received CONNACK
Client mosqpub/18773-sunda sending PUBLISH (d0, q0, r0, m1, 'sensors/temperature', ... (13 bytes))
Client mosqpub/18773-sunda sending DISCONNECT

See setup.sh for full example with Service/Device creation and observation publication via MQTT :

#!/bin/bash

# see https://github.com/telefonicaid/fiware-IoTAgent-Cplusplus/blob/develop/doc/MQTT_protocol.md
# provisioning API: https://github.com/telefonicaid/fiware-IoTAgent-Cplusplus/blob/develop/doc/north_api.md

DEVICE_ID=DavisMQTTDev
ORION_ENTITY=TempGeonovumMQTTEnt
# API_KEY=4jggokgpepnvsb2uv4s40d59ov
API_KEY=7qqa9uvkgketabc8ui4knu1onv
SERVICE_ID=fiwareiot
ORION_HOST=185.21.189.59
ORION_PORT=1026
MQTT_HOST=sensors.geonovum.nl
MQTT_PORT=1883
DEVICE_TEMPLATE=GEONOVUM_MQTT_TEMP

# Create the service
python CreateService.py ${SERVICE_ID} ${API_KEY} ${ORION_HOST} ${ORION_PORT}

# Provision Device
python RegisterDevice.py ${DEVICE_TEMPLATE} ${DEVICE_ID} ${ORION_ENTITY}

# Publish observations
# mosquitto_pub -h $HOST_IOTAGENT_MQTT -t <api_key>/mydevicemqtt/t -m 44.4 -u <api_key>
mosquitto_pub -r -d -h ${MQTT_HOST} -p ${MQTT_PORT} -u ${API_KEY} -t ${API_KEY}/${DEVICE_ID}/temp -m 11
mosquitto_pub -r -d -h ${MQTT_HOST} -p ${MQTT_PORT} -u ${API_KEY} -t ${API_KEY}/${DEVICE_ID}/pos -m '52.152435,5.37241'

and a sample device file:

	{
	 "devices": [
	    { "device_id": "DEV_ID",
	      "entity_name": "ENTITY_ID",
	      "entity_type": "thing",
	      "protocol": "PDI-IoTA-MQTT-UltraLight",
	      "timezone": "Europe/Amsterdam",
	"attributes": [
	        {
	          "object_id": "temp",
	          "name": "temperature",
	          "type": "int"
	        },
            {
	        "object_id": "pos",
            "name": "position",
            "type": "coords",
            "metadatas": [
                {
                    "name": "location",
                    "type": "string",
                    "value": "WGS84"
                }
              ]
            }
	      ],
	 "static_attributes": [
	        { "name": "organization",
	          "type": "string",
	          "value": "Geonovum"
	        }
	       ]
	      }
	     ]
	    }

The Service creation and Device provisioning uses the Admin API of the IotAgentCpp server. MQTT is only used to send observations. See also http://fiware-iot-stack.readthedocs.org/en/latest/device_api/index.html

The helper .py programs are ported from FIWARE FIGWAY Sensors_UL20 code.

TWO VERY IMPORTANT DIFFERENCES WITH UL20:

  • in the payload when creating the Service the following field needs to be set: "resource": "/iot/mqtt" otherwise the Device cannot be registered (“protocol is not correct” error).
  • Also in the Device Template (see here under devices/) the “protocol”: "PDI-IoTA-MQTT-UltraLight" needs to be present.

See also this issue: https://github.com/telefonicaid/fiware-IoTAgent-Cplusplus/issues/254

12.5.3. Inspect data in MongoDB

Within the mongodb Docker container we can inspect the persisted data in the Mongo shell : https://docs.mongodb.org/manual/reference/mongo-shell.

$ docker exec -it docker_mongodb_1 bash
root@43fd245b67ca:/# mongo
MongoDB shell version: 2.6.11
connecting to: test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        http://docs.mongodb.org/
Questions? Try the support group
        http://groups.google.com/group/mongodb-user
> show dbs
admin            (empty)
iot              0.031GB
local            0.031GB
orion            0.031GB
orion-fiwareiot  0.031GB
sth_fiwareiot    0.031GB

> use iot
switched to db iot
> show collections
COMMAND
DEVICE
PROTOCOL
SERVICE
SERVICE_MGMT
system.indexes
> use orion
switched to db orion
> show collections
entities
system.indexes
> db.entities.find()
> use orion-fiwareiot
switched to db orion-fiwareiot
> db.entities.find()
{ "_id" : { "id" : "WeatherOtterloEnt", "type" : "thing", "servicePath" : "/" },
"attrNames" : [ "TimeInstant", "location", "temperature" ],
"attrs" : { "TimeInstant" : { "value" : "2015-10-31T20:41:28.654329", "type" : "ISO8601", "creDate" : 1446324088, "modDate" : 1446324088 },
"location" : { "value" : "BosHut", "type" : "string", "md" : [ { "name" : "TimeInstant", "type" : "ISO8601", "value" : "2015-10-31T20:41:28.654370" } ],
"creDate" : 1446324088, "modDate" : 1446324088 },
"temperature" : { "value" : "11", "type" : "int", "md" : [ { "name" : "TimeInstant", "type" : "ISO8601", "value" : "2015-10-31T20:41:28.654329" } ],
"creDate" : 1446324088, "modDate" : 1446324088 } }, "creDate" : 1446324088, "modDate" : 1446324088 }

# timeseries storage
> use sth_fiwareiot
switched to db sth_fiwareiot
> show collections
sth_/_dust13ent_thing
sth_/_dust13ent_thing.aggr
sth_/_nexusent1_thing
sth_/_nexusent1_thing.aggr
sth_/_tempgeonovument_thing
sth_/_tempgeonovument_thing.aggr
sth_/_thing:dust13_thing
sth_/_thing:dust13_thing.aggr
system.indexes

12.5.4. Display Values with WireCloud

WireCloud http://conwet.fi.upm.es/wirecloud/ is a Mashup framework within FIWARE with instance at FIWARE Lab: https://mashup.lab.fiware.org

Here we can create Widgets to get data from the Orion CB, so indirectly observations sent to the IoTAgent from our devices.

12.5.4.1. First Steps

Trying simple the NGSI Browser, but did not succeed (help mail sent to fiware-lab-help@lists.fiware.org :

Trying to connect to my OCB  which has entities created via IDAS. Both are of latest Docker version.
Works fine using the FIGWAY Python scripts.

But using any Mashup widget that does requests to the OCB like the NGSI Browser the widget remans blanc,
since the OCB sends back:

{
  "errorCode" : {
    "code" : "404",
    "reasonPhrase" : "No context element found"
  }
}

This reply is also received when querying via curl:
curl <my_ocb_host>:1026/v1/contextEntities -S --header 'Accept: application/json'

But if I add the header  --header 'Fiware-Service: fiwareiot' (which was specified when creating the IoT
service w IDAS) then I get expected responses from the OCB.

However the Widgets, Operators in WC have no means to add the 'Fiware-Service' Header.

This problem was also posted at StackOverflow.

A quick local solution is to manually add the HTTP header using the browser’s Developer Tools to a WireCloud Widget JavaScript (usually main.js) where the HTTP request to the Orion NGSI API is created. The WC NGSI connector supports adding extra HTTP headers as per the NGSI documentation. See for example here below where the Chrome developer tools is used to modify the NGSIConnection :

_images/wc-fwheader-hack.jpg

Quick Fix: modify the HTTP header locally in browser

Another possibility is to download the widget, modify its config.xml and main.js to support configuring the FIWARE-Service header. This worked as well, but the real fixes should be done within the component source code. The issue affects all WC components (Operators, Widgets) that interact with Orion NSGI. The following issues have been opened:

As the NGSI Browser Widget was fixed and a new version was available, a first test could be performed.

12.5.4.2. Temperature in POI on Map

Using the NGSI Browser Widget (fixed v1.0.1) with an NSGI Entity to POI Converter connected to a Map Widget the temperature could be made visible on a map. The result of the mashup is below.

_images/wc-geonovum-temp1.jpg

First WireCloud Mashup: show Temperature from sensor as POI

The wiring for these components was as depicted below.

_images/wc-geonovum-temp1-wiring.jpg

First WireCloud Mashup: wiring view in editor

Next attempt was to use NGSI Subscriptions such that the widgets receive real-time updates. For this the NGSI Source Operator can be applied i.s.o. the NGSI Browser Widget used above. The NGSI Source Operator will use NGSI10 Subscriptions to register at an Orion CB using a callback mechanism. The wiring is depicted below.

_images/wc-geonovum-temp2-wiring.jpg

Second WireCloud Mashup: wiring view in editor

The NGSI Source Operator was first fixed via issue 3 such that the Fiware-service HTTP-header could be applied. But since the Orion CB runs without HTTPS within a remote (Docker) service, callbacks via HTTPS did not work. Also using HTTP via proxy http://ngsiproxy.lab.fiware.org did not work because of browser security restrictions. These problems have been documented and discussed in this issue. Polling mode may be another solution but possibly too strenous.

Using an HTTPS middleman Relayer, Rush, is suggested as a solution. This was tried first. The final range of commands is:

# within docker_orion_1 running container

# Dependencies
cd /root
yum -y install npm
# yum -y install redis   (gave errors see below)
yum -y install wget
wget http://download.redis.io/releases/redis-2.8.3.tar.gz
tar xzvf redis-2.8.3.tar.gz
cd redis-2.8.3
make
make install
# /etc/init.d/redis start  (no daemon installed)
redis-server redis.conf &

# Rush
cd /root
curl https://codeload.github.com/telefonicaid/Rush/tar.gz/1.8.3 > Rush-1.8.3.tar.gz
tar xzvf Rush-1.8.3.tar.gz
cd Rush-1.8.3
npm install --production

# set mongodb host as linked in docker-compose iota.yml
export RUSH_GEN_MONGO=mongodb
bin/listener &
bin/consumer &

NB yum -y install redis installed Redis 2.4 but gave recurring error: Generic Server Error: Error: ERR unknown command 'evalsha'. Redis should be higher than v2.4 as stated in the solution here

Activate Rush in Orion within iota.yaml by setting the orion entry (command args) to:

.
orion:
    image: fiware/orion
    links:
        - mongodb
    ports:
        - "1026:1026"
    command: -dbhost mongodb -rush localhost:5001 -logDir /var/log/contextBroker
.

Now notifications are seen immediately on sending events from the UL20 client! See picture below:

_images/wc-geonovum-temp2.jpg

Second WireCloud Mashup: direct notifications in Map (left) from UL20 client (right)

12.5.5. Display with OpenLayers/Heron

In order to facilitate viewing data in standard geo-web viewers, an OpenLayers NGSI10 Vector Layer component was developed. This allows viewers like the existing project HeronViewer to show (real-time) sensor data.

Below is a screenshot of the HeronViewer showing in blue dots 2 “bot-sensors” and the CityGIS Dustduino-based sensor. All show a temperature in realtime.

_images/heron-ngsi10-layer.jpg

NGSI10 OpenLayers Vector Layer (blue dots) in HeronViewer

12.5.6. Timeseries data from STH

The Short Term History (STH) component stores timeseries data. This is achieved by a subscribeContext on the Orion CB NGSI10 API. This has to be done explicitly. Documentation can be found here: https://github.com/telefonicaid/fiware-sth-comet

In our setup we fire a curl script to trigger a generic subscription on the thing entity:

#!/bin/bash

#POST /v1/subscribeContext HTTP/1.1
#Host: sensors.geonovum.nl:1026
#origin: https://mashup.lab.fiware.org
#Cookie: _ga=GA1.2.1632625772.1441807083, policy_cookie=on
#Content-Length: 257
#via: 1.1 mashup.lab.fiware.org (Wirecloud-python-Proxy/1.1)
#accept-language: en-US,en;q=0.8,de;q=0.6,fr;q=0.4,nl;q=0.2,it;q=0.2
#accept-encoding: gzip, deflate
#x-forwarded-host: sensors.geonovum.nl:1026
#x-forwarded-for: 82.217.164.50
#fiware-service: fiwareiot
#accept: application/json
#user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36
#connection: keep-alive
#x-requested-with: XMLHttpRequest
#referer: https://mashup.lab.fiware.org/justb4/NGSI%20Subscription
#content-type: application/json
#
#
#write error to stdout
#
#130.206.084.011.36914-185.021.189.059.01026:
#{"entities":[{"id":".*","isPattern":"true","type":"thing"}],
#"reference":"https://ngsiproxy.lab.fiware.org:443/callbacks/23:33:47-1:23:33:48-1",
#"duration":"PT3H",
#"notifyConditions":[
#{"type":"ONCHANGE","condValues":["position","temperature","organization"]
#}]}
#
#write error to stdout
#
#185.021.189.059.01026-130.206.084.011.36914: HTTP/1.1 200 OK
#Content-Length: 109
#Content-Type: application/json
#Date: Mon, 30 Nov 2015 21:31:14 GMT
#
#{
#  "subscribeResponse" : {
#    "subscriptionId" : "565cc0222b41bbad4c87656f",
#    "duration" : "PT3H"
#  }
#}

ORION_HOST=sensors.geonovum.nl
ORION_PORT=1026
STH_HOST=sensors.geonovum.nl
STH_PORT=8666

#  --header 'Fiware-ServicePath: /'
curl ${ORION_HOST}:${ORION_PORT}/v1/subscribeContext -s -S\
 --header 'Content-Type: application/json' \
 --header 'Accept: application/json' \
 --header 'fiware-service: fiwareiot' \
 -d @- <<EOF
{
    "entities": [
        {
            "type": "thing",
            "isPattern": "true",
            "id": ".*"
        }
    ],
    "reference": "http://sensors.geonovum.nl:8666/notify",
    "duration": "P1Y",
    "notifyConditions": [
        {
            "type": "ONCHANGE",
            "condValues": ["temperature", "humidity", "pm10", "pm2_5"]
        }
    ],
    "throttling": "PT5S"
}
EOF

This triggers a flow of data from the Orion CB via the STH to MongoDB. Via the STH REST API we can request timeseries data. As we need to send Fiware-HTTP headers we can invoke the STH API using a (Chrome) REST client as follows:

_images/sth-timeseries-rest.jpg

Request Timeseries Data from STH REST API

12.6. Installing FIWARE - from Source

Done on the Linux VPS (Ubuntu 14.04).

Abandoned, Orion CB compiled and ran with mongodb, but IoTAgent gave core dump, using Docker now, but kept for reference.

12.6.1. Orion Context Broker (OCB)

12.6.1.1. Build from source

On 28.okt.2015. Version: 0.24.0-next (git version: a938e68887fbc7070b544b75873af935d8c596ae). See https://github.com/telefonicaid/fiware-orion/blob/develop/doc/manuals/admin/build_source.md, but later found: https://github.com/telefonicaid/fiware-orion/blob/develop/scripts/bootstrap/ubuntu1404.sh

Install build deps.

apt-get  install cmake scons libmicrohttpd-dev libboost-all-dev
# what about libcurl-dev gcc-c++ ?
apt-get -y install make cmake build-essential scons git libmicrohttpd-dev libboost-dev libboost-thread-dev libboost-filesystem-dev libboost-program-options-dev  libcurl4-gnutls-dev clang libcunit1-dev mongodb-server python python-flask python-jinja2 curl libxml2 netcat-openbsd mongodb valgrind libxslt1.1 libssl-dev libcrypto++-dev

Setting up libboost1.54-dev (1.54.0-4ubuntu3.1) ...
Setting up libboost-dev (1.54.0.1ubuntu1) ...

Install the Mongo Driver from source:

    mkdir -p /opt/mongodb
cd /opt/mongodb
    wget https://github.com/mongodb/mongo-cxx-driver/archive/legacy-1.0.2.tar.gz
    tar xfvz legacy-1.0.2.tar.gz
    cd mongo-cxx-driver-legacy-1.0.2
    scons                                         # The build/linux2/normal/libmongoclient.a library is generated as outcome
    sudo scons install --prefix=/usr/local        # This puts .h files in /usr/local/include/mongo and libmongoclient.a in /usr/local/lib

Install rapidjson from sources:

    mkdir -p /opt/rapidjson
cd /opt/rapidjson
    wget https://github.com/miloyip/rapidjson/archive/v1.0.2.tar.gz
    tar xfvz v1.0.2.tar.gz
    sudo mv rapidjson-1.0.2/include/rapidjson/ /usr/local/include

Install Google Test/Mock from sources (:

    mkdir -p /opt/googlemock
cd /opt/googlemock
    wget http://googlemock.googlecode.com/files/gmock-1.5.0.tar.bz2
    tar xfvj gmock-1.5.0.tar.bz2
    cd gmock-1.5.0
    ./configure
    make
    sudo make install  # installation puts .h files in /usr/local/include and library in /usr/local/lib
    sudo ldconfig      # just in case... it doesn't hurt :)

Build Orion itself

    mkdir -p /opt/fiware/orion/
cd /opt/fiware/orion/
    git clone https://github.com/telefonicaid/fiware-orion git
cd git

    # Build errors on linking! libboost regex it seems
    [100%] Building CXX object src/app/contextBroker/CMakeFiles/contextBroker.dir/contextBroker.cpp.o
    Linking CXX executable contextBroker
    /usr/local/lib/libmongoclient.a(dbclient.o): In function `boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::assign(char const*, char const*, unsigned int)':
    /usr/include/boost/regex/v4/basic_regex.hpp:382: undefined reference to `boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::do_assign(char const*, char const*, unsigned int)'
    /usr/local/lib/libmongoclient.a(dbclient.o): In function `boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::unwind_extra_block(bool)':
    /usr/include/boost/regex/v4/perl_matcher_non_recursive.hpp:1117: undefined reference to `boost::re_detail::put_mem_block(void*)'
    /usr/local/lib/libmongoclient.a(dbclient.o): In function `boost::re_detail::cpp_regex_traits_implementation<char>::error_string(boost::regex_constants::error_type) const':
    /usr/include/boost/regex/v4/cpp_regex_traits.hpp:447: undefined reference to `boost::re_detail::get_default_error_string(boost::regex_constants::error_type)'
    /usr/local/lib/libmongoclient.a(dbclient.o): In function `void boost::re_detail::raise_error<boost::regex_traits_wrapper<boost::regex_traits<char, boost::cpp_regex_traits<char> > > >(boost::regex_traits_wrapper<boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::error_type)':
    /usr/include/boost/regex/pattern_except.hpp:75: undefined reference to `boost::re_detail::raise_runtime_error(std::runtime_error const&)'
    /usr/local/lib/libmongoclient.a(dbclient.o): In function `boost::re_detail::cpp_regex_traits_implementation<char>::error_string(boost::regex_constants::error_type) con

    # appears known problem: https://github.com/telefonicaid/fiware-orion/issues/1162
# DISTRO 14.04.3_LTS var not in add in src/app/contextBroker/CMakeLists.txt
# add
ELSEIF(${DISTRO} STREQUAL "Ubuntu_14.04.3_LTS")
   MESSAGE("contextBroker: Ubuntu ===TEST===== DISTRO: '${DISTRO}'")
   TARGET_LINK_LIBRARIES(contextBroker ${STATIC_LIBS} -lmicrohttpd -lmongoclient -lboost_thread -lboost_filesystem -lboost_system -lboost_regex -lpthread -lssl -lcryp\
to -lgnutls -lgcrypt)

(Optional but highly recommended) run unit test. Firstly, you have to install MongoDB (as the unit tests rely on mongod running in localhost).

apt-get install mongodb-server
apt-get install pcre            # otherwise, mongodb crashes in CentOS 6.3
service mongodb start
service mongodb status   # to check that mongodb is actually running
make unit_test

[----------] Global test environment tear-down
[==========] 1101 tests from 168 test cases ran. (4985 ms total)
[  PASSED  ] 1096 tests.
[  FAILED  ] 5 tests, listed below:
[  FAILED  ] mongoQueryContextRequest_filters.outsideRange_n
[  FAILED  ] mongoQueryContextRequest_filters.withoutEntityType
[  FAILED  ] mongoQueryContextGeoRequest.queryGeoCircleOut
[  FAILED  ] mongoQueryContextGeoRequest.queryGeoPolygonOut1
[  FAILED  ] mongoQueryContextGeoRequest.queryGeoPolygonOut2

5 FAILED TESTS
YOU HAVE 23 DISABLED TESTS

Also need to fix build error in make unit_tests

ELSEIF(${DISTRO} STREQUAL "Ubuntu_14.04.3_LTS")
  MESSAGE("contextBroker: Ubuntu ===TEST===== DISTRO: '${DISTRO}'")
  TARGET_LINK_LIBRARIES(unitTest ${STATIC_LIBS} -lmicrohttpd -lmongoclient -lboost_thread -lboost_filesystem -lboost_system -lboost_regex -lpthread -lssl -lcrypto -lg\
nutls -lgcrypt)

Install the binary. You can use INSTALL_DIR to set the installation prefix path (default is /usr), thus the broker is installed in $INSTALL_DIR/bin directory.

sudo make install INSTALL_DIR=/usr

#test install
contextBroker --version
0.24.0-next (git version: a938e68887fbc7070b544b75873af935d8c596ae)
Copyright 2013-2015 Telefonica Investigacion y Desarrollo, S.A.U
Orion Context Broker is free software: you can redistribute it and/or
modify it under the terms of the GNU Affero General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.

Orion Context Broker is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero
General Public License for more details.

Telefonica I+D

12.6.1.2. Running OCB

COnfig in /etc/default/contextBroker. As system service:

service contextBroker start
# calls /usr/bin/contextBroker
ERROR: start-stop-daemon: user 'orion' not found

# Found info in DockerFile: https://github.com/Bitergia/fiware-chanchan-docker/blob/master/images/fiware-orion/0.22.0/Dockerfile
# Add user without shell
useradd -s /bin/false -c "Disabled Orion User" orion

mkdir -p /var/log/contextBroker
mkdir -p /var/run/contextBroker
chown orion:orion /var/log/contextBroker
chown orion:orion /var/run/contextBroker
service contextBroker status
# contextBroker is running

Test with fiware-figway Python client:

sunda:ContextBroker just$ python  GetEntities.py ALL
* Asking to http://sensors.geonovum.nl:1026/ngsi10/queryContext
* Headers: {'Fiware-Service': 'fiwareiot', 'content-type': 'application/json', 'accept': 'application/json', 'X-Auth-Token': 'NULL'}
* Sending PAYLOAD:
{
    "entities": [
        {
            "type": "",
            "id": ".*",
            "isPattern": "true"
        }
    ],
    "attributes": []
}

...

* Status Code: 200
***** Number of Entity Types: 0

***** List of Entity Types

**** Number of Entity IDs: 0

**** List of Entity IDs

Do you want me to print all Entities? (yes/no)yes
<queryContextResponse>
  <errorCode>
    <code>404</code>
    <reasonPhrase>No context element found</reasonPhrase>
  </errorCode>
</queryContextResponse>

12.6.2. IoT Agent (CPP version)

From GitHub: https://github.com/telefonicaid/fiware-IoTAgent-Cplusplus, using Docker file https://github.com/telefonicaid/fiware-IoTAgent-Cplusplus/blob/develop/docker/Dockerfile for inspiration.

12.6.2.1. Build fiware-IoTAgent-Cplusplus

From source.

Steps:

    apt-get install -y tar gzip unzip file cpp gcc automake autoconf libtool git scons cmake
    apt-get install -y libssl-dev libbz2-dev zlib1g-dev doxygen

    mkdir -p /opt/fiware/iotagent
    git clone https://github.com/telefonicaid/fiware-IoTAgent-Cplusplus iotacpp
    cd /opt/fiware/iotagent/iotacpp

    # Disable and fix  CMakeLists.txt
    cp CMakeLists.txt CMakeLists.txt.backup
    sed -i CMakeLists.txt -e 's|^add_test|#add_test|g' -e 's|^add_subdirectory(tests)|#add_subdirectory(tests)|g' -e 's|^enable_testing|#enable_testing|g' -e 's|git@github.com:mongodb/mongo-cxx-driver.git|https://github.com/mongodb/mongo-cxx-driver|g'

    # Get version string
    $ source tools/get_version_string.sh
    $ get_rpm_version_string | cut -d ' ' -f 1
    1.3.0

    # Build in Release subdir
    mkdir -p ${CMAKE_CURRENT_SOURCE_DIR}/build/Release
    cd ${CMAKE_CURRENT_SOURCE_DIR}/build/Release
    cmake -DGIT_VERSION=1.3.0 -DGIT_COMMIT=1.3.0 -DMQTT=ON -DCMAKE_BUILD_TYPE=Release  ../../

    # very long build...lots of output...
$ make install

Create install scripts from RPM spec files. Need these files https://github.com/telefonicaid/fiware-IoTAgent-Cplusplus/tree/develop/rpm/SOURCES See https://github.com/Geonovum/sospilot/tree/master/src/fiware/iotagent

12.6.2.2. Running

Config JSON.

{
    "ngsi_url": {
        "cbroker": "http://127.0.0.1:1026",
        "updateContext": "/NGSI10/updateContext",
        "registerContext": "/NGSI9/registerContext",
        "queryContext": "/NGSI10/queryContext"
    },
    "timeout": 10,
    "http_proxy": "PUBLIC_PROXY_PORT",
    "public_ip": "8081",
    "dir_log": "/var/log/iot/",
    "timezones": "/etc/iot/date_time_zonespec.csv",
    "storage": {
        "host": "localhost",
        "type": "mongodb",
        "port": "27017",
        "dbname": "iot"
    },
   "resources": [
        {
            "resource": "/iot/d",
            "options": {
                "FileName": "UL20Service"
            }
        },
        {
            "resource": "/iot/mqtt",
            "options": {
                "ConfigFile" : "/etc/iot/MqttService.xml",
                "FileName": "MqttService"
            }
         }
   ]
}

Core dump...gave up.