Agora Edge Apps SDK

The Agora Edge Apps SDK is the next generation SDK for development of Agora Edge Applications. These SDK provide the minimal set of features that allows an application to be developed in either .NET or Python while providing a consistent, documented set of features for developers to use.

The four primary pieces of functionality offered by the SDKs are:

  • Logging
  • Configuration
  • Bus Communications and Message Handling
  • Utilities

For a guided introduction to use the SDK, refer to the Tutorials (SDK Tutorial).

Extra components for working with the Agora and the SDK are also included or available when application are deployed on the gateway:

  • Development Tools
    • CSVReader - Reads CSV files and publishes data messages.
    • CSVWriter - Listens for data messages and publishes tags specified in configuration file to csv file.
    • Mosquitto - Publicly available MQTT Broker useful during development providing an MQTT server for transferring messages between local applications.
  • Core Components
    • Agora Edge App Broker - The broker used on the gateway providing application level authentication/authorization as well as message routing.
    • OpsAgent - The AgoraOps Agent Primarily used to push alternate configurations from Cloud to Edge App/s running on gateway.
    • FileManager - AEA File Manager handles file transfer for AEA applications to/from the cloud as well hosts files for sharing at the edge.
  • Edge Web Services
    • Edge Application Gateway (EAG)
  • Edge Device Communication Modules
    • Modbus Module (HBM)
    • Enron Modbus Module
    • OPC UA Client
    • OPC UA Server
    • WITS Receiver
    • WITSML Client
    • WITSML Server
  • Northbound Adapters
    • Agora Cloud Adapter
    • Agora On-prem Adapter
    • Avalon Adapter

Units and Value Nomenclature and Time

When working with edge applications in the Agora ecosystem, it's important to ensure that data is being shared in units that are easily understood. To reduce complexity, a common unit system is adopted for data transfer between applications. Value names which are implicitly a type of value (ex. WellheadPressure) are expected to be in the common unit system unless explicitly added to the name. Agora Edge Apps adopt the following common SI Base Units with their following abbreviations:

  • time - seconds (s)
  • distance - meter (m)
  • area - square meters (m2)
  • volume - cubic meters (m3)
  • mass - kilogram (kg)
  • temperature - Celsius (degC)
  • current - ampere (A)
  • voltage - volts (V)
  • resistance - ohms (ohms)
  • inductance - henry (H)
  • capacitance - farad (F)
  • energy, work, amount of heat - joule (J)
  • force - newton (N)
  • pressure, stress - pascal (Pa)
  • magnetic flux - weber (Wb)
  • magnetic flux density - tesla (T)
  • frequency - hertz (Hz)
  • plane angle - radians (rad)
Important

Any value which has either a unit which is unusual or does not follow the units provided MUST append the unit to the value name.

Optional: To clarify the unit associated with a value name that follows the common unit system one MAY append the unit to the value name.

Values shall be appended with the unit as follows:

{ValueName}[_{Unit}]

ex:

  • WellheadPressure_psi - Wellhead Pressure in pounds per square inch (psi)
  • WellheadPressure_Pa - Wellhead Pressure in pascals (Pa)
  • WellheadPressure - Wellhead Pressure in pascals (Pa)

Timestamps

To ensure consistency and ease of communication between apps, all timestamps in the Agora ecosystem should be expressed as the number of milliseconds since January 1, 1970 (UTC), represented as a double. Developers can utilize the AgoraTimeStamp function within the SDKs to obtain this value. Unless otherwise specified in the variable name, this should be the default format for timestamps in Agora Edge Apps.

Deployment

When deployed, Agora Edge Apps are executed within containers to make sure the protection of both the application and the rest of the edge system from potential damage. As part of a deployment, each application is provided several shared volumes that allow it to be configured remotely and to exchange files with the cloud and local users.

Containerized App

To distribute containers to the edge, container registries are utilized. For commercial deployments, containers are placed in the Agora private container registry. Each container in this registry undergoes security scanning by AgoraOps personnel to make sure that any security or quality issues are resolved before inclusion. Additionally, all Agora Edge Application containers must be well-documented and comply with the configuration and file input/output methodology.

On the gateway, Agora Edge Applications are deployed as a group and communicate through the use of messages via a message broker.

Messaging

Similarly, gateways are deployed across the world and communicate with Agora Cloud or Agora On-Prem using messaging. Gateways DO NOT communicate with one-another via the MQTT IoT Hub.

Gateway Messaging

Gateways communicate to other systems at the edge using physical connections, which are allocated to specific containers. For example, a container may be provided access to a serial port to communicate with a WITS Server or allocated a TCP port to provide a Modbus Slave endpoint. This allows clear and secure communication between the gateway and other systems at the edge, and ensures that each container is only able to access the resources it needs to function.

Languages

The Agora Edge Apps SDK offers the fundamental tools to create applications and guidelines for packaging the application in containers for use on an Agora Gateway. The SDK enables developers to use their familiar development environments and utilize portable programming languages that facilitate the direct transfer of code from the development environment to a gateway. This streamlines the development process and allows for efficient deployment of applications to the edge.

Two languages are provided within the SDK, .NET and Python.

Configuration

The Agora Edge Apps SDKs provide a common method of sharing configuration across an application and with AgoraOps. Configurations are built at runtime using several sources, in the following order:

  1. Defaults - In Memory Config provided by app and SDK defaults.
  2. AEA.json - Internal configuration file, a.k.a. Primary Configuration.
  3. Environment variables prepended with "AEA__"
  4. Command-line arguments
  5. ./config/AEA.json - Alternate Configuration (optional) - Allows remote configuration by AgoraOps. During development, one typically does not use this.
  6. ./config/keys/{Key} - Key-Per-File: if {Key} file is specified, its value is the file contents (optional) - Allows remote configuration by AgoraOps. During development, one typically does not use this.
  7. Overrides - Overridden configuration values programmatically set by the application itself.
Note

Since environment variables and files (for Key-Per-File) cannot contain ':', a double underline '__' is used instead, ex. AEA2__LogLevel for key-file name, and AEA__AEA2__LogLevel for Environment Variable.

Configuration Architecture Overview

The AEA.json files and Key-Per-File are monitored for changes (see Runtime Configuration Changes at run-time.

Alternate Configuration and Key-Per-File

The purpose of the alternate configuration file is to allow the module to be configured once while running as a container via shared volume. To set the alternate configuration via docker use a command similar to the following:

docker -v {shared config folder}:/{AppDir}

where the shared config folder contains:

{application.exe folder}/
        config/AEA.json    <-- the alternate configuration
        config/keys/{Key-Per-File}

Within the config/keys folder, the file and file contents are converted to key-value pairs within the configuration.

For example, the file AEA2__LogLevel is the configuration setting 'key', and the files contents would be the configuration value.

Example: Contents of AEA2__LogLevel file to specify Debug level of logging:

Debug
Caution

You cannot specify a hierarchy of settings nor arrays using Key-Per-File.

Well known Configuration Settings

Below are the set of well known Configuration Settings provided by default:

  • Name - The Name of the App. This setting is used to set the MQTT Client Name for identifying the client to the Agora Edge Apps Broker. This is used for routing purposes. Default: Entry Assembly Name. Although "optional", it is recommended to specify the "Name" setting in the AEA.json configuration file.
  • DEVICE_ID - Device/Machine Id
  • GROUP_ID - Group Id of the Gateway
  • GATEWAY_ID - Gateway Id

Configurable Logging Settings

  • AEA2:LogLevel (string) - [optional] One of ("Trace", "Debug", "Info" (default), "Off"). This setting is not case sensitive and provides the capability to specify the minimum level of log statements included in the log. This setting affects all Logging Targets contained by the Logger.

Configurable BusClient Settings

  • AEA2:BusClient:Server (string): Server name hosting the MQTT Broker. Default: localhost.
  • AEA2:BusClient:Port (uint): Server Port for MQTT broker hosting the MQTT Broker. Default: 707.
  • AEA2:BusClient:DeviceId (int): Default Device Id to use when publishing DataOut messages. Use this instead of setting Device Id programmatically to make application more configurable.
  • AEA2:BusClient:Subscriptions (string array): Array of string topics to subscribe. Routing is required to receive messages. Default: <empty>. Common topics include:
    • DataIn - Subscribes to DataIn topic containing Io Data Reports.
    • RequestIn - Subscribes to RequestIn topic containing Request Messages.

File Manager Settings

  • AEA2:FileManager:SharedVolumeFolder (string): The folder used to interact with the FileManager.

Messaging

The BusClient and the Message Broker

The Agora Edge Apps SDKs include BusClients which are utilized to interact with an MQTT Message Broker. A BusClient is responsible for sending and receiving messages through the Broker.

When an application is deployed on the gateway, it will use the Agora Edge Apps Message Broker. Unlike a typical MQTT Broker, the Agora Edge Apps Message Broker manages the routing of messages between applications. For instance, AppA might generate DataOut messages which the operations team would like sent to AppB. Using the routing capability, the Broker can be configured to route /AppA/DataOut TO /AppB/DataIn, allowing for seamless communication and data transfer between different parts of the system.

Within the SDKs, the below topics are used by the BusClients:

  • DataOut/In - Used for passing Data Messages (IoDataReportMsg).
  • RequestOut/In - Used for requesting other application services (RequestMsg).

Generic Topics are routed similarly, i.e. publishing of a custom topic is done using SendMessage. Ex. When sending a message with the topic "T1", the topic is converted in the broker to '/{Name}/T1' and routed to other topics using the routes specified in the broker's routes.json configuration file.

Data and Request Messages

The BusClient sends Data Messages contained within IoDataReportMsg, Request Messages contained within RequestMsg, and custom messages (byte[]). Additionally, when data is received, the BusClient stores the messages in the BusClient Message Queues. Applications are expected to check and dequeue messages periodically. Currently, there is no mechanism provided that signals the app that a message has arrived.

Within Data and Request messages, there is a 'header' element which contains information that allows the receiver to determine:

  • SrcModule (string) - The Name of the originating sender module as specified in the configuration setting "Name".
  • MessageID (Int32) - An integer used to uniquely identify the message.
  • MessageType (string) - The type of message
  • ConfigVersion (Int32) - The subtype of message (used for versioning the message schema)
  • Timestamp (Int64) - Milliseconds since Jan. 1, 1970 (UTC) - AgoraTimeStamp

IoDataReportMsg

The IoDataReportMsg is used for accessing DataIn and creating DataOut messages.

The following is an example of JSON Data Message that is generated by serializing an IoDataReportMsg:

Note

It is not required for the developer to serialize or deserialize the Data and Request messages. The SDK does this for you.

{
  "header": {
    "SrcModule": "Sender",
    "MessageType": "IODataReport",
    "ConfigVersion": 1,
    "MessageID": 21556310,
    "TimeStamp": 1674686831036
  },
  "device": [
    {
      "id": "999",
      "tags": {
        "HKLD": {
          "value": 10,
          "quality_code": null,
          "timestamp": 1674686831036,
          "timeoffset_usec": null,
          "value_str": null
        }
      }
    }
  ]
}

RequestMsg

Request messages allow applications to pass requests to other applications and to provide responses (as needed). A Request message allows a string command to be send, as well as a set of parameters as string key/value pairs. A correlation id header.MessageID is provided on both messages to allow the request sender to correlate a Request with a possible Response (also sent as a Request message), as required.

To use Request Messages, subscribe to RequestIn. When Requests arrive, they are placed into the BusClient Message Queue as RequestMsg.

The following are example JSON Request and Response Messages:

Example of Request Message


{
  "header": {
    "SrcModule": "Sender",
    "MessageType": "RequestName",
    "ConfigVersion": 1,
    "MessageID": 21556311,
    "TimeStamp": 1674686832593,
  },
  "payload": {
    "Arg1": "Value1",
    "Arg2": "Value2"
  }
}

Example of Response Message for above Request

Note

MessageID is the same so that the request/response can be correlated.

{
  "header": {
    "SrcModule": "Receiver",
    "MessageType": "RequestName_RSP",
    "ConfigVersion": 1,
    "MessageID": 21556311,
    "TimeStamp": 1674686832649,
  },
  "payload": {
    "RetVal1": "Value1",
    "RetVal2": "Value2"
  },
  "response": {
    "status": "SUCCESS"
  }
}

File Transfers

A deployment of Agora Edge Apps provides the AEA_FileManager Application by default. This application is used to transfer files between the Cloud and the Applications on the Gateway.

To transfer files from your application to the Cloud, write files to the {fileManagerSharedVolume}/fileOut folder. Files beginning with '~' are ignored to allow writing files in place before sending.

Files received from the cloud will appear in the {fileManagerSharedVolume}/fileIn folder.

For consistency, use the following configuration setting to set the Shared Volume Folder.

  • AEA2:FileManager:SharedVolumeFolder (string): The folder used to interact with the FileManager.

On the Cloud, files can be transferred using the Nimbus File Manager Service API described at https://swagger.p4d.nimbus.slb-ds.com/swagger. An approved account is required to access the Cloud API documentation.