Remote Telemetry Data Acquisition

A full stack remote data acquisition project monitoring temperature every 5 seconds and using a WiFi link, the remote telemetry unit (RTU) connects to an Internet based data acquisition server (could be MQTT or HTTPS) to register the unit id, temperature and the time of the transmissions from the RTU.

An Internet facing API is available to retrieve and consume services based on the registered data and make it available for monitoring systems like mobile applications to get the current temperature and an oversight of how it changed looking back in time, or be alerted when temperatures exceed defined settings.

Remote Telemetry Data Aquisition Diagram

Data acquisition

Temperature measurements A MCP9808 high precision digital temperature sensor from the USA based company Microchip. It operates between -40 and 125 °C with an accuracy of ±0.5°C

Data processing and dispatch

Microcontroller An ESP8266 is a WiFi based chip that has built in support for TCP/IP with impressive microcontroller ability, made by the Chinese based Espressif Systems.

Online logging

Internet server service An online HTTPS API (or MQTT) has an Internet facing data logging interface that listens for inbound telemetry transmissions from the remote data collection units.

Data storage

Data storage A database registers and stores the incoming data via stored procedures and indexes data for efficient and effective on demand retrieval.

Data Processing

Internet server service Periodic tasks like calculating the average values per day. week, month and year are done by processes running on the telemetry receiving server.

Data Consumers

Data consumers Data consumer Example Outbound data feed processes, data consumer management as well as alarm and alert push notifications are dealt with via real-time outbound server processes.

External Data Processing

External Data Processing Secured unprocessed and processed APIs are made available for external systems to consume and use the data for own and internal application usage.

Application Feeds

Application Feeds Special compact preprocessed feeds of data are available to be consumed by portable and mobile wireless devices.


ESP8266 Small footprint WiFi module with programmable MCU


MCP9808 Compact precision temperature measurements

OLED Display

OLED Display A small low power OLED display was added to the remote telemetry collector to enable insight into whether it could and was connected to WiFi as well as whether the temperature sensor was operational.
Basically a pacifier for the installer.

Remote Telemetry Schematic

OLED Display


Everything runs of a 3.3 Volt power source. If you want to run it of a LIPO cell, you will want to put a voltage regulator inline.

Provision is made for an optional OLED display. The MCP9808 chip should not be mounted close to the ESP8266 as this can affect the correctness of the temperature readings.

Pin4 and Pin5 of the programming connector need to be shorted out in order to enable flashing of the ESP8266 module.

Ideally each chip should have a 100pF capacitor located as close as possible to the chip between VDD and GND.

Source code:

C++ source code for the ESP8266. It should be self explanatory how it works, however do make sure you have all the used libraries available. Easiest way to get up and running will be via the Arduino IDE. Replace the WiFi specifications for your own as well.

Remote Telemetry Reception

The section above is about reading data sensors, formatting the data into JSON data stream and dispatching it to a remote server. This is about receiving data from remote telemetry sensors.

Internet facing services There are many ways to develop APIs like web services frameworks and SOAP, however these are much to complex and cumbersome for this purpose. Considering we already have a firewall and surrounding processes monitoring inbound traffic as well as using HTTPS as transport medium, a single class with an Internet aware interface will do just fine.

Inbound data handler and decoder class

Simple straight forward JSON data telemetry receiver and decoder. The complete class can be downloaded here.

telemetry receiver and decoder How it works: when requests to the API URI are received by the server, they are routed to this class for initial processing. The data stream from the request is read in (note the web server is monitoring for max data length and buffer overflow), checked if it is JSON data and then decoded according to the telemetry data construct (ENUM). Should the data decode correctly it is then offered to the database service for storage in the database. The response back to the sending party if a good telemetry data packet was received is an 'OK' for success or a 'NOK' for a failure to register the data.

Public Sub ProcessRequest(context As HttpContext) Implements IHttpHandler.ProcessRequest
                '// Check for 'signature' in header of request, if found continue else report IP to interactive FW and drop request
                '// ....
                Dim strJson As String = New StreamReader(context.Request.InputStream).ReadToEnd()
                If Not String.IsNullOrEmpty(strJson) Then
                    Dim objMeas As MeasureInfo = JsonConvert.DeserializeObject(Of MeasureInfo)(strJson)
                    If DataService.PutLoggerData(objMeas.mac, objMeas.ip, objMeas.temp) Then
                    End If
                    context.Response.Write("No Data")
                End If
                context.Response.ContentType = "text/plain"
            Catch ex As Exception
            End Try
        End Sub

Data processing and forwarding

Secure server processes As the data receiver class is potentially connected directly to the Internet, isolating it from processes deeper in the systems is a smart idea. In order the get the checked and verified data into the database another (data) class that utilizes data base stored procedures is implemented. This prevents any direct access to the database from Internet facing processes. This is the call to

DataService.PutLoggerData(objMeas.mac, objMeas.ip, objMeas.temp)

Public Shared Function PutLoggerData(mac As String, ip As String, value As StringAs Boolean
        Dim ok As Boolean = False
            Dim ConnectionString As String = ConfigurationManager.ConnectionStrings("YOUR_DB_CONNECTIONSTRING").ConnectionString
            Using Connection As SqlConnection = New SqlConnection(ConnectionString)
                Using Command As SqlCommand = New SqlCommand("put_logger", Connection)
                    Command.CommandTimeout = 2
                    Command.Parameters.AddWithValue("@mac", mac)
                    Command.Parameters.AddWithValue("@ip", ip)
                    Command.Parameters.AddWithValue("@value", value)
                    Command.CommandType = CommandType.StoredProcedure
                    If Connection.State = ConnectionState.Closed Then
                    End If
                    ok = True
                End Using
            End Using
        Catch ex As Exception
            Errorhandler.ErrorHandler(0, ex.ToString)
        End Try
        Return ok
    End Function

Database Storage

Database To store the telemetry data, at this point two things are needed. A database table to store the information and the stored procedure to put the data into the data table. In this instance a MS SQL backend server is used located on the local LAN network.

Data table creation script

CREATE TABLE [dbo].[logger](
	[id] [bigint] IDENTITY(1,1) Not NULL,
	[mac] [nvarchar](17) Not NULL,
	[ip] [nvarchar](39) Not NULL,
	[productid] [int] Not NULL,
	[value] [nvarchar](50) Not NULL,
	[date] [datetime] Not NULL
ALTER TABLE [dbo].[logger] ADD
CONSTRAINT [DF_logger_productid]  Default ((0)) For [productid] GO ALTER TABLE [dbo].[logger] ADD
CONSTRAINT [DF_logger_date]  Default (getutcdate()) For [date] GO

Stored procedure generation script

CREATE PROCEDURE [dbo].[put_logger]
	@mac nvarchar(50),
	@ip nvarchar(50),
	@value nvarchar(50)
                         (mac, ip, value)

Data example

Database Example of logged data as stored in the data table on the database.

The 'id' column is the primary key with a self incrementing number. This way we can guarantee that if the numbers are consecutive, the database is intact and no data has been removed.

The date is populated with the UCT date time of the data record entry into the database. If it is important to register the time the data value was actually measured, then that will need to be sent from the remote telemetry unit.

The MAC and IP addresses serve to identify from which unit the data transmissions origionated from.

The product id, not used at present, is to associate the measured reading to a particular product, location, whatever.

As time goes by and more data is collected other processes can start using the historic data, like daily, weekly, monthly and yearly averages with the ability to compare current values to historic values, analyse the differences and determine trends.

Processed telemetry data output

Telemetry output Making the received and stored telemetry data available for use by external parties like mobile apps, web services are used again, but this time instead of inbound data streams they are going to be outbound. Examples of output data:

  • Get current data value.
  • Get last ten measurements.
  • Get average for time of day for week
  • Get average for today
  • Get trend (up or down)
  • Get chart data for period
  • Setup and subscribe to push notifications
  • .....

Live api pull example

Get last 10 received telemetry values

Data consumer raw output API

mac "80:7D:3A:6E:68:55"
ip  ""
productid   0
value   "27.38"
Date    "2022-08-20T10:44:28.303"
mac "80:7D:3A:6E:68:55"
ip  ""
productid   0
value   "27.37"
Date    "2022-08-20T10:44:14.35"
mac "80:7D:3A:6E:68:55"
ip  ""
productid   0
value   "27.31"
Date    "2022-08-20T10:44:00.62"
mac "80:7D:3A:6E:68:55"
ip  ""
productid   0
value   "27.30"
Date    "2022-08-20T10:43:46.9"