Telemetry Data Normalization: A Practical Guide

Telemetry normalization is the act of transforming metric streams from hundreds or thousands of sources into a common, canonical format of names, units, types and identifiers. Every graph, alert and data model consumes data products the same way. Normalize at the edge of your pipeline, define schemas before data flows and treat semantics, not just formatting, as immutable. Learn how, why and where teams fail at this process.

TL;DR

• Normalization is a contract, not a clean-up job. Specify the schema at producer or collector and validate against that schema; do not rely on fixing data after it is collected.
• Semantics beat syntax. Renaming fields is trivial. Projects break when two values that look similar do not actually represent the same thing.

Decouple identity from description. Separate mutable attributes such as IP address, firmware and location from stable IDs for devices and assets.

Normalize during collection. Unit conversion, renaming attributes, filtering and sanitization should happen at the collector prior to storage.

• Map sources via configuration. It should take minutes to add a new vendor format to a robust pipeline through a mapping file, rather than waiting on a new code release.

Table of Contents

• What telemetry data normalization means
• Why raw telemetry breaks at scale
• Normalize meaning first, format second
• Designing the schema
• Identity versus attributes
• Building the normalization pipeline
• Normalizing IoT and building data
• Common pitfalls
• How to start: a six-step plan
• Our perspective
• Where PODTECH can help
• FAQ
• Sources

What telemetry data normalization means

Telemetry is anything your systems broadcast about themselves: metrics, logs, traces, events and sensor data. For datacenters, this includes power meters, CRAC units, leak sensors, generators and network equipment. For buildings this means BMS sensors, access control, fire and life-safety panels. For factories it includes PLCs and gateways.

Every source uses its own slang. One chiller expresses temperature in Fahrenheit as SupplyTemp. Another uses Celsius as temp_supply_c. One more sends you an unscaled integer and you must apply a scale factor. Normalization converts all three to one definition: same name, same unit, same type, same meaning.

The practical benefit: normalize telemetry to a common format or schema according to AWS Well-Architected guidance, which enables you to collect and report on your data consistently and correlate signals across system performance, user behavior and security events.

Not the same as database normalization

Search results for this topic often combine two concepts. Database normalization, such as 1NF, 2NF and 3NF, is concerned with how tables are structured to reduce redundancy. Telemetry normalization involves processing heterogeneous incoming signals into comparable information. The concepts share a goal, consistency, but you are not splitting your sensor feed into third normal form. Think canonical model and contract, not table design.

Why raw telemetry breaks at scale

Hand-written maps will suffice for a tiny estate. As you expand, you run into three problems.

1. Correlation fails. If you have kW on one feed and W on another, someone can build a dashboard that adds them together without realizing anything is wrong.
2. Expenses increase. Redundant and overlapping series bloat storage and query costs. Attribute values that differ create additional unique series, the classic cardinality problem.
3. Trust is lost. There is an ISO/IEC 30178 standard for IoT data format, value and coding because variation in how sensor data is presented leads to a loss of trust from consumers and a loss of value.

The most insidious failures are quiet ones. A unit mismatch does not throw a dimensional error. It gives you a reasonable-looking number.

Normalize meaning first, format second

Probably the single most common error is approaching normalization as field renaming. Papers written about transforming between observability vendor schemas state the principle clearly: it's semantic, not syntactic. Vendors ascribe different definitions to fields with similar names, so simple mapping sacrifices accuracy.

A concrete example from facilities work: “alarm” may mean threshold exceeded, communications failure, or acknowledged-but-not-cleared depending on the BMS. Mapping all three onto alarm=true is syntactically convenient and operationally hazardous. A more robust canonical model breaks these into state (normal, warning, critical), cause (threshold, comms, fault) and acknowledged (boolean).

Before writing any mapping, document for each signal:

• What physical or logical quantity it measures
• The canonical unit and its precision
• Sampling interval and whether the value is instantaneous, averaged or cumulative
• What a missing value means: sensor offline, no event or zero
• Who owns the definition

Designing the schema

Telemetry schema is a contract between the sender and the receiver or receivers of telemetry. The Telemetry event-schema guide describes it as a contract and recommends agreeing upon each field before events are emitted, ensuring queries have the same units, names and expected results.

Start from an existing convention

Do not create your own attribute names if a convention already exists. OpenTelemetry semantic conventions describe commonly agreed attributes that bring meaning to telemetry data as it is generated, emitted and observed. Following them allows you to use standard units, seconds for duration for example, and standard attribute names like server.address, which helps reduce cardinality and makes data more portable across tools.

OpenTelemetry also has a concept of telemetry schemas. This solves the assumptions producers and consumers have about each other's data implicitly, and provides a documented path forward when names change between versions.

Extend, do not fork

Physical infrastructure stretches the edges of what general observability conventions cover. Use your own namespace for domain concepts like rack, row, containment aisle, leak zone and breaker. Just keep it documented and versioned. Do not pollute standard namespaces with custom attributes.

A minimal canonical record

For a sensor reading, a workable shape looks like this:

Retaining source and ingest timestamps is important. Otherwise you cannot detect a gateway that buffers for 10 minutes and then dumps the backlog.

Identity versus attributes

Good correlation requires immutable identifiers. The Telemetry application guide cautions against embedding dynamic values like a leased IP address in an entity's ID: every network change makes one device appear as multiple entities.

A simple rule:

Identity: a consistent asset tag, serial number or generated UUID that does not change throughout the life of the device.

• Attributes used to describe the device: IP address, firmware version, room, owner and status. These attributes are mutable, and they belong with the ID, not in the ID.

OpenTelemetry has the same dichotomy in its entity model. Entity events convey the entity type, its identifying attributes and its descriptive attributes separately, along with a lifecycle event type.

If you have an asset register inside a CMDB or DCIM tool, use that as your single source of truth for IDs and let the pipeline enrich inbound data against it. That is how a raw point name like AHU3_T1 can know which asset it lives on, what room that asset is in, what floor that room sits on and what building contains that floor.

Building the normalization pipeline

A reliable telemetry data pipeline has five stages. Normalization sits mostly in the third.

1. Collect. Poll or receive from sources: Modbus, BACnet, SNMP, MQTT and vendor APIs.
2. Parse and decode. Turn bytes and vendor payloads into structured records.
3. Normalize. Rename, convert units, retype, map enumerations, add identity and enrichment.
4. Validate and sanitize. Check against the schema and strip anything sensitive.
5. Route. Send to storage, alerting, analytics and long-term archive.

Do it in the collector

This is supported directly by OpenTelemetry's collector. Its transform processor allows you to define transformations for metrics, logs and traces using the OpenTelemetry Transformation Language, OTTL. It can also drop telemetry that matches certain conditions with a filter processor. This enables declarative cleanup in the pipeline instead of point-to-point converters scattered throughout your systems.

Use mapping files, not code

AWS has a reference architecture for telemetry normalization that solves a problem many teams wrestle with. Fleet operators ingest data from OEM cloud APIs, CAN bus and direct MQTT, with varying signal names, units and encodings. The architecture decouples the source from downstream applications via pre-defined transform manifests, so when a new source is added it arrives as a configuration file rather than new code. The domain there is vehicles, but the pattern generalizes easily to chillers, UPS units and gateways.

A transform manifest entry usually specifies:

• Source signal name and path
• Canonical metric name
• Scale factor and offset
• Source and target units
• Enumeration mapping, for example 0=off, 1=on, 2=fault
• Valid range, used to flag bad readings

Sanitize at the source

Sanitization is not optional. Take a lesson from OpenTelemetry's database span conventions: sensitive literals in query text are replaced with placeholders, letting you retain structure for performance analysis without leaking user data or causing cardinality blow-up due to unique values. The same concept applies to serial numbers found in free-text fields, operator names in logs or any other potentially identifying payload.

Go schema-first

The guiding philosophy is to work at the producer rather than post-ingestion. The Schema-First Application Telemetry position paper makes the case for defining telemetry contracts ahead of time so they can be validated prior to deployment rather than repaired after indexing. If you do not control the hardware, you may not be able to enforce it at the device. You can enforce it at the first collector you do control.

Normalizing IoT and building data

IoT data adds three complications: protocol diversity, intermittent connectivity and constrained devices.

Protocols: BACnet, Modbus, SNMP and MQTT all encode the meaning of the same physical quantity differently. Normalize after decoding messages, but never inside protocol decoding code.

• Delivery guarantees. Duplicates and gaps should be expected on lossy links. See our blog on mapping MQTT QoS to risk to determine what level is appropriate for each signal, and ensure your payload schema includes a sequence number or event ID so duplicates can be discarded.
• Connectivity. Many remote locations will have cellular backhauls. Select connectivity that allows devices to maintain stable network profiles. Example: business IoT SIM cards with the ability to roam across multiple networks, so gateways report reliably and timestamps remain accurate. This particular provider is Europe-based, so ensure you have coverage for domestic US locations if needed.

See ISO/IEC 30178 for format and coding standards of the sensor values themselves before inventing your own.

Common pitfalls

How to start: a six-step plan

1. Sources of inventory. Enumerate all telemetry-emitting systems, their protocol, owner and size.
2. Choose your canonical model. Follow OpenTelemetry convention where possible and document your extension otherwise.
3. Define identity. Agree the asset ID scheme and where the master register lives.
4. Write mappings for your top ten signals. Select the signals that are actionable, those that influence alerts and capacity decisions. Validate the pattern before mapping everything.
5. Fail early in the pipeline. Reject or quarantine broken records and tally them. An increasing reject rate clearly identifies the source that changed.
6. Version everything. Schema, mappings and enrichment tables all have versions. Consumers specify which version or versions they expect.

Our perspective

We make monitoring and integration software for datacenters, buildings and other mission critical facilities. Normalization is where most integration projects succeed or fail. It is almost never the hardware. The problem is that a BMS describes a room differently than a power management system or a network management system. Nobody standardized which description should prevail.

Our philosophy is straightforward. A normalized pipeline is a software abstraction between noisy signals and decisions you can act upon. It also insulates your strategy: by decoupling collection from analysis, you can switch out a monitoring platform without having to re-attach every sensor.

Three habits matter most in our experience:

Involve monitoring as early as possible. Late schema decisions are costly to reverse. That is why we preach include monitoring teams on day one of datacenter mobilisation.

One source of truth for assets. If integration between BMS, PMS and NMS is going to work, everything needs to resolve to the same identity for that asset. Master systems integration is designed with that concept in mind. Learn more.

• Plan for crappy hardware. Production environments have weird legacy equipment and strange protocols. See our observations on DCIM hardware foibles for details.

— Harry

Where PODTECH can help

Whether your telemetry comes from disparate systems and no one believes the compiled picture, we can assist you in architecting the canonical model, building the pipeline and stitching it into your existing platforms. Our projects include BMS and PMS integration, legacy modernization, and datacenter telemetry, with examples of our work on DCIM solutions. For an idea of the full range of services we provide, begin at podtech.com.

For those earlier on the journey, our SCADA to cloud migration blueprint explains why telemetry needs to come first. Also, our Industrial IoT security roadmap will guide you through securing those same data flows.

FAQ

What is telemetry data?

Telemetry data refers to the measurements and events that systems generate about their internal state and activity. Telemetry includes metrics, logs, traces and sensor readings such as temperature, power draw or door status. Telemetry data is collected remotely and sent to a central monitoring and analysis system.

How is telemetry normalization different from database normalization?

Database normalization, such as 1NF, 2NF and 3NF, organizes tables to minimize duplication. Telemetry normalization takes signals from multiple sources and conforms them to a single common schema of names, units and meanings. The goal is similar in both cases, consistency, but the methods used to achieve it are different.

Should I normalize before or after storing the data?

Store data wherever necessary beforehand. By normalizing in the collection pipeline you maintain consistency in storage and can validate and sanitize earlier. Store raw data in an archive tier if you may need to re-process it when mappings evolve.

Which standard should I base my schema on?

Start with OpenTelemetry semantic conventions for general observability signals. For IoT sensor data formats and encoding standards, see ISO/IEC 30178. Document and version your own extension for concepts that neither covers.

Is normalization worth it for a small estate?

Typically, yes, at least informally. A common naming convention, canonical units and persistent asset IDs are inexpensive early on, and much less expensive than applying them later to dozens of connected sources.

Sources

• AWS Well-Architected: standardize telemetry data with common formats
• ISO/IEC 30178 on IoT data format, value and coding
• Semantic conventions on semantic meaning in telemetry (OpenTelemetry)
• Designing an event schema (Telemetry)
• OpenTelemetry semantic conventions
• OpenTelemetry telemetry schemas
• Application telemetry: a practical guide (Telemetry)
• OpenTelemetry entity events
• Transforming telemetry (OpenTelemetry Collector)
• AWS guidance: telemetry normalization architecture
• OpenTelemetry database span conventions
• Schema-First Application Telemetry (arXiv)
• B2B IoT SIM cards: solutions for industries and use cases

Recommended

• Top telemetry tools for UK enterprise infrastructure in 2026
• Start With Telemetry: OT Engineers’ SCADA Cloud Migration Blueprint
• Avoid Cardinality Failures: Time Series Database Design for Engineers