Skip to main content
Back to Blog
OT Security

OT Patch Management: A Practical Guide for Critical Infrastructure Teams

November 202616 min read
Technician servicing an industrial controller during a planned maintenance window

OT patch management involves the process of identifying, evaluating, testing and implementing software and firmware updates to operational technology with zero interruption to the physical process being controlled. It's not just IT patching taking twice as long. It's a completely different decision-making model: every patch decision is balanced against uptime requirements, safety ramifications and vendor support capabilities, and no changes are allowed to reach production until they've been thoroughly tested on an exact replica of production hardware and software. This guide will walk you through how to build that process from the ground up, how to deal with systems that can't be patched, and how to ensure your patching process can stand up to a change-control audit.

PODTECH Custom Software and Automation for Critical Infrastructure

PODTECH specializes in creating custom software and automations tailored for critical infrastructure needs. This encompasses legacy modernization efforts and seamless integrations with building management systems. podtech.com

TL;DR

  • Patch according to risk, not calendar dates. CVSS is just one metric — consider threat vector, asset criticality and cost of downtime.
  • Test off-line first. Use test hardware that duplicates operations, and verify that the patch can be reversed.
  • Deploy in planned maintenance windows with a tested rollback and complete backups ready.
  • Where you can't patch, put compensating controls in place. Segmentation, least privilege permissions and application allow-listing drive risk down to acceptable levels.
  • Think of every patch as a change. Approval, testing and records provide the audit trail you want for SOX-style change control.

Table of Contents

What OT patch management is, and why it differs from IT

The IT default response to issues is patch and reboot. The OT default response to issues is to not touch running systems unless you have a good reason to. The reason is simple, those systems control physical processes.

The US Department of Homeland Security advises that one single solution is typically not sufficient for patch management across IT data networks and ICS, due in part to the fact that unexpected downtime of control systems can lead to operational impact. This means patches must be thoroughly validated before use. Source: Control Systems Security Guidance - Patch Management.

It goes beyond servers and laptops too. NIST SP 800-82r3 provides guidance on OT security throughout industrial control systems, building automation and transportation systems, considering requirements for performance, reliability and safety. This includes:

  • PLCs, controllers and their firmware
  • SCADA servers, HMIs and historians
  • Engineering workstations
  • Network devices and gateways
  • Building and facility systems such as BMS and power monitoring

For example, the ISA's technical report ISA-TR62443-2-3 covers patch management, but focuses specifically on industrial automation and control systems. It highlights asset owner concerns and product supplier concerns, as well as ways inadequate patching can hurt reliability or operability. Essentially, patching poorly is treated as an availability risk by itself by the standards world, not merely a security issue.

Three differences that shape the process

  1. Availability trumps all. If you need 99.999% uptime like many industrial networks do, then the DHS guidance suggests your patch window is roughly 5 minutes and 35 seconds annually. Unscheduled patching is a non-starter at that level.
  2. The vendor decides what is supported. Support packages are often certified for particular OS versions and patch levels. If you update to something outside that bundle, you risk your support status.
  3. Windows don't happen often. Updates to control environments often only happen during scheduled maintenance windows, typically quarterly or bi-annually. Many industrial protocols were also built years ago without authentication for uploading logic or firmware.

Why patching OT is risky

It's a double-edged sword. Failure to patch leaves an existing vulnerability exploitable. Patching poorly can halt a process, alter timing between components or void a certification.

ICS security expert SANS Institute's Dean Parsons explains that best-of-class organizations don't try to patch everything. They schedule updates during planned engineering maintenance windows based on operational requirements and risk analysis to operations, because OT environments contain unique attack surfaces, systems, and impacts when things go wrong.

Practitioners have also highlighted that sector context is important. In chemical processing for example, it might be common for a reactor to operate for extended periods without any controllers being touched. This is because the sensors and actuators they control cannot be stopped mid process. An interruption there can lead to serious safety, environmental and compliance consequences.

Industrial Cyber explains ISA/IEC 62443 and configuration management by stating the principle simply: even good security practices can lead to disruptions if not tested and validated. Scheduled windows, proper testing, and quick rollback are mitigations it recommends.

Step 1: Know what you are patching

You can't scan a hole that you can't see. Perform and maintain an inventory pre-patch decision that documents for every OT asset:

  • Hardware model, firmware and operating system version
  • Installed software and the vendor's certified patch level
  • Network zone and connectivity, including remote access paths
  • The process it supports and what happens if it stops
  • Owner, maintenance window and spare availability

This is as much a configuration management issue as it is a security one. Having a configuration management database (CMDB) that connects assets to the services they provide allows you to answer the question that's important during triage: if this device goes down, what stops?

Passive discovery is often the safest method of beginning enumeration in OT, as active scanning can crash delicate controllers. Correlate what you see on the network against engineering drawings and physical walk downs.

Step 2: Prioritise by risk, not by score

SANS doesn't beat around the bush on this topic: CVSS scores alone don't cut it anymore. Prioritizing strictly by severity rating could result in unnecessary downtime or safety hazards with minimal security benefit. Timely and effective mitigation is based on threat vector, criticality of the asset, potential for downtime, and the current threat landscape.

A workable scoring model weighs two sides.

Adversary opportunity if you do not patchEngineering impact of patchingPatch at next major outagePatch in next windowAccept and documentCompensating control nowMedium exposure, high deployment costCredible exploit, vendor-approved patchLow exposure, strong isolationExposed asset waiting on qualificationLowHighLowHigh

Engineering impact of patching

  • Downtime and production disruption
  • Chance of a failed deployment
  • Maintenance schedule coordination
  • Vendor certification and support

Adversary opportunity if you do not

  • Is the asset reachable from IT, the internet or a vendor connection?
  • Is there a known, working exploit?
  • Can an attacker pre-position and cause follow-on impact?
  • What are the safety consequences if the asset is compromised?

The outcome of this scoring is one of four decisions:

  1. Patch in the next window. High exposure, credible exploit, vendor-approved patch available.
  2. Patch at next major outage. Medium impact, expensive to deploy, mitigating controls present.
  3. Implement a compensating control now, apply patch later. Common with exposed assets waiting on patch qualification.
  4. Accept and document. Low exposure, high isolation. Residual risk accepted by asset owner.

Document it. We decided not to patch with no details looks like an omission if an auditor or incident investigator asks why later.

Step 3: Qualify and test the patch

The DHS recommended practice makes this explicit. It calls for dedicated test hardware and a production-like environment. Testing should verify that the patch:

  • Fixes the problem it was meant to fix
  • Does not conflict with coexisting applications
  • Can be removed without affecting operations
  • Leaves the program operational, demonstrated by successful execution of system operational acceptance test prior to return to service

It recommends checklists, procedures, and logged records of patch, test and configuration changes as well. Recommended Cybersecurity Practices for Industrial Control Systems from CISA reinforces this requirement: always test patches in off-line environments before implementation.

The test bench doesn't need to emulate an entire plant. It just needs the same models of controllers, same firmware, same OS images and same critical paths of communication for what's being tested. Virtualise what you can't afford duplicate hardware for. A virtual SCADA/historian layer, and a backup controller of each type covers most bases.

A pre-deployment checklist

  • Vendor has approved the patch for your product version
  • Patch integrity verified by hash or digital signature
  • Tested on the bench, including communication timing with neighbouring devices
  • Backup of the target system taken and restore tested
  • Rollback steps written, with a defined point of no return
  • Operations, engineering and safety sign-off recorded
  • Window booked and the process state confirmed safe for the work

Step 4: Deploy in a window, with rollback ready

The ISA's top recommended OT patch management best practice includes maintaining a rollback plan with full, tested backups before patching critical OT assets. Other recommendations include developing a baseline of patches across OT systems and reviewing patches that are already installed both before and after changes are made.

For deployment, a staged approach limits blast radius:

  1. Patch the least critical instance first, such as a standby or non-production twin.
  2. Verify service health and safety-function behaviour against written acceptance criteria.
  3. Proceed to the next tier only when the previous one is stable.
  4. Assign the rollback decision to a named person along with the deadline by which the decision must be made.
  5. Update the asset inventory and baseline immediately afterwards.

That ISA source also says patches should be deployed through the same change management process as any other OT change. Baseline. Record. Review. Document. And rollback, if needed. If your organization has a formal management of change (MOC) process already running, patching should be incorporated into it, not run on parallel paths.

Patching air-gapped and segmented OT

Isolation shifts the delivery path, not the rules. ENISA, who has done great work on patching SCADA systems, recommends that patch management systems reside inside of a hardened, properly isolated enclave. Some of their guidance:

  • Testing patches for malware and archiving them to read-only media
  • Installing two patch systems, one for downloading patches in isolation, and one for deploying patches that have been air-gapped
  • Using digital signatures or hash verification on every patch

Translated into an operating routine:

  1. Download vendor patches in a staging zone outside the OT network.
  2. Verify signatures and hashes, and scan for malware.
  3. Validate on the test bench.
  4. Transfer by controlled, logged removable media, and re-verify on arrival.
  5. Deploy in phases during the authorised window.
  6. Check installation, service health and safety-function behaviour, then log the result.

Removable media is an attack vector. Control of removable media is a patch governance issue: who is allowed to carry it, how it is scanned, how it is wiped, and where the log is maintained.

When you cannot patch: legacy and unsupported systems

Others can't be patched, period. According to the New York State Comptroller's industrial control systems handbook, many systems cannot easily be taken offline and then brought back online without impacting production. Additionally, patches may not be available for legacy components.

In fact, the ISA tells you this outright. If you are unable to install a patch due to incompatible hardware, lack of vendor approval, or failed testing, mitigate the risk by applying alternative controls to an acceptable level. Some of those listed are:

  • Disconnecting the asset from the business LAN or DMZ
  • Restricting user permissions
  • Application allow-listing or baselining

According to the above article, patch intelligently: not every OT asset can or should be patched. Weigh the price of patching versus the price of alternative controls. Document the exemption, compensating control and review date for each asset.

Where exposure is a chronic condition for your systems, the longer-term solution is decommissioning. This could involve encapsulating a legacy controller with an integration layer, replacing an obsolete OS platform or migrating supervisory functions to supported software. Our legacy modernisation services are designed for these scenarios, and see our system integration case study for a real-world example.

Firmware and connected devices

It's not just Windows hosts that need patching. Sensors, gateways and controllers deployed in the field have firmware that requires the same level of discipline, and often include a remote update mechanism. If you have an installed base consider reading our brief on Controlling OTA firmware updates before you turn on remote updates. For a broader hardening effort in conjunction with patching, review our Industrial IoT security roadmap which provides a phased 3 to 12 month plan.

Patch management as change control: the SOX angle

When people ask about SOX IT change control, they typically have one question in mind: what documentation will an auditor want to see for a change to a system? The answer is true for OT systems wherever an OT or building system feeds information to financial reporting systems or otherwise falls within audited scope. This can include metering and billing data, inventory systems tied to production systems, or facility systems that support data centres within scope.

PCAOB's AS 2201, Auditing Standard for internal control over financial reporting, considers effective IT general controls to impact the degree of reliance auditors place on automated controls. Change management is one of the typical disciplines that fall under those general controls. If a particular OT system is in scope for your organization is a question for your compliance team and external auditor, not something you should assume.

What a defensible change record looks like for a patch:

Control objectiveEvidence for a patch
AuthorisedChange request with named approver, separate from the implementer
TestedBench test results and acceptance criteria met
DocumentedVersion, date, target assets, before and after baseline
ReversibleTested backup and rollback plan
TraceableLog of who deployed what, and when

This table lines up perfectly with the ISA's baseline, record, review, document and rollback workflow. If you build your OT patch process around it, you'll inherently create an audit trail by doing the job right, rather than adding more work.

Perspective

It's no different in data centres, buildings and industrial sites: patching fails when it becomes siloed as a security responsibility of one team.

On critical infrastructure the patch decision impacts operations, engineering, safety and compliance all at the same time. The teams that handle it best have three traits in common. They know their assets, they test against a platform that mimics production, and they document every decision.

That's as much a software issue as a procedural one. Asset lists, change documentation, telemetry evidencing service health prior to and following a change, and interoperability between BMS, PMS and NMS layers are what enable a team to patch with confidence and validate that afterwards. Our focus on datacenter telemetry, system integration and modernization of legacy solutions centres around that philosophy: visibility first, structured change second, and less overall systems that no one is afraid to touch.

When you can't patch a system today you want a plan that eliminates the exposure on your timeline.

Harry

How PODTECH supports controlled OT change

We develop bespoke enterprise software and integration for critical systems and infrastructure backed by a 99.9% uptime SLA, with over 250 projects successfully delivered. When it comes to OT patch programmes, this normally involves:

  • Visibility: asset state and service health telemetry and monitoring with datacenter services and BMS and PMS integration.
  • Workflow: enterprise automation software to automate approvals, change records, maintenance scheduling and much more, customized to your environment.
  • Exiting from legacy risk: obsolete systems that cannot be patched safely anymore can be modernised.

To discuss a patch management or change-control project, talk to our team.

A 90-day starting plan

If you don't have an OT patch process today, start small and create repeatability.

  1. Days 1-30: Inventory the assets within your most critical zone. Document vendor-reported patch levels and network segmentation paths.
  2. Days 31-60: Agree scoring model and the categories for decisions. Name approvers. Configure staging area and test bench for one asset class.
  3. Days 61-90: Run one full cycle on a low risk asset: assess, test, approve, deploy, verify, document. Look at what your record looks like to an auditor, fix gaps.

Tested segment by segment. Something that works for one asset class is better than a blanket policy on paper.

FAQ

How often should OT systems be patched?

There is no one-size-fits-all interval. Many control environments can only be patched during scheduled maintenance windows, which can come once every quarter or two. Define cadence based on risk and window availability. Implement band-aid compensating controls as needed between intervals.

Is CVSS enough to decide what to patch?

No. SANS has stated that using CVSS by itself can cause downtime or safety incidents with minimal security gain. It should be used in conjunction with exposure, asset criticality, exploit availability and the operational impact of the remediation.

What if the vendor has not approved a patch?

Don't use it in production. Apply compensating controls like segmentation, limited permissions and application allow-listing in the meantime, as suggested by ISA. Document the waiver and review it when the vendor issues a justified update.

Does SOX apply to OT patching?

Only if an OT or building system falls within scope of financial reporting controls. The scope is determined by your compliance team and external auditor. If it falls within scope, expect auditors to be looking for authorisation, testing, documentation and traceable deployment of each change.

Should we patch OT assets as soon as a patch is released?

Yes, but with caveats. Validate vendor credentials, ensure it tests properly in a lab environment, verify rollback and install during a maintenance window. Hot fixes are generally only for high value targets which are being actively exploited and usually layered with compensating controls first.

Sources