Skip to main content
Back to Blog
Automation

Human-in-the-Loop Automation: Where People Belong

202614 min read
Operator reviewing a paused building automation decision

With HITL automation, software performs a series of steps and halts at various junctures to allow a human to take some form of action, such as approving, editing or declining, before the process proceeds. In infrastructure, built environments and physical assets, the debate is rarely whether humans should be involved at all. The real questions are which steps should be subject to human intervention, who should intervene, and how you verify that meaningful oversight is actually happening.

As providers of automation, machine learning and observability solutions for highly critical operations, PODTECH encounters these questions frequently. The framework below reflects how we think about placing people into automated systems without destroying the benefits of automation itself.

Put Human Judgment Where It Matters

PODTECH develops bespoke automation and observability software for critical operational contexts and can help you embed human intervention into the right processes.

Visit

TL;DR

  • A HITL process pauses an automated sequence at certain points, presents the information needed for a person to respond, and continues once they have.
  • Human involvement should be assigned per decision, not applied indiscriminately across an entire workflow.
  • Risk, reversibility and trust in the underlying data should drive the decision. Actions that cannot or should not be reversed will almost always require a person.
  • A simple approve button is not enough. You should measure decision volume, override frequency and the quality of information presented to reviewers.
  • In critical operations, the best HITL arrangement is often to surface the right notifications, supporting data and contacts so that the right person can intervene quickly.

Table of Contents

What human-in-the-loop automation means

According to IBM, HITL is any system or process in which a human plays an active role in the running, monitoring and/or decision-making of an automatic system. In machine learning, this is commonly used to preserve performance and guard against unsafe, incorrect or undesirable decisions.

The term is most often applied to a workflow. As Elastic describes it, a human-in-the-loop workflow pauses at a decision point, displays results to an appropriate individual, waits for their response and then continues. The important point is that the workflow is explicitly paused, not merely observed.

More broadly, HITL describes the position of human control relative to automation. It sits between complete autonomy and complete manual operation. AWS frames this as the placement of HITL between fully autonomous systems and those requiring constant human involvement. SS&C Blue Prism makes a similar distinction through attended versus unattended automation.

In the loop, on the loop, out of the loop

These phrases are often used together, but they describe different operating regimes and should not be treated as interchangeable.

ModeOperating principleApplicable to
In the loopWorkflow is paused and requires a human to provide an output at a specific juncture.Decisions that would be prohibitively expensive or difficult to correct if undertaken incorrectly.
On the loopAutomated system proceeds, with a human able to intervene and halt if necessary.Lower-risk decisions, or cases where it is simple to revert an erroneous course of action.
Out of the loopAutomated system operates largely free from contemporaneous human oversight.Straightforward decisions expected to occur regularly with high certainty.

Moxo makes a similar distinction, arguing that in-the-loop is appropriate where a mistake would create significant cost or difficulty to rectify, while on-the-loop is better suited to lower-risk and readily reversible activities. EY notes that in the on-the-loop phase, humans define target states, guardrails and areas of human decision, while autonomous agents carry out prescribed activities and flag anomalies.

Arize highlights an important caveat: if you place humans directly in the loop, you cap the speed of the system to the speed of those humans. By contrast, on-the-loop supervision can scale only if the supervising party is genuinely able to observe and react in time.

How a HITL workflow runs, step by step

There are many ways to implement HITL, but most follow a similar pattern. Describe Cloud, for example, outlines a sequence including data collection, input creation, ML inference, application of filters and policies, determination of an outcome, and finally execution and recording. Human checkpoints can be inserted at several of these stages.

Typically, the flow looks like this:

  1. Collect the relevant data, classify the situation and return a degree of certainty. Inputs may come from telemetry, ITSM records, content streams or financial activity.
  2. Filter and route based on certainty and risk. The more confident the system is, and the less hazardous the downstream impact, the more likely it is to proceed without intervention. In some designs, the system performs the task first and then evaluates its own certainty.
  3. Compile sufficient context for the reviewer. The human should receive the data, rationale and artefacts needed to make a proper decision.
  4. Send the decision to the right person. Seniority, technical capability and ownership of the relevant asset, account or function all matter.
  5. Enable the required action. The human may approve, modify, decline or provide a different response entirely. The workflow state should be preserved so it can resume cleanly.
  6. Feed the result back into the system. Errors and overrides should improve the automation through retraining, new business rules or better routing logic.
Collect dataScore riskPause workflowHuman reviewExecute taskRecord resultFeedback improves rules and modelsapprove / edit / decline

Two features distinguish a true closed-loop process. First, there should be an SLA for action, with an escalation chain if the task is not handled in time. Second, there must be a fixed record of the inputs shown to the human, the action taken, the timestamp and the identity of the individual who acted.

Deciding where a person belongs

A common mistake is to place a human across every step of a workflow. That simply recreates the existing process with more overhead. The opposite mistake is to remove all humans and expose the business to the long tail of catastrophic errors.

Shore Group captures this well: automation is often highly successful overall, but can still produce a small number of extreme outliers. Without a human at the right points, those outliers may not be detected until the damage is done.

Criteria for human involvement

A useful rule of thumb is this: if an action has low effect, uses trustworthy information, can be undone and has an exit route, then it is a good candidate for full automation. Where two or more of those conditions do not hold, the minimum starting point is usually either a suggested course of action for a human to approve or mandatory intervention before execution.

The key is to assess this per action, not per workflow. McKinsey proposes a similar approach by segmenting actions according to risk and complexity. Low-risk, low-complexity actions can be automated fully, while higher-risk or more complex actions are better supported by an AI co-pilot with continued human supervision.

Levels of human intervention

NiCE groups human-required actions into four buckets: pre-approval, post-approval, exceptions and fully human with AI support. In practice, these map well to operational environments:

ModeWhen to use itExample
Pre-approvalWhere there is significant risk in performing the task.Changing a chiller temperature or cutting a circuit.
Post-approvalLower risk and readily rectifiable actions.Automatically created work orders collated for a team.
ExceptionsHigh-volume, trusted tasks where only unusual cases need review.Standardised notifications and alerts.
With AIWhere expertise is needed to make a judgement or agreement.Investigating faults spanning multiple plant and systems.

Capgemini, referencing NIST, also warns that simply placing a human in oversight of an autonomous function is not enough to guarantee good control. Humans are susceptible to bias and over-trust. A better approach is to define an explicit list of permitted autonomous activities, such as routine low-risk processes, and separate them from areas requiring scrutiny, such as deviations from policy, confidential information or material values.

TechTarget makes a similar point: rules must define the level of autonomy the AI can operate under. Otherwise, users can drift into excessive trust and stop exercising real judgment.

Application to mission-critical environments

Data centres and other facilities

Data centres generate a constant stream of readings from a wide range of plant and systems. Most of these can be ignored, but some indicate a developing issue. The objective is to filter and route the meaningful signals to the relevant discipline.

Examples include:

  • Optimisation actions proposed to the facility and requiring sign-off before implementation through the BMS.
  • Aggregated notifications from BMS, PMS, NMS and related systems to surface discrete events that span multiple platforms.
  • Automatic generation of work orders to the appropriate function, subject to triage.

Of course, this depends on a suitable level of connectivity between otherwise disparate systems. In practice, integrating platforms such as BMS and PMS is often a prerequisite for useful HITL automation.

Works and sites

The need for direct human intervention is most obvious where lives are at stake. An EU-OSHA study describes an American firm deploying intelligent robotic earthmoving equipment while retaining the ability for the driver to take direct or remote control at will. Intelligent obstruction detection and live hazard communication support the operator, but do not replace them.

On worksites, computer vision can identify absent PPE, entry into restricted areas and similar issues. The resulting action should still be ratified and resolved by a designated member of site leadership. This is the right balance: automation detects, prioritises and routes; a human decides and enforces.

Human oversight also matters in commercial building processes. In the pre-sign-off of a construction bid, for example, there are several points where automated checks feed into a final stage of human judgment rather than replacing it.

Factory and production

In factory environments, equivalent controls include operators acknowledging a new production instruction, a QA team reviewing and clearing a product run, or a works manager authorising a revised auto-generated maintenance regime.

Research into Digital Twins identifies four key actors in industrial settings: Observer, Analyst, Decision Maker and Action Executor, spanning multiple degrees of automation for each. Understanding these distinctions is essential if you want to deploy the right level of automation across a facility rather than applying a single blanket policy.

Key considerations from academic work

A 2026 paper in PMC classifies HITL according to position in the control loop, level of human involvement and temporal scale. It identifies three major areas of difficulty: scalability, human mental effort and establishing the right level of trust in the loop.

  • Throughput is limited by the number of available humans. You should identify the control actions where it is prudent to constrain the rate.
  • Mental effort rises when users are presented with too many queries, leading to approval by inertia. Lower-consequence items should be grouped or removed.
  • Level of trust must be calibrated. Humans need transparency around predicted certainty and supporting rationale so they know when the technology can be relied upon.

Describe Cloud outlines a similar hierarchy from fully automatic through to single-point triage and more involved review patterns. The practical lesson is straightforward: the more critical the asset, the less sensible it is to think only in terms of “automate or don’t automate”. The real design task is to decide how much human control belongs at each point.

Designing and measuring your own HITL workflow

When deploying automation to control vital assets, we find that it is usually correct, but can fail catastrophically on the few occasions that it is wrong. For that reason, we start by defining the required level of human intervention before deciding how to automate the remaining functions.

In practice, that means establishing which parties are empowered to make the relevant decisions, what data they need to support those decisions, and how much lead time is required to complete the task. We then integrate outputs from the various control layers, such as BMS, PMS, NMS and plant sensors, so the individual can assess the situation and act from one place.

Requiring staff to consult multiple terminals undermines the value of placing them in the loop. A reviewer should not have to reconstruct the context manually from disconnected systems.

To assess whether your HITL design is working, monitor the following:

  • Volume of decisions routed to each reviewer or team.
  • Frequency of overrides, edits and declines.
  • Time to action against the SLA and escalation path.
  • Quality and completeness of context presented to the human.
  • Downstream outcomes, including whether the intervention prevented poor execution or simply added friction.

If reviewers are consistently able to click approve without meaningful consideration, that is not a sign of success. It may indicate that the intervention point is poorly chosen, the threshold is too low, or the process has become ceremonial rather than genuinely controlled.

Perspective

Having supported more than 250 initiatives and underpinned our managed services with 99.9% availability, we are well placed to embed these processes for clients looking to maximise performance. Our automation and machine learning practice enables large-scale volumes to be processed automatically while leaving higher-risk judgments to suitably experienced personnel.

Importantly, the goal is not to reduce headcount for its own sake. The aim is to free people to apply their skills where those skills matter most.

Harry

Implement the solution with PODTECH

We can help you decide where to automate and where an engineer should remain involved. For this, we offer an enterprise automation solution including approvers, escalations and record-keeping. Our ML Development function can develop the prediction and detection models behind the workflow.

Alongside DCIM, we also provide consultancy and integration for other data centre assets, including building management and electrical systems. Head to podtech.com and tell us which activity you want to run more safely.

FAQ

What's the distinction between "human-in-the-loop" and "human-on-the-loop"?

Human-in-the-loop halts the process and requires input from a person at specific points in the flow. Human-on-the-loop allows the process to continue while a person supervises and can intervene if required. In general, in-the-loop is preferable where there is significant risk or cost associated with executing an action.

Will requiring a human to approve things reduce the speed of my automation?

Yes, but only at the points where approval is required. If you place those checkpoints selectively, especially around higher-impact or lower-certainty activities, you can preserve good throughput. SLAs and routing rules also help prevent the workflow from blocking unnecessarily.

Are there any activities I should avoid automating entirely?

Where the risk and consequence of executing an action is sufficiently high, or where accountability must remain explicit, you should retain some level of human intervention. As GitLab notes, decisions relating to other people, such as recruitment, appraisals or granting privileges, should remain with a human even if an automated assistant is used. In physical environments, many location- or asset-based actions fall into the same category.

How will I be able to assess the effectiveness of having humans involved?

Monitor the volume of activity sent to reviewers, the number of overrides, the time taken to act and the quality of information presented. If reviewers can repeatedly click approve without real consideration, you should re-evaluate whether the intervention point is meaningful.

Sources