Augmented Reality (AR) in Industrial Heat Treatment

Remember physical paper maps? These were not easy to use and navigate while driving, and that’s why we tended to rely on our front-seat passenger to help guide the way or memorized the directions to our destination. These paper maps soon evolved into digital maps on specific GPS devices, and now everyone has a GPS navigation system available on their smartphone. These systems allow the driver to take back full control of navigation … but not without the occasional wrong-turn (and the GPS quickly pointing out our mistake).

More recent advances put the instructions in the driver’s line of sight, commonly referred to as heads-up displays (HUD). HUDs are a typical example of augmented reality (AR), where we overlay digital information on the real-world, which drastically cuts down on mistakes made with premature or late turns.

Parallel this to heat-treat operations. Within a couple of years of starting my heat-treat career (over 30 years ago), I moved from manual paper cards describing the process cycles in written detail to using programmable controllers, which automatically controlled the full setpoint cycle from start to finish. There was not much feedback about the cycle status from the control units – just digital readouts from the display indicating temperature and atmosphere levels. Paper had not completely gone away at this stage, and the chart record of the process was still in paper format.

The equivalent of a “wrong turn” tended to be selecting the wrong cycle/program number (in early days this was via a thumbwheel), or the chosen program was correct but the program’s detail had been altered for a specific run and not returned to the normal setpoints. An operator rushing to sign off the chart could miss this detail, and the issue could go unnoticed until ultimately picked up by final inspection. The paper chart was then examined to assess what went wrong.

In the mid-2000s, paper charts evolved into network-accessible digital charts, and the quality team could then monitor cycles in near real-time. This online access also enabled quicker checking of cycle details of suspect batches. Some equipment OEMs were also upgrading the technology on furnace control panels with dedicated HMIs (human-machine interfaces). These had been used extensively in other manufacturing industries to improve the quality, productivity and safety of a process.

These predominantly Windows-based HMIs started using the full graphic capabilities of the processors available at that time. From 3-D imaging to exhausting the color palette, HMIs could certainly light up the control panel. However, we still had issues with incorrect cycles, altered cycles and unexplainable equipment faults. A period of double-checking critical program entries with supervisor sign-off was possible before the economic collapse in 2008-2009. After this period, most heat treaters had to operate in a much leaner manner with a reduced team.

From the 2010s, the design of HMIs started to evolve and pick up a more situational-awareness graphic style. Research primarily in the oil-and-gas industry highlighted programming errors, and incorrect operator inputs accounted for 40% of failures in control systems.^[1]
It concluded that training, software testing, strict control of programming changes and design should help minimize these failures.

The pretty graphics were getting in the way of an operator understanding what was going on in the process. There needed to be a redesign of screens to reduce complexity and highlight critical elements that needed action. A typical example is looking at an HMI for something most of us usually use daily (Fig. 1).^[2]However, it is more common for this design to show just the critical information that drives proper control (Fig. 2).

Situational awareness design identified three levels.^[3]

Level 1: Perception – What is the current condition?
Level 2: Comprehension – What does that mean? What are the critical issues?
Level 3: Projection – Do I need to act now or soon, and what will be the likely outcome?

We now tend to focus on Level 1 and 2 in our control systems and HMIs and use operator training to help with Level 3. Losing some of the HMI detail does reduce the context and puts pressure back on training.

At this point, we are still in the GPS world with HMIs – looking down or sideways to view the information and (even only for a split second) taking our eyes off the road. The added issue is the mental activity necessary (termed cognitive workload) to interpret the detail on the GPS and evaluate when the next turn is. We then return our eyes to the road and take appropriate action.

AR can support. AR is, in effect, the HUD (heads-up display) for a heat treater. AR is not a replacement solution for the control system or HMI but an additional view into your operation. It focuses on situational awareness Levels 2 and 3 to help minimize errors and support timely action on equipment faults.

Human-Factors Engineering

If you dive into the detail, there is usually more than one contributory factor that leads to an error in heat treatment, and it’s not just as simple as training and experience. A way to examine this is with human-factors engineering, which looks at the interaction between operators, equipment and the environment. AR complements this by improving the way someone can easily interact with equipment and their environment.

Along with aiding process information (previously discussed), there are three other areas where AR can support the heat-treat operation.

Training

Heat-treat training is typically event-based – in the classroom or on-the-job. In classroom training, you tend to pre-allocate time slots during a day, require an instructor and conduct the training remotely from the shop floor. Studies show this type of training is not very useful for knowledge retention over time unless backed up by reinforced learning. Within 30 days, you can forget the majority of what you learned (Fig. 3).

On-the-job training is usually more successful because of regular ongoing learning, so you have built-in reinforcement of the training details. This type of training is in the shop-floor work environment with familiar equipment. It does tie up the instructor (lead operator or supervisor) during the training period, however, and success is highly dependent on the trainer’s skills and presentation.

AR training complements on-the-job training because it can be done on your own, during normal work activities, in a short burst of time and in the shop-floor work environment. It can also reinforce learning to help retain knowledge over the longer term.

Instructions

When a complex task is performed infrequently, even the most experienced workers may forget the steps required or the order in which specific steps much be performed.

For instructions or procedures to be adequate, you must use them. There are some reasons why this might not be the case:

Instructions are not readily available.
They are in the wrong (operator) language.
They are inaccurate. Typically, if <80% accurate, they won’t be used.
Instructions are out of date.

From an industrial study, 90% of “accidents” have at least one root cause related to mistakes within procedures, and an audit showed procedure accuracy ranged from 30-95%.^[1]AR instructions can help this situation. They are readily available on a smartphone or tablet. Many of the offers on the market include options for language translation. They are tested in a live working environment, and inaccuracies are identified and updated. Electronic record control ensures you have the latest issue.

Available Time

The worst enemy for quality of performance is available time. Tight deadlines or unrealistic measures encourage the behavior to “just get it done” and “check the box.” Shortcuts are taken, and mistakes are made and not caught. Plus, standards are not followed, variation increases, fatigue and stress levels increase, and ultimately accidents or errors happen. The current environment tends to push toward smaller teams. If you add in transient workers, you eventually end up with fewer resources with additional tasks and added training needs.

AR training and instructions help address this problem by cutting down time-draining issues involved with retraining and the accuracy of instructions. Added to this are digitized notes available in the context in the right time and location and access to remote experts through the AR application. This results in time-saving, efficient communication and problem-solving.

Next Steps

2020 has been a challenging year, and many businesses have focused on cash flow and costs to survive this period. Restarting furnaces to meet demand and controlling costs have been a focus. Critical maintenance and optimizing spend on regulatory compliance have been vital.

Following this stage, companies are starting to look at the next steps. We see this first with transformation – gaining efficiency and flexibility through digital solutions. Connected workers utilizing AR and other tools can help smaller teams with transient workers navigate these times to figure out how to do more with less. At the same time, process mastery can be used to secure the core operation.

Looking further into the future, we believe heat-treat operations need to be flexible, connected plants. Actions are necessary to enable risk mitigation and smart supply chains to help build resiliency to future potential risks, including pandemics, cyberattacks and supply-chain vulnerabilities. Ultimately, digital should be leveraged to its fullest extent to achieve self-organizing production.