Why wait to catch issues at end-of-line testing?

End-of-line testing in manufacturing refers to the quality assurance processes conducted at the final stage of the production line to ensure that products meet the necessary specifications and quality standards before they are shipped to customers.

However, new Quality 4.0 technology like machine learning and artificial intelligence now make it possible for manufacturers to detect defects much earlier in the process, before units reach the end of the manufacturing line. By addressing manufacturing defects and failures early on in production, engineers can stop relying on end of line testing alone to detect quality issues. Catching issues earlier results in a more efficient use of materials and resources.

This blog will dive deep into end of line testing, focusing on the benefits it provides, the challenges it poses for manufacturers… and the emerging new alternative to relying on end of line testing alone.

Why end of line testing matters

To ensure the quality of their products, manufacturers need to separate defective or out-of-spec units from those units they will ship to their customers. End of line testing ensures that the products meet the design specifications so they will perform properly and satisfy consumers.

Depending on the industry, end of line tests may also need to ensure they conform to standard regulations that ensure safety and longevity. For example, pacemakers are a common medical device that must meet manufacturing standards in order to leave the factory.

Many manufacturers in different types of manufacturing rely on end of line testing as the end-all be-all when it comes to quality control. They put all their resources to check product quality at the last possible moment before packing and shipping.

Types of end of line tests

Every product has vastly different requirements for end of line testing, depending on the method of manufacturing, the intended use of the product, the cost of materials, and many other factors. These are some of the common types of end of line tests you will find in a manufacturing plant:

• Functional Testing: This type assesses whether the product functions as intended. It involves applying various inputs and measuring outputs to ensure that all operational requirements are met. This test can cover a wide range of functionalities depending on the product, from simple mechanical operations to complex electronic functions.
• Performance Testing: This type evaluates the performance characteristics of the product, such as speed, efficiency, and responsiveness. It ensures that the product achieves the performance criteria set out in its specifications.
• Electrical Testing: For electronic and electrical products, this testing ensures that all electrical components and systems operate correctly, are safe, and meet the relevant standards. It can include tests for continuity, voltage, and resistance.
• Leak Testing: This is critical for products that must be airtight or watertight, such as containers, pipes, and some types of machinery. The testing checks for any leaks that could lead to product failure or safety issues.
• Visual Inspection: This involves a detailed examination of the physical appearance of the product. Inspectors look for defects, damage, or irregularities in the materials or assembly. While it can be performed manually, automated visual inspection systems are increasingly common.
• Safety Testing: This type of testing ensures that the product complies with relevant safety standards and regulations. It is particularly important for electrical appliances, vehicles, and machinery.

Three examples from automotive manufacturing

Automotive manufacturers who produce complex vehicle parts such as engines and transmissions have great examples of end of line testing. These vehicle parts are costly to build with many engineered subcomponents and strict safety and performance specifications to adhere to. These are three examples of specific end of line test protocols:

1. Hydraulic testing which provides valuable data on the rotary and linear motion of components and valves. Hydraulic testing uses a range of sensors that are controlled by software to detect fluctuations in pressure, flow rates, and leakages on parts and assemblies.
2. Roll brake testing involves simulating real-world road conditions on a mechanical platform to measure a vehicle’s braking functionality. This dynamic verification and diagnostic system provides accurate and repeatable test results in a controlled environment to deliver vital braking data for EOL quality control analysis.
3. Noise, vibration, and harshness (NVH) testing is done using an array of sensors, including accelerometers, microphones, and force sensors.  Noise, vibration and harshness testing is commonly used on vehicle components and subassemblies such as engines, transmissions, and suspension systems. The information collected is uploaded to an automotive quality testing solution and converted into digital data for analysis. For example, noises generated by geared assemblies can indicate manufacturing defects which can lead to passenger comfort problems or assembly failures that can risk passenger safety.

Limitations of relying on EOL testing

1. End-of-line testing is time-consuming and costly

While there will always be expenses associated with ensuring manufacturing quality, the costs of EOL testing in manufacturing tend to be particularly high. These expenditures are incurred from purchasing specialized equipment to conduct the end of line tests and extra staffing to execute and monitor test stations.

For example, completed vehicle engines are hot-tested on an engine test bench as seen below. The objective of the test is to check that all of the engine’s operating parameters are performing just as they would in a vehicle. Some engines may be tested for 20 minutes, not including the time spent setting up each unit. In an industry where seconds of cycle time count, 20 minutes of testing time can drastically limit the production capacity of a facility.

2. Waiting until the end of the line is inefficient

In an ideal world, maximum efficiency for a plant would mean 100% of all parts produced met specifications and passed all quality evaluations. This hypothetical plant would be operating at the highest level of efficiency, where all inputs equalled outputs. Optimizing throughput efficiency is an important goal for any manufacturing plant and could mean the difference between a viable operation or a closed plant.

When evaluating the effects of EOL testing on efficiency, it is important to look at the larger impact of what producing lower quality parts can have on downstream costs such as warranty issues and recalls. If the sole objective of EOL testing is to minimize the risks of warranty issues, then the test only needs to cost less than what the warranty issues would in order to make the test worthwhile. The problem with this scenario is that it doesn’t take into account all the other potential outcomes of EOL testing.

For example, when an EOL test identifies a defective unit, the manufacturer faces three choices:

1. Scrap the unit
2. Rework the unit
3. Test the unit again (and pray)

The end of the line is the costliest point in production at which a manufacturer can scrap or rework a part or assembly, since the maximum amount of energy and materials have already been expended to build it. Scrapping or reworking after a final test is highly inefficient.

The third option is not that uncommon, but it introduces even more uncertainty and efficiency into the quality control mix, not to mention production delays and increased costs.

3. End-of-line testing does not prevent quality spills

One might think that the cost of EOL testing and the potential for reduced efficiencies are outweighed by the security of having a final check that minimizes the risk of shipping defective products. However, anyone who has been working in a plant long enough knows that end-of-line testing is not foolproof.

In an AIAG & Deloitte Report, OEMs and Tier-1 suppliers in the automotive industry ranked problem solving as one of the most critical issues impacting quality. Additionally, both parties cited the lack of root cause analysis as one of the main reasons for this issue. While EOL testing can point to the existence of a problem, it cannot explain why it occurred or how to fix it. In other words, EOL testing may identify defective parts, but it cannot improve first time yield on its own.

Many manufacturers take a reactive approach to quality control. They take action only after a defective unit has already been produced.

Take the engine hot test example, where an engine is completely assembled, tested, and then classified as either having passed or failed. This “wait and see” method can be costly, especially for those manufacturers who build complex components that are expensive to process. Rarely is it cost-efficient to scrap a failed engine, and it is usually quite challenging to rework. Even if only a few engines are scrapped or reworked, testing every unit that comes off the line slows down the entire quality control process because the tests take so long to run.

What if there was a solution that could provide insights into why these engines are failing the EOL test in the first place, and could pinpoint at which stage of production the problem presented itself?

What if this solution could flag issues earlier in production, so that manufacturers could intervene sooner and prevent a quality event?

Identifying quality issues before the end of the line

Manufacturers generally expect some number of units to be defective, and they rely on EOL testing to catch these failures. But there are questions that engineers need to ask that cannot be answered at the end of the line, including:

• Which operations are causing end of line test failures?
• Which signals from the operations are most relevant to the failure?
• What was different about this unit from the others that caused it to fail?
• Where is the best corrective action to take to avoid potential future failures?

Quality 4.0 has ushered in a new wave of technology and new solutions to help solve these problems using machine learning and data analysis.

By supplementing end of line testing with a predictive quality solution, manufacturers can reduce their reliance on EOL testing. During production, algorithms in this type of software can accurately identify which units would pass the end of line test and have them bypass the EOL test altogether. Units identified to fail could be scrapped or reworked immediately, without having to wait to be tested to confirm that they are defective.

Preventing quality issues before the end of the line

End-of-line testing is still used because many products are complex with many points of potential failure. With an EOL test, units pass if they fall within specification limits, or fail if they don’t. But, if a manufacturer collects information on what went into making any given part, they can analyze their data with a predictive quality solution to predict how likely it is that a component will pass or fail an EOL test. This eliminates the need to test every part coming off the line.

Imagine a production line with three operations. After a unit comes off the line, it gets tested on how well it performs. A few of those units will fail the EOL test. Because the manufacturer knows that some parts will fail, it has no choice but to test every assembled unit to ensure inevitable defects are caught. This creates inefficiencies on the line.

If the manufacturer were to introduce a predictive quality analytics solution, an EOL test would only need to be conducted when there is reason to believe that a part is defective, or when it’s likely that the EOL test will identify a missed defect. Using machine learning, a predictive quality management solution can analyze data across multiple systems to quickly detect anomalies in machine signal data that differ from their normal behaviour, that are known to lead to quality errors.

The reason that a part is flagged as potentially defective could be an anomaly in operation two, or signals received are within current specifications individually but are a little too high or low across all three operations. Different operations impact the product in different ways, so the specific indicators of a defect vary from line to line as the series of operations changes.

If a manufacturer can analyze information from all operations using a predictive quality solution, they can predict which parts are likely to pass or fail an EOL test and only test parts that have a high likelihood of failing. Beyond eliminating the need to test each part coming off the line, collecting data from all upstream operations can also help engineers to identify where problems occur. This provides context when a defect is detected in EOL testing, enabling engineers to quickly understand the root issue.

Benefits to reduced EOL testing

Detecting potential quality issues during production can help manufacturers reduce the need for EOL tests, which would benefit them in a number of ways, including:

1. Minimizing warranty claims

In 2019 automakers paid out $45.9 billion on warranty claims, which equates to approximately 2.4% of their production revenue. Manufacturers in the automotive industry lose hundreds of millions of dollars to escaped warranty units every year. By implementing a predictive quality platform they can ensure that fewer defects slip through the cracks.

Since machine learning models can map the underlying structure of data and analyze the relationships between machine signal data on the line, test data, and any other data inputs that affect quality, they can detect issues that other quality control systems such as statistical process control might miss.

2. Lowering operating costs

Usually, a manufacturer’s production schedule is designed with lost time and money in mind. This is due to the inevitable retesting, scrapping, or reworking of parts, as a result of testing every single unit coming off the line. If every assembly is being tested, a full-time person must be assigned to each testing station. This individual needs to maintain a high level of expertise that requires ongoing training to ensure up-to-date skills, especially when producing complex products that require sophisticated testing.

Maintenance and unforeseen downtime due to unexpected calibrations or maintenance overhauls associated with EOL testing can also slow down production. By supplementing EOL testing with predictive quality analytics, manufacturers can free up human resources and reallocate them to areas of the business that drive profitability.

3. Improving throughput and first time through yield (FTT)

End-of-line testing creates a bottleneck. When every unit is tested, all upstream operations on the line have to be stalled. When fewer units need to undergo EOL testing, less time is required for the entire end-to-end assembly process. Manufacturers can increase throughput and FTT by supplementing EOL testing with predictive quality analytics.

Furthermore, predictive quality analytics can help increase the overall quality of production, resulting in fewer EOL test failures. Since manufacturers can forecast when a unit is likely to pass, high-quality units can bypass the end-of-line test, reducing the number of false positives.

Although there are certainly benefits to reducing EOL testing, it shouldn’t be removed from the manufacturing process altogether. It makes more sense to cut the number of components that need to be filtered through EOL tests, while also improving operations on the line to make better quality parts in general.

The future of quality testing

Manufacturers can now reduce their reliance on EOL testing and improve their product quality and throughput efficiency with a predictive quality solution like Acerta’s LinePulse. Read this eBook “End-of-line testing is good… but predictive quality analytics is better” for more detail on how it works.