Reducing signals that required manual investigation for root cause analysis by 99.75%.
- Reduce warranty claims and accelerate RCA for faulty transmissions by adding an abnormality score to EOL testing
- Identify 4 failed units from an unlabelled dataset of 50 based on abnormality score alone
- Training data was drawn from only 100 units, none of which failed EOL tests
- Models were trained exclusively on data from normal (i.e., non-defective) units
- Solution needed to deliver real-time results as part of a new EOL test
- Highest abnormality scores assigned to the 4 failed units in the test dataset
- Reduced number of signals requiring manual investigation for RCA by 99.75%
- Costs from warranty claims reduced by up to 30%
A leading Tier-1 transmission supplier based in Japan was looking to reduce warranty claims by identifying manufacturing defects using production and end-of-line (EOL) test data. The EOL test involved more than 100 steps for a variety of performance-based quality assessments, ultimately analyzed by an SPC program. However, the program only analyzed 10% of the data collected from each unit, requiring an engineer to inspect all the data manually in cases of potential failure.
The client requested a machine learning model that would automatically assign an abnormality score to each transmission as part of its EOL test. The score would provide engineers with an indication of unit quality, reducing the number of false negatives during EOL testing and minimizing the time spent evaluating datasets manually.
A dataset of 50 unlabelled units—four of which were faulty—served as a test of the model’s validity. The client provided data from 100 units, all of which passed the original EOL test and none that had returned a warranty claim.
Acerta’s team began by gathering information about the client’s manufacturing and data collection processes, which formed the basis of our intelligent feature engineering. Our industry experts identified non-polynomial features as potentially useful based on similarities between this application and past use cases. Acerta’s data scientists conducted a feature reduction, guided by machine-learning to prune out features that had little or no value to the final scoring algorithm.
Acerta created the requested model using unsupervised ensembled machine learning methods to generate an abnormality score for each transmission at EOL. The score simultaneously evaluated single signals and multi-signal relationships, along with their expected behavior in each test step, and across several steps. It was calculated using the reconstruction error of different models in the ensemble.
Transmissions from the test dataset were sorted based on their abnormality scores, with the model identifying the “least explainable” portions of the data.
Despite working from a small, unlabelled dataset, Acerta trained and delivered a model that generates a valid abnormality score for each transmission the client produces. The model uses data from multiple tests simultaneously to identify the signals with the greatest impact on abnormality score. In one instance, Acerta’s model identified a causal link between pressure delays and rotation delays. Detecting these specific signal relationships accelerated root cause analysis (RCA) of transmission issues by reducing the number of signals requiring manual investigation from approximately 4,000 per transmission to ten.
The model also demonstrated Acerta’s facility with multi-signal failure detection, something that was not possible with the client’s existing SPC program. As indicated in the accompanying charts, there was no clear threshold crossed (i.e., no extreme value) that would have triggered an alert from the existing SPC program. Acerta’s model, on the other hand, identified these subtle deviations automatically and cross-correlated the abnormal region with other signals from the same transmission unit to ensure a high confidence score on its reporting.
The client examined the transmissions that had passed their original EOL test but which were flagged as having a high abnormality score. They reported that these were indeed different from other units that had passed the original EOL test. According to the client, the new EOL test deployed via Acerta’s LinePulse platform would save the company up to 30% of the costs from warranty claims.