How Deep Learning Automates Inspections for the Electronics Industry
The electronic hardware manufacturing industry is well-served by machine vision. As product parts get ever smaller thanks to consumer demand for thin and light and as demand for cost-effective manufacturing increases, it’s key for producers to automate every aspect of their production processes without sacrificing quality.
But there are applications, especially in the semiconductor and mobile device industries, which resist automation because of their complexity and expense to program. In these cases, relying on human inspectors is a prevailing but inefficient compromise to handle judgment-based decisions involving multiple and different parts, assemblies, and scenes. Yet despite their reliability compared to machines, human inspectors are prone to introduce error when fatigue sets in.
For these reasons, electronics manufacturers are looking toward deep learning-based inspection solutions, which uniquely combine the flexibility of the human mind with the robustness of a computer for quick and highly reliable inspection results. Read on to see how deep learning revolutionizes inspections, code reading, and classification tasks for the electronics industry.
Assembled printed circuit boards (PCBs) rely on the precise welding of components so that electricity can flow. The smallest defects can cause interference and failure. This makes defect detection critical. And yet the great variety in welding defects—over-powered, under-powered, and uneven—appear differently to a vision system not only because of their subtle but innate differences but also because of image issues such as glare, which distort and change their presentation to the camera.
Any automated inspection system needs to tolerate significant part-to-part variation (for example, an underpowered vs. an overpowered weld) under tough image quality conditions. Deep learning-based solutions are capable of fixturing the region of interest and inspecting welds with significant part-to-part variation without rules-based programming, which would be too time-consuming, tedious, or sometimes even impossible.
Cognex’s deep learning tool for defect detection learns to identify a variety of welding defects after it has been trained on a representative set of known “good” (i.e., passing) and “bad” (failing) labeled images. Based on these images, Cognex Deep Learning can build a reference model of a weld’s normal appearance, as well as intolerable defects, that accounts for variations in appearance that might be attributable to imaging issues and glare.
Optical Character Recognition
Like any alphanumeric characters printed on electronic components and modules, the serial numbers printed on chips are vulnerable to deformities, skewing, and poor etching. These characteristics challenge traditional OCR tools, which need to be trained on a library of fonts and characters and their various presentations.
Even the best tools—those which allow users to train entire strings of characters in a single step, remove single characters, train on multiple instances of the same characters, and load or save trained fonts to new applications—are time-consuming and may still have trouble identifying a character with an unexpected deformity.
Deep learning-based tools use pre-trained libraries to recognize and verify characters, even as their presentations change, so that only missed characters need to be re-trained in the system during testing and validation. This gives the inspection system a better shot at accuracy out-of-the-box and minimizes downtime due to re-training.
Assembly verification is a notoriously difficult application to automate because of the number of different components which need to be accounted for. These components need to be verified as present, correctly mounted, and oriented. These inspections may need to occur in 2D as well as 3D, depending on the sub-assembly.
A classic example is the final assembly verification of a PCB, which has had LEDs, microprocessors, and other surface mount devices welded onto it. The inspection system is responsible for catching missing components and parts that may be incorrectly positioned, since these errors can damage a PCB’s performance and lifespan.
Machine vision systems can be trained on multiple regions of interest and learn to identify each individual component, but variations in appearance due to lighting contrasts, changes in perspective and orientation, or glare can still confuse the system. Especially on a densely populated PCB where components are close together, a machine vision system may have trouble distinguishing individual components, causing it to incorrectly fail an inspection. While human inspectors are able to distinguish between components, they simply cannot meet high-speed throughput demands.
Programming these inspections into a rule-based algorithm is time-consuming and still prone to error, not to mention challenging to maintain in the field. Fortunately, deep learning-based vision systems rival the flexibility, discernment, and judgment-based decision-making of a human with the additional benefit of a computer’s speed and robustness.
Trained on a labeled set of example references, the tool can build a reference model of a fully assembled PCB board. The model can identify individual components based on their generalized size, shape, and features—even though their appearance is bound to change during inspection—and predict their location on the board. During inspection, Cognex Deep Learning can identify multiple regions of interest to locate, count, and inspect components in order to ensure they’re present and correctly assembled.
Machine vision has inherent limitations, including the ability to classify. This becomes burdensome in electronics applications where components need to be identified and grouped into multiple classes and the inspection system needs to tolerate some visual variation.
Electronic capacitors are a good example of a component which varies by type (ceramic and electric) as well as by size (large and small) and color (gold and non-gold). A manufacturer needing to sort capacitors is faced with the difficult task of making sense of a single image which contains multiple classifications—for example, gold ceramic capacitors with black markings or gold electric capacitors with blue markings. The inspection system needs to be able to sort components according to the manufacturer’s criteria to distinguish electric capacitors by their color and markings while ignoring other criteria.
To do this automatically, an inspection application engineer must look to deep learning for a solution. Deep learning-based software operating in supervised mode can be trained to both detect a selective grouping of traits (for example, both gold and electric capacitors) and distinguish between each capacitor’s individual traits (gold, black, or blue markings) while ignoring additional variations within the same type. The deep learning-based system can accurately classify and sort multiple types of a single component in a single image—a huge benefit over machine vision.
Automating production processes and improving quality are two of the electronics industry’s greatest demands, yet some applications are too complicated and time-consuming to program into a rule-based algorithm. Cognex Deep Learning leverages the power of artificial intelligence on image analysis to solve challenging electronics applications involving part location, cosmetic inspection, classification, and character recognition.