Advances in Real Time Adaptive Outlier Removal During MMIC Production Testing

by Lina Abu-Absi*, Dr. Charles Trantanella, Ananya Gopalan**, and Michael Megan, Qorvo, Inc. *also with Northeastern University **also with Worcester Polytechnic Institute

Introduction 

Production testing of electrical components, including monolithic microwave integrated circuits (MMICs), often utilizes broad electrical specifications to maintain a high yield of “good” components. However, this approach has a significant drawback in that some units passing electrical specifications are outliers as compared to a typical part. For MMICs, outliers can have lower or higher DC current, lower gain, too much output power, etc. Removing these outliers is necessary for improving overall product quality and ensuring a tight distribution. The actual process for removal is a challenge, as natural variation in data, measurement drift, and test setup irregularities (among other issues) can complicate the algorithm used in the decision process.

Many outlier removal methods for MMICs, if not integrated circuits in general, use post-production techniques after all measurements are complete. Yilmaz et al. [1] describes a multivariant kernel-based density estimation model to identify outliers as units outside a set boundary. Similarly, Jauhri [2] uses density-based spatial clustering of applications with noise to identify outliers at the wafer level. O’Neill [3] applies principal component analysis (PCA) to reduce the dimensionality of data before implementing a univariate outlier test. Additionally, the Automotive Electronics Council [4] recommends a part average testing strategy to identify outliers based on how many standard deviations they stray from the mean. However, these methods can be impractical for post-production testing of MMICs, since units are often packaged in tape and reel and cannot be easily removed from the population.

Therefore, MMIC testing requires a real-time outlier removal method. Fang et al. [5] describes a real-time method using a cumulative mean and standard deviation to update univariate moving limits with each measurement. Roehr [6] implements measurement ratio testing with adaptive limits determined by predefined linear relationships between two variables. Alternatively, Yilmaz et al. [7] describes a technique that removes outliers based on expected correlation patterns from previous data. However, these three methods do not utilize more than two variables and cannot adjust for any measurement changes over time due to contact instabilities or equipment calibration drift. 

Conversely, the work by Remillard et al. [8] describes a multivariate, adaptive, and real-time PCA outlier removal method that was successfully implemented during the production testing of low noise amplifiers. In this paper, we expand the use of the method in [8], called Adaptive Outlier Removal (AOR), to more product types including mixers, driver amplifiers, and switches. We start by giving an overview of PCA as an outlier removal method and how it works within the AOR algorithm. Next, we provide details on the method and results of expanding AOR use to new product types on the production floor. Finally, we present our conclusions and plans for future work.

Principal Component Analysis and Adaptive Outlier Removal (AOR)

Analysis of large data sets with multiple, correlated parameters can often be simplified through mathematical techniques that reduce the dimensionality of the set while still maintaining the relationships between the parameters. Principal component analysis (PCA) is one such technique. PCA works by transforming the data (measured electrical parameters, in this case) to a set of orthogonal basis functions called principal components [9]. It is in this orthogonality where typical measurements are transformed into smaller numerical values, while abnormal measurements (outliers) are transformed into larger numerical values, thus making them easier to identify. Although the theory behind PCA is beyond the scope of this work, a more detailed description is given in [8]. 

Typically, PCA is employed as a post-processing method to identify outliers after all measurements have been completed. However, Remillard [8] described an implementation of PCA to isolate outliers in real time as they are measured on the production floor. We call this technique AOR, and in Figure 1, we present a flowchart of the algorithm.

In Figure 1, we note that AOR has two phases: set-up and test. The set-up phase includes the loading of a coefficient matrix and gathering baseline measurements before starting the test. The coefficient matrix is the foundation of PCA, as it represents the orthogonal relationships between the parameters. Since the calculations required to generate a coefficient matrix are too computationally expensive to be implemented in real time, we generate a pre-processed matrix using previous measurements as outlined in [8]. This information is loaded into the automated test equipment before starting. The set-up phase also includes testing until 50 units have passed all electrical specifications. These measurements generate the initial statistics needed as part of the real-time algorithm. The units used in set-up are then returned to the general population and the real-time AOR measurements can begin.

 We begin the test phase by first determining if a particular unit passes the electrical specifications. Units that fail are placed in a separate bin and never undergo AOR, whereas ones that pass are added to a rolling window of 50 units. From this rolling window, the algorithm computes a new median and standard deviation after each measurement. These statistics are used to normalize the measurements such that the set has a median of zero and standard deviation of one. Normalization is a crucial step in the algorithm, as it ensures that all measurements are treated with equal importance regardless of scale. For example, a 120 mA current measurement would outweigh a 1.5 dB noise figure measurement unless the data was normalized to the same scale. We use the median rather than the mean due to its decreased sensitivity to strong outliers, as simulations showed that AOR performed more consistently with this scheme. Finally, we chose a window size of 50 units after numerous simulations, for such a value minimizes the set-up time while still generating relevant statistics.

The normalized data is then multiplied by the coefficient matrix to generate the PCA score for each unit. Outliers are identified by high PCA scores. Therefore, a maximum PCA score can be used as a cutoff to isolate and remove outliers from the PASS population. As a result, if any element of the PCA score matrix is above the set PCA cutoff score, that unit fails and is removed from the PASS population as an outlier. The optimal PCA cutoff score for each product type was determined through numerous simulations of existing measurements using the algorithm in Figure 1.  

Expansion of AOR to More Product Types

As discussed in [8], the AOR algorithm had only been previously deployed on low noise amplifiers. We therefore considered the expansion of the algorithm to passive mixers, driver amplifiers, and passive switches, specifically single-pole, single-throw, and single-pole, double-throw varieties. The approach is described in detail in the mixer section below, and is repeated for the other product types.

Mixers

Mixers have three measured parameters: conversion gain, LO isolation, and IF/RF isolation. Outliers are units with high conversion gain, low isolation, or both. In addition, those with unusually low conversion gain or high isolation are considered outliers, as they are different from the typical response.

We first analyzed seven production lots of the CMD177C3 mixer, a commercially available X-band double balanced mixer from Qorvo, to examine the validity of applying AOR to passive mixers. We generated a coefficient matrix based on the three measured parameters using these seven lots, totaling approximately 16,000 units. This coefficient matrix was then used to run numerous AOR simulations on the same seven lots to determine the optimal PCA cutoff score. For this investigation, we considered outlier identification as well as the percent of units that failed AOR during comprehensive simulations where we ranged the PCA cutoff score from 4 to 7 in increments of 0.5. Table 1 shows the percent fail-AOR for each of the seven PCA cutoff scores for the different production lots. 

As shown by Table 1, the percent fail-AOR decreases as the PCA cutoff score increases, which is expected because higher PCA scores are associated with much stronger outliers, so quasi-outliers will be ignored. Based on these simulations, and those of the CMD178C3 mixer (not shown here), we determined the optimal PCA cutoff score for mixers was 5, as AOR identified the majority of strong outliers without removing too many quasi-outliers. 

Using a similar technique, we determined a coefficient matrix for the CMD178C3 mixer and deployed the algorithm to the production floor. Figure 2 shows measured AOR results from a recent lot of CMD178C3, where a PCA cutoff score of 5 was used to identify outliers. 

In Figure 2, we note that AOR is able to consistently identify the strongest outliers for conversion gain and most of the outliers in LO isolation. There were no strong outliers in IF/RF isolation. Any strong outliers missed for one parameter most likely have typical measurements for the other two parameters. 

Driver Amplifiers

Driver amplifiers typically have four measured parameters: Idd (drain current), gain, output power, and Igg (gate current). However, Igg is not measured for all driver amplifiers. For example, the self-biased Qorvo CMD158P3 has no external gate bias so there is no Igg measurement, while the Qorvo CMD245C4 driver amplifier utilizes a mirror circuit which draws significant current (~5 mA) and can show unit-to-unit variation. Outliers for driver amplifiers include units with abnormal Idd, high or low gain, or high or low output power, and if applicable, abnormal Igg. Therefore, AOR is implemented using three parameters for the driver amplifiers with no external gate bias (such as the CMD158P3), and four parameters for the driver amplifiers with mirror circuits (such as the CMD245C4). 

We implement AOR for the driver amplifiers using a similar procedure as described for the mixers. For the CMD158P3, we generated a coefficient matrix using twelve previous lots totaling approximately 33,000 units, whereas for the CMD245C4, we generated a coefficient matrix using six previous lots totaling 6,800 units. These coefficient matrices were used to run AOR simulations varying the PCA cutoff score from 3.5 to 5 in increments of 0.5 for both part types. Based on these simulations, we determined a PCA cutoff score of 4 to be optimal for the two amplifiers. Figures 3 and 4 show AOR results from the production floor for one lot of CMD158P3 and one lot of CMD245C4 driver amplifiers, respectively. 

In Figure 3, we note that for the CMD158P3, AOR is able to consistently identify the majority of strong outliers for each of the parameters. The results in Figure 4 for the CMD245C4 driver amplifier are less conclusive, as this particular lot was very consistent with little unit-to-unit variation. Nonetheless, AOR did identify the few CMD245C4 units with lower than normal gain or Idd.

SPST Switches

Single-pole, single-throw (SPST) switches have six measured parameters: two RF parameters (insertion loss and isolation) and four DC parameters (the control currents). Outliers include units with high insertion loss and low isolation measurements. However, we did not find anomalies or outliers in the control current measurements. Instead, anomalies in control current led directly to electrical failure, so these measurements were not considered in the implementation of AOR.

We implemented AOR with the Qorvo CMD204C3 SPST switches using a similar procedure as described for the mixers. Four production lots totaling approximately 5,400 units were used to create the coefficient matrix, and we ran AOR simulations varying the PCA cutoff score from 3.5 to 5 in increments of 0.5 to examine the viability. Based on these simulations, we determined the optimal PCA cutoff score to be 4 and deployed the algorithm on the production floor. Figure 5 shows measured AOR results for a recent lot of the CMD204C3 SPST switches.

In Figure 5, we note AOR identified many of the strongest outliers with either high insertion loss or low isolation, though a small cluster of outliers were missed around unit number 1500 in the sequence. This cluster led to a noted increase in the standard deviation, which affected the normalization and hence caused these units to be viewed not as outliers, but as typical in terms of response. However, if elements of this cluster had been spread out amongst the population, AOR would likely have caught them. The percent of units failed as outliers for the CMD204C3 SPST switches is typically less than 3%.

SPDT Switches

Single-pole, double-throw (SPDT) switches have eight measured parameters: four RF parameters (two insertion losses and two isolations) and four DC parameters (the control currents). Outliers are units with unusual insertion loss and isolation measurements. Typically, insertion loss outliers are above the main mode while isolation outliers are below. Just as with SPST switches, control current measurements were excluded when deploying AOR, as anomalies in control current led directly to unit electrical failures.

We implemented AOR with the Qorvo CMD196C3 SPDT switches using a similar procedure as described for the mixers. Six production lots totaling around 45,000 units were used to determine the coefficient matrix. We ran numerous AOR simulations varying the PCA cutoff score from 4 to 7 in increments of 0.5 to examine the viability of the approach. We determined the optimal PCA cutoff score to be 5 based on these simulations and then deployed the algorithm to the production floor. Figure 6 shows AOR results for a recent lot of the CMD196C3.

As shown in Figure 6, AOR identifies the majority of strong outliers in both insertion loss and isolation. This particular lot also had significant drift in parameters over time. AOR was able to account for this drift and fail only the strongest outliers, whereas techniques using fixed bounds would likely fail many more units. AOR typically fails less than 2% of units for the CMD196C3 switches while still capturing the strongest outliers.

Conclusion

In this paper, we demonstrated a real-time adaptive outlier removal algorithm called AOR as applied to mixers, driver amplifiers, and switches on the production floor. Compared to other real-time outlier removal methods, AOR can handle multiple parameters and measurement drift over time by utilizing a rolling window to update the necessary statistics. The results presented here indicate the success of AOR as an outlier removal method and its potential for other products. We will continue to work to expand AOR to other product types such as attenuators, distributed amplifiers, and power amplifiers. 

Acknowledgements

The authors would like to acknowledge Grace Remillard of the University of Massachusetts-Lowell for her previous work on understanding and applying PCA as an outlier removal method, and Helen Leung and Cham Heng of Qorvo for deploying the algorithm on the production test equipment.

References

[1] E. Yilmaz, S. Ozev, and K. Butler, “Adaptive Multidimensional Outlier Analysis for Analog and Mixed Signal Circuits,” Proceedings of the 2011 IEEE International Test Conference, Anaheim, CA, 2011.

[2] A. Jauhri, B. McDanel, and C. Connor, “Outlier Detection for Large Scale Manufacturing Process,” 2015 IEEE International Conference on Big Data: Big Data 29, Santa Clara, CA, 2015, pp. 2771-2774.

[3] P. O’Neill, “Production Multivariate Outlier Detection Using Principal Components,” Proceedings of the 2008 IEEE International Test Conference, Santa Clara, CA, 2008. 

[4] Guidelines for Part Average Testing, Dec. 2011, [online] Available: http://www.aecouncil.com/Documents/AEC_Q001_Rev_D.pdf . [Accessed Aug. 19, 2020].

[5] L. Fang, M. Lemnawar, and Y. Xing, “Cost Effective Outliers Screening with Moving Limits and Correlation Testing for Analogue ICs,” 2006 IEEE International Test Conference, 2006, pp. 1-10.

[6] J. L. Roehr, “Measurement Ratio Testing for Improved Quality and Outlier Detection,” 2007 IEEE International Test Conference, 2007, pp. 1-10.

[7] E. Yilmaz and S. Ozev, “Defect-based Test Optimization for Analog/RF Circuits for Near-zero DPPM Applications,” 2009 IEEE International Conference on Computer Design, 2009, pp. 313-18.

[8] G. Remillard, C. Trantanella, and M. Megan, “Removing MMIC Outliers in Production Test Using Real-Time Principal Component Analysis,” Microwave Journal, vol. 62, no. 11, pp. 66-80, Nov. 2019.

[9] T. Zhang, and B. Yang, “Big Data Dimension Reduction Using PCA,” 2016 IEEE International Conference on Smart Cloud (SmartCloud), 2016, pp. 152-57.

(645)