Process noise confounds our interpretation of when a change starts, when the process variable (PV) reaches steady state, and the true PV value.  In Figure 1, the dots represent a noisy measurement of a process that moved from one steady value to another. The line is a first-order filtered value of the data. 

Looking at the filtered value, one might claim that the transient started at a time of three and reached steady state at a time of 20. Looking at the noisy measurements, the beginning and end of the transient might be considered at times of two and 15.

During a transient, the filtered value lags. Nearly all measurement values in the time interval of five to 10 are above the filtered indication. If the filtered PV value were used for online process analysis, like a material balance closure, the filtered value would misrepresent the process value during the transient. However, during the initial and ending steady state values, the first order filtered value is in the centroid of the data, which does provide a relatively noiseless and valid indication of the true PV value.

During the initial or final steady state, the filtered trace is not steady; it wiggles a bit responding to the data perturbations. One could use less filtering to make the filter track the transient faster; but then it would not reduce the noise as much during steady periods. The lag and continual variation are undesirable trade-off features of first-order filtering.

This is simulated data, and Figure 2 also reveals the true, but unknown, PV trend, which generated the data for Figure 1. The true PV value makes a first-order rise, starting at a time of four and reaching 95% on the way to its asymptotic final value at a time of about 10. Neither observation from the data nor from the first order filtered value are great indications of the transient start or end times.

Noise can be produced by flow turbulence, which confounds pressure or flow rate measurements. It can be generated by incomplete in-process mixing, confounding composition or temperature measurements as rich or lean fluid packets pass by the sensor. It can be generated by physical vibrations from external equipment or in-process splashing, boiling, cavitation or bubbling. It can be generated by external electromagnetic fields. 

Noise is usually considered to be normally and independently distributed (NID) perturbations that are added to a process variable; and most often, the perturbations are close enough to being Gaussian distributed with a mean of zero and noise amplitude characterized by standard deviation, NID(0, σ). 

This ideal characterization of noise has permitted foundational mathematical analysis of many techniques for tempering noise, such as the first-order filter or a moving average. The NID(0, σ) and additive perturbation to a true PV value assumptions have also grounded the development of statistical filters, such as Kalman or SPC-based methods. Legitimized by theoretical analysis of idealized concepts, these methods dominate applications. 

Get your subscription to Control's tri-weekly newsletter.

Statistical filtering

Contrasting the behavior of a first-order filter, Figure 3 shows a statistically based filter behavior. It holds the prior filtered value until there is adequate evidence to confidently report a change, which is characterized by step-and-hold patterns. Here, filter delay (not lag) is an undesirable feature, but desirably the filtered value does not wiggle during periods of steady state, and it tracks the transient faster.

Kernel filtering

The first order and statistical filter methods update the filtered value in real time, as each sampling provides a new noisy PV value. They can only see present and past data values. They cannot see future values. By contrast, after the data is collected, a kernel filter goes back through the data and determines an average at each point in time using a few of both past and future data. An advantage is that it will more closely track transient responses, but the disadvantage for on-line control is that it is always reporting results represented half of a data window behind. 

Kernel methods would be good for accuracy and transient analysis in off-line analysis of process behavior. 

The kernel is the rule for weighting past and future data values. It could simply be an average, or it could be an exponential or radial basis weighting, or a median value. There are many weighting methods, including best fitting a local curve through the data and interpolating values from the curve. 

Tempering is not removing

Even so, methods to remove noise cannot truly remove it, they temper the perturbations and reduce the variance, but do not remove uncertainty. Further, online filtering methods cause either a lag or delay in the filtered signal catching up to the true signal. If the process transitioned to a new steady state, filtering takes some time to see the new average within the residual uncertainty, but during transients or a sequence of real changes, the filtered signal can remain aggravatingly behind, misdirecting action.

The greater the noise reduction, the greater the lag or delay. You must choose the filter parameter values to balance precision (at steady state) and speed of response with lag or delay.  If you don’t like the noise impact on the controller, only filter on the derivative function (not on the process variable) so that the proportional action can be responsive to data, not the lagged value.