Adaptive Noise Cancelling

Note: references 4 and 6 are independent sources with detailed coverage of the topic. Even though John Kaunitz is one of 8 co-authors of reference 4, it is largely the work of others. Reference 4 is the almost universally referenced definitive description of this topic. — Preceding unsigned comment added by Kaunitzj (talk • contribs)

---

Adaptive noise cancelling is an unorthodox signal processing technique that is highly effective in suppressing additive interference or noise corrupting a received target signal at the main or primary sensor in certain common situations where the interference is known and accessible but unavoidable and where the target signal and the interference are unrelated, that is, uncorrelated. Examples of such situations are where a microphone is attempting to collect speech near machinery or other noise sources in the environment or in the case of obtaining a fetal electrocardiogram (ECG) where the presence of the mother's stronger ECG represents the unavoidable interference.

Whereas conventional signal processing techniques rely on filtering the received signal so as to minimise the effect of the interference (maximising the signal-to-noise ratio), adaptive noise cancelling relies on a second sensor located near the source of the interference to obtain a relatively 'pure' version of the interference free from the target signal and other interference. This second version of the interference and the sensor receiving it are called the reference.

Adaptive noise cancelling uses an adjustable adaptive filter to automatically transform the reference signal into an optimal estimate of the interference corrupting the target signal before subtracting it from the received signal to cancel or minimise the effect of the interference. The adaptive filter adjusts itself continuously to minimise the residual interference affecting the target signal at its output. The power of the adaptive noise cancelling concept is that it requires no detailed a priori knowledge of the target signal or the interference. The adaptive algorithm that optimises the filter relies only on ongoing sampling of the reference and the noise canceller output.

Adaptive Noise Cancelling Configuration and Concept

The adaptive noise canceller configuration diagram below shows the target signal s(t) present at the primary sensor and the interference or noise source n(t) and its manifestations n_p(t) and  n_r(t) at the primary and reference sensors respectively.

Whilst n_p(t) and n_r(t) are the manifestations of the same interference source in different locations, they will usually differ significantly due to different transmission paths through the environment to the two sensors. So the reference n_r(t) cannot be used directly to cancel or reduce the interference corrupting the target signal. It must first be appropriately processed to generate ñ_p(t), the optimal estimate of the version of the interference present at the primary sensor, before it can be used to minimise the overall effect of the interference at the noise canceller output.

An adaptive noise canceller is based on a self-optimising adaptive filter that has as variable transform function shaped by adjustable parameters called weights. The adaptive filter transforms the reference n_r(t) into an optimal estimate ñ_p(t) of the interference n_p(t) corrupting the target signal and ‘cancelling’ the latter by subtraction, whilst leaving the target signal unchanged. So the output of the adaptive noise canceller shown below is: z(t) = s(t)+n_p(t)-ñ_p(t).

Adaptive filter configured as Noise Canceller

The power of the adaptive noise cancelling approach stems from the fact that the algorithm driving the iterative adjustment of weights in an adaptive filter, (for example Least-Mean-Square Filter) is a simple fully automatic iterative process that relies only on ongoing sequence of measurements of the noise canceller output z(t) and the weight inputs, which, in the case of the usual a tapped delay line filters is simply a sequence of samples of the reference signal r(t) = n_r(t).

Adaptive noise cancelling can be effective even when the target signal and the interference are similar in nature and the interference is considerably stronger than the target signal. Apart from the availability of a suitable reference signal the only other essential requirement is that the target signal and the corrupting noise source are unrelated, that is uncorrelated, so that the time average for all values of $\tau$, where the bar represents time averaging.

Adaptive noise cancelling does not require detailed a priori knowledge of the interference or the target signal.  However, the physical characteristics of the adaptive filter must be generally suitable for producing an adjustable frequency response or transfer function that will transform the reference signal n_r(t) into a close estimate of the corrupting interference, ñ_p(t), through the iterative adjustment of the filter weights.

Genesis

Adaptive noise cancelling evolved from the pioneering work on adaptive systems, filtering and signal processing carried out at the Information Systems Laboratories in the School of Engineering at Stanford University during the 1960’s under the leadership of Professor Bernard Widrow.  Adaptive filters incorporate` adjustable parameters called weights, controlled by iterative adaptive algorithms, to produce a desired transfer function which minimises the mean square of the error, the difference between the adaptive filter output and a desired response presented to the filter.

Adaptive filters were originally conceived to produce optimal filters by iteratively adjusting the filter weights during a training phase by presenting the filter with a known input and a training signal called a desired response. At the completing of the training phase the optimised filter in its operating phase would process received signals to improve the signal-to noise ratio.

In the adaptive noise cancelling context the primary input becomes the desired response, so the adaptive filtering of the reference actually strives to suppress the overall signal power at the output of the noise canceller.

This counterintuitive concept can be understood by keeping in mind that the target signal s(t) and the interference n(t) are uncorrelated. So in aiming to minimise the error, that is the noise canceller output, the best the adaptive filter can do is to generate the optimal estimate of the interference at the primary sensor ñ_p(t), thus minimising the overall effect of the interference at the noise canceller output whilst leaving the target signal s(t) unchanged. The iterative adaptive algorithms used in adaptive filtering require only an ongoing sequence of sampling measurements at the weight inputs and the error. In other words, the operation of an adaptive noise canceller requires only on an ongoing sequence of sampling measurements of the reference and the noise canceller output.

The noise cancelling approach and the proof of the concept, the first striking demonstrations that general broadband interference can be eliminated from a signal in practical situations using adaptive filters in a novel noise cancelling configuration, were set out and demonstrated in 1971-72 at the Stanford Information Systems Laboratory by Widrow and John Kaunitz, an Australian doctoral student, and documented in the latter’s PhD dissertation Adaptive Filtering of Broadband signals as Applied to Noise Cancelling (1972) (also available here) and also in a Stanford Information Systems Laboratory report by Kaunitz and Widrow, Noise Cancelling Filter Study (1973). The initial demonstration of the noise cancelling concept for eliminating broadband interference was carried out by means of a prototype hybrid adaptive signal processor designed and built by Kaunitz and described in a Stanford Information Systems Laboratory Report General Purpose Hybrid Adaptive Signal Processor (1971)  (also available here).  

A 1975 paper published in the Proceedings of the IEEE by Widrow et al., Adaptive Noise Cancelling: Principles and Applications , is now the most commonly referenced publication in the field that summarises subsequent early developments in adaptive noise cancelling and also mentions earlier efforts to eliminate narrowband 60Hz interference at the output of an electrocardiographic recorder as part of a student project at Stanford in the 1960s.

Applications

Examples of practical situations where noise cancelling can be used include the following:

• One of the original noise cancelling demonstrations involved eliminating ambient noise from the output of a microphone situated in a noisy environment by using a second microphone situated near the noise source as the reference signal.
• Similarly the background noise of a naval ship towing a sonar array searching for a target signal can be reduced or eliminated, making the receiver ‘quiet’, by using a reference signal of the towing ship’s own noise which can be readily obtained.
• Some noise cancelling headphones utilise adaptive noise cancelling techniques. The effects of ambient ambient noise which penetrates inside the earphone can be minimised by using the version of the ambient noise from a microphone situated on the headset as the reference signal.
• Another original demonstration of adaptive noise cancelling was the extraction of the remnant recipient pacemaker signal from a heart transplant animal from an ECG which also included the stronger ECG signal of the donor heart. A limb-to-limb ECG was used as the reference signal which was a version of the donor heart ECG in a relatively pure form.
• Similarly fetal electrocardiograms are received in the presence of the mother’s stronger ECG and can be extracted using adaptive noise cancelling to reduce the effect of the mother’s ECG.
• Adaptive noise cancelling techniques can also been used in the context of Active Noise Control to reduce acoustic noise in a physical space
• Adaptive noise cancelling has also been used in rail surface defect detection

In these situations a suitable reference signal can be readily obtained by placing a sensor near the source of the interference or by obtaining an independent version of the interfering ECG free from the target signal.

Adaptive noise cancelling can be effective even when the target signal and the interference are similar in nature and the interference is considerably stronger than the target signal. Apart from the availability of a suitable reference signal the only other essential requirement is that the target signal and the corrupting noise source are unrelated, that is uncorrelated, so that the time average 210x210pxfor all values of $\tau$, where the bar represents time averaging.

Adaptive noise cancelling does not require detailed a priori knowledge of the interference or the target signal.  However, the characteristics of the adaptive filter must be generally suitable for producing an adjustable frequency response or transfer function that will transform the reference signal n_r(t) into a close estimate of the corrupting interference, ñ_p(t), through the iterative adjustment of the filter weights. The interference in the above examples are usually irregular repetitive signals. Although the theory of adaptive filtering does not rely on this as an assumption, in practice this characteristic is very helpful as it limits the need for the adaptive filter to compensate for time shifts between the versions of the interference at the primary and reference sensors to appropriately compensating for phase shifts.

First Proof of Concept Demonstrations

The first noise cancelling demonstration carried out in 1971, typical of general practical situations involving broadband signals, eliminated the ambient noise from the output of a microphone used by a speaker in a very noisy room. A triangular signal, representing a typical broadband signal, emitted by a loudspeaker situated in the room, was used as the interfering noise source. A second microphone situated near this loudspeaker served to provide the reference input. The output of the noise canceller was channeled to the earphones of a listener outside the room.

The experimental arrangement used by Kaunitz in the photo below shows the loudspeaker emitting the interference, the two microphones used to provide the primary and reference signals, the equipment rack (third from left) containing the hybrid adaptive filter and the digital interface, and the HP 2116B minicomputer on the left of the picture.

Adaptive Noise Cancelling Demonstration by John Kaunitz at the Adaptive Systems Laboratory, Stanford University in 1971

The noise canceller effectively reduced the ambient noise overlaying the speech signal from an initially almost overwhelming level to barely audible and successfully re-adapted to the change in frequency of the triangular noise source and to changes in the environment when people moved around in the room. Recordings of these demonstrations are still available here and here.

The second application of the original noise canceller was to process ECGs from heart transplant animals studied by the pioneering heart transplant team at the Stanford Medical Centre at the time led by Dr Norman Shumway. Data was provided by Drs Eugene Dong and Walter B Cannon in the form of a multi-track magnetic tape recording.

In heart transplant recipients the part of the heart stem that contains the recipient’s pacemaker (called the sinoatrial or SA node) remains in place and continues to fire controlled by the brain and the nervous system. Normally the pacemaker controls the rate at which the heart is beating by triggering the atrioventricular (AV) nodes thus controlling heart rate to respond to the demands of the body. (See diagram below). In normal patients this represents a feedback loop, but in transplant patients the connection between the remnant SA node and the implanted AV nodes is severed and the remnant pacemaker and the implanted heart beat independently, at differing rates.

The behavior of the remnant pacemaker in the open loop situation of a transplant patient was of considerable interest to researchers, but studying the ECG of the pacemaker (the p-wave) was made difficult because the signal from the pacemaker is very weak and was swamped by the signal from the implanted heart even when a bipolar catheter sensor (primary sensor) is inserted through the jugular vein close to the SA-node. (See the third trace from top in the diagram below). The noise cancelling arrangement to eliminate the effect of the donor heart from the ECG of the p-wave is shown below.

A reference signal was obtained through a limb-to-limb ECG of the patient (See top trace in the diagram below), which provided the main ECG of the donor heart largely free from the pacemaker p-wave. Adaptive noise cancelling was used to transform the reference into an estimate of the donor heart signal present at the primary input (see second trace from top) and used to substantially reduce the effect of the donor heart from the primary ECG (third trace), providing a substantially cleaned up version of the p-wave at the noise canceller output (see bottom trace) suitable for further study and analysis.

Extracting Remnant Pacemaker Signal from Heart transplant ECG

Extracting Remnant Pacemaker Signal from Heart transplant ECG

Principles of Operation

Adaptive noise cancelling is simply the application of an adaptive filter in a certain configuration. The body of theory and analysis previously developed for adaptive filtering therefore applies directly to adaptive noise cancelling.

Adaptive filters are adjustable filter structures able to produce a range of transfer functions through adjustable parameters called weights which are adjusted using an iterative algorithm to achieve the desired transfer function, as measured by a certain performance indicating cost function.

An adaptive filter structure will include some form of pre-processor that provides the memory of the filter input, as the basis for producing a range of desired transfer functions for various situations. Although the adaptive filter used in the original proof of concept demonstration by Kaunitz was based on a RC pre-processor filter, the theory and practice of adaptive filtering using digital technology is based on the digital tapped delay line, where successive recent samples of the filter input signal serve as the weight inputs. The weight outputs  are then additively combined to form the filter output.

Representing the sequence of inputs  x₁(t), x₂(t)… x_n(t) as a vector X(t) and the sequence of weights as vectors w₁, w₂, …. w_n as a vector W, the adaptive filter output can be represented in vector notation as:

y(t) = X^T(t)W = W^TX(t ) = $\sum _{i=1}^n$x_i w_i

This type of adaptive filter is called a linear combiner .

Adaptive filters were originally conceived as a method of producing optimal filters for various situations by adjusting the filter’s frequency response during a training process using a training signal called the desired response. So, as originally conceived, adaptive filters had two modes of operation: a training mode and the normal operating mode.

During the training phase the parameters of the filter would be adjusted using a control input signal and a desired response signal designed to generate the desired filter transfer function which would maximize the signal to noise ratio at the filter output during normal operation, when presented with a target signals s(t) in the presence of interference n(t) both with certain known general characteristics.

File:Adaptive Filter - Operational Mode Crop.jpg

Optimised Adaptive Filter in Operating Mode

During the training phase the filter is adjusted or ‘trained’ to produce what is considered to be the desired optimum transfer function. This transfer function is produced by presenting the filter by a known input x(t) and a corresponding desired response d(t) and adjusting the weights by an iterative algorithm so that the filter output is an optimal estimate of the desired response. This configuration is deemed to be achieved where the mean squared error ξ(W) = $\overline{e^2(t)}$ = $\overline{(d(t)-y(t){)}^2}$, the time average of the difference between the desired response d(t) and the filter output y(t) is minimised.

File:Adaptive Filter - Training Mode crop.jpg

Adaptive Filter in Training Mode

When the adaptive filter is a linear combiner the mean squared error is: ξ(W) = $\overline{e^2(t)}$=$\overline{(d(t)-X^T(t)W){)}^2}$ =

Adaptive algorithms aim to minimise this mean squared error. The above  expression shows this to be a multi-dimensional paraboloid function of the weight vector with a single minimum that can be reached by gradient descent algorithms that adjust the weight vector opposite the gradient. Typically these iterative algorithms depend only on a series of measurements of the error signal and the weight inputs. For example the Least Mean Square L.M.S. algorithm iteratively adjusts weights according to the formula:

W_k+1 = W_k + µe_kX_k

Where k represents the k^th step in the iteration process and µ is the adaptation constant that controls the rate and stability of the adaptation process and e_k and X_k are samples of the error and the input vector respectively. In noise cancelling terminology this becomes:

W_k+1 = W_k + µz_kR_k

The typical algorithms that adaptive noise cancelling relies on is thus a simple process that requires only an ongoing series of simultaneous measurements at the adaptive filter input and the error, or in noise cancelling terminology, the reference input r(t) and the noise canceller output z(t) . The LMS (Least Mean Square) and other gradient descent algorithms will converge to a set of weight values which will tend to minimise the mean squared error.

The fundamental innovation of adaptive noise cancelling is to use adaptive filters differently by:

1. using the training mode of the filter as the operational mode
2. using the primary signal containing the target signal as the desired response and
3. using the error as the noise canceller output, that is, the difference between the primary signal the output of the adaptive filter.

Since the adaptation process will aim to minimise the error signal, it follows that in the noise canceller configuration the adaptation process will tend to minimise the overall signal power at the noise canceller output.

To understand how counterintuitive concept works, it helps to consider the situation where the target signal is absent s(t)=0. In this case the output of the noise canceller is the difference between the interference at the primary sensor and the adaptive filter output and the mean squared error are:

File:Formula 10 JK ANC.png

So in this case the desired response of the adaptive filter is the interference n_p(t) present at the primary sensor and the adaptation process will simply tend to minimise the mean squared error by transforming the reference input n_r(t) into an estimate ñ_p(t) of the interference present at the primary sensor. So in the absence of a target signal the noise canceller will suppress the overall effect of the interference at the noise canceller output and make its output ‘quiet’ by minimising the average output power.

When the target signal is additionally present:

File:Formula 8b JK ANC.png

Since ñ_p(t) = X^TW = N_r^TW  and the fundamental assumption that s(t) is uncorrelated with n_p(t) and n_r(t) the middle terms drop out and we are left with:

File:Formula 9 JK ANC.png

Since the first term is independent of W, the adaptation process will result in minimising the second term which is the same as when the target signal is absent.

So as long as the target signal and the interference are uncorrelated the adaptation will result in producing, at the adaptive filter output an optimal estimate of the interference at the primary sensor, thus minimising the overall effect of the interference n(t) at the noise canceller output whilst leaving the target signal unchanged.

A comprehensive analysis of algorithms designed to optimise adaptive filters when applied to stochastic signals is presented by Widrow and Stearns in their book Adaptive Signal Processing. An analysis of noise cancelling where s(t) and n(t) are assumed to be bounded deterministic signal was presented by Kaunitz in his PhD dissertation.^[1].

References

This article "Adaptive Noise Cancelling" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Adaptive Noise Cancelling. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.

---