This paper describes a fuzzy logic based architecture for real time sensor validation and fusion for vehicle following tasks for automated highways. Automated highways and more generally "smart vehicles" rely heavily on sensor data for a variety of control purposes. High sensor data fidelity is of prime concern because human life is at stake. However, sensor data are always uncertain to some extent due to noise and possible sensor failures. We address these issues by proposing to validate and fuse multiple sensor readings using a fuzzy time series prediction model, fuzzy validation gates, and a weighted average fusion scheme. The integration of these methods allows the assignment of degrees of confidence to each sensor reading. This is achieved through the use of validation gates which are areas in which the reading is expected to lie. They take into account the specific properties of each sensor as well as the physical limitations of the system considered. The placement of the validation gates in turn is dependent of the fuzzy time series prediction which uses fused past values and sensor readings as input. Examples from experiments performed for PATH (Partners for Automated Transportation Highways) California show that this method works successfully under a variety of operating conditions.
Introduction
Intelligent vehicle highway systems (IVHS) use information internal and external to the vehicle to activate responses appropriate for the given situation. The goal is to support the human operator in critical decision making situations or even eliminate the human operator altogether. Such systems are only desirable if they are able to perform at least as good as the human operator. They will rely heavily on sensor readings which measure internal quantities such as throttle angle, brake pressure, manifold flow, etc., and external quantities such as longitudinal and lateral distances to the next object, temperature, visibility, etc. The reality of sensor readings is that there is always some uncertainty involved due to noise, receptivity of the sensor to environmental conditions, possible failure, etc. To remedy this undesired situation, one can try to filter out the noise or use some kind of redundancy to back up the given sensor reading. Unfortunately, two (or more) sensor readings will never coincide. While this is not a big problem if they give readings in acceptable limits, it is a problem if the readings are far apart. In that case at least one sensor must be producing erronious readings, and means have to be used to find out which one is at fault. Generally, all sensor readings give incorrect readings to some extent. After the degree of uncertainty of each sensor has been established, this information must be used for evaluation of a better reading. The process of integrating information from several sensor readings is called sensor fusion. The standard approach to accomplish sensor fusion is to use probabilistic means. In order to simplify the computation, probabilistic approaches commonly assume zero mean, Gaussian distributions of noise. This assumption is often not always valid as our experiments have shown (Agogino, Goebel and Alag, 1995). Therefore, we propose to use fuzzy logic for sensor validation and sensor fusion because no assumption of a Gaussian distribution of the noise need be made. Furthermore, unlike many probabilistic approaches in which the variance of the system perturbation and the noise variables must be known in advance, no such assumptions about the variance are made for the fuzzy approach.
Architecture
The proposed architecture performs the tasks of sensor validation and sensor fusion. The architecture is depicted in figure 1. Input to this architecture are the raw sensor readings. Output is a corrected value. This value can be used for the machine level controller as well as for supervisory control tasks (Agogino, Alag, and Goebel, 1995). Additional information can be output about which sensor performed in which manner and whether there are indications for failure. A diagnostic module would utilize this information.
Fig.1: Architecture for Sensor Validation and Sensor Fusion
The Sensor Validation and Sensor Fusion modules are realized through a Fuzzy
Exponential Weighted Moving Average (FEWMA) Time Series Predictor, a validation
procedure using validation gates, and a weighted average fusion. The state
equations of the system model used for the FEWMA are described by
where
w(k) describes the system perturbation
v(k) represents the observation noise
The standard EWMA predictor has the form
If the parameter 
is set to a fixed value the ratio to which new
information from sensor readings is used to update the system state is fixed as
well. This means that the predictor will usually lag behind the true state to
some degree. On the other hand, outliers are filtered out and a relatively
smooth predictor curve is obtained assuming is large enough. To
circumvent the lag, a more responsive predictor is desired which can be
achieved through making smaller. However, the predictor follows now
more closely the true state but is much more receptive to noise. In many
systems it is desirable to have the predictor filter out noise when the system
is in steady state and in order to make it more receptive when it is in a
transient state. This means that has to be flexible. It should be
large when the system is in a steady state and it should be small when the
system is in a transient state (Khedkar and Keshav, 1992). To further overcome
the lag in the predictor, the terms of old value and incoming sensor reading
have to be de coupled. This can be achieved by replacing the term (1-)
by a 
. If is large, must be small and vice versa.
However, the relationship between and is nonlinear.
Therefore, the system will have a smaller gap between predictor and true
value. The resulting FEWMA predictor has the form:
where
and are related to the system perturbation and the observation variances
The basic fuzzy rules used for the FEWMA are:
* IF large THEN small
* IF large THEN small
* IF change of readings small THEN large
* IF change of readings large THEN large
The reasoning behind the first two rules captures the notion of the EWMA to allow
exponential decay of the residual. The third rule is motivated by the fact that
if the system is steady, then the change of readings is due more to noise and
less to changes of the system itself, therefore more weight should be given to
the past history and less to the new reading which is likely corrupted by noise
(Khedkar and Keshav, 1992). If, on the other hand, the system is in a transient
state, then the change of readings will be due more to the change of the state
and less due to noise. Therefore, more weight should be given to the incoming
reading and less to the past history to allow good responsiveness and little
lag. Design of the membership functions is of prime importance. We used
parameterized representation of the membership functions (Jang, 1993) as follows:
Learning
of parameters is accomplished through machine learning using genetic
algorithms, driven by data obtained experimentally in the real environment.
All sensor values are assigned a confidence value. This confidence value depends on
the specific sensor characteristics, the predicted value, and the physical
limitations of the sensor value. The assignment takes place in a validation
gate which is bound by the physically possible changes of the system. In the
case of longitudinal sensors this means the limits are set by the change from
the old value to what could be achieved by maximum acceleration of the follower
car and the maximum deceleration of the lead car on the one side and maximum
deceleration of the follower car and maximum acceleration of the lead car on
the other side. Beyond these limits no sensor reading makes sense and it would
be assigned a confidence of 0 if it falls outside this region. Inside the
region, the maximum value of 1 will be assigned to readings which coincide with
the predicted value. The curve between the maximum and the two minima is
dependent on the sensor behavior. Generally, it is a non-symmetric curve which
is wider around the maximum value if the sensor is known to have little
variance and narrower if the sensor exhibits noisy behavior. The curves are
flexible over the operating conditions which allows to capture the change in
behavior of the sensor over its operating span. The validation gate is shown in
fig. 2.
Fig. 2: Validation gate for the assignment of confidence values
The fusion is performed through a weighted average of confidence values and
distance measured as
where
xf:
fused value
yi: measurements
confidence values
Note that if all sensors lie on one side of the predicted value, the fused value will also be on the same side. This ensures that evidence from the sensors is closely followed yet discounted the further it gets away from the predicted value.
The validation and fusion algorithm works in the following manner: Incoming sensor readings are validated using the validation gate and the old fused value. This fused value is then used for prediction which in turn is necessary to perform the validation of the next time step. The fused value is also used for the machine level controller as well as supervisory control tasks. The algorithm is displayed in fig. 3.
Fig. 3: Algorithm for fuzzy sensor validation and fusion
Experiments
Data were taken from PATH vehicles equipped with three different type of longitudinal distance sensors (Agogino, Goebel, and Alag, 1995). These sensors were radar sensor, sonar sensor, and optical sensor. The vehicle performed several maneuvers such as join and split, i.e. reduction of distance and increase of the distance between the cars in a platoon. Fig. 4 shows the readings of the three sensors as well as the fused value. The radar sensor had little variance throughout the experiment but experienced "bumps" around 4.5m and 9m which has been attributed to a quantization error. The sonar sensor showed the smallest variance throughout its operating region but exhibited outliers which showed up above 4m and increased with distance between the follower vehicle and the lead vehicle. Above 8m no good readings were found. The optical sensor had the highest variance of all sensors which increased with growing distance between follower and lead vehicle but shower otherwise no adverse effects. The fused value filters out the spikes of the sonar sensor, the bumps of the radar sensor, and the noise of all sensors.
Fig. 4: Open loop validation and fusion of three longitudinal sensors (radar, sonar, optical)
While the fused value shown in fig. 4 was used in open loop fashion, fig. 6 shows the effect of feed back of the fused value to the machine level controller which effects the throttle angle. For this, simulation software release 1.0 from the Vehicle Dynamics lab of UC Berkeley was used. Fig. 5 shows the velocity and acceleration profile of the maneuver simulated. For comparison, Kalman filter and Probabilistic Data Association Filters (PDAF) were used for fusion as well (Alag, Goebel, and Agogino, 1995). Over a time period of 30 seconds, the spacing error was summed up. The sum squared error (SSE) for perfect information, i.e. no noise, was 0.6693. When non-Gaussian noise was introduced, the SSE was 186.5875. A Kalman filter reduced the SSE to 1.9186, the PDAF to 1.3901, the fuzzy filter alone to 0.8638, and the fuzzy fusion to 0.8454.
Fig. 5: Velocity and acceleration profiles for simulated maneuver
Fig.6: Error spacing of follower car for closed loop sensor validation and fusion
Summary and Conclusion
The proposed architecture for sensor validation and fusion provides a tool which conveniently deals with both Gaussian and non-Gaussian noise. Machine learning algorithms are used to modify the parameters over time in order to continuously update the system model. Computational expense is held at a minimum to allow for real time applications such as the dynamic environment of fast moving vehicles with short sampling intervals and high demand on sensor data fidelity.
Acknowledgments
The authors acknowledge the support by PATH California grants MOU-132 and MOU-157
References
Agogino, A., Goebel, K., and Alag, S., "Intelligent Sensor Validation and Sensor Fusion for Reliability and Safety Enhancement in Vehicle Control", MOU132, Final Report, ITS Research Report D95-36, California PATH, 1995.
Agogino, A., Alag, S., and Goebel, K., "A Framework for Intelligent Sensor Validation, Sensor Fusion, and Supervisory Control of Automated Vehicles in IVHS", Intelligent Transportation: Serving the User through Deployment, Proceedings of the 1995 Annual Meeting of ITS America, pp. 77-87, 1995.
Alag, S., Goebel, K., and Agogino, A., "A Methodology for Intelligent Sensor Validation and Fusion used in Tracking and Avoidance of Objects for Automated Vehicles", Proceedings of the ACC 1995 Conference, Seattle, WA, 1995.
Jang, J.-S.R., "ANFIS: Adaptive-Network-Based Fuzzy Inference System", IEEE Transactions on Systems, Man and Cybernetics, vol.23, (no.3):665-85, 1993.
Khedkar, P., and Keshav, S., "Fuzzy Prediction of Time Series", Proceedings of the IEEE International Conference on Fuzzy Systems, San Diego, CA, USA, 8-12 March 1992.