THE APPLICATION OF CASE-BASED REASONING TO
ULTRASONIC DATA FOR USE IN INSPECTION QUALIFICATION
D. McNab1, A. McNab1, P. Leggat1, and S. McArthur2
Centre for Ultrasonic Engineering, Department of Electronic and Electrical Engineering,
University of Strathclyde, Glasgow, Gl 1XW, UK
2
Centre for Electrical Power Engineering, Department of Electronic and Electrical
Engineering, University of Strathclyde, Glasgow, Gl 1XW, UK
ABSTRACT. This paper describes the application of case-based reasoning (CBR) to provide
decision support for the NOT engineer during Inspection Qualification. A key element of this process
is to present experimental evidence on the effectiveness of the inspection. A prototype for the system
has been developed using a case-base which describes the defect, the probe and response. This paper
discusses the motivation for the system, the case structure, weightings, similarity measures, retrieval
techniques and its design and implementation.
INTRODUCTION
This paper discusses a case-based reasoning (CBR) system that provides decision
support for the NDT engineer during Inspection Qualification (IQ). A main component of
IQ is to present evidence on the effectiveness of an inspection. This may take the form of
theoretical modelling and/or practical trial data from costly test pieces. In Inspection
Qualification, practical trial data carries more weight than modeled data. However,
manufacturing test blocks and obtaining trial data is costly and time consuming; because of
this trial data is limited. Therefore a system that allows the re-use of previous practical
trial data and modeled data in a CBR system, as evidence of a current inspection's
performance, would be advantageous. This paper describes such a prototype CBR system,
entitled "smartlQ". The first section will deal with the motivation for the system. A brief
description of case-based reasoning will follow and this will lead onto the description of
the smartlQ system. Subsequent sections will deal with the case structure, weightings and
similarity measures for smartlQ. The paper then concludes by discussing the benefits of
the system.
BACKGROUND
As stated above, obtaining practical trial data is costly and time consuming and
therefore data is limited. Currently existing data is not realizing its full potential, even if
made available on a relational database; for non-database users, multiple field queries can
become too complex and traditional database systems are only designed for exact
CP657, Review of Quantitative Nondestructive Evaluation Vol. 22, ed. by D. O. Thompson and D. E. Chimenti
© 2003 American Institute of Physics 0-7354-0117-9/03/S20.00
1862
matching between fields. However case-based reasoning can overcome these problems. A
CBR system will allow retrieval of the most similar set of previously solved problems
(called cases) instead of exact matching. A currently available database of experimental
trial data, containing information on all relevant parameters of the ultrasound inspection,
has been adapted as a foundation for the smartlQ CBR system.
CASE-BASED REASONING
The standard definition of case-based reasoning [1-4] states that it is an artificial
intelligence technique that emulates the human thought process of retrieving solutions to a
problem based on past experiences. That is, previously solved problems are used to
generate solutions for new problems. Figure 1 explains how this is achieved. Firstly the
problem must be described in a structured manner, usually with numerical or textual
values. These are the case parameters or attributes. Then, using the defined case structure
a query case can be created that describes the current problem. The CBR cycle shown in
Figure 1 explains the 'classic' CBR activities.
The current problem's characteristics are compared against retrieved
examples from a case base. Similar examples can be re-used directly or can be used to
generate revised examples to be retained in the case base. This cycle is very rarely done
without human interaction and usually a domain expert will make any final decisions.
CBR systems have been shown to be useful in a number of NDT problem areas. Both
Jarmulak et al [5-6] and Perner [7] have investigated using CBR for interpretation and
classification. Jarmulak et al [5-6] built two systems, one for the interpretation and
classification of eddy current signals and another to interpret B scan signals from rail
defects, whilst Perner [7] investigated using CBR for image segmentation. The following
section will show how this set of principles has been used to aid the qualification process.
THE CBR
CYCLE
FIGURE 1. The working CBR Cycle based on Aamodt and Plaza [3].
THE "SMARTIQ" SYSTEM ARCHITECTURE
The smartlQ CBR process follows the above principles, relating them specifically
to the domain of Inspection Qualification. The 'input problem' is to successfully qualify
an inspection using experimental evidence to prove its effectiveness. Previous trial data
1863
that is most similar to the current inspection is retrieved. Justifiably similar responses
from previous inspections can be re-used as evidence. Revision can be carried out by
comparing modeled data to real data from the case base and interpolating to predict
"experimental" data for the new case.
SmartlQ software has been developed to carry this process out, and operates using
3 layers. As a back-end, and method of storing cases of previous trial data, a Microsoft
Access database is used. Flexible CBR software called "Orenge" acts as middleware.
This enables CBR within the application and is also used to create the case structure,
weightings and similarities. A web-based interface is used as a front end, which connects
to the CBR software using a web server and CGI scripts. When building this system there
were three important issues to be considered regarding case-based reasoning. These are:
(i) defining what the case structure should be, i.e. what parameters are important in
describing an inspection ; (ii) defining the relative importance of parameters via an
assigned 'weight'; (iii) defining a scale of 'similarity' within the possible range of values
of an individual parameter. The next section will deal with the first of these issues, case
structure.
CASE STRUCTURE
Case structure must effectively describe the problem/solution and must fit the enduser's requirements. Therefore case structure development was carried out through a
structured interview with two experts in the field of ultrasonic NDT and who would
eventually be end-users of the system. The purpose of a case within the smartlQ system is
to describe an inspection (practical trial). Therefore case parameters had to be chosen to
satisfy this. Twenty-one parameters were chosen in total. The inspection was split into
three major elements, which were the probe, the defect and the inspection itself. Then
each element was discussed and important parameters were recorded.
Examples of probe parameters used are probe angle and wavelength, both of which
are numerical values. Wavelength was found to be more useful than just frequency on its
own, which has not been implicitly stored in the case. Examples of defect parameters used
are tilt mis orientation and defect type. Defect tilt is not meaningful on its own but has to
be considered in relation to the beam axis, hence the parameter tilt misorientation. This is
an important parameter as the response from the defect is strongly dependent on its value.
Defect type however is an enumerated type and contains some general class of defect such
as lack of fusion, smooth crack or rough crack etc. This is also important since it is
desirable to have the system retrieve similar types of defect.
These twenty-one parameters or attributes make up a case in the smartlQ system. This
case structure is then used to create the case base. This is carried out using an MS Access
database and CBR software "Orenge" by Empolis Knowledge Management. The case
structure is created in Orenge and then mapped to the database to identify the correct
records. Each record in the database then becomes a case in the case base. Cases can be
easily added or removed using the database. With the case structure defined a query case
can be created and sent to the smartlQ system. The next section will explain how the
smartlQ system carries out a CBR search and calculates the overall similarity.
1864
CASE RETRIEVAL
There are various retrieval techniques in CBR. Two of the most commonly used
techniques offered by commercial CBR tools are nearest neighbor and induction. The
latter uses a decision tree structure and usually is used for help desk and diagnosis type
problems. This is not suited to Inspection Qualification, which is much more akin to
design type problems. Therefore the smart IQ system uses a nearest neighbor technique to
calculate overall similarity. This is perhaps the most widely used retrieval technique of all
the CBR tools. A general nearest neighbor algorithm for overall similarity is given as:
Overall Similarity = ————————
(1 )
Where T is the target case; S the source case; n the number of parameters in the case
structure; / is the current parameter index from 1 to «;/a function describing the individual
similarity in cases T and S for the current parameter /; and w the weighting of the
parameter. This calculation is carried out for every case S in the case base and is usually
shown as a percentage, with 100% meaning an exact match. The nearest neighbors to the
target case T are then ranked by similarity. It can be seen from Equation 1 that the overall
similarity depends on the values of the weightings w and the similarity function f(T, S).
Therefore the knowledge behind the choice of these weightings and similarities is the most
critical component in the design of any CBR application. These topics will be covered in
the following sections.
WEIGHTINGS
Once the case structure has been constructed then each parameter of the case has to
be assessed and weighted according to its importance within a search. The weighting
assessment strategy involves carrying out knowledge elicitation. This means holding a
structured interview with experts, during which discussions take place that allow the
parameters of a case to be placed in order of importance and numerical weighting values
assigned. The logic behind such decisions is also recorded since this expert knowledge is
useful in its own right.
In order to prioritize twenty-one parameters, three weighting categories were
decided upon - strong, intermediate and weak. Each parameter was then assessed and
assigned to a group. This reduced the number of parameters that were worked upon at one
time. A 'paired weighting' prioritization technique was then employed to order the
parameters within each category. Paired weighting only works with small group sizes up
to approximately ten values. The technique involves comparing parameters in pairs and
passing the more important parameter on; this process is repeated until only the most
important parameter is left. This parameter is removed from the group and the whole
process is then iterated to yield a prioritized list, to which numerical weights are then
assigned. A scale of 2 to 10 is used, with 10 the most important.
If tilt misorientation is taken as an example, this is considered to be quite important
and was given a value of 7 by the experts since the ultrasonic response is strongly
1865
dependent on this factor. The weighting of some parameters, such as range, is more
complex, being dependent on other parameter values. These weights need to be changed
'dynamically'. For example the weighting of the range parameter varies from 5 to 8
depending on whether the defect is in the far-field of the probe, and whether the response
is DAC-corrected. For parameters that are not to be included in the search and associated
similarity calculation, a weight of zero can be assigned.
INDIVIDUAL SIMILARITIES
During a case-based search, queries are compared to cases in the case base and
overall similarity values calculated according to equation (1). For this purpose it is
necessary to define a set of individual similarity measures that apply to the range of values
of the particular parameters in the case. The type of similarity measure varies according to
the case parameter. The smartlQ system uses two types of parameter, numeric and textual,
resulting in two types of similarities, numeric and tabular. During knowledge elicitation
each case parameter is assessed and an intuitive, reasonable similarity measure decided
upon.
For example, Figure 2 displays a possible similarity measure for the case-parameter tilt
misorientation as a bell-shaped curve; similarity begins to fall off with increasing severity
as tilt misorientation deviates from the query value. At around ±10° deviation, similarity
is assigned a value around 80%, dropping off rapidly to zero for deviations of around ±15°.
These types of intuitive estimates are at the heart of a CBR system and allow the system to
be progressively refined according to user feed-back. Developers of CBR systems are now
looking to include the rational behind the chosen similarity in the result that is displayed to
the user. This would be beneficial as the resulting use of the system hinges on
believability.
llllllllil^^
:::;::::^:;::::;::::::::;:::;g:|;;:::s::::::s::::::;g:;::::g:s:?s|';::s:s
:::S:K::S:;:SS:E:-:E::fK:::K:::S:HS:-:ES:::KS::™t:E*:xE:
:::j::S:::j::::S;;::::::!::;:::::|i:::::::::iS:::::iS::::J:::::::S:::::S::::::<f::::::
i:::::::::::::::::::::::::::::::;::::::!;::::::::::::::::::::::;::::::::::::::::::::::::!:::::::::::::::
1
;:*:?•
SftHS
•III
|
|
|
|
|
|
|
|
|
|
:
|
|
i
^
|
|
|
|
|
|
|
|
|
|
|
|
|
|
:
|
|
|
|
|
H
|
sis;
::s:::s:g:::?g:!:^:::|:g:!:::;?:::::;^:::g:ss:s|sis:s
::s::::;::s:::::ss::::i;s:::|:K::::::sK::::si;s:s:s:s:::Ss:::i:s:::::
ill!
•
11
liiiiiiiiii^^
FIGURE 2. The numeric similarity curve for the Tilt Misorientation case parameter.
The other form of similarity used in the smartlQ system is that shown in Figure 3.
This is a tabular similarity for the case parameter 'defect type'. Here there are six possible
defect types in the query, which are compared with their counterparts in the casebase. The
diagonal represents a 'perfect' similarity and other legitimate comparisons have been
assigned reasonable estimated values. Thus, a lack of fusion defect has been described as
80% similar to a smooth crack and 20% to a rough crack.
1866
FIGURE 3. The tabular similarity showing the values for 'Defect Type' case parameter.
SYSTEM DEMONSTRATION
The user interface to the smartlQ system is a standard web browser. From there
the user can enter a query and click 'find'; the system will then retrieve a set of cases from
the case base which are similar to the query case. These are ranked in order with the most
similar first, allowing the user to immediately quote similar data where defect responses
have been detectable and if necessary construct arguments to justify their similarity.
The interface for constructing a query is shown in Figure 4. Data is entered via
drop-down lists or dialogue boxes. When "find" is clicked the system will return a page
displaying the query together with a set of ranked cases. An example is shown in Figure 5,
where cases are presented with an overall percentage similarity. The individual similarity
of each parameter is shown via a tool tip display when the cursor is held over a variable on
the screen.
/detailed search
/ Inspection and Defect Case
/ Inspection & Probe
Detection MQ
Probe Angle p..SO]
Range [Q..336]
Nearfield/Farfield
Wavel ength [0.31.. 1.62]
Defect Type
[^
[-none selected-|
Height [0.0..12.0] mm
1
Length [0.0..30.0] mm
^
Defect Skew [0.0..14.0] mm
[
Tilt Misorientation [O.O..S5.0]
Single/Twin
Defect Location
- none selected -
Wave Mode
Skip
/ Quick Find: Enteryour query regarding the defectfresponse
Dialog Service [true
find!
Marker Service [true
FIGURE 4. A screen shot of the query page from the smartlQ system.
1867
/ results of search
/ Y o u are Looking for:
Response: Detection Mode: Direct p-e . Probe Angle: 4€.Q degrees
Defect with height 10.0 mm , length 20.0 mm , Tilt MisoMentation 40.0 degrees
/ Found.....
/ Responses
DetectionMode: Direct p-e . Probe Angle: 45.0 degrees . Range 7.1 mm . Region: Hearfield , Wavelength: 0.81 mm . Crystal:
Single , WaveMode: S h e a r , SkipMode: Half, DAC Corrected: true , 12db Beamwidth ref: 3.7 degrees , 12db Beamwidth unref:
3.2 degrees , Range in Nearfields: 0.3 N , Response: -13.0 db
Type: Planar with height 10.0 mm . length 20.0 mm . skew 0.0 degrees . Tilt Misorientation 45.0 degrees , located: Surface
Breaking , Smooth?: Smooth (99 % similarity)
DetectionMode: Dired p-e , Probe Angle: 45.Q degrees , Range 7.1 mm , Region: Hearfield , Wavelength: 0.81 mm , Crystal:
Single , WaveMode: S h e a r . SkipMode: Half. DAC Corrected: true . 12db Beam width ref: 3.7 degrees . 12db Beamwidth unref:
3.2 degrees , Range in Nearfields: 0.3 N , Response: -12.0 db
Type: Planarwith height 10.0 mm , length 20.0 mm , skewS.Q degrees , Tilt Misorientation 45,0 degrees , located: Surface
Breaking . Smooth?: Smooth fl99 % similarity)
FIGURE 5. A screen shot of the results page from the smartlQ system.
CONCLUSIONS
Case-based reasoning has been shown to be a useful methodology when dealing
with experimental data during inspection qualification. It encapsulates knowledge relating
to the intelligent re-use of experimental data. This knowledge is the key to a good CBR
system and the resulting system is only as good as the knowledge captured for the
weightings and similarities. Early indications show that this system will be beneficial for
providing access to past results and for generating evidence for inspection qualification.
This will hopefully reduce the number of test blocks required.
ACKNOWLEDGEMENTS
This work has been funded by the UK EPSRC, BNFL-Magnox Generation and
British Energy pic.
REFERENCES
1. Kolodner, J., "Case-based reasoning", Morgan Kauffman Publishers Inc, San Mateo,
CA, 1993.
2. Watson, I., "Applying case-based reasoning for enterprise systems", Morgan Kauffman
Publishers Inc., San Francisco, CA, 1997.
1868
3. Aamodt, A., Plaza, E., "Case-Based Reasoning: Foundational Issues, Methodological
Variations and System Approaches", AI Communications, IOS Press, Vol. 7: 1, 1994,
pp. 39-59.
4. Leake, D.B., "CBR in Context: The Present and Future" in Case-Based Reasoning:
Experiences, Lessons and Future Directions, Menlo Park, AAAI Press, 1996.
5. Van't Hoff, P., Jarmulak, J., Kerckhoffs, E. J. H., Van't Venn, P., "Case-Based
Reasoning in a Ultrasonic Rail Inspection System" in 2nd International Conference on
Case-based Reasoning Proceedings, Springer Verlag, Berlin, July 1997, pp. 43-52.
6. Jarmulak, J., Van't Venn, P. P., Kerckhoffs, E. J. H., "Use of case-based reasoning for
interpretation of data from NDT", INSIGHT, Vol. 41, No 10, 1999, pp. 642-649.
7. Perner, P., "Case-based Reasoning for Image Interpretation in Non-Destructive
testing", EWCBR-93 Proceedings, Vol. 2, 1993, pp. 403-409.
1869