Applying Visual Data Analysis Techniques to Ultrasound Images

Solutions
Benefits
  • Viewing medical images
  • Custom analysis and visualization routines
The Reproductive Biology Research Unit at the Department of Obstetrics and Gynecology at the University of Saskatchewan, Canada, is mainly concerned with the study of infertility, while providing clinical services to couples who are having difficulty conceiving children, a problem shared by about 20 percent of the North American population.

Like many clinics of this type, researchers there rely heavily on ultrasound technology to generate pictures of internal body tissues. But Roger Pierson, an associate professor and the unit's director, has developed a method to enhance those images, opening new areas of medical inquiry.

The System

Over the past five years, Pierson has put together a computer-based system in which preprocessed ultrasound images are acquired by a digital acquisition system as they stream off the digital scan connector within the ultrasound device. They are then ported to a powerful SunTM Microsystems workstation for interrogation with what has come to be called visual data analysis (VDA) software. "We grab the data just before it goes into the video processor and stream it into the computers," Pierson says.

To analyze the data, Pierson and his team use PV-WAVE®, a VDA package from Visual Numerics, Inc. of Houston, Texas. PV-WAVE includes a fourth-generation language (4GL) for creating custom analysis and visualization routines, along with a library of canned functions for such tasks as convolution, filtering, and edge enhancement. Advanced math functions include Gaussian integrals, Fast Fourier Transforms (FFT), data point differentiation and interpolation.

"The software does some things exceptionally well, such as intensity mapping over images," Pierson says. "You can flip up an image and make a diagnosis very quickly. It helps keep track of all the images and provides the building blocks for our specialized research and analysis."

John Deptuch, a computer programmer who works with Pierson in the lab, says the most useful routine is one that allows him to take a two-dimensional array and shade it as a three-dimensional surface, where the height is based on the value in the array. "Writing that routine myself would have been an awful lot of work," Deptuch says.

The Application

Researchers at the University's College of Medicine are using the system in several applications. By studying computerized images of ovarian follicles, for example, they are learning to tell good follicles from bad. Good follicles contain a visible egg and are likely to perform their proper physiologic function. This increases the success rates for conception. In the long run, the information gathered could help scientists understand how to "turn on" the egg-producing mechanism in cases in which it is not functioning properly. Alternatively, researchers also want to understand how to turn the egg-producing mechanism off for contraceptive purposes. Another very promising area is cancer research. When Pierson and his team turned their equipment to study ovarian tumors, they realized that the visual data analysis could help them diagnose whether a cyst or tumor is malignant or benign based on a computer analysis of its structure.

This insight quickly spread to an interdisciplinary research project to verify the findings. Herb Yang in the university's Computer Science Department works closely with Pierson to guide the research. Today the team includes people throughout the university and medical community, from surgeons to computer graphics specialists, working together to establish a rigorous scientific and statistical basis for expanding these important findings beyond the clinic.

Old and New Analysis

Standard output from an ultrasound machine produces black-and-white images that are studied against a light board or on a video monitor. Whereas most people can only discern about 80 shades of gray through a visual inspection of ultrasound images, Pierson's computer can distinguish 256 shades. The addition of bandwidth filters, superimposed colors, and 3-D visualization techniques from PV-WAVE enables even finer distinctions and variations to be observed.

"For example, during a metastatic process, the blood flow to the organ increases," Pierson explains. This becomes apparent in an ultrasound image because the soundwaves decrease in their intensity and amplitude. Blood is a fluid, and fluids reflect a lower value echo. But in contrast to standard approaches, the computer can discern subtle differences that would be difficult to detect, such as the difference between an amplitude of 130 and an amplitude of 150 from a visual perusal of a black-and-white photographic image.

"Instead of an arbitrary scale of reference, we have exact figures that we can apply against an absolute scale," Pierson observes.

The distinctions Pierson can now make are important in understanding the processes associated with infertility because the release of an egg from the follicle is related to blood flow and to the action of the hormones on the tissue of the follicle. For example, when the leutinizing hormone (LH) is present in sufficient quantities to trigger the release of an egg, it causes those cells to hypertrophy. This gives off a different echo-intensity pattern than the pattern received from cells that don't respond to that hormone.

"We can look at the tissue and determine LH response based on tissue response," Pierson says. "If the LH receptors aren't there, the follicle tends to be very thin and bright-walled. The computer allows us to put hard numbers on these subtle changes. From these we can make predictions, such as, within this range of numbers, the likelihood of releasing an egg is 80 percent."

The new techniques are equally useful in the study of contraception. "We're trying to understand what the hormones released by the pituitary gland do to the ovaries on a monthly basis," Pierson explains. "We can use our infertility patients as models of the hormonal state necessary to prevent conception."

Buoyed by their results in infertility and contraception studies, Pierson and his colleagues have now turned to the study and detection of breast tumors to establish whether they can detect lumps that are not apparent through a physical exam, whether they can tell a fast-growing lump from a slow-growing one by its visual patterns, and, ultimately, a fibrocystic lump from a malignancy.

"Based on what we have seen so far, there is a different visual pattern displayed in the benign lumps than there is in the malignant ones, but it is too early to say much more," Pierson says. "We need to be able to identify those different tissue types, and we need to get the segmentation of the image to the point where we can reliably detect the lumps. Then we can begin to apply that knowledge to a generalized screening scan."

Looking Ahead

It's taken Pierson and his team about five years to perfect the techniques of ultrasound visualization. "Its worked; that's been the thrilling part," Pierson says. "Once you understand the physical components of bouncing sound through someone's body and generating an image from that, everything you do after that is nothing more than standard image-processing," Pierson explains. "It's what you interpret from the results of your studies that's important, and that is what the VDA software has given us -- a new way to interpret our data and better ways to manipulate it."