Projects
Annotation and Image Markup (AIM) Project
Images, in particular medical and scientific images, contain vast amounts of information. While this information may include metadata about the image, such as how or when the image was acquired, the majority of image information is encoded in the images pixels; however, information about how images are perceived by human or machine observers is not currently captured in a form that is directly tied to the images. A wealth of information pertaining to image content is thus disconnected from the images, limiting the value of radiology imaging to be related to other non-imaging data. We need tools that will allow both human and machine image annotations to be created and stored in a standard format that is syntactically and semantically interoperable with the infrastructure with other biomedical resources while supporting standards such as DICOM, HL7 and those being created by the W3C semantic Web community.
We are developing methods to describe the semantic content in images using ontologies, explicit representations of the entities and relations in biomedicine. We are also creating tools to compose ontology-based descriptions of image content and associate them with images. This work will change the paradigm of medical imaging—instead of clinical systems storing just pixels, they will store image data plus the image meaning. This will enable a broad range of computational analytic functionality, including semantic search (see IQ Project), integration of image- and non-image data, statistical modeling of disease, and intelligent decision support applications for image-based personalized care.
Content Based Image Retrieval for Decision Support
The number of images in Radiology is exploding. Diagnostic radiologists are confronted with the challenge of efficiently and accurately interpreting cross sectional imaging exams that often now contain thousands of images per patient study. Currently, this is largely an unassisted process, and a given reader's accuracy is established through training and experience. There is significant variation in interpretation between radiologists, and accuracy varies widely, a problem compounded by increasing image numbers. There is an opportunity to improve diagnostic decision making by enabling radiologists to search databases of radiological images and reports for cases that are similar in terms of shared imaging features to the images they are interpreting.
We are creating software tools that can be used to create and to search databases of radiological images based on image features, which include detailed information about lesions: (1) feature descriptors coded by radiologists using RadLex, a comprehensive controlled terminology, and (2) computer-generated features of pixels characterizing the lesion's interior texture and the sharpness of its boundary.
Our goal is to develop methods to facilitate the retrieval of radiological images that contain similarly appearing lesions. We are currently developing a CBIR system in CT images of the liver.
Bayesian Modeling for Decision Support
Radiology interpretation is challenging because there are many different imaging features and variation in how different radiologists
combine the evidence of multiple combinations of these features into a decision about diagnosis or patient management (e.g., should
this patient undergo biopsy?) Such decisions can potentially be improved through artificial intelligence methods such as Bayesian
networks.
We are currently pursuing a number of projects to create models relating observed radiology imaging features to the possible diagnoses and decision points. We are developing these models to improve the diagnosis of breast cancer, to evaluate whether negative results of biopsy could be due to sampling error, and to help radiologists evaluate the malignant potential of thyroid nodules.
DICOM Ontology (DO) Project
DICOM (Digital Imaging and Communications in Medicine) is the global standard for medical image information; a position it has held for over 20 years. It is pervasive throughout the medical imaging community, and nearly every medical imaging device supports some aspect of the standard. DICOM models the image acquisition process and information objects related to imaging, and it specifies how the image data, the metadata, and related objects are represented in a binary format. For example, DICOM models patients as both clinical and clinical trial subjects, imaging studies that consist of series of images as well as all of the technical parameters of imaging modalities. Despite its size and complexity, DICOM lacks a Reference Information Model of the imaging domain. A reference information model is a formal description of a domain that enables users to share consistent meaning and establish semantic interoperability beyond a local context.
There is a pressing need for an information model of imaging based on DICOM to enable the community to create intelligent imaging-based applications that are interoperable. We are developing the DICOM Ontology (DO), an ontology that will be a single common reference information model for the imaging domain. The DO will be analogous to the Gene Ontology (GO) and serve a similar role in radiology that GO serves in biology. The DO will unify and make explicit all the key entities and relations in DICOM in a human-usable and machine-processable format. The DO will ultimately become a reference ontology—one that comprehensively represents knowledge about the medical imaging domain independent from specific objectives or applications, guided by a theory of the imaging domain and by robust ontology design principles that encourages reuse.
Image Query (IQ) Project
Image query is a critical functional component of systems that integrate biomedical images with non-image data. Query tools are vital in order for them to find and retrieve information in all bioinformatics databases. Many biomedical repositories are accruing a wealth of images, such as the National Cancer Imaging Archive (NCIA) and the American College of Radiology Imaging Network (ACRIN), which are building image collections from diverse clinical trials. The current repositories provide the research community technologies to federate data archives, but techniques are needed to permit researchers to explore the various resources, pose questions, correlate image data with related non-image data, and formulate new hypotheses and research directions. There is an emerging need for intelligent image query tools to enable users to search the image resources in an intuitive way. Our goal is to create image query tools to help users create queries that exploit the capability of biomedical ontologies to enable search for images that are annotated using these knowledge sources.
In the IQ project, we will develop semantic methods for searching for annotated images. We will address these challenges: (1) Complexity if image content and semantics, (1) Relating radiology imaging to other non-imaging data, and (3) Terminology challenges of synonymy and polysemy. We will address these challenges by creating an ontology to support image query. An ontology is an explicit knowledge representation that specifies the entities and relations among those entities in a domain in a human-readable and machine-processable format. We will create methods to permit users to search for images based ontology terms, and the ontologies will also be used to expand user queries. We will also develop an intuitive interface to accessing the ontologies and composing queries.