Project

Agents for Integrated Annotation and Retrieval of Images

Effective image annotation and retrieval is bound up with image use. In this project, annotation, retrieval, and use are integrated, facilitating the finding and using images. A proactive user-interface agent seeks chances for image annotation and retrieval in the context of the user's everyday work, using an agent that sit in the user's text editor or other application and continuously monitors typing. Searches are automatically performed from an image library, and images relevant to the current text can be inserted in a single operation. Descriptions of images for storytelling can be seamlessly employed as raw material for annotation. Common-sense knowledge about situations in which pictures are taken, described, or used can help provide semi-automatic annotation and indirect inference for retrieval. Our approach does not completely automate the annotation/retrieval process, but it does reduce user-interface overhead, leading to better-annotated image libraries and fewer missed opportunities for image use.