Raconteur is a story-editing system for conversational storytelling that provides intelligent assistance in illustrating a story with photos and videos from an annotated media library. It performs natural language processing on a text chat between two or more participants, and recommends appropriate items from a personal media library to illustrate a story. A large common-sense knowledge base and a novel common-sense inference technique are used to find relevant media materials to match the story intent in a way that goes beyond keyword matching or word co-occurrence based techniques. Commonsense inference can identify larger-scale story patterns such as expectation violation or conflict and resolution, and helps a storyteller to chat and brainstorm personal stories with a friend.