SmartBook: AI-Assisted Situation Report Generation
Revanth Reddy, Yi Fung, Vicki Zeng, Manling Li, Ziqi Wang, Heng Ji
Contact: hengji@illinois.edu, revanth3@illinois.com
Please email Revanth Reddy if you experience any technical issues using our software or need further information.
About
Emerging events require timely and comprehensive situation understanding for effective decision-making and response. Automated generation of situation reports can reduce the time, effort, and cost of domain experts preparing their official reports. We propose SmartBook, a novel task for generating situation reports: given large volumes of news data, produce a structured report with multiple hypotheses (claims) summarized and grounded to sources with factual evidence. These reports are structured as a sequence of timespans, each consisting of major events, subdivided by strategic questions with their grounded summaries. SmartBook automatically discovers event-related questions, offering a more focused approach than nuanced, high-level analyst questions. Qualitative evaluations indicate over 80\% of questions probe for strategic information, and over 90\% of summaries produce tactically useful content. Experts tend to add more information to the generated reports than they remove (under 2.5\% of existing tokens), indicating SmartBook provides analysts with a valuable foundation for situation reports.
Software
SmartBook can be accessed and downloaded at https://github.com/blender-nlp/SmartBook.
Acknowledgement
We are grateful to Dr. Clare Voss from the Army Research Laboratory for her helpful insights and feedback. We would also like to acknowledge Lisa Ferro and Brad Goodman from MITRE for their valuable comments and help with expert evaluation. This research is based upon work supported by U.S. DARPA AIDA Program No. FA8750-18-2-0014, DARPA KAIROS Program No. FA8750-19-2-1004, DARPA SemaFor Program No. HR001120C0123, DARPA INCAS Program No. HR001121C0165 and DARPA MIPS Program No. HR00112290105. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of DARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.
References
Revanth Reddy, Yi Fung, Vicki Zeng, Manling Li, Ziqi Wang, Heng Ji. 2023. SmartBook: AI-Assisted Situation Report Generation. Arxiv 2023.