SmartBook: AI-Assisted Situation Report Generation

Revanth Reddy, Yi Fung, Vicki Zeng, Manling Li, Ziqi Wang, Heng Ji
Contact: hengji@illinois.edu, revanth3@illinois.com

Please email Revanth Reddy if you experience any technical issues using our software or need further information.


About

Emerging events require timely and comprehensive situation understanding for effective decision-making and response. Automated generation of situation reports can reduce the time, effort, and cost of domain experts preparing their official reports. We propose SmartBook, a novel task for generating situation reports: given large volumes of news data, produce a structured report with multiple hypotheses (claims) summarized and grounded to sources with factual evidence. These reports are structured as a sequence of timespans, each consisting of major events, subdivided by strategic questions with their grounded summaries. SmartBook automatically discovers event-related questions, offering a more focused approach than nuanced, high-level analyst questions. Qualitative evaluations indicate over 80\% of questions probe for strategic information, and over 90\% of summaries produce tactically useful content. Experts tend to add more information to the generated reports than they remove (under 2.5\% of existing tokens), indicating SmartBook provides analysts with a valuable foundation for situation reports.


Figure 1: An overview of SmartBook. The user can select the situation (F1), explore strategic questions related to a topic (F2), navigate sections by dates(F3), read an overarching summary on the strategic question (F4), and investigate the claims in the summary (F5).


Software

SmartBook can be accessed and downloaded at https://github.com/blender-nlp/SmartBook.


Acknowledgement

We are grateful to Dr. Clare Voss from the Army Research Laboratory for her helpful insights and feedback. We would also like to acknowledge Lisa Ferro and Brad Goodman from MITRE for their valuable comments and help with expert evaluation. This research is based upon work supported by U.S. DARPA AIDA Program No. FA8750-18-2-0014, DARPA KAIROS Program No. FA8750-19-2-1004, DARPA SemaFor Program No. HR001120C0123, DARPA INCAS Program No. HR001121C0165 and DARPA MIPS Program No. HR00112290105. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of DARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.


References

Revanth Reddy, Yi Fung, Vicki Zeng, Manling Li, Ziqi Wang, Heng Ji. 2023. SmartBook: AI-Assisted Situation Report Generation. Arxiv 2023.

@article{reddy2023smartbook, title={SmartBook: AI-Assisted Situation Report Generation}, author={Reddy, Revanth Gangi and Fung, Yi R and Zeng, Qi and Li, Manling and Wang, Ziqi and Sullivan, Paul and others}, journal={arXiv preprint arXiv:2303.14337}, year={2023} }