Lecture video summarization

Detecting and summarizing handwritten whiteboard content in lecture videos.

Lecture video is a useful record of a class, and it is hard to search. In a whiteboard lecture the useful signal is the handwriting, the math, and the sketches, and the lecturer keeps walking in front of it.

This line of work, with Bhargava Urala Kota, Kenny Davila, Srirangaraj Setlur, and Venu Govindaraju, detects that handwritten content and turns it into a compact summary. A text detector is adapted to handwriting, formulas, and drawings. Later methods represent the detected regions so that the same content can be tracked while it grows, disappears, or is occluded, and a spatio-temporal conflict criterion picks keyframes or key regions. The 2019 workshop paper on key handwritten content regions received a best paper award.

The experiments use the public AccessMath dataset. We also added timestamped bounding-box annotations for the handwritten content. The papers are listed below, with public PDFs.

References

2021

  1. Automated Whiteboard Lecture Video Summarization by Content Region Detection and Representation
    Bhargava Urala Kota, Alexander Stone, Kenny Davila, and 2 more authors
    In 2020 25th International Conference on Pattern Recognition (ICPR), 2021

2019

  1. Generalized Framework for Summarization of Fixed-Camera Lecture Videos by Detecting and Binarizing Handwritten Content
    Bhargava Urala Kota, Kenny Davila, Alexander Stone, and 2 more authors
    International Journal on Document Analysis and Recognition (IJDAR), 2019
  2. Summarizing Lecture Videos by Key Handwritten Content Regions
    Bhargava Urala Kota, Saleem Ahmed, Alexander Stone, and 3 more authors
    In 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), 2019
    Best paper award

2018

  1. Automated Detection of Handwritten Whiteboard Content in Lecture Videos for Summarization
    Bhargava Urala Kota, Kenny Davila, Alexander Stone, and 2 more authors
    In 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), 2018