Ongoing projects

Activity recognition for tool-tissue interaction understanding

• Collate and create datasets of surgical videos labeled as triplets <instrument, verb, target> for detailed and fine-grained modeling of surgical activities.
• Develop deep learning models based on multi-task learning, weak supervision, and attention/transformer for surgical action triplet detection.
• Develop standard python library “ivtmetrics”- open source, now available as pip & conda python package installers for public use.

  project page	datasets	publication 1
  publication 2	int’l challenge 1	int’l challenge 2

IVTMETRICS: Open source project on triplet performance metrics

• Design algorithm for the unified evaluation of state-of-the-arts methods on surgical action triplet recognition and detection.
• Build python package library for the evaluation metrics
• Host and deploy metrics for all OS platforms.

  project page

  publication

  GitHub

  PyPi.org

  Anaconda.org

Surgical tool tracking for context-aware intraoperative assistance

• Building deep learning models for surgical instrument tracking in endoscopic videos, achieved by modeling spatial localization and temporal consistency in moving images in a video using weak supervision.
• Resultant app now integrated in the NVIDIA Holoscan SDK App.

  project page

  publication

  applications

  news

Completed projects

• Melanoma skin cancer detection using machine learning

• Automatic diagnosis of cardiovascular diseases

• Short message service converter for android operating system

• Fingerprint recognition system for attendance management

• Billing system for mobile telephone networks