Center for Artificial Intelligence in Medicine

"Deep Learning can contribute to improving surgical outcomes."

August 2023

Negin Ghamsarian is a postdoctoral researcher at the ARTORG Center for Biomedical Engineering Research. She believes that self-supervised and semi-supervised deep learning can overcome current constraints in the feasibility of AI techniques for surgical video analysis to better predict postoperative complications and offer more precise surgical interventions.

With her research, Negin Ghamsarian seeks to open up more medical application areas for AI-based analytical systems to support medical teaching, predict potential post-operative complications and improve patient outcomes (zvg.).

Negin, can you tell us about your research?
My current research is focused on bridging the gap between deep-learning-assisted medical image and video analysis in laboratories and real-world conditions. An important aspect that is taken for granted in many deep-learning-based applications is the availability of labeled data. Indeed, despite the promising performance of supervised deep-learning methods in many healthcare-related tasks, including surgical phase recognition, object detection, localization, and semantic segmentation, their performance heavily relies on adequate annotations.
On the other hand, the diversities in different image and video acquisition devices, settings, and environmental conditions in hospitals or healthcare centers, may lead to a large domain shift between different datasets and hinder the generalizability of trained networks. In other words, a neural network trained on labeled images from one hospital may not achieve acceptable performance when evaluated on a dataset from another hospital or device.
The questions I pose in my research are: How can we reduce this reliance on annotations while not sacrificing the neural networks’ performance? And: How can we take advantage of the abundant unlabeled available data to bridge the cross-dataset distribution gap?

AI-based systems are poised to become increasingly indispensable in surgery.

Can you give an example?
Consider cataract surgery, the most frequently performed eye surgery and a highly demanded surgery worldwide. I have worked on DL-assisted analysis of cataract surgery videos since my Ph.D. studies. Working in an interdisciplinary environment in close collaboration with expert surgeons, I can argue that deep learning can considerably contribute to accelerating surgical training and improving surgical outcomes via phase recognition [1], relevance-based compression [2], intra-operative irregularity detection and postoperative complication prediction [3], and surgical scene understanding [4-5].

Deep-learning-assisted intraocular lens irregularity detection during cataract surgery.

Each of the mentioned tasks requires a specific level of annotation ranging from image to region to pixel level. Moreover, since the distribution of videos taken from different devices and in different hospitals is quite different, the trained networks can mainly provide satisfactory performance for the videos from the same device and hospital. Providing new annotations for each new dataset is hard to meet in terms of time and costs, especially in the medical domain, where domain knowledge is crucial. This difficulty in providing annotations is the bottleneck in deep-learning-based medical image analysis. While providing adequate annotations is challenging, there are abundant unlabeled images and videos in hospitals and healthcare centers, and the usability of such data to improve healthcare performance is undervalued.
Semi-supervised learning, self-supervised learning, and domain adaptation frameworks with different strategies, such as pseudo-supervision, consistency regularization, and contrastive learning, are the ways to address this problem. In the current stage of my research, I am developing semi-supervised learning and domain adaptation techniques tailored to the inherent features in surgical videos and volumetric images such as OCT and MRI to cut down the annotation requirements for reliable semantic segmentation in these domains [6].

A key factor to the applicability of deep learning in medicine is reducing the reliance on annotations.

You have studied in Iran and Austria. How do you like it in Bern?
The University of Bern has excellent connections with Inselspital, creating an ideal interdisciplinary environment for researchers in AI for medicine. Access to medical datasets and feedback from physicians and surgeons are crucial components of reliable research in this field, and the university provides ample opportunities for both. Additionally, the AIMI laboratory boasts a top-of-the-line hardware infrastructure that enables large-scale deep-learning-based evaluations integral to cutting-edge research. Beyond these academic advantages, the multicultural environment of the University of Bern creates a vibrant setting for professional growth.
It is a great opportunity for me to be a member of CAIM and work with a team of computer scientists and medical engineers. CAIM provides me with the opportunity to collaborate with other researchers in my field. This allows me to bounce ideas off my colleagues, get feedback on my work, and learn from their expertise.

What motivates you?
I have always been fascinated by the intersection of artificial intelligence and medicine, an area where AI can be beneficial, peaceful, and not destructive. Developing novel methods to deal with complex medical problems is deeply rewarding, as it can contribute to advancements in healthcare and positively impact patient well-being. Deep learning can provide a shortcut for how we approach medical diagnosis and surgical interventions, and this is the power engine that motivates me to go the extra mile.

How important will AI-based systems be in surgery in the future?
Thanks to technological advancements in surgery, operation rooms are evolving into intelligent environments. Context-aware systems are emerging as pivotal components of this evolution, empowered to advance pre-operative surgical planning, automate skill assessment, support operation room planning, and interpret the surgical context comprehensively. By enabling real-time alerts and offering decision-making support, these systems prove especially invaluable for less-experienced surgeons. Their capabilities extend to the automatic analysis of surgical videos, encompassing functions like indexing, documentation, and generating post-operative reports.
Indeed, AI-based systems are poised to become increasingly indispensable in the realm of surgery in the years to come. Addressing the existing limitations in AI-based medical image and video analysis will make computer-assisted diagnosis and surgery significantly more attainable in the foreseeable future.


Negin Ghamsarian has M.Sc. in Electrical Engineering and Ph.D. in Computer Science. During her Ph.D. studies at Klagenfurt University (Austria), she conducted thorough research on "Deep-learning-assisted Analysis of Cataract Surgery Videos." Her Ph.D. dissertation covers a broad spectrum of subjects, including supervised, semi-supervised, and self-supervised deep learning methods for image quality enhancement, video action recognition, object detection, and semantic segmentation for surgical videos.

Collaborating closely with renowned medical professionals in a multidisciplinary setting, she worked on developing innovative neural network architectures and deep-learning-based frameworks. These efforts aimed to tackle various challenges in surgical video analysis, contributing to the potential improvement of diagnosis, detection of irregularities, and prediction of postoperative complications. She is a postdoctoral researcher at the AI for Medical Imaging (AIMI) lab, ARTORG Center for Biomedical Engineering Research. Her work primarily focuses on semi-supervised learning and domain adaptation for medical and general images.


[1] Ghamsarian, N., Taschwer, M., Putzgruber-Adamitsch, D., Sarny, S. and Schoeffmann, K., 2021, January. Relevance detection in cataract surgery videos by spatio-temporal action localization. In 2020 25th International Conference on Pattern Recognition (ICPR) (pp. 10720-10727). IEEE.

[2] Ghamsarian, N., Amirpourazarian, H., Timmerer, C., Taschwer, M. and Schöffmann, K., 2020, October. Relevance-based compression of cataract surgery videos using convolutional neural networks. In Proceedings of the 28th ACM International Conference on Multimedia (pp. 3577-3585).

[3] Ghamsarian, N., Taschwer, M., Putzgruber-Adamitsch, D., Sarny, S., El-Shabrawi, Y. and Schoeffmann, K., 2021. LensID: a CNN-RNN-based framework towards lens irregularity detection in cataract surgery videos. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part VIII 24 (pp. 76-86). Springer International Publishing.

[4] Ghamsarian, N., Taschwer, M., Putzgruber-Adamitsch, D., Sarny, S., El-Shabrawi, Y. and Schöffmann, K., 2021. ReCal-Net: Joint Region-Channel-Wise Calibrated Network for Semantic Segmentation in Cataract Surgery Videos. In Neural Information Processing: 28th International Conference, ICONIP 2021, Sanur, Bali, Indonesia, December 8–12, 2021, Proceedings, Part III 28 (pp. 391-402). Springer International Publishing.

[5] Ghamsarian, N., Taschwer, M., Sznitman, R. and Schoeffmann, K., 2022, September. DeepPyramid: Enabling Pyramid View and Deformable Pyramid Reception for Semantic Segmentation in Cataract Surgery Videos. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 276-286). Cham: Springer Nature Switzerland.

[6] Ghamsarian, N., Tejero, J.G., Neila, P.M., Wolf, S., Zinkernagel, M., Schoeffmann, K. and Sznitman, R., 2023. Domain Adaptation for Medical Image Segmentation using Transformation-Invariant Self-TrainingarXiv preprint arXiv:2307.16660.