%0 Conference Proceedings %T Toward a Neural-Symbolic Framework for Automated Workflow Analysis in Surgery %+ Department of Computer Science [Verona] (UNIVR | DI) %+ Department of Electronics, Information, and Bioengineering [Milano] (DEIB) %+ European Institute of Oncology [Milan] (ESMO) %+ Laboratoire Traitement du Signal et de l'Image (LTSI) %A Nakawala, H. %A de Momi, E. %A Bianchi, R. %A Catellani, M. %A de Cobelli, O. %A Jannin, Pierre %A Ferrigno, G. %A Fiorini, P. %Z European Metrology Programme for Innovation and Research, EMPIR 742671European Research Council, ERCAmerican Radium Society, ARS732515 %< avec comité de lecture %B 15th Mediterranean Conference on Medical and Biological Engineering and Computing, MEDICON 2019 %C Coimbra, Portugal %Y Henriques J.de Carvalho P.Neves N. %I Springer %V 76 %P 1551-1558 %8 2019-09-26 %D 2019 %R 10.1007/978-3-030-31635-8_192 %K Deep learning %K Inductive logic programming %K Production rules %K Robot-Assisted Partial Nephrectomy %K Surgical workflow %Z Life Sciences [q-bio]/BioengineeringConference papers %X Learning production rules from continuous data streams, e.g. surgical videos, is a challenging problem. To learn production rules, we present a novel framework consisting of deep learning models and inductive logic programming (ILP) system for learning surgical workflow entities that are needed in subsequent surgical tasks, e.g. “what kind of instruments will be needed in the next step?” As a prototypical scenario, we analyzed the Robot-Assisted Partial Nephrectomy (RAPN) workflow. To verify our framework, first consistent and complete rules were learnt from the video annotations which can classify RAPN surgical workflow and temporal sequence at high-granularity e.g. steps. After we found that RAPN workflow is hierarchical, we used combination of learned predicates, presenting workflow hierarchy, to predict the information on the next step followed by a classification of step sequences with deep learning models. The predicted rules on the RAPN workflow was verified by an expert urologist and conforms with the standard workflow of RAPN. © 2020, Springer Nature Switzerland AG. %G English %L hal-02442593 %U https://univ-rennes.hal.science/hal-02442593 %~ UNIV-RENNES1 %~ LTSI %~ STATS-UR1 %~ LTSI-MEDICIS %~ UR1-HAL %~ UR1-MATH-STIC %~ TEST-UR-CSS %~ UNIV-RENNES %~ UR1-MATH-NUM %~ UR1-BIO-SA