DaVinci Robot Supervisory Control

Supervisory Control of a DaVinci Surgical Robot

• This section elaborates on a novel approach tailored to the supervisory control of the DaVinci surgical robot. This advanced method aims to enhance the efficiency and precision of robotic surgical procedures, which are increasingly common in modern medicine.
• Current surgical robots primarily operate through teleoperation, which significantly complicates the visualization of kinematic restrictions. This limitation poses challenges for operators, particularly during complex tasks that require precise movements, such as circular needle driving, where spatial awareness is critical for successful outcomes.
• The interface designed for this supervisory control system effectively transitions the operator from the classic teleoperation mode to a more advanced supervisory mode, allowing for greater oversight and control over surgical tasks.
• Operators engage with an innovative point-cloud display, enabling them to select specific task specifications with greater ease. This sophisticated interaction facilitates the automatic computation and execution of the selected tasks by the robotic system, thereby enhancing the workflow.
• The primary aim of this development is to streamline operational responsibilities, allowing the surgeons to concentrate on defining task parameters while the automation system handles the execution of procedures, resulting in quicker and more accurate surgical outcomes.

Introduction to Robotic Surgery

• Robotic Assisted Minimally Invasive Surgery (RAMIS) is a transformative approach that provides numerous advantages, including smaller incisions, accelerated recovery times, and diminished infection risks. These benefits are crucial in improving patient outcomes and reducing surgical complications.
• Despite its advantages, current robotic surgery techniques often lag behind traditional surgery regarding speed and dexterity, highlighting the need for further advancements in robotics.
• A significant hurdle in robotic surgery is the lack of tactile feedback, which is integral to many surgical procedures. Furthermore, issues related to limited endoscopic vision and the restricted workspace of robotic systems complicate surgical maneuvers.
• Particularly in minimally invasive surgeries, tasks such as suturing and knot-tying become increasingly challenging, requiring substantial time and skill from operators, which can lead to operational bottlenecks.
• The applications of RAMIS remain primarily focused on direct teleoperation, which limits the potential for automation and efficiency improvements.
• The potential for future developments could involve the automation of several key tasks, including needle driving, suture pulling, and knot-tying, carried out under the supervision of trained personnel. This could significantly enhance the workflow within surgical teams.
• In practice, surgeons would utilize a visual display to select target locations for surgical interventions, effectively allowing the robotic system to execute actions that are both faster and more precise than manual alternatives.

Circular Needle-Driving Path

• Numerous researchers have put forward various circular needle-driving pathways to mitigate tissue deformation, a critical consideration in preserving organ integrity during surgical procedures.
• This paper emphasizes the automatic execution of RAMIS circular needle-driving, an innovative process that aims to modernize and enhance the effectiveness of current surgical practices.
• A key feature of this system is the incorporation of fast inverse kinematics, which facilitates efficient and accurate needle driving using the DaVinci robot.
• An easily navigable machine interface is introduced to streamline the process of establishing task specifications for needle driving, thus making the operator's role less cumbersome.
• This development encompasses comprehensive automated planning and execution methodologies, which are critical for the successful management of needle driving tasks.
• The solution is currently being developed and evaluated within advanced simulations facilitated by the Robot Operating System (ROS), thereby extending the operational capabilities of the DaVinci Research Kit (DVRK) and its applications.

System Modeling

• The system utilizes a DaVinci surgical robot equipped with a stereo endoscope and two patient-side manipulators, which form the backbone of its operational functionality.
• Notably, a Patient-Side Manipulator (PSM) can be strategically pre-positioned via non-servoed joints to accurately establish the three-dimensional coordinates of an entry portal, enhancing precision before the surgical procedure begins.
• It is essential to highlight that repositioning the non-servoed joints can be accomplished only under the constraint that the portal remains fixed, ensuring stability throughout the process.
• During teleoperation stages, the focus shifts to moving only the distal, servoed joints, which include advanced functioning capabilities that aid in precise surgical manipulation.
• The servoed joints consist of three foundational “base” joints, in addition to as many as four tool joints that accommodate interchangeable tools for various surgical tasks.
• The primary research interest focuses on a specially designed needle-driver tool that possesses four degrees of freedom, allowing for roll, pitch, and rotational movements essential for diverse surgical applications.
• A detailed model of the DaVinci surgical robot has also been conceptualized in previous research conducted at notable institutions, including Johns Hopkins and Worcester Polytechnic Institute, which adds substantial depth to the existing knowledge base.
• This existing model enables the visualization of the servoed joints through “Rviz,” a highly interactive visualization tool embedded within the ROS ecosystem.
• Joint angles can be directly commanded through ROS, resulting in a dynamic and animated representation of the robotic arms, thus providing valuable feedback for operators.
• Visualization through this tool can prove highly beneficial for developing, testing, and refining motion plans prior to executing surgical interventions, lowering the risk of errors during actual procedures.

DaVinci Model and Environment

• It is crucial to note that Rviz primarily functions as a visualization tool and does not account for various factors such as robot dynamics, the physical interactions between different components, nor does it emulate sensor inputs.
• In contrast, "Gazebo" simulation offers a range of advanced capabilities centered around animated visualization, which is instrumental for practical applications in robotic surgery.
• Gazebo integrates a robust physics engine that provides dynamic simulation capabilities and the flexibility to modify the modeled environment according to specific operational needs.
• Additions to the Gazebo simulation of the DaVinci robot facilitate the emulation of stereo cameras, creating opportunities for developing and simulating sensory-driven behaviors that enhance the robot's responsiveness.
• The realistic physics simulation of the DaVinci endoscope capitalizes on image capture of 640x480 resolution, offering a horizontal field-of-view of 0.70 radians, alongside a baseline (inter-ocular distance) of approximately 5.8mm, which collectively support authentic visual feedback.
• Emulation efforts aim to replicate stereo vision that can be reflective of the physical realities the surgical robot will encounter during procedures, thereby improving operational accuracy.
• A particularly noteworthy feature is the introduction of textured models within the simulation, which are generated from a snapshot of a textured surface captured approximately 80mm away using the DaVinci endoscope, enhancing realistic interactions.
• Rviz’s functionalities have been significantly improved, allowing for immersive three-dimensional displays of point-cloud values that are derived from stereo cameras, thereby enhancing visualization clarity.
• Integrating point-cloud data in the Rviz environment allows for superior 3D visualization, providing the ability to rotate, zoom, and manipulate the visual output for better assessment and decision-making.
• The interface also features a natural user experience, allowing the operator to interactively select and manipulate points of interest through a straightforward click-drag operation with a mouse, thereby intuitively indicating 3D points that require focus.
• Simulations incorporate a comprehensive understanding of the joint limits associated with the servoed joints, which is imperative for accurate automated trajectory planning.
• Taking joint limits into account is a critical aspect, as these constraints are vital for ensuring smooth operational flows during robotic surgeries and providing graphic support to operators throughout the procedure.

Supervisory Control Interface

• One of the most significant components of this system is the ability to plan potential trajectories efficiently, notifying the operator about viable options, which necessitates the use of a rapid and dependable inverse kinematics solution.

Analytic Inverse Kinematics

• Previously conducted research has led to the development of specialized inverse-kinematic (IK) software tailored for the DaVinci surgical robot, improving the speed and accuracy of surgical movements.
• This prior work employed mathematical techniques utilizing Jacobians and numerical iterations to derive solutions to IK problems, which has been fundamental in robotic motion planning.
• However, numerical inverse-kinematic approaches often face challenges such as failure to converge to a solution and difficulties in obtaining multiple, valid solutions.
• An evaluation of the current numerical IK implementations has revealed multiple instances where the system struggled to find known valid solutions, encountered convergence issues, or ended up with solutions that breached legal bounds.
• In contrast, adopting an analytic solution framework alleviates these drawbacks, providing a more reliable and stable means of solving inverse kinematics challenges.

Inverse Kinematics Approach

• The specific needle-driver tool under consideration in this analysis is known as the Endowrist, featuring advanced capabilities that enhance precision in various surgical tasks.
• This tool, in conjunction with DaVinci joints, offers a considerable six degrees of freedom for the motion of the tool tip, which includes grip actuation, thereby expanding the potential for complex surgical maneuvers.
• Implementing an analytic inverse-kinematic (IK) solution is preferred due to its speed and reliability in calculating joint movements made by the robot, ensuring operational efficiency during procedures.
• While iterative numerical techniques focused on the calculation of the Jacobian inverse represent an alternative for solving these problems, they tend to present less favorable conditions in terms of speed and convergence.
• The specific inverse kinematics capability provided by the DaVinci Research Kit is largely derived from a numerical Newton algorithm. Although this algorithm has merits, it is not always ideal for high-speed applications.
• As noted, numerical IK solutions may experience slow convergence rates, complete failure to converge, or resolve to inappropriate solutions that do not conform to the physical constraints of the robotic system.
• The kinematic framework of this robot system can be effectively expressed through Denavit-Hartenberg (D-H) representation, a standard methodology in the field.
• The specific operational limits for the joints are outlined as follows: -1 ≤ θ1 < 1, -0.7 ≤ θ2 ≤ 0.7, 0.01 ≤ d ≤ 0.23, -2.25 < θ4 < 2.25, -1.57 < θ5 ≤ 1.57, and -1.39 ≤ θ6 ≤ 1.39. Such constraints provide important baselines for valid movement calculations.
• The complete geometric solution to the IK problem is often decomposed into two distinct parts to enhance efficiency and clarity.
• Specifically, a spherical "wrist point" is defined, which relates to the origin of D-H frame 4 and serves as a crucial reference within this context.

IK Approach

• Starting from the defined wrist point, solutions for the first three joint displacements, denoted as θ1, θ2, and d3, can be systematically calculated to establish initial positioning.
• Upon computation of these first three joint values, determining the remaining three joint values, which pertain to the wrist rotations, becomes a more straightforward process.
• Visualization of the system requires the establishment of a base frame, with the origin (represented as obase) aligned to the portal coordinates. Frame 4's z-axis is directed through the wrist-bend rotation axis, while the z-axis for position z5 aligns with the last joint degree of freedom, which governs the gripper-jaw rotation.
• The x5 axis indicates the orientation from origin 04 towards origin 05 while incorporating a D-H offset parameter labeled a5.
• It is advantageous to consider a desired configuration for the gripper, represented as baseTtip, which describes the frame associated with the gripper tip relative to the established base frame.
• A geometrical approach to resolving the IK can be implemented effectively, facilitating the accuracy of the proposed orientations and movements.
• The formation of two planes, Pparallel and Pperp, assists in streamlining the calculations and improves the overall process of establishing a solution.
• By deriving a means to compute x5 from a desired baseTtip, one can ascertain the wrist point 04, resulting in definitive numerical values that align with the operation framework.

Analytical vs Conventional Methodology

• The distance necessary for tool insertion, denoted as d3, is equivalent to ||04||, which represents the fourth column of the product of 4x4 matrices utilized in the calculations.
• Importantly, within the established limits for the Endowrist joints, these resultant solutions are found to be unique, lending reliability to the approach.
• The discovered values for θ1, θ2, and d3 are then employed in the computation of A3(θ1, θ2, d3), facilitating extended motion planning.
• Notably, a unique and useful property of the IK solution pertaining to the DaVinci robotic arm in conjunction with the Endowrist is that when a valid solution is obtainable (within defined joint limits), this solution is indeed unique.
• The analytic IK solution approach efficiently determines whether a valid IK solution exists; furthermore, if a solution is present, it quickly and precisely calculates the unique answer sought after.
• To validate the inverse-kinematics solution's integrity, a numerical test was conducted, culminating in successful confirmation that the IK solution was both reliable and unique.
• The analytic solution exhibited performance that was approximately eight times faster than the numerical approach, showcasing its advantages for real-time applications.

Human/Machine Interface

• Utilizing the Gazebo model of the DaVinci system, which incorporates stereo vision emulation, it is possible to leverage Rviz’s capabilities to develop a sophisticated graphical user interface that supports intelligent supervisory control mechanisms.
• The first visual scene presented to the operator depicts a colored 3-D point-cloud, which represents the operator’s perspective as they engage with the surgical sample during robotic interventions.
• This visual setup will be enhanced with green fiducials strategically placed to serve as reference points, facilitating navigational clarity.
• The operator is afforded the opportunity to rotate and translate the scene, promoting a more comprehensive understanding of the three-dimensional environment and supporting optimal viewpoint selection.
• The user experience is furthered by recommending the use of Rviz's "Publish Selected Points" feature, which aids in establishing centroids for task execution.
• A separate planner node is designed to subscribe to information pertaining to the centroids of selected points, which iteratively refines proposed surgical maneuvers based on incoming data.
• As part of the operational approach, the sequence of gripper poses is systematically expressed in camera space, allowing for more effective tracking and execution of movements.
• This representation involves four key parameters: the 3-D coordinates of the anticipated needle center, alongside the "yaw" angle that governs the needle axis in relation to a normal rotational axis concerning tissue.
• Constraints on the circular-arc path focus on the transformation of the needle-to-gripper setup, emphasizing the gripping mechanism applied to the needle itself.
• Additionally, there is an assumption made that the plane of the needle remains perpendicular to the tissue surface throughout the surgical process, an important consideration for maintaining precision.

Needle Driving

• Within the framework of circular needle-drive pathways, employing a practical reference frame is essential. This reference frame incorporates the center of the circular needle arc and a vector that is normal to the needle plane, known colloquially as the "needle axis."
• During circular driving maneuvers, the needle consistently maintains a constant center and axis, which is critical for minimizing tissue trauma and ensuring reliable outcomes.
• Operators select viable exit points, which are then published for real-time utilization by the Human-Machine Interface (HMI).
• The HMI node actively subscribes to the exit-points topic, subsequently displaying markers at precise coordinates calculated for potential exit points, thereby aiding operator decision-making.
• The needle-planner node operates by considering the entirety of needle-drive paths, evaluating them in light of hypothetical exit points that have been established as viable.

Auto Execution

• To engage effectively, the operator will click on a point positioned near one of the illustrated and viable exit points presented within the interface.
• Upon selecting an exit point, the HMI effectively erases alternative exit-point options, visually marking the chosen exit point with a blue indicator to signify its activation.
• The process is illustrated through real-time 3D rendering in Rviz; thus, allowing the operator to observe the selection of a desirable entry point and, if feasible, designate a corresponding exit point for the needle driving.
• Upon completion of the maneuver, the trajectory executed by the robotic system ensures that the needle penetrates the tissue precisely at the prescribed entry point and subsequently exits at the designated exit point, thereby fulfilling the objective with high accuracy.

Conclusions and Future Work

• The findings presented in this document mark an important initial step towards the development of enhanced supervisory control for robotic surgical systems, signaling a progression in surgical technologies.
• The computation of viable operational paths is enabled through the application of a fast and reliable inverse kinematics solver, which is vital for enhancing surgical workflow efficiency.
• Future work will focus on extending the supervisory control capabilities associated with needle driving to encompass a broader set of operational options and strategic pathways.
• Recognizing that such requirements, constraints, options, and strategies may not be immediately evident to the operator—a situation compounded by the inherent complexities of joint-limit constraints that are often difficult to visualize—efforts will prioritize intuitive design.
• Currently ongoing developments aim to extend the automation capabilities further, incorporating not only needle driving but also needle hand-off procedures to finalize the needle drive and sophisticated knot-tying mechanisms that could significantly streamline surgical workflows.