www.medpagetoday.com/Surgery/GeneralSurgery/42180?xid=nl_mpt_DHE_2013-10-10&utm_content=&utm_medium=email&utm_campaign=DailyHeadlines&utm_source=WC&eun=g217727d0r&userid=217727&[email protected]&mu_id=5213083
(This page requires a free membership, but I'll copy it anyway)
Peer Rating Strategy Predicts Bariatric Outcome
Published: Oct 9, 2013 | Updated: Oct 10, 2013
By Salynn Boyles, Contributing Writer, MedPage Today
Reviewed by F. Perry Wilson, MD, MSCE; Instructor of Medicine, Perelman School of Medicine at the University of Pennsylvania and Dorothy Caputo, MA, BSN, RN, Nurse Planner
Action Points
* Note that this cohort study suggests that bariatric surgeons with higher peer-assessed operative skill have lower surgical complication rates.
*Be aware that higher surgical volume, not years in practice, was associated with higher skill ratings.
* Patients of bariatric surgeons deemed to have poor skills by peers who watched them perform a procedure were almost three times more likely to have complications and five times more likely to die than those treated by top-rated surgeons, a study found.
The complication rate among patients treated by surgeons in the bottom quartile was 14.5%, compared with 5.2% among those treated by surgeons in the highest quartile (P<0.001) John D. Birkmeyer, MD, of the University of Michigan Center for Healthcare Outcomes and Policy in Ann Arbor, and colleagues reported.
The mortality rate for patients of the lowest-ranked surgeons was 0.26% versus 0.05% (P=0.01) and operation times averaged 137 minutes compared with 98 minutes (P<0.001), the researchers wrote in the Oct. 10 issue of The New England Journal of Medicine.
Birkmeyer told MedPage Today he was surprised by the strength of the association between the peer-assessed skill rating and surgical outcomes, and he suggested that a strategy similar to the one used in the study could serve as a tool for evaluating competency among bariatric surgeons and other surgeons performing operations that require a high degree of technical skill.
"Many people, including me, assumed skill would have some effect on surgical outcomes, but nobody predicted that the effect would be as strong as it was, or that it would be so consistent across virtually every type of poor outcome," he said.
The fact that the surgeons who participated in the study were also participants in the ongoing Michigan Bariatric Surgery Collaborative (MBSC), a clinical outcomes registry and surgical improvement program, made the research possible, Birkmeyer added.
"The main reason we were able to pull this off is that we had been working with this pool of surgeons to measure and improve outcomes for so long that they trusted our motives," he said.
For the study, 20 bariatric surgeons in Michigan were asked to submit a single representative videotape of themselves performing a laparoscopic gastric bypass, which was rated for technical skill by at least 10 other surgeons who did not know the operating surgeon's identity.
The researchers asked the bariatric surgeons to submit a "representative" laparoscopic bypass surgery video that included only the operative field and was stripped of any obvious patient or surgeon identifiers. The 20 videos were then edited to remove all but the critical components of the procedure, which included the creation of the gastric pouch, the gastrojejunostomy, and the jejunojejunostomy.
Edited videos ranged from 25 to 40 minutes each and were distributed electronically for rating by peer surgeons. One video was distributed approximately every 2 weeks between July 2011 and June 2012.
All 75 surgeons participating in the MBSC were invited to rate each video and 33 of them from 24 hospitals ended up serving as raters.
In addition to providing a summary judgment of overall skill, raters assessed each video on a scale of 1 to 5 for specific technical skills including gentleness, tissue exposure, instrument handling, time and motion and flow of operation, with 1 indicating lowest skill level and 5 indicating the highest. A score of 3 reflected the skill of an average practicing bariatric surgeon.
The main outcome was the occurrence of any postoperative complication. Surgical complications included surgical-site infection, wound infection or dehiscence, abdominal abscess requiring drainage or reoperation, anastomotic stricture, bowel obstruction requiring reoperation, and bleeding requiring blood transfusion, reoperation or a splenectomy. Medical complications included pneumonia, respiratory failure, renal failure, venous thromboembolism, myocardial infarction, cardiac arrest, and death.
Summary ratings of the technical skill of the 20 surgeons varied from a low of 2.6 to a high of 4.8. The five surgeons in the bottom quartile had a mean rating of 2.9, while the top five surgeons had a mean rating of 4.4.
Among the main findings:
* Surgical skill was more closely related to surgical volume than years in bariatric surgery practice. Compared with the top-rated surgeons, those in the bottom quartile had lower mean annual volumes of laparoscopic gastric bypass procedures (53 versus 157, P=0.005) and of any bariatric procedure (106 versus 241, P=0.02).
Lower skill ratings were associated with higher rates of both surgical and medical complications.
* Compared with patients operated on by the top-rated surgeons, patients of the bottom-rated surgeons had higher rates of surgical-site infections (4.60% versus 1.04%, P=0.001) and pulmonary complications (3.91% versus 0.71%, P=0.004).
* As compared with the top quartile of skill, the bottom quartile was associated with significantly higher rates of reoperation (3.4% versus 1.6%; P=0.01), readmission within 30 days (6.3% versus 2.7%, P<0.001), and return visits to the emergency department (10.2% versus 3.8%, P=0.004).
* Surgical volume was an indicator of technical skill, but it was by no means a perfect proxy, Birkmeyer said.
"Among highly skilled surgeons there were some that did relatively low volumes, and among surgeons more poorly rated there were a few that did especially high volumes," he said.
"Clearly practice matters a lot, but so does inherent ability and (task-specific) skill," said Birkmeyer, himself a bariatric surgeon, adding that the findings may have implications for how the surgical profession educates, trains, and self-polices itself.
The findings may also inform the current debate about how to best evaluate prospective and practicing surgeons, he said.
"Having surgeons submit one video representative of the thing they do the most and having it rated anonymously by a jury of their peers is not only very simple to do, it provides a great deal of information about who is good and who is not," he said.
(This page requires a free membership, but I'll copy it anyway)
Peer Rating Strategy Predicts Bariatric Outcome
Published: Oct 9, 2013 | Updated: Oct 10, 2013
By Salynn Boyles, Contributing Writer, MedPage Today
Reviewed by F. Perry Wilson, MD, MSCE; Instructor of Medicine, Perelman School of Medicine at the University of Pennsylvania and Dorothy Caputo, MA, BSN, RN, Nurse Planner
Action Points
* Note that this cohort study suggests that bariatric surgeons with higher peer-assessed operative skill have lower surgical complication rates.
*Be aware that higher surgical volume, not years in practice, was associated with higher skill ratings.
* Patients of bariatric surgeons deemed to have poor skills by peers who watched them perform a procedure were almost three times more likely to have complications and five times more likely to die than those treated by top-rated surgeons, a study found.
The complication rate among patients treated by surgeons in the bottom quartile was 14.5%, compared with 5.2% among those treated by surgeons in the highest quartile (P<0.001) John D. Birkmeyer, MD, of the University of Michigan Center for Healthcare Outcomes and Policy in Ann Arbor, and colleagues reported.
The mortality rate for patients of the lowest-ranked surgeons was 0.26% versus 0.05% (P=0.01) and operation times averaged 137 minutes compared with 98 minutes (P<0.001), the researchers wrote in the Oct. 10 issue of The New England Journal of Medicine.
Birkmeyer told MedPage Today he was surprised by the strength of the association between the peer-assessed skill rating and surgical outcomes, and he suggested that a strategy similar to the one used in the study could serve as a tool for evaluating competency among bariatric surgeons and other surgeons performing operations that require a high degree of technical skill.
"Many people, including me, assumed skill would have some effect on surgical outcomes, but nobody predicted that the effect would be as strong as it was, or that it would be so consistent across virtually every type of poor outcome," he said.
The fact that the surgeons who participated in the study were also participants in the ongoing Michigan Bariatric Surgery Collaborative (MBSC), a clinical outcomes registry and surgical improvement program, made the research possible, Birkmeyer added.
"The main reason we were able to pull this off is that we had been working with this pool of surgeons to measure and improve outcomes for so long that they trusted our motives," he said.
For the study, 20 bariatric surgeons in Michigan were asked to submit a single representative videotape of themselves performing a laparoscopic gastric bypass, which was rated for technical skill by at least 10 other surgeons who did not know the operating surgeon's identity.
The researchers asked the bariatric surgeons to submit a "representative" laparoscopic bypass surgery video that included only the operative field and was stripped of any obvious patient or surgeon identifiers. The 20 videos were then edited to remove all but the critical components of the procedure, which included the creation of the gastric pouch, the gastrojejunostomy, and the jejunojejunostomy.
Edited videos ranged from 25 to 40 minutes each and were distributed electronically for rating by peer surgeons. One video was distributed approximately every 2 weeks between July 2011 and June 2012.
All 75 surgeons participating in the MBSC were invited to rate each video and 33 of them from 24 hospitals ended up serving as raters.
In addition to providing a summary judgment of overall skill, raters assessed each video on a scale of 1 to 5 for specific technical skills including gentleness, tissue exposure, instrument handling, time and motion and flow of operation, with 1 indicating lowest skill level and 5 indicating the highest. A score of 3 reflected the skill of an average practicing bariatric surgeon.
The main outcome was the occurrence of any postoperative complication. Surgical complications included surgical-site infection, wound infection or dehiscence, abdominal abscess requiring drainage or reoperation, anastomotic stricture, bowel obstruction requiring reoperation, and bleeding requiring blood transfusion, reoperation or a splenectomy. Medical complications included pneumonia, respiratory failure, renal failure, venous thromboembolism, myocardial infarction, cardiac arrest, and death.
Summary ratings of the technical skill of the 20 surgeons varied from a low of 2.6 to a high of 4.8. The five surgeons in the bottom quartile had a mean rating of 2.9, while the top five surgeons had a mean rating of 4.4.
Among the main findings:
* Surgical skill was more closely related to surgical volume than years in bariatric surgery practice. Compared with the top-rated surgeons, those in the bottom quartile had lower mean annual volumes of laparoscopic gastric bypass procedures (53 versus 157, P=0.005) and of any bariatric procedure (106 versus 241, P=0.02).
Lower skill ratings were associated with higher rates of both surgical and medical complications.
* Compared with patients operated on by the top-rated surgeons, patients of the bottom-rated surgeons had higher rates of surgical-site infections (4.60% versus 1.04%, P=0.001) and pulmonary complications (3.91% versus 0.71%, P=0.004).
* As compared with the top quartile of skill, the bottom quartile was associated with significantly higher rates of reoperation (3.4% versus 1.6%; P=0.01), readmission within 30 days (6.3% versus 2.7%, P<0.001), and return visits to the emergency department (10.2% versus 3.8%, P=0.004).
* Surgical volume was an indicator of technical skill, but it was by no means a perfect proxy, Birkmeyer said.
"Among highly skilled surgeons there were some that did relatively low volumes, and among surgeons more poorly rated there were a few that did especially high volumes," he said.
"Clearly practice matters a lot, but so does inherent ability and (task-specific) skill," said Birkmeyer, himself a bariatric surgeon, adding that the findings may have implications for how the surgical profession educates, trains, and self-polices itself.
The findings may also inform the current debate about how to best evaluate prospective and practicing surgeons, he said.
"Having surgeons submit one video representative of the thing they do the most and having it rated anonymously by a jury of their peers is not only very simple to do, it provides a great deal of information about who is good and who is not," he said.
