Career outcomes of graduates who initially failed Step 1

I know that I have been gone for a while. Sorry about that...

I saw a really interesting article in Advances in Health Sciences Education (1) this month.  The authors decided to study the career outcomes of graduates of six Midwestern medical schools who had initially failed USMLE Step 1. In this retrospective study, the authors sought to determine the academic and professional career outcomes of medical school graduates who failed Step 1 on the first attempt. They took a cohort of students who graduated from the Ohio State University College of Medicine, Michigan State University College of Human Medicine , Washington University School of Medicine, SouthernIllinois University School of Medicine, University ofIowa Roy J. and Lucille A. Carver College of Medicine, and the University of Michigan School of Medicine. In this cohort of 2,003 graduates from 1997-2002 were 50 (2.5%) students who initially failed Step 1 and these students were compared to the 1,953 students who passed Step 1 on the first attempt.

There were several interesting findings in this study. The authors used information from the MSQ (Medical Student Questionnaire), the GQ (Graduate Questionnaire), the AMA Physician Masterfile, ABMS Board certification, and the AAMC Faculty Roster System. Data was gathered from all six schools and merged into a single database. Some of this data has issues, for example the MSQ and the GQ both rely on student self-report. The AMA Masterfile may mis-categorize some doctors and the cohort only includes students who made it to graduation and for whom they had complete data available (about 43% of the total graduates). But with that being said, this is a pretty good study with a large cohort of graduates.

So, what did they find?  As you would guess, passing USMLE Step 1 on the first attempt has major repercussions for medical students. Most students (94%) pass the test, but not all. Students who fail Step 1 are less likely to pass Step 2 and less likely to ultimately graduate from medical school. (2) But this group of students had all graduated so are they still impacted? There is not a lot of evidence that medical school test performance is correlated to residency clinical performance, but Program Directors still put too much emphasis on Step 1. In fact, a national survey of Program Directors (3) found that 84% would seldom or never interview a student who had failed Step 1 even if they eventually passed Step 1.

In this study, those who failed were more likely to be women, minorities, and older. In fact, there were significant differences between these groups. When compared to men who graduated, women were 3.2 times more likely to have failed Step 1 on the first attempt (p < 0.001). African Americans were 13.4 times more likely to fail when compared to whites (p < 0.0001), and Latinos were 7.4 times more likely (p < 0.0001) to fail when compared to whites. They are more likely to end up in primary care residencies, to be older, and to come from families of lower income.

This data obviously has huge implications. Every medical school has a few failures on Step 1 and should be concerned about the implications. Are we willing to tolerate this difference in the relative risk of failing a nationally standardized high stakes examination? There may be pre-matriculate variables that explain some of the differences, but there also may be ways to identify and intervene in high-risk students’ academic career in ways that can decrease their risk of failure.

Take a look at the programs that are in place at the University of Texas Medical Branch in Galveston and Southern Illinois University. They have been successful in helping students that were identified as at-risk students. Unfortunately, but many schools would rather try to decrease the number of at-risk students that they admit.

The method that is often used is to try to admit students with higher MCAT scores and higher undergraduate GPAs. The problem with this strategy is that in doing this the school will also increase the number of rich, white, male students who come from urban backgrounds. This leads to a student body that is less diverse. That is something that our schools should not tolerate.

References

(1) McDougle L, et al.  Academic and professional career outcomes of medical school graduates who failed USMLE Step 1 on the first attempt. Adv in Health Sci Edu.  7 April 2012 (Online First).

(2) Biskobing DM, et al. Study of selected outcomes of medical students who fail USMLE Step 1. Medical Education Online  2006;11(11):1–7.

(3) National Resident Matching Program, Data Release and Research Committee. Results of the 2008 NRMP Program Director Survey. Washington, DC: National Resident Matching Program. 2008

Impact of medical school tuition

In my blog, How much should medical school cost?, I wrote about an innovative proposal by Peter Bach and Robert Kocher to make medical school free. Dr. Bach is the director of the Center for Health Policy and Outcomes at Sloan-Kettering Cancer Center. Dr. Kocher is a special assistant to President Obama on health care and economic policy. Their proposal has gotten a lot of press. It seems that many people have an opinion about the cost of medical education. Dr Joshua Freeman wrote in his blog, Medicine and Social Justice Would free medical schools increase primary care?.  Dr Kenny Lin, in his blog the Common Sense Family Doctor wrote Strengthening the primary care pipeline.  Dr Pauline Chen wrote in the New Times Health blog about The Hidden Cost of Medical Student Debt. 

I was thinking about another aspect of the cost of medical school.  It is something that I have not heard people talking about. Most of the negative aspects that have been talked about are related to the negatives to the student and the impact on their future career choices. There is also some (important) talk about the societal negative of less students choosing primary care and family medicine.

But what about students who never graduate from medical school?  Medical schools try very hard to keep students in school because of the big investment in that individual student. If you get into medical school in the US, your chances of graduating are extremely high. Something like 96 percent.¹  But what happens if you don't graduate? At most schools, the student would be on the hook for the cost of their education.  Wow!  Saddling a medical school graduate with $150,000 in debt is bad, but we say--they are going to be doctors, they can afford it. But if a student leaves school or is dismissed from medical school for academic reasons, they are still going to have that debt of $150,000.

It is a big deal. When the faculty are talking about the academic performance of a medical student, they should not have to worry about the financial ramifications of their decision. But, honestly, how can they not consider the implications. They are essentially condemning someone to financial ruin by saddling them with a debt that rivals that of a new home. And state schools are relatively cheap. If they are at a private medical school or one of the osteopathic schools, their debt may be as much as $300,000. 

The faculty on a dismissal committee are given an unenviable task. They are acutely aware of their responsibility to the public and society. They do not want to graduate a student who is not qualified to be a physician. By the time a student gets to this point they have been on the dean's office radar for a long time. These students have generally failed several courses or national standardized examinations, they have been through a lot of remediation, and there has been an enormous effort to salvage their medical career. The faculty have invested a lot of time and effort into the student. The faculty really care about the students. The school has invested a lot of money in them. A position in the medical school class can never be regained, so if a student is dismissed that spot is gone forever. The faculty feel that they have failed when a student is dismissed. The financial ramifications just make it worse.

So, I would like medical schools to consider a no-fault system.  If a student is dismissed from medical school, their tuition and fees are refunded to them. Let's take the financial part completely out of the discussion.  This will be better for the students and for the faculty. An potential effect of this policy, would be to encourage schools to take a chance on more at-risk students.  Some of these students might be from disadvantaged backgrounds, some might be underrepresented minorities, and some might be from rural areas. These students are less likely to apply to medical school and even when they apply, less likely to graduate.¹ Disadvantaged students may choose to not even try medical school because of the financial consequences of failure. A no-fault system would eliminate those consequences.

What do you think?

References

1) Garrison G, Mikesell C, Matthew D. Medical School Graduation and Attrition Rates [https://www.aamc.org/download/102346/data/aibvol7no2.pdf]. Analysis in Brief AAMC.  2007. Vol 7 Number 2.

Research in medical education

I had a crazy thought the other day,  why don't we do more research in medical schools. I know you're saying that there is a ton of research going on in the modern medical school.  But I bet if you are reading this you don't care that much about most of the research that is going on. Sure, there is plenty of basic science research. For instance, my good friend and co-conspirator/ co- director of our first medical school module is the famous Joe Fontes. He is in the Department of Biochemistry and he has two RO1s from the NIH.  Pretty impressive!  We also do some clinical research. One of my partners has an RO1 from the NIH to study colon cancer screening in minority populations.

But I am talking about research on and about the educational process. How much of what we do on a daily basis is evidence based?  I don't mean the knowledge content that is being delivered but the way that the information is delivered.  Is a lecture better than a small group?  Is an experienced lecturer better than one who is young and inexperienced?  Is a preceptor visit a better way to learn about professional behavior than lecturers. Are preceptorship visits better for teaching clinical skills or faculty physicians or trained patients?

These are all important questions. Why don't we know the answers to them?  

You might say that we know the answer to some of those questions. I would say that we probably know part of the answer to some but we don't know the whole answer to any of them. But does that matter? I think it does. We talk all the time about how we think evidence-based medicine is important but do we apply the same standard to the education of our students? I don't think so. I am not even sure that we could all agree on the definition of "better".

But don't you think we should have an evidence-based standard for educational practice just like we have for clinical practice?  This is what I propose. We need a medical school that is totally based on the best available educational research. Wait a minute, that won't work, will it?  We already know that there is not enough evidence for most of what we do. So, if we can't do it now maybe we could in the future. But how can we get there? Here is the radical part. We need to experiment on our students. (a big gasp was heard!)

The funny part is that I think we already do this. But right now we are not getting their permission.  Every time that we use an unproven educational method in our teaching, we are experimenting on them. We just don't get their consent.  I propose that we start asking for consent and randomize them to the standard educational method or to a well thought out intervention that is based on sound educational theory. This will require us to have a good grounding in educational theory which most physicians don't. And it will require an acknowlegement by us that our methods may not be right. We might need to change.

If I was a med student, I would be excited by this. Our students in Salina (see my blog about Salina) were told at the beginning that they would be the innovators, not guinea pigs. It was and is exciting to them. They were excited to be the first to prove that a medical school could be run in a small town in rural Kansas.  I think that the same would be true on other campuses with other methods. Let's be innovators.  

What do you think?

Empathy

A recent study Rosenthal, et al (1) in the journal Academic Medicine, studied the Jefferson Empathy Scale in medical students at the Robert Wood Johnson Medical School . This study was born out of a committee that included medical students, residents, and faculty. Their purpose was to design a curricular intervention that could be done in the third year of medical school that would help to preserve empathy throughout the year. 

The purpose of the study was "to evaluate JSPE-MS scores of two consecutive medical school classes in order to assess the impact of an empathy-preserving curricular innovation". At RWJ Med school they believed that there is a decrease in empathy across the third year of medical school. Anecdotally, I think that most medical educators know this to be true.

Wiggleton and colleagues (2) found moral distress, burnout, and depression in third year students. In this study, the authors described 50 potentially distressing situations which medical students might encounter in clinical rotations. Situations included: a patient had very advanced disease because they faced barriers to accessing care; a member of the team was disrespectful to someone below them in team ranking; optimal care was not provided as a result of alcoholism, drug use or homelessness; and our team provided care that only prolonged a patient's suffering. Over half of these situations had been experienced at least once. 35% of the situations caused mild to moderate distress.  

Diseker and Michielutte (3) found that empathy decreased before and after clinical experiences in the third year. The author's administered the Hogan empathy scale to all medical students. They found the empathy scale to be negatively correlated to MCAT scores. (see my previous blog about the MCAT). And they found that there was a significant decline in the empathy score from the beginning of medical school to the end.

Hojat and colleagues (4) found empathy significantly declined across the third year of medical school. They found that empathy did not really change that much in the first and second year, but the decline during the third year persisted until graduation. Interestingly, 27% of the students did not have any decline in empathy.

So, in this study the authors designed a curricular intervention that was given in the clinical year. Their intervention was six interactive sessions that were given during each of the required third-year clerkships. The sessions were one hour long and included time for debriefing on intense experiences, reflective essays/blogs,  and discussions of role models, patient care, morally distressing events, and the students' reactions.

This intervention seems fairly simple and similar to a longitudinal experience that our school has in the third year of medical school. The authors were able to document a lack of change in empathy across the third year. There was no statistically significant change in pretest/posttest empathy scores (pre = 115.4, post = 113.9, p =.135). One interesting fact was that student who entered family medicine, internal medicine and pediatrics had significantly higher empathy scores than those entering other specialties such as surgery, urology, otolaryngology, and anesthesiology.

Hopefully, there will be more interventions designed to help our students during the third year. It is hard. They are vulnerable and are often treated like crap. Dr. Steven Kanter (5) in his opening editorial in the March issue of Academic Medicine, reminds us that we need to think with our head as well as our heart to provide the best patient care. If we damage our students they will no longer have the ability to think with their heart, to care for their patients.  OK, I will get off the soap box for a while.

REFERENCES

(1) Rosenthal S, et al Humanism at Heart: Preserving Empathy in Third-Year Medical Students. Acad Med 2011;86(3):350-358.

(2) Wiggleton C, et al. Medical students' experiences of moral distress. Acad Med 2010; 85:111-117.

(3) Diseker RA, Michielutte R. An analysis of empathy in medical students before and following clinical experiences. J Med Educ. 1981;56:1004–1010.

(4) Hojat M, et al. The devil is in the third year: A longitudinal study of erosion of empathy in medical school. Acad Med. 2009;84:1182–1191.

(5) Kanter S. Think With Your Head and With Your Heart. Acad Med  2011;86(3):273.

Professional dress: does it matter any more?

Sorry, I have been absent for a couple of weeks.

We started a new class of students off on their medical journey last month. The first year students arrive on campus bright-eyed and bushy tailed as it were. At our school most are fresh out of college. They have spent the last four or five years as Biology or Chemistry majors. They went to class or maybe they didn't. Most college courses don't have attendance requirements. They are generally allowed to dress however they want, this time of year shorts, flip flops and ball caps are the norm.

In 2005, there was a big flap (or should I say flip/flap) when the national championship women's Lacrosse team from Northwestern University was invited to the White House to meet President George Bush.  The scandal began when several people noticed that a picture taken of the ladies showed four of the nine players in the front row wearing flip flops.  Now these ladies were dressed up in skirts and nice clothes. After all they were meeting the President, but dressing up apparently did not include changing shoes.

 

So here we are with another class of new medical students. I am the co-director of the first module so I am sitting there in class most every day. The students are polite, they almost always address me as Dr Delzell, and so far I have not seen any rude behavior in class. But at least half the class is wearing flip flops. T-shirts are the norm. Many extolling their undergraduate school or their fraternity/sorority. Many of the guys wear ball caps. 

Now don't get me wrong, I like to dress casual. As soon as I get home from work I put on shorts and a t-shirt. I love to wear flip flops. I would love to have a job at a medical school that is located on the beach so I could wear casual Hawaiian-style shirts and flip flops every day.  But I don't.  And neither do our students. 

I know that there are some schools that require professional dress whenever the students are on campus. Dr. David Steele, Senior Associate Dean for Medical Education at the Paul L Foster Texas Tech School of Medicine in El Paso has told me that at their new medical school the faculty decided to require students to dress professionally every day. Even during the basic science lectures. And last year, we were invited to be visiting professors at the Yerevan State Medical University in Yerevan Armenia. We visited the campus and toured one of their large lecture halls that would hold about 600 first year medical students. It had hard wooden benches and no air conditioning and the students were required to dress up (suit and tie for the men) each day for lecture.

I don't know if it makes a difference. We talk all the time about how Millennials-Generation Y is different from past generations. I am sure that in the sixties when the hippies started their first day of medical school, they were wearing bell bottoms and tie-dye shirts. I am sure the professors were concerned about the lack of professionalism that those students displayed.  This is to some extent a generational issue that is seen every year. But where is the line?  When is it a generational issue-where the younger generation have a different set of internal rules and values that guide them in different ways than a previous generation? And when is it a maturation issue-where you need to learn behavior from those that are your teachers and mentors?

New Salina med school campus unique in U.S.

The following is an excerpt (posted with permission) of an article by Dave Ranney from the Kansas Health Institute News Service that was originally posted on July 5, 2011 on the KHI website. Dave Ranney graciously gave permission for this reposting.

SALINA — Next month (actually tomorrow), the University of Kansas School of Medicine will open a four-year, fully accredited school – officials prefer to call it a campus – next door to the Salina Regional Health Center.

“This will be the smallest medical school campus in the country and Salina will be the smallest city in the country - outside of a few major suburbs - to have a medical school campus,” said Dr. Heidi Chumley, senior associate dean for medical education at KU Medical Center.

The first class will have eight students, seven of whom are from Kansas.  KU Medical Center officials said they plan to add eight students a year at the campus in each of the next four years.

More primary-care docs

“The goal is to develop more primary care doctors for rural Kansas,” Chumley said. “We’re shooting for 75 percent (of the new school’s eventual graduates) choosing primary care, and 75 percent rural.” Much of the new school’s curriculum will be tied to the classroom offerings at the KU Medical Center’s campuses in Kansas City and Wichita. “We completely redid our curriculum about six years ago. It’s very computerized now,” said KU Medical Center Executive Vice Chancellor Dr. Barbara Atkinson. “All the lectures are podcasts. They’re all going to be teleconferenced (in Salina), though some will be generated on-site.”

The cost of remodeling the three-story building has been picked up by Salina Regional Health Center. And the hospital’s foundation and several private donors are covering many of the operational costs and scholarship offers. “So far, we’ve not asked the state for any money for this because we understand the financial situation the state is in,” Atkinson said. “But we have asked donors to support it and they’ve been very, very generous. Incredibly generous.” Salina-area benefactors, she said, hope to raise $2.5 million over the next four years. They’ve already raised $1.5 million with $1 million coming from the Salina hospital.

Salina Regional Health Center has hosted a residency program for KU Medical School graduates for about 30 years.  Most of its residents went on to start or join rural practices.

Model for other states

“What the University of Kansas is doing, I think, will be a template for having a positive impact on the number of medical practitioners in rural communities,” said Brock Slabach, senior vice president at the National Rural Health Association. “Other universities will be watching because, really, for a major medical school to commit itself to meeting rural-community needs like this is truly novel. It shouldn’t be, but it is.”

Slabach said he’s long been baffled by the fact that medical school officials in many rural states’ fail to see the connection between their states' shortages of health care providers and their students leaving for big cities in other states. “I wouldn’t include Kansas in that group,” he said. Last year, a national survey of how well medical schools were fulfilling their “social mission to train doctors…” ranked KU School of Medicine fifth in the nation. The school was ranked eighth for its percentage of graduates (44 percent) practicing in underserved areas.

Earlier this year, a Kansas Department of Health and Environment survey found that 51 of the state’s 105 counties had less than one physician per 2,695 residents and were considered medically underserved.

Looming retirements

“The shortage of health care professionals has been a critical issue for a long time,” said Dr. William Cathcart-Rake, director at the KU-Salina campus. “It’s not new, but what is new is that now we have a number of physicians who are nearing retirement age, and a good number of them are practicing in the rural communities.” Generating enough new doctors to replace those who are retiring – especially in rural areas - will be difficult, he said. “We have to do something,” Cathcart-Rake said. “We can’t keep doing what we’ve always done. It’s not enough.”

The Salina campus, he said, will be geared toward allowing students from small towns to complete their studies and residencies in a small-town environment. “There is evidence that shows that if someone is trained in a rural area they are more likely to stay in a rural area,” he said. “So the idea is that from day one we’ll be training our students in in non-metropolitan settings and exposing them to all the good and the bad that comes with life in rural Kansas. After that, we’ll hope for the best.”

Cathcart-Rake, who grew up in small town in Orange County, Calif. (“…back when there was still a small town in Orange County”), has been practicing medicine in Salina for 32 years. “The perception is that if you go to a small town you’ll work yourself to death, you’ll never get a vacation and you won’t get to spend time with your family,” he said. “The way to get around that is to be with a group of physicians so you can cover for each other so you don’t have to do everything for yourself.

Going where no medical school has gone before....

On the front page of the Saturday (July 23, 2011) New York Times, an article by A.G. Sulzberger proclaimed Small-Town Doctors Made in a Small Kansas Town.  This week, the University of Kansas School of Medicine is opening a new campus in Salina, Kansas.  Salina will be the smallest town in the US to host a four-year medical campus. The stated purpose of the new campus is produce primary care doctors that will practice in rural Kansas. Dr. Heidi Chumley, the KU Senior Associate Dean for Medical Education and Associate Vice Chancellor for Educational Resources and Interprofessional Education, is leading KU's development of this campus with strong support from Chancellor Bernadette Gray-Little and EVC / Executive Dean Barbara Atkinson. The fact that this article ran on the front page of the NY Times is a testament to the uniqueness of this new campus. The LCME reviewers of the Salina program thought that it was "an interesting model" that "could be a stimulus for other schools".

As you might expect, the program has generated a lot of interest:

Dave Ranney from the Kansas Health Institute had a nice article on the KHI web site on July 5, 2011, New Salina med school campus unique in U.S. I will be reposting an excerpted version of this article in the next few days.

Elana Gordon of KCUR, the University of Missouri-KC NPR affiliate, did a nice article about the new campus, Tiny School to Create Tiny Town Docs. This ten-minute interview with Dr. Chumley ran on the air and a podcast is posted at this link.

Roger Cornish, from KWCH 12 Eyewitness News (a Wichita CBS affiliate), has a great interview with Kayla Johnson and several of the other new first year medical students at the Salina campus. You can watch the video of the interview titled, Classes begin at K-U Med / Salina.

Lily Wu from KAKE (another Wichita TV station) did an interview of the students and staff at the new campus, Medical Students Interested in Serving Small Towns.

I am sure that there will be more press on this over the next few days and weeks. Congratulations to Dr. Chumley and the University of Kansas for this historic endeavor.