education

How good was the test? A look at the Mississippi medical licensing exam

/ davidfsmb / Leave a comment

My last several posts drew inspiration from materials donated by the Mississippi Board of Medical Licensure to the FSMB Historical Collection. I used a representative sample set drawn from license applications submitted to that board as a means of exploring the demographics of the early physician licensees in Mississippi. My last blog post addressed test construction and performance data on the Mississippi exam spanning the period from 1924-1958.

Today’s post focuses on the actual questions presented on the Mississippi exam. As my colleague Cyndi Streun, D. Eng., and I perused the two-volumes containing all the Mississippi board’s test questions spanning 1924-1958 any number of questions came to mind. We wondered if these were quality test items? Did they require candidates to apply relevant medical knowledge? Or were they exercises in recall asking for factoids and medical trivia? In essence, we wondered “How good was the exam?”

To answer these questions, we sought formal review from a group of subject matter experts (SMEs): licensed physicians. The four physicians who participated with us all held ABMS certification in either family medicine or internal medicine; and they all had experience in assessment through their service as committee members working on the United States Medical Licensing Examination® (USMLE®).

The sheer volume of questions available for review presented both an opportunity and a quandary. There were more than 3,900 test questions available to us with three hundred or more items in each of the subject areas, e.g., anatomy, physiology, surgery, etc. Even reviewing all the test items from a single year meant asking an SME to read and critique roughly 80 or more questions. In order to make this task more manageable for these SMEs volunteering their time, we settled upon a small subset of questions from a single subject area.

The SMEs reviewed 56 physical diagnosis questions drawn from multiple administrations of the Mississippi board’s medical licensing exam panning over thirty years. We pulled items from this subject area of the test administrations from 1925, 1930, 1935, 1940, 1945, 1950 and 1955. Each SME reviewed these questions independently. For every test item they applied a Yes/No response to a series of prompts.

Is the question clinically relevant?

Is the question appropriate for inclusion on a medical licensure examination?

Does the question involve clinical reasoning?

If the SME answered affirmatively for clinical reasoning, they were then asked to assign a specific level from Bloom’s taxonomy (i.e., remembering, understanding, analyzing, applying, evaluating, creating) to their rating of the question.

This methodology allowed us to provide a consensus score for the SMEs’ collective rating of each test item.  Perfect agreement with all raters answering yes would be a 1.0 score for that test item. Three of four raters saying yes equaled 0.75, etc. The same approach was used to attach a score to appropriateness and whether clinical reasoning was required. As an example, perfect agreement among the raters for clinical relevance would be a score of 8 in 1925.

There was no perfect consensus among the four raters for any of the three categories in any of the years in this study. In general, the SMEs retrospective review deemed the Physical Diagnosis questions clinically relevant (overall .85) and appropriate for a medical licensing exam (overall .78).  See Table 1.

Table 1: SME evaluation of clinical relevance, appropriateness and clinical reasoning on 56-test item set drawn from Mississippi’s medical licensing exam

 # itemsRelevance pointsAppropriateness pointsClinical reasoning points
192585.755.53
193076.2564.75
1935876.754.25
1940875.754
194598.57.755
195087.56.255.25
19558663.5
     
total56 questions48 pts44 pts29.75 pts
  0.857 avg0.785 avg0.531avg

The SMEs judged clinical reasoning as the weakest of the three domains considered. Indeed, this area proved challenging for the SMEs themselves as it is clear that some did not view the lowest levels on Bloom’s taxonomy (remember, understand) as falling within their own definition of clinical reasoning in a medical context.

We quantified the application of Bloom’s taxonomy by identifying instances in which two SMEs marked the same question as requiring a higher clinical reasoning (analyze, apply, evaluate). Only 12 of the 56 test items met this threshold.

To the extent clinical reasoning was deemed by the SMEs as being required of the examinee to answer the question, it was generally seen as occurring at the lower levels on Bloom’s taxonomy (remember, understand). The SMEs assigned the lowest levels (remember, understand) on Bloom to 55% of the total 56 test items.

So what might we surmise for all of this? Several things but first let’s be clear on the limitations of what I’ve shared. This project involved a subset of test items from a single subject area (Physical Diagnosis). More ambitious work might tackle all 300+ test items from a single subject area…or all the questions posed from a single year’s test administration…or from multiple administrations of the Mississippi exam. And while there is nothing to suggest that Mississippi was unique in the construction and content of its licensing examination, this remains a snapshot from a single state.

Having said all this, several things are worth stressing. It seems fair to say that our SMEs had generally positive views for the appropriateness of the questions being asked and their clinical relevance. In essence, what was being asked seemed reasonable even though the questions did not consistently and routinely require candidates to apply their knowledge.

It is also fair to say that these test items (and the examinations in general) were very much products of their time. On multiple occasions the SMEs called out “antiquated” or “dated” language or that today one would apply a “more specific diagnosis.”. The SMEs flagged instances in which a question might be far less relevant today than at the time it was used on this exam. For example, a question diagnosing typhus or rheumatic fever today (compared to the 1930s) might feel like “esoterica.” Additionally, imaging and lab tests would be the standard approach today for conditions previously diagnosed through physical findings several generation ago.

Indeed, one of the major challenges that arose early in working with the SMEs was agreeing upon the lens through which their evaluation should be made—specifically, was the SME rating these questions for appropriateness, clinical relevance, etc. through the lens of medicine as taught and practiced today? Or through the lens of what was understood and practiced at the time of the test? Ultimately, we directed the SMEs to evaluate the test questions through the lens of medicine today. The rationale being that we did not expect our SMEs to be historians of medicine evaluating each item through the lens of medical knowledge known at specific points in time spanning mid-20th century America.  

A final element of this study involved using an AI-model to generate answers to each of the 56 physical diagnoses questions drawn from the Mississippi exam. We then asked our four SMEs to rate the AI-generated answers. Our SMEs evaluated the AI-developed answers based upon the following criteria:

Did the answer show a clear understanding of the question?

Was the answer clear, concise and without irrelevant information?

Did the answer use appropriate language and terminology?

Did the answer provide factually correct information?

Did the answer show evidence of clinical reasoning?

Our SME human raters scored AI generated answers to the dataset questions highest in providing factually correct information, using appropriate language and avoiding irrelevant information. Our SMEs showed markedly different views of the AI answers in their demonstrating clear understanding of the question. Finally, and perhaps not surprisingly, our SMEs scored the AI-generated answers lowest for showing evidence of clinical reasoning. (see below)

Final reflections:

Mississippi’s approach to assessment was conventional and consistent with long-standing state board methods (extended response test items). It was also vulnerable to the deficiencies inherent to that format, e.g., reliability. The board members were not measurement scientists and perhaps not even expert in the subject area(s) they were asked to write test items.

The questions posed on the Mississippi exam were generally relevant and appropriate to a licensing examination. They may not have required the physician candidates to consistently apply a high degree of clinical reasoning but, based upon my familiarity with medical licensing exam questions in other states, the Mississippi exam seems no better or no worse in that regard than other states.

Perhaps most important was the standard being applied as evidenced by the pass rate on the Mississippi exam. It seems clear that regardless of what was being asked on the exam, the depth and breadth of knowledge deemed acceptable as evidence of proficiency was an easily achievable hurdle for nearly all the Mississippi candidates in most years. Remember, Mississippi passed every candidate tested on their exam in 23 of the 35 years in this study. Only 1.3% of all their candidates were failed by the Mississippi board.

Finally, we’ve all heard an iteration of the quote describing medicine as a blend of science and art. If so, it seems that testing during the era of state medical licensing exams probably belonged as much to the latter as it did to the former.

The opinions expressed are those of the author and do not represent the views of the Federation of State Medical Boards.

How tough was it? A look at the Mississippi medical licensing exam

/ davidfsmb / 1 Comment

I’ve written before about the donated materials contributed by the Mississippi Board of Medical Licensure to the FSMB Historical Collection (FHC). https://armchairhistorian.blog/2025/10/07/preserving-the-history-of-medical-regulation/  Among the donated materials are a two-volume set containing all the questions (organized by year and subject area) presented to candidates on that state’s medical licensing exam between 1924-1958.

This gift piqued my interest as well as that of a colleague. As we considered the source materials donated to the FHC, various ideas and questions came to mind: How challenging were these state-developed medical licensing exams? Were they quality exams requiring candidates to apply relevant medical knowledge? Or were they exercises in recall too often seeking factoids or obscure medical trivia? There was a lot riding on the outcome of these exams—a medical license, a career and professional livelihood. So were these exams constructed, administered and scored in a manner commensurate with the high-stakes nature involved with the licensed practice of medicine?

We undertook something akin to a case study of the Mississippi licensing exam, blending two approaches: (1) AI and human rater analysis of these the Mississippi test questions, and (2) review of performance data on the Mississippi exam. My next blog post will share some of what we learned about the former. Today, I want to focus on this latter, starting with how these exams were constructed.

Structure and format of the Mississippi exam

From its originating legislation in 1882, Mississippi required physicians to pass a written examination conducted by its examining body. (Note: There was an exemption in this original law that allowed established practitioners in the state to forego the exam).

A ten member Mississippi Board of Health wrote and administered their state’s licensing exam. The twelve subject areas covered on the exam during the 1924=1958 period are listed here.

Anatomy*      Physiology*      Hygiene*      Obstetrics* and Gynecology       Surgery*            Pathology*     Histology/Bacteriology          Chemistry* Diseases of eye, ear, nose, throat      Materia Medica* Theory and Practice of Medicine         Physical Diagnosis

*indicates original subject areas called for in the 1882 legislation

Individual members of the Mississippi Board wrote six to ten questions for their assigned subject area. All questions were constructed as “open” or extended response items. There were no multiple-choice questions (MCQs) on the Mississippi exam during this period. This is hardly surprising. MCQs did not feature prominently on any state exams for medical licensure or on the National Board of Medical Examiners Parts exam until the 1950s.

Absent information to the contrary, it is presumed that the board member writing the questions for a subject area also carried the responsibility for scoring those items.

I have found no detailed information on the scoring scales or requirements specific to achieving an overall passing level performance on this exam. The 1882 law called only for candidates to present an examination that “prove[d] satisfactory” to the Board.1 As late as 1947, the law only referred to the examination results providing evidence of “sufficient learning.”2 The various volumes donated by the Mississippi board to the FHC do not present any specific information on the scoring. Thus, we don’t know the answer to a few basic questions: Did the Board require a passing score on each individual subject area in order to achieve an overall passing outcome? Or were subject area scores rolled up into an aggregate score with its own specified minimum for passing? (Note: States commonly specified 75 in their state statute as the minimum pass.)

Exam performance data

So how tough was this Mississippi exam? To answer this question, I leveraged the data collected and published annually in JAMA as “State Board Statistics” (1924-1929) and subsequently retitled as “Medical Licensure Statistics” (1930-1958).

For the 35-year period spanning the years 1924-1958, a grand total of 1,768 individuals were examined by the Mississippi Board—an average of 50 candidates per year. And the results? We are definitely not talking about the Bataan death march of assessment. All but 23 candidates passed the exam—a 1.3% fail rate for this entire period! (See the Appendix 1 below)

From a purely statistical perspective, this exam presented a minor hurdle for the vast majority of candidates. Indeed, in twenty-three of the 35 years of this study no candidates failed the exam in Mississippi. Though this may seem surprising it was actually not uncommon based upon the reported performance outcomes on other states’ medical licensing exams. Looking nationally at the period 1924-1958, twenty-three states reported years in which no candidates failed their medical licensing exam. Thus, Mississippi’s modest 1.3% overall fail rate ranks 25th (median) among all states during this period. (see Appendix 2)

Interestingly, this was definitely not the case previously on the Mississippi exam. Only a decade earlier, the fail rate on Mississippi’s exam fell on the other end of the spectrum. The first 5-year period reported by JAMA showed Mississippi with the highest failure rate on its exam compared to all other states. 

Years# MS candidates% failedNational rank
1908-19121,17255%1st
1914-191828423%11th
1919-19241248%25th

A dramatic swing involving the Mississippi exam took place sometime around the start of World War I. What happened precisely is unclear but we can speculate on reasons for both the sharp drop-off in number of candidates being examined by Mississippi after 1912 as well as the steady decline in their fail rate.

In terms of the volume drop off, prospective licensees may have begun avoiding Mississippi because of the high fail rate on its exam which would have been publicly known through JAMA’s annual publication of state specific exam results. It is also possible that changes to state law or board policy may have directed more incoming physicians from other jurisdictions into licensure by reciprocity or endorsement scenarios that allowed them to bypass the exam. Perhaps the strongest factor in the declining numbers and also the fail rate was the change to state law in 1919 that required licensure candidates to be graduates of Class A school2 under the AMA Council on Medical Education’s classification system.

It also bears mention that Mississippi had one of the lowest physician-to-population ratios in the United States at 1:1640. Board Chair Felix Underwood called it the “most serious and complex public health problem” in the state’s history.3 With the Mississippi Board keenly aware of this shortage, I can’t help but wonder if this worked at least unconsciously on the Board’s vetting of physician candidates and even its scoring of candidates’ exams. See Appendix 3

By 1914 and through the start of World War II, the number of physician candidates examined by the Board stabilized at a modest twenty to thirty annually in most years with a fail rate hovering around 2%.

Mississippi board composition and impact on the exam

The ten-member Mississippi board wrote and administered its medical licensing examination. Board members served six-year staggered terms. Assuming scoring was done by a single board member for each subject area, changes to board composition directly impacted both the exam’s content and its scoring.

Thirty-five (35) individuals served on the Mississippi board during the period 1924 to 1958. The mean length of service was nearly a decade (9.57 years) with a dozen individuals serving 12 or more years on the MS board. Board Secretary Felix Underwood served 34 years, nearly the entire span of the time period under review. See Appendix 1

Paying attention to the Board’s composition is more than just historical minutiae. With an extended-response item format, the individuals making decisions about the correctness and sufficiency of the answers (i.e., the rater) represents the key factor impacting scoring and overall pass rates on the Mississippi exam. The information available to us does allow for analysis concerning how the number of raters scoring each subject area impacted potential scoring (in)consistency.

While two subject areas (Histology/Bacteriology and OB/GYN) had higher potential consistency by virtue of the small number of board members scoring those areas during this period, most subject areas had 3-4 different board members scoring that component of the exam from 1924-1958. Two areas (surgery, EENT) had probably the greatest potential for scoring variability.

Subject area# scorersSubject area# scorers
Histology/Bacteriology1Pathology4
OB/GYN2Physical Diagnosis4
Anatomy3Theory & practice of medicine4
Hygiene3Materia Medica5
Physiology4Surgery7
Chemistry4Diseases of eye, ear, nose, throat7

We might think that a single examiner scoring a subject area over the entire period is the ideal. While such was the case for histology/ bacteriology, the practical reality is that the time-period in question is so long that there were likely multiple changes over time impacting the judgment of a sole rater on an biannual test administration. For instance, rater tendencies and predilections involving specific focus or emphasis in crafting questions within the subject area; changing expectations of what constitutes an acceptable minimum level of performance by a candidate, etc.  In other words, how a rater approached both what questions to ask and how they should be scored ultimately shift over time even when only a single rater was involved in a subject area. As one scholar put it, “assigning a grade to an essay is not a precise science.”4 This is not conjecture. Ample evidence exists in the scholarly and technical literature specific to scoring extended response items demonstrating that rater (in)consistency is highly problematic to exam reliability.

At the same time, it seems unlikely that the rater composition, regardless of (in)stability, had much impact on licensure candidates in Mississippi. Why? Going back to the performance data, few individuals (n=23) failed to demonstrate adequate knowledge to that board. The examination in Mississippi presented an easily surmounted hurdle in the candidate’s journey to licensure.

So was the medical licensing exam in Mississippi little more than pro forma ritual? A performative exercise for gaining admittance to the profession? Perhaps. But in order to answer this we have to look directly at the questions being posed on the Mississippi examination. How relevant were they? Were they quality items allowing the rater to glean insights into the knowledge and competence of these candidates?  For this, we sought subject matter review. That will be subject of my next blog post.

To be continued….

The opinions expressed are those of the author and do not represent the views of the Federation of State Medical Boards.

Endnotes

  1. See Section 17 of An Act to Regulate the Practice of Medicine in the State of Mississippi (Jackson: State Printer, 1882).
  2. Mississippi Laws and Extracts of Laws Dealing with Public Health (Jackson: Mississippi State Board of Health, 1947), 104
  3. Lucie Robertson Bridgforth, “Politics of Public Health Reform,” The Public Historian, 1984, p. 19
  4. Gavin T. L. Brown, “The Reliability of Essay Scores: The Necessity of Rubrics and Moderation,” in Tertiary Assessment and Higher Education Student Outcomes: Policy, Practice and Research. Ed. Luanna H. Meyer, et. al. 2009

Appendix 1

All candidates for Mississippi examination
YearTotal# pass# failFail %MS fail % compared to nat’l# states w/ 0% fail
1924212014.7%21st28
1925282800%24th27
19262018210%5th25
1927212100%35th16
1928242400%32nd19
1929272700%30th21
1930333126.0%10th27
1931353500%25th26
1932272700%25th26
1933242400%26th25
1934292813.4%14th22
1935313013.2%16th19
19362622415.3%8th26
1937182229.0%14th18
1938323113.1%16th21
1939222200%25th26
1940424200%28th23
1941383800%27th24
1942454324.4%14th24
194311411310.9%23rd22
1944606000%24th27
1945505000%25th26
1946454500%28th23
1947595900%22nd29
1948474700%27th24
19494035512.5%7th19
1950646311.5%22nd21
1951777700%28th23
1952838300%27th24
1953848400%25th26
1954808000%27th24
1955757500%33rd18
195610110100%32nd19
195712212200%34th17
195812412400%26th25
Total1,7681,745231.30%Mean 20thMean 23
   Median 25thMedian 24
Mode 25thMode 26

Source: Compiled from JAMA “State Board Statistics” (1924-1929) and “Medical Licensure Statistics” (1930-1958). This appeared annually—usually in April or May.

Appendix 2

# of Physicians in MS, 1938-1947

YearTotalWhite“Colored”
19381446139254
19391436138155
19401425137253
19411356130650
19421330127951
19431200115149
19441160111149
19451112106052
19461213116350
19471351129853

319 MS physicians served in WWII

Source: Public Health and Medical Licensure in Mississippi, Vol 2. FJ Underwood, RN Whitfield. Jackson: Tucker Printing House, 1938, p. 378

Appendix 3

Mississippi Board members 1924-1958

Board memberYears servingTotal # YrsSubject area
Arrington19581pathology
Austin1924-194320Anatomy; Materia medica
Avent1947-195812Chemistry
Banks1934-195724Pathology
Blackburn1950-19589Hygiene
Brock1936-19416Physiology
Brown19251Material medica
Caldwell1944-195815EENT; Surgery
Crawford1926-19294Surgery
Culpepper1942-19476Physical diagnosis
Dampeer1924-19296Physiology
Darrington1931-19355Surgery
Dearman1931-19355Physical diagnosis
Eason1924-195118OBGYN
Field1942-195817Physiology; Materia medica
Frizell1930-19356Physiology
Gamble1944-195613Anatomy
Garrison1948-195811Theory, practice of medicine
Gavin1924-1925`2Hygiene
Haralson1924-19252Pathology
Holmes1925-19273Chemistry
Hooper19241Chemistry
House1952-19587OBGYN
Howell1936-19416Surgery
Lipscomb1926-194318Theory, practice of medicine
Long1942-19487Surgery
McKinnon1936-1941, 1948-195817EENT, Physical diagnosis
Seale19251Theory practice of medicine
Shaw1928-194619 
Stennis1926-19337Pathology
Underwood1925-195834Histology/bacteriology
Wall1924-19252Surgery
Watson1926-19294EENT
Wilkins1957-19582Anatomy
Wright1926-194924Hygiene

35 members   

Average length of service in years: Mean 9.57 Median 6  Mode 6

Source: Two-volume set of Mississippi Board of Health licensing exam questions housed at the FSMB Historical Collection

Who were these guys? A Look at Early Licensed Physicians in Mississippi

/ davidfsmb / 1 Comment

We know a good deal about the early history and development of our system for medical licensure in this country. Historian of medicine Richard Shryock and medical regulator Robert Derbyshire1 contributed short but valuable works in the 1960s. Sociologist Paul Starr addressed licensing as one part of his masterwork on the social development on the medical profession.2 Scholars like Ronald Hamowy and Samuel Baker authored oft-cited journal articles looking specifically at the legislative origins and evolution of early medical licensure3 while the historian James Mohr explored the legal basis for medical regulation.4 What all of these authors contributed were broad brush stroke portraits of our medical licensure system—narratives delivering a valuable macro-level view of the subject.

What has been less frequently presented are state-level analyses of these origin stories for medical licensure. A scholarly few exist: Clinton Sandvick for Illinois, Samuel Baker for Massachusetts. Yet there is little in the scholarly literature that takes even a cursory look at the demographic profile of early licensees in any state. To put it bluntly, we have seldom posed the question, “Who were these guys?” (And yes, I know there were women too; small in number attempting to succeed in an entrenched patriarchal landscape)

The gift of a series of registry books from the Mississippi State Board of Medical Licensure to the FSMB Historical Collection provided me with an opportunity to explore that ‘Who were they?’ question directly. The donated volumes contained thousands of individual records assembled as part of a 1930s W.P.A. project for the Mississippi Department of Health—specifically individual applications for medical licensure in that state.

The time available to me precluded reviewing the entirety of the records contained in those volumes. Instead, I undertook a targeted, hopefully representative sampling from those volumes—approximately 472 records. What follows is a demographic summary of that sample set supplying insight into the early licensed practice of medicine in Mississippi. Who were they? Where were they educated? How much (in)formal training did they bring to their practice?

The 472 individuals were licensed over six decades (1882-1940). The majority of the records (80%) fell within the first three decades of that period. It comes as no surprise that the licensees over overwhelmingly male over this time span. Only six women were found to have been licensed in this sampling. Similarly, this sample found only four likely black physicians if we infer race from their associated medical school (Meharry-one of the historical black medical colleges).

While intra-professional strife characterized the relations between “regular” (i.e., allopathic) physicians and their “irregular” brethren for the second half of the nineteenth century, the small number of non-allopaths during this extended period (n=15) suggests these tensions may have minor irritants at best in Mississippi. The fifteen were categorized by the board as either eclectic, homeopathic, botanic or mineral physicians. Interestingly, no osteopathic physicians appeared in this data set spanning six decades.

Time period# of licenseesHolds Medical degreePreceptorship“Irregular” practitioners
1882-189017268 (39%)125 (73%)15
1891-19007948 (61%)41 (52%)0
1901-191012660 (47%)57 (45%)0
1911-19204134 (83%)21
1921-19303534 (97%)30
1931-19401916 (84%)00
     
Total 1882-1940N=472N=260 (55%)N=228 (48%)N=16 (3%)

One question peaking my curiosity involved education—what type of medical education and training did these early licensed physicians bring to the treatment of their patients in Mississippi? 

Only 55% of all the physicians in this sample reported having graduated with a medical degree. For those who did, where they attended medical school is worth sharing. Few of these physicians obtained their medical education and degree outside of the South. The medical colleges in Tulane and Memphis were the primary source institutions with 116 and 115 licensees respectively. Degree holders from schools in Louisville, Nashville and Jackson were smaller in number but still significant sources of the degreed-physicians in Mississippi.

Only 15 of these licensees presenting a medical degree obtained that credential from an institution outside of the South. Schools in Chicago, Cincinnati, New York City and Philadelphia predominated. (Note: My definition of the “South” included border states such as Missouri, Kentucky, Maryland)

If the strong southern orientation of the medical schools for these degree holding physicians didn’t surprise me, something else did: The number of individuals reporting medical education from more than one institution. More than one quarter (n=128) of the 472 physicians in the sample reported attending lectures at two or more medical colleges. While speculative, it may be surmised that repeating a set of didactic lectures at a second school was, if not common, then at least not unusual. Additionally, seeking clinical experience away from the school providing the core medical lectures seems understandable in an era when the onus for obtaining training fell more to the physician rather than their institution.

One of the more telling figures in suggesting the profoundly different educational landscape for medicine at that time involves the number of licensees presenting to the Mississippi board absent a medical degree. Less than half (46%) contained in this sample covering the period 1882-1910 possessed a medical degree.

This may seem strange to us today. However, like many states, the introduction of licensure laws in Mississippi included provision for licensing individuals who did not attend or graduate from a medical college.5 The reality is that such provisions were pragmatic in nature as state legislatures opted more often than not to grandfather in those individuals already practicing in the state who might not otherwise meet newly implemented requirements.

A total of 212 individuals reported no medical degree…and yet all but 27 of these physicians reported having attended at least one set of lectures at a medical college. Thus, licensed physicians who were wholly self-taught or trained solely via preceptor or apprentice relationships constituted only 6% of all the licensees in this sample.

The most common experience among the Mississippi licensees spanning these first three decades involved preceptorship. 59% of all the licensees during that period claimed one. Interestingly, familial connections may have played a prominent role in this as 55 of the 223 licensees did their preceptorship with a physician bearing the same last name. While some of these may have been coincidence, it seems likely that familial connection (father, brother, uncle, cousin) facilitated the training. The reported length of time under preceptorship seems not insignificant—a 3-years mean/median across the entirety of the sampling. Even if this self-reported data is inflated by half, it suggests that informal training occurred longer and more extensively than one might have thought.

One caveat to the preceptor data. This practice was common enough that it remained a standard field on the licensing application throughout the entire 1882-1940 time period. However, it was clear to me that some physicians (or the staff member filling in the form) classified the clinical portion of their medical education under the preceptor category. I excluded those instances where it seems clear this was being done.

So what to make of this data? Perhaps the key word to keep in mind is dynamic. This data, while specific to a single state, reflects the evolving nature of the medical educational and regulatory systems in late nineteenth and early twentieth century America—essentially a system in flux. Educational standards were still being formulated through a de facto accreditation process via the AMA Council on Medical Education and AAMC. Legislatures were following that lead by codifying these classifications for medical schools as the basis for licensure: Graduates of Class A and B schools as eligible; Class C graduates ineligible. Mississippi required graduation from a Class A school starting in 1919.6 Legislatures enacted and medical boards operationalized licensure requirements mindful of this still evolving educational landscape as well as the presence of thousands of established practitioners whose medical careers predated these changes.

Of course, we should be cautious in drawing conclusions from what is a sample set of data. Still, the data is consistent with what we would expect during a transition era in the history of the profession and the regulatory system. Perhaps an ambitious scholar would like to visit the FSMB Historical Collection and use these records for a more systematic survey and analysis?  

The opinions expressed are those of the author and do not represent the views of the Federation of State Medical Boards.

  1. Richard Shyrock, Medical Licensing in America, 1650-1965 (Johns Hopkins, 1967); Robert Derbyshire, Medical Licensure and Discipline in the United States (Johns Hopkins, 1969)
  2. Paul Starr, The Social Transformation of American Medicine (Basic Books, 1982)
  3. Ronald Hamowy, “The Early Development of Medical Licensing Laws in the United States, 1875-1900,” Journal of Libertarian Studies, 1979; Samuel Baker, “Physician Licensure Laws in the United States, 1865-1915,” Journal of History of Medicine and Allied Sciences, 1984
  4. James Mohr, Licensed to Practice (Johns Hopkins, 2013)
  5. See Section 17 of An Act to Regulate the Practice of Medicine in the State of Mississippi (Jackson: State Printer, 1882)
  6. Mississippi Laws and Extracts of Laws Dealing with Public Health (Jackson: Mississippi State Board of Health, 1947), 104

Unexpected Encounters with History

/ davidfsmb / Leave a comment

Earlier this summer my wife and I were traveling through Colorado and spent a couple nights in Trinidad. After a nice dinner at a Mexican restaurant in their downtown, we strolled the streets, window shopping and enjoying the mild weather. We ran across a placard inside one building that gave a bit of history to that site and Trinidad’s early days.

The placard related a number of “firsts” dating to 1867, including mention of Dr. Michael Beshoar as the proprietor of the “only drug store Santa Fe to Denver.”

I was taken aback when I read the name. Michael Beshoar didn’t just ring a bell. I knew precisely who he was and why I remembered that name. It turns out that one of Trinidad’s most famous early physicians had a direct connection to America’s largest medical diploma mill. As Dr. Beshoar learned firsthand, eighteen hundred miles wasn’t far enough to distance himself from a questionable decision and even more dubious connection.

Prior to seeing Beshoar identified as a prominent early citizen of Trinidad, my knowledge of him derived from familiarity with seeing his name on the printed list of faculty for John Buchanan’s medical college in the mid-1870s. I learned a bit more about Beshoar when I discovered the Denver Public Library contained archival materials including his “papers.” From these I learned that Beshoar’s connection with the infamous diploma mill dated to 1873.

On April 10 of that year, John Buchanan responded to an inquiry from Beshoar stating:

“Dear Sir, if you comply with the [?] directions in your letter,  the desired article will be forwarded to you at once. Your  matriculation ticket or whatever you send will be carefully [?]. Please send money for P.O. order or draft.”

The indirect language and euphemism (“desired article”) were common features of John Buchanan’s operative style when dealing with prospective customers by mail. The context for this letter and later exchanges between the two men make it clear that Beshoar had expressed interest in purchasing a diploma from one of Buchanan’s legally chartered schools: the Eclectic Medical College (EMC) or the American University of Philadelphia.

Another letter from 1879 included an interesting pledge from Buchanan. He assured Beshoar that, “we shall take care of your interests if there is any inquiries.” Whether this was specific to securing students for his fall 1879 classes at the EMC or if this alluded to the press inquiries growing more frequent concerning the nature of Buchanan’s operations is unclear. One suspects that Buchanan stood behind both.

When I finished my book on the subject (Diploma Mill), my view of Dr. Michael Beshoar was uncomplicated. He was simply one of the rogues unmasked by the Philadelphia Record’s crusading editor, John Norris after authorities’ arrest and search of Buchanan’s offices at 514 Pine Street. But that placard in Trinidad gave me pause. What did I really know about Dr. Beshoar? Was he just another of Buchanan’s scoundrels? Or was there more to his story?

I decided to look more closely. As you’ll see, the picture becomes more nuanced and less clear. (Note: The same is true for another EMC “graduate” that I wrote about in 2019)

Michael Beshoar was born in 1833 and later graduated from the University of Michigan’s medical college in 1853. Beshoar relocated to Arkansas where he established deep roots: he married, established a medical practice and served in the state legislature. When the Civil War broke out, he served as a medical officer for the Confederacy. Captured in 1863, he began treating Union troops and ultimately served the U.S. Army at Fort Kearney in Nebraska Territory.

A flamboyant frontier persona

By 1867, he drifted into Colorado, first Pueblo and then Trinidad. In the latter town, his wide interests and boundless energy saw him practicing medicine, operating a drug store, creating his own patent medicines; and later opening a newspaper, engaging in land speculation, serving on a vigilance committee, pursuing local and state political offices and much more. Dr. Michael Beshoar’s boundless energy and entrepreneurial interests touched just about everything and everyone in the Trinidad region.

All of which begs the question: Why did Michael Beshoar get mixed up with America’s most notorious medical diploma mill? 

To understand this, we need to go back to Beshoar’s days in Arkansas. Like so many Southerners, the Civil War ruined him financially. In an 1879 letter, he claimed to have come out of the war with “$1.20 suit of summer clothes” and little else. Thousands of acres that he had procured earlier in northern Arkansas were lost to back taxes. His dreams (“I was sure to being a millionaire in a few years”) were dashed.

Still, ambition never seemed to wane in Beshoar. While he studied and practiced medicine, Beshoar appeared to see himself as much more than just a physician. Ambition is the key to understanding Dr. Michael Beshoar and ultimately his interactions with Dr. John Buchanan.

Let’s go back to that 1873 correspondence with John Buchanan. At that time, medicine was effectively an unregulated field in this country. Only North Carolina and Texas had laws in place controlling medical practice. Even then I doubt whether much, if any, enforcement of these laws took place.

Most Americans’ experience of health care involved what we would categorize today as folk medicine, i.e., botanical remedies delivered in the home by family, friend or a trusted neighbor. The “doctor” called in to render aid was more often a local healer whose skills (real or perceived) were valued in their own right. Probably only a minority of Americans received treatment at the hands of a doctor as defined by someone possessing a medical degree or diploma.

Recall that Beshoar received a medical diploma from Michigan. So why would he have been seeking one of Buchanan’s diplomas? Initially, I thought the likely explanation was simply that Beshoar no longer had the sheepskin credential in his possession after his many moves. That turned out not to be the case as newspapers covered Beshoar’s attendance at a 1903 reunion in Ann Arbor where he produced the original diploma and seal issued to him half a century earlier.

So if it wasn’t a case of replacing a lost diploma, why would have wanted one from Buchanan? The answer may be as simple as this: While a medical degree wasn’t necessary to practice in Colorado, it was a great distinguisher that could only enhance Beshoar’s standing in the community. So, if he was going to have a diploma, it couldn’t hurt to have a degree from a school in Philadelphia, the unchallenged epicenter of American medicine. Beshoar’s would have been an ad eundem degree, i.e. one conferred in addition to an earlier degree. This is entirely speculative on my part but wholly plausible. This was a fairly common practice among 19th century colleges and universities.

Beshoar and Buchanan engaged in periodic correspondence in the 1870s. In his February 1875 letter to Beshoar, Buchanan wrote, “Your letter and proposition accepted….” Beshoar had apparently requested appointment to the EMC faculty and though the appointment ultimately was just titular (Beshoar’s name on the faculty list), their correspondence makes clear that Buchanan actually expected Beshoar to join him in Philadelphia to teach the fall 1875 session. Their correspondence continued into 1878 with Buchanan still hoping for Beshoar’s presence on-site and asking for a photograph so that all faculty could be represented in their advertising.

Ultimately, Beshoar did receive a Buchanan diploma and had his name on the faculty list presented in the EMC’s journal and the school prospectus. While these items might have been showcased in Beshoar’s office, they later proved problematic for him.

In the summer of 1880, John Buchanan was arrested and his offices searched. His subsequent faked suicide and flight from justice made front page news across the country. The Philadelphia Record and other newspapers printed various materials captured in the police raid, including exhaustive lists of the many recipients of Buchanan diplomas. Nestled among the the names: Michael Beshoar of Trinidad, Colorado

In an undated letter, the president of the Colorado State Medical Society demanded that Beshoar appear to answer multiple charges including questionable advertising, misrepresentations about his relationship to the ‘Rocky Mountain Medical Association’ and his status as “Emeritus Professor of the Practice of Medicine in the American University of Philadelphia—a notorious mill concern.”

Since the Colorado State Board of Medical Examiners wasn’t established until 1881 there were no licensing repercussions for Beshoar. And at the end of the day, membership in the state medical society wasn’t necessary to practice medicine.

So where did this leave Beshoar? Perhaps slightly embarrassed and the subject of snide comments from locals who heard the rumors coming from Beshoar’s medical brethren but probably not much more.

His interactions with Buchanan and his dubious medical colleges carried little risk for most of the 1870s. Beshoar sought a credential and a little gravitas through a titular faculty appointment that he likely saw as just a means to an end—bolstering his reputation and his medical practice in the community. These were precisely the moves of an energetic entrepreneur. Unfortunately for him, changing times brought legislation regulating the practice of medicine to every state and territory. The go-get’em spirit that rewarded bold, opportunistic types like Dr. Michael Beshoar looked a lot different by 1880 when the full nature of Buchanan’s diploma mill became known. Beshoar’s wide interests and influence in southern Colorado’s medical, political and newspaper communities made a fair number of rivals and outright enemies. His connection to Buchanan and the EMC provided easier fodder for critics looking to smear Beshoar’s reputation.

I have yet to run into anything in the historical record for Beshoar that suggests this blemish on his career made any lasting impression. While his grandson’s book, Hippocrates in a Red Vest, addressed the issue directly, his review of the evidence strongly suggests the links to Buchanan were a minor irritant in an otherwise remarkably diverse and successful career. Most of the sources and stories characterize him as a bit of free-spirited pioneer. Fair enough. Sleep well, Dr. Beshoar.

Dr. Beshoar in center right light colored top hat. Standing before one of the newspapers he founded.

The opinions expressed are those of the author and do not represent the views of the Federation of State Medical Boards.

Sources:

Michael Beshoar Papers, Box 7, Folder FF19

Barron Beshoar, Hippocrates in a Red Vest

Carl Bartecchi, “Practicing Medicine in the Wild West.” Access August 22, 2025 at https://www.michiganmedicine.org/medicine-michigan/practicing-medicine-wild-west

Nancy Kristofferson, “Michael Beshoar: Doctor and Developer,” World Journal. Huerfano, Colorado. November 20, 1914. Accessed August 22, 2025 at https://worldjournalnewspaper.com/michael-beshoar-doctor-and-developer/