Self-grading your MEE/MPT answers
One question examinees often ask me is whether they should self-grade their MEE/MPT answers. While anyone can self-grade MBE questions, it’s more difficult to get a gauge for how much you need to know and how much you do know for purposes of essay questions when all you have is model answers. As a result, most examinees shy away from self-grading their written MEE/MPT answers. However, this is where the materials available through this module can help you immensely as one of their major benefits is the ability to self-grade. Examinees can use the MEE Essay Compilation or MEE Issue Spotting outline to identify the issues/legal problems and then supplement with the MEE/MPT Comparisons. These Comparison Banks mimic the process a grader goes through, and the more you put yourself into the shoes of the grader, the better you will score on the MEE/MPT.
Almost all states scale their written component to the MBE. This sometimes leads to examinees failing the exam who would have passed if it weren’t for the scaling system. Basically, an examinee can fully demonstrate knowledge on an essay, but once the scaling and adjustment take place, they may no longer have a passing grade. Because of this, Oklahoma stopped scaling their essays to the MBE in 2016.
However, since almost all states continue to scale their essays to the MBE, this is a battle you must fight on your opponent’s battlefield according to your opponent’s rules. For these scaling states, NCBE tells graders to put their graded essays into “buckets” which contain an equal number of 1 essays, 2 essays, etc. This allows for a normal distribution of scores, but can lead to arbitrariness/unfairness in essay grading. For example, if your essay deserves a 4, but the grader’s 4 bucket is too full, he may unfairly move your essay into the 3 bucket.
So the next question is how does the grader decide what essay goes into which bucket. According to a 1977 study entitled An Analysis Of Grading Practices On The California Bar Examination by Stephen P. Klein, Ph.D., the “grading standards for the California bar exam essays are based on: analysis of the problem, knowledge of the law, application of legal principles, reasoning, and the appropriateness of the conclusions reached. The objective “correctness” of the answer are not supposed to affect the grade assigned.” Bar exam graders have to be trained to apply these scoring rules consistently through a process called “calibration.”
At a March 2011 bar exam workshop at New York Law School, Bryan R. Williams of the NYS Board of Law Examiners stated: “The grading of the exam is done by seven people throughout state – all practicing lawyers. There are five board members of the NY Board of Law Examiners who are appointed by the Court of Appeals. Each one of those board members have seven people who are in their team. Each person is responsible for one essay, and that team of people, then they grade the essay and the MPT. So what happens is we have the question written, and then we have a model answer. And just like this exam that was just given, a few days after the exam, all of us, the seven graders and myself, will receive about 50 sample answers given by candidates, so we all get the same 50, and we individually go and we grade those exams based upon the model answer that they did, and then we have a meeting and we come together and we make sure that we are all grading the same way, so we can get calibrated, and there has never been a time since I’ve been doing this, and I’ve been doing this since 1986, there has never been a time where we would have had that meeting and because of the kinds of answers we get back, we don’t in some way change our model answer because what we are trying to do, we are trying to rank order people.”
The more calibration sessions, the more reliable the essay grading. In California, a group of graders hold three calibration sessions to ensure that grading is done consistently as the graders continue to score answers. In New York, it is not clear how many calibration sessions are undertaken, but it is likely less than three. Graders use calibration sessions to standardize the scores for various quality levels of answers. This ensures that theoretically at least, the same essay would receive the same points from any of the graders assigned to that essay. Essay grading reliability diminishes if the graders are not sufficiently trained to apply the scoring rules consistently. This was confirmed in the 1977 Klein study: “there was far more consistency among the readers before the regular reading process began (calibration data set) than there was once this process was underway. This difference is evident on all three indices of agreement and clearly illustrates that the initial calibration data does not reflect accurately the degree of agreement among, the readers in the scores that are subsequently used in determining an applicant’s pass/fail status. For instance, with the calibration sample there was a range of 70-85 percent agreement on the pass/fail decision~ whereas this range dropped to 27-57 percent at the beginning and to 23-53 percent at the end of the regular reading period. In other words, during the normal reading process, the readers agreed with one another about one-half as well as they did during the calibration process!” see The State Bar Of California Committee Of Bar Examiners/Office Of Admissions Description And Grading Of The California Bar Examination – General Bar Examination And Attorneys’ Examination
NCBE provides a very detailed answer and analysis to each MEE/MPT question. This makes it much easier to self-grade. Following is a detailed explanation of HOW an examinee can accurately self-grade their MEE/MPT answers using the materials on this site.
In tutoring, I grade essays on a scale of 0-10 where 5 is exactly passing. A legend for the scoring is below.
0 = Did not discuss any issue or the answer is completely irrelevant
1 = Very poor discussion of the issues. Wrong law and poor analysis.
2 = Poor discussion of the issues missing important law/analysis OR right conclusions but no issue statements, rules or analysis.
3 = Inadequate answer. Missing parts of law and/or analysis.
4 = Not quite passing answer. Some incorrect law and adequate analysis OR correct law and some inadequate analysis.
5 = Exactly passing answer. Answered the issues sufficiently with adequate analysis, even if the answer not correct.
6 = Above passing answer. Answered the issues correctly with more detailed analysis.
7 = Good answer. Answered the issues correctly with good analysis.
8 = Very good answer. Solid answer with detailed elements and good analysis.
9 = Great answer. Correct on all aspects of the issues and detailed elements. Used the appropriate terms of art. Solid detailed analysis.
10 = Excellent answer. Fully answered the issues correctly with strong lengthy analysis. Discussed each element and used all relevant facts.
For each MEE practice question issue, you should self-evaluate by comparing your answer to the MEE Answer and then entering a grade ranging from 0-10 into a spreadsheet to track your progress. For example, if you think the first issue for an MEE question was difficult and you missed some law, you should enter a code between 2 or 3. You should do this for each issue contained in each MEE question. Use the Issue Point percentage (e.g. 50%) to help you decide if your writing was sufficient for the topic. For example, if a topic was 50% of your score but represented only 25% of your writing, you should downgrade your score for that topic. Even if your jurisdiction uses a different grading scale, you should still use the above legend as a guidepost when you self-grade your practice essay answers.
You should use the MEE and MPT Comparisons to get an idea of whether your word count and analysis is consistent with passing essays. Put simply, these are the best resources available for accurately self-grading your MEE and MPT answers. As you review the better essays, your format will start to change and you will focus on framing the issues with more particularity. Much like a grader will calibrate his grading by looking at the quality of a range of different answers, you will learn how to adjust your responses to mimic those regarded as superior. Your primary goals when using the Comparison Banks should be: (1) to see how the high scoring essays are structured (e.g. how they use CIRAC/IRAC, how they address the issues, how they format their answer in regards to issue statements, conclusions and bolding/underlining/italicizing); and (2) to see how the high scoring essays properly analyze the issues. Good analysis is a pre-requisite for a passing essay score. For example, as stated in a Southern Illinois University School of Law presentation, the most common problem seen on the exams of those who failed was the absence of factual analysis. Analysis is the most important element of IRAC because this is where the legal reasoning occurs. Continually observing how high scoring answers analyze the facts will give you insight on how to replicate such analysis in your essays. As one passing examinee told me: “Reviewing the essay comparison tool gave me a better idea of where I was going wrong. Once you get past the mental hurdle of how much information is available, or really, once you figure out what to focus on, it’s such an invaluable tool. I’m not sure if I was right on the substance of what I wrote this time, but I am damn confident my writing style and overall tone improved.”
When you are self-grading practice essays, be a bit harsh in your grading. This helps you avoid lulling yourself into a sense of false security that may prevent you from really studying the material more. As one top scoring F20 examinee said: “I got one 4 and all the rest 3s (for IL the range is 1-6, passing starts at 4) from the graders. On the real deal I got 175. During the last two weeks, I focused on essays. I graded myself really harshly (like if I didn’t split up the rules like they did, I didn’t give myself that point) so that I would learn.” Don’t be overly harsh – simply imagine yourself as a skeptical grader and be realistic about how the grader would score your essays if it were the actual exam. Always keep in mind that self-grading can be a double-edged sword. On some essays, you will feel completely demoralized, while on other essays you will give yourself an inflated sense of confidence in your ability. Making MEE rules helps overcome these potential problems. When you self-grade using the model answers, copy the sections of the answer text that contain the rules you missed. For example, a domestic-educated examinee who passed on his 2nd attempt wrote full essay answers in practice and made an essay rules outline. He told me: “Bar reviews put emphasis is on building your own outlines from the lectures. That only works if you’re a person who learns while typing. For me, it was easier to review other people’s outlines — like yours — and build lists of rules based only on my mistakes.” Keep these MEE rules list the same way you keep your MBE rules lists – categorized by subject (e.g. a batch of Wills MEE rules, Trusts MEE rules, etc.) and even by sub-category. Every so often, print out these missed rules (large bold type is best) to read/review. Also, occasionally read these MEE rules out-loud to yourself or record yourself reading the rules and listen to the recording. Making these rules will help prevent self-grading from demoralizing you. Bottom line – you don’t want to give yourself an inflated sense of confidence in your ability.
An indirect way to start your essay study is to use the MEE essays to supplement your MBE studies. For example, if you had problems with joinder in MBE practice, read the MEE essays on joinder (Ctrl+F using the MEE Essay Compilation). Since your knowledge for the MEE is going to be limited in early study. you should answer MEE essays open book where you read the question and answer, and then write/type the exact answer as if you were transcribing it. The act of writing it forces you to think about how everything is put together. Then as your knowledge develops, go from open book to closed book. Depending on how much time you have to study, you should issue spot essays if your time is limited, or answer full essays if you have more time. A good rule of thumb is one full essay and 2-3 issue spotted essays per day in the afternoon or evening (after you did your morning study). Ultimately, you want exposure to as many MEE essays as possible. For example, one foreign examinee who received the highest MEE essay score on the Secured Transaction essay told me she simply reviewed the Secured Transaction essays going back 15 years. Often, a topic from a past essay will show up on the MEE, so the exposure will help you spot the issue and know what to write.
Even NCBE acknowledges that testing and practice are important learning tools. According to NCBE: In order to drive student learning, law school faculty should ask themselves whether their tests assess what they want students to learn and whether their tests are given in time to provide feedback that will enhance student learning. Tests are a powerful motivator, and testing time is not a waste of instructional time if the tests are focused on important concepts. Likewise, studying for a test is a good use of learning time if the tests are testing important concepts. Testing early and often is important to provide guidance to students about whether they are on track or whether they need to study more in order to succeed in the course. A quick quiz, perhaps in the last 5 to 10 minutes of each class, would be one way to provide this feedback. The quiz could consist of short-answer or multiple-choice questions, and it could be self-graded. see The Bar Examiner, December 2011
I am in the process of developing a spreadsheet which will be keyed to my MEE Outline to enable examinees to more efficiently self-grade their MEE answers. I expect to release this here in late-June 2020.
