Self-grading your MEE/MPT answers

One question examinees often ask me is whether they should self-grade their MEE/MPT answers. While anyone can self-grade MBE questions, it’s more difficult to get a gauge for how much you need to know and how much you do know for purposes of essay questions when all you have is model answers. As a result, most examinees shy away from self-grading their written MEE/MPT answers. However, this is where the materials available through this module can help you immensely as one of their major benefits is the ability to self-grade. Examinees can use the MEE Essay Compilation or MEE Issue Spotting outline to identify the issues/legal problems and then supplement with the MEE/MPT Comparisons. These Comparison Banks mimic the process a grader goes through, and the more you put yourself into the shoes of the grader, the better you will score on the MEE/MPT.

Almost all states scale their written component to the MBE. This sometimes leads to examinees failing the exam who would have passed if it weren’t for the scaling system. Basically, an examinee can fully demonstrate knowledge on an essay, but once the scaling and adjustment take place, they may no longer have a passing grade. Because of this, Oklahoma stopped scaling their essays to the MBE in 2016.

However, since almost all states continue to scale their essays to the MBE, this is a battle you must fight on your opponent’s battlefield according to your opponent’s rules. For these scaling states, NCBE tells graders to put their graded essays into “buckets” which contain an equal number of 1 essays, 2 essays, etc. This allows for a normal distribution of scores, but can lead to arbitrariness/unfairness in essay grading. For example, if your essay deserves a 4, but the grader’s 4 bucket is too full, he may unfairly move your essay into the 3 bucket.

So the next question is how does the grader decide what essay goes into which bucket. According to a 1977 study entitled An Analysis Of Grading Practices On The California Bar Examination by Stephen P. Klein, Ph.D., the “grading standards for the California bar exam essays are based on: analysis of the problem, knowledge of the law, application of legal principles, reasoning, and the appropriateness of the conclusions reached. The objective “correctness” of the answer are not supposed to affect the grade assigned.” Bar exam graders have to be trained to apply these scoring rules consistently through a process called “calibration.”

At a March 2011 bar exam workshop at New York Law School, Bryan R. Williams of the NYS Board of Law Examiners stated: “The grading of the exam is done by seven people throughout state – all practicing lawyers. There are five board members of the NY Board of Law Examiners who are appointed by the Court of Appeals. Each one of those board members have seven people who are in their team. Each person is responsible for one essay, and that team of people, then they grade the essay and the MPT. So what happens is we have the question written, and then we have a model answer. And just like this exam that was just given, a few days after the exam, all of us, the seven graders and myself, will receive about 50 sample answers given by candidates, so we all get the same 50, and we individually go and we grade those exams based upon the model answer that they did, and then we have a meeting and we come together and we make sure that we are all grading the same way, so we can get calibrated, and there has never been a time since I’ve been doing this, and I’ve been doing this since 1986, there has never been a time where we would have had that meeting and because of the kinds of answers we get back, we don’t in some way change our model answer because what we are trying to do, we are trying to rank order people.”

The more calibration sessions, the more reliable the essay grading. In California, a group of graders hold three calibration sessions to ensure that grading is done consistently as the graders continue to score answers. In New York, it is not clear how many calibration sessions are undertaken, but it is likely less than three. Graders use calibration sessions to standardize the scores for various quality levels of answers. This ensures that theoretically at least, the same essay would receive the same points from any of the graders assigned to that essay. Essay grading reliability diminishes if the graders are not sufficiently trained to apply the scoring rules consistently. This was confirmed in the 1977 Klein study: “there was far more consistency among the readers before the regular reading process began (calibration data set) than there was once this process was underway. This difference is evident on all three indices of agreement and clearly illustrates that the initial calibration data does not reflect accurately the degree of agreement among, the readers in the scores that are subsequently used in determining an applicant’s pass/fail status. For instance, with the calibration sample there was a range of 70-85 percent agreement on the pass/fail decision~ whereas this range dropped to 27-57 percent at the beginning and to 23-53 percent at the end of the regular reading period. In other words, during the normal reading process, the readers agreed with one another about one-half as well as they did during the calibration process!” see The State Bar Of California Committee Of Bar Examiners/Office Of Admissions Description And Grading Of The California Bar Examination – General Bar Examination And Attorneys’ Examination

NCBE provides a very detailed answer and analysis to each MEE/MPT question. This makes it much easier to self-grade. Following is a detailed explanation of HOW an examinee can accurately self-grade their MEE/MPT answers using the materials on this site.

In tutoring, I grade essays on a scale of 0-10 where 5 is exactly passing. A legend for the scoring is below.

0 = Did not discuss any issue or the answer is completely irrelevant
1 = Very poor discussion of the issues. Wrong law and poor analysis.
2 = Poor discussion of the issues missing important law/analysis OR right conclusions but no issue statements, rules or analysis.
3 = Inadequate answer. Missing parts of law and/or analysis.
4 = Not quite passing answer. Some incorrect law and adequate analysis OR correct law and some inadequate analysis.
5 = Exactly passing answer. Answered the issues sufficiently with adequate analysis, even if the answer not correct.
6 = Above passing answer. Answered the issues correctly with more detailed analysis.
7 = Good answer. Answered the issues correctly with good analysis.
8 = Very good answer. Solid answer with detailed elements and good analysis.
9 = Great answer. Correct on all aspects of the issues and detailed elements. Used the appropriate terms of art. Solid detailed analysis.
10 = Excellent answer. Fully answered the issues correctly with strong lengthy analysis. Discussed each element and used all relevant facts.

For each MEE practice question issue, you should self-evaluate by comparing your answer to the MEE Answer and then entering a grade ranging from 0-10 into a spreadsheet to track your progress. For example, if you think the first issue for an MEE question was difficult and you missed some law, you should enter a code between 2 or 3. You should do this for each issue contained in each MEE question. Use the Issue Point percentage (e.g. 50%) to help you decide if your writing was sufficient for the topic. For example, if a topic was 50% of your score but represented only 25% of your writing, you should downgrade your score for that topic. Even if your jurisdiction uses a different grading scale, you should still use the above legend as a guidepost when you self-grade your practice essay answers.

You should use the MEE and MPT Comparisons to get an idea of whether your word count and analysis is consistent with passing essays. Put simply, these are the best resources available for accurately self-grading your MEE and MPT answers. As you review the better essays, your format will start to change and you will focus on framing the issues with more particularity. Much like a grader will calibrate his grading by looking at the quality of a range of different answers, you will learn how to adjust your responses to mimic those regarded as superior. Your primary goals when using the Comparison Banks should be: (1) to see how the high scoring essays are structured (e.g. how they use CIRAC/IRAC, how they address the issues, how they format their answer in regards to issue statements, conclusions and bolding/underlining/italicizing); and (2) to see how the high scoring essays properly analyze the issues. Good analysis is a pre-requisite for a passing essay score. For example, as stated in a Southern Illinois University School of Law presentation, the most common problem seen on the exams of those who failed was the absence of factual analysis. Analysis is the most important element of IRAC because this is where the legal reasoning occurs. Continually observing how high scoring answers analyze the facts will give you insight on how to replicate such analysis in your essays. As one passing examinee told me: “Reviewing the essay comparison tool gave me a better idea of where I was going wrong. Once you get past the mental hurdle of how much information is available, or really, once you figure out what to focus on, it’s such an invaluable tool. I’m not sure if I was right on the substance of what I wrote this time, but I am damn confident my writing style and overall tone improved.”

When you are self-grading practice essays, be a bit harsh in your grading. This helps you avoid lulling yourself into a sense of false security that may prevent you from really studying the material more. As one top scoring F20 examinee said: “I got one 4 and all the rest 3s (for IL the range is 1-6, passing starts at 4) from the graders. On the real deal I got 175. During the last two weeks, I focused on essays. I graded myself really harshly (like if I didn’t split up the rules like they did, I didn’t give myself that point) so that I would learn.” Don’t be overly harsh – simply imagine yourself as a skeptical grader and be realistic about how the grader would score your essays if it were the actual exam. Always keep in mind that self-grading can be a double-edged sword. On some essays, you will feel completely demoralized, while on other essays you will give yourself an inflated sense of confidence in your ability. Making MEE rules helps overcome these potential problems. When you self-grade using the model answers, copy the sections of the answer text that contain the rules you missed. For example, a domestic-educated examinee who passed on his 2nd attempt wrote full essay answers in practice and made an essay rules outline. He told me: “Bar reviews put emphasis is on building your own outlines from the lectures. That only works if you’re a person who learns while typing. For me, it was easier to review other people’s outlines — like yours — and build lists of rules based only on my mistakes.” Keep these MEE rules list the same way you keep your MBE rules lists – categorized by subject (e.g. a batch of Wills MEE rules, Trusts MEE rules, etc.) and even by sub-category. Every so often, print out these missed rules (large bold type is best) to read/review. Also, occasionally read these MEE rules out-loud to yourself or record yourself reading the rules and listen to the recording. Making these rules will help prevent self-grading from demoralizing you. Bottom line – you don’t want to give yourself an inflated sense of confidence in your ability.

An indirect way to start your essay study is to use the MEE essays to supplement your MBE studies. For example, if you had problems with joinder in MBE practice, read the MEE essays on joinder (Ctrl+F using the MEE Essay Compilation). Since your knowledge for the MEE is going to be limited in early study. you should answer MEE essays open book where you read the question and answer, and then write/type the exact answer as if you were transcribing it. The act of writing it forces you to think about how everything is put together. Then as your knowledge develops, go from open book to closed book. Depending on how much time you have to study, you should issue spot essays if your time is limited, or answer full essays if you have more time. A good rule of thumb is one full essay and 2-3 issue spotted essays per day in the afternoon or evening (after you did your morning study). Ultimately, you want exposure to as many MEE essays as possible. For example, one foreign examinee who received the highest MEE essay score on the Secured Transaction essay told me she simply reviewed the Secured Transaction essays going back 15 years. Often, a topic from a past essay will show up on the MEE, so the exposure will help you spot the issue and know what to write.

Even NCBE acknowledges that testing and practice are important learning tools. According to NCBE: In order to drive student learning, law school faculty should ask themselves whether their tests assess what they want students to learn and whether their tests are given in time to provide feedback that will enhance student learning. Tests are a powerful motivator, and testing time is not a waste of instructional time if the tests are focused on important concepts. Likewise, studying for a test is a good use of learning time if the tests are testing important concepts. Testing early and often is important to provide guidance to students about whether they are on track or whether they need to study more in order to succeed in the course. A quick quiz, perhaps in the last 5 to 10 minutes of each class, would be one way to provide this feedback. The quiz could consist of short-answer or multiple-choice questions, and it could be self-graded. see The Bar Examiner, December 2011

I am in the process of developing a spreadsheet which will be keyed to my MEE Outline to enable examinees to more efficiently self-grade their MEE answers. I expect to release this here in late-June 2020.

What is takes to get a 120 on the written portion of the UBE

Based on NCBE statistics, a July examinee with a written score of 120 did better than only 10-12% of examinees nationwide on the MEE/MPT. To better understand what 120 written score looks like, following is a July 2016 MEE answer that received a score of 42 in NY. If a July NY UBE examinee received a score of 42 for each of his 6 MEE essays and 2 MPTs, that examinee would have received a total written score of 120 (meaning they would have needed a 146 on the MBE to pass with a 266). Please note that this is a single sample essay and some examinees write much less and receive higher scores while other examinees write more and receive lower scores.

Study suggestions for subscribers

I suggest that UBE ESSAYS account subscribers use the materials as follows:

RELEASED MEE ESSAY COMPILATION AND MP3s: Examinees should start with a combination of reading the Released MEE Essay Compilation and listening to the Released MEE Essay MP3s. Start with the February 2016 MEE and work backwards (e.g. July 2015, February 2015, etc.). Much like the recent OPE exams are a good reflection of the current MBE, the recent MEEs are a good reflection of the current MEE. You will not be reviewing the most recent MEEs (July 2016 to Feb 2018) because you will be testing on these questions later. It should take an examinee 6-10 minutes to read one MEE question and answer analysis in the Released MEE Essay Compilation or 12-18 minutes to listen to the MP3 version (unless you speed up the MP3 to 1.5x-2x speed using a player like VLC to bring it in line with reading speed). How many of these MEE questions/answers you review depends on how much study time you have, but I suggest reading a minimum of 30 questions and listening to a minimum of 30 different questions. This would require 12 hours of your study time. An added benefit to reviewing the law in the MEE questions is that the questions based on MBE subjects will help you on the MBE. For example, when NCBE introduced the MBE subject of Civil Procedure in 2015, they needed to create a large bank of MBE Civil Procedure questions, and I believe they borrowed issues from past MEE questions that could be turned into short answer questions. Thus, reviewing these MEE questions will count towards MEE and MBE study.

MEE ISSUE SPOTTING PRACTICE OUTLINE AND MP3s: I make the past MEEs even more useful/efficient by creating an MEE Issue Spotting Practice outline along with MP3s of it. After reviewing full NCBE MEE questions to familiarize themselves with the MEE and using them as learning tools, examinees should then go through the entire MEE Issue Spotting Practice outline. This outline will take about 10-12 hours to read once in its entirety (or 17 hours to listen to the MP3 version). In doing so, you will be reviewing the 200 most recent MEE questions as efficiently as possible since the only things you will be reviewing are the question itself, then each issue tested (along with how much that issue contributed to your score), and then a short summary of the answer just to give you a gist of the law being tested. This is absolutely the most efficient way to review a large number of MEE questions in volume. This outline serves not only as a study outline, but also as a practice outline since examinees can read/listen to a question and then pause to issue spot in their head, and then continue on to the answer. My suggestion is to read the entire MEE Issue Spotting Practice outline and then listen to the subjects you are having the most difficulty with (e.g. Civil Procedure). By employing two different methods of review/memorization, you will hopefully form multiple memory impressions to help you better recall the information on the exam.

MEE COMPARISON BANKS: Next, you should practice answering MEE questions using the most recently released MEE questions. At present, there are 18 MEE Comparison Banks. Thus, once you have reviewed a subject (e.g. Civil Procedure), go to the MEE Comparison Bank for that subject. At the top is the MEE question. Open the PDF for the question (dont look at the answer) and then either print/read it and then answer the question under timed conditions (30 minutes in total). After you answer the question, spend 30 minutes comparing your answer to the answers in the MEE Comparison Bank. Start by looking at the NCBE Answer Analysis which breaks down each of the issues and how much each issue is worth. Then go and look at exactly passing answers to see how close you were to them. Then look at high scoring examinee answers to see what they did differently (did they do more analysis, address issues differently from you). Then look at the very low scoring answers to see if you did anything they did (areas to avoid). Over one month, repeat this for every single essay in the MEE Comparison Bank (roughly 18 hours of your study time in total)

MPT FORMAT BIBLE: For the MPT, review the MPT Format Bible for about 2-4 hours. Use the MPT Format Bible to review the most commonly tested formats and styles. Examinees should review the MPT answers to understand how a Persuasive MPT answer is written versus an Objective MPT answer. Examinees should review the answers for the MPT types that did not have guidelines or had minimal guidelines to ensure they understand the formats required for these types of MPTs. Finally, examinees should briefly review MPT answers that contain a Statement of the Case or a Statement of Facts to understand how to compose one if necessary.

MPT COMPARISON BANKS: Next, examinees should answer 2-4 MPTs in practice depending on the amount they are studying. At present, there are 19 MPT Comparison Banks. There are 6 MPTs from 2010-2012 which can be downloaded free here: Pick one of these MPTs (choose the ones which correspond with the MPT Comparison Bank) and either print/read it and then answer the MPT question under timed conditions (90 minutes in total). After you answer the question, spend 90 minutes comparing your answer to the answers in the MPT Comparison Bank. Start by looking at the NCBE Point Sheet which breaks down each of the issues and how much each issue is worth. Then go and look at exactly passing answers to see how close you were to them (look at word count also). Then look at high scoring examinee answers to see what they did differently (did they do more analysis, better formatting, better layout, addressed issues differently from you). Then look at the very low scoring answers to see if you did anything they did (important areas to avoid). Over one month, do this once a week until you answer 2-4 MPTs in total (roughly 6-12 hours of your study time in total). Note: If you have access to the MPT questions from 2013-2018, you should choose a few MPTs from this date range that are also in the MPT Comparison Bank.

All together, the above MEE/MPT study and practice will require a minimum of 50 hours of your study-time if you are a full-time examinee. However, depending on how much you study, you should adjust it accordingly. For example, if you are studying part-time rather than full time, only follow 50% of every suggestion. In contrast, if you are studying early and therefore studying more than a typical examinee, increase the proportions accordingly.