How exams are built and scored
An exam score is a measurement made against a standard that a panel set, not a percentage of what you know. Knowing how items are written, how forms are equated into scaled scores and how adaptive tests stop removes most of the mystery, and several common worries about difficulty turn out to be misreadings.
Lessons in short
- EVIDENCE-BACKED A scaled score is not a percentage. Forms differ in difficulty, so each form has its own raw-to-scale conversion.
- EVIDENCE-BACKED The pass mark comes from a standard-setting procedure, so do not guess it from a round number.
- EVIDENCE-BACKED On an adaptive test, items feel hard because the test aims them at your level. A hard stretch is not a sign you are failing.
- EVIDENCE-BACKED Three good answer options are enough for a well-built item, so expect some options to be weak.
- EVIDENCE-BACKED Do not plan around item-writing flaws. They exist, but their effect on scores is uneven.
An exam is a measurement with a purpose
A certification or licensing exam exists to separate people who meet a minimum standard from people who do not. The professional standards for testing treat everything that follows from that purpose, from writing items to reporting scores and being fair to different test takers, as things a test owner must be able to defend (AERA, APA and NCME, 2014). The standards are open access, and reading their table of contents is a fast way to see how much design sits behind a single pass or fail.
This matters to you for one reason. If you believe the exam is a trick, you study tricks. If you understand it as a sample of tasks scored against a bar, you study the tasks. The research in the next chapters favors the second approach, and so does the way the exams are built.
How a multiple-choice item is written
Item writers follow guidelines. One review built a taxonomy of 31 multiple-choice item-writing guidelines by comparing 27 textbooks on educational testing with 27 research studies and reviews (Haladyna et al., 2002). The guidelines cover the stem, the correct answer and the wrong options, which testing people call distractors. Typical guidelines ask for plausible options of similar length that do not give each other away.
A meta-analysis of 80 years of research found that three options are enough: tests can carry more three-option items in the same time without hurting the quality of the scores (Rodriguez, 2005). The practical reading is that a four-option item often has one option that few people pick. When you eliminate options later in this guide, expect that one or two will be easy to remove and the last two will do the real work.
Flawed items exist, and they do not help as much as you think
Writers are human, and flawed items slip through. A study of ten high-stakes nursing test papers found that 47.3 percent of items contained at least one flaw. The flawed items did not disadvantage borderline students, and high-achieving students were more likely to be penalised by them (Tarrant and Ware, 2008). That finding cuts against the idea that test-wise tricks reliably rescue weak knowledge. It also warns that a flawed item can punish a strong reader who sees a second defensible answer.
Answer position is another popular trick. Researchers found that both test makers and test takers prefer to hide or seek the correct answer in middle positions, but that operational tests balance their answer keys on purpose, so the bias almost disappears in real keys (Attali and Bar-Hillel, 2003). Choosing a favorite letter when you guess gives you no systematic edge on a balanced key.
Raw scores, scaled scores and why 700 is not 70 percent
A raw score is the number of questions you answered correctly. Raw scores are hard to compare across forms, because one form can be harder than another. Testing organizations solve this by reporting scaled scores. A scaled score is adjusted to compensate for differences in the difficulty of the questions, so the easier the questions, the more of them you must answer correctly to reach a given scaled score (Livingston, 2004). Each form has its own raw-to-scale conversion.
Two things follow. First, a printed pass mark such as 700 on a 1,000-point scale is a position on a scale, not 70 percent of the questions. Second, your friend's report that they needed to get "only" a certain count right tells you little about your form. If you want a practice benchmark, treat your percent correct in practice as a study signal only, and treat the real conversion as unknown unless the owner publishes it.
Unscored items are normal
Some exams include questions that do not count toward your score because the owner is testing them for future forms. The BCBA examination, for example, includes 175 scored questions and 10 unscored pilot questions (BACB, BCBA test content outline). You cannot tell which are which, so treat every question as if it counts. The useful side effect is calming: a question that seems strange or unfair may be one of the unscored ones, and you should not let it eat your time.
Where the pass mark comes from
The pass mark is a cut score, and cut scores come from standard setting. A published instructional module on the topic describes standard setting as the set of procedures used to derive performance levels for achievement tests in education, licensure and certification, and explains why such a step is needed at all (Cizek et al., 2004). Many methods ask panels of experts to judge how a minimally competent person would perform, and the cut score is derived from those judgments.
For you, the point is that the bar is a judgment about minimum competence in a field, and it is not set to pass a fixed share of candidates. Do not read any pass rate you see online as a statement of difficulty without knowing who took the test. Our pages do not quote pass rates for that reason.
Adaptive tests: why every question feels hard
Some exams are computerized adaptive tests. The best-documented public example is the nursing licensure exam, whose owner explains how it works. Every time a candidate answers an item, the computer re-estimates ability from all the earlier answers and the difficulty of those items, then chooses the next item to be neither too easy nor too hard. The owner states that candidates should find each item challenging, because each item is targeted to their ability (NCSBN, computerized adaptive testing).
The same page lists three rules for deciding pass or fail. The most common is a 95 percent confidence interval rule: the computer stops when it is 95 percent certain that the candidate's ability is clearly above or clearly below the passing standard. Two other rules cover candidates near the standard who reach the maximum length, and candidates who run out of time (NCSBN, computerized adaptive testing). Research on building adaptive tests covers item selection algorithms, item exposure controls and the termination criteria that produce those stopping rules (Thompson and Weiss, 2011).
Put it to work
Pick one of the exams you are preparing for and find its candidate handbook. Write down four facts: the number of scored questions, whether any are unscored, the time limit and the form of the score report. You will need them for the plan in chapter 8 and the timing in chapter 10. Then try a short set on one of the live practice exams, such as ISC2 CC or Florida life and health insurance, and notice which options you eliminate first.
Practice what you just read
Six exams have a live practice set, free, with right or wrong on every answer and your score: ISC2 Certified in Cybersecurity (CC) free practice questions, SAFE MLO Test free practice questions, Life and health insurance (Florida) free practice questions, Life and health insurance (Texas) free practice questions, Property and casualty insurance (Texas) free practice questions, Life insurance agent (California) free practice questions, Real estate salesperson (California) free practice questions, and Property and casualty insurance (California) free practice questions. After you have practiced, the full analysis for an exam shows the right answer, explanation and source for every question, your readiness by domain, confidence calibration, pace, your missed list and a study plan. The paid part: DecipherU Pro $59.99 once. We are not affiliated with any exam owner named here.
Get one question a day, or a note when your exam changes
Pick your exam. Each form sends only a confirmation email first and sends nothing else until you confirm. You can stop with one click.
ISC2 Certified in Cybersecurity (CC)
SAFE MLO Test
Life and health insurance (Florida)
Life and health insurance (Texas)
Property and casualty insurance (Texas)
Life insurance agent (California)
Real estate salesperson (California)
Property and casualty insurance (California)
Cite this page
DecipherU Editorial. (2026, October 18). How exams are built and scored. DecipherU. https://decipheru.com/exam-skills/how-exams-are-built-and-scored
Stable address: decipheru.com/exam-skills/how-exams-are-built-and-scored. First published . Sources last checked . If a later check changes a statement, the date above changes with it and the change is listed in the corrections log.
Sources
- American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. AERA. www.testingstandards.net/open-access-files.html Read , recheck by .
- Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-333. doi.org/10.1207/S15324818AME1503_5 Read , recheck by .
- Rodriguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, 24(2), 3-13. doi.org/10.1111/j.1745-3992.2005.00006.x Read , recheck by .
- Tarrant, M., & Ware, J. (2008). Impact of item-writing flaws in multiple-choice questions on student achievement in high-stakes nursing assessments. Medical Education, 42(2), 198-206. doi.org/10.1111/j.1365-2923.2007.02957.x Read , recheck by .
- Attali, Y., & Bar-Hillel, M. (2003). Guess where: The position of correct answers in multiple-choice test items as a psychometric variable. Journal of Educational Measurement, 40(2), 109-128. doi.org/10.1111/j.1745-3984.2003.tb01099.x Read , recheck by .
- Livingston, S. A. (2004). Equating test scores (without IRT). Educational Testing Service. www.ets.org/Media/Research/pdf/LIVINGSTON.pdf Read , recheck by .
- Behavior Analyst Certification Board. (2024). BCBA test content outline (6th ed.). www.bacb.com/wp-content/uploads/2022/01/BCBA-6th-Edition-Test-Content-Outline-240903-a.pdf Read , recheck by .
- Cizek, G. J., Bunch, M. B., & Koons, H. (2004). Setting performance standards: Contemporary methods. Educational Measurement: Issues and Practice, 23(4), 31. doi.org/10.1111/j.1745-3992.2004.tb00166.x Read , recheck by .
- National Council of State Boards of Nursing. (n.d.). Computerized adaptive testing (CAT). NCLEX.com. www.nclex.com/computerized-adaptive-testing.page Read , recheck by .
- Thompson, N. A., & Weiss, D. (2011). A framework for the development of computerized adaptive tests. Practical Assessment, Research, and Evaluation, 16(1), Article 1. doi.org/10.7275/wqzt-9427 Read , recheck by .