From the Item Bank

The Professional Testing Blog

What is a Psychometrician?

A psychometrician is a professional who practices the science of measurement, or psychometrics.  Psychometrics is the field of science associated with the development of instruments (such as examinations) that measure knowledge, skills and attributes (KSAs).  A psychometrician develops assessments such as examinations for educational, employment or professional credentialing purposes.  From the Greek root word psych which means mind and metr which means measure, psychometrician is just what it sounds like — a person who measures what is in someone’s mind.

Certification and licensing bodies that develop assessments for professional credentials employ a psychometrician (or outsource to a psychometrician) to make sure that they are developing examinations that are fair and valid.  After all, it is important to be sure that any certification or licensing body using an examination to verify competence is certain that the examination really is measuring knowledge and skills required for competent performance.  Consider a multiple choice test question that could have a hidden bias in the way the question is worded thus affecting a candidate’s ability to accurately respond to the question.  Or an essay question that is in fact assessing reading or writing skills instead of knowledge of the concept the essay question is asking.  These are the types of careful considerations that psychometricians evaluate to ensure a quality, fair, valid and reliable examination process.

Someone with the title of psychometrician might work in any area of testing.  For example, a psychometrician might spend most of his or her time overseeing an examination program.  This could include facilitating a job/task analysis, managing the item writing process, and conducting standard setting (or passing score) studies.  A psychometrician might also spend considerable time conducting data analyses.  He or she might score exams, run item analyses, produce score reports, and conduct reliability and validity studies.

Typically, a psychometrician has an advanced graduate degree from a university, usually either from an Educational Measurement program, from an Industrial Organization program or from a Quantitative Psychology program.  Master’s and PhD programs in measurement can be found in colleges of education, psychology and occasionally statistics. Holding advanced studies from a university helps psychometricians understand the complexities associated with developing good examinations.  However, there is no certification or license that a person has to complete or hold in order to become a psychometrician.  A person working in any area of testing could simply choose to call himself or herself a psychometrician.

Psychometricians often focus on a few areas of expertise.  Very few psychometricians have expertise in all of the areas of specialization associated with measurement.  Some psychometricians spend a lot of time working with subject matter experts (SMEs) facilitating meetings and serving as project directors and liaisons with certification and licensing bodies.  Other psychometricians choose to stay in the background working with the data or the test questions and limiting their interaction with certification and licensing bodies.  Some psychometricians are experts at defining the knowledge and skills required for competence (such as in a job/task analysis) while others are experts at establishing the passing standard.  Some psychometricians are experts at reviewing statistics for purposes of cheat detection, while others might be experts at the statistics associated with developing adaptive examinations.  When determining the psychometric expertise required to produce quality examinations, certification and licensing bodies should consider these expertise variations, among others.

It would be difficult to find one psychometrician with expertise in all areas associated with assessment and measurement.  Therefore many certification and licensing bodies outsource their measurement to a psychometric firm.  Certification programs may benefit from working with a testing company with a staff of psychometricians in a number of different areas of expertise.  This will allow for the adjustment of psychometric expertise as the examination and the program evolves.  Changes to the examination life-cycle will affect many other decisions of a psychometric nature and certification and licensure programs committed to producing quality assessments will benefit from finding the right psychometric talent to partner with throughout the life of their examinations.

Categorized in: Industry News