VALIDITY & RELIABILITY in Assessment: Why It Matters

SHARING IS CARING

Validity and reliability in assessments are the two pivotal indicators to tell you if an assessment’s results can be trusted.  If both of these indicators are present, you can use the assessment for something as important as hiring.  And the PXT Select shines in both validity and reliability.

All assessments are not created equal.  Creating good tests is not a job for amateurs, but trained experts.  Creating assessments is a very sophisticated science involving years of time and frequently lots of money.  There are two distinct clues to knowing whether you can trust the test or not:  validity and reliability.

Validity: Does the Assessment Measure What It is Supposed to Measure?

In assessments, validity is a term that means the instrument measures what it is supposed to measure. 

A high validity score (on a scale of 10) says the assessment does, in fact, measure what it said it does.  The lower the score, the less valid it is in measuring what it says it measures.  There is no such thing as a 10 validity score since humans are not dipsticks. (Well, okay, usually not, except for your X.)

If someone went under your car hood with a tire gauge and measured your oil, the technician is not measuring what is supposed to be measured.

Reliability:  Is the Assessment Consistent Over Time?  

If someone is picking leaves off of a daisy and when she picks one leaf off, she says, “I love you.”  Next leaf, “I love you not.”  Next leaf, “I love you.” And her final answer to her love dilemma is simply which alternative she lands on with the last leaf, I would be concerned about both her and her methods of deciding.

What if you had a test that did that?  One leaf, “You are enterprising.”  Next leaf, “You are not enterprising.”  Next leaf, “Yes, you are.”  Would you trust the results of leaf picking?  Hopefully not.  Yet, people trust tests that are unreliable.

If a test is unreliable, that means it is not consistent over time.

It lacks precision.  You don’t want a Fit test that will tell you on Monday that you need to be a drummer, but if you take it on Wednesday, it will tell you to be a professional hang glider.  And on Friday, it will tell you to play the drums while you are hang-gliding.  Some information in life you just cannot trust.

Typically, two factors make good reliability scores:  test length and item quality. 

Item quality is what consumes hours and hours of researchers’ time; but for your purposes, just know that if a test takes five minutes to complete, while interesting, it is not top-of-the-line reliable.

Why Your Bathroom Scale Could Be Your Friend

Well, actually I should say, “How a Bathroom Scale Can Explain Results to Us.” So, here goes…  Good tests have both validity and reliability.  An example that is often used is a bathroom scale.  (I know.  I know.  I hate to even bring it up.)  If someone weighs 200 pounds and steps on the scale 10 consecutive times (why would you want to do that I don’t know, but let’s just say) and gets readings of 180, 55, 240, and 75, the scale is not reliable.  You would feel like weighing every day is a lottery experience.

If the scale consistently reads 165 every time (and you really weigh 200 pounds), then the scale is reliable, but not valid. (Though in this particular instance, if the scale always measured less, I would keep the scale just for the happiness it brings.)  Okay, back to tests instead of scales.

Reliability is analogous to precision, while validity is analogous to accuracy.

Find a Trusted Resource

Because these days, there are lots and lots of tests on the internet, and if you are not an expert investigator, you can be led astray.  I would never let that happen to you. When it comes to hiring, there must be strong reliability and validity in order to hold up under any legal scrutiny.

It still astounds me after over twenty years of helping companies find people who fit best, there are business owners out there who hire without using objective data.  The rational, objective, logical mind is not the end-all to thinking, but it certainly has its place.  Forensic science is compelling because of its accuracy and objectivity, and workplace science is equally as compelling because of its ability to accurately and objectively get your business to Fit.

 

 

 

“Data!data!data!” he cried impatiently. “I can’t make bricks without clay.”1

Sherlock Holmes (Arthur Conan Doyle, The Adventure Of The Copper Beeches)

Use the Logical Mind and Objective Science

The logical, rational, conscious mind can aid you in your hiring, just as the intuitive, sensory, unconscious mind can help as well.  They are great partners.  Why make such a big decision with one part of your brain?

Use reliable and valid assessments such as the Profiles PXT Select Assessment as THE tool to help you find the right hiring Fit.  As a rule of thumb, the PXT Select should be one-third of your hiring decision.  The other two-thirds should be the interview and the resume or curriculum vitae.  Use every possible source of information at your disposal.

As a closing thought, there is one more pitfall to avoid.  It’s called face validity.

Face Validity Means “On its face, it makes sense.”

There are a few assessments out there (no names) that have high Face Validity, but not Real Validity or Reliability.  They can never be used for hiring.

A good example of a paradigm/assessment that had “Face Validity” was learning styles.  It made so much sense and was used for so long in classrooms everywhere, but alas when it was subjected to rigorous objective testing, there was no research proof for learning styles. (In course creation, which we also always base on research and science, we would never lead astray with faulty research in that domain either).

Look for the science.  A good assessment, like the Profiles PXT Select, will always have solid validity and reliability.

References:

1 Konnikova, M. (2011). Lessons from Sherlock Holmes: Don’t Try to Make Bricks without Clay. Scientific American. Retrieved from https://bit.ly/3SPJJ15

SHARING IS CARING