New York Magazine has published a devastating critique of the widely cited implicit association test (IAT). The creators of the IAT claim that it can ferret unconscious biases. From the point of view of psychometrics, the first important question we must ask about any instrument is its reliability, that is, does the measure yield consistent results if the measured phenomenon has not changed. It is a basic law of measurement that an unreliable measure can not be valid.
Here is what the article says about reliability:
What constitutes an acceptable level of test-retest reliability? It depends a lot on context, but, generally speaking, researchers are comfortable if a given instrument hits r = .8 or so. The IAT’s architects have reported that overall, when you lump together the IAT’s many different varieties, from race to disability to gender, it has a test-retest reliability of about r = .55. By the normal standards of psychology, this puts these IATs well below the threshold of being useful in most practical, real-world settings.
peakmemory 
to the extent all these “tests” are based on subjective self reports, they are all inherently flawed
Quick follow up: this post rekindled (in me) the old problem of making “interpersonal comparisons of utility.” More here: https://priorprobability.com/2017/07/08/true-or-false-can-we-make-interpersonal-comparisons-of-utility/
Thanks for the comment and the link!
I don’t completely dismiss self report data, if it is reliable and if it has predictive validity, that is, if responses can predict some real world outcome that we care about. For example, there are a number of clinical self report instruments in psychology that are helpful in diagnosing conditions such as depression.
It occurs to me that utility is a much more slippery concept than depression. Depression, on the other hand, has a number fairly well defined behavioral correlates.
Yes, I have always found the concept of “utility” in economics to be very problematic, as you cannot measure what you cannot define (to paraphrase Lord Kelvin!)