Wednesday, September 10, 2014

Validation

You can't be sure. There is no such thing as a "survival level of proficiency." The world has a 100% death rate and no matter how skilled, equipped, or physically gifted you are, there is stuff out there that can splat you like a bug on a windshield. That's just the way it is. The one thing that's a safe bet is that if you are sure your stuff is adequate, you are already setting yourself up for failure.

No matter how tested something is or under what conditions it has been tested, all you know is that you haven't found the failure point yet. But the failure point is out there. So is your stuff valid? That depends how far you have tested your stuff. There is a point where it will cease to work. And the uncertainty increases when it is not tested. When there is no way to validate a thing, humans seek validation instead.

You can't be 100% sure of very much. 1+1=2 with high reliability when applied to rocks. It's less reliable when applied to rabbits. When you can't be sure (validity) people want to feel sure (validation).

How does one go about validation? They like be told by other people that they are good. There are a lot of rituals and trappings to it, but that's the essence. A black belt. Certificates and trophies. Creating "Councils of Masters" who cross-certify each other as "Masters." In the RBSD world, you have instructors who are combing academic abstracts looking for studies that appear to justify their own beliefs or discredit a competitor's. Everybody wants a guy in a white coat with a PhD after his name to validate their approach. The academic researcher takes the place of the shaman is this quest in this culture.

And that last, science, isn't bad. If you are scientifically literate (understand experimental design, the scientific method and the basics of statistical analysis as a start) and read the actual article, not just the abstract. And don't cherry-pick too hard.

But the rest aren't bad, either. Sort of. I want validation too. My validation comes from the respect of people that I respect. Hmmmm. Sort of. I respect almost everyone as a matter of courtesy. But when I look at my closest friends, I'm a little humbled to be accepted in their company. But it can be a fine line between a group of operators and former operators telling war stories and and a cross-certifying Master's Council. I'm fairly positive that each of those "masters" convince themselves that the others on the council are extraordinary and being allowed in is a compliment (even if one Hall of Fame award was offered to every member of a certain martial arts forum one year. Sigh.)

There are certificates that mean a lot to me because of who they came from and how they were earned. And I know there are, or used to be, certificates that came in a sheaf with a box of DVDs all pre-signed by the "master" so that you could fill them out and show potential students your hundreds of certifications.

And trophies-- you win an olympic judo medal or a UFC title and you are one tough son of a bitch, dedicated and skilled. Or you can just go to an event that has three times as many categories as competitors and come home with a pocketful of gold medals from events where you had no opposition. The good and worthless trophies look just the same on the wall.

It can look like the goal is to be strong enough not to need outside validation, to be so sure that you don't need other people telling you how good you are. But that doesn't work either, because some of the worst instructors I have seen had a profoundly over-developed ego. Someone who truly feels superior usually sucks (Dunning-Kruger) and are most likely to reject outside opinions yet most likely to need them.

Sometimes I  think about offering a certification program in thinking for yourself. The catch being that if you want a certificate in autonomy from someone else, you don't get it. You don't get the certificate or the concept.


11 comments:

Kamil said...

That catch of yours at the end is almost precisely the same as the original Catch-22 in Joe Heller's book of the same name. "You can be discharged from the army only by reasons of insanity. Wanting to be discharged from the army is irrefutable proof of your sanity."

Tia said...

Getting people to think for themselves: Add to your end of seminar ritual. Continue to ask for one thing participants took away from the seminar but also have them tell you one thing they think you're wrong about. Or something you answered incompletely or taught with too much authority and not enough knowledge.
Gets them to think critically and gives you new questions to think about and equations to solve.

Unknown said...

With the Dunning-Kruger effect, didn't the majority rate themselves at about 70%?

Unknown said...

Rory- your blog is very good for helping me clarify my thinking (a little)and forgive me if some of this just paraphrases what you have said already.
Validation-
1. The college in the summer asked for an instructor's certificate, an instructors insurance certificate and a first aid certificate before they would let the students book me for a day's course.They were satisfied as long as I had the correct piece of paper.
2.The manager at the students union wanted to know who I had trained with, and knew personally, or knew of, the people I mentioned.
3. The women in the classes usually want to know if I have survived a violent attack. No, because I have never been put in that situation. (closest things, a few dodgy experiences while travelling on my own, one of which, just outside Teheran 40 years ago, I think the 'attackers' were more nervous than we were!)
Validity- I feel a lot of responsibility for anything I try to teach, but I haven't tried it in a 'real' situation. So I have to try to learn from others who have been in these situations themselves and have found things that do work, and then try to extract from these things that I think I could make work myself or that someone could make work after a few evening classes with no prior training or experience. Classes often end up as discussion with the women adding their own experiences and how they have dealt with them (or sometimes not) ,plus some principles (but rarely techniques) ukemi - or rather falling over without hurting themselves, from the dojo.

Unknown said...

oh- and the students union suggested we give the women 'certificates' for completing a course lasting a few evenings.(so they had something to take away with them afterwards!)
uhh- No!

Quint said...

I'm now going to try this out. I will say in my freshman study skills course that any student who shows me they can think for themselves will get an A.

The only way that students can get an A in this scheme is by directly challenging me on the logic that they have to show me they can think for themselves.

I realize that this will only work for a maximum of 2 years before word gets out and I have every student challenging my authority at every turn (but that might not be bad in and of itself).

pax said...

Re your last para - laughed. But IMO, certificates aren't really for the student (or shouldn't be). They're mostly documentation for other people, primarily for the justice system should it come to that.

Kai Jones said...

Certification is what you show somebody else, not what you need for yourself. I have used you as my certification repeatedly, even while sitting next to you and talking to somebody else.

Delta Juliet Papa said...

Re: Validation--To paraphrase Groucho Marx, "I wouldn't want to be a member of a club that'd want me as a member."

Travis said...

"You can't be sure. "

Are you positive?

;)

Unknown said...

Thank you for the validation of not needing to buy your book. :) Good reasoning. I enjoyed your article.