First in an occasional series. The surname of the writer may be familiar to you.

I was told off last week at a course that I had organised that as the “learning outcomes” were shown on neither the feedback form nor the attendance certificate then the Genera Dental Council might look unfavourably on the validity of the course content. My thoughts are that people who devise these rules, those who enforce the rules and particularly those who take any notice of them really belong in the Golgafrinchan middle classes. Look out, because the space goat will get you and no amount of learning outcomes will save you.

From the med.ed.ed blog 

Original here: http://mededed.me/2014/02/05/jorge-luis-borges-and-learning-outcomes/?utm_source=rss&utm_medium=rss&utm_campaign=jorge-luis-borges-and-learning-outcomes

Jorge Luis Borges and Learning outcomes

by reestheskin on February 5, 2014

Whenever I hear or read the phrase ‘learning outcomes’ I think of the story ‘On Exactitude in Science’ by Jorge Luis Borges. It is a short story, very short in fact, coming in at fewer than 150 words. So the danger in attempting to describe what it is about, is that you use more words than Borges himself. The hazards of summary or précis is of course part of its subject. So here it is:

…In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast Map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography.

—Suarez Miranda,Viajes de varones prudentes, Libro IV,Cap. XLV, Lerida, 1658

By learning outcomes I simply mean stating what you expect students to know or be able to do. I suspect there are lots of exegeses in the academic literature, but I assume this definition will suffice. In the context of medicine it seems especially important to be able to tell students what you expect them to know (for the record I do not believe that this is a sensible strategy in all contexts, just most).

You can signal to students what you want them to know in various ways. One way of course is feedback on any written work or how they are performing clinically. This approach will not work well for medicine because written work is the exception and, because medical courses are highly fragmented, there is little long-term staff-student interaction. Assessing clinical skills in everyday practice is problematic for similar reasons, and also because most teaching of medical students in the UK is by staff who are not directly employed to teach students. (Note: this does not mean they many do not teach very well, merely that I do not think anybody would think the current design is optimal nor, if they were starting afresh, design it this way). If we consider postgraduate training of residents then the design is a much better fit. Most consultants have a good idea of what you are trying to get your registrars to know and do. For medical students, this is frequently not the case, simply because of the complexity and fragmentation of the medical course. It is of course bizarre to have to explain this. Imagine a school teacher who didn’t know what their pupils were required to know, in order to progress.

Another way of mapping out learning outcomes is by questions, ideally ones that the students can mark themselves such as MCQs. If you can provide a link to summative marks, frequent formative tests allow students to see what it is that they will be tested on. This is the way many postgraduates revise for postgraduate professional exams. It will work well if there are a large number of questions and there is tight linkage of the formative to the summative assessments. The difficulty in fully embracing this approach is that I think you have to have a very large numbers of formative questions, and in practice, devising such questions is expensive —unless you have the scale you see in some postgraduate exams. Speaking personally, I find coming up with good questions, standard setting them, very, very time consuming. Scale would help, but single institutions do not achieve this scale, and cooperation between institutions is lacking— not least because of the lack of agreement over what knowledge is required. And of course, this agreement, is what learning outcomes are supposed to be about.

Most of the learning outcomes I have seen appear to me to be far from ideal. What do I mean? Well, I see statements like ‘the student will know the signs of psoriasis; the student will understand the basis of commonly used treatments…..’ and so on. Well, such statements are not entirely devoid of merit, but they are not very smart either. They do not operationalise. What signs do you mean? What treatments, should they know about, and what is it about these treatments they are expected to know? In a similar vein, statements that students should know how to diagnose common skin cancers, conveys little. What sort of skin cancers, and which particular clinical cases do you think the they should get right? A total novice may diagnose some melanomas, but performance is a continuous trait: where are you expecting the student to be on this scale? And how have you operationally defined this position? So, you can see my problem with this whole approach, and also why I view Borges essay as relevant.

In some domains of knowledge you can indeed summarise succinctly a capability that might present in an infinite number of ways. So, we can say that we wish students to be able to carry out certain procedures (e.g. addition, subtraction) on say (real) numbers. We can test this ability easily because we can come up with exam examples easily. Similarly, we might say we want students to be able to solve certain types of quadratic equation. What we require can be stated in few words, but the ‘real world’ of possible instances, is almost infinite.

For many topics in medicine this approach is impossible. I might want students to know about psoriasis, but exactly what is it about psoriasis I want them to know, and how can I codify this required knowledge? I could say I want them to know about the key treatments but, in turn, I will have to enumerate what these treatment are and which are not they key treatments, and then in turn enumerate what it is that I want them to know. My map scale approaches reality. Eventually, my learning outcomes will be congruent with all the information that I will need to provide to explain these learning outcomes. There are some exceptions: you might say you want students to be able to list the names of the various layers in the epidermis, for example. So some data compression might be possible here and there, but in many instances I suspect not a lot.

In practice, this sort of issue receives little attention. Students may get told to read a book, without noting that even ‘basic’ texts may differ wildly in what information they provide. Often the advice is given without detailed annotation of what bits of a book to concentrate on. Often those doing the teaching have not even read the book. I can imagine a mythical past time when large chunks of undergraduate medical teaching worked as an apprenticeship. Here continued interaction, and clinical presence allowed a very good idea of what was ‘essential’ knowledge. But I think this is now rare.
There is a final twist that bugs me and I haven’t resolved. Much of knowing what is wrong with a patient rests on simultaneously knowing what is not wrong with a patient. Diagnosis is in large part a categorisation task, and it involves processes analogous (not identical) to the statistician’s likelihood ratio. One simplification that you might invoke to solve the conundrum that Borges writes about, is to say that students only need to know and understand the 10 key presentations (within a particular area). I worry this may be an intellectual sleight of hand, as students can only learn what things are, by simultaneously knowing what they are not.