Author: Nina O'Reilly

Davies, A. (2012) ‘Learning outcomes and assessment in art and design. What’s the recurring problem?’, Networks ADM-HEA Magazine, Issue 18, July 2012. Available at: http://arts.brighton.ac.uk/projects/networks/issue-18-july-2012/learning-outcomes-and-assessment-criteria-in-art-and-design.-whats-the-recurring-problem (Accessed: 20 January 2023).

Abstract

This paper is a critical reflection on the development of learning outcomes in art and design. It builds on a Learning and Teaching Support Network (LTSN) – funded project from 2003 that sought to provide support for colleagues seeking to make sense of the then new quality agenda. It seeks to identify recurrent issues and make a number of recommendations for further development.

Reflections

In my recent blog post “Out of the humanist matrix” I read and reflected on Eleanor Dare’s article which critiques humanist pedagogical approaches, particularly within the context of art and design education, and imagines learning taxonomies beyond Bloom. It chimed with my own experience of being compelled, within various institutions, to adopt these taxonomies within my learning design practice. This article certainly had my synapses firing up so I was really pleased to read more on this topic in Allan Davies’ article Learning outcomes and assessment in art and design. What’s the recurring problem?

There were certainly frameworks and bodies that were newish to me – I know the Quality Assurance Agency (QAA) but Biggs’s SOLO taxonomy (2003) was new to me. Bloom’s taxonomy popped again, with the reiteration of it not wholly relevant within an arts and design context. Indeed, the verbs seem to more neatly correspond to the practice of empirical assessment or essays, where knowledge can be said to be acquired or not. But the taxonomy cannot meaningful capture some of the processes that happen within creative education, as Davies notes:

I would argue that the insistence that learning outcomes should be sufficiently clear ‘to be measurable’ has not helped those subject areas, such as the creative arts, in which articulating outcomes that involve the development of intuition, inventiveness, imagination, visualisation, risk-taking, etc, is challenging. In terms of meaningfulness, they equate to the notion of ‘understanding’, a cognitive term which is regarded as too complex and which should be substituted by other, more measurable, terms such as, ‘explain’, ‘analyse’, etc. Another drawback in the use for these terms, acknowledged by Biggs (2003), is that they are regarded as ‘divergent’ and as such do not invite one appropriate answer but a range of possibilities.

Davies (2012)

Davies illustrates this by the skill of visualising which an arts and design student will develop the ability for over time. It is an ability that resists capturing within a neat taxonomy or locating to a specific assessable event. And yet if the learning happens without a well written learning outcome, can it be said to have have happened at all?

Davies highlights how arts educators and students often eschew the rigidity of overly learning outcomes prescriptive learning outcomes and this does not negatively impact the learning experience. The learning goals can be successfully communicated to students in other ways and students are often encouraged to create their own understandings of what these outcomes means for them.

Our obsession with establishing the accuracy/clarity of learning outcomes in the belief that this an essential prerequisite for quality learning to take place is undermined by those courses in which the written learning outcomes are largely unclear but the students are performing well. A lack of clarity in the learning outcomes, it seems, does not mean that students are not clear about what they have to do. Indeed, learning outcomes, ambiguous or otherwise, appear to be no substitute for established learner support systems and other frameworks that help students understand what they have to do in order to successfully complete a programme of work. Briefs and briefings are familiar in art and design along with tutorials, interim crits and feedback forums. It is during these supportive scenarios that art and design students formulate their intentions and actions and come to understand what ‘imagination’, ‘creativity’, ‘risk-taking’, etc, (the very terms regarded as potentially ambiguous) actually mean for them.

Davies (2012)

Davies suggests that “rather than measurability, the focus should be on meaningfulness.” In my own context this chimes with the risk of having an overly prescriptive approach to learning design. It feels that rather than saying I exclusively use ADDIE or Dick and Carey etc etc to “do” learning design, a thoughtful, mindful and flexible approach is better. In a way this seems obvious, but it is easy to fall into repetitive and cyclical patterns within practice, particularly if our practice is aligned with term times and the rolling of one academic year into another. What does a meaningful approach mean in arts and design education in a practical sense? As we don’t want an overly prescriptive learning outcomes approach that limits learning but we do want to map the context of the learning in some way. Davies says:

So, I suggest, in art and design whilst it is important that students know what they have to do on any course of study, it is not necessarily through published learning outcomes. Learning outcomes might be seen as necessary for administrative purposes but they are not sufficient in helping students develop an idea of what they will be learning and how they will go about it. Indeed, in a highly supportive context, learning outcomes might be so generalised as to only define the landscape and the boundaries of their intended learning. The knowing of what to do becomes developmental and personalised.

Davies (2012)

In my own practice I feel having LOs does help to focus the learning and ensure that the topics, materials, and assessments don’t go completely off piste. But I feel like Davies’ advice that they support rather than hamper what the learning might be, especially for subject areas where critical thinking and creativity are important. Davies speaks about the difficulty of aligning LOs with assessment criteria. It seems that UAL has opted to have a university wide assessment criteria (rather than a subject level one), perhaps amongst the factors influencing this is the desire to avoid being overly specific. I suppose the idea is that the LOs are specific on the subject level but they line align with a wider & broader ethos of arts and design education/assessment?

UAL’s Course Designer toolkit

Then onto UAL’s course designer toolkit, which includes a guide for writing LOs and embraces the outcomes based approach to learning. I really am ignorant to how much choice any given HE institute has with this, I imagine they are legally or fiscally bound to comply with the QAA, which would explain why LOs feature here and why they must align with QAA subject benchmark statement.

In terms of structuring the LOs, they must begin with “an action verb to describe the behaviour (what the student will do) which demonstrates the student’s learning”. So it’s Bloomian in a general sense but doesn’t make specific reference to that taxonomy. This gives a bit of freedom and perhaps bypasses some of the critique Davies had with regard to Bloom’s taxonomy not affording enough options for arts and design subjects. I note that the guidance includes examples of poorly written LOs, which is helpful, but not well written ones…. I wonder why this is, perhaps for want of not being overly prescriptive about what is right? I would have liked to see a good example though so as to see if they are general enough to allow the play and experimentation of learning in arts & design. And to see if in practice it’s possible to write an LO that is both as specific and as broad as it needs to be.

Then aligning LOs with UAL assessment criteria – Enquiry, Knowledge, Process, Communication, Realisation – is a helpful way of bringing a creatively focused pedagogy into practice. Each of these concepts are tangible and important in arts education so ensuring that the LOs map to these

Group session 27th January 23

Our cohort met for the first time in person at LCC on the 27th January. In groups of three we examined an artefact from our teaching practice and considered its aims, how it is used, what is assessed and how it is assessed. To complete this activity, Tonia, Irti and I looked at the unit brief for one of Irti’s modules at the Creative Coding Institute. We talked about the difficulty of assessing the module because, on the one hand students are intended to learn technical skills from the module, but on the other more profound level the aim is for them to understand how they can use within the context of their creative practice. This certainly chimed with the points Davies makes in the article above about creating effective learning outcomes for art and design. Irti spoke about how the module formerly had tests, making it similar to common approaches to learning computer science, but how ultimately the assessment has changed to become more project based.

We were then asked to create a poster to explain how we might redesign the artefact. We spoke about how unit briefs are often long documents that students don’t often read although they might benefit a lot of they did, as they contain key information about the course structure, learning outcomes and assessment. In the context of the creative computing frameworks module, we considered whether there is further risk of this, as the technical terminology may further dissuade students from reading. An additional consideration is that lengthy course documents are not an inclusive way of presenting this information for those with dyslexia.

In our poster, we therefore wondered if there was a way of visually presenting the same information so that it is more accessible to students. We attempted to convey the learning outcomes visually (below) although this was difficult to pull off in the time we had!

I do certainly thing that there is something in this approach though. Even if the information was presented in a flow chart stye with a template that could be re-used across different courses and programmes. It may encourage a greater engagement with this key information from the outset.

TPP’s learning outcomes

• LO1: Interpret theories, policies and pedagogies in the context of your evolving practice. [Knowledge]
• LO2: Critically evaluate your approach to planning, teaching and assessment using self-reflective frameworks and observations/reviews of practice. [Process]
• LO3: Appraise your ongoing personal and professional development. [Realisation] LO4: Articulate your pedagogic ideas, experience and expertise for the benefit of the programme community. [Communication]
• LO4: Articulate your pedagogic ideas, experience and expertise for the benefit of the programme community. [Communication]

In the face to face session we focused on LO1 for the course and how this blog could be assessed to measure our mastery of the LO. In my group of three, we struggled with applying UAL’s level 7 assessment criteria to the blog activity. Initially we thought a D would apply to the bare minimum in terms of submitting 4 X 250 word blog posts that align with the topics on the course. Building on that a C would have a more critical element rather than purely being descriptive. But as we continued to discuss this, it felt as though criticality as well as inclusion of practical/conceptual/technical knowledges was also the minimum requirement. We then couldn’t understand how to practically decide how, once a student has submitted all of this, that someone could be marked as excellent while another might just be very good.

We therefore felt that a pass and a refer system made more sense. And that this allowed for everyone to be excellent, as excellent will be different for each individual person… specifically in the context of this module where we are engaging with theories and reflecting on our practice, this process will be individual and different for every member of the course, so provided everyone meets the criteria they should pass.

Abstract:

This article critiques the ‘humanist’ legacy by questioning the cognitivist and constructivist paradigms which underpin dominant models of adult learning. It asks whether they are suitable for evaluating the way art and design students work with digital technology, questioning humanist and cognitivist models of learning, such as Bloom’s Cognitive Taxonomy (Bloom et al., 1956) and whether it supports curiosity, criticality and imaginative risk. It connects this issue to the problem of ‘normative validity’, which describes how that which is measured is valued – ‘the indicator of quality becomes the definition of quality’ (Biesta, 2013, p.1) – overshadowing more inclusive approaches to learning.

Reflections

I was drawn to this article as Bloom’s taxonomy, as well as constructivist/cognitivist/ connectivist pedagogies, have underpinned the approaches of the publishing and higher education institutions I have worked in. I have carried my concern over the “escalating culture of ‘accountability and audit,’”(p.47) from institution to institution, watching this increasingly manifest itself in how teaching and learning is approached. Dare’s article articulated this bubbling concern of mine to myself. It has made me consider and critique my own assumptions and practice. I now see the extent of the issue as well as possible futures.

As a learning technologist, I have been trained to embed these pedagogical approaches within my practice. While there are some practical takeaways (learners constructing knowledge rather passively receiving it from a “sage on stage”), I have felt troubled about this reductionist approach to learning. It is as though, provided one designs within the lines of humanist pedagogies, all learners and learning can be uniformly contained within the same approach. I do find the learning technology community to be well meaning and genuinely concerned with evolving practice, yet this somewhat mercenary approach pervades.

Humanist pedagogies are problematic because they are predicated on a white male Eurocentric preset for human experience. We need only think of the multitude of ways different groups have continually been dehumanised based on their bodies, beliefs, abilities, backgrounds to see this as real and harmful. Therefore, the idea of humanist education is exclusionary and privileges one kind of human and how that human supposedly learns. As Dare states:

The connections between Papert’s constructivism and Bloom’s Cognitive Taxonomy are created by a set of humanist presumptions. In both models the individual is swept into ‘a concept of rationality which is an ahistorical, universal model leading to a view of learning that fails to deal directly with considerations and questions of […] ideology, culture, power and race-class-genderdifferences’ (Illeris, 2009, p.95). This also raises the question of what happens to students who have little or no familiarity with positivist constructions of knowledge, meaning knowledge empirically verified via sense data, but also that which is filtered through mathematical systems of logic.

p.46

This opened up to me how we could consider decolonising learning technology. As a learning technologist I am at a remove from the curriculum and so have wondered about my role in decolonizing it. Questioning and critiquing the theory that underpins my practice therefore feels like a concrete action I can personally take towards decolonization.

The article questions how “This pedagogic model underpins many learning technologies, including the virtual learning environment (VLE), Moodle, which was created by the technologist Martin Dougiamas (1998; 2000) with a specifically constructivist agenda. Dougiamas is clear in his allegiance to constructivism, and its embedding within Moodle, as evidenced by his many publications, not least of all in his blog post ‘A Journey into Constructivism’ (1998).” (p. 46) This made me reflect on my extensive use of Moodle. While I had considered Moodle’s possible inbuilt bias towards accessibility, I hadn’t considered what ideology might underpin it. As Dare states:

…the models we draw upon when deploying the digital in teaching and evaluation are far from neutral, as the subject of these evaluations –the learner –is an ideological construct that serves an outdated ontology, a humanist agenda, which excludes a more expansive consideration of the world in which we exist.

p.45

Dare also underscores how humanist pedagogical approaches serve the problematic marketisation of education.

•”Are the hard-and-fast metrics which educators and technologists find themselves bombarded with in any way reconcilable with an evaluation of what happens when students engage with creative practice? Practices in which ‘the kinds of transitions we are considering are not linear, not the learning of simple isolated concepts’ but ‘messy, abstract transformations’?”

p.47

This made me think of the UK government’s value system for categorising disciplines and how Sheffield Hallam suspended an English Lit course for being “low value” within this framework (Weale 2022). When education is reduced to its relation to the economy, employment figures, tax payments, aren’t learners dehumanised? It’s as if learning and creativity are ores that can be extracted to power the economy, a row in the government’s balance sheet. 

Theodor Adorno (2011) wrote about the insidious pervasiveness of low culture in capitalist societies. Low culture is cheap to produce but its impact on our souls and being is also inferior. It is simply there to appease and relax us, to make us docile cogs within the capitalist machine. In contrast, high culture challenges us, develops us intellectually and can have an emancipatory effect. Notwithstanding the classist idea of high and low art, I wonder about the fate of art that provokes, shocks and teaches us things, if neoliberalism means that art education must have a commercially viable output.

However, Dare’s article closes with a note of hope for the future:

Posthumanist educational theory proposes that we ‘have never been separate from machines and that notions of “humanness” could not be produced without machines’ –in other words, we ‘have always been technological’ (Snaza et al., 2014, p.44). This idea radicalises Vygotsky’s notion of technology as a ‘mediating object’ (1978). A posthumanpedagogy challenges us to consider what Braidotti calls ‘life beyond the self’, and ‘life beyond the species’, to bridge the ‘nature-culture’ and all other ontological divides (Braidotti, 2013, p.186). According to Braidotti, we need to consider virtual entities, animals, codes, networks and flows of energy (2013, p.190), the complex assemblage of agencies inextricably entangled with our always emergent, dynamic subjectivities.

p.46

This made me think of technicians who have always been at the forefront of emerging technologies in their disciplines. The idea of always being technological is therefore not new. However, embracing virtual entities, animals, flows of energy is so relevant now as we strive to deliver socially, environmentally conscious teaching. For my part it’s about how we do this in a meaningfully blended way.

References

Adorno, T.W. (2010) The Culture Industry: Selected Essays on Mass Culture. Edited by J.M. Bernstein. London, UK: Routledge. 

Dare, E. (2018) ‘Out of the humanist matrix: Learning taxonomies beyond Bloom’, Spark: UAL Creative Teaching and Learning Journal, 3(1), pp. 43-51. Available at: https://sparkjournal.arts.ac.uk/index.php/spark/article/view/79 (Accessed: 06 January 2023).

Weale, S. (2022) Philip Pullman leads outcry after Sheffield Hallam withdraws English Lit Degree, The Guardian. Guardian News and Media. Available at: https://www.theguardian.com/education/2022/jun/27/sheffield-hallam-university-suspends-low-value-english-literature-degree (Accessed March 20, 2023). 

My presentation based on my reading of this article