Category: Action Research Project

Please find the slides for my presentation (on the 29th January 2024) below.

This page is a menu of my ARP blog posts, in case it is difficult to navigate these otherwise.

1. ARP overview: research questions, action research cycle, action plan/schedule
2. ARP Context and rationale
3. Ethical enquiry form
4. Research methods: selection and reflections
5. Data analysis – questionnaire
6. Document analysis: Web Sustainability Guidelines (WSG)
7. Gamifying learning – inspiration from the Carbon Literacy Training
8. Game design, development, and prototype
9. Reflections on my ARP
10. References list
11. Presentation upload

Introduction

One of my ARP’s research questions is “How can knowledge and confidence of this topic be increased to create best practice?”. Over the course of my research – including my document analysis of the web sustainability guidelines, understaning learning technologists’ confidence levels with the topic, attending UAL’s carbon literacy training programme – I have come to learn that institutionally we are at the beginning of understanding and implementin an appraogch to this topic and therefore my intervention needs to effectively convey the foundational aspects of digital sustainability for e-learning in a accessible and entry level way. I have therefore become interested in how a gamified approach is appropriate for this.

There are a number of things that led to my interest in creating a game around sustainable design practice for e-learning. One was that my analysis of the questionnaire of learning technologists was that there was an awareness of general sustainability best practice for being online (ie cloud storage, device management etc) but less about creating the content students learn from sustainability. Most respondents felt unconfident about sharing best practice with others across the university. This led to me to analsye the web sustainability guidelines for simple and actionable tips that could be shared and adopted across the university.

However, in addition to deriving these top ten tips, my document anaylsis of the WSGs also revealed how dense the literature on this topic can be and this presents a difficulty for comprehension and engagement. As a technical document, it provides key information on the topic of digital sustainability, but is not suitable as a standalone learning opportunity, at least for gaining foundational knowledge of the topic and simple actionable steps towards sustainable practice.

Doing UAL’s Carbon Literacy Training

I therefore knew an additional step was required in order to answer the question of how we might develop sustainable digital practice at UAL and I took the opportunity to take UAL’s Carbon Literacy Training course to see if it could help me (and as I myself am by no means an expert on sustainability!)

As well as going through the fundamentals of what is causing the climate crisis as well the UAL context, doing this training allowed me to view the topic from the perspective of a learner. I was therefore able to learn about the topic but also be reflective about my learning experience.

One of the highlights of this programme was the Carboned Out! game presented by two UAL alumni who had developed the game during their studies. Players competed against each other to make the least carbon intensive choices, only learning the true carbon cost of things such as washing the dishes, eating pizza and even getting a mortgage after they made their choice. If players made too many carbon intensive choices they would become “carboned out” and lose the game.

The gamification of this content helped me to retain these facts about carbon cost as it was enjoyable and more engaging than simply reading the same information. I was inspired by this approach, particularly as a way of teaching around a heavy topic.

Gamification theory

To build on what I observed during the Carbon Literacy Training, I closely read “Gamification of adult learning: Gamifying employee training and development” (2019) by Landers et al. I chose this particular reading because as well as being well cited and reatively recent, it is aptly focused on the correct audience for the game I wish to create, ie adults in a professional context rather than children, young people, etc. Additionally to counter the under-researched hype around gamification, the motives of the paper are to “define gamification and provide a comprehensive introduction to it […] explore which theories describe its known and potential effects […] provide extensive practical literature-driven recommendations for those seeking to gamify training themselves.” (2019)

One of my key takeaways from the paper was that learning in a gamified way can be extremely effective in increasing learners’ ability to meet learning outcomes. Additionally the efficiency of gamified elements including assessment, game fiction, immersion, points, badges, leaderboards, progress bars, quests, meaningful stories, and avatars are proven in relation to pyschological theories including self determination theory, test-enhanced learning, expectancy theory, goal-setting theory, and self-determination theory. In addition to how well gamification elements may bear out in relation to these theories, in a professional context the article points out that “it is important to consider a variety of situational moderators of training effectiveness, including climate/culture, supervisor support, and employee buy-in”. Luckily in UAL, climate consciousness is a key cornerstone of our social juctice mission as an institution. Equally my own ARP questionnaire of UAL learning technologists showed that 92% were interested in training and guidance for best practice in reducing emissions through online course build and design.

However, what the article also stresses is that gamification is an instructional design tool rather than a wholesale solution to creating effective learning solutions. If the content is not of quality or in alignment with the intended learning outcomes gamification is not going to make a difference to that. Additionally the article states that “Critically, gamification of training should not be attempted unless there is a specific, identifiable problem with a training as it currently exists.” I have rationalised my reasons for wanting to use gamification in the introduction above. However, I am starting out with this training rather than seeking to improve something pre-existing. Therefore for me, the article’s subsequent point about gradually including gamified approach is important “Gamification of training is a process in which training content and methods are modified using game elements. This process is incremental in nature, whereby the instructional designer modifies pieces of the training bit by bit to improve learning outcomes.” Therefore being thoughtful in my approach, focusing on the content itself rather than all the gamified bells and whistles is a sensible approach for the initial iteration. 

My takeaway is that I will operate within the constraints of whatever game authoring system I will use.Equally while badges/certificates could be possible and are proven to help with learners’ motivation, I think the game is still too early in its development to be connected to a badge although it is an option to consider at a later point. I am interested in using opportunities for learners to individualise their learning, so that tasks and activities correspond with their own context. Additionally, similar to a progress bar or collection of points, I think the a decreasing carbon count as players progress through the game could be effective.

Application to my game prototype

In the blog post Game design, development, and prototype I write about how I developed the game drawing on the inspiration from the students’ Carboned Out game and through my reading of the Landers at al.

References

Barker, G. (2023) UAL Carbon Calculation: Educational Working Group. Available at: https://canvas.arts.ac.uk/News/240267/ual-carbon-calculation-educational-working-group. (Accessed 12 January 2024)

Landers, R.N., Auer, E.M., Helms, A.B., Marin, S. and Armstrong, M.B., (2019) Gamification of adult learning: Gamifying employee training and development. The Cambridge handbook of technology and employee behavior, pp.271-295.

Barker, G. (2023) UAL Carbon Calculation: Educational Working Group. Available at: https://canvas.arts.ac.uk/News/240267/ual-carbon-calculation-educational-working-group. (Accessed 12 January 2024)

Berners-Lee, M (2020). How bad are bananas? : the carbon footprint of everything. London: Profile Books.

Bowen, G.A. (2009) ‘Document analysis as a qualitative research method’, Qualitative Research Journal, 9(2), pp. 27–40. doi:10.3316/qrj0902027. 

Braun, V. and Clarke, V. (no date) Doing reflexive Thematic Analysis. Available at: https://www.thematicanalysis.net/doing-reflexive-ta/ (Accessed: 23 November 2023). 

British Educational Research Association (2018) Ethical Guidelines for Educational Research. 4th ed. Available at: https://www.bera.ac.uk/publication/ethical-guidelines-for-educational-research-2018 (Accessed 25 October 2023)

Dawson, A. and Frick, T. (eds.) (2023) Web Sustainability Guidelines (WSG) 1.0, Web sustainability guidelines (WSG) 1.0. Available at: https://w3c.github.io/sustyweb/ (Accessed: 09 October 2023). 

Daeninck, C., Kioupi, V. and Vercammen, A. (2023) ‘Climate anxiety, coping strategies and planning for the future in environmental degree students in the UK’, Frontiers in Psychology, 14. doi:10.3389/fpsyg.2023.1126031. 

Ecograder.com (2023) Impact report for https://www.arts.ac.uk/. [Webpage] Available at: https://ecograder.com/report/rnbx0SlRapy8Ax39sXAtbVRR (Accessed: 06 October 2023). 

Ecograder.com (2023) Impact report for https://moodle.arts.ac.uk/login/index.php. [Webpage] Available at: https://ecograder.com/report/orCS0jpIHGiJjy8Ddp51yvnw (Accessed: 06 October 2023). 

Ellis, C.S. and Bochner, A.P. (2006) ‘Analyzing analytic autoethnography’, Journal of Contemporary Ethnography, 35(4), pp. 429–449. doi:10.1177/0891241606286979. 

Gibbs G (1988). Learning by Doing: A guide to teaching and learning methods. Oxford Polytechnic: Oxford.

GreenNet (2023) Understanding file sizes. Available at: https://www.greennet.org.uk/support/understanding-file-sizes (Accessed 22 January 2024).

Greenwood, T. (2023) 20 ways to make your website more energy efficient, Wholegrain Digital. Available at: https://www.wholegraindigital.com/blog/website-energy-efficiency/ (Accessed: 09 October 2023). 

Griffiths, S. (2022) Why your internet habits are not as clean as you think, BBC Future. Available at: https://www.bbc.com/future/article/20200305-why-your-internet-habits-are-not-as-clean-as-you-think#:~:text=The%20carbon%20footprint%20of%20our,a%20researcher%20at%20Lancaster%20University. (Accessed: 06 October 2023). 

Heselden, M. (2022) Digital content design strategy for arts.ac.uk 2022-2025. Available at: https://canvas.arts.ac.uk/documents/sppreview/0eb0ce28-2598-47c4-b672-787b3f906f97 (Accessed: 19 October 2023).

Jackson, T. and Hodgkinson, I.R. (2022) What is ‘dark data’ and how is it adding to all of our carbon footprints?, World Economic Forum. Available at: https://www.weforum.org/agenda/2022/10/dark-data-is-killing-the-planet-we-need-digital-decarbonisation/?utm_campaign=social_video_2022 (Accessed: 25 January 2024). 

Jisc (2022) Student digital experience insights survey 2021/22 UK higher education (HE) survey findings. Available at: https://repository.jisc.ac.uk/8850/1/2022-07%20%28iDFltdP024.11%29%20DEI%20HE%20%26%20FE%20Reports%202022%20%28HE%29%20v1-05.pdf (Accessed 25 January 2024).

Joyce, E. (2023) All staff briefings Q&A session | Monday 16 October Available at: https://canvas.arts.ac.uk/documents/sppreview/e2818b82-6dc5-448e-8c73-1669759abea9 (Accessed 25 January 2024).

Landers, R.N., Auer, E.M., Helms, A.B., Marin, S. and Armstrong, M.B., (2019) Gamification of adult learning: Gamifying employee training and development. The Cambridge handbook of technology and employee behavior, pp.271-295.

Mansoux, A. et al. (2023) ‘Permacomputing Aesthetics: Potential and Limits of Constraints in Computational Art, Design and Culture’, in Ninth Computing within Limits 2023. LIMITS. doi:10.21428/bf6fb269.6690fc2e

McNiff, J. (2020) ‘Action Research for Professional Development’ [electronic resource]. https://www.jeanmcniff.com/ar-booklet.asp (Accessed 20 November 2023)

Odrozek, K. (2018) The internet uses more electricity than…, Internet Health Report. Available at: https://internethealthreport.org/2018/the-internet-uses-more-electricity-than/ (Accessed: 06 October 2023). 

Permacomputing.org (2023) Permacomputing. Available at: https://permacomputing.net/ (Accessed 06 October 2023).

Paisey, C. and Paisey, N.J. (2005) ‘Improving accounting education through the use of action research’, Journal of Accounting Education, 23(1), pp. 1–19. doi:10.1016/j.jaccedu.2004.10.001. 

Pew Research Center (2021) Writing survey questions. Available at: https://www.pewresearch.org/our-methods/u-s-surveys/writing-survey-questions/ (Accessed: 05 December 2023). 

Qualtrics (2023) Survey Methodology & Compliance Best Practices: Predicted Duration, Qualtrics XM: The Leading Experience Management Software. Available at: https://www.qualtrics.com/support/survey-platform/survey-module/survey-checker/survey-methodology-compliance-best-practices/?utm_medium=ExpertReview&utm_source=product&utm_campaign=PredictedDuration&utm_content=#PredictedDuration (Accessed: 20 November 2023).

Siegman, T. (2023) Introducing web sustainability guidelines, W3C. Available at: https://www.w3.org/blog/2023/introducing-web-sustainability-guidelines/ (Accessed 09 October 2023).

Smith, A. (2023) Climate Change and Student Mental Health Report. Available at: https://www.studentminds.org.uk/uploads/3/7/8/4/3784584/climate_change_and_student_mental_health.pdf (Accessed 25 January 2024)

University of Edinburgh (2020) Gibbs’ Reflective Cycle. Available at: https://www.ed.ac.uk/reflection/reflectors-toolkit/reflecting-on-experience/gibbs-reflective-cycle (Accessed 25 January 2024).

University of Sheffield. (2018) Emotionally demanding research: risks to the researcher. [Specialist Research Ethics Guidance Paper]. Available via: https://moodle.arts.ac.uk/pluginfile.php/1587855/mod_folder/content/0/SREGP-Emotionally-Demanding-Research%20-%20University%20of%20Sheffield%202018.pdf?forcedownload=1 (Accessed 25 October 2023)

University of the Arts London (2023) Electricity Consumption. [Report] Available at: https://www.arts.ac.uk/__data/assets/pdf_file/0025/374128/Climate-Action-Plan_Nov2022.pdf (Accessed 09 October 2023)

University of the Arts London (2022) Climate Action Plan. [Report] Available at: https://www.arts.ac.uk/__data/assets/pdf_file/0025/374128/Climate-Action-Plan_Nov2022.pdf (Accessed 25 October 2023) 

University of the Arts London (2022) Guiding policy 2. Available at: https://www.arts.ac.uk/about-ual/strategy-and-governance/strategy/guiding-policy-2 (Accessed: 20 November 2023). 

Website Carbon Calculator. (2023) Carbon results for arts.ac.uk. Available at: https://www.websitecarbon.com/website/arts-ac-uk/ (Accessed: 06 October 2023). 

Website Carbon Calculator. (2023) Carbon results for moodle.arts.ac.uk/login/index.php. Available at: https://www.websitecarbon.com/website/moodle-arts-ac-uk-login-index-php/ (Accessed: 06 October 2023).

World Wide Web Consortium (2023) About Us, W3C. Available at: https://www.w3.org/about/ (Accessed: 18 December 2023). 

World Wide Web Consortium (2023) Web sustainability guidelines (WSG) 1.0. Available at: https://w3c.github.io/sustyweb/ (Accessed: 04 January 2024).

In this post I will reflect on my ARP – what went well, what didn’t, to what extent was I able to answer the research questions, what part of the AR cycle did I complete and what’s next? This post is intended to bookend my ARP alongside the ARP overview post, where I laid out the research questions and action research cycle.

I am going to use Gibbs reflective cycle (1988) and The University of Edinburgh’s reflective toolkit (2020) in order to think through these questions.

Description

In this ARP, I wanted to answer the following questions:

• How aware are UAL digital learning staff members about digital sustainability?
• How can knowledge and confidence of this topic be increased to create best practice?

I wanted to do this because digital sustainability appears to be an under explored area of UAL’s Climate Action Plan as well as the progress update report (2023) that was released towards the end of my project. It felt like there was a disconnect between this and the university’s strategy to reach more students outside of London through online means. I couldn’t point fingers because I myself, working in an online learning role, knew little about the topic of digital sustainability. So this project was about trying to take action in my own context, hoping that that leads to something larger.

To answer these research questions I had a mixed methods approach. First I needed to survey UAL digital learning staff, which I did through an anonymous online questionnaire. I need to also understand what knowledge and guidance were out there, so I completed document analysis of the Web Sustainability Guidelines (WSG). Lastly, to understand how training and knowledge of this topic could be cultivated, I attended UAL’s Carbon Literacy Training programme. The outcome of each method ultimately was that I had a better understanding of where UAL learning technologists are in relation to digital sustainability – much to learn but eager to do it. I found ten low-effort and actionable guidelines from the WSG. And I observed the efficacy of gamified learning approaches to this heavy and complex topic. Ultimately all of these things have contributed to a prototype low-tech game I have made and will test out with my CSM colleagues.

Feelings

Uncertainty and the feeling of being over my head were common feelings. Those are symptomatic of any research project, but specifically for me they arose because of my own lack of knowledge of digital sustainability. There were points, particularly when making the game that I felt I was positioning myself as someone experienced and knowledgable of this topic. I was drawing on the WSG to author the game, rather than pulling things from my head, so perhaps there’s a bit of imposter syndrome going on there.

There were also hash truths and uncomfortable learnings about the role of the internet – and my extension my professional practice – in the climate crisis. This made me feel guilty for my ignorance. But I did feel empowered by the ARP as it gave me an opportunity to enhance my understanding and importantly to do something with that.

How do I feel now – whatever feeling is the opposite of accomplished! Not because I didn’t make headway against my research questions – I feel that I have. But because this really is the tip of the iceberg with regard to this topic and the wider UAL conversation around it. But I guess that is the nature of an ARP – carving out small areas bit by bit to make change.

Evaluation

What worked and didn’t work about the ARP? I had about a 36% response rate to my questionnaire, which while it gave me a range of perspectives, may not have been the largest sample size. Another reality of doing this ARP as part of the PgCert is that it was quite an individual activity… it was crucial for me to include the perspectives of other UAL digital learning staff via the questionnaire, but I think for the next stage of where this goes it needs to really happen in the context of an action group. This is because achieving digital sustainability requires institutional change. I’m pleased to have started something and have the fire to take it forward beyond the ARP but I can’t do justice to it on my own!

Equally the game feels very much like a prototype. I don’t think I’ve quite pulled that off yet as a super engaging and fun game, so that’s an area that needs improvement, hopefully with the input of my digital learning colleagues at CSM.

Analysis

As well as what I’ve learnt from answering my research questions, an overarching takeaway from this project is that digital sustainability is a burgeoning area of knowledge and practice, not just at UAL but in general. I think this is partly because the internet feels less tangible and ethereal climate change culprit than, say, taking a flight. I also think it’s largely about awareness raising though; we only know about the huge environmental cost of industries because activists and campaign groups shine a light on this. I don’t think this has happened yet for the internet – if anything we are driven to using it more and more, to the benefit of large social media and tech companies.

So I certainly started this ARP from the beginning and in a place of ignorance. But I learnt that I certainly wasn’t alone in that. And I learnt about the complexity of the topic – what do we mean by digital sustainability, is it hardware procurement and disposal, is it how we use the internet, or is it (how I define it for elearning ) about how we design content for the web? It’s also hard to get a sense of how applying best practice to elearning tangibly makes a difference, largely because we don’t have carbon costs for this specific area yet – although Mike Berners Lee (2020) has mapped many other things!

So these complexities contributed to the direction of the project. I initially started out thinking I could apply some changes to Moodle based on my reading of the WSG and see some tangible results to the carbon count of that site. Things sadly were not as simple as that, but that complexity woke me up to the potentials of other actions, ie the ideas of engaging our digital learning team in the topic, which I think is a more meaningful approach overall.

Conclusions

Apart from the specific learnings – ie digital learning’s staff knowledge of the topic, guidelines we can follow, pedagogical methods of delivering training – one of my conclusions is that the social justice angle of the project has left an impression on me. I hope to maintain this after the project and have applied to join UAL’s Carbon Calculation working group in order to do so. Again, I learnt that this is a huge and burgeoning topic, but there is a willingness to develop it at UAL and there beginning to be tools emerging out there for us to do that.

Action plan

1. Share the game prototype with my CSM Digital Learning colleagues in order to improve and develop it for wider distribution.
2. Participate in the UAL Carbon Calculation group to get digital learning activities (and more broadly UAL’s internet usage) on the table for discussion on and institutional level.

If I were to do things differently, I’d have maybe bounced my reading of the WSG against some others in UAL, ie digital learning staff. Because I realise in analysing the guidelines I have made assumptions about what are feasible for us at UAL, so it would be great to have others’ input into that. Also – are the ten guidelines too much? If we’re at the beginning of trying to create more sustainable elearning content, maybe I should have really focused on the absolute basics with 5 points or even 3. I do intend to explore that though while testing the prototype with digital learning colleagues and by choosing a low stakes format of game – powerpoint – it is not a huge effort to revise and change things.

Overall I feel things went to plan for this ARP and also not to plan – I registered for the Carbon Literacy Training thinking it might help me learn about the topic but realising swiftly it was also a chance to observe the teaching of this topic. And the planned and unplanned aspects of the project have helped me to answer my research questions. Ultimately I feel able to carry out the full action research cycle beyond the scope of this Pgcert and to continue to reiterate it in the future.

References

Berners-Lee, M (2020). How bad are bananas? : the carbon footprint of everything. London: Profile Books.

Gibbs G (1988). Learning by Doing: A guide to teaching and learning methods. Oxford Polytechnic: Oxford.

University of Edinburgh (2020) Gibbs’ Reflective Cycle. Available at: https://www.ed.ac.uk/reflection/reflectors-toolkit/reflecting-on-experience/gibbs-reflective-cycle (Accessed 25 January 2024).

I have chosen a mixed methods approach: document analysis, questionnaire (quantitative + qualitative), and individual learning/training. I did consider whether autoethnographic would be a suitable approach, as I am beginning this project on digital sustainability from a position of little knowledge and intend to develop my understanding of this over the course of the project. However, through reading of Ellis and Bochner (2006) I realised that development and growth of knowledge through an ARP is common (indeed to be expected!) and this is not the same thing as using an autoethnographic method which specifically explores the intersection between oneself and society, focusing on the relation with regard to a social, historical, political, cultural context.

Document analysis

Glenn Bowen’s article “Document analysis as a qualitative research method” provides an overview of this research method. He highlights the advantages of this method: it is efficient, generally without cost, applicable to a wide range of documents (including online sources, newspapers, personal diaries, public records).

I have chosen to use document analysis for my Action Research Project, in order to respond to my research question “How can knowledge and confidence of this topic be increased to create best practice?”. In addition to the review of existing literature and reports around sustainability more generally as well as online sustainable practice, I think it’s important to do an an in-depth analysis of the newly released Web Sustainability Guidelines. These guidelines are the new standard for digital sustainability, and are based on measurable evidence based research.

This is a large document with over ninety suggested actions for digital sustainability. Each action has been designated an impact and effort value. Because I am at the beginning of my digital sustainability journey, my first point of analysis will be to pull all of the low effort actions from the guidelines. I feel this will allow me to discover some quick wins or easy actions that could feature into my artefact.

Bowen mentions there are limitations to document analysis, including “biased selectivity” (p. 32) and “insufficient detail” (p. 31-32). The latter is the most apparent limitation of this research, as I have found that UAL documentation does not specifically speak about the role of digital sustainability in online learning practice, and conversely the external guidelines are just that, guidelines, and not specific to our UAL context. Therefore, as Bowen suggests, it’s a good idea to triangulate this research method with others, and I will do this for my ARP as well through a questionnaire.

Please see my document analysis blog post for further information about how I employed this research method for my ARP.

Questionnaire – qualitative and quantitative research

As one of my research questions is “How aware are UAL digital learning staff members about digital sustainability?”, I need to get UAL specific data to answer this question. I have decided to gather this via an online questionnaire. This felt like the most appropriate research method as colleagues work across a number of campuses across London and in a hybrid manner, so allowing a flexible online method for participation was key. Equally, Qualtrics (the software I have chosen) predicts that my questionnaire falls within the optimum duration for completion, indicating that this questionnaire will be less burdensome for participants to complete. Qualtrics (2023) has interestingly discovered that “surveys longer than 12 minutes (and 9 minutes on mobile) start to see substantial levels of respondent break-off.”

While I have used questionnaires as a research method in the past, it was important to engage with the ARP unit’s reading on this topic as well. I therefore read the Pew Research Center’s article on Writing Survey Questions. This article provided some great guidance about questionnaire design for optimum data gathering. The article provides guidance:

• question wording: asking one question at a time rather than overloading, using neutral rather than loaded/biased wording
• open/closed questions: where do I want free text answers, where do I just want stats?
• logical ordering of questions and options for multiple choices
• managing social desirability bias, which is the tendency of survey respondents to provide answers that make themselves look good rather than responding truthfully. I hope that by making this survey anonymous I can counteract that from happening.
• asking about sensitive topics – in my context sustainability could definitely be considered sensitive as climate change can be a triggering topic and also I am asking colleagues to comment on their professional knowledge of the topic. Again by making the questionnaire anonymous I hope to counteract this as well as letting participants know that they can withdraw at any time.

Based on my reading of this article, I wrote the following questions:

1. How familiar are you with the concept of digital sustainability? 
   • Please share any details via the free text box.
2. Are you aware of any guidelines or tools for reducing online carbon emissions? 
   • Please share any guidelines or tools you are aware of.
3. How would you rate your confidence in sharing sustainable best practice for online learning with fellow staff, ie academics, technicians, professional staff?
   • Please share any details via the free text box.
4. Would you be interested in training or guidance for best practice in reducing emissions in online course build and design?
5. Would you be interested in being further involved in further projects around sustainable practice for online learning at UAL?
6. If you have comments or suggestions about this research or its topic please add them here.

The questionnaire was the area of my research that presented the most ethical considerations, which I explore further in this post. You can also read more about the questionnaire data analysis in this post.

Individual learning/training

By using the questionnaire and document analysis I believe I can answer both of my research questions:

• How aware are UAL digital learning staff members about digital sustainability?
• How can knowledge and confidence of this topic be increased to create best practice?

Carrying out the questionnaire corresponds to the first question and carrying out the document analysis enables me to understand what best practice looks like in this area in order for digital learning professionals to be able to adopt it.

However, for the second question, I also felt like the document analysis is not enough to understand how the knowledge and confidence in the topic can be developed in other staff members. So for this reason I decided it was important to the project for to undertake training via UAL’s carbon literacy training programme to

1. become more literate in this area and understand it in the UAL context;
2. observe how these sessions work pedagogically.

I was particularly in the pedagogical methods as this is a topic that can both overwhelm in terms of information and in terms of anxiety. Attending the training therefore felt integral to my ARP and I have thus included here as sort of research method to inform the pedagogical approach of my own intervention. I speak more about the Carboned Out game and the exploration of gamification that it prompted in this blog post.

References

Bowen, G.A. (2009) ‘Document analysis as a qualitative research method’, Qualitative Research Journal, 9(2), pp. 27–40. doi:10.3316/qrj0902027. 

Ellis, C.S. and Bochner, A.P. (2006) ‘Analyzing analytic autoethnography’, Journal of Contemporary Ethnography, 35(4), pp. 429–449. doi:10.1177/0891241606286979. 

Pew Research Center (2021) Writing survey questions. Available at: https://www.pewresearch.org/our-methods/u-s-surveys/writing-survey-questions/ (Accessed: 05 December 2023). 

Qualtrics (2023) Survey Methodology & Compliance Best Practices: Predicted Duration, Qualtrics XM: The Leading Experience Management Software. Available at: https://www.qualtrics.com/support/survey-platform/survey-module/survey-checker/survey-methodology-compliance-best-practices/?utm_medium=ExpertReview&utm_source=product&utm_campaign=PredictedDuration&utm_content=#PredictedDuration (Accessed: 20 November 2023). 

Authoring the game: using a gamified powerpoint presentation

With my reading of the Landers et al, and my knowledge of the Web Sustainability Guidelines, and my knowledge of my audience (UAL learning technologists and others engaged in online learning) I had to think about how to deploy the game. Ultimately, I have decided to develop the game within Powerpoint for the following reasons:

• It’s a low tech, non internet dependent way of developing and playing a digital game. So while it might be nice to create something high tech, that wouldn’t be true to the overarching mission of the game and this project which is to seek out sustainable options for online learning.
• It can played in a group dynamic with a facilitator, or it can be played by an individual in present mode. Having flexibility about synchronous/asynchronous modes at this early stage felt important.
• Easy to edit, easy to scale. Again at this early stage being able to quickly develop a game that can be iterated is important and in line with sustainable approaches to content development.
• Easy to make available to others. Powerpoint is widely available and used programme to that makes it easier for people to access the game and play. Powerpoint slides can be made available as downloads from pretty much anywhere, ie intranet, blogs, Moodle etc.

Storyboarding content and activities

In this Miro board, I’ve included my draft for the game storyboard. It focuses on the ten areas of sustainable guidance, ideas for questions connected to these points, and some ideas sparked from reading Landers et al (2019) including the progress bar and milestones. Using all these elements I created the first draft of the game – I say first draft as I do feel it needs some user feedback to improve.

Game aesthetic design

The last part of creating the game was its visual design in Powerpoint. I knew that I wanted to keep the file as low in size as I could and the design would be part of that. I used ual branded colours and fonts used icons/shapes over his res photos where possible (I also compressed all graphics in the file as well). I decided that this minimised aesthetic, could also embrace the humour and familiarity of more pastiched elements of Powerpoint including simple slide design, word art and animations – these can be design (and sustainable) strengths rather than a weaknesses. This ties in with the concept of permacomputing (mentioned elsewhere in this blog post). As Mansoux et al (2019) state:

As a result, permacomputing aims to provide a countervoice to digital practices that promote maximisation, hyper-consumption and waste. It seeks to encourage practices as an applied critique of contemporary computer technology that privileges maximalist aesthetics where more pixels, more frame rate, more computation and more power equals more potential at any cost and without any consequences. We believe that such a critical practice can be relevant to artists, designers and cultural practitioners working with computer and network technology who are interested in engaging with environmental issues. 

What next for this intervention?

I feel that the next steps for this game is to test it with my fellow digital learning colleagues to understand its effectiveness, areas for improvement and to gather any thoughts they have on it. Testing iteratively in this way ties in with the sustainable concept principle of ideation.

You can download the first draft of the game below. Coming in at 4MB the carbon impact of downloading or sharing it is significantly reduced compared to a higher tech format with audio, photos and video content. (In fact it’s somewhere between the same size as a blurry low quality picture taken on a smartphone and streaming one minute low res video! (GreenNet 2023)

References

GreenNet (2023) Understanding file sizes. Available at: https://www.greennet.org.uk/support/understanding-file-sizes (Accessed 22 January 2024).

Mansoux, A. et al. (2023) ‘Permacomputing Aesthetics: Potential and Limits of Constraints in Computational Art, Design and Culture’, in Ninth Computing within Limits 2023. LIMITS. doi:10.21428/bf6fb269.6690fc2e

I identified document analysis as a pertinent research approach for my ARP, based on my reading of Bowen (2009).

Why choose the WSG for analysis?

The World Wide Web Consortium (W3C) was founded by Tim Berners-Lee (creator of the web) in 1994 to ensure the long-term growth of the web. (World Wide Web Consortium 2023). They provide web standards that those developing websites should follow in order to ensure quality.

These include the Web Content Accessibility Guidelines (WCAG) 2.2, which in line with the 2018 accessibility regulations must be followed by all public sector bodies – including universities. In my role as a learning technologist, understanding and following these guidelines is vital to ensure online learning content at UAL is accessible and inclusive and this has been a large focus of my PgCert coursework.

Therefore when W3C released the draft version of the WSG in 2023 this signalled a big moment for web sustainability as there is now a recognised standard for this and designated guidelines for best practice. While it is not yet a legally binding standard like the accessibility guidelines, it signals the intent for where we need to go next.

Overview of the WSG

• Nearly 100 guidelines for making websites and digital products more sustainable.
• Guidelines are divided into user experience design, web development, hosting infrastructure & systems, and business strategy & product management.
• Guidelines are also categorised by their impact level from low to medium to high.
• They are also categorised by their effort level from low to medium to high.

Strategy for analysis

One parallel I can draw between accessibility guidelines and sustainability guidelines is that both can feel like complex concepts that are difficult to understand and deliver. To help build a foundation of understanding, learning technologists in helping colleagues understand digital accessibility often focus on the easier topics (the more complex ones being things that learning technologists themselves can focus on). For this project I have taken a similar approach, as I think mastering the simpler aspects of web sustainability is a good place to start. Looking at the questionnaire results, UAL learning technologists 64% of respondents fell unconfident about advising other staff members about digital sustainability best practice.

For that reason I manually created a report that included only the low effort actions, as what I want to get from these guidelines are a select number of very easy things someone working in online learning can do now to bring sustainability into their practice.

Once I found all of the low effort actions, which totalled to 37, I needed to read through each them carefully and decide whether they would be feasible actions within the the context of online learning and UAL. So for example, guideline 4.5 “Limit Usage Of Additional Environments” might be a low effort action for someone involved in hosting, infrastructure and systems. However, it isn’t something a learning technologist or lecturer could do to make their online learning practice more sustainable. I therefore needed to read through each of the low effort guidelines to determine whether these could easily adopted within the online learning context at UAL.

The below file includes the analysis of low effort WSG and to what degree each guidelines could be feasible for UAL digital learning.

Findings and conclusions

There were ultimately 10 low effort digital sustainability guidelines that I considered to be feasible for UAL digital learning teams:

1. Take a More Sustainable Approach To Image (and media) Assets: “Of all the data which comprises the largest over-the-wire transfer rates within the average website or application, images are usually those which are responsible due to their quantity and usefulness. As such, doing all you can to reduce their size and unnecessary loading will be beneficial for reducing emissions.” This is especially pertinent for an art and design institution, where our image output and usage may well be higher than average and thus the need to do everything we can to reduce our emissions here. This involved assessing the need for images, considering the quantity, format, and size necessary for implementation, optimising images by resizing and compressing, having a maintenance and usage policy for images.
2. Write With Purpose, In An Accessible, Easy To Understand Format: this point nicely intersects with accessibility principles of writing clearly (without jargon or unnecessarily complicated language), using clear document structure, visual hierarchy, headings, bulleted lists, line spacing, and using defined and consistent terminologies/nomenclatures.
3. Reduce The Impact Of Downloadable Or Physical Documents: Sometimes PDFs are required but not always – can you convey the information through a Moodle page and then encourage students not to print where necessary? If you are using PDFs or other files, remember that most users don’t want documents to force download to their devices (especially on mobile). Be mindful of the file settings on Moodle so that documents open in a tab which is less carbon intensive that downloading and hosting unwanted files on multiple devices.
4. Be Mindful Of Duplicate Data: this is particularly relevant for course back ups. It’s wise to back up a Moodle courses in case something happens and data/content is wiped. However, for sustainability it’s good to have a storage plan for back-ups, ie removing outdated back-ups, storing them on Sharepoint or Moodle not in both places etc.
5. Consider Sustainability In Early Ideation ie by wire framing and prototyping (either online or on paper) Moodle sites rather than building a full blown experience for each idea.
6. Ensure Navigation And Way-finding Is Well-structured because the less time students need to trawl through a Moodle site to find what they’re looking for, the less energy will be used. Sometimes Moodle sites can be used as a sort of “dumping ground” for recordings and other random files without clear structure and design. There are therefore a multitude of benefits in avoiding this approach, with sustainability being one of them.
7. Respect The Visitor’s Attention this can be achieved by allowing them to easily control how (and when) they receive information (ie Moodle forum notifications) as well as avoiding using infinite scroll or related attention-keeping tactics. In previous research with students, infinite scroll has come up as a frustration with using Moodle and results in wasted effort of electricity.
8. Use Recognised Design Patterns: this is similar to the point about navigation, in the sense that interfaces should deploy visual styles (patterns) that are easily recognised and used. So in our Moodle context that might mean having a consistent design or template of Moodle pages for a programme so that students aren’t relearning how to use a Moodle site from unit to unit.
9. Develop A Mobile-first Layout: We don’t currently have the Moodle app rolled out at UAL. However testing our courses in mobile view is important to see if the design is responsive and easy to use, ultimately averting extra time online wasted trying to navigate through a poor mobile experience.
10. Establish If A Digital Product Or Service Is Necessary: Moodle sites are usually non-negotiable, ie every unit will need one etc. But what about digital resources supporting courses? Are there duplications in videos/PDF guides/blog posts? Achieving this one may require working with colleagues to identify overlaps and refine things.

What next?

The next thing will be to use these ten guidelines as a basis for an intervention that will help those involved in digital learning at UAL to incorporate sustainable practices into their work. You can read more about the intervention in this post.

References

Bowen, G.A. (2009) ‘Document analysis as a qualitative research method’, Qualitative Research Journal, 9(2), pp. 27–40. doi:10.3316/qrj0902027. 

World Wide Web Consortium (2023) About Us, W3C. Available at: https://www.w3.org/about/ (Accessed: 18 December 2023). 

World Wide Web Consortium (2023) Web sustainability guidelines (WSG) 1.0. Available at: https://w3c.github.io/sustyweb/ (Accessed: 04 January 2024).

I opened my questionnaire to digital learning staff via the UAL Digital Learning Support channel on MS Teams. It was open for a period of three weeks from 06/11 to 27/11. Overall I had a 36% response rate, as 14 of the 39 members in the chat responded. While I would have liked to have had a larger sample size – at least 50% would have been great – I decided to call it in the interest of time constraints of the project. Additionally, if I can reflect on why I didn’t get a larger response rate, there may have been multiple factors at play: not everyone uses MS Teams, staff absences, disinterest in the topic, lack of incentive to complete the questionnaire, workload.

Ultimately however I do think that 14 responses does still equate to a range of participants and perspectives. I have used Braun and Clarke’s model of reflexive thematic analysis (no date) to understand the questionnaire results.

Reflexive thematic analysis of questionnaire results

Familiarising myself with the dataset

With my data collected, it was time to get familiar with it. Helpfully, Qualtrics has a report builder functionality so I was able to convert the responses of each of the questions into visualisations. This helped me to take in and retain the findings from the survey. It also gave me the sense that overall, we are in a positive position with most (77%) of respondents being somewhat familiar about digital sustainability (although as further analysis showed, definitions of that meant varied) and 92% being interested in training or guidance for best practice in reducing emissions in online course build and design.

The full report I created in order to familiarise myself with the data below.

Coding

While coding is a suggested step in the thematic analysis process, as Braun and Clarke (n.d.) state “These phases do not prescribe a rigid process you must follow”. Therefore due to the concise dataset I have gathered, I don’t believe the coding process, whereby codes are applied to the data in multiple rounds of analysis, is required. Rather working with the themes the data presents is more appropriate for this data.

Themes: generating, developing, reviewing, refining, and naming

• The first stage of this process is to generate initial themes. To do this I’ve copied each of the data visualisations for each question to a Miro board, so that I can move them around and group them depending on theme.
  • Existing strengths + areas of growth: I split the data into these categories, and the majority of the data signalled existing strengths in the community; 79% were familiar with the concept of digital sustainability, more than half were able to suggest guidelines or tools for reducing online carbon emissions, 92% were interested in training or guidance for best practice, and 69% were interested in being further involved in projects around this topic. One area for growth though, was that the majority felt unconfident in sharing sustainable best practice for online learning with fellow staff. This isn’t a negative thing, it presents the need for a resource to help digital learning staff to do this – this is where my intervention can come in.
• I then looked at the free text fields to generate some themes. From analysing this data I discovered that digital sustainability means different things to different people, so I grouped these responses into the following categories of what we talk about when we talk about digital sustainability:
  • Hardware + hosting: This includes hardware procurement and disposal, the internet servers an organisation includes, etc. It is generally under the remit of IT and university leadership. This was mentioned by three respondents.
  • General sustainable best practice for being online: This is general guidance for using devices, browsing the web, managing files on cloud storage, sending emails, etc. This was the most common category the free text field responses fell under.
  • Sustainable design of online learning materials: This is specifically how we design and deliver our courses and online learning materials. No one actually specifically mentioned this, which suggests there is a gap in guidance and understanding around this specifically and also points to the complexity of this topic.
• The last theme I gleaned from the data was around external guidelines, there were three resources suggested:
  • Government Digital Sustainability Alliance: This one was new to me. It seems to be a consortium of a number of government departments and tech companies. There is a blog that details their work but it doesn’t seem have direct application or guidance for our context at UAL.
  • Website carbon calculator: I knew about this and used it to generate reports for the UAL website and Moodle.
  • Wholegrain digital: I also knew about this, it’s connected to the website carbon calculator. It’s a useful and comprehensive overview of the different elements that contribute to online carbon emissions and is a more condensed and accessible version of the Web Sustainability Guidelines. These weren’t mentioned by anyone which isn’t too much of a surprise as they are quite new and quite dense to navigate through. That said, even the wholegrain digital blog is wordy and hard to read end to end for someone engaged on this topic let alone asking other staff to read it.

Conclusions

• Within the digital learning team there is a base level of existing knowledge around digital sustainability. However this is largely with regard to devices and general internet usage. There wasn’t evidence of a large understanding of digital sustainability for content creation.
• However, there is a desire to learn more about this topic and how it connects to their role.
• Online learning staff are hugely involved in training for the wider university with learning technologies. We advise on technical aspects of this, as well as pedagogy for online and digital accessibility. However, advising on digital sustainability is not an area that staff currently feel confident in advising colleagues about.
• The resources that online learning staff know about with regard to digital sustainability are not always relevant to our practice, or they give a basic level of knowledge or too much information. This can make it hard for staff to tangibly apply this to their own practice, or share with others across the college.

Writing up

The write up is constituted by this blog and the accompanying Miro board (screenshot below) which details how I analysed the data.

Additionally I created the below infographic to visualise what I find. Ultimately, the questionnaire signalled to me that we have some promising foundations for growth in the area of digital sustainability. These are: familiarity with the concept, awareness of carbon calculation tools, confidence to share best practice, desire for training/guidance and interest in a UAL approach – and are represented as roots in the infographic. But the digital sustainability cannot flourish with these roots alone, it needs UAL specific guidance and institutional policy in order to become a fully fledged practice.

Next steps

Analysing this data is an important part of informing what comes next in my ARP, that is creating an intervention. From the data I can see that the digital learning community has a baseline knowledge of this topic but wants to do more and needs help with sharing sustainable best practice with colleagues. This data alongside the analysis I have done of the web sustainability guidelines and the gamified approach I saw demonstrated in the UAL Carbon Literacy training [read more in this blog] will lead me to creating my intervention to help develop digital sustainability at UAL.

References

Braun, V. and Clarke, V. (no date) Doing reflexive Thematic Analysis. Available at: https://www.thematicanalysis.net/doing-reflexive-ta/ (Accessed: 23 November 2023). 

Research questions

How aware are UAL digital learning staff members about digital sustainability?

How can knowledge and confidence of this topic be increased to create best practice?

Please see my “context and rationale” blog post for more information on the development of these questions.

Position on the Action Research cycle

In line with my reading of McNiff and Paisey and Paisey’s action research diagram, I have formulated my ARP to involve the following:

Step	My ARP
Step 1: Define the problem and and frame the research questions	Identifying and reading around area of concern: digital sustainability (Read this blog post for more detail). Deciding on the research methods: document analysis, questionnaire and participating in UAL’s Carbon Literacy Training programme. (Read this blog post for more detail).
Step 2: Collect data and decide how practice could be changed	Analysing the Web Sustainability Guidelines (see this blog for the analysis) to see what guidelines could be relevant and implementable for UAL digital learning staff, using reflexive thematic analysis to draw findings from the questionnaire responses (blog post), reflecting on the carbon literacy training module in order to develop my intervention (blog post).
Step 3: Implement the changes	Creation of intervention: an interactive online game that explains the different concepts of digital sustainability in an engaging, friendly, digestible way. It can therefore be used by more than just digital learning technologists. (Blog post)

Unfortunately I will not have time to complete Step 4 (Monitor and evaluate the changes made) and Step 5 (Review and reflect upon the changes) within the timeframe of this ARP. However, this ARP has woken me up to the climate implications of my role and I am dedicated to continuing the cycle after the ARP concludes by joining the UAL Climate Emergency Network.

Action plan/schedule

I used Miro to plan out my project, please visit this Miro board to see the interactive plan or the screenshot of it below.

References

McNiff, J. (2020) ‘Action Research for Professional Development’ [electronic resource]. https://www.jeanmcniff.com/ar-booklet.asp (Accessed 20 November 2023)

Paisey, C. and Paisey, N.J. (2005) ‘Improving accounting education through the use of action research’, Journal of Accounting Education, 23(1), pp. 1–19. doi:10.1016/j.jaccedu.2004.10.001.