I have chosen a mixed methods approach: document analysis, questionnaire (quantitative + qualitative), and individual learning/training. I did consider whether autoethnographic would be a suitable approach, as I am beginning this project on digital sustainability from a position of little knowledge and intend to develop my understanding of this over the course of the project. However, through reading of Ellis and Bochner (2006) I realised that development and growth of knowledge through an ARP is common (indeed to be expected!) and this is not the same thing as using an autoethnographic method which specifically explores the intersection between oneself and society, focusing on the relation with regard to a social, historical, political, cultural context.
Document analysis
Glenn Bowen’s article “Document analysis as a qualitative research method” provides an overview of this research method. He highlights the advantages of this method: it is efficient, generally without cost, applicable to a wide range of documents (including online sources, newspapers, personal diaries, public records).
I have chosen to use document analysis for my Action Research Project, in order to respond to my research question “How can knowledge and confidence of this topic be increased to create best practice?”. In addition to the review of existing literature and reports around sustainability more generally as well as online sustainable practice, I think it’s important to do an an in-depth analysis of the newly released Web Sustainability Guidelines. These guidelines are the new standard for digital sustainability, and are based on measurable evidence based research.
This is a large document with over ninety suggested actions for digital sustainability. Each action has been designated an impact and effort value. Because I am at the beginning of my digital sustainability journey, my first point of analysis will be to pull all of the low effort actions from the guidelines. I feel this will allow me to discover some quick wins or easy actions that could feature into my artefact.
Bowen mentions there are limitations to document analysis, including “biased selectivity” (p. 32) and “insufficient detail” (p. 31-32). The latter is the most apparent limitation of this research, as I have found that UAL documentation does not specifically speak about the role of digital sustainability in online learning practice, and conversely the external guidelines are just that, guidelines, and not specific to our UAL context. Therefore, as Bowen suggests, it’s a good idea to triangulate this research method with others, and I will do this for my ARP as well through a questionnaire.
Please see my document analysis blog post for further information about how I employed this research method for my ARP.
Questionnaire – qualitative and quantitative research
As one of my research questions is “How aware are UAL digital learning staff members about digital sustainability?”, I need to get UAL specific data to answer this question. I have decided to gather this via an online questionnaire. This felt like the most appropriate research method as colleagues work across a number of campuses across London and in a hybrid manner, so allowing a flexible online method for participation was key. Equally, Qualtrics (the software I have chosen) predicts that my questionnaire falls within the optimum duration for completion, indicating that this questionnaire will be less burdensome for participants to complete. Qualtrics (2023) has interestingly discovered that “surveys longer than 12 minutes (and 9 minutes on mobile) start to see substantial levels of respondent break-off.”
While I have used questionnaires as a research method in the past, it was important to engage with the ARP unit’s reading on this topic as well. I therefore read the Pew Research Center’s article on Writing Survey Questions. This article provided some great guidance about questionnaire design for optimum data gathering. The article provides guidance:
- question wording: asking one question at a time rather than overloading, using neutral rather than loaded/biased wording
- open/closed questions: where do I want free text answers, where do I just want stats?
- logical ordering of questions and options for multiple choices
- managing social desirability bias, which is the tendency of survey respondents to provide answers that make themselves look good rather than responding truthfully. I hope that by making this survey anonymous I can counteract that from happening.
- asking about sensitive topics – in my context sustainability could definitely be considered sensitive as climate change can be a triggering topic and also I am asking colleagues to comment on their professional knowledge of the topic. Again by making the questionnaire anonymous I hope to counteract this as well as letting participants know that they can withdraw at any time.
Based on my reading of this article, I wrote the following questions:
- How familiar are you with the concept of digital sustainability?
- Please share any details via the free text box.
- Are you aware of any guidelines or tools for reducing online carbon emissions?
- Please share any guidelines or tools you are aware of.
- How would you rate your confidence in sharing sustainable best practice for online learning with fellow staff, ie academics, technicians, professional staff?
- Please share any details via the free text box.
- Would you be interested in training or guidance for best practice in reducing emissions in online course build and design?
- Would you be interested in being further involved in further projects around sustainable practice for online learning at UAL?
- If you have comments or suggestions about this research or its topic please add them here.
The questionnaire was the area of my research that presented the most ethical considerations, which I explore further in this post. You can also read more about the questionnaire data analysis in this post.
Individual learning/training
By using the questionnaire and document analysis I believe I can answer both of my research questions:
- How aware are UAL digital learning staff members about digital sustainability?
- How can knowledge and confidence of this topic be increased to create best practice?
Carrying out the questionnaire corresponds to the first question and carrying out the document analysis enables me to understand what best practice looks like in this area in order for digital learning professionals to be able to adopt it.
However, for the second question, I also felt like the document analysis is not enough to understand how the knowledge and confidence in the topic can be developed in other staff members. So for this reason I decided it was important to the project for to undertake training via UAL’s carbon literacy training programme to
- become more literate in this area and understand it in the UAL context;
- observe how these sessions work pedagogically.
I was particularly in the pedagogical methods as this is a topic that can both overwhelm in terms of information and in terms of anxiety. Attending the training therefore felt integral to my ARP and I have thus included here as sort of research method to inform the pedagogical approach of my own intervention. I speak more about the Carboned Out game and the exploration of gamification that it prompted in this blog post.
References
Bowen, G.A. (2009) ‘Document analysis as a qualitative research method’, Qualitative Research Journal, 9(2), pp. 27–40. doi:10.3316/qrj0902027.
Ellis, C.S. and Bochner, A.P. (2006) ‘Analyzing analytic autoethnography’, Journal of Contemporary Ethnography, 35(4), pp. 429–449. doi:10.1177/0891241606286979.
Pew Research Center (2021) Writing survey questions. Available at: https://www.pewresearch.org/our-methods/u-s-surveys/writing-survey-questions/ (Accessed: 05 December 2023).
Qualtrics (2023) Survey Methodology & Compliance Best Practices: Predicted Duration, Qualtrics XM: The Leading Experience Management Software. Available at: https://www.qualtrics.com/support/survey-platform/survey-module/survey-checker/survey-methodology-compliance-best-practices/?utm_medium=ExpertReview&utm_source=product&utm_campaign=PredictedDuration&utm_content=#PredictedDuration (Accessed: 20 November 2023).