This is the first in a series of posts on the evaluation work the eLearning Unit is conducting this academic year, investigating the institutional impact of our VLE module standard, the VITAL Baseline, amongst staff and students. We’ll begin by introducing some of the background and some thinking about what we want to evaluate. As Christmas is close adjacent to writing this I’m unashamedly also going to somehow try and stir up a mildly seasonal metaphor – cake mixtures and consistencies – which might rise and fall over the course of the post. This is not going to be entirely awkwardly folded-in because a key idea that crops up repeatedly is the notion of consistency of provision. The eLearning Unit played a significant role in the development, implementation and promotion (weighing, mixing and baking?) of the Baseline, and so one year in to its full roll-out we are keen to find out how it is has been received.
The VITAL Baseline is the University of Liverpool’s standard for all modules in our VLE (VITAL) which we launched for the academic year 2014-15. The VITAL Baseline is six key areas of content and information students would most like to see in all modules. You can read about it in more detail here, get a ‘How to’ guide here and watch our students describe what it is and why they value it here. The initial impetus came from the student body and concerns that there was uneven use of the VLE across the institution so that some modules were highly developed whilst others might have very little or nothing in them. The Guild of Students’ report ‘Make the Most of I.T.’ (2013) made the following recommendation, based on survey and focus group data:
Policy Recommendation 1: The University of Liverpool pass and implement a policy requiring all academic modules to have a presence on VITAL with agreed material available through it. This material should remain accessible for students throughout the course of their degree. At the most basic level this should include: i. A presence of all modules on VITAL ii. Module specification uploaded to VITAL iii. Lecture notes uploaded to VITAL iv. Reading lists uploaded to VITAL v. Past exam papers uploaded to VITAL where appropriate
It would have been useful at this point to have made an institutional evaluation of practice around the VLE. This would have provided benchmarking detail on the ways and the actual extent to which staff were using VITAL modules before the Baseline was introduced, and also where there appeared to be no or little engagement with this online space, why not, whether staff were using other tools and why. We might also have gained valuable information on interesting and scalable good practice and this kind of benchmarking is definitely something to recommend for future TEL initiatives at the institutional level. Unfortunately we do not have the resource to tackle this retrospectively although we are thinking about at least a sampling strategy to see what was happening in modules before the Baseline arrived. It might also be a question for our focus groups/interviews whether any staff felt that they had examples of their own good practice overlooked at the time which could have been part of the thinking about the Baseline mix.
In response to the Guild report, the VITAL Baseline was one of the first major steps for the University’s then-new (2013) Technology Enhanced Learning strategy, and probably the most visible so far. Interestingly VITAL modules have had a default template since the introduction of the VLE ten years ago and this suggested a basic structure and content to include, not vastly dissimilar to the new standard, with the accompanying rationale being a relative freedom in the way that the VLE was used (within the limits of the systems) and this suggested structure was as open as possible to reflect that freedom. A policy focus on formalising a standard basic structure to be met in all modules and communicating this expectation to the institution was a positive, useful and timely process. One interesting piece of work here is to map the previous template and its rationale against the new Baseline standard and its rationale and examine the points of difference and whether these signal a shift in the ways that the VLE is viewed institutionally and further whether such a shift (if it exists) trickles down to staff and students or meets the pre-existing attitudes of both groups. The current University TEL strategy (2013) states:
“As we develop new and innovative approaches to structuring technology enhanced learning, it is clear that students would value consistent use of VITAL across all modules that they study and they need to feel that they are getting a similar experience to their friends studying on other programmes. This could be seen as part of the contract that the University is developing with current and potential students, describing minimum standards in the use of VITAL.”
I think we need to look carefully at what might be the underlying assumptions about what the space in a VLE means in both rationales. Is there a view of the VLE that it is a functional, administrative space in which student-defined expectations can and should be met and is there a view that the VLE is just one tool with innovative or useful pedagogic potential from the range of which teaching staff can deploy as they see suits their requirements? Are these views opposed or even false extrapolations from the stated rationales? I would want to find out from staff whether they felt that a Baseline came down on one side or another or if this simply isn’t an issue. Is it the case that something like a Baseline was always waiting to burst out of the specific VLE system that we use, given the structures and activities it appears most obviously to support? That is, has it always encouraged a particular way of thinking about teaching with learning technology by its design and its toolset? For staff who have not previously engaged with the VLE, for whatever reasons, now that is there an expectation to fulfil a certain level of engagement with the VLE is there an underlying or implicit pedagogical model determined by a standard like this which fixes their view about how learning technologies can be designed into their learning and teaching practice? If there is an implied underlying model does it in some subtle way set a course for the institution’s view of learning and teaching more generally? There have been attempts to alternatively frame such standards in customer satisfaction terms but is this to undersell or miss the pedagogic implications and can we find any evidence of this? Also, we need to look at the individual elements of the Baseline and what they represent in part and in whole in terms of a way of thinking and acting in the VLE space.
I’ve drifted waywardly but I hope not uninterestingly from the starting point of this post, and abandoned any metaphorical acrobatics, perhaps mercifully. I’ve tried to begin rummaging around the notion of “consistency” given its prominence in the rationale charging the VITAL Baseline. What is this particular kind of consistency, what is its value and desirability? Is it constrained to comprising certain information components or is it encouraging a consistency of thinking about and designing learning with technologies including the VLE? These questions are one small part of the evaluation work we hope to carry out but in many ways they will be the most critical when considering what the future shape any Baseline should take and to unearth the full impact of this new approach to the VLE.