On 4th June 2014 at the Birmingham Rep Theatre, I felt a bit like going to Cannes for a sneak preview. If Cannes is one of the events to be at for films, for surveys and student evaluation, the HEA Surveys for Enhancements conference is THE one to be at. Admittedly, without the red carpet though!
This year’s hot topic was the review of the 10-year old NSS (National Student Survey) by Professor Janet Beer‘s opening keynote as current chair of the steering group for the National Student Survey (HEPISG).
From Professor Beer, we learnt the current state of the NSS review: the NSS is staying with some rewording and modifications (it has been 10 years since the NSS started!), and the consideration of including questions on student engagement and academic experience. The 3 purposes of NSS remain valid (Quality Enhancement, Quality Assurance and public information). As Professor Beer observed, there was ‘limited appetite for change and support for retaining most questions’; the timeline of changes is as follows:
- Autumn 2014 – cognitive testing of new NSS survey
- Spring 2015 – piloting new NSS survey
- 2017: first new NSS survey
Prof Beer offered a teasing taster of a data analysis tool that will be available on the 10-year data set of the NSS, the full data will be available for analysis (not by institution) – but we have to wait until early 2015 for this! Nevertheless, it will be a customisable tool, which caused the excitement.
Taking a holistic and more critical approach, Dr Camille Kandiko Howson‘s keynote was on “Metrics that Matter: Student Expectations and Effective Educational Practices”, pressing the point that it is the process of student evaluation – the ‘who does what’ and ‘for what purpose’ with the data – that is more crucial than the ‘what data’ is collected. I loved Camille’s analogy of deep/surface learning applied to the way institutions deal with survey data. She argued that many institutions take a surface approach to using their survey data: trying to climb up on the league tables rather than the deep approach: really engaging with and thinking about their data well and how they could lead to enhancement.
My highlights from the parallel sessions were:
* Hearing about an interesting initiative from next door: LJMU‘s Dr Elena Zaitseva talking about how they have changed their module surveys to be much more concise: 4 quantitative and 4 free text questions only, focusing on student learning and development. This led to a discernible shift in free text responses from satisfaction to student reflection and learning in students’ responses.
* I didn’t see the session by Dr Sarah Townley at the Liverpool School of Tropical Medicine on their use of focus groups for student feedback, but it resonates with our Vet School’s student feedback system based on focus group methods, which has been very successful and which Ellen Singer will talk about at this year’s Learning and Teaching Conference. One to get the slides from!
* I also picked up as a skill from Dr Metaxia Pavlakou, Oxford Brookes University, who led a workshop session on cognitive interviewing to test survey responses. It is something I have done before piloting a survey but it was nice to consider and learn about the ‘official methodology’. A good book link given was Cognitive Interviewing by Willis (2005).
Another initiative I have been keeping an eye on is the UKES (UK Engagement Survey) pilot. UKES is the UK-adaptation of NSSE (National Student Survey of Engagement, US) which is currently being piloted at 36 institutions across the UK. Once piloted, it will be made available from 2015 to all institutions by the HEA. The purpose of UKES is quality enhancement – its data are never publicly available, they are for the institutions to make use of.
The panel discussion about the ‘Future of surveys’ was a stimulating debate including the keynotes, students, academics and HEA staff. At times I had felt opinions waged a battle between a Hollywood blockbuster aimed at satisfaction (the NSS) and a thoughtful independent movie (NSSE/UKES aimed at student engagement). But it was not that simple. There was a range of views from commitment to the NSS as a tool for helping the student voice through to experiences of it being used as a ‘stick to be beaten with’. The audience was pointedly asked: “How many of you know the departments with the worst NSS scores?” “And how many of know you know those with the best scores?” (Has anyone got any praise?). Views also highlighted the importance that the NSS should never be the only data collected: “Triangulate!”. The importance of the process of student evaluation (what happens with the data) was stressed again. Audience questions also suggested at the institution’s service there are many other means – focus groups, well-working student rep systems etc. – of eliciting student feedback and ways of responding to them – at their centre being systems and processes built on constructive dialogue between staff and students.
Tünde Varga-Atkins, eLearning Unit
- Presentations and resources from the Conference
- More information about the UKES pilot;
- If you want to hear more about what went on in the conference- check our the twitter back-chat on Twitter feed of the conference #s4e2014 (Storify);
- The HEA also highlighted another recent report that explored the academic experience of students, especially in the context of the new fees-regime: the HEPI-HEA student experience academic survey 2014 report.
Researching the learner experience
Inaugural Symposium organised by ELESIG North West
Thursday 15 May 2014, 12.00 – 16.30
Are you interested in sharing knowledge and practice around
researching the learner experience and technology?
ELESIG North West is a regional group aiming to
promote networking and the sharing of good practice
in learner experience research within a collegial setting.
Come and find out more at our inaugural event.
Speakers include Dr Paul Ashwin, University of Lancaster.
Paul will lead an interactive session on researching the learner experience
to inform teaching and institutional practices and add to the existing literature on this topic. During the session, you will have a chance
develop your own idea and turn it into an action plan for research.
Venue: G-Flex Room, Central Teaching Laboratories (building number 802 on campus map), University of Liverpool, Liverpool, L69 7ZJ
Register: It is a free event open to staff at North West England HEIs/FE colleges. You can register at http://www.eventbrite.com/e/elesig-north-west-researching-the-learner-experience-tickets-10988965291 (For those outside the North West, we will post details of the event after.)
Full programme details: ELESIG North West Symposium flyer.
Contact: Tunde Varga-Atkins, email@example.com
ALT (Association of Learning Technology) is 20 years old, so this year’s conference (ALTC-2013) themed as ‘Building new cultures of learning’ was marked by firework celebrations at the lovely green campus of Nottingham University. The 3 conference days were well spent networking, presenting our work and observing what others are up to in the field of learning technology in UK and global HEIs and other educational institutions. Naturally, with the 20 year-anniversary came an element of reflection.
One of my highlights of the conference was seeing one of our very own academic member of staff, Dr Tim Bullough (Engineering), amongst the presenters. We have so many innovative good practices at Liverpool worthy of being shared with others. Tim presented Kritikos, a customised visual media search tool for students, in front of a captive audience (see our earlier blog post on Kritikos being presented at a uni e-learning network meeting).
The main flavours of the conference were: institutions presenting data and insights on their minimum VLE requirement programmes, such as Jess Power from Huddersfield on ‘Blockages in relation to VLE use’, or Suzanne Wright from Nottingham on ‘Maximising Moodle’. Lecture capture was another much-spoken-of topic, typically presenters sharing their experiences with institution or department-wide lecture capture implementations, such Leonie Sloman from King’s College London’s medical school using echo360 who collected some impressive rigorous data both from students and staff on their preferences and experiences with it; or Ben Steeples from University of Essex, who shared with us a fast-pace whole-scale implementation of a lecture capture system within 30 months, using Panopto as their chosen system. Many universities in the audience had institution-wide lecture capture systems. Ben pointed out a legal guidance document by JISC on recording lectures. Essex will be looking at guidance on writing rights management and license awareness of staff and implementing a more gradient way of guiding rights of recordings from staff opting out from recordings through to enabling the creation of OERs and anywhere in-between. I also went along to a demonstration of Lecturetools (an echo360 company) by Prof Perry Samson, USA, who demonstrated his use of this as an in-class tool. An impressive aspect of it was not only that in-class polls can be asked after slides, but also the fact that each student can add their electronic notes/marks on any slide which are recorded for each student and can be retrieved by them later – saving on having to make notes on paper!
Of course the conference couldn’t happen without the mention of MOOCs, which others have documented amply, perhaps just to mention an award-winner example by Nottingham Uni who have developed their own ‘NOOCs‘ (N for Nottingham, rest is the same as in MOOC), the one on sustainability has won the prize.
Talking of award-winners, each year a ‘learning technologist of the year’ is announced, with Sheila McNeill from CETIS winning this year’s title amongst warm support of the ALT community. The Best Proceedings paper was won by Richard Osborne and colleagues at Exeter Uni on ‘Integrating technologies into ‘‘authentic’’ assessment design: an affordances approach‘.
A useful session was given by Lesley Gourlay, Martin Oliver (and sorry, I forgot the third presenter’s name!) on how to get your research published in the Research in Learning Technology journal – if anyone is interested in this contact me (Tunde) for more details. It’s a great journal to be published in: it is a high quality peer-reviewed, open access and free journal! See also two of our staff’s recent publicatons in it:
Susanne Voelkel’s (School of Life Sciences, University of Liverpool) on “Combining the formative with the summative: the development of a twostage online test to encourage engagement and provide personal feedback in large classes” and
And finally, last but not least, I wanted to mention our own presentation by our eLearning Unit team, which Phil Walker and I (Tunde) presented on everyone’s behalf entitled ‘eLearning Unit, can we help you?’. It was about our team development. The session was well attended – people were buzzing with activity and we got many positive comments afterwards. A separate blog will follow on this shortly.
So the overall impression of ALTC? It is definitely a useful forum to benchmark in our role as learning technologist. I just wish it wasn’t so expensive as its high costs is getting preventive wiping out one’s annual conference budget! And one of the best unexpected bits: not only meeting and getting to know national and international colleagues but also having a chance to catch up with one’s very own colleagues (yes, Peter Reed and Tim Bullough!).
by Tünde Varga-Atkins, eLearning Unit
Further links and resources
- ALTC2013-related blog posts
- Peter Reed’s (Health and Life Sciences, UoL): conference blog post
- Conference tweets #ALTC2013
Tools mentioned at the conference – may be worth checking out?
Imagine hovering your smartphone camera display over a poster on a wall and seeing a video appear on screen related to the content of the poster. Augmented reality (AR) works in this way. See the video below for an example. It merges content created virtually (videos, images, animations, graphics) with real world environments. This virtual content is layered over a real-life object, whether that is a picture, a person or even a building, the options are endless. Viewed through the camera display on a smartphone or tablet device, these AR ‘overlays’ can be triggered by a GPS location, a sound, or through the recognition of an image (usually a photograph, graphical display, painting or poster). This is dependent on the application used but if content is synchronised, using an AR app, then content can ‘overlay’ information, for example, onto a famous landmark, or help someone to navigate around an a city area.
Newspapers, magazines and advertising agencies have already made use of AR. I’ve seen one app used to animate logos and text displays in the headlines of newspapers. It is fun seeing an advertisement in a newspaper come to life as an animation, however, this use of AR appears, in my opinion, rather gimmicky and the ‘fun’ effect was one that quickly waned. In terms of using two forms of media together, I think that comes down to preference. Certainly in my own experience I’m unlikely to use my phone or tablet for AR purposes whilst simultaneously reading a newspaper. AR in the context of entertainment will inevitably face practical questions as to why audiences should use AR in conjunction with other firmly established forms of media. Will the technological development of AR echo the anticipated rise and now more recent fall of the 3D TV? I do believe AR has a place that, once identified, will integrate it into certain elements of everyday practice. My hope is that this practice includes education and the interactivity found in learning environments.
AR in Education
It’s hard to know where to start when choosing an app to work with AR. A brief search through the Google Play and online App stores reveals an explosion of AR apps all focusing on different types augmented experiences. This ranges from live virtual views of someone’s location, using map software, to producing music using a hand-drawn piano layout.
After trying a lot of different AR apps, I came across Aurasma. Aurasma is a smart device app that was demonstrated by Judy Bloxham at the e-Assessment Association hosted by Manchester Metropolitan University in November 2012. Judy’s involvement in an AR project, supported by JISC and their RSC Northwest Network, is powered by Aurasma and has utilised AR for the purposes of enhancing educational content. The project compiled a series of posters that aided formative assessment and became interactive when students used the Aurasma app alongside them. This helped students to engage in a reflective thinking process drawing upon some of the key principles they had covered in their learning of the subject. Using AR and interactive content like this can also provide an ideal entry point for a learning activity to begin – with options to break down content into a series of options that students can choose to follow.
A good example is outlined in the poster below (provided by Judy Bloxham), allowing the user has to use Aurasma to interact with a series of buttons and videos to see which is the correct shelf for storing raw meat in the fridge.
[Link to the original website for this poster: http://tinyurl.com/dxhnbhm]
You can use Aurasma’s image recognition tool and see how this poster can trigger interactive learning content. This can be tested on a computer screen or using a printed version of one of the posters linked below. There are two ways to do this depending on the smart device that you are using. Some instructions are outlined below.
1. Visit the Google Play store to search and download Aurasma
2. Open Aurasma and press the ‘A’ icon at the bottom of the screen
3. This takes you to the Explore section. Press the magnifying glass at the bottom of the screen to go to the Search page.
4. Type in RSC Northwest to find the RSC Northwest Channel. Click the image and then click the ‘Following’ button on the next screen.
5. Now press the bracket icon [ ] on the bottom line to return to the camera view. Then view over one of the posters below.
1. You can download Aurasma in the same way as above (for the Android devices) and follow the options to connect to the RSC Northwest channel.
2. Alternatively, search RSC Northwest in the App Store and install it to your Apple device.
3. Use the camera view and hover one of the posters below. Make sure the whole of the poster is displayed on your smart device.
[Link to the original website for this poster: http://tinyurl.com/dxhnbhm]
[Link to the original website for this poster: http://tinyurl.com/dxhnbhm]
These are examples of how an image can trigger an Aurasma ‘aura’. Open one of the PDF files to full screen and hover your smartphone device over the image. If the app has joined the channel correctly the image should trigger the interactive ‘aura’ content. Do let us know in the comment box below about your experiences using this app.
To read more about the use of AR used in education then visit the following sites below. These were recommended by Judy Bloxham, who I would like to thank on behalf of the eLearning Unit here at the University of Liverpool, for allowing us permission to make use of the above posters, demonstrating the use of Augmented Reality in education. Her work in raising awareness about the potential benefits of AR in education is ongoing and well worth following. Below are some further links related to the information above and to other AR projects, if you would like to delve further.
Scarlet Project – http://teamscarlet.wordpress.com/
cARe Project http://blogs.city.ac.uk/care/
Living Learning: Plumbing from Kendal College https://itunes.apple.com/gb/app/living-learning/id549156508?mt=8
written by Phil Walker
Some thoughts on our first VITAL online training session – Weds 20th Feb
The eLU have just successfully trialled offering VITAL training for University staff via a webinar. After recently attending quite a few excellent webinar sessions run by national bodies (JISC, ALT, TechDis) and having run a couple of successful webinar ventures nationally with Ed Dev colleagues last year, we could see some very valuable potential in using this format for our in-house learning technology training, namely:
- No need for people to trek across campus to a training room – just access from your office PC – in fact from anywhere.
- Shorter sessions are now possible on some of the lesser-known but hugely useful tools and facilities in VITAL (in this instance Rubrics cards – see note at the end of the post).
- Sessions are recorded for anyone to be able to access any time after the session.
- The ‘chat’ text box records the discussion and the questions from the session.
- Give staff experience of the webinar format and so consider how it could be useful in their own practice.
We can see multiple applications of the webinar format, particularly as the campus expands, our international partnerships go from strength-to-strength and more CPD courses are offered online by schools and departments. From our own point of view we are always looking for new ways of reaching as many people as possible, and to develop ways in which to deliver our teaching and training, and to pass on our experiences of doing this.
What is a Webinar? The webinar format is an online session which is joined from an office PC and all that is needed to participate in the session is a pair of headphones for your computer and an internet connection. A microphone can also be useful but not essential as text ‘chat’ is mainly used for side-communication in many sessions. The session comprises the elements of a face-to-face session but run virtually, so depending on the presenter and the session there will a video stream, an audio stream, interaction via either of these and/or a chat box. Interactions such as polls and quizzes can form part of the session. The whole can be recorded and edited and offered for general access afterwards.
So some initial thoughts and reflections on using webinars in this way include:
- It is quite scary as a presenter the first time you run a webinar (click image left for face of fear)! Be prepared for this!
- You need a co-pilot to manage the chat room and pick up on questions, problems, conversations developing and so on (see image at the bottom of this post).
- Good pre-session materials are needed to help people understand the concept of a webinar and the environment you are going to use, as much as the content of the session itself.
- Rehearse the session you want to run in as much fine detail as you can – you will find that some of your planned interactive items don’t quite work or need better guidance materials on what attendees have to do in the session.
- It is a very positive and satisfying experience to run a webinar session and understand how, in the absence of face-to-face interactions, the toolset available in the conferencing software offers a different kind of personal contact and interaction.
We tried out a number of ideas for ways in which attendees could interact, from running simple polling questions and discussing the results, to asking people to switch to a different browser and access a VITAL module to try out what had just been demonstrated. All of these were managed with varying degrees of success but there is very little we wouldn’t do again.
If anyone would like to watch the edited recording of the session on marking by criteria with the Rubrics facility then please do get in touch with the eLearning Unit. If you would like to find out more about using the Adobe Connect system for running such a session yourself and to get any advice on how to run a webinar then we would be delighted to share our experiences in more detail with you. As we said at the start of this post, there are multiple applications for this kind of session and we are very keen to explore these with you.
Tünde and Dan
About VITAL Rubrics: Interactive, electronic versions of criteria sheets can be created, attached to most assessable items in a VITAL module and used for marking. Cards can be used to generate a grade or can be used qualitatively, they can be made visible to your students before they submit their work or after, and they allow for extra written feedback if required for every level on each criterion. Further, criteria cards can be shared amongst marking teams, as well as adapted, re-purposed and re-used.
Attending our annual professional conference, the Association for Learning Technology Conference (ALT-C), was special for two reasons this year. For one, this was the first time I was involved in presenting, and secondly, I was co-presenting with Jaye McIsaac, our colleague from Educational Development (see Jaye’s blog entry of the even here).
Our presentations involved a student evaluation method which we have been using successfully for curriculum and module evaluation for a number of years now. The technological relevance was that we tried a number of technologies to see whether we can improve on the process, called the Nominal Group Technique. (Slideshare links here to our presentation & demonstration.)
One of the most useful aspects of the conference is to be able to chat to other institutions and see what they are up to. Having seen plenty of engaging presentations, credit must go to our academic staff at the University of Liverpool, many of whom who could have been there presenting their innovative e-learning practices at ALT-C. In fact, if you are one of them – do consider presenting at ALT-C in 2013! The eLearning Unit can support you with the process.
My personal highlights
My highlights included (hard to keep list short):
- Eric Mazur on active learning – covering how to work the lectures so that they reduce gender bias, ‘confusion’ as a sign of learning (i.e. if students are confused, it is a good sign that they are learning), and the ‘usefulness’ of demonstrations in science lectures.
- Attending and presenting together with my colleague, Jaye McIsaac, as this brought a fresh perspective on what ALT is and why it’s useful.
- The Digital Literacies projects presentations, including a session by Helen Beetham and others on ‘Tools of the Digital Trade’.
- Getting to know the Pecha Kucha format (at ALT, it’s 9 slides with 45 seconds each).
- A presentation on ‘session capture‘ , with the idea that through extending the naming from ‘lecture capture’ to ‘session capture’, comes the extension possibilities of using recordings in many more settings, such as induction activities, aiding a more institution-wide take-up of the practice. (Loughbourough has Echo360.)
- The London JISC project, Generation 4.5, which uses virtual patients through multi-branching PBL (problem-based learning) scenarios.
Tools & tips
As usual, there are always tools or tips to pick up. Just out of interest, the trendy tool of the conference was the Instagram app that everyone seem to be using (see first image of this blogpost).
- ScoopIt – a curation service which colleates content from blogs and other web resources on a given topic, e.g. see a ScoopIt for ALTC2012.
- Articulate Storyline – an (according to the presenters) exciting e-learning authoring software which has Flash-like capability for animations.
- Twitter tool – which combines twitter feeds in a PowerPoint.
- Mobile Xerte
- Khan Academy: free videos on various topics.
- Clapometer: a tool that measures the applause level of an audience (one lecturer uses it as a fun alternative to a clicker indicating best response from students).
- YouTube wraps: a tool that allows you to tailor an existing YouTube resource.
and finally, the awards:
I have never made it to the ALT Gala dinner before. It was inspiring to see the best of the profession receiving awards and recognition for their work.
One of the Learning Technologist of 2012 awards went to a close neighbour, Philip Taubman, Lancaster University, for his work on Open Educational Resources and their VLE. The runner up team award (joint second place) was awarded to the SCARLET team at Manchester for ‘their excellent development and implementation of the pioneering SCARLET Augmented Reality (AR) toolkit’ – enhancing learner engagement with artefacts and bring library special collections to life – and for Skills@Library, University of Leeds, for ‘their outstanding development and implementation of internationally recognised open e-learning resources’.
At ALT-C, prizes are also awarded for the Best Proceedings papers, which for the second year in a row was given to Tom Cochrane, this time for his paper on “Secrets of mlearning failures: confronting reality“. The two best Pecha Kucha Presentations were: “Transforming Learning Technologists into Design Researchers“ by Brenda Bannan, and “Engagement by stealth: Can a PG Cert get teachers excited about tech?“ by Lindsay Jordan who dressed as a miner for this: ‘at the coalface’.
So overall, a really good conference – and here at the eLearning Unit we have hatched plans for presenting at ALT-C 2013 as a team.