Summer School – Technology Enhanced Learning sessions 2018

Summer rolls round into view and everyone’s perennial favourite, the E-learning Summer School bursts into bloom. This year we are nurturing Curriculum 2021’s Liverpool Hallmarks in our verdant garden of workshops and network meetings, with a particular focus on authentic assessment and active learning. As well as our regular introductory sessions we’ll be looking at classroom polling technologies, getting started with Twitter for HE, tools and tips for visual presentations in lectures, introducing you to PebblePad’s assessment capabilities, and our #livunisocial network will be meeting. We’re also running a short session on the tools we have for student peer review and self assessment. Finally, it’s the end of our season of workshops on Turnitin and Blackboard Assignments which support the University’s policy on electronic submission so book in quick if you want to get some training in either of these e-submission and feedback systems.

The complete schedule follows below. For bookings please email eddev @ liv.ac.uk if there are any sessions you want to attend. Full descriptions for each session are available on the booking page here. You can also book from this page where clicking the ‘book here’ link generates an email with the subject filled in with the workshop you want to attend.

2018 Summer School Schedule

Wednesday 6th June (10:00 – 12:00) – VITAL Essentials – Beginner’s practical session on using VITAL also covering the VITAL Baseline, copyright and accessibility.

Monday 11th June (10:00 – 12:00) – PebblePad – an introduction. Pebblepad is the University’s online system for a range of activities
such as portfolios and student placements. This session will introduce users to the main features and identify different ways the system can be used.

Monday 11th June (13:30 – 15:00) – PebblePad for assessment.

Tuesday 12th June (10:00 – 12:00) – Social Media Round Table #livunisocial. Our popular network forum for staff using social media to support and enhance the student journey in all its forms. We’ll bring together examples of good practice from across the University, offer practical tips and advice, as well as exploring new tools and approaches.

Tuesday 12th June (13:30 – 15:30) – Engaging Learners visually in lectures – tools, tips and tricks. Examining principles of communication and good design for the presentation materials we use in lectures. We will explore sharing approaches and tools that allow you to make engaging slides in PowerPoint, access Creative Commons images, and present data in appealing and engaging ways.

Wednesday 13th June (10:00 – 12:00) – The Blackboard Assignment tool for e-submission and feedback . A practical session looking at how you set up and manage coursework submissions through the Blackboard Assignment tool and how you can offer feedback electronically using the grading.

Wednesday 13th June (13:30 – 15:30) – Turnitin and Feedback Studio for e-submission and feedback. A practical session looking at how you set up and manage coursework submissions through the Turnitin Assignment tool and how you can offer feedback electronically using Feedback Studio.

Thursday 14th June (13:30 – 15:30) – Classroom Polling Technologies. A look at some of the classroom polling systems widely available and their applications. NB this session will use technologies and software which are currently not available centrally through CSD, but you will be shown low cost or free (but limited) systems which you can use in your teaching.

Friday 15th June (10:00 – 12:00) – An introduction to Twitter in Higher Education. This session will provide an overview of the tool, demonstrate examples of how it is used in teaching to support learning, event hashtags and sharing of information.

Friday 15th June (13:30 – 15:00) – Online tools for student self and peer review. Here we introduce the two main tools for student self and peer review activities, PeerMark (from Turnitin) and Self and Peer Assessment (from Blackboard).

Also running soon after the main summer school sessions are our TEL introduction sessions. These are discursive, reflective sessions, along with a good overview of the technologies, policies and strategies, rather than practical skills-based sessions run in the lab, and are a key component of the CPS programme, but open to all.

Monday 18th June (09:30 – 12:30) – An introduction to technology-enhanced learning. Looks at the technologies we have available centrally at Liverpool, their applications for learning and teaching, and highlights policy, strategy and guidelines relating to TEL. Reflect on your digital capabilities and that of your students within the context of your own discipline.

Monday 25th June (13:30 – 15:00) – An introduction to technology-enhanced learning. Another run of the above session.

All of the workshops are listed on our booking site linked to here.

The main pre-requisite for the summer school sessions (apart from the introductory workshops) is that you are familiar with VITAL and navigating your way around, but please contact us if you want to discuss the suitability of any of the sessions. You are welcome to book on as many sessions as you wish, although we would ask that if you find yourself unable to attend to unenroll as soon as possible.

Please also remember that we are able to run tailored workshops for schools and departments (with a minimum of five attendees). Get in touch with the Centre for Innovation in Education to discuss this option.

Dan

Learning World Teaching Machine – 1985.

Image source: http://www.Inthe80s.comhttp://www.inthe80s.com/toys/images/user-image-1291152512.jpg )

The University Challenge of using BlackBoard Test Tool

“Magna Carta! Disraeli! Debussy!” are usually my exclamations when watching University Challenge as I desperately attempt to get any question correct over anyone else watching with me.

I’d never get anywhere near a team for University Challenge, I’d like to think I’ve retired.  So I was delighted when Jane Coles, Societies Coordinator for the Guild, contacted our team for help with the trails for the University of Liverpool team.

Jane was looking for a way to run the trial through our VLE BlackBoard and to use the Test assessment tool.  The issues Jane had was:

  • To move a paper test into an online format.
  • Avoid manual marking each attempt.
  • Analyse the results of each student.

After meeting to discuss the issues, we went into the Test tool and looked at the different question types.  To get automatic marking to work, the best option was Fill in the blank option.

One of the benefits of Fill in the Blank option is it’s one of the few question types in Blackboard that allows you to set the correct answer to text that is either an exact match or contains that text.

uni challenge1

So for example, if the question was “Name colour is a ripe banana?”- you could either set the correct answer to be an exact match of ‘yellow’ being correct or contains ‘yellow’ so the student would still get the correct answer if they submitted ‘bright yellow’.

The option would also allow automatic marking, which can be viewed as soon as the student submitted their exam in either the Grade Centre or Retention Centre.

Jane went away and built a test consisting of 127 questions, varying from subjects covered on University Challenge.  Jane also manually enrolled the students onto the module and invited them to attend one of three time slots, Monday-Wednesday, where the trials would be taking place.

To avoid students potentially cheating by searching the web, Jane used NetSupport School software and locked down all the computers in the computer lab to only access the student intranet and Blackboard.

The students sat the trials and were encouraged to answer as many questions as they could during a two-hour slot.  They could leave anytime they liked but could not return once they had submitted their answers.

The trials themselves were successful, with the system not crashing or students having issues navigating through Blackboard and the Quiz.  I attended the final day and everything went smoothly.

Afterwards, Jane and I met to push the results through to the Grade Centre as exclamation marks were appearing.  This was due to the first few questions being of Essay type to simply ask availability, contact details and year studies finished.

Using the ‘Grade Questions’ option in the Grade Centre allowed us to quickly mark the Essay type questions for all student attempts on one page and finally push through the students’ score.

The other issues we had was if the answer was set as a plural answer (e.g. eggs) but the student set their answer as singular (egg) – the software identified the answer as wrong.

In the future we would need to set the answer as singular, but thankfully the score could be overwritten to be correct before marks are finalised.  Other issues to be aware of are to avoid capital lettering, set numbers as figures and text, as well as ensure apostrophes are correct or remove them.

uni challenge2

Also any questions that required two answers, if the student put “Atlantic and Pacific” but the correct answer set was “Pacific and Atlantic” it would be marked as incorrect.  So for future use, we would need to put both combinations in as the correct answer, as the Quiz tool allows more than one set of correct answers.

I found it to be a positive learning experience.  I enjoyed working with Jane and the Guild to get this project completed. I also learnt a lot of about the capabilities and restrictions the Blackboard Test tool offers.

In the time of writing this, the team has been selected to represent the University and are going through the first stages of getting onto the TV show.  I wish them the best of luck and hope Jeremy Paxman goes easy on them.

Ben McGrae

Back to school VITAL kitbag 2017

The new academic year looms and everyone’s thoughts instantly turn to VITAL, of course. It can be a chore to remember what buttons to click and what processes to follow when preparing the new year’s modules. This post includes reminders,  links and resources which we hope will make it a little easier to reacquaint yourself with VITAL. The main thing to be aware of in terms of changes to VITAL is that Turnitin GradeMark has been upgraded and given a new design and name, Feedback Studio. We are offering some short workshops and webinars on Feedback Studio in September and October and there are plenty of online resources. We are also running VITAL training sessions for people who are new to the system and again there are online support resources as well. The post below has help on:

  1. Course copy
  2. The VITAL Baseline
  3. New features 2017-18
  4. Known Issues and Bugs
  5. More help and training for VITAL

1. How do I copy content over from another module?

You do not need to re-make you modules from scratch. You can copy content from any previous module. The easiest way to copy the bulk of your content from another module (e.g. last year’s version) is to use the Course Copy facility. You need to be at the Instructor level on both the source and destination modules. Here is the latest version of our guide to the course copy process. The screenshot here shows you where to locate this facility on the Control Panel menu.

Once on the course copy page firstly in the very first section SELECT COPY TYPE make sure that you have selected the option ‘Copy Course Materials into an Existing Course’ (and not ‘Copy Course Materials into a New Course’)

Then use the Browse button to find the Destination Course ID (don’t type in the module ID) and do not  select the Include Enrolments in the Copy option (at the bottom of the copy page). There are also some useful tools and tips on course copy below this screenshot.

  • Course copy – useful Link-checker tool. See all of the weblinks in the copied module content on one page and check whether any are broken so that these can be hidden and/or fixed. Access this from the module’s Control Panel in the Course Tools section. When you click the Link Checker link the tool will start the checking process so there may be a few seconds until the results display.
  • Course copy – Date-manager tool. See all of the due dates, availability dates and date adaptive release rules for the copied module content on one page and adjust these for the new academic year (also useful if you need to make changes to the schedule of a module during term-time). Again, access this from the Control Panel in the Course Tools section. Click the Date Management link here. Select the List all Dates for Review option at the start of the process to see and adjust dates individually.
  • Course copy – Tidy up modules. Course Copy is also an opportunity to think about tidying up modules. For example, does every content area from the older module still need to be copied over? You can specify which sections you want copying and which to leave (and you can always come back and copy individual sections over if it turns out you need them after all). Thinking about the VITAL Baseline, are the sections you have copied over organised in a structure that is easy to navigate for your students and are folders, content areas and files clearly labelled? The VITAL Baseline guide below offers some advice on doing this. A word of caution though, there is no Undo facility in VITAL and if you delete anything it cannot currently be recovered.

Course copy – Video content. Something else to think about is any video content uploaded directly to your module. Firstly, please do not upload any further video content to VITAL modules, it is too large and the way that VITAL works means that students have first to download it before they can view it which can be highly problematic. Instead, where you own the copyright to the video then use the University’s streaming media service to host and to stream the videos. If the video is hosted on another streaming site (YouTube for instance) you can embed it in your VITAL module. For material broadcast on UK television you can use the Box of Broadcasts service and stream full programmes and extracts from here. For video that does not fall into these categories then please contact the eLearning Unit for help. Then for video content that is already in the module please have a chat with CSD technical team about moving your video content out of VITAL and storing and streaming it elsewhere. For advice on copyright of video please see this library guidance site.

Copyright

To ensure you remain copyright compliant in your teaching it is worth noting that before uploading anything to VITAL you should stop and think about the materials you want to upload.  Do you have permission to do so, or do you need to seek permission?

Important! Scanned journal articles/book extracts. 

The University of Liverpool holds a licence from the Copyright Licensing Agency (CLA), which allows staff to scan either ONE WHOLE CHAPTER from a book / ONE WHOLE ARTICLE from a journal issue OR 10% of the whole publication, whichever amount is greater, from permitted copyright works and upload them to VITAL in order to support teaching and learning.

BUT you have to comply with the terms of the licence for each item uploaded to VITAL and show you have done so by prefacing each scan with a Copyright Notice.

There are 3 options for ensuring you meet the terms of the Licence:

1. Complete the online Digitisation Request Form – which will automatically check the terms have been met.

  • If copyright cleared, the Library will scan the item and return it to you with the required Copyright Notice.
  • If you already have a pdf of the requested chapter/article you can supply this with the form and the system will attach the Copyright Notice to your pdf for you.

Or

2. Use the Library’s Digitisation Service which will automatically check materials for copyright-clearance, digitise the materials and embed the scans within Reading Lists @ Liverpool (contact your Liaison Librarian to find out how). You may also use the link provided to the scan and Copyright Notice in VITAL.

Or

3. Complete a Copyright Notice Form – to show YOU have manually checked the terms have been met. You need to send this form to your Liaison Librarian for reporting to the CLA.

For details on the CLA process, digitisation, and for a comprehensive guide to copyright and VITAL generally this site from the library on copyright is an excellent resource. Follow the link and look at the Procedures for VITAL for a starting point. You can also contact your liaison librarian as another excellent source of help for any copyright questions.

2. What’s the VITAL Baseline again?

Once course copy is completed, check that the module will meet the VITAL Baseline. More has been automated for this academic year so that as well as the Module Overview page link and the Exam Resources section appearing by default in your module menu, you will also see that the link to your Reading Lists @ Liverpool list has been included in the default module template. This quick guide to the six elements of the Baseline shows you how to add this link and for you to check your module meets all six elements. The all-staff module VITAL help for staff includes a detailed section on the VITAL Baseline and how to meet it.

As a reminder there is also a default section in your module called ‘Stream Lectures’. If you are using Stream Capture to record Orbit-timetabled lectures then this is the section of your module where the recordings appear automatically.

3. Is there anything new in VITAL this year?

1. GradeMark redesign – Introducing Feedback Studio

The biggest change you will see is the new design for GradeMark, which also gets a new name, Feedback Studio. None of the current grading and feedback functionality will be changed or lost. There will only be small additions to the system, for example very simple formatting for text comments is introduced. Full guidance on the new design is available at the VITAL help for staff module here.

To have an introductory look at the new design, visit this interactive demo page. If you have used the GradeMark iPad app then you will already be familiar with this new design.

This video demonstrates the differences between the current GradeMark and the new design of Feedback Studio.

 

2. VITAL new features
This summer release sees some small but significant improvements to the main Blackboard system, many of which are relevant to electronic submission and grading.

A. Drag and drop file uploads. The Create Item tool now lets you drag and drop a file or multiple files to upload them to a content area. You’ll also be able to upload file attachments in this way to the Web Link tool and the Assignment tool. Students will be able to drag and drop to the Blackboard Assignment tool when submitting their file(s). Where you see this dotted line box you will be able to drag and drop a file or files for upload. 

B. Easier to email non-submitters for Blackboard Assignments. A new option for Blackboard Assignments, Tests and Self and Peer Assessments, with one click you can send a system-generated, generic reminder text to students who have not submitted, rather than selecting individual students to contact. Particularly useful for anonymised assignments where you cannot select students individually in the Grade Centre to email.

C. Needs Grading area improvement. If you use this feature, and have assignments which allow multiple submission attempts by your students, you will now only be shown the work you have specified (e.g. last attempt). This is a big improvement again for anonymised assignments to avoid marking multiple, similar submissions from one student.

D. Course Activity reports by groups. New in the Course Reports usage statistics area of the module, the report Course Activity Overview (which summarises all activity across the module for students) includes a filter which lets you select a report for a group or groups rather than all students.

E. Submit button is now always visible. Instead of chasing up and down the screen to find the button it will always be available at the bottom of your current view of the page when editing.

F. The VITAL default menu template now includes the Reading Lists @ Liverpool link automatically. If you do not use this tool you can hide it from your menu.

And a farewell to…

The old Chat and Virtual Classroom tools have finally been discontinued and are no longer available in VITAL.

4. Any problems you can tell me about now?

There is a regularly-updated Known Issues page here which also covers Turnitin and Campus Pack problems.

If you encounter a problem with VITAL which is not listed on the Known Issues page please do report this to the CSD ServiceDesk.

5. Where can I get some more help, training or online resources for VITAL?

VITAL help module.  Everyone is enrolled on the VITAL help for staff module which details how to meet the Baseline standard. If you come on one of our workshops you will also be enrolled on our workshop resources module which contains further guides, examples, case studies, research and more.

Guides. This A-Z guides page has functional help on setting up and using the toolset in VITAL. If you don’t see the guide you need just ask us and we’ll find something for you!

Workshops. We have some introductory workshops to limber people up for the new academic year (book here).

  • VITAL Essentials – a short, practical, beginner’s guide to VITAL (21st September and 27th September)
  • An introduction to Feedback Studio – Grademark upgrade workshop (19th and 26th September)
  • An introduction to Technology Enhanced Learning – a broader overview of  learning technologies at the University and planning to use these in your learning and teaching (25th October)

Watch out for other term-time offerings and the January 2018 Winter School (Mon 8th – Fri 12th).

Email and ‘phone support. Contact us for advice and help. We will also be running a bookable Studio Wednesday drop-in once we move back into our building in late October.

Is there anyone else out there I can talk to?

As well as the eLearning Unit and CSD Servicedesk (who will deal with all student queries and all technical problems like enrolments, error messages and so on) if you want help with the Reading Lists @ Liverpool tool or digital copyright then please contact your liaison librarian. There is also a University mailing list for people interested in learning technology where you can ask questions and get updates on what is happening with learning technologies around the institution; get in touch with the eLearning Unit if you want to join the e-learn.net. We also meet face-to-face at least once a term.

Dan

Blackboard or Turnitin – our EMA decision-making flowchart

The team are fresh back from the now-legendary 2017 ALT conference, and bursting to share and connect with this  fantastic community. One of the things presented by Alex, Tünde and Dan which caught people’s eye during our session on the sometimes arduous road to EMA was Tünde’s ‘Blackboard or Turnitin’ decision flow chart.

It was designed to help staff decide what were the most important factors when they first come to choose whether BB or Tii is the best fit for their context. Behind it is the usual mountain of documentation, guides, caveats, sub-clauses and subtleties, but we had a think about what were the most-often asked questions from our staff and the priority they gave different requirements and used this to help map an easier route through the terrain.

As it looked useful to a few people, we’ve tidied it up, and given it a CC licence so please feel free to grab it from here and try it out with your own colleagues. If you want to know any more about our approach to guiding staff through the BB and Tii tangle then please do get in touch.

Creative Commons LicenseBlackboard or Turnitin Assignment by Centre for Innovation in Education, University of Liverpool is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

You can also catch up with our slides from the session here.

Something else that caught people’s attention, quite literally, was our percussion department for bringing the breakout discussion parts of our workshop presentations to order. ALT delegates are incredibly eager to get stuck in to a good conversation about any aspect of learning technologies, so you definitely need some serious help on your side as a presenter if you want to get a word in again! Here’s our kit, level 1 – the bell, level 2 – the castanets, and level 3 – our last resort – the slide whistle. We went all the way to level 3.

Alex, Tünde and Dan

Black Sabbath to Busted – Report on Blackboard Enhanced Assessment and Feedback Event

Leaving home under an early-morning starry sky and up over the misty, snowy Pennines to Sheffield, I wasn’t sure what to expect from the Blackboard Enhanced Assessment and Feedback day to which I was travelling. I had a sketched outline of the themes for the day ahead but not much more, namely that we would engage in some way with:

Assessment and feedback – an institutional perspective

  • Examination of key drivers and challenges (reputations, quality of process, quality of data, legal, business efficiencies, risk etc.)
  • Placing use cases on a confidence/effectiveness model
  • Highlight relevant Blackboard solutions

 Assessment and feedback – innovation and organisational change

  • Identifications for drivers for change
  • Gap analysis of practice and stakeholder experiences
  • Prioritisation of opportunities for change
  • Highlight relevant Blackboard solutions

This kind of workshoppy day from Blackboard was something I hadn’t experienced before, so I was propelled by curiosity as much as that it seemed relevant to the work that our team are currently leading on e-submission and e-feedback at Liverpool. What I got was a useful day of frank discussion and sharing of experiences, ideas and commonalities, which was mostly reassuring, with colleagues from other institutions in the kind if depth you don’t often get. This is especially useful for a sense of the bigger picture in HE, to talk about the differently badged or described but largely similar activities, structures and strategies that are top of our agendas at the moment, e-submission and e-marking being one of Liverpool’s current strategic TEL focus. One universal and rapidly-emerging area of concern that became evident on the day was a need for a variety of programme-level views of assessment activity in the VLE for academic and administrative staff and students. This is a long-requested feature from Blackboard usergroups that’s time has come with the adoption of e-submission and e-marking policies across the sector of late and I hope this was the main takeaway message for the Blackboard team.

The event was run by our regional Blackboard Customer Success Team, in partnership with the BB North user group, recognising a need for a more extensive exploration of particular issues that get raised at user group meetings, where the format doesn’t allow fuller discussion. Whilst advertised as intended for senior leaders, learning technologists, TEL managers and academic staff, the majority attending today were learning technologist types. It was instructive to hear that on the previous day at Edinburgh a couple of PVCs had attended, sending some very positive signals about the depth of an institutions’ engagement and intent with the actual tools that students, academic and professional service staff use as a part of their everyday life at the University.

Music, sweet music…

Our first activity was to introduce ourselves telling the room the first piece of music that we had ever bought. An astonishing array of formats and first loves was paraded, from Now compilation tapes to Avril Levigne downloads, from Osmonds vinyls to Busted CDs. Blackboard’s Alan Masson and Gillian Fielding all admitted to their first purchases, but I’ll spare their blushes here. Top-trumping all these, however, was Blackboard’s Steve Hoole, whose overnight Novotel stay featured a vinyl deck (remember these, kids?) and a selection of Sheffield synth heaven albums to spin the night away.

Structure of the sessions

The morning and the afternoon were structured in a similar way so that we would first ‘brainstorm’ our thoughts in groups on a set of e-assessment themes, then work together on some specific ideas from those and bring something interesting back to the room. We’d finally end with a discussion of the potentially useful tools in Blackboard that could be a part of the thinking for some of these. I initially thought this last element was going to be a sales pitch but it was pleasingly nuanced in that the Blackboard team wanted to hear stories of how people are using these tools, where they’re working well and what the gaps are. A very clear point made more than once was that the recent Blackboard activity in developing the assignment tool to offer dual marking, moderation and anonymous marking had been excellent but it now seemed that the Blackboard focus had moved on from this, whilst the toolset still needs work, that there had only been one iteration of the process. Also that this process of close consultation should be constantly repeated for other areas as our needs are constantly evolving, not just for assessment.

Morning Session – what’s needed to enhance assessment and feedback practice?

For the morning, we’d thought about quality, processes and workflows and where the opportunities for enhancement lay. We chucked all of our ideas at the first Padlet below. You’ll find all of the issues that we have encountered in the course of our work as learning technology developers and as part of the University’s EMA project, from how to handle video submission and feedback, to combining some functionality of Blackboard assignments (group tools, double marking, staged release of feedback, etc) with that of Turnitin assignments (Originality Checking, GradeMark, offline marking). In fact, this was another big ask on the day, that Turnitin and Blackboard align/integrate their products in ways that will help us, as you will gather from some of the posts in this Padlet.

Made with Padlet

What leapt out at me was a so-far un-encountered issue at Liverpool of needing a read-only external examiner access to modules. In some institutions administrative staff are packaging up content and assignments into a special section of a module, which only the external has access to, and making the rest of the module inaccessible to them. This is to meet an anxiety around externals potentially changing grades and altering content, but it costs hours of administrative time, essentially duplicating what’s already in the module, so re-introducing at a later stage of the assessment cycle serious administrative burden that the electronic submission process had originally taken away from the front end. What’s needed is a read-only access enrollment level, which is another development idea for the Blackboard team to add to the suggestions box.

Moving on to the next activity in groups again, we were tasked with listing and describing up to five assessment and feedback enhancements that would have significant benefits for the listed stakeholders and the degree to which it would require resource. This photo shows our effort…

dsc01104

…and for those who aren’t adept at reading the handwriting of people who spend their whole day attached to a keyboard our five enhancements (all pretty standard) were:

  1. External Examiner Access – read-only access for external examiners or a similar idea.
  2. Student Assessment Journey – programme level views of student assessment activity for students and staff
  3. Flexible innovative assessment – making the assessment and feedback tools at all points in Blackboard, not just for assignments, so that you can start thinking about using any tool for assessment purposes.
  4. Double marking – further work on the current functionality to take it to a robust, fully-usable level.
  5. Programme Level Assessment – looking at assessment practice across entire programmes and thinking about programme-level learning outcomes.

Hearing back from the rest of the room we discussed in more depth some of the already described above (external examiner access, programme-level views of assessment) and the Blackboard team promised to send round some case study examples of good practice for external examiner processes using Blackboard tools. In a discussion around whether and how institutions were using the Delegated Grading functionality, which was designed for UK HEIs, again the Blackboard team said they would gather together some case studies of where these are being used well. The feeling from the room was that this kind of functionality should be available across all assessment tools rather than locked to a single tool.

A few other interesting discussion points to end the morning session were that many institutions are thinking at programme level about replacing traditional assignment assessment. Video assignments and feedback are rapidly on the rise but also causing headaches as infrastructure and policy isn’t keeping up.

Afternoon Session

As I said, the afternoon session followed the same structure. So our post-lunch digestif activity was another Padlet , this time thinking about innovation and new practices that would enhance assessment and feedback in our institutions. I’ll let the Padlet do the talking so scroll around to see the ideas. I was interested in things like students being able to select the kinds of assessment that they wanted to do, and learners and academics developing assessment literacies through feedback dialogue and feed-forward as a continuous process.

Made with Padlet

As in the morning, next was a group task, where we were asked to think “aspirationally” about how we imagined assessment could look, if we had a free rein. What change or innovation in assessment and/or feedback would have significant impact and how would it benefit learners, tutors, courses and institutions? Essentially we were encouraged to go wild in the aisles of transformative assessment practice.

Our group went Back to Basics and offered the transformative potential of programmes where learning outcomes were mapped to assessment.  Well, someone had to. Other groups had some tidy, Tomorrow’s World ideas including:

  • An assessment wizard which built the kind of assessment you wanted with one view for staff and students (no more multiple systems or at least hiding these from you).
  • A tool that surfaced programme level assessment data.
  • A tool for personalised feedback and assessment routes – feedback raises flags on further help students can get and other staff can see that in later assessments.

And as per the morning session the Blackboard team led a discussion on how their products could work to do some of these things. One thing they did bring back to my attention was the Goals and Outcomes system which has a new dashboard view of the data and I think it would be opportune to review this in the light of programme development work that is heading the way of our team, as this could present an opportunity for offering programme-level views of progress through modules.

The end

So not a sales day, not your regular product roadmap/roadshow day, this represented a deeper dive into Electronic Management of Assessment, including the Blackboard tools that can be a part of that that environment. The Blackboard team wanted the day to be about sharing practice, raising awareness of what Blackboard tools we have already and encouraging us to get the best we can out the Blackboard tools and products that you have and I think this was more than achieved on the day. Thanks to the team and to the Bb North UserGroup for arranging and hosting. I had some very useful conversations (which is pretty much usual for the BB North User Groups meetings) and plenty from all of the above to take back for the project board overseeing the implementation of an e-submission policy at Liverpool.

What music did I first buy? The Muppet Show album. And I’ve never needed any other in my life…

Dan

Report: e-Learning Network Meeting – January 2017

We were delighted to welcome Professor Helen O’Sullivan, APVC Online Learning, as speaker at the first e-Learning Network meeting of 2017. Helen spoke to us about the University’s new Education Strategy, giving the network an overview of the structures, leadership teams and immediate priorities. The recording of that talk is linked-to below. Helen then led a discussion workshop on what an institutional Digital Education vision might look like (this part of the session is not recorded). We also managed to make time for a couple of extra items: a first look at Turnitin Feedback Studio, the new design for GradeMark we will be moving to in July; and an introduction to the Go Mobile user group that began meeting this academic year. A pretty busy lunchtime for the forty staff members who came together for this valued networking event.

Professor Helen O’Sullivan

So much is going on at the University at the moment it was a welcome opportunity to spend some time thinking through and discussing how current strategies relate to our own interest area and Helen did a great job of this, even in the sweltering conditions of our meeting room. The Education Strategy’s core values, ‘Liverpool Hallmarks’, of ‘research-connected teaching, active learning and authentic assessment’ are immediately appealing to anyone interested in learning and teaching, and learning technologies can play a critical role in these. I won’t go into micro-detail but what I found really useful was an update on the top priorities for the coming year, including the setting-up of a new Programmes Development Team, a media technical support team, continued work on the Electronic Management of Assessment project, and also hearing about less familiar things including the focus on the London Campus portfolio and degree apprenticeships. Click the image below for the (Stream Capture) recording, about 32 minutes long.

bloghelenimage
Click the image above to watch the recorded talk by Prof O’Sullivan (32 minutes)

 

We then moved to some group discussions to consider a Liverpool take on David White’s digital leadership framework which is designed to help high-level discussion and decision-making about all things digital, giving some coherence for thinking about the whole organisation and how decisions can affect all of these layers. The framework diagram below is taken from David’s blog post (click the diagram to read) and was the starting point for the activity. In my group we focussed quite a bit on the Digital Service layer, which possibly reflected the areas we work in but which we felt was the bedrock of an organisation’s culture and medium.

Turnitin Feedback Studio – Dan Roberts

There was also a bit of time for a couple of extra items. First up was a look at the new design for Turnitin GradeMark, called Turnitin Feedback Studio. This was an out-of-the-box walkthrough and we were only examining the feedback environment. Essentially the desktop version has been rebuilt and the design is very similar to the current iPad app version, but now you will be able to use it on any device. This video below maps the key differences between our current version of GradeMark and what we will see after this summer’s upgrade. You can also try out a live, online demo if you follow this link.

No horses seemed to be startled by this new look. From a design point of view I think it is a much-improved, cleaner system, tidying away a lot of the distracting array of menus and buttons we are used to, and instead putting the most commonly-used feedback tools directly in front of you whilst marking work; no more hunting around for different comment types for instance. The rebuild has also focussed on making GradeMark fully-accessible which is great. Asking about what kinds of things people would be interested to test in the lead-in time to the summer upgrade, long-standing functionality/workflow requirements such as double marking were top of the list. Looking through the release notes whilst writing this post I can see that there is a Beta version of the multiple markers facility for which Turnitin are looking for some testers, so we will organise this through the network and the e-submission/EMA project board. Get in touch directly if you want to be a part of this testing.

Go Mobile Usergroup – Alex Spiers

We rounded off chatting about the new user group for anyone interested in anything mobile that Alex  has set up and has met a couple of times already this academic year. It is as wide-ranging as that sounds, so we’ve looked at apps, devices like the iPad pro and pen, and the kinds of things staff and students from all parts of the University are doing with mobile technologies for learning and teaching. Look out for the next meeting which we hope will be this side of Easter and we’ll release details ASAP or keep up with #LIVUNIGO.

gomobilemarch
Next meeting – April 27th

Many thanks to Helen for the valuable and engaging insight into the strategic thinking and work going on for the University’s Education Strategy, and the role that Technology-Enhanced Learning has to play as it moves into its implementation phase. It was also a great opportunity to have a first say on some emergent ideas around a Digital Vision for the University of Liverpool. This is an ongoing process and Helen would welcome more comments and feedback on anything covered in the presentation or discussion.

The next e-Learning Network meeting is scheduled for Thursday 27th April 12:30 – 2pm. The network lunch is intended primarily as a sharing event so if you have an idea for one of our meetings or anything you want to share about something you have been doing with TEL and to get some feedback and discussion from the group then please let us know.

Dan

Winter School 2017 report

winter-school‘Brilliant, engaging, authentic – thank you!’

This year’s Winter School, our sixth, has been the best attended of any we have run so far which was really pleasing for the team. We’ve met lots of new colleagues over the two weeks and been inspired by the kinds of things people want to do with technologies for their learning and teaching. Feedback from the sessions has given us some great ideas on how to develop our workshops and also the kinds of things people would like to find out more about, that’s why we collect it, so we’ll be scheduling in workshops on Evernote and technologies for polling over the next few months for starters. To keep up to date with when workshops are running you can follow us on Twitter (link below), follow us on this blog, keep an eye on the University Announcements page and also search the CLL Booking page here.

‘Lots of showing and sharing of experiences – colleagues sharing their use of various tools… Learning about those tools and trying them out.’

Our aim is to make the Winter and Summer schools a mix of introductory and more exploratory sessions, providing opportunities for staff to extend themselves. In the feedback we get, staff report a valued feature of the sessions is the opportunity to try out new tools and systems, to discuss the strengths and limits of these, how these can be trialed and then made a part of learning and teaching practice, and to share and learn from what colleagues in other schools and departments are interested in and are doing. We are also delighted to read about the wider impact and reach our sessions have in a range of contexts, from enriching personal practice, to staff championing in schools and departments the ideas and knowledge they gain, to applications for research groups.

‘I came to build up professional knowledge and skills for my own personal development and I think this will definitely come in useful in the future.’

‘This will quickly enable myself and other module leaders to elevate the skills of our students…’

Immediate use to help a research collaboration (2 UK sites, 2 European) meet and collaborate. I will include this system as a way of indicating how we will collaborate – thus strengthening a bid for funding. I will explore using this for T&L in a European meeting to demonstrate some of our T&L content.’

Another way we’ve responded to the feedback we get has been developing an advanced session on using Twitter, following a brace of well attended introductory Twitter sessions over the summer. We ran this in the winter school and knowing that it is crucial to include local examples of good practice we enlisted the help of Zelda Chatten, who is part of the team that supports the Library’s social media presence. Between us we were able to share practical tips and useful advice about our experiences using Twitter.

This was a fruitful partnership which extended the range of sessions we run collaboratively (we also run an Adobe Connect session with colleagues from CSD) and one which we will repeat and continue to build on, for example we are planning to develop a session jointly-run with CSD colleagues on Stream Capture.

‘The opportunity to explore some very useful tools in a supported way – I left with some great ways to make my slides more visual and engaging and will pass these tips onto others.’

It was also the first time we ran the session ‘Engaging learners visually’ in which Alex and Tünde shared a few tools that you can use to create more exciting, visual slides, including Notegraphy, Piktochart, Prezi, Haikudeck and some features of PowerPoint. We discussed accessibility considerations when putting together pictoral slides. Judging from the attendance and busy keyboard-working during the workshop, it was a lively and useful topic to cover!  We will definitely run this workshop again.

‘Hands-on guidance through the set-up, recording, stream process. Also love the well-scaffolded and visual resources laid out on the VITAL page. Great!’

If you attended one of our the Winter School sessions you will be enrolled on our Workshop Resources module in VITAL which will contain at least the slides from the sessions and usually further links and support resources on all of the sessions we run. Our Summer School will be running from Wednesday 7th June until Friday 16th June 2017 and some sessions are already scheduled in to run and are bookable here: https://www.liverpool.ac.uk/cll/booking/

‘Thank you – terrific, inspiring session!’

Thanks to everyone who came along and engaged so positively. We derive much inspiration and learn so much from all of our sessions, which we always strive to make a collaborative venture. But for now, the decorations are stowed back in the loft, the last green triangle Quality Street has reluctantly been scoffed, and the jumper from Santa exchanged for a Star Wars onesie, it can mean only one thing; the eLearning Unit Winter school has been and gone and the festivities over for another year. Hope to see you at the Summer School!

The eLearning Unit team

 

Old and New: Highlights from the Durham Blackboard Users Conference 2017 #DURBBU

Total Architecture

As I’ve said elsewhere, I’m no stranger to the Durham Blackboard Users Conference (now in its 17th year!) having attended and presented on number of occasions. However, this was my first visit in my new role at University of Liverpool and also the first visit to the new location at the Durham Business School. Very impressive venue located a little further out of town but close to the excellent Oriental Museum.  The conference began with a warm welcome from the new VC Professor Stuart Corbridge. Then we were back on familiar ground with Malcolm Murray introducing us to the conference theme (Better Assessment & Feedback) and using the Lego figures to settle us into a great couple of days.

The Keynote was delivered by Dr Susie J Schofield from the University of Dundee. Translating evidence-based principles to improved feedback practices” using the “interACT” study she co-authored. The main focus of the presentation was a review of the literature on a range of feedback models and principles (Gibbs, Nicol, Carless, Barton) and draw out some commonality. She shared the view that students can avoid bad teaching but they cannot avoid bad assessment. Stating how a shared understanding of assessment criteria is crucial to student success. I wonder how often there is the time and space in the curriculum to allow students to get a real grasp of this.  Sounds like a good opportunity for students to get involved and co-create their own assessments and criteria.

Wayne Britcliffe presented us with a useful overview of the breadth of tools being used at the University of York, to support assessment and feedback. Many of which are familiar to Blackboard users, although it was interesting to hear that they are a Google University He also shared some useful tips in and a few ‘gotchas’ This was a thought provoking and refreshing topic as it sums up the learning technologist experience at the moment. We are all dealing with similar issues and complexities of electronic assessment. This has certainly been the case for me. A deeper understanding of electronic submission and e-marking approaches have been at the centre of my working practice particularly as these methods gain wider adoption within institutions.

Patrick Viney from University of Northumbria presented a novel approach to using PebblePad V5. Not as an ePortfolio but to manage the dissertation proposal process for over 800 undergraduate students. A common problem to many institutions. The process was based on the workbook functionality in Pebblepad. This allowed students to submit proposals electronically and and select keywords to identify the topic and allow it to matched to an appropriate supervisor.  The effect of this change resulted in a speedier matching of supervisors to students. Reducing time taken from 2 weeks to 1 day! As well as time saving befits, this approach is paper free and auditable. Non submitters can easily be identified and contacted. I’m sure this process could be adapted to support a range of subjects at our insitution, and perhaps lead to greater adoption of the system.  Kudos to Partick for going live into the Pebblepad to demonstrate it. Always valuable to see it live if possible.
I’ve always been a big fan of screen-capture and I use screencast-o-matic a lot these days, since Screenr passed away a few years ago. I’m also very interested in visual learning, so I was keen to hear how University of Reading Lecturer, Emma Mayhew, used this technology to enhance student assessment.  Using Camtasia she developed a range of study support videos which quickly gained popularity, so she ended up creating more and using other tools such as Powtoon to get the message across.  Then applied this to her feedback on assessment. The video feedback reminded me of the useful qualities of audio feedback. Students get more feedback – quantity and detail, but also feed-forward suggestions about what can be done better next time. Staff demonstrate empathy and encouragement more clearly, but also visually identify issues where the student needs help. The process is necessarily efficient for the lecturer but is overwhelmingly receives positive responses from the students. An excellent presentation from an enthusiastic academic making positive changes.

Finally,  it has always been a bit of a tradition for Blackboard to share new advances in their technology at Durham. Nicholas Matthij was excited to share the work he’s been doing on a new product called Ally. This is another Blackboard acquisition and looks to be platform agnostic. The functionality that Ally provides is to make  course content within the VLE more accessible. The tool guides academic staff on how they can make their content more accessible and offers alternatives while keeping the original files. Clever eh? In addition it also has an interesting analytics dashboard that could make it easier for staff to recogise popular content and patterns of use. Developed in part as a response to the changes in legal requirements,  Blackboard will make this service available in Q2 2017. Sadly this will not be appearing in the product as standard but can be purchased separately.

Anyone that is using Blackboard or its associated products would be well served in attending this conference. I know I always have been. Its not just because I keep winning prizes (iPad mini in 2012, £50 amazon voucher this year), although that helps! The community is the crucial element of the conference. Enthusiastic people presenting, sharing and openly talking about issues they care about to help change and make things better. Now that sentiment never gets old.

Other blog posts
 Alex

2017 New Year – back to VITAL!

A slightly belated Happy New Year to everyone and in this truly chilly January we hope this post will help defrost VITAL for you, ready for the new term. We’ve put together a checklist with follow-on links and resources on:

  1. Finding a missing VITAL module
  2. Course copy (copying module content from previous/other modules)
  3. The VITAL Baseline
  4. E-submission and e-marking support
  5. Was there anything new in VITAL this year?
  6. Known Issues and Bugs
  7. More help and training for VITAL

1. I can’t see my module in VITAL! Where is it?

If you log in to VITAL and cannot find the module(s) on which you are teaching this coming year in your module list then for most standard taught modules click here for a checklist on how to resolve this problem.

2. How do I copy content over from another module?

The easiest way to copy the bulk of your content from another module (e.g. last year’s version) is to use the Course Copy facility. You need to be at the Instructor level on both the source and destination modules. Here is the latest version of our guide to the course copy process. The screenshot here shows you where to locate this facility on the Control Panel menu. Once on the course copy page do use the Browse button to find the Destination Course ID and do not  select the Include Enrolments in the Copy option (at the bottom of the copy page). There are also some useful tools and tips on course copy below this screenshot.

Course copy – useful Link-checker tool. See all of the weblinks in the copied module content on one page and check whether any are broken so that these can be hidden and/or fixed. Access this from the module’s Control Panel in the Course Tools section. When you click the Link Checker link the tool will start the checking process so there may be a few seconds until the results display.

Course copy – Date-manager tool. See all of the due dates, availability dates and date adaptive release rules for the copied module content on one page and adjust these for the new academic term (also useful if you need to make changes to the schedule of a module during term-time). Again, access this from the Control Panel in the Course Tools section. Click the Date Management link here. Select the List all Dates for Review option at the start of the process to see and adjust dates individually. 

Course copy – Tidy up modules. Course Copy is also an opportunity to think about tidying up modules. For example, does every content area from the older module still need to be copied over? You can specify which sections you want copying and which to leave (and you can always come back and copy individual sections over if it turns out you need them after all). Thinking about the VITAL Baseline, are the sections you have copied over organised in a structure that is easy to navigate for your students and are folders, content areas and files clearly labelled? The VITAL Baseline guide below offers some advice on doing this. A word of caution though, there is no Undo facility in VITAL and if you delete anything it cannot currently be recovered.

Course copy – Video content. Something else to think about is any video content uploaded directly to your module. Firstly, please do not upload any further video content to VITAL modules, it is too large and the way that VITAL works means that students have first to download it before they can view it which can be highly problematic. Instead, where you own the copyright to the video then use the University’s streaming media service to host and to stream the videos. If the video is hosted on another streaming site (YouTube for instance) you can embed it in your VITAL module. For material broadcast on UK television you can use the Box of Broadcasts service and stream full programmes and extracts from here. For video that does not fall into these categories then please contact the eLearning Unit for help. Then for video content that is already in the module please have a chat with CSD technical team about moving your video content out of VITAL and storing and streaming it elsewhere. For advice on copyright of video please see this library guidance site.

Copyright – scanned journal articles/book extracts. One more thing to check for is the copyright status of materials in your module, in particular what to do about journal articles and extracts from books which you have scanned/have previously been scanned from a hard copy and then uploaded to VITAL. If you do have items like this and they have not been uploaded by following the CLA clearing process detailed here (click on the ‘Procedures for VITAL’ section) you need to ensure that these items comply. You can make things easier by using the library’s new digitisation service and offering the resource through Reading Lists @ Liverpool (click the ‘Digitisation Service’ link on the same page). Your liaison librarian can help you with this.

3. What’s the VITAL Baseline again?

Once course copy is completed, check that the module will meet the VITAL Baseline. The Module Overview page link and the Exam Resources section now appear by default in your module menu. You will still need to manually add a link to any Reading Lists @ Liverpool list you have set up for a module. This quick guide to the six elements of the Baseline shows you how to add this link and for you to check your module meets all six elements. There is also a module in VITAL on which you are enrolled ‘VITAL Baseline and guidance’ which takes you through the Baseline in more detail.

You’ll notice another default section in your module called ‘Stream Lectures’. If you are using Stream Capture to record Orbit-timetabled lectures then this is the section of your module where the recordings appear automatically. If you are not then this section will not appear to students (as does any empty module section).

4. What help is available for e-submission and e-marking?

The e-submission project has a regularly-updated project page here which contains some details on how it is managed and what work is going on at the Institutional level to facilitate the e-submission policy. You will also find a list of school/departmental contacts who you can contact about local implementation and approaches. If you would like to attend a workshop where you can find out how to use the tools supported centrally, their uses, limits and feedback contexts then we run monthly e-submission and e-marking days looking at Turnitin and Blackboard Assignment tools (dates below). In the VITAL Baseline and Support module you will also find a section on e-submission and e-feedback tools and workflows, current recommendations and so on. We will also be moving to a new-look Turnitin GradeMark called Feedback Studio and will be running sessions on this in the summer and into the new academic year, as well as providing online support materials. The eLearning Unit will help your school or department plan your approach to e-submission and e-marking, run training sessions for your staff and offer ‘phone and email helpdesk support.

E-submission days.

We run three sessions on one of our e-submission ‘days’. The titles and times they run are:

  • An introduction to e-submission (09:30 – 10:30)
  • The Turnitin Assignment tool for e-submission and GradeMark for feedback (11:00 – 13:00)
  • The Blackboard Assignment Tool For e-submission and feedback (14:00 – 16:00)

And the dates they are running this year are:

  • Weds 22nd Feb
  • Weds 15th March
  • Weds 26th April
  • Weds 14th June

You can book onto any of them at the CLL booking page here.

We also have a set of student guides and videos on submitting work and accessing feedback which are held on this page and which you are welcome to adapt for your own use (please note these pages are currently being redeveloped).

5. Was there anything new in VITAL this year?

No major additions to the toolset were made in the summer upgrade, which was mostly focussed on bug fixes and improvements to the hardware running our installation of VITAL. The Retention Centre was restored and the Self and Peer Assessment tool is now working fully. There were some important new features added to VITAL in the December 2014 upgrade, including a student preview mode,  anonymous marking for Blackboard assignments and delegated marking for Blackboard assignments which allows double marking and moderation of Blackboard Assignments (this does not apply to Turnitin assignments).

From previous upgrades to Turnitin there were a couple of useful enhancements, most importantly the file size limit for student assignments has doubled to 40 MB. You can also directly email late-submitting students from the assignment’s Turnitin classlist.

6. Any problems you can tell me about now?

There is a regularly-updated Known Issues page here which also covers Turnitin and Campus Pack problems. Some of the main current problems are:

  • Tracking statistics reports do not include Mobile app access of modules.
  • Bringing forward the post date of a Turnitin assignment marked with GradeMark  appears to make any final grades entered go missing (in fact they re-appear later on but this is alarming to see).

If you encounter a problem with VITAL which is not listed on the Known Issues page please do report this to the CSD ServiceDesk.

7. Where can I get some more help, training or online resources for VITAL?

Workshops. We’ve just run our 2017 Winter School and are now in the process of scheduling in workshops for the rest of the year. As well our Summer School (7th – 16th June) we’ll be organising a new ‘mini’ Easter School, and yes there will be chocolate eggs. Our e-submission and e-marking days run every month and are all bookable from the CLL booking page here (and see above). If you want a session running for your school or department then please contact us to arrange this as soon as possible.

VITAL Help Module.  All staff are enrolled on the module VITAL Baseline and guidance which has support sections for:

  • how to meet the Baseline standard (and get a basic introduction to VITAL)
  • e-submission and e-marking
  • lecture capture and screencasting

If you come on one of our workshops you will also be enrolled on our workshop resources module which contains further guides, examples, case studies, research and more on the specific sessions that you attend.

Guides. VITAL Self Service has searchable FAQs and Known Issues as well as some useful administrative tools for working with VITAL.This A-Z guides page has functional help on setting up and using the toolset in VITAL. If you don’t see the guide you need just ask us and we’ll find something for you!

Email and ‘phone support. Contact us for advice and help. We also run a bookable Studio Wednesday drop-in.

Is there anyone else out there I can talk to?

As well as the eLearning Unit (we’ll help with how you can use these technologies for your learning and teaching) and CSD Servicedesk (who will deal with all student queries and all technical problems like enrolments, error messages and so on) if you want help with the Reading Lists @ Liverpool tool or digital copyright then please contact your liaison librarian. There is also a University mailing list for people interested in learning technology where you can ask questions and get updates on what is happening with learning technologies around the institution, find out dates of upcoming workshops and e-learning network meetings and other similar groups around the University; get in touch with the eLearning Unit if you want to join.

Dan

Winter School 2017 – Week 2 Workshops – 16th – 20th January

The Winter School began today with a great session on engaging students visually in lectures with loads of ideas for presentation materials. There are a number of other sessions in the first week , which you can read about here and still book the last few places here.

This year we are also running a second week of sessions (Monday 16th to Friday 20th January) looking at technologies for learning and teaching beyond VITAL and a number of which are designed to tie-in with the online Bring Your Own Device for Learning 2017 events. All sessions listed below and bookable at: https://www.liverpool.ac.uk/cll/booking/

  • Monday 16th Jan – An introduction to Twitter (2 – 4pm)
  • Tuesday 17th Jan – GoMobile user group meeting (1 – 3pm)
  • Wednesday 18th Jan – e-submission and e-marking workshops
    • An Introduction To Electronic Submission Of Coursework (9:30 – 11am)
    • The Turnitin Assignment Tool For e-submission (Part 1) And GradeMark For Feedback (Part 2) (12 – 2pm)
    • The Blackboard Assignment Tool For e-submission (Part 1) And Feedback (Part 2) (2:30 – 4pm)
  • Thursday 19th Jan – Student as co-creators (11am – 1pm)
  • Friday 20th Jan – Advanced Twitter (2 – 4pm)

Finally, after these two weeks are a couple of events which might interest you.

  • Wednesday 25th Jan – Building good VITAL modules – a practical session looking at ways of building on the VITAL Baseline to create well-designed modules.
  • Thursday 26th Jan – eLearning Network meeting. In this meeting we will be getting a first look at the new Turnitin Feedback Studio.

All sessions bookable at: https://www.liverpool.ac.uk/cll/booking/

winter-school