Black Sabbath to Busted – Report on Blackboard Enhanced Assessment and Feedback Event

Leaving home under an early-morning starry sky and up over the misty, snowy Pennines to Sheffield, I wasn’t sure what to expect from the Blackboard Enhanced Assessment and Feedback day to which I was travelling. I had a sketched outline of the themes for the day ahead but not much more, namely that we would engage in some way with:

Assessment and feedback – an institutional perspective

  • Examination of key drivers and challenges (reputations, quality of process, quality of data, legal, business efficiencies, risk etc.)
  • Placing use cases on a confidence/effectiveness model
  • Highlight relevant Blackboard solutions

 Assessment and feedback – innovation and organisational change

  • Identifications for drivers for change
  • Gap analysis of practice and stakeholder experiences
  • Prioritisation of opportunities for change
  • Highlight relevant Blackboard solutions

This kind of workshoppy day from Blackboard was something I hadn’t experienced before, so I was propelled by curiosity as much as that it seemed relevant to the work that our team are currently leading on e-submission and e-feedback at Liverpool. What I got was a useful day of frank discussion and sharing of experiences, ideas and commonalities, which was mostly reassuring, with colleagues from other institutions in the kind if depth you don’t often get. This is especially useful for a sense of the bigger picture in HE, to talk about the differently badged or described but largely similar activities, structures and strategies that are top of our agendas at the moment, e-submission and e-marking being one of Liverpool’s current strategic TEL focus. One universal and rapidly-emerging area of concern that became evident on the day was a need for a variety of programme-level views of assessment activity in the VLE for academic and administrative staff and students. This is a long-requested feature from Blackboard usergroups that’s time has come with the adoption of e-submission and e-marking policies across the sector of late and I hope this was the main takeaway message for the Blackboard team.

The event was run by our regional Blackboard Customer Success Team, in partnership with the BB North user group, recognising a need for a more extensive exploration of particular issues that get raised at user group meetings, where the format doesn’t allow fuller discussion. Whilst advertised as intended for senior leaders, learning technologists, TEL managers and academic staff, the majority attending today were learning technologist types. It was instructive to hear that on the previous day at Edinburgh a couple of PVCs had attended, sending some very positive signals about the depth of an institutions’ engagement and intent with the actual tools that students, academic and professional service staff use as a part of their everyday life at the University.

Music, sweet music…

Our first activity was to introduce ourselves telling the room the first piece of music that we had ever bought. An astonishing array of formats and first loves was paraded, from Now compilation tapes to Avril Levigne downloads, from Osmonds vinyls to Busted CDs. Blackboard’s Alan Masson and Gillian Fielding all admitted to their first purchases, but I’ll spare their blushes here. Top-trumping all these, however, was Blackboard’s Steve Hoole, whose overnight Novotel stay featured a vinyl deck (remember these, kids?) and a selection of Sheffield synth heaven albums to spin the night away.

Structure of the sessions

The morning and the afternoon were structured in a similar way so that we would first ‘brainstorm’ our thoughts in groups on a set of e-assessment themes, then work together on some specific ideas from those and bring something interesting back to the room. We’d finally end with a discussion of the potentially useful tools in Blackboard that could be a part of the thinking for some of these. I initially thought this last element was going to be a sales pitch but it was pleasingly nuanced in that the Blackboard team wanted to hear stories of how people are using these tools, where they’re working well and what the gaps are. A very clear point made more than once was that the recent Blackboard activity in developing the assignment tool to offer dual marking, moderation and anonymous marking had been excellent but it now seemed that the Blackboard focus had moved on from this, whilst the toolset still needs work, that there had only been one iteration of the process. Also that this process of close consultation should be constantly repeated for other areas as our needs are constantly evolving, not just for assessment.

Morning Session – what’s needed to enhance assessment and feedback practice?

For the morning, we’d thought about quality, processes and workflows and where the opportunities for enhancement lay. We chucked all of our ideas at the first Padlet below. You’ll find all of the issues that we have encountered in the course of our work as learning technology developers and as part of the University’s EMA project, from how to handle video submission and feedback, to combining some functionality of Blackboard assignments (group tools, double marking, staged release of feedback, etc) with that of Turnitin assignments (Originality Checking, GradeMark, offline marking). In fact, this was another big ask on the day, that Turnitin and Blackboard align/integrate their products in ways that will help us, as you will gather from some of the posts in this Padlet.

Made with Padlet

What leapt out at me was a so-far un-encountered issue at Liverpool of needing a read-only external examiner access to modules. In some institutions administrative staff are packaging up content and assignments into a special section of a module, which only the external has access to, and making the rest of the module inaccessible to them. This is to meet an anxiety around externals potentially changing grades and altering content, but it costs hours of administrative time, essentially duplicating what’s already in the module, so re-introducing at a later stage of the assessment cycle serious administrative burden that the electronic submission process had originally taken away from the front end. What’s needed is a read-only access enrollment level, which is another development idea for the Blackboard team to add to the suggestions box.

Moving on to the next activity in groups again, we were tasked with listing and describing up to five assessment and feedback enhancements that would have significant benefits for the listed stakeholders and the degree to which it would require resource. This photo shows our effort…

dsc01104

…and for those who aren’t adept at reading the handwriting of people who spend their whole day attached to a keyboard our five enhancements (all pretty standard) were:

  1. External Examiner Access – read-only access for external examiners or a similar idea.
  2. Student Assessment Journey – programme level views of student assessment activity for students and staff
  3. Flexible innovative assessment – making the assessment and feedback tools at all points in Blackboard, not just for assignments, so that you can start thinking about using any tool for assessment purposes.
  4. Double marking – further work on the current functionality to take it to a robust, fully-usable level.
  5. Programme Level Assessment – looking at assessment practice across entire programmes and thinking about programme-level learning outcomes.

Hearing back from the rest of the room we discussed in more depth some of the already described above (external examiner access, programme-level views of assessment) and the Blackboard team promised to send round some case study examples of good practice for external examiner processes using Blackboard tools. In a discussion around whether and how institutions were using the Delegated Grading functionality, which was designed for UK HEIs, again the Blackboard team said they would gather together some case studies of where these are being used well. The feeling from the room was that this kind of functionality should be available across all assessment tools rather than locked to a single tool.

A few other interesting discussion points to end the morning session were that many institutions are thinking at programme level about replacing traditional assignment assessment. Video assignments and feedback are rapidly on the rise but also causing headaches as infrastructure and policy isn’t keeping up.

Afternoon Session

As I said, the afternoon session followed the same structure. So our post-lunch digestif activity was another Padlet , this time thinking about innovation and new practices that would enhance assessment and feedback in our institutions. I’ll let the Padlet do the talking so scroll around to see the ideas. I was interested in things like students being able to select the kinds of assessment that they wanted to do, and learners and academics developing assessment literacies through feedback dialogue and feed-forward as a continuous process.

Made with Padlet

As in the morning, next was a group task, where we were asked to think “aspirationally” about how we imagined assessment could look, if we had a free rein. What change or innovation in assessment and/or feedback would have significant impact and how would it benefit learners, tutors, courses and institutions? Essentially we were encouraged to go wild in the aisles of transformative assessment practice.

Our group went Back to Basics and offered the transformative potential of programmes where learning outcomes were mapped to assessment.  Well, someone had to. Other groups had some tidy, Tomorrow’s World ideas including:

  • An assessment wizard which built the kind of assessment you wanted with one view for staff and students (no more multiple systems or at least hiding these from you).
  • A tool that surfaced programme level assessment data.
  • A tool for personalised feedback and assessment routes – feedback raises flags on further help students can get and other staff can see that in later assessments.

And as per the morning session the Blackboard team led a discussion on how their products could work to do some of these things. One thing they did bring back to my attention was the Goals and Outcomes system which has a new dashboard view of the data and I think it would be opportune to review this in the light of programme development work that is heading the way of our team, as this could present an opportunity for offering programme-level views of progress through modules.

The end

So not a sales day, not your regular product roadmap/roadshow day, this represented a deeper dive into Electronic Management of Assessment, including the Blackboard tools that can be a part of that that environment. The Blackboard team wanted the day to be about sharing practice, raising awareness of what Blackboard tools we have already and encouraging us to get the best we can out the Blackboard tools and products that you have and I think this was more than achieved on the day. Thanks to the team and to the Bb North UserGroup for arranging and hosting. I had some very useful conversations (which is pretty much usual for the BB North User Groups meetings) and plenty from all of the above to take back for the project board overseeing the implementation of an e-submission policy at Liverpool.

What music did I first buy? The Muppet Show album. And I’ve never needed any other in my life…

Dan

Advertisements

Winter School 2017 – Week 2 Workshops – 16th – 20th January

The Winter School began today with a great session on engaging students visually in lectures with loads of ideas for presentation materials. There are a number of other sessions in the first week , which you can read about here and still book the last few places here.

This year we are also running a second week of sessions (Monday 16th to Friday 20th January) looking at technologies for learning and teaching beyond VITAL and a number of which are designed to tie-in with the online Bring Your Own Device for Learning 2017 events. All sessions listed below and bookable at: https://www.liverpool.ac.uk/cll/booking/

  • Monday 16th Jan – An introduction to Twitter (2 – 4pm)
  • Tuesday 17th Jan – GoMobile user group meeting (1 – 3pm)
  • Wednesday 18th Jan – e-submission and e-marking workshops
    • An Introduction To Electronic Submission Of Coursework (9:30 – 11am)
    • The Turnitin Assignment Tool For e-submission (Part 1) And GradeMark For Feedback (Part 2) (12 – 2pm)
    • The Blackboard Assignment Tool For e-submission (Part 1) And Feedback (Part 2) (2:30 – 4pm)
  • Thursday 19th Jan – Student as co-creators (11am – 1pm)
  • Friday 20th Jan – Advanced Twitter (2 – 4pm)

Finally, after these two weeks are a couple of events which might interest you.

  • Wednesday 25th Jan – Building good VITAL modules – a practical session looking at ways of building on the VITAL Baseline to create well-designed modules.
  • Thursday 26th Jan – eLearning Network meeting. In this meeting we will be getting a first look at the new Turnitin Feedback Studio.

All sessions bookable at: https://www.liverpool.ac.uk/cll/booking/

winter-school

E-submission and e-marking workshops and support 2017

esubmission long

In support of the University’s e-submission policy for the academic year 2016-17 the eLearning Unit continue to run e-submission and e-marking workshop days throughout the academic year. The days consist of three sessions described below and you are welcome to attend any or all, in whole or in part. We also run tailored school and departmental workshops and if you are considering training or consultancy from the team for e-submission and marking processes and systems, please do get in touch as soon as possible. You will also find dedicated online support resources from the eLearning Unit in the VITAL module VITAL Baseline and support, on which all staff are enrolled.

To search for and book onto any session please see the CLL booking page here. Scheduled e-marking workshop days (all on Wednesdays) for the rest of 2017 are:

  • January 18th
  • February 22nd
  • March 15th
  • April 26th
  • June 14th

Session 1: An introduction to electronic submission of coursework

(Please note this is not a lab-based session). The University is moving to online submission for coursework assignments (see this blog post for more detail on policy). To help staff get an overview of the available tools, understand associated current issues, and plan your own approach to e-submission the eLearning Unit offer this session comparing and discussing the two tools in VITAL for electronic submission, the Blackboard assignment tool and Turnitin.

In this two hour session we will look at:

  • An overview of e-submission in Higher Education and at Liverpool, including the student view
  • A discussion of attendees’ current practice
  • Analysis and examples of the two available assignment submission tools, Blackboard Assignment and Turnitin Assignments and the linked tools available for each.
  • Institutional guidance and key issues for planning assignment submission ‘workflows’.
  • Follow-up support and guidance.

Session 2: Turnitin Assignment tool for e-submission (part 1) and GradeMark for feedback (part 2)

This practical session runs in two parts. After a quick recap of the key differences between Turnitin and Blackboard we will firstly look at the different ways in which the Turnitin Assignment tool can be set up for online assignment submission (part 1 – 45 minutes approx). Then in the optional second part (45 minutes approx) we will try out the GradeMark feedback facilities available, from Criteria marking sheets to inline comments, both in desktop and mobile versions. If you would like to try out the mobile version of the grading tool please could you bring your device. This is available for iOS devices only at the moment and can be found here: https://itunes.apple.com/gb/app/turnitin/id657602524?mt=8

Session 3: The Blackboard Assignment tool for e-submission (part 1) and feedback (part 2)

This practical session runs in two parts. After a quick recap of the key differences between Turnitin and Blackboard we will firstly look at the different ways in which the Blackboard Assignment tool can be set up for online assignment submission (part 1 – 45 minutes approx). Then in the optional second part (45 minutes approx) we will try out the grading facilities available, from Criteria marking sheets to inline comments, both in desktop and mobile versions.

Please contact the team if you have any questions about these sessions.

Dan

New Turnitin features – August 2015

In this summer’s upgrade to VITAL there were a couple of enhancements to Turnitin worth knowing about.

1. New 40MB file upload size limit. The previous limit was 20MB. Students can now submit one file of up to 40MB to a Turnitin assignment.

2. ‘Email late submitters’ facility.​ Send an email to all students who have not yet submitted to an assignment. This feature also works with anonymised assignments.

And as a refresher, there have been a number of useful new features introduced into Turnitin over the last couple of years which are listed below along with some links to more details. Any questions about any of the below please do contact the eLearning Unit (elearning@liv.ac.uk).

1. Submit PowerPoints, Excel and Google docs for originality checking. The range of file types that can be submitted by students and which Turnitin will scan for originality and create an Originality Report includes PowerPoint, Excel and Google Docs as well as Microsoft Word, WordPerfect, PostScript, PDF, HTML, RTF, OpenOffice (ODT), Hangul (HWP), and plain text. They should be less than 40MB and contain at least 20 words of text. When creating the assignment, on the page where dates and titles are entered select the option “Allow only file types that Turnitin can check for originality”.

2. Submit images. Students can submit images in the JPEG, GIF, PNG, TIFF, BMP, or PICT formats. These are not originality checked so will not generate an Originality Report but can be opened directly in GradeMark for comment, feedback and grading. They should be less than 40MB. When creating the assignment, on the page where you add in the dates and title select the option “Allow any file type”.

3. Submit any file type. A new option at the assignment set-up stage will let students submit any file type they wish to the Turnitin assignment. (When creating the assignment, on the page where you add in the dates and title select the option “Allow any file type”). Originality checking cannot be guaranteed here. The Originality Check facility will try to scan student submissions but will not return a value or report if it cannot. GradeMark will also try to display the file directly in the Turnitin document viewer but again, if it cannot do this for a particular filetype then it will display the submission as a link to download the file to open in the relevant application. You can still use all of the GradeMark feedback facilities and a blank page for QuickMark comments is available in the case where the file is only available as a download. Files must be less than 40MB.

4. Grade without submissions. You can use the GradeMark facilities with no need for the students to submit work. An example context of this kind of use is an assessed presentation – GradeMark could be used during the presentation on an iPad or laptop to make grading notes, notes on any criteria marking, and offer the overall comment, grade, criteria marks and/or audio comment afterwards.

5. Link inline comments and marks with Rubric card criterion. If using a rubric card and also the QuickMarks to comment inline on student work you can choose to link a particular QuickMark/comment with one of your criterion which is then visible to the student on the rubric – extremely useful for evidencing the levels attained by students in the criterion. For more details of this feature see this blogpost from Turnitin.

6. ‘Grading Forms’ (simple criteria marking sheets). Essentially a scaled-down rubric/criteria electronic marking sheet, as exemplified in the screen shot below. For full details on this new feature see this blogpost from Turnitin.

An example of the new grading form for use with Turnitin Grade Mark

7. Upload criteria from Excel. Use the Turnitin Excel template to import your criteria from Excel which can make creating Turnitin criteria cards much faster when you have the cards existing in other digital formats. See this Turnitin item for details.

8. Student assignment submission ‘workflow’. The student submission workflow is a three stage process as it always has been. The stage where students check their work before submission is a paginated and expandable window on the submission confirmation page (stage two) containing the fully formatted work as submitted by the student (it used to be an unformatted view of the submitted text only). Students click on the image of their work if they need to enlarge it, and click through it using the on screen arrow icons to check their work.

This is an important detail as students need to confirm that the content displayed here corresponds to what they expect to submit (i.e. there have been no glitches etc in the upload process) and any submission guidance should be updated to reflect this. Students can also now submit directly from DropBox and GoogleDocs, as well as upload from their desktop machine. For more details please read this Turnitin blogpost.

9. Quantitative % grading schemes – do not need to add up to 100%. They can be less than 100% if required.

Turnitin continually seek ideas and feedback from users on how to improve the system (which includes GradeMark and PeerMark also). You can see the current list of ideas, vote for them, add you own etc on the Turnitin Roadmap forum from any Turnitin assignment classlist. See this blog post for more details on the Turnitin Roadmap.

We’re offering workshops on GradeMark and Turnitin this academic year. Keep an eye on the CLL Booking site or our twitter feed for dates.

Dan

Turnitin Roadmap Forum – Have Your Say – update

Suggest and vote for ideas and improvements

A while back we wrote about the Turnitin Feedback forum where you could suggest and vote for new features, facilities, tools and improvements to the Turnitin, GradeMark and PeerMark products.

This facility has since been renamed to the Turnitin Roadmap and expanded to include details of when new versions of Turnitin will be released and what will be the new features in those releases. It also clearly labels all ideas in the forum with their status in the development schedule and how far away the implementation of each idea might be.

If you use the Turnitin, PeerMark and/or GradeMark products as a member of staff then you also have easy access to the Roadmap through VITAL and staff at Liverpool have been adding product suggestions, voting for and adding comments on ideas such as allowing group work submissions, facilitating multiple markers and making anonymous marking workflows more flexible.

In VITAL, find the Roadmap by going in to any Turnitin assignment class list (Control Panel>Course Tools>TurnitinUK assignments and select an assignment) and then in the upper right of the page find the link “Roadmap” as shown in the screenshot below.

TurnitinRoadmap

Clicking on this tab takes you to the front page of the Roadmap forum. The top half of the page has the scheduled release dates with links to details of what these will contain.

Scroll down the page and you will see the current most-voted for feature requests. You can open these to read the detail, add a comment and vote for the idea (you get an initial 20 votes). You can also use the search box on the main page to search for a particular feature request to see if it has already been submitted and so you can add your vote. If it is does not yet exist click the ‘Post a new idea…’ button, write up your idea and wait for the votes to start rolling in. Encourage your colleagues to vote for your request and get it moved up the ‘Top ideas’ list.

You can also view the ideas grouped by the subject-specific categories in the right-hand side menu which include ‘Turnitin for iPad’, ‘Grade Mark’ and ‘OriginalityCheck’.

The forum is constantly reviewed by the Turnitin support and software development teams and many of the latest developments to Turnitin and GradeMark have come from this forum.

Dan

Turnitin GradeMark on your iPad…

…and hurrah! it includes a facility for working offline.

This summer the iPad app for Turnitin GradeMark was generally released after a long period of Beta-testing and a lot of anticipation.

Available free from iTunes (https://itunes.apple.com/us/app/turnitin/id657602524) this app will let you carry out all of the grading and feedback functions you are used to with GradeMark.

Before you can use the app you will need to make a settings change which is detailed in this FAQ from our VITAL Self Service tool (click the link). The FAQ also explains how you create a class code so that you can link from the VITAL module containing the assignments to the iPad app.

We’d be very interested in hearing any reports from staff trying this app out so please do get in touch with the eLearning Unit to discuss your experiences.

E-learning Summer School 2013

Writing this post during a freezing-cold, wind-lashed rainstorm, what more cheering thought could there be than the prospect of this year’s eLearning Unit summer school? None? Then read on…

Running from Wednesday 12th to Thursday 20th June the sessions will include:

  • The introductory workshop for academic staff ‘An introduction to technology-enhanced learning using VITAL’
  • Turnitin assignments and Originality Reports
  • GradeMark for online marking and feedback
  • Marking by criteria with the Blackboard Rubric tool
  • VITAL Plus – tips and ideas for taking your use of VITAL to the next level
  • A workshop on the Wiki, blog and journal tools available in VITAL

We’re planning a small-scale session and some drop-ins around podcasting and ideas for screencasting (although please note that there are very limited tools available at the University for screencasting). These will appear on the booking system soon.

Also, the annual Learning and Teaching conference falls during this Summer School and there will be a dedicated room with three sessions on innovative uses of learning technologies from around the University, with a chance to try these out yourself, as well as colleagues from the library and CSD demonstrating some new online applications throughout the day.

Full details on each workshop and booking can be found at our booking site linked to here, look for sessions prefixed ‘E-learning summer school’. Below is the current full schedule and brief descriptions of each session (available in other formats – please email).

summer2013

Click to view current schedule for 2013 e-learning summer school

The sessions will generally be a mix of hands-on practical work, application to your learning and teaching, and case studies of practice at the University. The introductory workshop is suitable for  staff who have not used VITAL or Blackboard at another institution at all – if you are already familiar with the system then this session will be too basic. The main pre-requisite for the other summer school sessions is that you are familiar with VITAL and using its main facilities, but please contact us if you want to discuss the suitability of any of the sessions. You are welcome to book on as many sessions as you wish, although we would ask that if you find yourself unable to attend to unenrol as soon as possible.

As usual, we can only run these sessions with a minimum of 5 attendees. If we have to cancel a workshop that you have booked then we will arrange to run the session with you on a one-to-one/small group basis at another time convenient to you. We will confirm all workshops by Monday 10th June.