Blackboard or Turnitin – our EMA decision-making flowchart

The team are fresh back from the now-legendary 2017 ALT conference, and bursting to share and connect with this  fantastic community. One of the things presented by Alex, Tünde and Dan which caught people’s eye during our session on the sometimes arduous road to EMA was Tünde’s ‘Blackboard or Turnitin’ decision flow chart.

It was designed to help staff decide what were the most important factors when they first come to choose whether BB or Tii is the best fit for their context. Behind it is the usual mountain of documentation, guides, caveats, sub-clauses and subtleties, but we had a think about what were the most-often asked questions from our staff and the priority they gave different requirements and used this to help map an easier route through the terrain.

As it looked useful to a few people, we’ve tidied it up, and given it a CC licence so please feel free to grab it from here and try it out with your own colleagues. If you want to know any more about our approach to guiding staff through the BB and Tii tangle then please do get in touch.

Creative Commons LicenseBlackboard or Turnitin Assignment by Centre for Innovation in Education, University of Liverpool is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

You can also catch up with our slides from the session here.

Something else that caught people’s attention, quite literally, was our percussion department for bringing the breakout discussion parts of our workshop presentations to order. ALT delegates are incredibly eager to get stuck in to a good conversation about any aspect of learning technologies, so you definitely need some serious help on your side as a presenter if you want to get a word in again! Here’s our kit, level 1 – the bell, level 2 – the castanets, and level 3 – our last resort – the slide whistle. We went all the way to level 3.

Alex, Tünde and Dan

Advertisements

Black Sabbath to Busted – Report on Blackboard Enhanced Assessment and Feedback Event

Leaving home under an early-morning starry sky and up over the misty, snowy Pennines to Sheffield, I wasn’t sure what to expect from the Blackboard Enhanced Assessment and Feedback day to which I was travelling. I had a sketched outline of the themes for the day ahead but not much more, namely that we would engage in some way with:

Assessment and feedback – an institutional perspective

  • Examination of key drivers and challenges (reputations, quality of process, quality of data, legal, business efficiencies, risk etc.)
  • Placing use cases on a confidence/effectiveness model
  • Highlight relevant Blackboard solutions

 Assessment and feedback – innovation and organisational change

  • Identifications for drivers for change
  • Gap analysis of practice and stakeholder experiences
  • Prioritisation of opportunities for change
  • Highlight relevant Blackboard solutions

This kind of workshoppy day from Blackboard was something I hadn’t experienced before, so I was propelled by curiosity as much as that it seemed relevant to the work that our team are currently leading on e-submission and e-feedback at Liverpool. What I got was a useful day of frank discussion and sharing of experiences, ideas and commonalities, which was mostly reassuring, with colleagues from other institutions in the kind if depth you don’t often get. This is especially useful for a sense of the bigger picture in HE, to talk about the differently badged or described but largely similar activities, structures and strategies that are top of our agendas at the moment, e-submission and e-marking being one of Liverpool’s current strategic TEL focus. One universal and rapidly-emerging area of concern that became evident on the day was a need for a variety of programme-level views of assessment activity in the VLE for academic and administrative staff and students. This is a long-requested feature from Blackboard usergroups that’s time has come with the adoption of e-submission and e-marking policies across the sector of late and I hope this was the main takeaway message for the Blackboard team.

The event was run by our regional Blackboard Customer Success Team, in partnership with the BB North user group, recognising a need for a more extensive exploration of particular issues that get raised at user group meetings, where the format doesn’t allow fuller discussion. Whilst advertised as intended for senior leaders, learning technologists, TEL managers and academic staff, the majority attending today were learning technologist types. It was instructive to hear that on the previous day at Edinburgh a couple of PVCs had attended, sending some very positive signals about the depth of an institutions’ engagement and intent with the actual tools that students, academic and professional service staff use as a part of their everyday life at the University.

Music, sweet music…

Our first activity was to introduce ourselves telling the room the first piece of music that we had ever bought. An astonishing array of formats and first loves was paraded, from Now compilation tapes to Avril Levigne downloads, from Osmonds vinyls to Busted CDs. Blackboard’s Alan Masson and Gillian Fielding all admitted to their first purchases, but I’ll spare their blushes here. Top-trumping all these, however, was Blackboard’s Steve Hoole, whose overnight Novotel stay featured a vinyl deck (remember these, kids?) and a selection of Sheffield synth heaven albums to spin the night away.

Structure of the sessions

The morning and the afternoon were structured in a similar way so that we would first ‘brainstorm’ our thoughts in groups on a set of e-assessment themes, then work together on some specific ideas from those and bring something interesting back to the room. We’d finally end with a discussion of the potentially useful tools in Blackboard that could be a part of the thinking for some of these. I initially thought this last element was going to be a sales pitch but it was pleasingly nuanced in that the Blackboard team wanted to hear stories of how people are using these tools, where they’re working well and what the gaps are. A very clear point made more than once was that the recent Blackboard activity in developing the assignment tool to offer dual marking, moderation and anonymous marking had been excellent but it now seemed that the Blackboard focus had moved on from this, whilst the toolset still needs work, that there had only been one iteration of the process. Also that this process of close consultation should be constantly repeated for other areas as our needs are constantly evolving, not just for assessment.

Morning Session – what’s needed to enhance assessment and feedback practice?

For the morning, we’d thought about quality, processes and workflows and where the opportunities for enhancement lay. We chucked all of our ideas at the first Padlet below. You’ll find all of the issues that we have encountered in the course of our work as learning technology developers and as part of the University’s EMA project, from how to handle video submission and feedback, to combining some functionality of Blackboard assignments (group tools, double marking, staged release of feedback, etc) with that of Turnitin assignments (Originality Checking, GradeMark, offline marking). In fact, this was another big ask on the day, that Turnitin and Blackboard align/integrate their products in ways that will help us, as you will gather from some of the posts in this Padlet.

Made with Padlet

What leapt out at me was a so-far un-encountered issue at Liverpool of needing a read-only external examiner access to modules. In some institutions administrative staff are packaging up content and assignments into a special section of a module, which only the external has access to, and making the rest of the module inaccessible to them. This is to meet an anxiety around externals potentially changing grades and altering content, but it costs hours of administrative time, essentially duplicating what’s already in the module, so re-introducing at a later stage of the assessment cycle serious administrative burden that the electronic submission process had originally taken away from the front end. What’s needed is a read-only access enrollment level, which is another development idea for the Blackboard team to add to the suggestions box.

Moving on to the next activity in groups again, we were tasked with listing and describing up to five assessment and feedback enhancements that would have significant benefits for the listed stakeholders and the degree to which it would require resource. This photo shows our effort…

dsc01104

…and for those who aren’t adept at reading the handwriting of people who spend their whole day attached to a keyboard our five enhancements (all pretty standard) were:

  1. External Examiner Access – read-only access for external examiners or a similar idea.
  2. Student Assessment Journey – programme level views of student assessment activity for students and staff
  3. Flexible innovative assessment – making the assessment and feedback tools at all points in Blackboard, not just for assignments, so that you can start thinking about using any tool for assessment purposes.
  4. Double marking – further work on the current functionality to take it to a robust, fully-usable level.
  5. Programme Level Assessment – looking at assessment practice across entire programmes and thinking about programme-level learning outcomes.

Hearing back from the rest of the room we discussed in more depth some of the already described above (external examiner access, programme-level views of assessment) and the Blackboard team promised to send round some case study examples of good practice for external examiner processes using Blackboard tools. In a discussion around whether and how institutions were using the Delegated Grading functionality, which was designed for UK HEIs, again the Blackboard team said they would gather together some case studies of where these are being used well. The feeling from the room was that this kind of functionality should be available across all assessment tools rather than locked to a single tool.

A few other interesting discussion points to end the morning session were that many institutions are thinking at programme level about replacing traditional assignment assessment. Video assignments and feedback are rapidly on the rise but also causing headaches as infrastructure and policy isn’t keeping up.

Afternoon Session

As I said, the afternoon session followed the same structure. So our post-lunch digestif activity was another Padlet , this time thinking about innovation and new practices that would enhance assessment and feedback in our institutions. I’ll let the Padlet do the talking so scroll around to see the ideas. I was interested in things like students being able to select the kinds of assessment that they wanted to do, and learners and academics developing assessment literacies through feedback dialogue and feed-forward as a continuous process.

Made with Padlet

As in the morning, next was a group task, where we were asked to think “aspirationally” about how we imagined assessment could look, if we had a free rein. What change or innovation in assessment and/or feedback would have significant impact and how would it benefit learners, tutors, courses and institutions? Essentially we were encouraged to go wild in the aisles of transformative assessment practice.

Our group went Back to Basics and offered the transformative potential of programmes where learning outcomes were mapped to assessment.  Well, someone had to. Other groups had some tidy, Tomorrow’s World ideas including:

  • An assessment wizard which built the kind of assessment you wanted with one view for staff and students (no more multiple systems or at least hiding these from you).
  • A tool that surfaced programme level assessment data.
  • A tool for personalised feedback and assessment routes – feedback raises flags on further help students can get and other staff can see that in later assessments.

And as per the morning session the Blackboard team led a discussion on how their products could work to do some of these things. One thing they did bring back to my attention was the Goals and Outcomes system which has a new dashboard view of the data and I think it would be opportune to review this in the light of programme development work that is heading the way of our team, as this could present an opportunity for offering programme-level views of progress through modules.

The end

So not a sales day, not your regular product roadmap/roadshow day, this represented a deeper dive into Electronic Management of Assessment, including the Blackboard tools that can be a part of that that environment. The Blackboard team wanted the day to be about sharing practice, raising awareness of what Blackboard tools we have already and encouraging us to get the best we can out the Blackboard tools and products that you have and I think this was more than achieved on the day. Thanks to the team and to the Bb North UserGroup for arranging and hosting. I had some very useful conversations (which is pretty much usual for the BB North User Groups meetings) and plenty from all of the above to take back for the project board overseeing the implementation of an e-submission policy at Liverpool.

What music did I first buy? The Muppet Show album. And I’ve never needed any other in my life…

Dan

Winter School 2017 – Week 2 Workshops – 16th – 20th January

The Winter School began today with a great session on engaging students visually in lectures with loads of ideas for presentation materials. There are a number of other sessions in the first week , which you can read about here and still book the last few places here.

This year we are also running a second week of sessions (Monday 16th to Friday 20th January) looking at technologies for learning and teaching beyond VITAL and a number of which are designed to tie-in with the online Bring Your Own Device for Learning 2017 events. All sessions listed below and bookable at: https://www.liverpool.ac.uk/cll/booking/

  • Monday 16th Jan – An introduction to Twitter (2 – 4pm)
  • Tuesday 17th Jan – GoMobile user group meeting (1 – 3pm)
  • Wednesday 18th Jan – e-submission and e-marking workshops
    • An Introduction To Electronic Submission Of Coursework (9:30 – 11am)
    • The Turnitin Assignment Tool For e-submission (Part 1) And GradeMark For Feedback (Part 2) (12 – 2pm)
    • The Blackboard Assignment Tool For e-submission (Part 1) And Feedback (Part 2) (2:30 – 4pm)
  • Thursday 19th Jan – Student as co-creators (11am – 1pm)
  • Friday 20th Jan – Advanced Twitter (2 – 4pm)

Finally, after these two weeks are a couple of events which might interest you.

  • Wednesday 25th Jan – Building good VITAL modules – a practical session looking at ways of building on the VITAL Baseline to create well-designed modules.
  • Thursday 26th Jan – eLearning Network meeting. In this meeting we will be getting a first look at the new Turnitin Feedback Studio.

All sessions bookable at: https://www.liverpool.ac.uk/cll/booking/

winter-school

Winter School 2017 diary dates

winter-school

A quick post to let you know about the eLearning Unit’s 2017 Winter School. All sessions bookable at: https://www.liverpool.ac.uk/cll/booking/

We’ve scheduled the first full week of sessions as listed here.

  • Monday 9th Jan – Engaging Learners visually in lectures – tools, tips and tricks (1 – 3pm)
  • Tuesday 10th Jan – VITAL Essentials (10am – 12pm)
  • Tuesday 10th Jan – An introduction to Technology Enhanced Learning (13:00 – 16:00)
  • Wednesday 11th Jan – Running webinars and online classrooms with Adobe Connect (10am – 12pm)
  • Thursday 12th Jan – 2nd annual Pedagogic Research Conference (all day)
  • Thursday 12th Jan – Turnitin GradeMark (2 – 3:30pm)
  • Friday 13th Jan – Stream Capture for lecture capture and screencasting (1 – 3pm)

This year we are also running a second week second week of sessions (Monday 16th to Friday 20th January) a number of which are designed to tie-in with the online Bring Your Own Device for Learning 2017 events. All sessions bookable at: https://www.liverpool.ac.uk/cll/booking/

  • Monday 16th Jan – An introduction to Twitter (2 – 4pm)
  • Tuesday 17th Jan – GoMobile user group meeting (1 – 3pm)
  • Wednesday 18th Jan – e-submission and e-marking workshops
    • An Introduction To Electronic Submission Of Coursework (9:30 – 11am)
    • The Turnitin Assignment Tool For e-submission (Part 1) And GradeMark For Feedback (Part 2) (12 – 2pm)
    • The Blackboard Assignment Tool For e-submission (Part 1) And Feedback (Part 2) (2:30 – 4pm)
  • Thursday 19th Jan – Student as co-creators (11am – 1pm)
  • Friday 20th Jan – Advanced Twitter (2 – 4pm)

Finally, after these two weeks are a couple of events which might interest you.

  • Wednesday 25th Jan – Building good VITAL modules – a practical session looking at ways of building on the VITAL Baseline to create well-designed modules.
  • Thursday 26th Jan – eLearning Network meeting. In this meeting we will be getting a first look at the new Turnitin Feedback Studio.

All sessions bookable at: https://www.liverpool.ac.uk/cll/booking/

E-submission and e-marking workshops and support 2017

esubmission long

In support of the University’s e-submission policy for the academic year 2016-17 the eLearning Unit continue to run e-submission and e-marking workshop days throughout the academic year. The days consist of three sessions described below and you are welcome to attend any or all, in whole or in part. We also run tailored school and departmental workshops and if you are considering training or consultancy from the team for e-submission and marking processes and systems, please do get in touch as soon as possible. You will also find dedicated online support resources from the eLearning Unit in the VITAL module VITAL Baseline and support, on which all staff are enrolled.

To search for and book onto any session please see the CLL booking page here. Scheduled e-marking workshop days (all on Wednesdays) for the rest of 2017 are:

  • January 18th
  • February 22nd
  • March 15th
  • April 26th
  • June 14th

Session 1: An introduction to electronic submission of coursework

(Please note this is not a lab-based session). The University is moving to online submission for coursework assignments (see this blog post for more detail on policy). To help staff get an overview of the available tools, understand associated current issues, and plan your own approach to e-submission the eLearning Unit offer this session comparing and discussing the two tools in VITAL for electronic submission, the Blackboard assignment tool and Turnitin.

In this two hour session we will look at:

  • An overview of e-submission in Higher Education and at Liverpool, including the student view
  • A discussion of attendees’ current practice
  • Analysis and examples of the two available assignment submission tools, Blackboard Assignment and Turnitin Assignments and the linked tools available for each.
  • Institutional guidance and key issues for planning assignment submission ‘workflows’.
  • Follow-up support and guidance.

Session 2: Turnitin Assignment tool for e-submission (part 1) and GradeMark for feedback (part 2)

This practical session runs in two parts. After a quick recap of the key differences between Turnitin and Blackboard we will firstly look at the different ways in which the Turnitin Assignment tool can be set up for online assignment submission (part 1 – 45 minutes approx). Then in the optional second part (45 minutes approx) we will try out the GradeMark feedback facilities available, from Criteria marking sheets to inline comments, both in desktop and mobile versions. If you would like to try out the mobile version of the grading tool please could you bring your device. This is available for iOS devices only at the moment and can be found here: https://itunes.apple.com/gb/app/turnitin/id657602524?mt=8

Session 3: The Blackboard Assignment tool for e-submission (part 1) and feedback (part 2)

This practical session runs in two parts. After a quick recap of the key differences between Turnitin and Blackboard we will firstly look at the different ways in which the Blackboard Assignment tool can be set up for online assignment submission (part 1 – 45 minutes approx). Then in the optional second part (45 minutes approx) we will try out the grading facilities available, from Criteria marking sheets to inline comments, both in desktop and mobile versions.

Please contact the team if you have any questions about these sessions.

Dan

New Turnitin features – August 2015

In this summer’s upgrade to VITAL there were a couple of enhancements to Turnitin worth knowing about.

1. New 40MB file upload size limit. The previous limit was 20MB. Students can now submit one file of up to 40MB to a Turnitin assignment.

2. ‘Email late submitters’ facility.​ Send an email to all students who have not yet submitted to an assignment. This feature also works with anonymised assignments.

And as a refresher, there have been a number of useful new features introduced into Turnitin over the last couple of years which are listed below along with some links to more details. Any questions about any of the below please do contact the eLearning Unit (elearning@liv.ac.uk).

1. Submit PowerPoints, Excel and Google docs for originality checking. The range of file types that can be submitted by students and which Turnitin will scan for originality and create an Originality Report includes PowerPoint, Excel and Google Docs as well as Microsoft Word, WordPerfect, PostScript, PDF, HTML, RTF, OpenOffice (ODT), Hangul (HWP), and plain text. They should be less than 40MB and contain at least 20 words of text. When creating the assignment, on the page where dates and titles are entered select the option “Allow only file types that Turnitin can check for originality”.

2. Submit images. Students can submit images in the JPEG, GIF, PNG, TIFF, BMP, or PICT formats. These are not originality checked so will not generate an Originality Report but can be opened directly in GradeMark for comment, feedback and grading. They should be less than 40MB. When creating the assignment, on the page where you add in the dates and title select the option “Allow any file type”.

3. Submit any file type. A new option at the assignment set-up stage will let students submit any file type they wish to the Turnitin assignment. (When creating the assignment, on the page where you add in the dates and title select the option “Allow any file type”). Originality checking cannot be guaranteed here. The Originality Check facility will try to scan student submissions but will not return a value or report if it cannot. GradeMark will also try to display the file directly in the Turnitin document viewer but again, if it cannot do this for a particular filetype then it will display the submission as a link to download the file to open in the relevant application. You can still use all of the GradeMark feedback facilities and a blank page for QuickMark comments is available in the case where the file is only available as a download. Files must be less than 40MB.

4. Grade without submissions. You can use the GradeMark facilities with no need for the students to submit work. An example context of this kind of use is an assessed presentation – GradeMark could be used during the presentation on an iPad or laptop to make grading notes, notes on any criteria marking, and offer the overall comment, grade, criteria marks and/or audio comment afterwards.

5. Link inline comments and marks with Rubric card criterion. If using a rubric card and also the QuickMarks to comment inline on student work you can choose to link a particular QuickMark/comment with one of your criterion which is then visible to the student on the rubric – extremely useful for evidencing the levels attained by students in the criterion. For more details of this feature see this blogpost from Turnitin.

6. ‘Grading Forms’ (simple criteria marking sheets). Essentially a scaled-down rubric/criteria electronic marking sheet, as exemplified in the screen shot below. For full details on this new feature see this blogpost from Turnitin.

An example of the new grading form for use with Turnitin Grade Mark

7. Upload criteria from Excel. Use the Turnitin Excel template to import your criteria from Excel which can make creating Turnitin criteria cards much faster when you have the cards existing in other digital formats. See this Turnitin item for details.

8. Student assignment submission ‘workflow’. The student submission workflow is a three stage process as it always has been. The stage where students check their work before submission is a paginated and expandable window on the submission confirmation page (stage two) containing the fully formatted work as submitted by the student (it used to be an unformatted view of the submitted text only). Students click on the image of their work if they need to enlarge it, and click through it using the on screen arrow icons to check their work.

This is an important detail as students need to confirm that the content displayed here corresponds to what they expect to submit (i.e. there have been no glitches etc in the upload process) and any submission guidance should be updated to reflect this. Students can also now submit directly from DropBox and GoogleDocs, as well as upload from their desktop machine. For more details please read this Turnitin blogpost.

9. Quantitative % grading schemes – do not need to add up to 100%. They can be less than 100% if required.

Turnitin continually seek ideas and feedback from users on how to improve the system (which includes GradeMark and PeerMark also). You can see the current list of ideas, vote for them, add you own etc on the Turnitin Roadmap forum from any Turnitin assignment classlist. See this blog post for more details on the Turnitin Roadmap.

We’re offering workshops on GradeMark and Turnitin this academic year. Keep an eye on the CLL Booking site or our twitter feed for dates.

Dan

Turnitin GradeMark on your iPad…

…and hurrah! it includes a facility for working offline.

This summer the iPad app for Turnitin GradeMark was generally released after a long period of Beta-testing and a lot of anticipation.

Available free from iTunes (https://itunes.apple.com/us/app/turnitin/id657602524) this app will let you carry out all of the grading and feedback functions you are used to with GradeMark.

Before you can use the app you will need to make a settings change which is detailed in this FAQ from our VITAL Self Service tool (click the link). The FAQ also explains how you create a class code so that you can link from the VITAL module containing the assignments to the iPad app.

We’d be very interested in hearing any reports from staff trying this app out so please do get in touch with the eLearning Unit to discuss your experiences.