The Faculty of Medicine is currently piloting e-exams within a small number of courses.  For more information on this pilot please visit the e-exams in the Faculty of Medicine pilot page.

Online exams at UQ

UQ instructors have been making use of our learning management system's online test capabilities for school based tests. Use of these systems is increasing, and five faculties have listed this as an area they would like ITS to add services to in the near future. At UQ this endeavor is carefully informed by the expertise, experiences and needs of the university's instructors and educational designers, and education professionals in ITaLI and ITS. 

In support of e-exams, ITS have:

  • Provided online support guides and lessons learned for instructors doing e-tests (e.g. Pharmacy, Business)
  • Organised and delivered joint ITS-ITaLI training courses on online tests
  • Observed the Pharmacy pilot of the ProctorU online invigilation service
  • Observed ITaLI's online exam pilot of a USB application on BYOD devices
  • Been activelly pursuing projects in support of BYOD with initiatives such as:
    • a device purchasing equity program for students in support of: in-class tool use; student response systems; and e-exams
    • App-Central portal for distribution of applications to student owned devices
    • VDI pilot
  • Helped faculties and schools investigate online invigilation services, particularly schools already running home based exams (currently on hold at request of DVC(A))
  • Helped faculties and schools evaluate off-the-shelf options for quizes, tests and exams, like (e.g. Respondus, and Examsoft etc)

Description of e-exams:

At UQ examinations can be administered centrally or by the schools. Central examinations are administered by UQ's examinations section.  School-based examinations are administered by the relevant School (Policy 3.10.11 Examinations). For the purposes of this endeavour, an online test or e-exam is any quiz based assessment item where the examinee uses a computer or other electronic device to record their responses.  The responses are generally collected electronically, either manually or automatically, and the answers can be marked either manually or automatically, depending on the nature of the exam, and the software used.

Moving from paper-based exams to electronic exams can create efficiencies in exam creation, distribution, marking and collation of results.  Some solutions provide opportunities for including rich media in exams that improve the validity of assessments and provide a more consistent experience for students accustomed to participating in their learning via electronic devices. Electronic exams also come with opportunities for de-centralising exam sittings, freeing up exam rooms and creating flexibility with timetable scheduling and attendance for distance students.  Some solutions include a "lock-down" functionality that prevents students from accessing other applications and network devices during the exam.

However, as with every new technology, these opportunities come with new risks and challenges that include invigilation, technical architecture and hardware compatibility, business changes and the resulting training and support for students and staff.

What is the problem we are trying to solve?

Single solution options for on-campus exams:

  • Are we simply looking for an exam creation solution that provides a more sophisticated range of question authoring options than is currently available in the Blackboard tests and quizzes?
  • Are we satisfied with the current Blackboard quizzes, but looking for a technology that improves exam security in the exsisting lab environments - i.e. a lock-down browser?
  • Are we looking for a solution that allows students to bring their own devices into an exam room, and access an online exam in a secure environment?

Blended solution options for on-campus exams:

  • Do we want both improved question creation and improved security in an existing lab environment?
  • Do we want both improved question creation and improved security for students using their own machines?
 
Single solution options for off-campus exams:
 
  • Are we satisfied with the current Blackboard quizzes, but looking for a technology that improves exam security and flexibility in the students home or work location?  This could include recording (video/audio/screen/keystroke) of the exam with instructor review after; or live invigilation as well as recordings.
 
Blended solution options for off-campus exams:
 
  • Are we looking for a solution that provides more sophisticated exam questions, as well as a live/recorded online invigilation service?
 

Candidate solutions

There is a range of products which provide capability to implement the approaches listed above and can be used individually or various combinations. Generally custom built system are avoided because they are expensive and slow to develop, and quickly overtaken by superior commercial off the shelf (COTS) products that are better supported and achieve better economies of scale.

Other Considerations:

  • UQ policy 3.10.11 Examinations specifies that “unsupervised online assessment” falls outside the scope of the existing examinations policy.
  • UQ does not yet have a BYOD (Bring your own device) policy, and resulting equity issues must be taken into account.

Participants

Faculties who have identified this as a high priority include: Business, Science, Health, Engineering, SBS. ITaLI have expertise and a expressed a keen interest in the adoption of e-Exams.

Status

  • An initial product scan has been completed ✔ 
  • A scan for business drivers that support online exams is underway ✔ 
  • The School of Medicine has now commenced a Pilot project using Examsoft solution to test the concept and viability of invigilated, on campus, BYOD-based online exams within SoM’s selected courses. (live pilot to take place in Semester 2, 2016) ✔

Next steps

  • Establish key issues driving requirement for e-exams.
  • Collate lessons learnt from others who have used various e-exams solutions.
  • Perform trials / pilots of a range of solutions to identify whether they are suitable for the UQ environment.
  • Shortlist any suitable products and assist the faculties to develop a request for funding to implement larger-scale implementations that provide a complimentary suite of e-exam options.
  • Convert the successful options to supported eLearning services that are part of the SLA and include training and support for University – wide adoption.

Back to the potential project list.