Reading group notes: Models for evaluating e-learning

Evaluating E-learning, A Guide to the Evaluation of E-learning, Graham Attwell, 2006 (Suzi) – Framework of factors to take into consideration, then lists 5-6 clusters of types of evaluation. Not much detail of how these types of evaluation actually work, but may be useful as a starting point. Section 6: SPEAK tool – used in community education – may be useful for “students as agents of change”. Section 9 – evaluating e-learning policies – gives guidelines and prompts for developing a set of questions with which to evaluate a policy. Section 10 – management-oriented evaluation – possibly worth a look during project planning.

Evaluation of e-learning courses– Institute of Education , 2008  (Roger)   Covers evaluating courses, wholly or partly online.  Aims to provide an overview of practical evaluation resources. And its target audience is academics at IOE. Includes a literature review.Recommendations:

  • Plan evaluation before the course starts
  • Collect feedback from all stakeholders including students, tutors, admins and tech support staff.   For staff this could be done on an ongoing basis through frequent team meetings, and an end of course survey
  • Collect student feedback during and at the end of the course
  • Consider all relevant aspects of the use of tech for t and l in the course e.g. usefulness of the  content, how well online  activities run e.g. timing, sequencing (could be blended), instructions, the user experience ( levels of engagement, tutor participation, workload)
  • Make use of the specific tools available in typical online tools e.g. course stats in BB to get an idea of levels of activity (not quality)

Suggested reading

There was discussion about this on the ALT list in May 2011. References given include:


Reading group notes: Digital literacy

Information and digital competencies in HE  (Roger) – read one article in this collection  “HE and the knowledge society. Information and digital competencies”. (by Juan de Pablos Pons, 2010)  Knowledge society requires changes to teaching models – lecturers no longer the only source of knowledge, opportunities for technology to help the social dimension of teaching (“socialization of knowledge”). Universities are too compartmentalised, their structures and offer needs to reflect more the interdisciplinary nature of modern knowledge and research.   Distinguishes between IT competencies (how to use ICT) and “information competencies” (which would relate to more of Doug Belshaw’s 8 elements, e.g. criticality, creativity, cognitive etc. Lastly universities need to recognise that they are educating each student for multiple jobs (and these may not yet even exist)

The essential elements of digital literacies, Slideshare (Nic) This presentation offers a useful description of digital literacy (DL), i.e. DL is often understood as the ability to participate in a range of activities and creative practices that involve understanding, sharing and creating meaning with different kinds of media and technology”. This emphasise ICT and media. The presenation also introduces a ‘5-step process model’ and a matrix that can be used to map steps to different technologies. It also describes 8 essential elements of DL. Ones I found particularly interesting were:  As online texts are not exactly written, cyber literacy needs a different form of critical thinking; society is increasingly look for people who can confidently solve their own problem and manage their own life long learning, qualities ICT is believed to promote; DL must involve systemic awareness of how digital media are constructed. From our discussion, I’ve realised that althgough the term DL is complex and contested, the topic has an important relationship to TEL. Also, DL goes beyond reading (and writing) texts to actively and collaboratively creating meaning. This is where we, as learning technologist, can engage with it at a practical level, i.e. in the design of TEL activities. See a 38 page paper from this guy on ‘What is digital literacy?’ at

The SCONUL seven pillars of information literacy – core model – (Roger) aimed at Library staff engaged in information skills work. 7 pillars : 1. identify a personal need for info, 2. scope – assess current knowledge and identify gaps, 3. plan – construct strategies for finding info, 4. gather info, 5. evaluate – including reading critically and evaluating info, 6. manage – organise information professionally and ethically (academic integrity) , 7. present – including active creation of knowledge.

The SCONUL seven pillars of information literacy – research lens  (Suzi) rather complicated extended metaphor for digital-literacy

The Medium is not the Literacy (Suzi) – article critical of the term from 2002 – the example he gives about news is a bit dated (digital news is more than just reading news stories on a screen now) – but makes a good point about the shortcomings of the term

Digital Natives: Fact or Fiction? (Suzi) – Useful, brief critique of the term “digital natives” stating that it was based on opinion (not science)

Futurity Now: Bruce Sterling on Atemporality (Suzi) – Nice illustration of the changes we face – “old” Feynman on how science works (write down problem, think hard, write down answer) vs what we do now (start by Googling to see if someone already solved it, etc)

Education Technology Standards for Students – Nice clear breakdown of the components of digital literacy (what it should mean for students to be digitally literate) including: creativity, citizenship, critical thinking, research, collaboration, and an understanding of the technology infrastructure.

European e-Competance Framework – “A common European framework for ICT professionals in all industry sectors” (including education)

Suggested reading