Evaluating E-learning, A Guide to the Evaluation of E-learning, Graham Attwell, 2006 (Suzi) – Framework of factors to take into consideration, then lists 5-6 clusters of types of evaluation. Not much detail of how these types of evaluation actually work, but may be useful as a starting point. Section 6: SPEAK tool – used in community education – may be useful for “students as agents of change”. Section 9 – evaluating e-learning policies – gives guidelines and prompts for developing a set of questions with which to evaluate a policy. Section 10 – management-oriented evaluation – possibly worth a look during project planning.

Evaluation of e-learning courses– Institute of Education , 2008  (Roger)   Covers evaluating courses, wholly or partly online.  Aims to provide an overview of practical evaluation resources. And its target audience is academics at IOE. Includes a literature review.Recommendations:

• Plan evaluation before the course starts
• Collect feedback from all stakeholders including students, tutors, admins and tech support staff.   For staff this could be done on an ongoing basis through frequent team meetings, and an end of course survey
• Collect student feedback during and at the end of the course
• Consider all relevant aspects of the use of tech for t and l in the course e.g. usefulness of the  content, how well online  activities run e.g. timing, sequencing (could be blended), instructions, the user experience ( levels of engagement, tutor participation, workload)
• Make use of the specific tools available in typical online tools e.g. course stats in BB to get an idea of levels of activity (not quality)

Suggested reading

• A Technique for Quality Evaluation of E-Learning from Developers Perspective
• Online Education Evaluation: What Should We Evaluate?
• Transition and technology—Evaluation of blended learning, delivered by university staff to 6th form students
• Models and instruments for assessing Technology, Enhanced Learning Environments in higher. education
• e-learning benchmarks compared
• e-learning maturity model from Victoria University of Wellington – 5 process categories: Learning, Development, Support, Evaluation and Organisation – each process is broken down into practices, which are then rated
• HEA e-learning benchmarking methodologies
• HEA e-learning benchmarking Bibliography
• Jisc Curriculum design and delivery – http://jisccdd.jiscinvolve.org/wp/2011/11/09/tracks-in-the-snow-finding-and-making-sense-of-the-evidence-for-institutional-transformation/
• JISC – tangible benefits of e-learning
• Educause – Maintaining Quality in Blended Learning: From Classroom Assessment to Impact Evaluationhttp://www.educause.edu/Resources/MaintainingQualityinBlendedLea/213889
• Evaluation of e-learning courses WLE centre

There was discussion about this on the ALT list in May 2011. References given include:

• http://www.eep-edu.org/InnService/Start/what_addval_start.htm   – aimed at schools but still relevant
• https://sites.google.com/site/evaluationtoolkit/
• http://www.as.bham.ac.uk/learningspaces/LEI/Toolkit.shtml – University of Birmingham LT adoption toolkit
• http://www.sciencedirect.com/science/article/pii/S0360131505001338 – Critical success factors for e-learning acceptance UAE University
• http://www.sciencedirect.com/science/article/pii/S0360131506001874 – What drives successful e-learning

Information and digital competencies in HE  (Roger) – read one article in this collection  “HE and the knowledge society. Information and digital competencies”. (by Juan de Pablos Pons, 2010)  Knowledge society requires changes to teaching models – lecturers no longer the only source of knowledge, opportunities for technology to help the social dimension of teaching (“socialization of knowledge”). Universities are too compartmentalised, their structures and offer needs to reflect more the interdisciplinary nature of modern knowledge and research.   Distinguishes between IT competencies (how to use ICT) and “information competencies” (which would relate to more of Doug Belshaw’s 8 elements, e.g. criticality, creativity, cognitive etc. Lastly universities need to recognise that they are educating each student for multiple jobs (and these may not yet even exist)

The essential elements of digital literacies, Slideshare (Nic) This presentation offers a useful description of digital literacy (DL), i.e. DL is often understood as the ability to participate in a range of activities and creative practices that involve understanding, sharing and creating meaning with different kinds of media and technology”. This emphasise ICT and media. The presenation also introduces a ‘5-step process model’ and a matrix that can be used to map steps to different technologies. It also describes 8 essential elements of DL. Ones I found particularly interesting were:  As online texts are not exactly written, cyber literacy needs a different form of critical thinking; society is increasingly look for people who can confidently solve their own problem and manage their own life long learning, qualities ICT is believed to promote; DL must involve systemic awareness of how digital media are constructed. From our discussion, I’ve realised that althgough the term DL is complex and contested, the topic has an important relationship to TEL. Also, DL goes beyond reading (and writing) texts to actively and collaboratively creating meaning. This is where we, as learning technologist, can engage with it at a practical level, i.e. in the design of TEL activities. See a 38 page paper from this guy on ‘What is digital literacy?’ at http://neverendingthesis.com/doug-belshaw-thesis-as-submitted.pdf

The SCONUL seven pillars of information literacy – core model – (Roger) aimed at Library staff engaged in information skills work. 7 pillars : 1. identify a personal need for info, 2. scope – assess current knowledge and identify gaps, 3. plan – construct strategies for finding info, 4. gather info, 5. evaluate – including reading critically and evaluating info, 6. manage – organise information professionally and ethically (academic integrity) , 7. present – including active creation of knowledge.

The SCONUL seven pillars of information literacy – research lens  (Suzi) rather complicated extended metaphor for digital-literacy

The Medium is not the Literacy (Suzi) – article critical of the term from 2002 – the example he gives about news is a bit dated (digital news is more than just reading news stories on a screen now) – but makes a good point about the shortcomings of the term

Digital Natives: Fact or Fiction? (Suzi) – Useful, brief critique of the term “digital natives” stating that it was based on opinion (not science)

Futurity Now: Bruce Sterling on Atemporality (Suzi) – Nice illustration of the changes we face – “old” Feynman on how science works (write down problem, think hard, write down answer) vs what we do now (start by Googling to see if someone already solved it, etc)

Education Technology Standards for Students – Nice clear breakdown of the components of digital literacy (what it should mean for students to be digitally literate) including: creativity, citizenship, critical thinking, research, collaboration, and an understanding of the technology infrastructure.

European e-Competance Framework – “A common European framework for ICT professionals in all industry sectors” (including education)

Suggested reading

• TLRP briefing – digital literacies
• JISC Digital literacies programme
• Learning literacies in a digital age 
• Next generation user skills
• Information literacy in Europe – Literature review 
• What is ‘digital literacy’? A Pragmatic investigation, Doug Belshaw – thesis 
• Developing digital literacies – briefing paperhttp://www.jisc.ac.uk/media/documents/funding/2011/04/Briefingpaper.pdf