Curriculum design – notes from the reading group

Exploring curriculum design approaches (report by Suzanne Collins)

Suzanne talked about her work exploring curriculum design approaches (separate blog post) where she looks at looks at methodologies such as ABC, Carpe Diem, CAIeRO and ELDeR.

ABC Learning Design (notes by Suzi Wells)

ABC learning design is a rapid design / review methodology developed by Clive Young and Nataša Perović in 2014, and drawing on Laurillard’s ‘Conversational Framework’.

The method is centered around 90 minute workshop, during which participants:

  • Describe their unit in a tweet
  • Map out the types of activity, currently undertaken or planned, against Laurillard’s six learning types
  • Storyboard the unit using the ABC cards

In the DEO a few of us – Roger Gardner, Suzanne Collins, and I – have trialled this approach. Initially this was with a small number of academics interested in redesigning their credit-bearing units. We made much fuller use of it when supporting the design of the FutureLearn courses, following which Suzanne and I presented on this at a UCL conference in 2018: our presentation on using ABC at Bristol.

One advantage of the methodology is that you could run a single 90 minute workshop looking at an entire programme, allowing potential links between the units to become apparent. The short length of the workshop gives at least some chance to get everyone from the unit together in one place.

The cards are Creative Commons licensed and have been widely adapted, adding in activities and terminology that is more relevant to their context. On the ABC site you can download ABC cards for MOOCs designed for FutureLearn and EdX. At the conference we heard how people have used stickers in the storyboarding stage to surface things that they are interested in: employability skills, alignment with the education strategy, and university-wide themes (such as Bristol’s global citizenship, innovation & enterprise, and sustainable futures themes).

Obviously a 90 minute workshop is not going to give you time to finalise many details but ABC is quick to learn, very adaptable, and sparks good conversations. It’s remarkable how much can be done in a short time.

Beyond podcasting: creative approaches to designing educational audio (notes by Chrysanthi Tseloudi)

This paper talks about a pilot that aimed to encourage academics to use podcasts in their teaching through a tool in their VLE. The pilot included initial workshops and various types of support for the 25 participants that decided to try it out. All participants used audio, apart from 1 team, which used video podcasts. 9 of them shared their experience for this paper. They had produced various types of resources: videos about clinical techniques (nursing), audio based on research projects which also received audio feedback from the academic (sport), “questions you’re afraid to ask” (art & design), answers to distance learning students’ questions to reduce the sense of isolation in the VLE (communications), etc.

Academics enjoyed using audio for learner-centred pedagogies, but they also encountered some barriers. Expectations for high quality may be a barrier for both staff and students, while assessing student work in this format is time consuming. Not being familiar with the technology can be frustrating and impeding for staff, as they would rather not ask students to do something they didn’t feel confident they could do well themselves. Students are not necessarily more confident than them in using this technology. Following the pilot, the institution’s capacity to support such activities was evaluated and some solutions to support staff were devised.

This was a nice paper with a variety of ideas on using audio for teaching. I found the point about voice on the VLE increasing connectivity and reducing isolation particularly interesting, and would love to see any relevant research on this.

Suggested reading

Approaches to curriculum design

Related articles

Digital Examinations Forum

Yesterday I attended the Digital Examinations Forum at the University of Bath which was an extremely useful day.  The event was very well-attended and we had the opportunity to hear from and discuss with colleagues from a range of institutions who are well advanced in the journey of implementing digital examinations.  Mostly for me the day re-emphasised some key points, summarised as follows, with a couple of related photos below:

  1. Have an assessment policy which specifically mentions appropriate use of digital, but emphasise the opportunities digital offers to do assessment differently and better, not just digitising existing practice or restricting yourselves to MCQs.  For example incorporate video into questions, use authentic case studies in digital format,  produce a mini piece of coursework under exam conditions, and there were lots more ideas
  2. Keep developing staff understanding of assessment, ensuring alignment with learning outcomes and activities
  3. Marking online is different to marking on paper. Allow colleagues time, space and support to get used to this.
  4. Standard processes help, but maintain flexibility in the approach – there will be no one tool/approach which meets all requirements
  5. Some things may just not be appropriate to digitise
  6. Educational leadership is needed at all levels to support this change
  7. It costs!  eg building / kitting out space, IT infrastructure (including for BYOD), support eg training for invigilators 

Finally a couple of other items of interest: there was plenty of encouraging feedback about the Inspera platform, and I hadn’t previously come across the work by Martin Bush and Lucia Otoyo from London South Bank on reducing the need for guesswork in multiple choice tests. There are some examples here: https://quizslides.co.uk/home , and a journal article from Assessment and Evaluation in Higher Education.

Astrid Birgitte Eggen from University of Agder on assessment policy

Anja Sisarica from Inspera on the need for flexibility in marking tools

Curriculum theories – notes from reading group

Thanks to Sarah Davies for setting us some fascinating reading!

Connected curriculum chapter 1 (notes from Chris Adams)

The connected curriculum is a piece of work by Dilly Fung from UCL. It is an explicit attempt to outline how departments in research-intensive universities can develop excellent teaching by integrating their research into it; the ‘connected’ part of the title is the link between research and teaching. At it’s heart is the idea that the predominant mode of learning for undergraduates should be active enquiry, but that rather than students discovering for themselves things which are well-established, they should be discovering things at the boundaries of what is known, just like researchers do.

It has six strands:

  • Students connect with researchers and with the institution’s research. Or, in other words, the research work of he department is explicitly built into the curriculum
  • A throughline of research activity is built into each programme. Properly design the curriculum so that research strands run though it, and it builds stepwise on what has come before.
  • Students make connections across subjects and out to the world. Interdisciplinarity! Real world relevance.
  • Students connect academic learning with workplace learning. Not only should we be teaching them transferable skills for a world of rapid technological change, but we need to tell them that too.
  • Students learn to produce outputs – assessments directed at an audience. Don’t just test them with exams
  • Students connect with each other, across phases and with alumni. This will create a sense of community and belonging.

This last point is then expanded upon. Fung posits that the curriculum is not just a list of what should be learned, but is the whole experience as lived by the student. Viewing the curriculum as a narrow set of learning outcomes does not product the kind of people that society needs, but is a consequence of the audit culture that pervades higher education nowadays. Not all audit is bad – the days when ‘academic freedom’ gave people tenure and the freedom to teach terribly and not do any research are disappearing, and peer-review is an integral part of the university system – but in order to address complex global challenges we need a values based curriculum ‘defined as the development of new understandings and practices, through dialogue and human relationships, which make an impact for good in the world.’

I liked it sufficiently to buy the whole book. It addresses a lot of issues that I see in my own department – the separation of research from teaching, and the over-reliance on exams, and the lack of community, for example.

Connected curriculum chapter 2 (notes from Suzi Wells)

As mentioned in chapter 1, the core proposition is that the curriculum should be ‘research-based’ – ie most student learning “should reflect the kinds of active, critical and analytic enquiry undertaken by researchers”.

Fung gives a this useful definition of what that means in practice. Students should:

  • Generate new knowledge through data gathering and analysis
  • Disseminate their findings
  • Refine their understanding through feedback on the dissemination

All of it seems fairly uncontroversial in theory and tends to reflect current practice, or at least what we aspire to in current practice. There’s some discussion of the differences in what research means to different disciplines, and how that filters through into assessment of students, and potentially some useful studies on just how effective this all is.

Fung mentions the Boyer Commission (US 1998) and its proposed academic bill of rights, including (for research intensive institutions): “expectation of and opportunity for work with talented senior researchers to help and guide the student’s efforts”. Given increasing student numbers, this is possibly a less realistic expectation to meaningfully meet than it once was.

There’s some useful discussion about what is needed to make research-based-teaching work.

I was particularly interested in the idea that providing opportunity for this form of learning isn’t everything. Socio-economic factors mean that students may have differing beliefs about their own agency. Fung cites Baxter-Magdola (2004) on the importance of students having ‘self-authorship’ which includes ‘belief in oneself as possessing the capacity to create new knowledge’ and ‘the ability to play a part within knowledge-building communities’. You can’t assume all students arrive with the same level of this, and this will affect their ability to participate.

This part of the chapter also talks about the importance of not just sending students off “into the unknown to fend for themselves” – imagine a forest of ivory towers – but to give them support & structure. Activities need to be framed within human interactions (including peer support).

Towards the end there is a nod to it being anglo-centric – African and Asian educational philosophy and practice may be different – but little detail is given.

How Emotion Matters in Four Key Relationships in Teaching and Learning in Higher Education” (notes from Roger Gardner)

This is a 2016 article by Kathleen Quinlan, who is now Director of the Centre for the Study of Higher Education and Reader in Higher Education at University of Kent, but was working at Oxford when this was written.

She writes that while historically there has been less focus on Bloom’s affective domain than the cognitive, recently interest in the relation of emotions to learning has been growing although it is still under-researched. The article comes out of a review of the existing literature and conversations with teachers at the National University of Singapore in August 2014.

The paper focusses on four relationships: students with the subject matter, teachers, their peers and what she calls “their developing selves”. For each section Quinlan includes a summary of implications for teaching practice, which provide some very useful suggestions, ranging from simple things such as encouraging students to introduce each other when starting activities to help foster peer relationships, to advocating further research and exploration into when it is appropriate and educationally beneficial for teachers to express emotions and when not.

Quinlan says “discussions about intangibles such as emotions and relationships are often sidelined”, but it now seems essential to prioritise this if we are to support student wellbeing, and this paper provides some helpful prompts and suggestions for reflection and developing our practice.  If you are short of time I recommend looking at the bullet point “implications for practice”.

What is “significant learning”? (notes from Chrysanthi Tseloudi)

In this piece, Dr. Fink talks about the Taxonomy of Significant Learning; a taxonomy that refers to new kinds of learning that go beyond the cognitive learning that Bloom’s taxonomy addresses. The taxonomy of significant learning – where significant learning occurs when there is a lasting change in the learner that is important in their life – is not hierarchical, but relational and interactive. It includes six categories of learning:

Foundational knowledge: the ability to remember and understand specific information as well as ideas and perspectives, providing the basis for other kinds of learning.

Application: learning to engage in a new kind of action (intellectual, physical, social, etc) and develop skills that allow the learner to act on other kinds of learning, making them useful.

Intergration: learning to see, understand, and make new connections between different things, people, ideas, realms of ideas or realms of life. This gives learners new (especially intellectual) power.

Human Dimension: learning about the human significance of things they are learning – understanding something about themselves or others, getting a new vision of who they want to become, understanding the social implications of things they have learned or how to better interact with others.

Caring: developing new feelings, interests, values and/ or caring more about something that before; caring about something feeds the learner’s energy to learn about it and make it a part of their lives.

Learning how to learn: learning about the learning process; how to learn more efficiently, how to learn about a specific method or in a specific way, which enables the learner to keep on learning in the future with increasing effectiveness.

The author notes that each kind of learning is related to the others and achieving one kind helps achieve the others. The more kinds of learning involved, the more significant is the learning that occurs – with the most significant kind being the one that encompasses all six categories of the taxonomy.

Education Principles: Designing learning and assessment in the digital age (notes from Naomi Beckett)

This short paper is part of a guide written by Jisc. It covers what Education Principles are and why they are such a vital characteristic of any strategy. Coming from someone unspecialised in this area it was an interesting read to understand how principles can bring staff together to engage and develop different education strategies. The guide talks about how principles can ‘provide a common language, and reference point for evaluating change’.

The paper talks about having a benchmark in which everyone can check their progress. I like this idea. So often projects become too big and the ideas and values are lost on what was first decided as a team. Having a set of principles is a way to bring everything back together and is a useful way to enable a wide variety of staff to engage with each other. The guide mentions how having these principles means there is a ‘common agreement on what is fundamentally important.’

Having these principles developed at the beginning of a project puts the important ideas and values into motion and is a place to look back to when problems arise. Principles should be action oriented, and not state the obvious. Developing them in this way allows for a range of staff members to bring in different ideas and think about how they want to communicate their own message.

I also followed up by reading ‘Why use assessment and feedback principles?’ from Strathclyde’s Re-Engineering Assessment Practices (REAP) project.

Suggested reading

Feedback, NSS & TEF – notes from reading group

Chrysanthi read “Thanks, but no-thanks for the feedback”. The paper examines how students’ implicit beliefs about the malleability of their intelligence and abilities influence how they respond to, integrate and deliberately act on the feedback they receive. It does so, based on a set of questionnaires completed by 151 students (113 females and 38 males), mainly from social sciences.

Mindset: There are two kinds of mindsets regarding malleability of one’s personal characteristics; People with a growth mindset believe that their abilities can grow through learning and experience; people with a fixed mindset believe they have a fixed amount of intelligence which cannot be significantly developed. “If intelligence is perceived as unchangeable, the meaning of failure is transformed from an action (i failed) to an identity (i am a failure)” (p851).

Attitudes towards feedbackSeveral factors that influence whether a person accepts a piece of feedback – e.g. how reflective it is of their knowledge and whether it is positive or negative – were measured, as well as 2 outcome measures.

Defence mechanisms: Defence mechanisms are useful in situations we perceive as threatening, as they help us control our anxiety and protect ourselves. But if we are very defensive, we are less able to perceive the information we receive accurately, which can be counterproductive; e.g. a student may focus on who has done worse, to restore their self-esteem, rather than who has done better, which can be a learning opportunity.

The results of the questionnaires measuring the above showed that more students had a fixed mindset (86) than growth (65) and that their mindset indeed affected how they responded to and acted on feedback.

  • Growth mindset students are more likely to challenge themselves and see the feedback giver as someone who can push them out of their comfort zone in a good way that will help them learn. They are more motivated to change their behaviour in response to the received feedback, engage in developmental activities and use the defence mechanisms considered helpful.
  • Fixed mindset students are also motivated to learn, but they are more likely to go about it in an unhelpful way. They make choices that help protect their self-esteem, rather than learn, they are not as good at using the helpful defence mechanisms, they distort the facts of the feedback or think of an experience as all good or all bad. The authors seemed puzzled by the indication that fixed students are motivated to engage with the feedback, but they do so by reshaping reality or dissociating themselves from the thoughts and feelings surrounding said feedback.

Their recommendations?

  • Academics should be careful in how they deliver highly emotive feedback, even if they don’t have the time to make it good and individualised.
  • Lectures & seminars early in students’ studies, teaching them about feedback’s goal and related theory and practice, as well as action action-orientated interventions (eg coaching), so they learn how to recognize any self-sabotaging behaviours and manage them intelligently.
  • Strategies to help students become more willing to experience – and stay with – the emotional experience of failure. Eg, enhance the curriculum with opportunities for students to take risks, so they become comfortable with both “possibility” and “failure”.

I think trying to change students’ beliefs about the malleability of their intelligence would go a long way. If one believes their abilities are fixed and therefore if they don’t do well, they are a failure, a negative response to feedback is hardly surprising. That said, the responsibility of managing feedback should not fall entirely on the student; it still needs to be constructive, helpful and given in an appropriate manner.

Suzi read: An outsider’s view of subject level TEFA beginner’s guide to the Teaching Excellence FrameworkPolicy Watch: Subject TEF year 2 by the end of which she was not convinced anyone knows what the TEF is or how it will work.

Some useful quotes about TEF 1

Each institution is presented with six metrics, two in each of three categories: Teaching QualityLearning Environment and Student Outcomes and Learning Gain. For each of these measures, they are deemed to be performing well, or less well, against a benchmarked expectation for their student intake.

… and …

Right now, the metrics in TEF are in three categories. Student satisfaction looks at how positive students are with their course, as measured by teaching quality and assessment and feedback responses to the NSS. Continuation includes the proportion of students that continue their studies from year to year, as measured by data collected by the Higher Education Statistics Agency (HESA). And employment outcomes measures what students do (and then earn) after they graduate, as measured by responses to the Destination of Leavers from Higher Education survey – which will soon morph into Graduate Outcomes.

Points of interest re TEF 2

  • Teaching intensity (contact hours) won’t be in the next TEF
  • All subjects will be assessed (at all institutions), with results available in 2021
  • Insufficient data for a subject at an institution could lead to “no award” (so you won’t fail for being too small to measure)
  • Resources will be assessed
  • More focus on longitudinal educational outcomes, not (binary) employment on graduation
  • It takes into account the incoming qualifications of the students (so it does something like the “value add” thing that school rankings do) but some people have expressed concern that it will disincentivise admitting candidates from non-traditional backgrounds.
  • There will be a statutory review of the TEF during 2019 (reporting at the end of the year) which could change anything (including the gold / silver / bronze rankings)

Suzi also read Don’t students deserve a TEF of their own which talks about giving students a way in to play with the data so that, for example, if you’re more interested in graduate career destinations than in assessment & feedback you can pick on that basis (not on the aggregated data). It’s an interesting idea and may well happen but as a prospective student I can’t say I understood myself — or the experience of being at university — well enough for that to be useful. There’s also a good response talking about the kind of things (the library is badly designed, lectures are at hours that don’t make sense because rooms are at a premium, no real module choice) you might find out too late about a university that would not be covered by statistics.

Roger read “How to do well in the National Student Survey (NSS)” an article from Wonkhe,  written in March 2018. The author, Adrian Burgess, Professor of Psychology at Aston University, offers some reflections based on an analysis of NSS results from 2007 to 2016.

Whilst many universities have placed great emphasis on improving assessment and feedback, this has “brought relatively modest rewards in terms of student satisfaction” and remains the area with the lowest satisfaction.

Burgess’ analysis found that the strongest predictors of overall satisfaction were “organisation and management” closely followed by “teaching quality”.

Amy read Feedback is a two-way street. So why does the NSS only look one way?, an article by Naomi Winstone and Edd Pitt. This piece highlighted the issue that the NSS questions on feedback are framed as if feedback should be a passive experience – that students should be given their feedback. In 2017, the question was changed from “I have received detailed comments” to “I have received useful comments”. Both the old and new question frames feedback as something that is received, a ‘transmission-focussed mindset’, whereas Winstone and Pitt argue that feedback should be a two-way relationship – with the student working with the feedback and their tutor to develop.
The authors do not believe that changing the NSS question will solve all of the problems with students perception of feedback (though it will definitely help!) but they do believe that by promoting feedback as something that individuals work with, have responsibility for and seek out if they feel they need to develop in a certain area, that gradually the mindset will change and become a more sustainable form of learning for students.

Suggested reading

From WonkHE

From the last time we did assessment & feedback, which was July 2017 (I’ve left in who read what then)

Thoughts from a recent GW4 meeting at University of Bath

 

On Friday 23rd March, Mike, Naomi, Robyn, Han and I headed over to Bath for the latest GW4 meeting of minds. As decided in the previous meeting, the main topics for discussion were e-assessment and portfolios, but we also discussed MOOC development and learning analytics. Unfortunately, no one from Exeter could make it up this time, so it was us from Bristol, along with colleagues from Bath and Cardiff. As before, we used Padlets to pool ideas and discussion points as we discussed in smaller groups.

Portfolios 

Portfolios seem to be a common focus (dare I even say, headache). Bath and Cardiff have been using Mahara, and have been trying to overcome some of its limitations in-house. There was a strong feeling that none of us have found a portfolio which delivers what we need, and that if we ganged up on the providers they might be able to find a solution. The next step is to try to define what it is we do need from a portfolio, which tools we use (or have already investigated), and what we can do to find a common solution. Some immediate themes were e-portfolios as assessment tools (and how they integrate with current systems), GDPR implications, students being able to share parts of portfolios externally and internally, and how long students can have access to their portfolio.

MOOCS

As something we all have experience of, to a greater or lesser degree, there was inevitably quite a bit of discussion around MOOCs. We talked about the processes we follow to develop MOOCs, and the different support we provide to academics. For example, Gavin from Bath showed us how he uses Camtasia to produce videos in house; in fact, he was able to knock up an example of such a video in 20 minutes during the session, with mini interviews and shots from the day. We also discussed the data we get from FutureLearn, and how we all find it difficult to do anything with that data. With so much information, and not much time, it tends to become something we’d all like to do more with but never quite find the time for.

The discussion also retuned to an idea we’ve been kicking around GQ4 for a while, that of a collaborative MOOC. We discussed the idea of perhaps making courses for pre-entry undergrads, or students embarking on PhDs, or perhaps staff development and CPD courses for new academics (which Cardiff are already building a bank of in FutureLearn). The idea of creating small modular courses or MOOCS, where each of us could provide a section which is based on our own expertise and interests, was also popular…let’s see how this develops!

E-assessment

Tools and systems around e-assessment was also a common theme. As well as thinking about Blackboard assignments, use of Turnitin and QMP, there was also talk about peer assessment tools and practice and adopting a BYOD approach. It seemed that we all had experiences of e-assessment being very mixed, with huge disparity in adoption and approach within our institutions. We’re all working on e-assessment, it seems, for example our EMA project, which is quite similar to that of Bath. However, other trials are also going ahead, such as Cardiff’s trial of ‘Inspera‘. I think we’re all keen to see what their experiences of that project are, as the Scandinavian approach to e-exams has often been heralded as the future!

What next?

For the future, we discussed more of a ‘show and tell’ approach, where we could get a closer look at some of the things we’re up to. There was also talk of upping our use of communication channels in between meeting in person, particularly using the Yammer group more frequently, and perhaps having smaller virtual meetings for specific topics.

It wasn’t decided who would host the next session, particularly as Exeter weren’t represented, although we did tentatively offer to host here at Bristol. But, seeing as Bath really did set the bar high for lunch expectations – with special mention to the excellent pies and homemade cake – if we do host I think we’d better start planning the food already…!

 

 

Flexible and inclusive learning – notes from reading group

Amy read: Why are we still using LMSs, which discusses the reasons LMS systems have not advanced dramatically since they came onto the market. The key points were:

  • There are five core features that all major LMS systems have: they’re convenient; they offer a one-stop-shop for all University materials, assessments and grades; they have many accessibility features built in; they’re well integrated into other institutional systems and there is a great deal of training available for them.
  • Until a new system with all these features comes onto the market, the status quo with regard to LMS systems will prevail.
  • Instructors should look to use their current LMS system in a more creative way.

Mike read: Flexible pedagogies: technology-enhanced learning HEA report

This paper provided a useful overview of flexible learning, including explanations of what it might mean, dilemmas and challenges for HE. The paper is interesting to consider alongside Bristol’s Flexible and Inclusive learning paper. For the authors, Flexible learning gives students choice in the pace, place and mode of their learning. This is achieved through application of pedagogical practice, with TEL positioned as an enable or way of enhancing this practice. Pace is about schedules (faster or slower), or allowing students to work at their own pace. Place is about  physical location and distance. Mode includes notions of distance and blended learning.

Pedagogies covered include personalised learning, flexible learning – (suggesting it is similar to adaptive learning in which materials adapt to individual progress), gamification, fully online and blended approaches. The paper considers the implications of offering choice to students for example, over what kind of assessment. An idealised form would offer a very individualised choice of learning pathway, but with huge implications on stakeholders.

In the reading, group, we had an interesting discussion as to whether students are always best equipped to understand and make such choices. We also wondered how we would resource the provision of numerous pathways.  Other  risks include potential for information overload for students, ensuring systems and approaches work with quality assurance processes. Barriers include interpretations of KIS data which favours contact time.

We would have a long way to go in achieving the idealised model set out here. Would a first step be to change the overall diet of learning approaches across a programme, rather than offering choice at each stage? Could we then introduce some elements of flexibility in certain areas of programmes, perhaps a bit like the Medical School’s Self Selected Components, giving students choice in a more manageable space within the curriculum?

Suzanne read:  Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. The main points were:

  • Self-regulated learning is something which happens naturally in HE, as students will assess their own work and give themselves feedback internally. This paper suggests this should be harnessed and built on in feedback strategies in HE.
  • Shift in focus to see students having a proactive rather than reactive role in feedback practices, particularly focused on deciphering, negotiating and acting on feedback.
  • The paper suggests 7 principles for good feedback practice, which encourages this self-regulation: 1. clarifying what good performance is; 2. facilitating self-assessment; 3. delivering high quality feedback information; 4. encouraging dialogue; encouraging self-esteem and motivation; 6. giving opportunities to close the gap between where the student is now and where they need/want to be; 7. using feedback to improve teaching.
  • For our context, this gives some food for thought in terms of the limitations of a MOOC environment for establishing effective feedback practices (dialogue with every student is difficult if not impossible, for example), and emphasises the importance of scaffolding or training effective peer and self-assessment, to give students the confidence and ability to ‘close the gap’ for themselves.

Suzanne also read: Professional Development Through MOOCs in Higher Education Institutions: Challenges and Opportunities for PhD Students Working as Mentors

This paper reports on a small-scale (20 participants), qualitative study into the challenges and opportunities for PhD students acting as mentors in the FutureLearn MOOC environment. As a follow-on from the above reading, using mentors can be a way to help students with the peer and self-assessment practices, which is why I decided to read it in parallel. However, it also focuses on the learning experiences of the PhD student themselves as they perform the mentor role, also giving these students a different (potentially more flexible and inclusive) platform to develop skills.

Overall, the paper is positive about the experiences of PhD MOOC mentors, claiming that they can develop skills in various areas, including:

  • confidence in sharing their knowledge and interacting with people outside their own field (especially for early career researchers, who may not yet have established themselves as ‘expert’ in their field);
  • teaching skills, particularly related to online communication, the need for empathy and patience, and tailoring the message to a diverse audience of learners. It’s noteworthy here that many of these mentors had little or no teaching experience, so this is also about giving them teaching experience generally, not teaching in MOOCs specifically;
  • subject knowledge, as having to discuss with the diverse learning community (of expert and not expert learners) helped them consolidate their understanding, and in some cases pushed them to find answers to questions they had not previously considered.

Roger read Authentic and Differentiated Assessments

This is a guide aimed at School teachers. Differentiated assessment involves students being active in setting goals, including the topic, how and when they want to be evaluated. It also involves teachers continuously assessing student readiness in order to provide support and evaluate when students are ready to move on in the curriculum.

The first part of the article describes authentic assessment, which it defines as asking students to apply knowledge and skills to real world settings, which can be a powerful motivator for them. A four stage process to design authentic assessment is outlined.

The second part of the article focuses on differentiated assessment. We all have different strengths and weaknesses in how we best demonstrate our learning, and multiple and varied assessments can help accommodate these. The article stresses that choice is key, including of learning activity as well as assessment. Project and problem based learning are particularly useful.  Learning activities should always consider multiple intelligences and the range of students’ preferred ways of learning, and there should be opportunities for individual and group tasks as some students will perform better in one or the other.

Hannah read: Research into digital inclusion and learning helps empower people to make the best choices, a blog by the Association for Learning and Teaching about bridging the gap between digital inclusion and learning technology. The main points were:

  • Britain is failing to exploit opportunities to give everyone fair and equal access to learning technology through not doing enough research into identifying the best way to tackle the problem of digital exclusion
  • Learning technology will become much more inclusive a way of learning once the digital divide is addressed
  • More must be done to ensure effective intervention; lack of human support and lack of access to digital technology are cited as two main barriers to using learning technology in a meaningful way
  • We need to broaden understanding of the opportunities for inclusion, look into how to overcome obstacles, develop a better understanding of the experiences felt by the excluded and understand why technological opportunities are often not taken up

Suzi read:  Disabled Students in higher education: Experiences and outcomes which discusses the experience of disabled students, based on surveys, analysis of results, interviews, and case studies at four, relatively varied, UK universities. Key points for me were:

  • Disability covers a wide range of types and severity of issues but adjustments tend to be formulaic, particularly for assessment (25% extra time in exams)
  • Disability is a problematic label, not all students who could do will choose to identify as disabled
  • Universal design is the approach they would advocate where possible

Suzi also read: Creating Better Tests for Everyone Through Universally Designed Assessments a paper written for the context of large-scale tests for US school students, which nonetheless contains interesting background and advice useful (if not earth-shattering). The key messages are:

  • Be clear about what you want to assess
  • Only assess that – be careful not to include barriers (cognitive, sensory, emotional, or physical) in the assessment that mean other things are being measured
  • Apply basic good design and writing approaches – clear instructions, legible fonts, plain language

 

Teaching at scale: engagement, assessment and feedback – notes from reading group

Chris read #53ideas 27 – Making feedback work involves more than giving feedback – Part 1 the assessment context. A great little paper full of epithets that perfectly describe the situation I find myself in. ‘You can write perfect feedback and it still be an almost complete waste of time’. ‘University policies to ensure all feedback is provided within three weeks seem feeble’.’On many courses no thought has been given to the purpose of the learning other than that there is some subject matter that the teacher knows about’. ‘Part time teachers are seldom briefed properly about the course and its aims and rationale, and often ignore criteria’. The take home message, for me, was that the OU is an exemplar in the area of giving good, useful, consistent feedback even when the marking load is spread over a number of people: ‘If a course is going to hire part-time markers then it had better adopt some of the Open University’s practices or suffer the consequences.’

Jane recommended: Sea monsters& whirlpools: Navigating between examination and reflection in medical education. Hodges, D. (2015). Medical Teacher 37: 3, 261-266. Interesting paper around how diverse forms of reflective practice employed by medical educators are compatible with assessment. She also mentioned “They liked it if you said you cried”: how medical students perceive the teaching of professionalism

Suzi read E-portfolios enhancing students’ self-directed learning: a systematic review of influencing factors

This 2016 paper is based on a systematic literature review of the use of online portfolios, with most of the studies taking place in an HE context. They looked at what was required for portfolio use to foster self-directed learning. Their conclusions were that students need the time and motivation to use them, and also that portfolios must:

  • Be seamlessly-integrated into teaching
  • Use appropriate technology
  • Be supported by coaching from staff (this is “important if not essential”)

Useful classification: purpose (selection vs learning) and volition (voluntary vs mandated) from Smith and Tillema (2003). Useful “Practical implications” section towards the end.

Suzi read How & Why to Use Social Media to Create Meaningful Learning Assignments

A nice example of a hypothetical (but well thought-through) Instagram assignment for a history of art course, using hashtags and light gamification. Included good instructions and motivation for students.

Has some provocative claims about the use of social media:
“It’s inevitable if we want to make learning relevant, practical and effective.”
“social media, by the behaviours it generates, lends itself to involving students in learning”
Also an interesting further reading section.

Suzi read #53ideas 40 – Self assessment is central to intrinsic motivation

Feeling a sense of control over learning leads to higher levels of engagement and persistence. If possible this would be the what, how, where and when. But “taking responsibility for judgements about their own learning” – so good self & peer assessment – may be enough. Goes through an example of self & peer assessment at Oxford Polytechnic. Challenging to our context, as this was highly scaffolded, with the students practicing structured self-assessment for a year before engaging in peer assessment. Draws on Carl Rogers principles for significant learning. Interesting wrt the need to create a nurturing, emotionally supportive space for learning.

Suggested reading

Engagement and motivation

Social media and online communities

Assessment and feedback

More general, learning at scale