Online communities – notes from the reading group

Amy read Professors share ideas for building community in online courses. The over-arching narrative of this piece was that ‘humanizing learning’ was the most effective was to build online learning communities, which occurs when students connect on a emotional and social level when engaging with the community. The author, Sharon O’Malley, suggest six methods for achieving this:

  1. Let students get to know you – instructors need to present themselves as ‘real people’ – this can be done by appearing goofy or telling genuine anecdotes in videos, for example. Students should also be encouraged to reveal their non-academic lives, in order for others to feel more like they know them personally, rather than just in the learning context
  2. Incorporating video and audio resources and feedback
  3. Meet in real time – students can talk to each other in real time and make instant connections
  4. Work in small groups – students get connected with others in their group – instead of feeling like they’re in a class of fifty, they feel they are in a class of 5, 10 etc.
  5. Require constant interaction – group projects and collaborative writing assignments force students to engage with each other out of the session
  6. Rise to the challenge – building community takes time – it takes planning and experimentation. Stick with it if it doesn’t immediately work!

Roger introduced a Building learning communities card activity. This is an activity from QAA Scotland, designed to stimulate discussion about what helps an effective learning community. The activity cards suggest the following factors:

  • Clearly defined and inclusive values
  • A clearly articulated and shared purpose
  • Clearly articulated and shared purpose goals
  • Active and vibrant interaction
  • Owned and managed by its people
  • Dedicated structure
  • Collaboration
  • Adequate and appropriate support
  • Understood and respected expectations
  • Adequate and appropriate resources
  • Built in evaluation

The instructions ask the group to consider which of these are essential and which are “nice to haves”.   The activity was certainly effective in stimulating discussion in reading group.]

Suzi watched Building Community: A Conversation with Dave Cormier – a recording of an edX webinar from 2014 – video. Here Cormier, who coined the term MOOC, talks to edX about how they could and should use online learning communities.

Cormier talks about four models of learning that you could scale up online:

  • One-to-one (adaptive learning, tutoring on skype?)
  • One-to-many (video lectures on MOOCs)
  • Cooperative learning: many-to-many, all working on the same thing
  • Collaborative learning: many-to-many, shared interest but each with own project

Collaborative learning is the one which he thinks is particularly – perhaps only – served by online communities. The real life equivalent being chaos, or maybe conferences (which, arguably, don’t work well for learning).

He draws the distinction between mastery learning (where skills can be ticked off a list as you progress) and complexity. Communities are not a particularly useful tool for mastery, or for checking who has learnt what. They are much better suited for complexity. This seemed to echo discussions we’d had about the difference between using gamification and using playfulness in learning – gamification being more for mastery, playfulness for complexity.

Cormier offers some tips on building a successful community.

  • A community should have, should move people towards building, shared values and a shared language.
  • Drive participation by having a famous person (but this can become one-to-many) or by asking annoying questions that people can’t resist engaging with (eg “how do we recognise cheating as a valuable part of education?”).
  • Shape participation by assigning roles to people and having course leader presence to set the tone.
  • Give people ways to get to know each other and make connections: recognising who people are and recognising aspects of yourself in them.

His view on evaluation and measuring success might be more specific to the MOOC context. He suggests borrowing techniques from advertising to demonstrate their value (but he doesn’t give details). The outcomes he suggests you might hope for are things like building more interest in your research area, or building the brand of an academic / department / institution.

He also asks some interesting questions. About the authenticity of work we give to students – how will their work persist? Can it be right that so much of students work is destined to be thrown away? About life beyond the community – how will the community persist? Communities are emotional – you shouldn’t just pull the plug at the end.

Lots of this is challenging in an educational context. For instance, communities take time to build but we generally work with units that last for a teaching block at most. Our online Bristol Futures courses only last four weeks. I wonder if this is to do with setting expectations. Perhaps we need thin and thick communities: the thin communities being time-bound but with much more scaffolding and a narrower purpose, the thick communities being more what Cormier is talking about here.

I also read The year we wanted the internet to be smaller (on the growth of niche communities in 2017) and 11 tips for building an engaged online community (practical advice aimed at NGOs). Both are interesting in their own right and worth a read. In both the idea of shared values, shared language and a sense of purpose came up. They also talk also recognition: communities as a place where you find “your people”. This resonates with my positive experiences of online communities but is, again, challenging in an education context. As Suzanne pointed out I think – if the tone and being among “your people” is important you must be able to walk out and find something different if you don’t feel comfortable. And it may be far better that you work with people who aren’t just  “your people”, or at least who don’t start that way.

Suggested reading

Online communities in education

From other sectors

Education communities – articles that are 10+ years old

Suggested listening

Evidence – notes from the reading group

Suzi read Real geek: Measuring indirect beneficiaries – attempting to square the circle? From the Oxfam Policy & Practice blog. I was interested in the parallels with our work:

  • They seek to measure indirect beneficiaries of our work
  • Evaluation is used to improve programme quality (rather than organisational accountability)
  • In both cases there’s a pressure for “vanity metrics”
  • The approaches they talk about sound like an application of “agile” to a fundamentally non-technological processes

The paper is written at an early point in the process of redesigning their measurement and evaluation of influencing. Their aim is to improve the measurement of indirect beneficiaries at different stages of the chain, adjust plans, “test our theory of change and the assumptions we make”. Evaluation is different when you are a direct service provider than when you are a “convenor, broker or catalyst”. They are designing an evaluation approach that will be integrated into day to day running of any initiative – there’s a balance between rigor and amount of work to make it happen.

The approach they are looking at – which is something that came up in a number of the papers other people read – is sampling: identifying groups of people who they expect their intervention to benefit and evaluating it for them.

Linked to from this paper was Adopt adapt expand respond – a framework for managing and measuring systemic change processes. This paper presents a set of reflection questions (and gives some suggested measures) which I can see being adapted for an educational perspective:

  • Adopt – If you left now, would partners return to their previous way of working?
  • Adapt – If you left now, would partners build upon the changes they’ve adopted without us?
  • Expand – If you left now, would pro-poor outcomes depend on too few people, firms, or organisations?
  • Respond – If you left now, would the system be supportive of the changes introduced (allowing them to be upheld, grow, and evolve)?

Roger read “Technology and the TEF” from the 2017 Higher Education Policy Institute (HEPI)  report “Rebooting learning for the digital age: What next for technology-enhanced higher education?”.

This looks at how TEL can support the three TEF components, which evidence teaching excellence.

For the first TEF component, teaching quality, the report highlights the potential of TEL in increasing active learning, employability especially digital capabilities development, formative assessment, different forms of feedback and EMA generally, and personalisation. In terms of evidence for knowing how TEL is making an impact in these areas HEPI emphasises the role of learning analytics.

For the second component, learning environment, the report focusses on access to online resources, the role of digital technologies in disciplinary research-informed teaching, and again learning analytics as a means to provide targeted and timely support for learning. In terms of how to gather reliable evidence it mentions the JISC student digital experience tracker, a survey which is currently being used by 45 HE institutions.

For the third component, student outcomes and learning gain, the report once again highlights student digital capabilities development whilst emphasising the need to support development of digitally skilled staff to enable this. It also mentions the potential of TEL in developing authentic learning experiences, linking and networking with employers and showcasing student skills.

The final part of this section of the report covers innovation in relation to the TEF.  It warns that “It would be a disaster” if the TEF stifled innovation and increased risk-averse approaches in institutions. It welcomes the inclusion of ’impact and effectiveness of innovative approaches, new technology or educational research’ in the list of possible examples of additional evidence as a “welcome step.” (see Year 2 TEF specification Table 8)

Mike read  Sue Watling – TEL-ing tales, where is the evidence of impact and In defence of technology by Kerry Pinny. These blog posts reflect on an email thread started by Sue Watling in which she asked for evidence of the effectiveness of TEL. The evidence is needed if we are to persuade academics of the need to change practice.  In response, she received lots of discussion, including and what she perceived to be some highly defensive posts.  The responses contained very little by way of well- researched evidence. Watling, after Jenkins, ascribes ‘Cinderella Status’ to TEL research, which I take to mean based on stories, rather than fact.  She acknowledges the challenges of reward, time and space for academics engaged with TEL. She nevertheless  makes a pleas that we are reflective in our practice and look to gather a body of evidence we can use in support of the impact of TEL. Watling describes some fairly defensive responses to her original post (including the blog post from James Clay that Hannah read for this reading group). By contrast. Kerry Pinny’s post responds to some of the defensiveness, agreeing with Watling – if we can’t defend what we do with evidence, then this in itself is evidence that something is wrong.

The problem is clear, how we get the evidence is less clear. One point from Watling that I think is pertinent is that it is not just TEL research, but HE pedagogic research as a whole, that lacks evidence and has ‘Cinderella status’. Is it then surprising that TEL HE research, as a  subset of  HE pedagogic research, reflects the lack of proof and rigour? This may in part be down to the lack of research funding. As Pinny points out, it is often the school or academic has little time to evaluate their work with rigour.  I think it also relates to the nature of TEL as a  set of tools or enablers of pedagogy, rather than a singular approach or set of approaches. You can use TEL to support a range of pedagogies, both effective and non-effective, and a variety of factors will affect its impact.  Additionally, I think it relates to the way Higher Education works – the practice there is and what evidence results tends to be very localised, for example to a course, teacher or school. Drawing broader conclusions is much, much harder.  A lot of the evidence is at best anecdotal. That said, in my experience, anecdotes (particularly form peers) can be as persuasive as research evidence in persuading colleagues to change practice (though I have no rigorous research to prove that).

Suzanne read Mandernach, J. 2015, ” Assessment of Student Engagement in Higher  Education: A Synthesis of Literature and Assessment Tools“, International Journal of Learning, Teaching and Educational Research Vol. 12, No. 2, pp. 1-14, June 2015

This text was slightly tangential, as it didn’t discuss the ideas behind evidence in TEL specifically, but was a good example of an area in which we often find it difficult to find or produce meaningful evidence to support practice. The paper begins by recognising the difficulties in gauging, monitoring and assessing engagement as part of the overall learning experience, despite the fact that engagement is often discussed within HE. Mandernach goes back to the idea of ‘cognitive’ ‘behavioural’ and ‘affective’ criteria for assessing engagement, particularly related to Bowen’s ideas that engagement happens with the leaning process, the object of study, the context of study, and the human condition (or service learning). Interestingly for our current context of building MOOC-based courses, a lot of the suggestions for how these engagement types can be assessed is mainly classroom based – for example the teacher noticing the preparedness of the student at the start of a lesson, or the investment they put into their learning. On a MOOC platform, where there is little meaningful interaction on an individual level between the ‘educator’ and the learner, this clearly becomes more difficult to monitor, and self-reporting becomes increasingly important. In terms of how to go about measuring and assessing engagement, student surveys are discussed – such as the Student Engagement Questionnaire and the Student Course Engagement Questionnaire. The idea of experience sampling – where a selection of students are asked at intervals to rate their engagement at that specific time – is also discussed as a way of measuring overall flow of engagement across a course, which may also be an interesting idea to discuss for our context.

Suggested reading