Finding evidence

There are a couple of things that make patterns distinct from other forms of design guidance. One is that they are routed in practice – they are not simply “good ideas” or theories (not that there is anything wrong with those) but they are drawn from real examples of things that have worked to solve problems in specific contexts. Another is that they cannot just be based on a single example – a pattern can only really be called a pattern if it can be demonstrated to have worked in at least three distinct cases. We call this the rule of three. We recently reviewed all the “patterns” that have arisen from our workshops and most fall into the “candidate pattern” category – we think they might be patterns, we have one – occasionally two examples of the pattern – but we don’t yet have sufficient evidence to be confident that this really is a pattern.

So we are looking for evidence in the stories of successful practice that we all have as educators. Our Elluminate story telling session on Tuesday with the EXTEND project brought up several examples which – at least on the surface – seem to be evidence of one or other of our candidate patterns. We plan to hold more open story telling sessions over the next couple of weeks. Prior to that we will be publishing summaries of our existing candidate patterns so that people can easily review them and let us know if they have had any experiences relevant to any of them.

Pattern elicitation is an ongoing community process – and we expect the Planet patterns to be evolving and developing long after the of the project through other initiatives that are already in progress. But we’d like to make the most of the wealth of experience in Emerge to gather evidence! Watch this space.

CETL ALiC Workshop Jan 27th

Janet and I ran a workshop with the CETL ALiC user group yesterday where we got the participants to review their own approaches to designing student learning experiences and then to reflect on how useful some of the frameworks under consideration were in this context. We gave them copies of both Sally Fincher’s time based decision points framework and also the 6 themes used in the Disciplinary Commons (DC) project (Context, Content, Instructional Design, Delivery, Assessment and Evaluation). This was followed by an activity in which participants were asked to map the patterns stored in the Planet platform against their preferred framework.

One group choose to work with the DC themes modified to include levels concerned with individual, institutional and discipline wide decisions. This was chosen as it was simple and easy to work with. They also added an extra theme for ‘Students’ on the basis that students were a key element in the learning design process however when they mapped the patterns onto this framework none of the patterns were allocated to this theme! (perhaps because the students were an implicit aspect of everything). This group were able to review all the patterns in Planet and were able to map most of them – there were a number that were impossible to map as they were either empty or incomprehensible as patterns.

The second group combined both the decision based framework with the DC themes producing quite a complex structure that required them to give some deep thought to the mapping of patterns. As a result they were able to cover a subset of the available patterns but their mapping showed an interesting spread of patterns across time, numbers of students and level of decisions. They also found that some patterns could not be mapped again either because they were only a title or were difficult to understand.

It was clear from the discussions and comments of  participants that the different levels of development of the patterns made the activity very difficult and they did observe that a number of things presented as patterns were not actually patterns but examples of something that had worked in one context only (one ‘pattern’ was identified as a problem without a solution!).

We have videos of the groups working and will seek to provide some clips of the mapping outcomes. What is evident is that we do need to revisit the patterns we actually have with a view to identifying those that are clearly useful to us and moderating them to bring them up to pattern standard. As part of this process we need to introduce a separation between those items that are truly patterns and those that are ‘candidate patterns’ in order to help staff who may wish to use our patterns.

At the end of the workshop we spent a little time with the participants in order to get them to complete the workshops evaluation questionnaire as offered by Isobel. We will bring these to the meeting at York by which time I should have looked at them at identified useful outcomes.

We plan to run a further workshop with this group in March where they bring a specific issue or problem from their teaching experiences and support them in using a framework that helps them identify useful patterns for solving that problem.

Planet 2-day meeting

My head is buzzing following our very productive two day meeting in London which finished yesterday. The first day was a meeting of the Planet team; on the second we were joined by distinguished guests with expertise in various areas of patterns and representations of practice. These included: Lorna Burns from Barnet College; Mark Childs from Coventry; Juliette Culver from the OU; Sally Fincher from University of Kent; Christian Kohls from the Knowledge Media Research Center in Tübingen, Germany; Diana Laurillard from London Knowledge Lab; Helen Sharp from the OU; and Niall Winters from London Knowledge Lab. Jill Jameson also joined us for the afternoon on the second day in her role as critical friend to the project. In between the two working days, the team and guests met for dinner at the wonderful Ottolenghi restaurant in Islington – well worth a visit! But back to the main business. 

Frankly it is difficult to know where to start. On day one we thrashed through some major issues to do with the process of eliciting patterns, the scaffolding we offer through our wiki, and the need for (and current lack of) an organising structure for the patterns that are emerging from our workshop activities. On the second day we had invited our guests to submit stories about their own successful teaching practice which we then used in the morning to give them a taste of our workshop approach to pattern elicitation. In the afternoon we invited them to feedback on this which led to a valuable discussion of the strengths and weaknesses in our approach and alternative approaches which really helped us to pin down the aspects we need to focus on in the remaining months of the project.

Each of these needs further consideration (and warrants its own blog post) but to summarize:

• We are proposing a three workshop model, with active facilitation from a pattern-knowledgeable moderator pre and post each activity. Much of this is in place but needs closer specification so that what is currently “craft” knowledge is made explicit, the activities required of participants are more clearly defined and the case and pattern structures currently on the Wiki reflect what we are seeking in these two forms.
• We need to agree what and how we are abstracting from case stories to make patterns: what are the salient questions to ask? And what order is it appropriate to ask them?
• We urgently need an organising structure to help us make sense of the patterns that are already emerging, to identify gaps where new patterns are needed, and to scaffold the use of patterns in practice. We have some candidates and we need to start working with them: how do our existing patterns map onto these? where are the gaps? what sense do they make to users? The latter is key: whatever structure we choose must reflect the way teaching practitioners work and think about their practice or the patterns will not be used.
• We currently have upwards of a dozen user groups, with whom we are working and talking. All are at different stages in the process, but it is important that one or two at least complete and evaluate the whole three workshop cycle. CETL ALiC and the e-formative assessment groups are furthest along this path so we need to make sure their forthcoming workshops reflect the process as it is developing.

There is much more to say and other team members will give their own reflections on the event. But for me this has been a significant activity and one which has really enabled us to examine what we are doing. There is a lot still to do but we are definitely making progress! The challenge now is to keep focused on these critical elements of work.

e-learning patterns: workshop at KMRC, Tuebingen, March 4-6

Once you start its hard to stop, but really I should. After all, this is the Planet project site – not the pattern herald. Still, this workshop is organized by Christian, so it has to be worth the trip. Plus, you can catch a gig at Tuebingen’s epplehaus.

Design patterns capture proven solutions for recurrent problems. The goal is to externalize the implicit knowledge of an expert, using a highly structured description format for documentation. Patterns have been around for decades, they are a success story in the field of software design, and recently have become very popular to describe recurrent teaching scenarios, instructional methods and tools.

This workshop aims to 

Capture and document the state-of-the art – Establish quality standards – Dissemination – Ties to related fields – Co-operation

For more information about the workshop, please contact Christian Kohls.
Mail: c.kohls(at)iwm-kmrc.de

HCI Disciplinary Commons patterns workshop – September 1st 2008

Throughout the academic year 2007-08, a group of Human-Computer Interaction (HCI) Educators have been engaging in an HCI Disciplinary Commons, meeting on a monthly basis to discuss their teaching practice, undertake peer observation and engage in peer review. Each participant aimed to produce a portfolio representing their HCI teaching during this period. These portfolios form a body of knowledge about HCI teaching in the UK that may be of use to other practitioners.

Planet is joining with the Disciplinary Commons project to run a workshop at the HCI2008 conference in Liverpool in September. This workshop has three aims:
• To inform the wider community about the process, activity and outcomes of the Commons;
• To give participants a taste of how the Commons operated;
• To consider how we might progress to develop meaningful representations for HCI Educators to share practice, using the patterns approach.

The workshop will be in two parts. In the morning, the facilitators of and the participants in the Commons will give a series of presentations on the priniciples, process and experience of the Commons approach to sharing practice. In the afternoon, workshop participants will take part in activities focused on sharing and representing practice. These activities will be anchored in case studies of HCI teaching practice, submitted prior to the workshop.

The workshop will be of interest to anyone teaching HCI who wishes to share practice with others and anyone interested in methods for sharing practice in general. For more information on the workshop and how to participate please see the HCI Disciplinary Commons Workshop on Planet Wiki.

This workshop will bring together practitioners from the same discipline, rather than those with an interest in the same technologies. It will be interesting to see whether the patterns captured will be more generally applicable and to consider how closely they relate to those captured in the Web 2.0 for learning focused workshops. The workshop will also allow us to evaluate the Planet methodology in another context, with an established user group. It promises to be an exciting session.

Both Planet and the HCI Disciplinary Commons projects are feeding into a major new three year project, funded under the National Teaching Fellowship Project scheme: To see ourselves as others see us: sharing and representing disciplinary classroom practice (we’re working on a snappier name!). Led by Sally Fincher, of the University of Kent, and involving Planet members Janet Finlay and Isobel Falconer, this project will explore teaching practice in detail, including developing Disciplinary Commons activities across different disciplines; evaluating different representations of practice (including patterns); and examining in depth how and why practitioners change their practice.

Sharing practice figured large in the JISC Innovation Forum earlier this week – it was recognised by several speakers that we need to learn to do this better or we will waste both effort and resource. Planet and its new partner project, are helping to address this issue.

PatternSeer and the Planet platform: this looks like the beginning of a wonderful friendship

I met Ademar Aguiar at EuroPLoP last week (note to self: need to report on the conference, it was a great event). Ademar is a long-standing member of the pattern community and something of a WikiGuru, he’s one of the organizers of WikiSym, but I’m digressing.

Ademar and his students are working on PatternSeer.org, which is a web2.0-esk clearing house for all things pattern. PatternSeer allows you to submit design patterns and pattern related papers, rate them, discuss them and share them. Needless to say, it allows you to search across sites.

This covers just about everything that the Planet platform doesn’t do. We provide a structured participatory methodology for developing patterns and pattern languages, and the authoring tools to support that. We’re strong on the editing and storage, but pathetic on the social aspects and cross-site search.

This has mashup written all over it. We need a pattern language interchange API, which would allow PatternSeer to crawl the Planet platform and the the Planet platform to piggy-back on PatternSeer for search and social features. And here’s the first shot at a project spec.

Planet makes stuff together

I had the rather rare but very welcome opportunity of acting as non-participant observer in yesterday’s “Making Stuff Together” Planet workshop at LKL. As we had started the day with a Planet project meeting we had a full complement of “staff” for the workshop so some of us stepped back to operate recording equipment and observe proceedings.

The theme for the day was “Making stuff together”, the aim to consider submitted case studies of teaching and learning in collaborative environments and tease out the elements that were successful in order to seed patterns. We had representatives from a couple of other Emerge projects on the day (APSTAIRS and MOOSE) as well as colleagues from elsewhere. It was a mixed group – some with particular interest in patterns and pattern capture, some with little knowledge of patterns but an interest in making things in MUVEs, others with interests in creative online collaboration tools. The potential for not finding commonality seemed high!

However we should not have worried. From the explanation of the first case study (of using Flashmeeting to support collaborative design activity) the group were engaged and animated, picking up connections relating to collaboration across a range of applications from World of Warcraft to Google! By lunchtime we had half a dozen potential “seed patterns” which were then taken up enthusiastically by the group in the afternoon and developed further. There is still work to do on these but the outputs of these labours can be found on the Planet wiki pattern page and feedback and comments are welcome from all (check the Created on 7th July ones – though comments are welcome on them all!).

As an observer it was also very interesting to see how the process worked and I can see some potential patterns emerging here as well. Group formation was certainly an issue both here and in our online workshop at the June Emerge event. Facilitation needs to carefully balance team and participant engagement and it is important to establish common ground in advance. Interestingly all of these were also highlighted in the discussions of making stuff together suggesting much similarity whether we are dealing with Second Life, face to face workshops, Elluminate or Google Docs. It is the human activity of collaboration which is critical, not the technology.

Thanks to all the participants for a really stimulating day – and to Yishay, Steve and Jim for facilitating it. Our next workshop is scheduled for July 21st in Leeds and is focused on one of our user groups, the multi-institutional CETL on Active Learning in Computing. We will be looking at experiences in assessment, project work, learning spaces and Web 2.0. Although primarily for this user group, other participants are welcome. Full details will be posted here as soon as possible but in the meantime if you are interested in coming along just let me know – j.finlay@leedsmet.ac.uk

Emerge Conference Event: Pattern Workshop on Developing and sustaining learning communities, Jun 23rd

Planet is running an online workshop at the Emerge Online Event on 23rd June, around the theme of “Developing and sustaining learning communities“.

Education in the 21st century is increasingly based on different models to those of the past – emphasising peer-to-peer interaction, continuing and lifelong learning, and ubiquitous access. Learning communities are an essential part of making these new approaches work. But what makes learning communities successful, and what are the key methods and processes that need to be used to create and maintain them? The aim of the workshop is to look at examples of different learning communities, and attempt to identify the common features of successful practice, and how this relates to the context in which the learning takes place.

User contributed case studies are the starting point for the workshop. Each of these will be discussed in different subgroups to refine the cases and determine the key elements that led to the outcomes observed. Following this, the group as a whole will discuss the cases and begin the process of identifying the underlying design patterns. These nascent patterns will be then be refined and elaborated in other subgroups, with the aim by the end of session to have a set of candidate patterns, as well as an idea of the type of additional work and evidence needed to develop these further.

You are invited to submit case studies for the workshop. There is a form for this on the Planet wiki – register there at http://patternlanguagenetwork.myxwiki.org/xwiki/bin/view/Main/WebHome. Case studies should be short and focused on a particular incident or experience rather than describe a whole project. Please take a few minutes to think about your experiences of learning communities and whether you have a case study that you can share with us!

Don’t forget you also need to register for the Emerge Event to take part in the workshop.
For any enquiries, please contact: Jim Hensman – j.hensman@coventry.ac.uk or Janet Finlay – j.finlay@leedsmet.ac.uk