ELTAC Poster

RARE poster

ELTAC2

More Photos

Feedback :: Report

Report on delegate feedback ‘Institutional Impact’ – JISC Institutional Innovation/ Lifelong Learning & Workforce Engagement Programme Event, 9 July 2009

Of the 62 delegates attending the event (this is excluding the support team and JISC members), 16 submitted responses to the feedback form which is a 25.6% return rate.  (The feedback form can be accessed at:

http://ssbr0709.inin.jisc-ssbr.net/feedback-form/)

This summary report outlines the feedback from the submitted forms including how the platform was experienced, nature of participation, what delegates valued, and what they would like to be different at future events. It concludes with a list of recommendations arising from selected feedback.

Representation of programme phases

Table 1: Feedback forms submitted and attendance totals

Group

Feedback forms

Totals attended

IIN (Phase 2 projects) 10 35
LLLWFD (Phase 3 projects) 05 26
Other 01 01
TOTAL submitted 16 62

All submissions found registration easy with one delegate (perhaps external to the email lists?) receiving information insufficiently in advance. From the registration details it appears that a few delegates registered more than one with one delegate mentioning they did this when they had not received confirmation of registration.

Using the Elluminate audiographic platform

To meet the needs of delegates who might be unfamiliar with the online Elluminate audiographic application platform, a series of familiarisation sessions with the platform were run during the weeks leading up to the event. These were small group, 2-hour sessions of basic training with the platform as well as hands-on practical experience with participating and presenting. Of the 62 delegates who attended the event 34 signed up for a familiarisation session. Taken only from the feedback forms, the figures in Table 2 illustrate the levels of familiarity with Elluminate together with the proportion that signed up for familiarisation sessions. These indicate that not all delegates new to the platform took up the opportunity to get practice in a hands-on interactive session. Conversely there were delegates who did take up the opportunity despite previous experience with the platform.

Table 2 Elluminate platform familiarity and sign ups to training sessions

Level

No.

Novice 11 (6 signed up for training session)
Familiar 05 (3 signed up for training session)
Total 16

Based on their experience of the event, 12 of the 16 who submitted feedback would consider using Elluminate for their future project meetings or assemblies.

Favourable comments about the platform referred to:

  • the system as a good means to get large numbers together online
  • the learning experience
  • the opportunity to work on other things and dip in when interested.
  • the text chat
  • being able to collect questions from the chat for  moderators to return to

Conversely difficulties with the platform included:

  • Sound problems including audio drop-out causing disjointed dialogue
  • Loss of video
  • The need to multi-task with simultaneous audio, visuals and text chat.
  • The ‘cluttered’ interface.

While the above difficulties were not confined to those who had not attended familiarisation sessions the following issues that were dealt with in familiarisation sessions were:

  • Managing the interface
  • Setting profile information in order to provided more information on individuals and project membership.

Tables 3 and 4 illustrate respectively the audio equipment used by delegates during the event and the type of Internet connectivity.  In addition to the audio equipment listed three delegates linked up with a webcam.

A quarter of those who submitted feedback forms made use of external speakers for the sessions. Given that some of the audio problems encountered may have been aggravated by the use of external speakers, this is referred to later in the section on Recommendations.

Table 3 Equipment used during the event

Audio/video equipment

No.

USB headset 5
Minijack headset 6
External speakers/
built in mic
4
Not stated 1
Total 16

Table 4 Internet connectivity

Connectivity

No.

Univ. wired network 9
Univ. WIFI 3
Home WIFI 1
Not stated 3
Total 16

Participation methods and level

Almost all (13/16) who submitted feedback forms participated with text chat during the event, with the remaining three listening and observing only without using text or voice options. Eleven also participated in the voice chat while three of the Phase 3 and one of the Phase 2 delegates who responded also gave presentations during the event. Two of these also connected via webcam.
.

Table 5 Participation methods

Participation methods

No.

Text chat 13
Voice 11
Gave a presentation 4
Listened and watched only 3
Webcam 2

Session participation

Table 6 includes the totals that attended the various types of plenary sessions (excluding support team members) as well as of those who submitted feedback forms.

Table 6 Session participation

Sessions Total  attended No. submitting feedback
Keynote 65 14
SSBR presentations 63 14
Project presentation 63 12
Radio Not available 4

The totals reflecting participation on the day in the break-out sessions is illustrated in Table 7.

Table 7 Attendance at break-out sessions

Session1:

No. attended
LLLWFD – Making connection part ` 24
The physical world, green ICT, estates, mobile and location-aware 12
Social and cultural enabling practices, (e) Learning Teaching & Assessment 9
Technical enabling practices, data management, systems integration, IT architectures 8

Session 2:

No. attended
LLLWFD – Making connection part 2 13
Developing Benefits Realisation Projects 13
Developing Assemblies – Institutional Change Agents 12
Second Life – an Introduction 5
The Institutional Innovation Support Platform – user engagement 3

To approximate the overall level of engagement as perceived by delegates they were asked to individually rate this on a 5 point scale from very weak to very strong.  The level of engagement of those who provided feedback ranged across all the categories with an equal number of delegates rating their level of engagement 2 and below as did those 3 and above (see Figure 1), with median scores of 2.5.

Figure 1: Level of engagement provided by delegates in feedback forms

Most valued at the event

Reported by those who submitted feedback, what they valued most from the event included:

  • Networking with people from similar projects with a view to future collaboration
  • Follow up emails relating to assemblies
  • Interesting and informative talks
  • Webinar like session with Q & A
  • Discussion in breakout groups
  • A separate LLL&WFD (Phase 3) session
  • The internet radio show that gave the sense of being together during breaks
  • The experience of the environment providing pointers (good and bad) for its future use
  • Not having to travel.

In contrast to the value given as ‘taking away many positives’ there was also one response that there was ‘Nothing’ of value, two that suggested a ‘return’ to face to face events, while another elaborated on the event as a ‘..grotesque parody of an academic meeting…’

Requests for future events

In response to what delegates would like to be different in future online events the requests in the feedback included:

  • Clearer event purpose
  • More allowances for the time it takes for group dynamics to process online
  • More imaginative use of the platform than reading off slides
  • Shorter duration than a full day
  • Smaller sub-groups
  • Shorter and fewer plenary events
  • Longer breakout sessions with more information to enable informed choices
  • More breaks with opportunities to network during breaks
  • More opportunity and time to present project work and progress

Recommendations

Organisation

  • Provide confirmation messages in response to registration.
  • Consider timing and duration in terms of the suggestions made
    • shorter plenary session
    • longer breakout sessions
    • longer breaks with networking opportunities
    • smaller groupings (but keeping in mind the need to encourage links across the programme)
    • opportunities for continued presentation by projects of their work and progress to facilitate linkages

Elluminate

As some of the sound problems were caused by delegates using external speakers and others not disengaging sound when they were not speaking, consider providing a set of key recommendations for equipment use with the platform.  These would be useful not only for future events but also for those projects who have expressed interest in using the platform for their project meetings and assemblies. The guidelines could include the practicalities of interface management and profiling and other functions as were outlined during the familiarisation sessions.

There are likely to be delegates, as there were in this event, who consider text chat ‘rude’ during presentations. There may be the need to provide information on the genre of (online) events given the practice of participating in multiple channels that is developing in online and face to face conferences.

Suggestions of how to manage multi-tasking by dividing tasks across team members might also be helpful to those new to online event interfaces. In addition a reminder of the opportunity to revisit recorded sessions after the event could be included.

Compiled by Patsy Clarke
July 2009