Evaluation from 2000/01

 Evaluation has been sought from both students and tutors. The major results have been discussed at staff/student liaison meetings. For numerical data from students' questionnaires, go to the eemec pages. Numerical data from tutors' questionnaires is below. The following comments come from analysis of the numerical data, and of the written comments provided by students, tutors, examiners and other assessors. Similar comments from 1999/2000 are below.
 
General
The course generally ran well in the first block of its second year.  The format seems to be established successfully despite the novelty of some aspects of the tasks and organisation.  Overall ratings concerning the design and content of the course were high.

Large groups
51 students were in large groups.  Critical Care Physiology and Radiology were popular beforehand and received excellent ratings afterwards.  Medical Microbiology however was unpopular beforehand (few students chose it; presumably through word of mouth reports), and received a number of adverse comments and the poorest scores in the evaluation questionnaire ­ 63% wished they had been allocated to a different group.  The course has not changed substantially recently and it may be suffering by comparison with the other subjects and approaches now on offer.  Its unpopularity tends to mean that a proportion of its places are allocated to low-motivation students who haven't bothered to submit a list of preferences, but in truth, without them it would scarcely have been viable.

Need to address how to improve the attractiveness of studying microbiology in 2yO for next year. 

 
Small groups
158 students were divided between 28 small groups.  Most of these ran very well.  One group was clearly dysfunctional.  In general the format was rated highly by students (note responses to 2,5,13,16,17,23,24) and probably even more so by tutors (see below).  Peer marking was used slightly more discriminatingly than last time, perhaps because of the introduction of an optional bonus mark.  40% (25% last year) of tutors would have liked the opportunity to give students individual marks (at present they only approve students' assessments).  Five students received worryingly low individual marks but none of these led to failure. 

It is proposed that next year tutors should be able to give students an individual mark in the same way as the students do (and as an equal).  This is probably fair and would not add administrative work.  They would be encouraged to use the full range of the scale, and are probably more likely to do so than students, which would increase the influence of individual marks on the final grade for some students at either extreme.  

Website composition
This caused less voiced anxiety than before and there has been a substantial shift in expectations and attitude: instead of 'why' (asked by a small but vociferous few last year), we are being asked whether tuition and resources to achieve better results could be provided.  The new self-learning package was an improvement, and tutorials seemed to go well, although for those late in the alphabet these came late (week 3).  55% of students rated help with web authoring poorly but our question did not discriminate between wanting more help to do the same, and help to do more.  There was an organisational anomaly in that the deadline for personal pages was after the deadline for group websites, to which they were supposed to be linked, and which were supposed to have benefitted from their experience in making personal pages.  Difficulties in uploading pages in order to submit the projects were the commonest complaint. 

The deadline for personal pages will be moved to before Christmas next year if possible, so that all students have been forced to confront any difficulties before contributing to group websites. 

LTS propose to introduce a better system for uploading files next year. 

LTS will investigate timetabling of tutorials earlier in the course for next year.  The feasibility of offering some optional 'advanced classes' in some aspects of authoring will also be considered. 

Library and computer facilities.
25% of small group students (the ones using the library the most) found library facilites to be inadequate.  This is worrying, and relevant to the current discussion on the feasibility of merging the Erskine and Main libraries in the future.  The 'learning style' of SSMs is projected to become more common with time, which may further increase pressure on library facilities.  In contrast, numerical and written comments about computing facilities improved this year, with the increased number of PCs and earmarking of the Greenfield Suite for medics during the day. 

Pass on this information. 

Course timetabling
The proximity of the two blocks of the course is presenting difficulties.  Straddling vacations was predicted to be unsatisfactory to some extent, but we did not forsee how the assessment of the first block could seriously overlap with the organisation of the next.  The students may also benefit more educationally if they had longer to reflect on the first block before planning the second. 

Pass on this information.

Tutors' evaluation, 1st block 2yO 2000/01
25 responses were received from 28 tutors (89%). Answers to questions about their time commitment to the course showed a median (also the mean) of 11 hours (cf. 10h in 99/00), including 2 hours for assessment. The range was 6-23h.  Replies are percentages; >70% and shifts are highlighted. 

Evaluation from 1999/2000

• Most people enjoyed the course and felt that the aims of the course were achieved. Some were very positively enthusiastic. 
• In general the new small group format worked well, this impressed tutors perhaps even more than students. Only one or two groups malfunctioned. The differences between groups bothered some people - in workload, in subject area, in ethos.
• There was considerable concern about the difficulty of website composition, the resources available for instruction in it, and one or two doubts about how relevant it was for a future doctor.

Comment: Most external observers were impressed, and believed that the experience provided was relevant and worthwhile. The quality of presentation of the websites finally produced was above the average internet site reached by casual browsing, and far exceeded the average in quality of information and discussion. Many of the concerns about resources for learning web composition skills have been taken up though, and should be addressed next year.

• From written comments, it is seems that computer facilities were found to be adequate most of the time, but came under great pressure close to the submission deadlines. 

Comment: More space is being created and this question will be monitored closely each year. 

• Some concern was expressed that criticism of scientific papers was a bit of a shock.

Comment: SSMs are perhaps about delivering that kind of shock, but it may be possible to provide more support and guidance next year, even very informally through suggestions to tutors.

• People were happy with the assessment process in general, although some were concerned (at least one or two were very concerned) at the difficulty of making comparisons between groups that are so diverse. One comment questioned the validity of using marks for non-medical subjects to go towards a medical degree.

Comment: This is a question that the course organisers and examiners have discussed often. In retrospect we don't think we've made too bad a job of it - the only way to make a system 100% objectively fair would be to give everyone the same task - but then it wouldn't be 'options'. In a course of this type some comparisons are going to be relatively subjective - but we are taking multiple views in each case, and bearing in mind all information that comes to us from tutors. On the subject of non-medical topics' marks contributing to the marks for Medicine, that's not difficult to justify. The problems being set are generic - find out about a subject and report about it - and the subject matter is almost immaterial. The GMC raised the possibility of using non-medical topics in their original document 'Tomorrow's Doctors'.   

• Peer assessment raised some hackles but seemed to work just fine and didn't upset too many people by the end of the second block.

Comment: Many groups awarded each other all 3.0, which is OK by us. 6-8 students were marked down quite significantly by their colleagues, and tutors and examiners have taken this seriously.

Evaluation as a whole: In the first block there was a problem with the mechanism of returning feedback forms. They were often handed to tutors instead of going unseen into envelopes. Some groups never returned forms. In the second block we gave out forms at a lecture early the week following the course. Replies were lower as a proportion, but gave better representation across all groups. Because of the circumstances, fewer handwritten comments were added, which is a shame, because these offer valuable explanations and expansion to the numerical responses. 

Tutors' evaluation, 1999/2000

Numerical feedback is restricted to Small Group tutors. Responses were received from 79% and from 75% of tutors in the first and second blocks of 1999/2000 respectively. The numbers given are percentages. Dominant answers are highlighted in red.

Tutors in both blocks estimated that they each spent a mean of approximately 14 hours on the course, which was also the median (range, 5 to 27 hours)