Assessing 4th Year Theses

Background

The current system of awarding a single final mark for Thesis B, based on five criteria, which may or may not be applied, is tenuous on several grounds:

the granularity is too fine; is a thesis that scores 73 "better" than one that scores 69?
marks are influenced by the expectations of markers, and are not necessarily objective
the criteria were developed when CSE was part of EET and don't seem totally appropriate to modern CSE theses

We propose a simplified system, based on grading with respect to a new set of criteria, to try to improve the reliability of assessment. Markers award grades, not marks, for each criterion. The final mark is computed by the system by mapping each grade to a mark and comouting a weighted-sum of the individual criterion marks

Criteria

Evaluation of own work

Grades

Overall Presentation	quality of written english structure of thesis (chapters/sections) logical flow of arguments effective citation and referencing
Background/Analysis	comprehensive description of problem space reference to and analysis of other work
Original Contributions	originality of approach to the problem quality of the final results or system
used appropriate analystical instruments carried out analysis effectively analysed results appropriately realistic appraisal of achievements/limitations

Grade-to-Mark Mapping

A+	absolutely top-quality work, best I've seen publishable in good conference with little change
A	excellent work, does everything required results are good, could be published with some re-working
B	good quality work, but with some deficiencies would need substantially more work to be publishable
C	adequate the topic could have been treated much better
D	just satisfactory, minimal standard for a CSE thesis
E	not up to standard required of CSE thesis

Criteria Weightings

The weights of the individual criteria towards the final mark could be determined in several ways:

fixed for all theses: 1 = 20%, 2 = 20%, 3 = 30%, 4 = 30%
determined up-front as a "contract" between student and supervisor
different for the different thesis types, e.g.

A+	95% of the mark for that component
A	88% of the mark for that component
B	77% of the mark for that compnent
C	66% of the mark for that component
D	55% of the mark for that component
E	40% of the mark for that component

R	1 = 20%, 2 = 20%, 3 = 35%, 4 = 25%
R+D	1 = 20%, 2 = 20%, 3 = 40%, 4 = 20%
D	1 = 20%, 2 = 15%, 3 = 45%, 4 = 20%

John Shepherd, March 2008