Comments on our Bespoke Survey
I'm very pleased to see that all but two of the enrolled students completed the survey (bribing with a t-shirt may have helped ;-) This certainly provides valuable feedback.
General comments
The results mostly speak for themselves. The course seems to be in good shape, overall satisfaction was very high, some critical comments notwithstanding.
We are grateful for all the positive and encouraging comments, I won't discuss them, but be assured they are well received by the team! Let's look at the critical comments that address issues under our control (and changing which would not obviously be at odds with the overall objectives for the course). Also, I'm skipping comments that are already addressed in the myExperience response.
Also, compared to previous years, the survey provider has significantly downgraded the export features, I struggled to get half-decent formatting. They also randomised matrix-style questions (see Q11) which is incredibly stupid. Apologies for that...
Blue font indicates concrete action items.
Q1: Quick evaluation
It's great to see the strong approval scores for all people involved (individual lecturers as well as the class of tutors), with "excellent" clearly dominating. It's pleasing to see that their commitment is appreciated.
Q4: Content Balance
Main observation is that a fair number of people would like to see more current research issues covered (but note that this aspect seems to shift a fair bit between years). I'll see whether I can find something that isn't ML ;-)
Q9: Worst things
Many of these came up in the myExperience survey and are addressed in that context. However, one seems to merit more discussion:
- Lectures not assessed?
- They are, but not in the usual way. If you don't go through the lectures, you'll lack the background for the exam. I strongly suspect that these are the cases where exam performance is way below the project mark.
- Labs scheduled late.
- We have to take demonstrator availability into account, and we also look at time tables of enrolled students to identify good slots. We can only do that during Week 1.
- Have threads-vs-events lecture earlier
- Yeah, we had shift the second part of this into W3 (due to personal reasons), which was sub-optimal. We'll avoid this in the future.
- Would've liked to learn about the parts of the operating system that were done for us
- This is actually covered in the lectures, and you had the source?
- No option to solo the project, even at own risk
- We occasionally allow this, but only where the student can convince us that they have sufficient experience, which is rare. We do our best to prevent students from failing.
- Milestone marking seemed inconsistent between tutors
- We do what we can to ensure consistency, incl defining show-stoppers. Demonstrators are told to reject what will likely cause issues down the road.
- Low on current research, time protection not covered?
- We'll try to get this back in...
- Due dates on Friday are particularly painful as in practice it doesn't give you the weekend to work through things
- We're up against uni policies here. However, late penalties only kicked in after the week-end, so I don't understand the issue here.
- ... demos are not available until after the weekend. If showstopped, in practice it would give you only a few days during the week until you can complete the next milestone without penalty
- There's nothing stopping you demonstrating early, and in fact this is what I explicitly recommended in class!
- 24h exam stressful, 2x8h would be better
- the intention isn't to spend 24h on the exam, but that you can pick your time within the 24h period. This gives you more flexibility than forcing into specific 8h periods.
- seL4 bias
- This is normal for a deep-dive course like this one. The exam papers are chosen to fit the background provided by the lectures. And all the material from previous years is on-line and easy to check.
Q14: What should be added?
- unikernels
- They are really a fancy name for library OSes (see slide 43 of the virtualisation lecture)
- More on other OS designs
- We have the Linux lecture. I don't really feel confident talking about macOS or Windows design, but maybe we can at least talk about it at a high level...
- Time Protection, more security etc.
- Let me see if I can revive that lecture...
- Microkit etc
- I'll try to dig deeper into current TS research.
Q15: What should be excluded?
No clear votes for anything specific, and the things mentioned are all topics that were popular with the majority of people. My take-away is that there's no strong argument for dropping any specific content. The ones that seem least clear-cut are:
- Some of the formal verification, especially the history of OS verification
- I already reduced this, and it's really core to the seL4 story...
Q19: Other comments on lectures
- Questionnaire about lectures?
- Good idea, I'll give it serious consideration
Q20: Difficulty of various parts
This looks reasonably balanced...
Q21: How well specified?
This looks good, looks like improvements made this year helped.
Q21: Quality of...
Again, looks like improvements made this year helped.
Q24: GDB
Looks like GDB support was less mature/usable than we thought. Hopefully better next time!
Q29: Partners
We should warn that good partner interaction is essential, and that people should look for partners before the course starts...
Q31: Thesis interest
This looks like a record high!
Final
Thanks for the feedback, and your participation in the course!
Gernot