Screen Version

On-Line Survey 2014

Survey ID 1389 Title COMP9242 14 Description Course Evaluation Survey for COMP9242 Advanced Operating Systems. Version for Session 2, 2014. Anonymous Yes Fill Ratio 56% (9/16) # Filled 9 # Suspended 1 # Not Filled 6

indicates required field

Your comments will help us to assess and improve our courses, not only for future generations, but for your further study in CS&E. We really look at the results and appreciate your feedback! Several changes to the course over the years were a direct result of student feedback. And, as always, we'll publish the uncensored results on the course web site. Note: Please do not enter "no comment" or something similar into comment boxes. If you don't have anything to say, just leave the box empty.

1. Quick Evaluation

1. Give a high rating if you have a good opinion of something (e.g. interesting, useful, well-structured, etc.). Give a low rating if you have a bad opinion of something (e.g. too slow, confusing, disorganised, etc.)   Question type : Single answer -- Radio Button   Excellent Satisfactory Poor Gernot Heiser 9 (100%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Kevin Elphinstone 9 (100%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Guest lecturer Ihor Kuz 8 (89%) 0 (0%) 1 (11%) 0 (0%) 0 (0%) Guest lecturer Toby Murray 7 (78%) 1 (11%) 1 (11%) 0 (0%) 0 (0%) Guest lecturer Peter Chubb 8 (89%) 0 (0%) 1 (11%) 0 (0%) 0 (0%) Tutors/demonstrators 4 (44%) 5 (56%) 0 (0%) 0 (0%) 0 (0%) Exam 5 (56%) 4 (44%) 0 (0%) 0 (0%) 0 (0%) Course web pages 4 (44%) 5 (56%) 0 (0%) 0 (0%) 0 (0%) Reference material 3 (33%) 4 (44%) 2 (22%) 0 (0%) 0 (0%) Computing resources 4 (44%) 4 (44%) 1 (11%) 0 (0%) 0 (0%) COMP9242 overall 8 (89%) 1 (11%) 0 (0%) 0 (0%) 0 (0%)

2. General

2. Which factors most influenced your decision to enrol in this course?   Question type : Multiple answer -- Check Box Interest in operating systems as an area of study 7 (78%) Chance to build a system 8 (89%) Chance to get fingers really dirty 8 (89%) Would like to do some systems research 3 (33%) Looking for a challenge 8 (89%) Looking for an easy course 1 (11%) Rupert told me to 0 (0%) Friends told me it was good 7 (78%)

3. Other factors not mentioned above? Question type : Short-answer Answer at the bottom page (1 comments)

4. Would you recommend this course to another student such as yourself?   Question type : Single answer -- Radio Button Yes 9 (100%) No 0 (0%)

5. The course is heavy on design and implementation issues. It also tries to remain close to present research issues (although that aspect has suffered with the move to 12 teaching weeks). What do you think about the content allocation?   Question type : Single answer -- Radio Button   Too much Just right Too little Theory/general principles 0 (0%) 2 (22%) 7 (78%) 0 (0%) 0 (0%) OS design and implementation 0 (0%) 1 (11%) 6 (67%) 2 (22%) 0 (0%) Current research issues 0 (0%) 0 (0%) 7 (78%) 2 (22%) 0 (0%)

6. What were the best things about this course? Question type : Long-answer Answer at the bottom page (8 comments)

7. What were the worst things about this course? Question type : Long-answer Answer at the bottom page (6 comments)

8. How does the workload in this course compare to workloads in other ...   Question type : Single answer -- Radio Button   Much Lighter Similar Much Heavier COMP courses at this level 0 (0%) 0 (0%) 0 (0%) 6 (67%) 3 (33%) COMP courses in general 0 (0%) 0 (0%) 0 (0%) 1 (11%) 8 (89%) Courses in general 0 (0%) 0 (0%) 0 (0%) 1 (11%) 8 (89%)

9. How does the overall quality/value of this course compare to other ...   Question type : Single answer -- Radio Button   Among the best Average Among the worst COMP courses at this level 9 (100%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) COMP courses in general 9 (100%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) courses in general 9 (100%) 0 (0%) 0 (0%) 0 (0%) 0 (0%)

10. What background knowledge do you think you were missing that would have helped you in this course? Is distinction in COMP3231/9201 a suitable preparation? Is it too harsh? Question type : Short-answer Answer at the bottom page (5 comments)

3. Content/Syllabus

11. Please rate the relevance/appropriateness of the lecture topics.   Question type : Single answer -- Radio Button   Very relevant Average Inappropriate N/A Microkernels and seL4 6 (67%) 2 (22%) 1 (11%) 0 (0%) 0 (0%) 0 (0%) Caches 6 (67%) 3 (33%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Threads vs Events 7 (78%) 2 (22%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Performance Evaluation 9 (100%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) SMP and Locking 4 (44%) 4 (44%) 1 (11%) 0 (0%) 0 (0%) 0 (0%) Virtual Machines 4 (44%) 4 (44%) 1 (11%) 0 (0%) 0 (0%) 0 (0%) Microkernel Design 4 (44%) 5 (56%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Linux 7 (78%) 1 (11%) 1 (11%) 0 (0%) 0 (0%) 0 (0%) Real-Time Systems 4 (44%) 4 (44%) 0 (0%) 1 (11%) 0 (0%) 0 (0%) Multiprocessors 2, Drawbridge 4 (44%) 4 (44%) 1 (11%) 0 (0%) 0 (0%) 0 (0%) Security 6 (67%) 2 (22%) 1 (11%) 0 (0%) 0 (0%) 0 (0%) Local Systems Research 4 (44%) 4 (44%) 0 (0%) 0 (0%) 0 (0%) 1 (11%) Sample paper analysis 5 (56%) 4 (44%) 0 (0%) 0 (0%) 0 (0%) 0 (0%)

12. Please tell us how interesting you found the lecture topics.   Question type : Single answer -- Radio Button   Very interesting Ok Boooooring! Skipped Microkernels and seL4 5 (56%) 3 (33%) 1 (11%) 0 (0%) 0 (0%) 0 (0%) Caches 6 (67%) 3 (33%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Threads and Events 5 (56%) 3 (33%) 1 (11%) 0 (0%) 0 (0%) 0 (0%) Performance Evaluation 6 (67%) 2 (22%) 1 (11%) 0 (0%) 0 (0%) 0 (0%) SMP and Locking 1 (11%) 7 (78%) 1 (11%) 0 (0%) 0 (0%) 0 (0%) Virtual Machines 7 (78%) 2 (22%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Microkernel Design 5 (56%) 3 (33%) 1 (11%) 0 (0%) 0 (0%) 0 (0%) Linux 6 (67%) 1 (11%) 2 (22%) 0 (0%) 0 (0%) 0 (0%) Real-Time Systems 6 (67%) 2 (22%) 0 (0%) 1 (11%) 0 (0%) 0 (0%) Multiprocessors 2, Drawbridge 4 (44%) 3 (33%) 2 (22%) 0 (0%) 0 (0%) 0 (0%) Security 6 (67%) 1 (11%) 2 (22%) 0 (0%) 0 (0%) 0 (0%) Local Systems Research 2 (22%) 5 (56%) 0 (0%) 0 (0%) 0 (0%) 2 (22%) Sample paper analysis 2 (22%) 4 (44%) 2 (22%) 1 (11%) 0 (0%) 0 (0%)

13. Which material do you think will be most useful to you in the future?   Question type : Long-answer Answer at the bottom page (9 comments)

14. Which material, not currently in this course, would you liked to have seen covered? Question type : Long-answer Answer at the bottom page (3 comments)

15. Which of the current topics would you like to see scaled back or excluded? Question type : Long-answer Answer at the bottom page (3 comments)

4. Lectures

16. What factors caused you to attend lectures?   Question type : Multiple answer -- Check Box I had enough spare time 5 (56%) The lectures were too good to miss 6 (67%) Given the pace and lack of a textbook, I could not afford to miss the lectures 2 (22%) It was as good a place as any to take a nap 1 (11%) I wanted to be seen to be there 2 (22%) None, I skipped most 0 (0%)

17. What were the reasons for skipping lectures?   Question type : Multiple answer -- Check Box Overall workload in this and other courses 1 (11%) Lecture notes and references cover the material adequately 0 (0%) Lectures are boring 0 (0%) There was not enough material to justify attending lectures 0 (0%) First half of the course was more interesting than second half 0 (0%) None, I attended (almost) all 8 (89%)

18. Any suggestions for improving lectures? Question type : Long-answer Answer at the bottom page (4 comments)

5. Project

19. What was the level of difficulty various parts of the project?   Question type : Single answer -- Radio Button   Too easy Just right Too hard Milestone 0 2 (22%) 1 (11%) 6 (67%) 0 (0%) 0 (0%) Milestone 1 0 (0%) 1 (11%) 5 (56%) 2 (22%) 1 (11%) Milestone 2 0 (0%) 1 (11%) 7 (78%) 1 (11%) 0 (0%) Milestone 3 0 (0%) 1 (11%) 6 (67%) 2 (22%) 0 (0%) Milestone 4 1 (11%) 0 (0%) 7 (78%) 1 (11%) 0 (0%) Milestone 5 0 (0%) 0 (0%) 6 (67%) 3 (33%) 0 (0%) Milestone 6 0 (0%) 0 (0%) 4 (44%) 4 (44%) 1 (11%) Milestone 7 0 (0%) 0 (0%) 5 (56%) 2 (22%) 2 (22%) Milestone 8 0 (0%) 3 (33%) 6 (67%) 0 (0%) 0 (0%) System documentation 0 (0%) 1 (11%) 6 (67%) 2 (22%) 0 (0%) Project overall 0 (0%) 0 (0%) 6 (67%) 3 (33%) 0 (0%)

20. How well was the project specified?   Question type : Single answer -- Radio Button   Very clear Ok Confusing Milestone 0 5 (56%) 3 (33%) 1 (11%) 0 (0%) 0 (0%) Milestone 1 5 (56%) 2 (22%) 2 (22%) 0 (0%) 0 (0%) Milestone 2 5 (56%) 2 (22%) 2 (22%) 0 (0%) 0 (0%) Milestone 3 5 (56%) 1 (11%) 2 (22%) 1 (11%) 0 (0%) Milestone 4 5 (56%) 2 (22%) 2 (22%) 0 (0%) 0 (0%) Milestone 5 4 (44%) 1 (11%) 3 (33%) 0 (0%) 1 (11%) Milestone 6 3 (33%) 4 (44%) 2 (22%) 0 (0%) 0 (0%) Milestone 7 4 (44%) 1 (11%) 2 (22%) 2 (22%) 0 (0%) Milestone 8 4 (44%) 4 (44%) 1 (11%) 0 (0%) 0 (0%) System documentation 3 (33%) 3 (33%) 3 (33%) 0 (0%) 0 (0%) Project overall 3 (33%) 3 (33%) 3 (33%) 0 (0%) 0 (0%)

21. What was the quality of...   Question type : Single answer -- Radio Button   Excellent Ok Poor Documentation/reference material 2 (22%) 3 (33%) 2 (22%) 2 (22%) 0 (0%) Supplied code 2 (22%) 3 (33%) 4 (44%) 0 (0%) 0 (0%) Hardware platform 4 (44%) 4 (44%) 1 (11%) 0 (0%) 0 (0%) Consultation time help/support 8 (89%) 0 (0%) 0 (0%) 1 (11%) 0 (0%) On-line help/support 3 (33%) 3 (33%) 2 (22%) 1 (11%) 0 (0%)

6. Project mechanics

This year we introduced a number of changes in the way the project was done and administered. We're interested in feedback on those.

22. Do you have comments/opinions on the demo submission and web-based demonstration process? Question type : Long-answer Answer at the bottom page (6 comments)

23. Do you have comments on the compulsory use of git for revision control? Question type : Long-answer Answer at the bottom page (6 comments)

24. We released the project as a git "repo", with the ability to pull patches (and push pull requests for your fixes) while the project was underway. Do you have comments on this? Question type : Long-answer Answer at the bottom page (5 comments)

7. Summer Internships

25. Each year we encourage the students in this course to do a summer internship with us (i.e. the Trustworthy Systems group at NICTA). Typically less than half take up the opportunity, and we'd like to understand why. Question type : Single answer -- Radio Button I am doing a ToR/NICTA Summer Internship with Trustworthy Systems 1 (11%) I am doing a ToR/NICTA Summer Internship with another group 2 (22%) I received an offer but declined 1 (11%) I applied but did not get an offer 0 (0%) I did not apply 5 (56%)

26. If you are not doing a ToR/Summer Internship with us, please let us know why. Question type : Long-answer Answer at the bottom page (7 comments)

8. Anything Else

27. Any other comments/suggestions that might help us to improve the course in the future? Question type : Long-answer Answer at the bottom page (6 comments)

3. Other factors not mentioned above? 1: The challenge of AOS and maintaining a healthy relationship with girlfriend at the same time 6. What were the best things about this course? 1: - the challenge - the people - the tshirt - the bonding...with my aos partner 2: It's very challenging and the topics are very relevant. 3: Build an operating system - this is pretty neat. Freedom of implementation/no handholding. Interesting, fast-paced. Most demonstrators were very good. 4: Excellent assignment tasks spaced out well over the semester. 5: The tutors are very supportive 6: Practical project, good lecturers, mostly helpful tutors. 7: Revision of OS concepts. Improved my C and lower level coding skills. Challenging. Got to build a huge system instead of tiny bits and pieces. Approaching more complex systems, and research papers, are much less daunting now. 8: Project In person discussions of various implementation issues with two of the tutors 7. What were the worst things about this course? 1: The workload can be overwhelming and stressful, especially when working alone. The deadlines are a tad unreasonable too: how is it that M0 and M6/M7 are given exactly the same time to complete? Once you're behind with one milestone, the mark penalties will continue to haunt you throughout the project. 2: Consults were inefficient; a lot of wasted time; demonstrators were not all on the same page. Fucking blue ethernet thing. 3: - the exam is too hard - said girlfriend did not enjoy those late nights without me 4: One particular tutor (not naming names) always took forever to mark stuff. There was one time where they asked us to demo a specific thing, which we demo-ed, and then said that he really wanted a different demo, and wouldn't mark us off because he had a meeting now. We weren't the only group with issues like this. 5: Intense workload for 13 weeks. Impossible to catch up unless you have lots of spare time, resulting in snowballing of lost marks. If you made mistakes prior to m7, you ended up with an endless stream of bugs to fix. Compound this with being behind and it made it impossible to submit a final system. For students reading this in the future: lateness deductions come off your gained and NOT maximum mark! 6: Differences between tutors could be annoying at times. Some were insanely awesome while others were avoided by most students. 10. What background knowledge do you think you were missing that would have helped you in this course? Is distinction in COMP3231/9201 a suitable preparation? Is it too harsh? 1: Prereq is fine. 2: Seems fine 3: Distinction in OS/EOS should be the minimum for entry. 4: I suppose along with the basic OS course, a background in computer architectures would be helpful. 5: D in OS is good. MAYBE COMP4920...jokes not really 13. Which material do you think will be most useful to you in the future?   1: Linux internals from Peter was engaging and very relevant. Reading papers was useful. 2: Threads and Events, Performance Evaluation, Microkernel Design 3: Low-level OS knowledge and ability to build complete systems with experience in making valid tradeoffs during design/testing. 4: Caching, multiprocessors 5: Caches, concurrency models, virtual machines, and the linux section. 6: Hard to say, I would guess caches, locking, threads and events, virtual machines, benchmarking, possibly even microkernels 7: All of it. 8: Caches, virtual machines, performance evalutation 9: The lecture slides. 14. Which material, not currently in this course, would you liked to have seen covered? 1: More on design considerations of systems that aren't microkernels 2: Nil. Though I would love to see some practical real-time system implementation tasks (since no other UNSW comp course offers this atm). 3: Windows. 15. Which of the current topics would you like to see scaled back or excluded? 1: Virutal Machines should be scaled back - it seemed like it covered a lot of what was in EOS last semester. 2: Nil. 3: (maybe) very heavy (but still useful) on microkernels 18. Any suggestions for improving lectures? 1: More condensed 2: Nil. 3: Some of the lectures this year were recorded, which I found helpful due to the pace, despite attending all of the lectures. (That said, I know what effect the recorded lectures had on EOS and why you may not want to record everything.) Somewhat related, it would have been good if the slides for a lecture could be up beforehand to skim through - I find things much easier to follow that way (draft slides are fine for this - even indicating which topic it would be so we could skim previous year's slides.) 4: Why does only Professor Elphinstone record his lectures? It'd be so much better if all lectures were recorded, given that the notes are extremely succinct. 22. Do you have comments/opinions on the demo submission and web-based demonstration process? 1: Good! 2: Worked OK. Should not have to submit beforehand if you are ahead of deadline - this wastes time in consults. Patch size should be larger. 3: One particular tutor is annoying to demo to (as outlined previously). What is the ``web-based demonstration process''? 4: There was web-based demonstration? 5: Worked fine 6: Great. Flexibility in when we can demonstrate was incredibly helpful for say, being sick one day, class clashes or oversleeping. 23. Do you have comments on the compulsory use of git for revision control? 1: As someone who's used Git for a very long time it was fantastic news after having to use SVN in 3231/3891, but for those less familiar with it, things can go wrong. I needed to mess around with resetting and reflogs at times 2: It worked. 3: Good idea. 4: Yes, Git is awesome... please continue using it. 5: Love it! 6: I like git - it also frees us up to use Github, which is nice for quickly looking at code, quickly finding specific versions, etc. I'm also more familiar with git than other version control systems. 24. We released the project as a git "repo", with the ability to pull patches (and push pull requests for your fixes) while the project was underway. Do you have comments on this? 1: Good idea. 2: 'twas great! 3: It worked. 4: I liked it. Can't think of an obvious way to improve on the system, and it was also a handy way to learn some features of git I've never used before. 5: Worked well. 26. If you are not doing a ToR/Summer Internship with us, please let us know why. 1: Have an internship elsewhere, but would have definitely applied if I did not 2: I got an internship that looks fun and pays well. I also might be doing honours at NICTA, depending on whether I can be bothered with a fourth year. 3: Other internships 4: I did not feel confident enough to apply for a TOR/Summer internship with SSRG during the time I was taking AOS. 5: I am graduating and have already been offered a full-time position. 6: I already accepted an internship offer for this summer. 7: Better offers, unfortunately. Looking for Special Project though. 27. Any other comments/suggestions that might help us to improve the course in the future? 1: I found it unfair one group was allowed access to the USB keyboard stack code, but we were denied. This was not fair - we did not know it could be received, and when we knew we were shut down. Bonus marks should be there for all. One of the newer demonstrators weren't helpful, in that he took the entire consultation to mark a demonstration, and once asked questions for an hour and DID NOT MARK US. We had to wait until the next lab...where a more experienced one got us marked in less than 5 minutes. Labs should be marked week before more easily. It allows us to, if there is a problem, change stuff ahead of time. Otherwise, what's the point of being ahead of time? David Cock is awesome. 2: This course ought to be worth 2 normal courses. 3: Get Kevin to go out for beers after lectures more :) 4: I was told several times that 'process deletion' was the hard part of m7 - it wasn't obvious that this meant kill until we did that milestone! Project would have been less painful with a very slight improvement in the debugging tools available - e.g. hooking that backtrace function into any assertion fails and a_crash (muslc intentionally calls a_crash in the malloc library...) 5: There shouldn't be a cap on bonus marks; extra bonus marks should be allowed to count for boring courses that you have neglected. 6: Little bits here and there you don't get around to or didn't think of earlier will grow exponentially and blow up in your face towards the end of the project. This needs very heavy emphasis, as it's incredibly disheartening to have a semester's worth of work end up with an incomplete system, while watching others already having long finished.