Screen Version

On-Line Survey 2008

Survey ID 1267 Title COMP9242 08s2 Description Course Evaluation Survey for COMP9242 Advanced Operating Systems. Version for Session 2, 2008. Anonymous Yes Fill Ratio 86% (12/14) # Filled 12 # Suspended 2 # Not Filled 0

indicates required field

Your comments will help us to assess and improve our courses, not only for future generations, but for your further study in CS&E. We really look at the results and appreciate your feedback! Several changes to the course over the years were a direct result of student feedback. Note: Please do not enter "no comment" or something similar into comment boxes. If you don't have anything to say, just leave the box empty.

1. Quick Evaluation

1. Give a high rating if you have a good opinion of something (e.g. interesting, useful, well-structured, etc.). Give a low rating if you have a bad opinion of something (e.g. too slow, confusing, disorganised, etc.)   Question type : Single answer -- Radio Button   Excellent Satisfactory Poor Gernot Heiser 12 (100%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Kevin Elphinstone 12 (100%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Guest lecturer Stefan Petters 4 (33%) 2 (17%) 5 (42%) 1 (8%) 0 (0%) Guest lecturer Herbert Bos 8 (67%) 2 (17%) 2 (17%) 0 (0%) 0 (0%) Tutors/demonstrators 4 (33%) 5 (42%) 3 (25%) 0 (0%) 0 (0%) Course web pages 1 (8%) 8 (67%) 2 (17%) 1 (8%) 0 (0%) Exam 1 (8%) 5 (42%) 4 (33%) 2 (17%) 0 (0%) Reference material 2 (17%) 1 (8%) 5 (42%) 1 (8%) 3 (25%) Computing resources 5 (42%) 3 (25%) 3 (25%) 1 (8%) 0 (0%) COMP9242 overall 10 (83%) 2 (17%) 0 (0%) 0 (0%) 0 (0%)

2. General

2. Which factors most influenced your decision to enrol in this course?   Question type : Multiple answer -- Check Box Interest in operating systems as an area of study 10 (83%) Chance to build a system 7 (58%) Chance to get fingers really dirty 6 (50%) Would like to do some systems research 7 (58%) Looking for a challenge 11 (92%) Looking for an easy course 0 (0%) Friends told me it was good 5 (42%)

3. Other factors not mentioned above? Question type : Short-answer Answer at the bottom page (0 comments)

4. Would you recommend this course to another student such as yourself?   Question type : Single answer -- Radio Button Yes 12 (100%) No 0 (0%)

5. The course is heavy on design and implementation issues. It also tries to remain close to present research issues. What do you think about this?   Question type : Single answer -- Radio Button   Too much Just right Too little Theory/general principles 0 (0%) 1 (8%) 9 (75%) 2 (17%) 0 (0%) OS design and implementation 0 (0%) 1 (8%) 10 (83%) 1 (8%) 0 (0%) Current research issues 0 (0%) 0 (0%) 8 (67%) 4 (33%) 0 (0%)

6. What were the best things about this course? Question type : Long-answer Answer at the bottom page (12 comments)

7. What were the worst things about this course? Question type : Long-answer Answer at the bottom page (11 comments)

8. How does the workload in this course compare to workloads in other ...   Question type : Single answer -- Radio Button   Much Lighter Similar Much Heavier COMP courses at this level 0 (0%) 0 (0%) 0 (0%) 3 (25%) 9 (75%) COMP courses in general 0 (0%) 0 (0%) 0 (0%) 0 (0%) 12 (100%) Courses in general 0 (0%) 0 (0%) 0 (0%) 0 (0%) 12 (100%)

9. How does the overall quality/value of this course compare to other ...   Question type : Single answer -- Radio Button   Among the best Average Among the worst COMP courses at this level 10 (83%) 1 (8%) 1 (8%) 0 (0%) 0 (0%) COMP courses in general 10 (83%) 2 (17%) 0 (0%) 0 (0%) 0 (0%) courses in general 11 (92%) 1 (8%) 0 (0%) 0 (0%) 0 (0%)

10. What background knowledge do you think you were missing that would have helped you in this course? Is distinction in COMP3231/9201 a suitable preparation? Is it too harsh? Question type : Short-answer Answer at the bottom page (10 comments)

3. Content/Syllabus 11. Please rate the relevance/appropriateness of the lecture topics.   Question type : Single answer -- Radio Button   Very relevant Average Inappropriate N/A L4 general and L4 API 5 (42%) 5 (42%) 1 (8%) 0 (0%) 0 (0%) 1 (8%) Microkernels in general 6 (50%) 3 (25%) 2 (17%) 0 (0%) 0 (0%) 1 (8%) Caching and TLBs 9 (75%) 2 (17%) 1 (8%) 0 (0%) 0 (0%) 0 (0%) Threads and Events 8 (67%) 2 (17%) 1 (8%) 1 (8%) 0 (0%) 0 (0%) Security 6 (50%) 3 (25%) 2 (17%) 1 (8%) 0 (0%) 0 (0%) Dynamic Memory 9 (75%) 3 (25%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Microkernel construction issues 6 (50%) 4 (33%) 1 (8%) 0 (0%) 1 (8%) 0 (0%) Real-time systems 4 (33%) 3 (25%) 2 (17%) 2 (17%) 1 (8%) 0 (0%) Papers in Week 10 - Verified and high assurance kernels 4 (33%) 6 (50%) 1 (8%) 0 (0%) 1 (8%) 0 (0%) Virtual machines 9 (75%) 3 (25%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Multiprocessor Systems 9 (75%) 2 (17%) 1 (8%) 0 (0%) 0 (0%) 0 (0%) Local R&D Overview 6 (50%) 3 (25%) 2 (17%) 0 (0%) 0 (0%) 1 (8%)

12. Please tell us how interesting you found the lecture topics.   Question type : Single answer -- Radio Button   Very interesting Ok Boooooring! L4 general and L4 API 3 (25%) 6 (50%) 3 (25%) 0 (0%) 0 (0%) Microkernels in general 6 (50%) 4 (33%) 1 (8%) 1 (8%) 0 (0%) Caching and TLBs 8 (67%) 3 (25%) 1 (8%) 0 (0%) 0 (0%) Threads and Events 8 (67%) 2 (17%) 2 (17%) 0 (0%) 0 (0%) Security 3 (25%) 4 (33%) 3 (25%) 1 (8%) 1 (8%) Dynamic Memory 5 (42%) 7 (58%) 0 (0%) 0 (0%) 0 (0%) Microkernel construction issues 6 (50%) 5 (42%) 0 (0%) 0 (0%) 1 (8%) Real-time systems 2 (17%) 1 (8%) 4 (33%) 3 (25%) 2 (17%) Papers in Week 10 - Verified and high assurance kernels 3 (25%) 1 (8%) 7 (58%) 1 (8%) 0 (0%) Virtual machines 8 (67%) 4 (33%) 0 (0%) 0 (0%) 0 (0%) Multiprocessor Systems 8 (67%) 4 (33%) 0 (0%) 0 (0%) 0 (0%) Local R&D Overview 3 (25%) 5 (42%) 4 (33%) 0 (0%) 0 (0%)

13. Which material do you think will be most useful to you in the future?   Question type : Long-answer Answer at the bottom page (12 comments)

14. Which material, not currently in this course, would you liked to have seen covered? Question type : Long-answer Answer at the bottom page (6 comments)

15. Which of the current topics would you like to see scaled back or excluded? Question type : Long-answer Answer at the bottom page (9 comments)

4. Lectures

16. What factors caused you to attend lectures?   Question type : Multiple answer -- Check Box I had enough spare time 6 (50%) The lectures were too good to miss 11 (92%) Given the pace and lack of a textbook, I could not afford to miss the lectures 4 (33%) It was as good a place as any to take a nap 2 (17%) I wanted to be seen to be there 4 (33%) None, I skipped most 1 (8%)

17. What were the reasons for skipping lectures?   Question type : Multiple answer -- Check Box Overall workload in this and other courses 4 (33%) Lecture notes and references cover the material adequately 2 (17%) Lectures are boring 0 (0%) There was not enough material to justify attending lectures 0 (0%) First half of the course was more interesting than second half 0 (0%) None, I attended (almost) all 9 (75%)

18. Any suggestions for improving lectures? Question type : Long-answer Answer at the bottom page (5 comments)

5. Project

19. What was the level of difficulty various parts of the project?   Question type : Single answer -- Radio Button   Too easy Just right Too hard Milestone 0 1 (8%) 2 (17%) 7 (58%) 2 (17%) 0 (0%) Milestone 1 2 (17%) 1 (8%) 8 (67%) 0 (0%) 1 (8%) Milestone 2 2 (17%) 1 (8%) 5 (42%) 3 (25%) 1 (8%) Milestone 3 1 (8%) 0 (0%) 9 (75%) 2 (17%) 0 (0%) Milestone 4 0 (0%) 0 (0%) 6 (50%) 6 (50%) 0 (0%) Milestone 5 0 (0%) 2 (17%) 8 (67%) 2 (17%) 0 (0%) Milestone 6 0 (0%) 0 (0%) 5 (42%) 6 (50%) 1 (8%) Milestone 7 0 (0%) 2 (17%) 8 (67%) 2 (17%) 0 (0%) Milestone 8 0 (0%) 0 (0%) 10 (83%) 1 (8%) 1 (8%) System documentation 0 (0%) 0 (0%) 10 (83%) 1 (8%) 1 (8%) Project overall 0 (0%) 0 (0%) 9 (75%) 2 (17%) 1 (8%)

20. How well was the project specified?   Question type : Single answer -- Radio Button   Very clear Ok Confusing Milestone 0 1 (8%) 4 (33%) 6 (50%) 1 (8%) 0 (0%) Milestone 1 3 (25%) 2 (17%) 5 (42%) 1 (8%) 1 (8%) Milestone 2 2 (17%) 1 (8%) 4 (33%) 3 (25%) 2 (17%) Milestone 3 0 (0%) 3 (25%) 5 (42%) 4 (33%) 0 (0%) Milestone 4 3 (25%) 1 (8%) 3 (25%) 4 (33%) 1 (8%) Milestone 5 4 (33%) 2 (17%) 3 (25%) 3 (25%) 0 (0%) Milestone 6 2 (17%) 2 (17%) 5 (42%) 3 (25%) 0 (0%) Milestone 7 1 (8%) 5 (42%) 4 (33%) 2 (17%) 0 (0%) Milestone 8 3 (25%) 3 (25%) 2 (17%) 3 (25%) 1 (8%) System documentation 2 (17%) 3 (25%) 5 (42%) 1 (8%) 1 (8%) Project overall 0 (0%) 3 (25%) 5 (42%) 4 (33%) 0 (0%)

21. What was the quality of...   Question type : Single answer -- Radio Button   Excellent Ok Poor Documentation/reference material 0 (0%) 2 (17%) 1 (8%) 5 (42%) 4 (33%) Supplied code 0 (0%) 1 (8%) 4 (33%) 4 (33%) 3 (25%) Hardware platform 8 (67%) 4 (33%) 0 (0%) 0 (0%) 0 (0%) Consultation time help/support 2 (17%) 4 (33%) 4 (33%) 1 (8%) 1 (8%) On-line help/support 1 (8%) 7 (58%) 4 (33%) 0 (0%) 0 (0%)

22. Any suggestions for improving the project? Question type : Long-answer Answer at the bottom page (10 comments)

6. Anything Else

23. Any other comments/suggestions that might help us to improve the course in the future? Question type : Long-answer Answer at the bottom page (5 comments)

3. Other factors not mentioned above? 6. What were the best things about this course? 1: Amazing subject - the amount you learn in a short space of time is staggering. 2: Assignment was much more interesting than any other assignment I've done. 3: Challenge, design aspect of the project. Casual nature of the lectures. 4: Hands on with own OS personality of which the design of was left pretty much all to us. Learning c very very well and becoming quite comfortable with OS design and implementation. 5: It mention most of the aspect of Operating System with implementation based. 6: See 5 :) 7: The assignment was very rewarding, and we could get our fingers dirty and build a large, complex system. 8: The chance to build an OS "from scratch". The nights I didn't sleep because of the project. The nights I could sleep after all, happy for finishing it well. 9: The course was entirely practical work which is a refreshing change. Implementing concepts learnt in class is a far more effective way of reinforcing those ideas than just testing students on their memory/knowledge in an exam. 10: The project is challenging and interesting 11: The project was great. It was very challenging but very rewarding. 12: difficult and challenging, no handholding 7. What were the worst things about this course? 1: Assignment spec was a little unclear.. 2: I can't find anything wrong with it. 3: I've never really read any research papers before, so I found the exam quite difficult and it was very stressful. 4: Lack of sleep :p follow by lecture after milestone ... 5: Oud is too cold, L4 documentation not great - in general spent relatively too much time doing tedious/mundane things. 6: The amount of time required for the assignments (though it is worth it) 7: The workload is very very high. 12 week semesters are putting huge strain on students and this is not adequately compensated for. 12UOC!!!! 8: Work load. It is necessary though. Documentation! It was pretty bad, especially the first half of the course while you were getting used to L4. The documentation should include a user guide as well since initially the reference manual isn't useful at all since it provides far too much information of which most you don't care about. A nice quick documentation of the L4 helper library would have been great as well since the plethora of L4_Send... commands is quite confusing at first. The details on what was required to complete M2 and general documentation on the boot process and boot-info was also very bad. It wasn't helped by the fact that some of the tutors only had a fairly general idea as well. 9: Workload. Effect thereof on other marks, to say nothing of personal health. 10: having to do the assignment with a partner. I think the project should be individual. 11: the L4 ref doc is sucks 10. What background knowledge do you think you were missing that would have helped you in this course? Is distinction in COMP3231/9201 a suitable preparation? Is it too harsh? 1: A distinction is probably suitable restriction, but an HD should be recommended. 2: D or HD in operating systems is definitely appropriate - you need to know and understand all the concepts from the get-go. 3: Distinction in 3231 is suitable 4: How to implement a file system. 5: I think a distinction in Comp3231 is needed. 6: I was taking Computer Architecture COMP3211 side-by-side, and I found this useful. 7: If you don't get HD in COMP3231/9201 I don't think you can do this course without working **really** hard. 8: It's enough. You pick up the rest. 9: Yes its suitable. Perhaps an extra optional lecture at the start or hand out for revising the core OS topics from COMP3231. This would have benefited a fair amount those students who had taken COMP3231 more than 1 year ago. 10: a bit more detail of IPC and thread loop 13. Which material do you think will be most useful to you in the future?   1: All of the above in 12. 2: Caching, TLB and dynamic memory issues. 3: Depends where I go in life. Generally I would say coding skills and OS design/general knowledge. If I end up in research, then probably everything. 4: Greater understanding on microkernels in general, as well as real time and virtual machines 5: I guess Virtual machines and Microkernel in general 6: Lecture notes 7: Memory, Virtual machines, threads, microkernel construction issues. 8: Mostly the skills - this is a subject that turns you from a good programmer to a great one. 9: Multi-processor systems 10: Nothing specific, just the overall understanding of the construction of kernels and the problems that arise. 11: Virtual Machines 12: microkernel construction issues 14. Which material, not currently in this course, would you liked to have seen covered? 1: Can't think of anything 2: I would have liked to have covered more OS current research areas. 3: Internals of L4 (OKL4), how is it implemented, tricks and cool implementation details. 4: More info on monolithic kernels. OSX/Windows/Linux make up the bulk of currently used systems but we did not cover these in any real depth. This course (rightly or wrongly) felt like it was aimed at micro-kernel indoctrination. 5: Not sure. 6: n/a 15. Which of the current topics would you like to see scaled back or excluded? 1: Don't know, they are all relevant. 2: General-purpose computing. All of the applications are in real-time systems - microkernel-based OSs equally have applications in non-real-time environments, with different uses and failings. 3: Microkernels in general. 4: Perhaps less history lessons on microkernels. I think I remember atleast 2 lectures where this came up, 1 is enough. 5: Real time systems 6: Real time systems probably deserves a half lecture, not a full 3 hour one. 7: Would have liked to have seen the real-time systems information excluded as I had already covered nearly all of it in COMP3231 but I think this is because it was cut out when unsw changed to 12 week semesters. 8: n/a 9: nothing 18. Any suggestions for improving lectures? 1: Don't have them on or near demo day! 2: More breaks. More reliable hardware... 3: Put them on the monday of the week if milestones are on friday. -- at least 2 days before the milestone :p 4: The main problem with them was being on friday afternoon, just after the milestones are due. Being up most of the night before made it hard to stay awake. 5: Wider range of topics. Sometimes there was too much focus on L4 or microkernels. 22. Any suggestions for improving the project? 1: * decent L4 documentation! * skeleton codebase that implements some aspect of decent software design (if not canonical microkernel-OS design) * clearer/appropriate specs, for example... - what does "block" in "your system must not block on... I/O" (M9) mean, specifically? - register_timer() is only useful in non-cap-based versions of L4 - LWIP as given is not thread-safe - malloc is not thread-safe and will explode! (and the necessary headers to make it thread safe via the #define don't exist) - process control should be far earlier: comes in m7, but we needed to learn how to add processes to the bootimage and parse bootinfo by m4 2: Aligning lectures and project milestones would be a good start. For example, the caching problem on the ARM processor we were using on the slug was not explained until the lecture after that milestone was due. The same thing happened for threads vs events after we'd just designed our RPC interface. If possible, lectures that assist milestones should be delivered before the milestone is due. 3: Better Documentation. I think it would have been better also in the assignment work to perhaps scale back some of the required milestones (there was a lot on memory) and had one or two open ended milestones where we had to pick and implement an extension of our own choosing, perhaps with a novel design or API. (e.g network layer through VFS API). While we could do extensions, I would have prefered more time for this. 4: Better reference manual ... 5: I think a bit more guidance on design would be nice, at the start. Not much (since the fun/interesting/challenging part is to develop your own design) but nevertheless... just a bit to get started. Another thing is, a nicer skeleton OS. 6: Less bugs in the given code =p Some milestones are a bit unclear, and some clarification would be useful. 7: Need to think about the impact of 12 week sessions. The relentless pace of 1 submission per week is such that no other subject can get a look in. 8: No, very good project. Some supplied code is horrible (SOS, network stack), but we can live with it. 9: Occasionally some of the documentation was a bit misleading, or inaccurate, particularly the bootinfo parts. The L4 documentation was also missing some very useful functions 10: make it individual work 23. Any other comments/suggestions that might help us to improve the course in the future? 1: * highlight the difficulty of the herd-of-servers approach (or make it less hard with adjustments to the skeleton codebase) * is there a slug-like simulator? would make the dev cycle MUCH faster * make sure the fundamental nature of IPC/RPC is really understood by students, as early as possible * thread-safe LWIP and libc * NIC code that can realistically be relocated out of the rootthread (or Big Red Warnings not to even try) * 12UoC! 2: Better documentation would be of great help, especially about the bootinfo or just the build system in general of pistachio and sos. 3: L4 lecture should be done before the first milestone - it would help the basic understanding of IPC so much more. 4: Make it harder... :) 5: Probably the best CSE course.