Assignment 1: Give

version: 1.19 last updated: 2024-07-01 14:40

Aims

This assignment aims to give you

  • practice in Shell programming generally

Introduction

You are going to implement simple but functional versions of give and autotest.

Your task in this assignment is to write 8 shell scripts named give-add give-submit give-summary give-status give-fetch give-autotest give-mark give-rm.

Reference implementation

Many aspects of this assignment are not fully specified in this document; instead, you must match the behaviour of a reference implementation.

For example, your script give-add should match the behaviour of 2041 give-add exactly, including producing the same error messages.

A reference implementation is a common method to provide an operational specification, and it's something you will likely encounter after you leave UNSW.

Discovering and matching the reference implementation's behaviour is deliberately part of the assignment.

While the code in the reference implementation is fairly straightforward, reverse-engineering its behaviour is obviously not so simple, and is a nice example of how coming to grips with the precise semantics of an apparently obvious task can still be challenging.

If you discover what you believe to be a bug in the reference implementation, report it in the class forum. We may fix the bug, or indicate that you do not need to match the reference implementation's behaviour in this case.

Storing information in .give

Your Give commands will need to store information.

They should do this by creating a directory in the current directory named .give and creating files and/or sub-directories with it as needed. They should not store information elsewhere.

You can choose what information you store in .give and you can choose how to store it.

The reference implementation also stores information in .give in a subdirectory named .reference. Do not change anything in this directory or store files in this directory. You do not have to create the same file or directories as the reference implementation.

Note, normally a system like this would store this information in a separate directory owned by another user, e.g it might be stored /home/cs2041/work and owned by cs2041. For the purposes of this assignment, we are storing it in .give

Note, also some of the commands below would normally be run by the course account (add, summary, mark, rm) and some by students (submit, status, fetch, autotest). For the purposes of this assignment, you are running all of them.

Give Commands

give-add <assignment> <solution> <autotests> <automarking>

give-add command creates a new Give assignment.

Its first argument should be the name of the assignment. Assignment names must start with a lowercase letter ([a-z]) and be followed only by letters, number and underscore ([a-zA-Z0-9_]).

Its second argument should be the pathname of a file containing a correct solution for the assignment. The pathname should only contain letters [a-zA-Z], numbers ([0-9]), underscore ('_'), dash ('-'), dot ('.') and slash ('/').

Its third argument should be the name of a file specifying the autotests for the assignment.

Its fourth argument should be the name of a file specifying the automarking for the assignment.

The format of autotest and automarking specification files is described below.

give-submit <assignment> <zid> <filename>

give-submit makes a submission for the assignment by a student.

Its first argument should be the name of the assignment.

Its second argument should be the zid of the student.

Its third argument should be the pathname of file containing the submission. The pathname should only contain letters [a-zA-Z], numbers ([0-9]), underscore ('_'), dash ('-'), dot ('.') and slash ('/').

A student zid should be a 'z' followed by 7 digits.

Students can make multiple submissions. Only their last submission is marked but all submisisons can be retrieved (see below).

give-summary

give-summary lists all assignments. with a count of count of how many students have made submissions. It takes no arguments.

give-status <zid>

give-status lists all submissions a student has made.

Its first argument should be the zid of the student.

give-fetch <assignment> <zid> [n]

give-fetch outputs the contents of a submission the student has made.

Its first argument should be the name of the assignment

Its second argument should be the zid of the student.

Its optional third argument is the number of the submission to print. If should be an integer. If the third argument is omitted, the last submission should be printed.

Non-positive integers specify submissions relative to the last submission. Zero specifies the last submission, -1 the second-last submission, -2 the third-last submission, and so on.

give-autotest <assignment> <filename>

give-autotest runs the autotests for an assignment on a program.

Its first argument should be the name of the assignment

Its second argument should be the name of the file containing the program to be tested. The filename should only contain letters [a-zA-Z], numbers ([0-9]), underscore ('_'), dash ('_'), dot ('.') and slash ('/').

An autotest specification file specifies tests one per line.

Each line should contain 4 columns separated by pipe ('|') characters.

The first column should contain a label for the test. Test labels must start with a lowercase letter ([a-z]) and be followed only by letters, number and underscore ([a-zA-Z0-9_]). Test label should be unique for each test.

The second column should contain arguments, if any, for the test. Arguments should only contain the characters [a-zA-Z0-9_.-]. Multiple arguments should be separated by spaces.

The third column should contain stdin, if any, for the test. It should contain only non-white-space ASCII characters plus spaces. The two characters '\n' may appear to denote a newline. Backslash ('\') should only appear followed by an 'n'. It can not contain a pipe character ('|')

The fourth column should specify test options. This should be zero or more of the characters [bcdw] in any order. Options apply when comparing both stdout and stderr to the correct output.

If the option b is specified differences regarding empty lines should be ignored.

If the option c is specified differences in case (uppercase versus lowercase) should be ignored.

If the option d is specified differences regarding characters other than [0-9] and '\n' should be ignored.

If the option w is specified differences in space (' ') and tab ('\t') characters should be ignored.

Autotest specifications file can also contain lines with the first character '#'. These are comments and should be ignored. empty lines should be ignored.

give-mark <assignment>

give-mark runs the automarking for an assignment on the last submission of each student.

Its first argument should be the name of the assignment

An automarking specification file specifies tests one per line.

Automarking files have the same format as autotest files, except they have an extra fifth column specifying the marks for passing this test. It should be a non-negative integer.

give-rm <assignment>

give-rm removes an assignment.

Its first argument should be the name of the assignment

Examples

Here are some simple examples of how your 8 shell scripts should behave.

You will need to use the reference implementation to discover much more about exactly how your scripts should behave. 

ls -d .give
ls: cannot access '.give': No such file or directory
give-add lab1 multiply.sh multiply.autotests multiply.automarking
directory .give created
assignment lab1 created
ls -d .give
.give
give-add lab2 answer.sh answer.autotests answer.automarking
assignment lab2 created
give-submit lab1 z5000000 multiply_wrong.sh
Submission accepted - submission 1: multiply_wrong.sh 42 bytes @ Sun Nov 24 13:23:12 2024
give-submit lab1 z5000000 multiply_right.py
Submission accepted - submission 2: multiply_right.py 78 bytes @ Sun Nov 24 13:23:12 2024
give-submit lab2 z5000000 answer_wrong.sh
Submission accepted - submission 1: answer_wrong.sh 407 bytes @ Sun Nov 24 13:23:12 2024
give-submit lab2 z5111111 answer.sh
Submission accepted - submission 1: answer.sh 47 bytes @ Sun Nov 24 13:23:12 2024
give-summary
assignment lab1: submissions from 1 students
assignment lab2: submissions from 2 students
give-status z5111111
* 1 submissions for lab2
submission 1: answer.sh 47 bytes @ Sun Nov 24 13:23:12 2024
give-fetch lab1 z5000000
#!/usr/bin/python3
import sys
a=int(sys.argv[1])
b=int(input())
print(a * b)

give-autotest lab1 multiply_right.py
* Test test_a passed.
* Test test_b passed.
* Test test_c passed.
** 3 tests passed, 0 tests failed
give-mark lab1
*** Student z5000000 - submission 2: multiply_right.py 78 bytes @ Sun Nov 24 13:23:12 2024
* Test marking1 passed (10 marks).
* Test marking2 passed (30 marks).
* Test marking3 passed (20 marks).
* Test marking4 passed (42 marks).
** 4 tests passed, 0 tests failed - mark: 102/102
give-add lab1 grep.sh grep.autotests grep.automarking
give-add: assignment lab1 already exists
give-rm lab1
assignment lab1 removed
give-add lab1 grep.sh grep.autotests grep.automarking
assignment lab1 created

Test Scripts

You should submit ten Shell scripts, named test0.sh to test9.sh, which run give commands that test an aspect of Give.

The test?.sh scripts do not have to be examples that your program implements successfully.

You may share your test examples with your friends, but the ones you submit must be your own creation.

The test scripts should show how you've thought about testing carefully.

You are only expected to write test scripts testing parts of Give you have attempted to implement. For example, if you have not implemented give-autotest you are not expected to write test scripts testing give-autotest .

Permitted Languages

Your programs must be written entirely in POSIX-compatible shell.

Your programs will be run with dash(1), in /bin/dash. You can assume anything that works with the version of /bin/dash on CSE systems is POSIX compatible.

Start your programs with: 

#!/bin/dash

If you want to run these scripts on your own machine — for example, one running macOS — which has dash(1) installed somewhere other than /bin, use: 

#!/usr/bin/env dash

You are permitted to use any feature /bin/dash provides.

On CSE systems, /bin/sh is the Bash shell: /bin/sh is a symlink to /bin/bash. Bash implements many non-POSIX extensions, including regular expressions and arrays. These will not work with /bin/dash, and you are not permitted to use these for the assignment.

You are not permitted to use Perl, Python or any language other than POSIX-compatible shell.

You are permitted to use only these external programs:

  • basename(1)
  • bunzip2(1)
  • bzcat(1)
  • bzip2(1)
  • cat(1)
  • chmod(1)
  • cmp(1)
  • combine(1)
  • cp(1)
  • cpio(1)
  • csplit(1)
  • cut(1)
  • date(1)
  • dc(1)
  • dd(1)
  • df(1)
  • diff(1)
  • dirname(1)
  • du(1)
  • echo(1)
  • egrep(1)
  • env(1)
  • expand(1)
  • expr(1)
  • false(1)
  • fgrep(1)
  • find(1)
  • fold(1)
  • getopt(1)
  • getopts(1)
  • grep(1)
  • gunzip(1)
  • gzip(1)
  • head(1)
  • hostname(1)
  • ifne(1)
  • less(1)
  • ln(1)
  • ls(1)
  • lzcat(1)
  • lzma(1)
  • md5sum(1)
  • mkdir(1)
  • mktemp(1)
  • more(1)
  • mv(1)
  • nl(1)
  • patch(1)
  • printf(1)
  • pwd(1)
  • readlink(1)
  • realpath(1)
  • rev(1)
  • rm(1)
  • rmdir(1)
  • sed(1)
  • seq(1)
  • sha1sum(1)
  • sha256sum(1)
  • sha512sum(1)
  • sleep(1)
  • sort(1)
  • sponge(1)
  • stat(1)
  • strings(1)
  • tac(1)
  • tail(1)
  • tar(1)
  • tee(1)
  • test(1)
  • time(1)
  • top(1)
  • touch(1)
  • tr(1)
  • true(1)
  • uname(1)
  • uncompress(1)
  • unexpand(1)
  • uniq(1)
  • unlzma(1)
  • unxz(1)
  • unzip(1)
  • wc(1)
  • wget(1)
  • which(1)
  • who(1)
  • xargs(1)
  • xz(1)
  • xzcat(1)
  • yes(1)
  • zcat(1)

Only a few of the programs in the above list are likely to be useful for the assignment.

Note you are permitted to use built-in shell features including: cd, exit, for, if, read, shift and while.

If you wish to use an external program which is not in the above list, please ask in the class forum for it to be added.

You may submit extra shell files.

Autotests

As usual, some autotests will be available: 

2041 autotest give 
...

If you are using extra Shell files, include them on the autotest command line.

You can download the files used by autotest as a zip file or a tar file.

Fo example: 

curl -s https://cgi.cse.unsw.edu.au/~cs2041/24T2/assignments/ass1/tar|tar -xvf -
You will need to do most of the testing yourself.

Assumptions/Clarifications

Like all good programmers, you should make as few assumptions as possible.

You can assume that all give commands are always run in the same directory.

Error-checking is required. You should match the error messages produced by the reference implementation.

You should match the output streams used by the reference implementations. It writes error messages to stderr: so should you.

You should match the exit status used by the reference implementation. It exits with status 1 after an error: so should you.

You do not have to handle concurrency. You can assume only one instance of any give command is running at any time.

You can assume stdout and stderr produced by programs, and all files provided as arguments contain only ASCII bytes and you can assume the only non-printing ASCII bytes they contain are space (' '), tab ('\t') and newline('\n'). So, for example, you can assume stdout and stderr do not contain '\r' characters.

You can assume, pathnames provided as arguments, if they exist, are ordinary files. You can, for example, assume they are not a symbolic link, directory or device. Note, the command 2041 fetch give creates symbolic links. If this creates issues for your scripts, use the provided zip or tar files instead to create ordinary files.

You can assume the last character in files provided for autotests and automarking is a new line ('\n'), unless the file is empty (zero length).

You can assume programs when run terminate within 60 seconds and produce less than 1 megabyte of output on stdout and stderr

Error messages from the reference implementation include the program name. It is recommended you extract the program name (e.g. using basename) from $0 however it is also acceptable to hard-code the program name. The automarking and style marking will accept both.

You can assume the contents of .give are only modifed by your scripts.

Change Log

Version 1.0
(2024-06-20 12:00)
  • Initial release
Version 1.1
(2024-06-20 16:00)
  • Several major bug-fixes in reference-implementation
  • give-summary added to spec
Version 1.2
(2024-06-21 15:00)
  • autotests added, bug fixed in reference implementation
Version 1.3
(2024-06-22 13:15)
  • autotests fixed, bug fixed in reference implementation for give-autotest
Version 1.4
(2024-06-22 15:50)
  • reference implementation updated to fix bug in give-summary when failed submissions where still counted
Version 1.5
(2024-06-22 20:30)
  • fix the previous fix for give-summary so that it actually fixes the bug
Version 1.6
(2024-06-22 22:40)
  • clarify how to use $0 for error messages
Version 1.7
(2024-06-25 15:00)
  • usage message for give-submit and give-autotest changed in reference implementation
  • autotests added
Version 1.8
(2024-06-25 16:30)
  • clarify restrictions on filenames for give-add give-submit & give-autotest
Version 1.9
(2024-06-25 19:30)
  • clarify autotest and automarking files always finish with a newline
Version 1.10
(2024-06-25 19:35)
  • remove restriction on autotest and automarking filenames
Version 1.11
(2024-06-26 10:30)
  • add files used in autotest to zip
Version 1.12
(2024-06-29 10:30)
  • fix bug in reference-implementation for submission fielnames that don't start with letter
Version 1.13
(2024-06-29 10:30)
  • add autotests for fetch
Version 1.14
(2024-06-29 21:50)
  • change filename to pathname in give-submit description
Version 1.15
(2024-06-30 16:30)
  • clarification that pathname given as arguments if they exist can only be ordinary files
  • autotest added for incorrect stderr & exit status
Version 1.16
(2024-06-30 19:30)
  • assumption added that you can assume .give changed by your scripts
  • internal error fixed in reference implementation
Version 1.17
(2024-07-01 10:15)
  • reference implementation output for autotest changed: newlines added to description of why behaviour is incorrect
Version 1.18
(2024-07-01 13:00)
  • change reference implementation so stdout+stderr redirected to file is same as when output to terminal
Version 1.19
(2024-07-01 14:40)
  • reference implementation changed to (as the spec says) allow one letter test labels
Version 1.19
(2024-07-01 14:40)
  • autotests 18 & 19 changed to test what they are intended to

Assessment

Testing

When you think your program is working, you can use autotest to run some simple automated tests: 

2041 autotest give

2041 autotest will not test everything.
Always do your own testing.

Automarking will be run by the lecturer after the submission deadline, using a superset of tests to those autotest runs for you.

Submission

When you are finished working on the assignment, you must submit your work by running give: 

give cs2041 ass1_give give-* test[0-9].sh[any-other-files]

You must run give before Week 7 Monday 11:59:59 2024 (midday) to obtain the marks for this assignment. Note that this is an individual exercise, the work you submit with give must be entirely your own.

You can run give multiple times.
Only your last submission will be marked.

If you are working at home, you may find it more convenient to upload your work via give's web interface.

You cannot obtain marks by emailing your code to tutors or lecturers.

You can check your latest submission on CSE servers with: 

2041 classrun check ass1_give

You can check the files you have submitted here.

Manual marking will be done by your tutor, who will mark for style and readability, as described in the Assessment section below. After your tutor has assessed your work, you can view your results here; The resulting mark will also be available via give's web interface.

Due Date

This assignment is due Week 7 Monday 11:59:59 2024 (midday) (2024-07-08 11:59:00).

The UNSW standard late penalty for assessment is 5% per day for 5 days - this is implemented hourly for this assignment.

Your assignment mark will be reduced by 0.2% for each hour (or part thereof) late past the submission deadline.

For example, if an assignment worth 60% was submitted half an hour late, it would be awarded 59.8%, whereas if it was submitted past 10 hours late, it would be awarded 57.8%.

Beware - submissions 5 or more days late will receive zero marks. This again is the UNSW standard assessment policy.

Assessment Scheme

This assignment will contribute 15 marks to your final COMP(2041|9044) mark

15% of the marks for assignment 1 will come from hand-marking. These marks will be awarded on the basis of clarity, commenting, elegance and style: in other words, you will be assessed on how easy it is for a human to read and understand your program.

5% of the marks for assignment 1 will be based on the test suite you submit.

80% of the marks for assignment 1 will come from the performance of your code on a large series of tests.

An indicative assessment scheme follows. The lecturer may vary the assessment scheme after inspecting the assignment submissions, but it is likely to be broadly similar to the following:

HD (85+) All commands working correctly; code is beautiful
DN (75+) Most commands working correctly; code is readable
CR (65+) Some commands working correctly, code is mostly readable
PS (50+) Good progress on assignment, but not passing autotests
0% knowingly providing your work to anyone
and it is subsequently submitted (by anyone).
0 FL for
COMP(2041|9044) 		submitting any other person's work; this includes joint work.
academic
misconduct 		submitting another person's work without their consent;
paying another person to do work for you.

Intermediate Versions of Work

You are required to submit intermediate versions of your assignment.

Every time you work on the assignment and make some progress you should copy your work to your CSE account and submit it using the give command below. It is fine if intermediate versions do not compile or otherwise fail submission tests. Only the final submitted version of your assignment will be marked.

Attribution of Work

This is an individual assignment.

The work you submit must be entirely your own work, apart from any exceptions explicitly included in the assignment specification above. Submission of work partially or completely derived from any other person or jointly written with any other person is not permitted.

You are only permitted to request help with the assignment in the course forum, help sessions, or from the teaching staff (the lecturer(s) and tutors) of COMP(2041|9044).

Do not provide or show your assignment work to any other person (including by posting it on the forum), apart from the teaching staff of COMP(2041|9044). If you knowingly provide or show your assignment work to another person for any reason, and work derived from it is submitted, you may be penalized, even if that work was submitted without your knowledge or consent; this may apply even if your work is submitted by a third party unknown to you. You will not be penalized if your work is taken without your consent or knowledge.

Do not place your assignment work in online repositories such as github or anywhere else that is publicly accessible. You may use a private repository.

Submissions that violate these conditions will be penalised. Penalties may include negative marks, automatic failure of the course, and possibly other academic discipline. We are also required to report acts of plagiarism or other student misconduct: if students involved hold scholarships, this may result in a loss of the scholarship. This may also result in the loss of a student visa.

Assignment submissions will be examined, both automatically and manually, for such submissions.