Week 08 Tutorial Sample Answers

Objectives

In assignment 2, you will implement a simple subset of the functionality of the sed command.
Here are some sed commands and their output.
Discuss what these print and why:

quit
```
seq 42 44 | sed 1q
```
```
    42
```
1 is the address.

q is the command.

The address 1 is the address of the first line.

The command q is the command to quit.

So 1q will quit on the first line.

sed will print the current line before it quits.

Giving us a single line of output: whatever the first line is.

In this case, the first line is 42.
```
sed 10q < dictionary.txt
```
```
    ...
    ...
    the first 10 lines of the dictionary.txt file
```
10 is the address.

q is the command.

The address 10 is the address of the 10th line.

The command q is the command to quit.

So 10q will quit on the 10th line.

sed will print the current line before it quits.

Giving us 10 lines of output: the first 10 lines of dictionary.txt.
```
seq 41 43 | sed 4q
```
```
    41
    42
    43
```
4 is the address.

q is the command.

The addess 4 is the address of the 4th line.

The command q is the command to quit.

So 4q will quit on the 4th line.

But as there are only 3 lines of input, sed will hit EOF first.

Therefore, all three lines of output will be printed.

The q command never gets the chance to be used.
```
seq 90 110 | sed /.1/q
```
```
    90
    91
```
/.1/ is the address.

q is the command.

The addess /.1/ is the address of any line that matches the regex .1.

The command q is the command to quit.

The line 91 matches the regex .1.

As the regex . (any character) matches the 9.

So we will quit on the line 91.

As always print the current line before quitting.
```
sed '/r.*v/q' < dictionary.txt
```
```
    ...
    ...
    aardvark
```
/r.*v/ is the address.

q is the command.

The addess /r.*v/ is the address of any line that matches the regex r.*v.

The command q is the command to quit.

Depending on the contents of the dictionary.txt file, the lines printed may be different.

For my dictionary 247 line were printed before aardvark matches the regex r.*v.

All lines up to and including aardvark will be printed.
```
yes | sed 3q
```
```
    y
    y
    y
```
Note: the yes command will print y infinitely.

3 is the address.

q is the command.

The addess 3 is the address of the 3rd line.

The command q is the command to quit.

Because yes prints infinitely sed can't wait untill EOF to stop.

sed also can't read all input lines into an array.

sed must process lines one at a time.

Note: because of the $ address (last line) sed needs to read two lines at a time.
The current lines and the next line (to detect when there is no next line).
sed should not store more that two lines in memory at any time.
print
```
seq 41 43 | sed 2p
```
```
    41
    42
    42
    43
```
2 is the address.

p is the command.

The addess 2 is the address of the 2nd line.

The command p is the print command.

So 2p will print on the second line.

This print is in addition to the automatic print of the current line that sed already does.

This causes the second line to be printed twice.
```
head dictionary.txt | sed 3p
```
Third line of the dictionary.txt file is printed twice.
```
seq 41 43 | sed -n 2p
```
```
    42
```
The -n option is used suppress (turn off) the automatic printing of the current line.

Therefore we only print when explicitly asked to.

We are asked to print the second line, and so get the output of 42.
```
sed -n 42p < dictionary.txt
```
Similar to the previous example.

Only the 42nd line is printed.
```
head -n 1000 dictionary.txt | sed -n '/z.$/p'
```
Similar to the previous example.

Only print a line if it matches the regex z.$.

That is: if the second last character is a z.

substitute
```
seq 10 15 | sed 's/[15]/zzz/'
```
```
    zzz0
    zzz1
    zzz2
    zzz3
    zzz4
    zzz5
```
Run the substitute command on each line.

Replace the first instance of 1 or 5 on each line with zzz.
```
seq 10 15 | sed 's/[15]/zzz/g'
```
```
    zzz0
    zzzzzz
    zzz2
    zzz3
    zzz4
    zzzzzz
```
Run the substitute command on each line.

Replace all instances of 1 or 5 with zzz.
```
echo "Hello Andrew" | sed 's/e//'
```
```
    Hllo Andrew
```
Run the substitute command on each line.

Replace the first instance of e on each line with the empty string.
```
echo "Hello Andrew" | sed 's/e//g'
```
```
    Hllo Andrw
```
Run the substitute command on each line.

Replace all instances of e with the empty string.
addresses
```
seq 1 5 | sed '$d'
```
```
    1
    2
    3
    4
```
$ is the special address for the last line.

d is the delete command.

If a line is deleted then processing immediately moves on to the next line.

The line is not automatically printed.
```
seq 42 44 | sed 2,3d
```
```
    42
```
2,3 is a range address.

The command d is applied to all lines within the range (start and end line inclusive).

So it will delete all lines from line 2 to line 3.
```
seq 10 21 | sed 3,/2/d
```
```
    10
    11
    21
```
The range address is between 3 and /2/.

The first address is the third line.

The second address is the first line that matches the regex 2, that comes after the third line.

So it will delete all lines in between these two addresses (inclusive).
```
seq 10 21 | sed /2/,7d
```
```
    10
    11
    17
    18
    19
```
Similar to the previous example.
```
seq 10 21 | sed /2/,/7/d
```
```
    10
    11
    18
    19
```
The first address /2/ is the line that matches the regex 2.

The first address /7/ is the line that matches the regex 7, after the first address.

The command will delete all lines between any range that matches these two addresses.

Since 20 also matches the regex 2, it will continute to delete the last 2 lines 20 and 21.
substitute
```
seq 1 5 | sed 'sX[15]XzzzX'
```
```
    zzz
    2
    3
    4
    zzz
```
Similar to substitution except the X character is being used to delimit the substitute command instead of the / character.
multiple commands
```
seq 1 5 | sed '4q;/2/d'
```
```
    1
    3
    4
```
The first command is 4q which will quit printing after the 4th line of input.

The second command is /2/d which will delete any line matching the regex 2.

Each command runs on each line of input.
-f
```
echo "4q" > commands.script
echo "/2/d" >> commands.script
seq 1 5 | sed -f commands.script
```
```
    1
    3
    4
```
These are the same commands as the previous question, being read from the specified file.
input files
```
seq 1 2 > two.txt
seq 1 5 > five.txt
sed '4q;/2/d' two.txt five.txt
```
```
    1
    1
    2
```
sed runs the commands on the contents of the input files two.txt and five.txt instead of on the lines from stdin.

The edited lines are still printed to stdout without the -i argument.

The second 2 is still printed because it is on the 4th line and the q command quit before /2/d was run on this line.
whitespace and comments
```
seq 24 42 | sed ' 3, 17  d  # comment   ;   /2/p'
```
```
    24
    25
    41
    42
```
The command is 3,17d with whitespace characters around the addresses and command.

The # character starts a comment in the command.
-i
```
seq 1 5 > five.txt
sed -i /[24]/d five.txt
cat five.txt
```
```
    1
    3
    5
```
The file contents of five.txt are replaced with the output of the sed command.

sed does not print anything to stdout.
multiple commands
```
echo 'Punctuation characters include . , ; :' | sed 's/;/semicolon/g;/;/q'
```
```
    Punctuation characters include . , semicolon :
```
The first ; is part of the regex in the substitution command to replace ';' with 'semicolon'.

The second ; is separating multiple commands.

The third ; is part of the second regex.

What types are available as inbuilt types in Python?

int()
float()
str()

[] - list()

            [1, 2, 3]
            [] # empty list

{} - dict() - store data values in key:value pairs.

            {"key1": "value1", "key2": "value2"}
            {} # empty dict.

(,) - tuple() - immutable list

            (1, 2, 3, 4, 5)
            ("hello",) # single elements must end with a comma or python will think it's just a grouping operator
            () # NOT an empty tuple
            tuple() # is an empty tuple

{,} - set() - unordered collection of unique elements

            {1, 2, 3, 4, 5}
            {"hello",} # single elements must end with a comma or python will complain it looks like a dict
            {} # NOT an empty set (is a dict)
            set() # is an empty set

frozenset - like a set, but immutable (like a tuple is to a list)

What other useful types are available with Python's standard library?

collections.Counter

Works like a dictionary, but with some extra methods, plus has a default value of 0.

            from collections import Counter
            c = Counter()
            c['a'] += 1   # index error with a normal dictionary
            print(c['a'])

collections.defaultdict

Works like a dictionary, but you can set a default value for missing keys.

            from collections import defaultdict
            c = defaultdict(list)
            c['a'].append("hello")   # index error with a normal dictionary
            print(c['a'])

collections.OrderedDict

Works like a dictionary, but keeps the order of the keys.

ie. OrderedDict.keys() is the same order as the keys were added.

Since Python 3.7 this is also true for normal dictionaries. There are only subtle differences between dicts and OrderedDicts.
```
            from collections import OrderedDict
            c = OrderedDict()
            c['z'] = "COMP1511"
            c['a'] = "COMP1521"
            c['k'] = "COMP1531"
            c['b'] = "COMP2041"
            print(c.keys())
        
```

Write a Python function print_dict() that displays the contents of a dict in the format below:

[key] => value
[Andrew] => green
[Anne] => red
[John] => blue

Write a Python program, times.py which prints a table of multiplications.

Your program will be given the dimensions of the table and the width of the columns to be printed. For example:

./times.py 5 5 5
    1     2     3     4     5
    2     4     6     8    10
    3     6     9    12    15
    4     8    12    16    20
    5    10    15    20    25

./times.py 10 10 3
  1   2   3   4   5   6   7   8   9  10
  2   4   6   8  10  12  14  16  18  20
  3   6   9  12  15  18  21  24  27  30
  4   8  12  16  20  24  28  32  36  40
  5  10  15  20  25  30  35  40  45  50
  6  12  18  24  30  36  42  48  54  60
  7  14  21  28  35  42  49  56  63  70
  8  16  24  32  40  48  56  64  72  80
  9  18  27  36  45  54  63  72  81  90
 10  20  30  40  50  60  70  80  90 100

Write a Python program missing_words.py which given two files as arguments prints, in sorted order, all the words found in file1 but not file2.

You can assume words occur one per line in each file.

Write a Python program source_count.py which prints the number of lines of C source code in the current directory. In other words, this Python program should behave similarly to wc -l *.[ch]. (Note: you are not allowed to use wc or other Unix programs from within the Python script). For example:

./source_count.py
    383 cyclorana.c
    280 cyclorana.h
     15 enum.c
    194 frequency.c
    624 model.c
    293 parse.c
    115 random.c
     51 smooth.c
    132 util.c
     16 util.h
    410 waveform.c
   2513 total

Write a Python program, phone_numbers.py which given the URL of a web page fetches it by running wget and prints any strings that might be phone numbers in the web page.

Assume the digits of phone numbers may be separated by zero or more spaces or hyphens ('-') and can contain between 8 and 15 digits inclusive.

You should print the phone numbers one per line with spaces & hyphens removed.

./phone_numbers.py https://www.unsw.edu.au
20151028
11187777
841430912571345
413200225
61293851000
57195873179

quit

print

substitute

addresses

substitute

multiple commands

-f

input files

whitespace and comments

-i

multiple commands