It looks like it is becoming a tradition – another vacation ends with a guitar video. This time – a short slow blues in Stefan Grossman’s style. The usual apologies for the sound, video and playing quality apply.

2. Professional

I was accepted as a student member to the Chartered Institute for IT (former British Computer Society). For the next two years (at least) I should be reachable through this e-mail: manfredas.zabarauskas@bcs.org.

 
3. Studies

Without getting into the gory details, the outcomes of the Lent (read: second) term in Cambridge can be summarized by:

Conway's Game of Life

Mandelbrot set (1280x1024)

1. a wallpaper “spit out” by ML depicting a rather standard Mandelbrot set fractal (on the right),
2. an animation “spit out” by Java depicting eight-hundred generations of the spacefiller pattern in the Conway’s Game of Life (further on the right),
3. and an insane amount of material to prepare for the upcoming exams; both broad and not as shallow as I expected.

I am still extremely enjoying it, even though the amount of my spare-time has decreased. (As an afterthought – I still had time to play basketball for Blues, Lions and my college, so maybe it was not that bad).

 
4. Internship

Over the summer I will be working as a Technology Analyst for Morgan Stanley, in their Innovative Data, Environments, Analytics & Systems (IDEAS) group.

Sunrise over Canary Wharf

During the interviews Morgan Stanley really left a very good impression both at the level of knowledge of the people working there and the communication and culture inside the company. We will see if that applies in day-to-day situations. In any case, I have nothing against spending the summer in Canary Wharf and seeing the investment banking industry from inside.

It will surely give me more things to write about in this blog, so be sure to check back once in a while.

Competition: internship offer packs from Morgan Stanley and Citigroup.

Description

In 1970s John Horton Conway (British mathematician and University of Cambridge graduate) opened a whole new field of mathematical research by publishing a revolutionary paper on the cellular automaton called the Game of Life. Suffice it to say that the game which he has described with four simple rules has the power of a universal Turing machine, i.e. anything that can be computed algorithmically can be computed within Conway’s Game of Life (outlines of a proof for given by Berlekamp et al; implemented by Chapman as a universal register machine within the Game of Life in 2002).

Glider in the Game of Life

1. Any live cell with fewer than two live neighbours dies, as if caused by underpopulation.
2. Any live cell with more than three live neighbours dies, as if by overcrowding.
3. Any live cell with two or three live neighbours lives on to the next generation.
4. Any dead cell with exactly three live neighbours becomes a live cell.

For more information, patterns and current news about the research involving Game of Life check out the brilliant LifeWiki at conwaylife.com.
 

Implementation

The following applet visualising the Game of Life has been developed as part of the coursework for Object-Oriented Programming at the University of Cambridge, all code was written and compiled in Sun’s Java SE 1.6.

Click on any of the screenshots or the button below to launch the Game of Life (and if nothing shows up, make sure that you have the Java Runtime Environment (JRE) installed).

Spacefiller (Game of Life applet)

Traffic circle (Game of Life applet)

Pattern editor (Game of Life applet)

References

1. Berlekamp, E. R.; Conway, J. H.; and Guy, R. K. “What Is Life?” Ch. 25 in Winning Ways for Your Mathematical Plays, Vol. 2: Games in Particular. London: Academic Press, 1982.

Gel electrophoresis of Senecio vulgaris, S. squalidus and S. cambrensis DNA strands.

Post-vacation status update: apparently I did have to discuss plant evolution through polyploidy in my exam! What were the odds…

Anyway, here’s the video. Oh, yes, and while we’re at it – happy New Year to you all.

"Well done! - joint shortest traditional (and practical :-) sort for the OS1a prize tick". Cambridge, 2009.

# Copyright Manfredas Zabarauskas, 2009.
# MIPS routine that reads an array of ten integers
# and prints the sorted array to console.
.text
main:   sub $t7, $sp, 40
l_read: li $v0, 5
        syscall
        sw $v0, 0($t7)
        add $t7, $t7, 4
        bne $t7, $sp, l_read
l_out:  sub $t8, $sp, 36
        sub $t7, 40
l_inn:  add $t8, $t8, 4
        lw $t2, -8($t8)
        lw $t3, -4($t8)
        ble $t2, $t3, no_swp
        sw $t2, -4($t8)
        sw $t3, -8($t8)
        move $t7, $sp
no_swp: bne $t8, $sp, l_inn
        beq $t7, $sp, l_out
l_prnt: li $v0, 11
        li $a0, 10
        syscall
        li $v0, 1
        lw $a0, 0($t7)
        syscall
        add $t7, $t7, 4
        bne $t7, $sp, l_prnt
        li $v0, 10
        syscall

Introduction

The idea behind eigenfaces is similar (to a certain extent) to the one behind the periodic signal representation as a sum of simple oscillating functions in a Fourier decomposition. The technique described in this tutorial, as well as in the original papers, also aims to represent a face as a linear composition of the base images (called the eigenfaces).

The recognition/detection process consists of initialization, during which the eigenface basis is established and face classification, during which a new image is projected onto the “face space” and the resulting image is categorized by the weight patterns as a known-face, an unknown-face or a non-face image.

Demonstration

To download the software shown in video for 32-bit x86 platform, click here. It was compiled using Microsoft Visual C++ 2008 and uses GSL for Windows.

Establishing the Eigenface Basis

First of all, we have to obtain a training set of grayscale face images . They should be:

1. face-wise aligned, with eyes in the same level and faces of the same scale,
2. normalized so that every pixel has a value between 0 and 255 (i.e. one byte per pixel encoding), and
3. of the same size.

So just capturing everything formally, we want to obtain a set:

, where and

Once we have that, we should change the representation of a face image from a matrix, to a point in -dimensional space. Now here is how we do it: we concatenate all the rows of the matrix into one big vector of dimension . Can it get any more simpler than that?

This is how it looks formally:

, where and

Since we are much more interested in the characteristic features of those faces, let’s subtract everything what is common between them, i.e. the average face.
The average face of the previous mean-adjusted images can be defined as , then each face differs from the average by the vector .

Now we should attempt to find a set of orthonormal vectors which best describe the distribution of our data. The necessary steps in this at a first glance daunting task would seem to be:

1. Obtain a covariance matrix
   , where .
2. Find the eigenvectors and eigenvalues of .

However, note two things here: is of the size and hence the matrix is of the size . To put things into perspective – if your image size is , then the size of the matrix would be . Determining eigenvectors and eigenvalues for a matrix this size would be an absolutely intractable task!

So how do we go about it? A simple mathematical trick: first let’s calculate the inner product matrix , of the size . Then let’s find it’s eigenvectors of (of the -th dimension). Now observe, that if , then

,

and hence and are respectively the eigenvectors (of -th dimension) and eigenvalues of . Make sure to normalize , such that .

We will call these eigenvectors the eigenfaces. Scale them to 255 and render on the screen, to see why.

It turns out that quite a few eigenfaces with the smallest eigenvalues can be discarded, so leave only the ones with the largest eigenvalues (i.e. only the ones making the greatest contribution to the variance of the original image set) and chuck them into the matrix

After you have done that – congratulations! We won’t need anything else, but the matrix for the next steps – face detection and classification.

Face Classification Using Eigenfaces

Once the eigenfaces are created, a new face image can be transformed into it’s eigenface components by a simple operation:

.

The weights describe the contribution of each eigenface in representing the input face image. We can use this vector for face recognition by finding the smallest Euclidean distance between the input face and training faces weight vectors, i.e. by calculating . If , where is a treshold chosen heuristically, then we can say that the input image is recognized as the image with which it gives the lowest score.

The weights vector can also be used for an unknown face detection, exploiting the fact that the images of faces do not change radically when projected into the face space, while the projection of non-face images appear quite different. To do so, we can calculate the distance from the mean-adjusted input image and its projection onto face space , i.e. . Again, if for some treshold (also obtained heuristically, for example, by observing for an input set consisting only of face images and a set of non-face images) we can conclude that the input image is a face.

References

1. Face Recognition Using Eigenfaces, Matthew A. Turk and Alex P. Pentland, MIT Vision and Modeling Lab, CVPR ‘91.
2. Eigenfaces for Recognition, Matthew A. Turk and Alex P. Pentland, Journal of Cognitive Neuroscience ‘91.
3. Eigenfaces. Sheng Zhang and Matthew Turk (2008), Scholarpedia, 3(9):4244.

A status update: apparently I have been awarded the Simon Gray prize by the Department of Philosophy at the University of Edinburgh. That, a KAL industrial scholarship, a couple of other prizes… I almost regret having to leave.

Anyway, here’s the video. Try to figure out which is worse: my playing or my webcam’s video quality.

Labs @ Edinburgh University

As I have promised earlier, here’s the second part of my impressions about the individual courses in the first year of Computer Science and Artificial Intelligence at the University of Edinburgh. Hopefully you will find it useful, e.g. when choosing your outside courses, preparing for the exams or simply getting a taste of the CS undergraduate life in the University of Edinburgh.

Once again, please bear in mind that all what is written below represents only my personal opinion.

Innovation and Enterprise for Scientists and Engineers
Professor: Khaled Benkrid

Lectures: a brilliant introduction to the main components and aspects of modern entrepreneurship, definitely one of the most interesting business courses that I’ve been to so far. Not only will you get the basic theoretic concepts of the modern business (like industry, marketing, operational, financial analyses, risk assessment), but you’ll also put them into practice while writing a business plan for your own idea, which will then account for 50% of your final grade.

What’s more, you’ll get the guest talks of local entrepreneurs (either spin-offs from the university, or related in some other way), an overview of the legal system regarding UK start-ups and businesses (PLCs, sole-traders, etc) and on top of that – it will all be related to the science and engineering (with Dr. Benkrid being a computer scientist himself!).
All in all – a highly inspiring, exciting and recommended course!

Exam: well… it all can’t be that interesting. In the first half of the two hours closed-book exam you’ll have to prove that you’re capable of learning things by heart: you might be asked to list Porter’s five competitive forces, describe the four elements of any business model or give the five most common myths about entrepreneurs.

During the other hour, you’ll be given a much more reasonable task (in my humble opinion) to analyze and discuss a real-life business success/failure story from the various entrepreneurial aspects and answer a couple of more specific questions. To sum it up – even if you’re not very fond of learning things by heart, you’ll be all right if you’ll put some effort into your business plan. It won’t be easy to get a 100, but it surely is possible to still get an A (that’s what I got).

Data and Analysis
Professor: Alex Simpson

Lectures: a fairly broad (and sadly/fortunately not that deep) grounding course on the data collection, representation and interpretation methods across different areas of Computer Science. Starting with the relational databases (E-R diagrams, relational algebra, tuple-relational calculus, SQL), XML (DTDs, XPath) and corpora (properties, annotation, tagging, querying using CQP), and finishing with information retrieval system topics (the vector space model, precision-recall tradeoff) and basic methods of statistical data analysis (Pearson’s correlation coefficient, Chi-square test).

Exams: quite surprisingly Dr. Simpson (being a brilliant mathematician himself, and hence supposedly a rational man) wants you to learn by heart a lot of things that you would typically look-up online. For the exam you will be supposed to know syntaxes of SQL and SQL DDL queries, XML document type definitions, XPath expressions and even the syntax of CQP queries for the CorpusWorkBench.

Besides all that, during the exam you will probably need to design a database from a given scenario (and to draw an E-R diagram, to write SQL DDL statements and a couple of relational/TRC/SQL queries for it as well). You will also have to produce an XML document, together with a couple of XPath queries; do some evaluation for the given IR systems, discuss the necessary properties of corpora, find the correlation between the statistical data and check the hypothesis using the Chi-square test, so…  better do your tutorial exercises during the semester, ask a lot of questions in the lectures/tutorials and go through the past papers against the clock, because the exam is time-wise intense as well. It’s surely worth doing so, since the exam counts for the 100% of your grade (I managed to reach 97).

Object-Oriented Programming
Professor: Ewan Klein

Lectures: not much to say here, actually. If you have done any Java programming before (and since you have applied to study Computer Science at one of the best IT departments in the whole world, chances are that you have) it’s going to be a breeze. Nothing sophisticated (read: interesting) here: methods, objects, inheritance, polymorphism, interfaces and abstract classes.

And I wouldn’t bother spending too much time doing the lab assignments (especially the optional ones): most likely you won’t be able to submit them to Web-CAT (the automatic grading system), or the Web-CAT won’t be able to compile, run or grade them. Oh, and they count for nothing to the final grade anyway. To be perfectly honest, they’re as useless, as the tutorials during which you should discuss your lab assignments, which a) you have done, and since they’re straightforward – there’s nothing more to discuss, or b) you haven’t done, and i) you have no clue what’s going on or ii) you’d like to learn how to do them, but what you haven’t realized is that learning to program by talking about it is as reasonable as learning to swim by reading the manual.

Exam: two 20-minute tasks for which you will have two hours, one involving implementing a couple of simple methods with the arrays and the second involving implementing a list of classes from the given UML featuring some simple inheritance and polymorphism.

Maybe you shouldn’t read this, since I have had a reasonable amount of experience with Java before starting the course, but… if you have some experience with Java as well – most likely you won’t have to spend a single minute preparing for the exam, you’ll leave after the first half an hour and you’ll get a 100. At least, that’s what I did. And if you don’t have any experience, you’ll do fine as well.

My only advice to you then is – forget the lab assignments. Come up with the project that you always wanted to do – a simple adventure game or a text-based LAN chat application – and implement that in Java. You’ll get much more pleasure and benefit from it (and not only for the exam).

Mathematics for Informatics 2
Professors: Martin Dindos, Antony Maciocia

Lectures: compared to the first semester the maths will get more interesting (especially the calculus). You’ll delve into growth rates, complex numbers, power series, partial differentiation and first order differential equations; graph theory, recurrent relations, geometry in R² and R³, and end-up with matrices and Gaussian elimination.

However, due to the nature of the course itself, it’s broad, but not that deep. The lecturers will do a great job making all the material easy to grasp as well, but it surely doesn’t mean that you won’t have to make an effort to understand it, quite the opposite (it’s still mathematics, at the end of the day). The key thing to remember here (and I cannot overstress it) is DO the hand-in assignments! Not only will it help you to stay on top of the material, to formulate and ask the right questions during lectures and tutorials, but it will also account for the 15% of your final grade.

Exams: a two and a half hours run through all the material. Just to scare you off a bit – in year ‘07/’08, 53% percent of all the students taking the course have failed this exam (after the re-sits).

Now, it was a lot easier when I took it (in ‘08/’09) – I don’t know the statistics, but I managed to get a 100, and I’m not brilliant at maths, so… draw your conclusions.

Anyway, there seem to be no guaranteed shortcuts in this course. The exams tend to be comprised mainly of the same type of exercises that you have done in your tutorials/hand-ins, and they’re focused more on the quantity than on the quality, so, if you managed to do those exercises – you should be on the right path.

Wolfson College

I’m already looking forward to it. To the legendary supervision sessions, to the world class experts in the field, to the competitiveness amongst the students. I’m sure I will finally arrive at the place where I’ll be the average guy between the best one’s… not the other way around.

(I’m not convinced that the latter applies to the basketball team, though. Well, I guess we’ll find that out pretty soon).

Confirmation Letter. Cambridge University, Wolfson College, 2009.

1 Night in Haskell

Since the exam results came back and the first year is finally over, I have decided to share my impressions about the individual lectures in the first two semesters. Hopefully it will help someone to choose one subject over another, not to make the same mistakes when preparing for the exams that I did, or simply to get the taste of the first year in college. Please bear in mind that all what is written below represents only my personal, highly subjective opinion.

Functional Programming
Professor: Philip Wadler

Lectures: a wonderful introduction to the functional programming. Amazing teaching skills and professor’s authority in the classroom – from ripping a T-shirt to reveal a big superman-lambda, to being one of the principal engineers and designers of the Haskell programming language – 10 out of 10.

Exam: Final grade is made by a programming class test (10%) and a final exam (90%). The programming class test is pretty straightforward – three simple tasks to test one’s basic knowledge of Haskell (things you need to know are the syntax, because it’s done on paper, list comprehension, recursion and simple library functions). The final exam is done in front of computer with plenty of time (2 hours) for three simple tasks (e.g. take a list of integers and return the sum of the cubes of the positive numbers in that list). The only catch is that the same problem has to be solved in a couple of different ways: using basic functions, using list comprehension, using recursion and using the higher-order functions, however… at the end of the day – it’s still the same problem.

My advice in succeding in this subject – participate in the class’ programming competition (see this years competition and my entry). If you’ll manage to get something done – you won’t need to spend any more time to prepare for the final exam, and I can hardly imagine how you’d get less than an A (my personal result – 99).

Computation and Logic
Professor: Dave Robertson

Lectures: a basic introduction to reasoning – propositional logic, concepts of satisfiability, validity and proof, regular expressions, finite state transducers/acceptors, HMMs and probabilistic finite state machines. However, the professor seems to be a little distracted by his other responsibilities (being the Head of School of Informatics) so the lectures aren’t as involving as they could be. Be careful about the tutorial exercises as well – many of them contain mistakes, and some of the TA’s (master and PhD students) seem to have already forgotten the material quite a bit.

Exam: a 2-hour rush through all of material – I remember writing eight A4 pages (that leaves 15 mins per page) on truth tables – inconsistencies, contingencies, tautologies, proving that formulas are equivalent, proof trees (including the negation rules), resolution – normal and clausal forms, transducer/acceptor FSMs, and their equivalence with regular expressions (Kleene’s Theorem); HMMs and probabilistic FSMs.

I’m afraid that there’s no easy way here – you just have to work your way through the bunch of tutorial exercises and past papers (I managed to drudge away to 89). The good thing is that exams don’t change too much and you will know what’s expected from you.

Mathematics for Informatics 1
Professors: John Byatt-Smith, Antony Maciocia

Lectures: a semester-long review of a high school material trying to get everyone up to the same speed. It includes revision of real functions, integers, rationals, real numbers, number bases; differentiation: limits, tangent to a curve, stationary points, inverse functions; exponentials, logs, integral definition, integration methods and the Fundamental Theorem of Calculus; general algebra background: sets, functions, relations, mathematical induction; number theory: GCD, the extended Euclid’s algorithm, residues and equations, RSA; counting: permutations, binomial theorem, Pascal’s triangle; basic probability: discrete spaces, random variables, expectation, variance, Bernoulli trials, binomial and geometric distributions; entropy, Shannon’s source coding theorem and Huffman coding.

It may sound like a lot, but it’s more to the breadth than to the depth, really. Plus the lecturers will make all this material quite easy to grasp – just make sure to do your hand-in assignments (15% of the final grade) and do your fair share of the work in the tutorials (that includes asking the questions).

Exam: two and a half hours of the hand-in assignment problems. If you’ve done those and went through a couple of the past years exams (you can find most of them here) you won’t have any trouble getting an A (my result – 96).

Logic 1
Professor: Paul Schweizer

Lectures: with deep regret – incredibly boring. Professor Paul Schweizer doesn’t seem to be bothered too much about anything happening in the audience – noises, non-working microphones, or half of the students playing online poker. This is partially caused by Logic 1 being a compulsory course for many School of Philosophy, Psychology and Language Sciences students – and being way over their heads.

It’s extremely dissapointing because it could become a very good grounding course in logic (if anyone would pay at least some attention) – starting with the propositional logic and basic concepts, moving on to sematic tableaux, equivalence and adequacy, predicate logic, binary and identity relations and finally touching some of the theories of thruth and non-classical logic.

Exam: another two hour rush through all the material – make sure to do as many of the previous year exams against the clock as you can!
I’m afraid that there are no shortcuts here – you’ll have to work a lot on your own if you haven’t done any logic courses before (my result in the exam – 98. Yes, I have done logic courses before). Do as many exercises as you can find, ask your tutor the right questions and the golden rule – don’t leave anything unclear: look it up on the internet, ask your friends or the professor after the lecture. If you’ll attend the lecture. You probably won’t.