Gel electrophoresis of Senecio vulgaris, S. squalidus and S. cambrensis DNA strands.

Post-vacation status update: apparently I did have to discuss plant evolution through polyploidy in my exam! What were the odds…

Anyway, here’s the video. Oh, yes, and while we’re at it – happy New Year to you all.

"Well done! - joint shortest traditional (and practical :-) sort for the OS1a prize tick". Cambridge, 2009.

# Copyright Manfredas Zabarauskas, 2009.
# MIPS routine that reads an array of ten integers
# and prints the sorted array to console.
.text
main:   sub $t7, $sp, 40
l_read: li $v0, 5
        syscall
        sw $v0, 0($t7)
        add $t7, $t7, 4
        bne $t7, $sp, l_read
l_out:  sub $t8, $sp, 36
        sub $t7, 40
l_inn:  add $t8, $t8, 4
        lw $t2, -8($t8)
        lw $t3, -4($t8)
        ble $t2, $t3, no_swp
        sw $t2, -4($t8)
        sw $t3, -8($t8)
        move $t7, $sp
no_swp: bne $t8, $sp, l_inn
        beq $t7, $sp, l_out
l_prnt: li $v0, 11
        li $a0, 10
        syscall
        li $v0, 1
        lw $a0, 0($t7)
        syscall
        add $t7, $t7, 4
        bne $t7, $sp, l_prnt
        li $v0, 10
        syscall

Introduction

The idea behind eigenfaces is similar (to a certain extent) to the one behind the periodic signal representation as a sum of simple oscillating functions in a Fourier decomposition. The technique described in this tutorial, as well as in the original papers, also aims to represent a face as a linear composition of the base images (called the eigenfaces).

The recognition/detection process consists of initialization, during which the eigenface basis is established and face classification, during which a new image is projected onto the “face space” and the resulting image is categorized by the weight patterns as a known-face, an unknown-face or a non-face image.

Demonstration

To download the software shown in video for 32-bit x86 platform, click here. It was compiled using Microsoft Visual C++ 2008 and uses GSL for Windows.

Establishing the Eigenface Basis

First of all, we have to obtain a training set of grayscale face images . They should be:

1. face-wise aligned, with eyes in the same level and faces of the same scale,
2. normalized so that every pixel has a value between 0 and 255 (i.e. one byte per pixel encoding), and
3. of the same size.

So just capturing everything formally, we want to obtain a set:

, where and

Once we have that, we should change the representation of a face image from a matrix, to a point in -dimensional space. Now here is how we do it: we concatenate all the rows of the matrix into one big vector of dimension . Can it get any more simpler than that?

This is how it looks formally:

, where and

Since we are much more interested in the characteristic features of those faces, let’s subtract everything what is common between them, i.e. the average face.
The average face of the previous mean-adjusted images can be defined as , then each face differs from the average by the vector .

Now we should attempt to find a set of orthonormal vectors which best describe the distribution of our data. The necessary steps in this at a first glance daunting task would seem to be:

1. Obtain a covariance matrix
   , where .
2. Find the eigenvectors and eigenvalues of .

However, note two things here: is of the size and hence the matrix is of the size . To put things into perspective – if your image size is , then the size of the matrix would be . Determining eigenvectors and eigenvalues for a matrix this size would be an absolutely intractable task!

So how do we go about it? A simple mathematical trick: first let’s calculate the inner product matrix , of the size . Then let’s find it’s eigenvectors of (of the -th dimension). Now observe, that if , then

,

and hence and are respectively the eigenvectors (of -th dimension) and eigenvalues of . Make sure to normalize , such that .

We will call these eigenvectors the eigenfaces. Scale them to 255 and render on the screen, to see why.

It turns out that quite a few eigenfaces with the smallest eigenvalues can be discarded, so leave only the ones with the largest eigenvalues (i.e. only the ones making the greatest contribution to the variance of the original image set) and chuck them into the matrix

After you have done that – congratulations! We won’t need anything else, but the matrix for the next steps – face detection and classification.

Face Classification Using Eigenfaces

Once the eigenfaces are created, a new face image can be transformed into it’s eigenface components by a simple operation:

.

The weights describe the contribution of each eigenface in representing the input face image. We can use this vector for face recognition by finding the smallest Euclidean distance between the input face and training faces weight vectors, i.e. by calculating . If , where is a treshold chosen heuristically, then we can say that the input image is recognized as the image with which it gives the lowest score.

The weights vector can also be used for an unknown face detection, exploiting the fact that the images of faces do not change radically when projected into the face space, while the projection of non-face images appear quite different. To do so, we can calculate the distance from the mean-adjusted input image and its projection onto face space , i.e. . Again, if for some treshold (also obtained heuristically, for example, by observing for an input set consisting only of face images and a set of non-face images) we can conclude that the input image is a face.

References

1. Face Recognition Using Eigenfaces, Matthew A. Turk and Alex P. Pentland, MIT Vision and Modeling Lab, CVPR ‘91.
2. Eigenfaces for Recognition, Matthew A. Turk and Alex P. Pentland, Journal of Cognitive Neuroscience ‘91.
3. Eigenfaces. Sheng Zhang and Matthew Turk (2008), Scholarpedia, 3(9):4244.

A status update: apparently I have been awarded the Simon Gray prize by the Department of Philosophy at the University of Edinburgh. That, a KAL industrial scholarship, a couple of other prizes… I almost regret having to leave.

Anyway, here’s the video. Try to figure out which is worse: my playing or my webcam’s video quality.

Labs @ Edinburgh University

As I have promised earlier, here’s the second part of my impressions about the individual courses in the first year of Computer Science and Artificial Intelligence at the University of Edinburgh. Hopefully you will find it useful, e.g. when choosing your outside courses, preparing for the exams or simply getting a taste of the CS undergraduate life in the University of Edinburgh.

Once again, please bear in mind that all what is written below represents only my personal opinion.

Innovation and Enterprise for Scientists and Engineers
Professor: Khaled Benkrid

Lectures: a brilliant introduction to the main components and aspects of modern entrepreneurship, definitely one of the most interesting business courses that I’ve been to so far. Not only will you get the basic theoretic concepts of the modern business (like industry, marketing, operational, financial analyses, risk assessment), but you’ll also put them into practice while writing a business plan for your own idea, which will then account for 50% of your final grade.

What’s more, you’ll get the guest talks of local entrepreneurs (either spin-offs from the university, or related in some other way), an overview of the legal system regarding UK start-ups and businesses (PLCs, sole-traders, etc) and on top of that – it will all be related to the science and engineering (with Dr. Benkrid being a computer scientist himself!).
All in all – a highly inspiring, exciting and recommended course!

Exam: well… it all can’t be that interesting. In the first half of the two hours closed-book exam you’ll have to prove that you’re capable of learning things by heart: you might be asked to list Porter’s five competitive forces, describe the four elements of any business model or give the five most common myths about entrepreneurs.

During the other hour, you’ll be given a much more reasonable task (in my humble opinion) to analyze and discuss a real-life business success/failure story from the various entrepreneurial aspects and answer a couple of more specific questions. To sum it up – even if you’re not very fond of learning things by heart, you’ll be all right if you’ll put some effort into your business plan. It won’t be easy to get a 100, but it surely is possible to still get an A (that’s what I got).

Data and Analysis
Professor: Alex Simpson

Lectures: a fairly broad (and sadly/fortunately not that deep) grounding course on the data collection, representation and interpretation methods across different areas of Computer Science. Starting with the relational databases (E-R diagrams, relational algebra, tuple-relational calculus, SQL), XML (DTDs, XPath) and corpora (properties, annotation, tagging, querying using CQP), and finishing with information retrieval system topics (the vector space model, precision-recall tradeoff) and basic methods of statistical data analysis (Pearson’s correlation coefficient, Chi-square test).

Exams: quite surprisingly Dr. Simpson (being a brilliant mathematician himself, and hence supposedly a rational man) wants you to learn by heart a lot of things that you would typically look-up online. For the exam you will be supposed to know syntaxes of SQL and SQL DDL queries, XML document type definitions, XPath expressions and even the syntax of CQP queries for the CorpusWorkBench.

Besides all that, during the exam you will probably need to design a database from a given scenario (and to draw an E-R diagram, to write SQL DDL statements and a couple of relational/TRC/SQL queries for it as well). You will also have to produce an XML document, together with a couple of XPath queries; do some evaluation for the given IR systems, discuss the necessary properties of corpora, find the correlation between the statistical data and check the hypothesis using the Chi-square test, so…  better do your tutorial exercises during the semester, ask a lot of questions in the lectures/tutorials and go through the past papers against the clock, because the exam is time-wise intense as well. It’s surely worth doing so, since the exam counts for the 100% of your grade (I managed to reach 97).

Object-Oriented Programming
Professor: Ewan Klein

Lectures: not much to say here, actually. If you have done any Java programming before (and since you have applied to study Computer Science at one of the best IT departments in the whole world, chances are that you have) it’s going to be a breeze. Nothing sophisticated (read: interesting) here: methods, objects, inheritance, polymorphism, interfaces and abstract classes.

And I wouldn’t bother spending too much time doing the lab assignments (especially the optional ones): most likely you won’t be able to submit them to Web-CAT (the automatic grading system), or the Web-CAT won’t be able to compile, run or grade them. Oh, and they count for nothing to the final grade anyway. To be perfectly honest, they’re as useless, as the tutorials during which you should discuss your lab assignments, which a) you have done, and since they’re straightforward – there’s nothing more to discuss, or b) you haven’t done, and i) you have no clue what’s going on or ii) you’d like to learn how to do them, but what you haven’t realized is that learning to program by talking about it is as reasonable as learning to swim by reading the manual.

Exam: two 20-minute tasks for which you will have two hours, one involving implementing a couple of simple methods with the arrays and the second involving implementing a list of classes from the given UML featuring some simple inheritance and polymorphism.

Maybe you shouldn’t read this, since I have had a reasonable amount of experience with Java before starting the course, but… if you have some experience with Java as well – most likely you won’t have to spend a single minute preparing for the exam, you’ll leave after the first half an hour and you’ll get a 100. At least, that’s what I did. And if you don’t have any experience, you’ll do fine as well.

My only advice to you then is – forget the lab assignments. Come up with the project that you always wanted to do – a simple adventure game or a text-based LAN chat application – and implement that in Java. You’ll get much more pleasure and benefit from it (and not only for the exam).

Mathematics for Informatics 2
Professors: Martin Dindos, Antony Maciocia

Lectures: compared to the first semester the maths will get more interesting (especially the calculus). You’ll delve into growth rates, complex numbers, power series, partial differentiation and first order differential equations; graph theory, recurrent relations, geometry in R² and R³, and end-up with matrices and Gaussian elimination.

However, due to the nature of the course itself, it’s broad, but not that deep. The lecturers will do a great job making all the material easy to grasp as well, but it surely doesn’t mean that you won’t have to make an effort to understand it, quite the opposite (it’s still mathematics, at the end of the day). The key thing to remember here (and I cannot overstress it) is DO the hand-in assignments! Not only will it help you to stay on top of the material, to formulate and ask the right questions during lectures and tutorials, but it will also account for the 15% of your final grade.

Exams: a two and a half hours run through all the material. Just to scare you off a bit – in year ‘07/’08, 53% percent of all the students taking the course have failed this exam (after the re-sits).

Now, it was a lot easier when I took it (in ‘08/’09) – I don’t know the statistics, but I managed to get a 100, and I’m not brilliant at maths, so… draw your conclusions.

Anyway, there seem to be no guaranteed shortcuts in this course. The exams tend to be comprised mainly of the same type of exercises that you have done in your tutorials/hand-ins, and they’re focused more on the quantity than on the quality, so, if you managed to do those exercises – you should be on the right path.

Wolfson College

I’m already looking forward to it. To the legendary supervision sessions, to the world class experts in the field, to the competitiveness amongst the students. I’m sure I will finally arrive at the place where I’ll be the average guy between the best one’s… not the other way around.

(I’m not convinced that the latter applies to the basketball team, though. Well, I guess we’ll find that out pretty soon).

Confirmation Letter. Cambridge University, Wolfson College, 2009.

1 Night in Haskell

Since the exam results came back and the first year is finally over, I have decided to share my impressions about the individual lectures in the first two semesters. Hopefully it will help someone to choose one subject over another, not to make the same mistakes when preparing for the exams that I did, or simply to get the taste of the first year in college. Please bear in mind that all what is written below represents only my personal, highly subjective opinion.

Functional Programming
Professor: Philip Wadler

Lectures: a wonderful introduction to the functional programming. Amazing teaching skills and professor’s authority in the classroom – from ripping a T-shirt to reveal a big superman-lambda, to being one of the principal engineers and designers of the Haskell programming language – 10 out of 10.

Exam: Final grade is made by a programming class test (10%) and a final exam (90%). The programming class test is pretty straightforward – three simple tasks to test one’s basic knowledge of Haskell (things you need to know are the syntax, because it’s done on paper, list comprehension, recursion and simple library functions). The final exam is done in front of computer with plenty of time (2 hours) for three simple tasks (e.g. take a list of integers and return the sum of the cubes of the positive numbers in that list). The only catch is that the same problem has to be solved in a couple of different ways: using basic functions, using list comprehension, using recursion and using the higher-order functions, however… at the end of the day – it’s still the same problem.

My advice in succeding in this subject – participate in the class’ programming competition (see this years competition and my entry). If you’ll manage to get something done – you won’t need to spend any more time to prepare for the final exam, and I can hardly imagine how you’d get less than an A (my personal result – 99).

Computation and Logic
Professor: Dave Robertson

Lectures: a basic introduction to reasoning – propositional logic, concepts of satisfiability, validity and proof, regular expressions, finite state transducers/acceptors, HMMs and probabilistic finite state machines. However, the professor seems to be a little distracted by his other responsibilities (being the Head of School of Informatics) so the lectures aren’t as involving as they could be. Be careful about the tutorial exercises as well – many of them contain mistakes, and some of the TA’s (master and PhD students) seem to have already forgotten the material quite a bit.

Exam: a 2-hour rush through all of material – I remember writing eight A4 pages (that leaves 15 mins per page) on truth tables – inconsistencies, contingencies, tautologies, proving that formulas are equivalent, proof trees (including the negation rules), resolution – normal and clausal forms, transducer/acceptor FSMs, and their equivalence with regular expressions (Kleene’s Theorem); HMMs and probabilistic FSMs.

I’m afraid that there’s no easy way here – you just have to work your way through the bunch of tutorial exercises and past papers (I managed to drudge away to 89). The good thing is that exams don’t change too much and you will know what’s expected from you.

Mathematics for Informatics 1
Professors: John Byatt-Smith, Antony Maciocia

Lectures: a semester-long review of a high school material trying to get everyone up to the same speed. It includes revision of real functions, integers, rationals, real numbers, number bases; differentiation: limits, tangent to a curve, stationary points, inverse functions; exponentials, logs, integral definition, integration methods and the Fundamental Theorem of Calculus; general algebra background: sets, functions, relations, mathematical induction; number theory: GCD, the extended Euclid’s algorithm, residues and equations, RSA; counting: permutations, binomial theorem, Pascal’s triangle; basic probability: discrete spaces, random variables, expectation, variance, Bernoulli trials, binomial and geometric distributions; entropy, Shannon’s source coding theorem and Huffman coding.

It may sound like a lot, but it’s more to the breadth than to the depth, really. Plus the lecturers will make all this material quite easy to grasp – just make sure to do your hand-in assignments (15% of the final grade) and do your fair share of the work in the tutorials (that includes asking the questions).

Exam: two and a half hours of the hand-in assignment problems. If you’ve done those and went through a couple of the past years exams (you can find most of them here) you won’t have any trouble getting an A (my result – 96).

Logic 1
Professor: Paul Schweizer

Lectures: with deep regret – incredibly boring. Professor Paul Schweizer doesn’t seem to be bothered too much about anything happening in the audience – noises, non-working microphones, or half of the students playing online poker. This is partially caused by Logic 1 being a compulsory course for many School of Philosophy, Psychology and Language Sciences students – and being way over their heads.

It’s extremely dissapointing because it could become a very good grounding course in logic (if anyone would pay at least some attention) – starting with the propositional logic and basic concepts, moving on to sematic tableaux, equivalence and adequacy, predicate logic, binary and identity relations and finally touching some of the theories of thruth and non-classical logic.

Exam: another two hour rush through all the material – make sure to do as many of the previous year exams against the clock as you can!
I’m afraid that there are no shortcuts here – you’ll have to work a lot on your own if you haven’t done any logic courses before (my result in the exam – 98. Yes, I have done logic courses before). Do as many exercises as you can find, ask your tutor the right questions and the golden rule – don’t leave anything unclear: look it up on the internet, ask your friends or the professor after the lecture. If you’ll attend the lecture. You probably won’t.

Debugging Samsung SMRP6400

Since last week we have spent quite some time debugging Samsung SMRP6400/SMDK64X0 IIC drivers, I thought I might share with one particular example here. It is both a good showcase of the hardware/software synchronization issues and since the bugs are in the latest version of the drivers shipped together with the development platforms, it might save someone from additional headaches.

So, while working on the drivers for IIC one of our automated tests to make sure that IIC still works was to write some data on the bus, do an immediate read-back and verify that both data written and read back matches; something similar to this:

UCHAR outData[3] = { REGISTER, DATA_BYTE_1, DATA_BYTE2 };
UCHAR inData[2] = { 0, 0 };

IIC_Write(SLAVE_ADDRESS, outData, 3);
IIC_Read(SLAVE_ADDRESS, REGISTER, inData, 2);

if ((inData[0] != DATA_BYTE_1) || (inData[1] != DATA_BYTE_2))
{
    DEBUGMSG(ERROR_MSG,
             (TEXT("Immediate write/read data mismatch: data sent [0x%X " \
                   "0x%X] differs from the data received [0x%X, 0x%X]."),
                   DATA_BYTE_1, DATA_BYTE_2, inData[0], inData[1]));
}

However, after we added some IIC read calls from another hardware driver it started spitting fire throwing the following error message:

Immediate write/read data mismatch: data sent [0xCA, 0xFE]
differs from the data received [0xCA, 0xFE].

Now this clearly meant we had a serious problem: the error message was saying that the data does not match, which obviously was not the case as shown by the message text!

Since we were pretty confident about our side of things, as well as the readings from the scope, it seemed like a good time to start looking at the Samsung IIC drivers (and especially s3c64X0_iic_lib.cpp). The driver structure there is pretty straightforward: the first read/write byte on the bus triggers the hardware IRQ, which is mapped to SysIntr triggering a transfer event; then any subsequent call to a read/write function blocks waiting for a transfer-done event, which is triggered when the last byte is read/written in the IST.

Everything looks sane up to the point where a transfer-done event is signalled from the IST (pseudocode, not the original code below, due to the legal issues):

static HANDLE ghTransferEvent;
static HANDLE ghTransferDone;
static DWORD IICSysIntr;

...

static DWORD IST(LPVOID lpContext)
{
    BOOL bTransferDone = FALSE;

    while (TRUE) {
        WaitForSingleObject(ghTransferEvent, INFINITE);

        switch (IIC_BUS_STATUS) {
            case MASTER_RECEIVE:
                // receive bytes and store them in the buffer
            break;

            case MASTER_TRANSMIT:
                // transmit bytes from the buffer in memory
                if (LAST_BYTE) bTransferDone = TRUE;
            break;

            InterruptDone(IICSysIntr);

            if (bTransferDone) {
                SetEvent(ghTransferDone);
            }
        }
    }
}

Two major problems with this code are:

1. The bTransferDone variable is never set from the IIC read, and hence the transfer-done event for bus reads is never triggered,
2. After the bTransferDone is set from the IIC write, it is never reset, hence the transfer-done event is triggered after reading/writing single byte on the bus in all subsequent transactions.

That explains the initial test case failure: during the write/immediate read data comparison the data arrives exactly between the if statement and the following printout, thus triggering the error message, but printing the correct data to the output due to the early signal of the event.

The way to solve this is also straightforward: make sure that the bTransferDone variable is cleared after the transfer-done event is triggered, and make sure that master-receive mode sets the bTransferDone variable after reading the last byte of the transaction from the IIC bus.

In pseudocode it would look similar to:

static DWORD IST(LPVOID context)
{
    BOOL bTransferDone = FALSE;

    while (TRUE) {
        WaitForSingleObject(ghTransferEvent, INFINITE);

        switch (IIC_BUS_STATUS) {
            case MASTER_RECEIVE:
                // receive bytes and store them in the buffer
                if (LAST_BYTE) bTransferDone = TRUE;
            break;

            case MASTER_TRANSMIT:
                // transmit bytes from the buffer in memory
                if (LAST_BYTE) bTransferDone = TRUE;
            break;

            InterruptDone(IICSysIntr);

            if (bTransferDone) {
                bTransferDone = FALSE;
                SetEvent(ghTransferDone);
            }
        }
    }
}

The lesson of the day (quoting my colleague) is: “The first rule about multithreading – you’re wrong“.

At work. Wolfson Microelectronics PLC, 2009

"Green Card"

It all comes at once.

A couple of days ago I received a call from Lithuania – there was a letter from U.S. waiting for me at home. After some discussion it became clear that I was one of the 100 000 selected to compete for 55 000 DV’s (”Green Cards”).

After some short initial joy, problems and worries started to creep in: one has to cross the U.S. border in six months after getting a DV – something which I’m not keen doing at all, because I’ll be studying in Cambridge University at that time, and since I’ve worked quite hard to get there, it’s priority #1 in the list.

One possible solution would be to go to U.S. for the summer and then comeback for a school year, but… we’ll see how that goes, it might introduce some additional problems (about which I’m not fully aware at the moment). It would be nice to visit some old friends though…

Anyway, I’ll try to keep the whole process documented here as well – it might give someone a good estimate of trouble involved in getting a DV (and help me to track the overall progress as well).

Thanks for reading and stay tuned – there will be more.

Boston, U.S.A. (09/2007)