Category Archives: Uni Life

ELROND #2: The Pipeline…

The second post about my final year Electronic Engineering project to identify landmarks using photographs taken from a mobile phone in seconds.

Imagine you are walking around London and you come across a building that you want to know more about. You whip out your mobile phone and open up the ELROND app. By taking a picture of the building you want to know more about, ELROND returns relevant information in just a few seconds. The name and purpose of the facility, weather it is open to the public, the opening times, the phone number of reception, a history, appropriate web links… wouldn’t that be useful? Well that’s just one potential application of my project.

It could equally be used to display information about portraits in a gallery just by holding your smartphone in front of a painting, with relevant information overlayed on the screen in real-time…

Or just as an alternative to GPS to locate you when GPS is not available… such as indoors…

My project, titled Elrond, aims to provide the backbone or infrastructure to enable such apps to be written much quicker. But how does it work? how does that picture of a building turn into information?

The diagram below gives an overview of the process.

ELROND

  1. Once the Android application has extracted the features from the image of the building using a feature detection algorithm (more on that in a later post) the extracted information is packaged into an XML format and transmitted over a data network (3G or Wi-Fi) to a Linux web server.
  2. The web server then parses the XML file into a format it can understand.
  3. Each extracted feature is then compared to all known features by Elrond, hence gathering a shortlist of which buildings this is most likely to be a picture of. Because the number of known features is likely to be in the order of millions, a neat way of searching the set needs to be used, called a KD-Tree.
  4. The searching returns a shortlist of images that most probably match the query image. The items at the top of the shortlist are the most likely, so the top 100 results are looked into in more detail. Elrond will look at all the features in both images, find ones that match, and store their locations. Then a homograph is calculated to see if there is a way to map the features from the query image to the stored image. A homograph is a matrix that describes the best way to map two sets of points to each other. If the homograph can map lots of points between the images, then that image is given a high match score.
  5. After all the homographs have been calculated, the best matching image can be determined; or no match is found, in which case Elrond cannot return any information. Assuming a reasonable match was found, Elrond now knows what the building is! Relevant information about that building is maintained on a database, and so it can be fetched.
  6. The database information is packaged into another XML file and sent to the mobile device.
  7. The mobile device interprets the XML and graphically displays the information on the screen. Voilà!

I know that some people are reading this gormless, but for those that are interested to know more I want to write more posts about how specific parts of the application work and perform. So if you have any thoughts and suggestions let me know.

The next post will be about less techy things…

ELROND #1: Feature Detection in Computer Vision

The first post about my final year Electronic Engineering project. I introduce you to the topic of feature detection and what my project is about.
Feature detection is an important problem in computer vision. Computer Vision is the study of extracting information from the real world and somehow use it as an input to computer programs. A simple example of Computer Vision would be how Gollum was created for the Lord of the Rings films…

The creation of Gollum used motion capture from real world camera shots to digitally create the character.

Feature detection is the process of computer algorithms detecting interesting, or “perceptually interesting” locations of an image. Early feature detection worked by detecting edges within an image, but there were several problems with this approach.
Edge Detection
^ from Wikipedia
One of the biggest uses of feature detection is a process known as Image Registration, where a computer can find similar images and determine that they are actually of the same building/object/face/landscape/tree/etc. And this is the basis for my project.
everyday object recognition
^ originally from http://ils.intel-research.net/ (now removed)
The premise of my project, titled Scalable Landmark Recognition on Mobile Devices, is to allow anyone with a mobile phone to take a photograph of a landmark and be told what it is, where it is and some interesting facts about it. For example, you could take a picture of the Eiffel Tower and phone app would recognise the landmark and return information about when it was built, how tall it is, how much ticket prices are today, opening times etc.

^ from http://mastersofmedia.hum.uva.nl/2011/10/10/app-review-google-goggles/
This will be a downloadable mobile app for Android phones, and it will use feature detection to find similar images in it’s database and return information about the closest match. The app will require internet access to communicate with a server that will run the query and holds the database.
The project was dubbed “ELROND” by Anne, who I lived with in second year, because it’s close to SLROMD (which is what the actual abbreviation would be) and it also has Lord of the Rings references, which I’m all for!
So far I’ve got most of the feature detection working, and it should be quick enough to identify a match from 1000’s of images in under a second… but only time will tell. I’ll explain more about how my current solution works in a later post, but for now here’s a screenshot of it working on my computer…

13 usingfile

  • On the top-right you can see the query image, this would be the image taken on the phone.
  • Below it is the image matched to it from the database.
  • The white lines are lines between features matched between the query and database image (using a process called FLANN matching).
  • The green box shows where the query image would fit onto the database image if you were to stitch them together in a panorama (called a homography).
  • The top-middle window (with the red circles) shows the features found in the query image. The features are found using the SURF algorithm.
  • The left of the image shows the code output, showing the progress of the database search and how the query image matched to the other images in the database.

What’s 2011 All About?

This post was written on Tuesday 4th January

2010 was a fantastic year. And as always I’m going to maintain that live is not about where you go, or things you see, but the people and the experiences you have along the way. Every single one of my top 40 moments of last year was great because of people, and that’s what will make 2011 too. But you can’t just sit inside and wait for people to turn up! Here’s the vague plan in my head of how 2011 is shaping up:

  • January. More working in Switzerland
  • February. Going to KoKo club in London to see Mr Scruff, and Becky comes to London
  • March. Giving up chocolate for lent (well, trying). Grand Prix season starts :)
  • April. Seeing The King Blues at KoKo. Another boat trip?
  • May. Nothing yet
  • June. End of tenancy at my current abode
  • July. Contracted to finish at Sony
  • August. Start of Inter-rail trip
  • September. End of inter-rail trip
  • October. Back at university
  • November. Nothing yet
  • December. Christmas (because some things don’t change)

I’ll talk about the inter-rail trip in another post, but that’s the chorus in a nutshell. I’ve never meant what I’m about to say with such passion: “BRING IT ON!”

  • Happiness. 9 out of 10
  • Tiredness. 0 out of 5
  • Free Time. 3 out of 5
  • Last Meal. Chicken Tikka Masala from the pub
  • Song of the day. Ellie Goulding – Your Song 
  • Thought for the day. 2010 was good, but 2011 will be better
  • What I’m Doing Now. Looking at the amazing curtains now in my room to block out the noise from the living room next door. Thank you dad!

100% Of The Time, I Get 75.2% All The Time

I haven’t really figured out what I’m doing with this blog while I’m on placement. It’s not like I have anything less to say, I just have less time to word it. If you have any ideas for what I can do while I’m on placement, then I’m all ears.

First thing to talk about is those results. I got them a couple weeks ago now, and my estimate of 76% wasn’t far off. I got 75.2% :)

That’s another first. My highest mark was 87% for programming. That’s not really a surprise when you factor in the 98% for the coursework. My lowest mark was Electronics 6, only 59%. I said on here that would be the hardest exam, and it definitely was. I’m really happy with my 77% in Computer Vision, that’s the one where I played with teapots. A little confused about how I only got 69% in labs, because the media project went really well…

Ah well, a first is a first. Although I do have to say I’m a little disappointed that I didn’t get over 80%, but if you look at the year as a whole, with MAD TV and Sony, I think I can live with my decisions.

There’s so much more to talk about, we’re a good 2 weeks behind. On Friday 19th Frankie and I went to Grad Ball with the MAD TV camera to film the occasion. It was a brilliant night and makes me look forward to graduating! Not long now :S

37613_559070621198_36701888_33347101_2502813_n

Frankie and I having a great evening gate crashing grad ball and eating lots of strawberries and chocolate :P

You may have noticed that I’ve changed my hair slightly, it more… ‘up’ now. I was getting bored of having short hair and I thought placement was a good opportunity to try something new. So far I’m loving it, can’t believe I never tried this before!

We’ll leave it there for today, but we’ve still got lots to include such as the bank I love to hate, The King Blues live (again) and then everything I did last week!

Phew! I would just do a bullet point list, but some recent events we’re too good for that.

Try and see you tomorrow!

  • Happiness. 10 out of 10
  • Tiredness. 3 out of 5
  • Free Time. 1 out of 5
  • Last Meal. Bacon sandwiches (roast tomorrow!)
  • Song of the day. John Mayer – Gravity
  • Thought for the day. At the end of the day, all you have is tomorrow morning
  • What I’m Doing Now. Being very happy :o)

PS pictures from this weekend :)

PPS I also have some very exciting news about work coming up…

No Curtains

The last week has flown by. Revision from dawn to dusk and beyond. And so far it looks like it’s paid off…

The last three exams have gone really well. I’ve left each one with the feeling that I’d revised the right thing and I did as well as I could.

Especially the Maths exam today since I was so worried last night that it was going to be a disaster. As it turned out it was probably the best exam yet! All the right questions came up, even some questions I’d looked at last night.

So only one exam left. On Friday. That gives me 3 days solid revision to prepare. I’m not very confident about this exam at the moment, but I’m hoping a lot can change in 72 hours.

Then, after Fridays exam I have a lot of partying to catch up on. The university has an annual lake party at the end of the year, that literally thousands of people go to, last year was a blast and this year promises to be even better. We’ve decided to go together as a whole house since we barely spend time together outside these 4 walls together.

As the end of this year gets ever nearer, as a house we realised we didn’t have a group picture. On Saturday we spent an hour in the garden rectifying this fact. I’ve got lots of brilliant photos, but at the moment this is our favourite:

image 

I also really like this one:

DSCF3121

  • Happiness. 7.5 out of 10
  • Tiredness. 3 out of 5 (thanks to Viv, I don’t have any curtains)
  • Workload. 8 out of 10
  • Last Meal. Anne made a cheesecake. You should’ve had some.
  • Song of the day. Free Peace – Shook Up
  • Thought for the day. “The tinsel’s on the table…”
  • What I’m Doing Now. Hoping for a good nights sleep since this happened:

image

Press The Issue!

My first exam on Monday went OK. The most important thing is that it’s over. I think it was definitely the hardest of the three electronics exams I have to sit this week. The second is this afternoon. I’m far more confident about this one, but there are a few questions that would cripple me, so here’s hoping they don’t show up :S

Revision has been going really well and the next two days of exams are going to be intense, but at least I feel as prepared as I could have been.

As a form of procrastination of the weekend, I created a new wallpaper for my computer. It’s a mosaic of over 500 of my pictures from my time since I’ve been at university. It’s amazing how many good times there are! A picture that I took a couple of days ago is definitely going to hold a lot of memories of this year:

DSCF3067

Tickling, Uno, Anne’s cakes, Anne’s frame of shame (more on that another time), Knocking on the wall for Josh, Anne and her OCD… and so many more :) It’s not even sunk in that we’ll be leaving this place in just 16 days…

  • Happiness. 7 out of 10
  • Tiredness. 2 out of 5
  • Workload. 8 out of 10
  • Last Meal. Anne cooked Josh and I bolognaise :)
  • Song of the day. FM Static – Tonight
  • Thought for the day. “Lets press the issue!”
  • What I’m Doing Now. Revision, then an exam, then revision, then sleep, then an exam, then revision…

The next 2 weeks

Hi all! I’m a big fan of not looking back. The past is full of memories and lessons written in stone, but the future has all those moments begging to be memories and the mistakes waiting to be lessons.

As such I’m taking a look forward at the next two weeks. Unfortunately, I don’t anticipate many memories being made, but you never know…

  • Monday. 1st exam. Semiconductors, EM Theory, Analogue Electronics and Machines & Power. One I am not looking forward too.
  • Wednesday. 2nd exam. Programming, CMOS and MIPS. Could be worse.
  • Thursday. 3rd exam. Communications, Circuit Theory, Electronics and Control System. I’m getting there with this stuff…
  • Monday. 4th exam. Maths!
  • Friday. Final exam! Computer vision and graphics… wish me luck for this one :S

At 11:30am on Friday I become a free man! Then I have two whole weeks before I start my placement! I’m going to see my sister for a few days, and I’m staying in Guildford for party times too :D

Interesting things are afoot for summer… Life: The fun never stops, but it can pause.

  • Happiness. 8 out of 10
  • Tiredness. 4 out of 5
  • Workload. 8.9 out of 10
  • Last Meal. Pie and chips
  • Song of the day. The King Blues – Headbutt (again)
  • Thought for the day. “What other names are there for cows other than Daisy and Buttercup?” (Josh’s forever insightful revision ponderings…)
  • What I’m Doing Now. Prepping for revision weekend bonanza!