Tracking system changed
After much testing we have decided to do a change in the way the system should work. Intensive testing revlealed that our first setup with the infrared lights inside the box had a lot of disadvantages.
- It was very hard to get an equal amount of light on the entire plate. It was very hard to get the lamps to sit in the right way. This would render some parts of the plate untrackable.
- Infrared light from above would confuse the system. A normal ceiling lamp creates enough infrared light to make the object we track turn into a shadow instead of a lit up dot.
- There was a safety risk regarding the use of the many infrared lamps. The human eye cannot see the light and so the reflex that makes us close the eye when the light is too strong does not work. This would not be a huge problem as the screen is a diffusion screen and the lamps are at an angle but nevertheless still a safety risk that would be nice to get rid of.
- The system is not very fast as the webcam will have a medium shutter time running at around 20-30 fps or even lower. This is not enough at all for tracking of fast movement.
- The system uses a LOT of power as the diodes can handle a forward current of up to 130mA. They are generally used at 100mA but we use them at 20-40mA, closer to the specs of a normal LED. For a useful solution we would need to make it work with a wall plug.
- The image quality was in general not good enough for effective tracking. Putting this thing in a club with lasers and powerful lights everywhere would most likely render it completely useless.
So, in general our first solution was very unstable at best. We then decided to once again test out a prototype we made a couple of weeks ago but left behind because we wanted to see if we could make a system that would not require the user to wear anything. The principle is basically the same - the camera will track a shiny dot if infrared light. This solution is based on using some kind of controller, a puck, pen, stick, glove, or even something else with a small battery and a single infrared LED in it. This would be placed on the plate and light down through it straight into the camera. We tested this out and it is by far the favorable solution.
- The safety issues are practically gone as long as people don't look straight into the LED which is very unlikely. The LED might even be powerful enough at a level of light that is not hazardous.
- The light is very easy to track because it's much more intense than a reflection.
- We can bring down the shutter time of the camera to the lowest value possible. First of all, this will filter out anything but the light from the LED. This works because the Vision 2000 screen is a diffusion screen. The diffusion is not very powerful when we have the infrared light pointing straight into it at a few millimeters distance as the light is very intense in a small spot. The light is easily visible beneath the plate. Other lights however, such as lamps above the system will not be seen, as the screen will diffuse the light so much that not enough will enter to be registered by the camera. At the lowest shutter time, even a very bright infrared LED will have to be no more than a couple of centimeters above the plate to be registered by the camera at all. There is absolutely no noise at all and we have a small completely white dot on a completely black background - this doesn't get much better for tracking as we hardly need any filters for noise reduction at all.
- The low shutter time has another advantage, namely speed. With the shutter time at the lowest setting, the speed of the camera(fps) go sky high compared to normal shutter times. This is very effective as our application requires as much speed as possible and therefore a nice side effect of the low shutter time.
- A lot less power is required as we can power the pointer from a normal 1.5V battery or even a small wrist watch battery.
- Because the system is almost completely invulnerable to noise, it will do a lot better in tricky environments such as a club with a lot of lights everywhere.
- There is no need to put many lamps inside the box.
- The system will be compatible with all sizes of screens as it basically only requires a single infrared LED, a screen and a camera.
The only downside to this system is that the user has to wear something or use something to point with. We don't consider this a huge disadvantage as this is something we have already discussed. A pointer can be a good idea no matter what as you make sure that only the intended user will be able to control the system. A big problem with the old solution was about interference from other things than just things that would confuse the camera. Imagine the DJ is playing and then suddenly somebody throws a peanut or a glowstick onto the plate. How would it react? If users could ruin the DJ's performance just by throwing a peanut onto the table it would most likely be quite a problem. Therefore a dedicated pointer would not be a bad idea as only the DJ would be in control and nothing else could interfere with the system.
We are now working on the new system and will post some pictures soon.
Building the lamps, meeting with Smilen
Today we had a long meeting with Smilen about the project. We learned a lot from this and got some new ideas for Max/MSP. We hope to get this working as fast as possible and get it to work with the tracking patch so we can show exactly how it is suppose to work.
We also had a long discussion about the electronics we use, namely the infrared light. First of all Smilen wanted to make sure that we were not directing a lot of infrared light directly into the eye of the user. As the eye cannot see infrared light, the eyes will not react to it - even if it is strong enough to damage the eye. This should not be a problem however, for several reasons.
1. The Vision 2000 screen we are using is a diffusion screen. A picture from a projector will be very sharp but all other kinds of light will be scattered in a huge diffusion on the screen. Even though some infrared light may still come through, it will be very far from dangerous in intensity. After all, you can stare straight into a projector located less than a meter below the screen without getting blinded by the light and by testing with the webcam, we can see that infrared light becomes diffused too.
2. The infrared lamps are angled and put on the sides just below the screen so that they do not light up into the user. The user will most likely not be hit by any infrared light at all, except for on the hands.
This is all very important to mention in the report. We also had a discussions about the circuits we have made as they are not using resistors. This is mainly due to the lamps being prototypes to see if they can light up enough to work with the camera. A better construction would be to include a resistor, and maybe be able to plug the circuit into a wall outlet for continuous power, as the circiut would more than likely not be able to handle this at the moment. This, along with drawings of the circuit, is very important to mention in the report too.
Testing the camera
Even before we plugged in the camera for the first time, we had a clear idea about how this whole camera procedure would work. This idea eventually turned out to be wrong due to a misunderstanding of light physics.
Since we, in our project, would have quite strong light projected onto the plate from underneath, we were sure that a finger placed on top of the plate would leave a black mark on the image seen from below. Since our camera were to be placed underneath the box, we expected that it would be fairly easy to track the black blob if we just kept the projected image as light as possible.
Basic Improvements
We replaced the paper and the mirror with our new equipment, and the result were as expected. The image got clearer and the green edge got thinner. Though this tiny adjustment was a major improvement, the image still wasn't as clear as we had hoped, and the green edge was still there. Furthermore it still wasn't possible to fully straighten the paper, since we still used the acrylic plate as table, and the bubbles in the paper corrupted the image.
While these issues were in fact only minor imperfections, the mirror stand was quite a problem. Due to the unstable cardboard structure we would never be able to have the excact same angle when testing, and it was too easy to move the stand by accident.
The problem lead to a childish, but efficient solution, and we build an entirely new stand made out of LEGO. The stand was mounted on a big plate, and a turnbutton gave us the possibility to increase or decrease the angle in tiny steps. This gave us a much more precise setup, and we were able to have the same offset everytime we made tests.
On the same plate we made a slot for the projector, in order to be able to place it in the same place every time.
Even though our prototype still looked like an unfinished model, it had surely been improved, and the fact that we were able to use nearly the same setup for all our tests, gave us a certain advantage.
Due to these new possibilities we went on to test the camera. Of course we allready had done some tests on the camera, but not nearly enough to have a clear idea on how to use it in regards to our project.
The first prototype
The required theory was in in order, and we went on a minor field trip to buy the different components needed to build the first prototype. The shopping list included a transparent plate, some semitransparent paper and a mirror, and Silvan seemed as the obvious provider of these items, so off we went.
The purpose of the plate should be supporting the paper, and make the surface pressable by the user. Therefore we sought a fully transparent plate that would not interfere with the light projected from underneath. Since the budget couldn't support buying a special cut glass plate, we bought an acrylic plate instead. It was equally transparent, but less expensive, so it seemed like a fair deal for testing purposes.
Next on the list were the semitransparent paper, but since Silvan really couldn't offer many useful solutions, we decided to see what a typical convenient store could offer.
In Føtex we found that normal baking paper could be a cheap and easy solution, and since they also had cheap mirrors in store, we bought both items and went home to set up our first, quite limited prototype.
Back in the group room a piece of cardboard and some tape became our first mirror stand, and we placed the acrylic plate across two tables with the baking paper on top. We connected the projector and the camera to a computer, and just like that we had our first working prototype.
At first glance it worked excactly as expected, and the image projected onto the paper looked quite good. At second glance there were some problems though.
First of all there were some minor problems with the mirror and the ”screen”. The mirror stand was quite unstable, and even minor movements would result in shaking of the image, or a total movement of the projection.
The acrylic plate turned out to be either too thin or too weak, which resulted in a bendy surface. This issue messed up the baking paper a little bit, which then resulted in an uneven image.
The paper also had too much structure, so the image was corrupted by slightly visible patterns in the paper.
Beside the issues mentioned above, we had an even more annoying issue with the mirror. Along the top and bottom edges of the projected image there were a greenish edge with a height of approximately 5 mm. We tried correcting the angle between the projector and the mirror, in order to see if that could be the issue. We also tried to play around with the settings in the projector software, but none of these possible solutions solved the issue.
Since nothing seemed to solve our problem, we decided to research the issue, and it turned out simply to be a problem with the type of mirror we were using.
The reflecting part of the mirror was covered by a thick piece of glass, and the image from the projector was therefore reflected twice, once at the glass surface, and once at the reflecting material.
This was of course a problem that we couldn't solve by setup, and we started searching for proper alternatives.
We sought inspiration from a former project, ConDio, and the group behind that project suggested to minimize the angle between projector and mirror in order to get the thinnest green edge possible. They also pointed to first surface mirrors as a possible solution. A little research revealed that a first surface mirror, as the name might imply, reflects at the surface of the mirror, and no ghost reflections would then appear. These types of mirrors are less resistant to scratches than normal mirrors, since there is no glass to protect the surface, so we concluded that a first surface mirror was quality over usage, since we wouldn't be able to clean it as normal.
The ConDio group also told us that they had used architechtonic paper for their project, and that this kind of paper had way less structure than baking paper. The paper should be quite expensive, but worth every penny, so we started considering whether to upgrade our equipment or not.
We decided to try another kind of paper that closely resembled the architechtonic paper, and we also borrowed another type of mirror with a thinner glass plate.
With this new equipment in stock, it was time to move on to the next step of our testing phase.
Introduction
Welcome to the blog about project Iceplate. This is a project located in Denmark at Aalborg University with a group consisting of 6 members. Project Iceplate, which is just a temporary codename, is an idea to make a digital version of a turntable for DJ's. It will work by using a touch screen that is build using digital perception - tracking via a camera. This means that the touch screen will not be an ordinary one but a new kind of touch screen based on a completely different idea than the usual matrix film. The project is meant for DJs but will most likely catch the interest of many other developers as it will be a very versatile screen that can have many uses - especially different uses from a normal touch screen.
In this blog you can read about how things are going, what's working and what's not, new ideas we get, problems we get and problems we solve. It will be a tool for sponsors, supervisors, other interested people and ourselves to keep track of how things are going and to later remember what we did to get to the final result.
