Wireless 3D Sensor

This project was done for MP2079 Engineering Innovation and Design module at NTU, and won first prize in the Health,Lifestyle and Assistive Technologies category. It consist of a sensor which detects it's orientation and transmit them to a PC via bluetooth, and a set of software that uses the 3D data to control a game. The purpose of this project is to guide and motivate stroke patients in carrying out exercises.

The module lasted 5 weeks, but with the first week used up in deciding on the project topic, and classes on the second and third week, the entire project was carried out in quite a rush. As the module's documentation requirements focuses on the business plan of the project, the technical documentation were all sacrificed in the interest of expedience. What you see on this page is probably all the documentation there is, and they are all written from memory nearly 4 months after the project ended.

3D Sensor

The 3D sensor is capable of sensing its own orientation in 3D using a combination of accelerometer and magneto-inductive sensor. It also comes with an optional attachment for detection of grip strength. The sensor can be strapped on the user's arm or leg, and can detect all 3 degrees of rotation, and also limited sensing of translational movement. It consist of X main components, which are all mounted on a stripboard (except the bluetooth module which is more dangling off the stripboard than mounted on it).

• Voltage Regulator
  The power is supplied by a 9V PP3 battery within the sensor casing. We use the ubiquitous 7805 linear voltage regulator to reduce it down to 5V, but any 5V voltage regulator would do just fine. A switching voltage regulator would extend battery life, but we wanted to keep the sensor small and could not afford the additional space needed.

• Micro-Controller
  We use a Atmel ATmega8 micro-controller, as it has built in support for SPI (for the magneto-inductive sensor), USART (for the bluetooth module), ADC (for the accelerometer and the pressure sensor), built in oscillator (save space), and can be programmed in C. Even more important would be the fact that I have all the hardware and software necessary for programming the ATmega8, as the university can't be depended upon for any sort of technical or equipment support.

• Accelerometer
  The accelerometer is a DE-ACCM3D 3-axis analog output model from Dimension Engineering. A common LM2904 Op-Amp is used to adjust the output voltage range to 0-2.56V, so that it'll fit neatly within the input range of the micro-controller built-in ADC. The purpose of the accelerometer is to detect the direction of the earth's gravitational acceleration, giving us one vector, sufficient to detect 2 degrees of rotation motion. It is also possible to detect translational motion by double integration of the acceleration vector, but this is not done as our sensor's resolution and sampling rate is too low to provide a useful result. Rapid movement can result in errors, as the accelerometer will detect the sum of the earth gravitational acceleration and the acceleration from the rapid movement. But then again, stroke patients aren't exactly know for rapid movement.

• Magneto-Inductive Sensor
  The magneto-inductive sensor is a MicroMag 3-axis digital output model with SPI interface bought from Sparkfun. It detects the direction of the earth's magnetic field, providing a second vector, roughly 90 degrees from the earth's gravitational acceleration vector. With both the magnetic field vector and the gravity vector, we can detect all 3 degrees of rotational motion.

• Pressure Sensor
  To detect grip strength, we used a rubber dust blower connected via a small hose to a pressure sensor. The pressure connectors were all machined by us from aluminum, as pre-made connectors were either too big or too expensive. The hose is made from heat shrink tubing, as we were unable to find a suitably small and flexible hose. The pressure sensor is a Honeywell 24PC15SMT, with a 0-15psi range. It was selected as it was small, accurate, available in a variety of ranges, but mostly because it was the cheapest pressure sensor on the Farnell website. The sensor is a surface mounted model, but with a bit of pliers work and some solid core wire, we were able to mount it on the stripboard easily.

• Bluetooth Module
  A BlueSMiRF Gold from Sparkfun is used to provide the bluetooth link between the sensor and the PC. A cheaper bluetooth module like the BlueSMiRF Silve would do just fine, and we only got the Gold model as the Silver model was back-ordered at the time of the project. The bluetooth module connects to the micro-controller via a USART serial link, and provides a virtual serial port on the PC. On Linux, the rfcomm command can be used to setup the rfcomm device, while on windows an appropriate bluetooth driver (eg. BlueSoleil) can be used. A cheaper alternative may be to use a MAX232 to provide a wired link to the PC.

Schematics

It'll be nice if we have some schematics of the sensor huh... Too bad we didn't keep proper documentation and I don't remember it well enough to redraw it here. On the bright side, it's not all that hard to figure out.

Software

The software were all written in Python. The choice of language was a simple one; I'm rather language agnostic, while the rest of the team were relatively unskilled in any programming language. Python is a modern, easy to use language, and it saves the inexperienced programmers in the team from the hells of buffer overrun, stale pointers, and type conversion. Pygame is also used to simplify many of the programing task and allow platform independence (all of my software development was done on Linux while the rest of the team uses windows).

The software comprises of 5 games, "Cupcake Muncher", "Photo Window", "Egg Catcher", "Catch the Fish", and "Breakout" (this last one was written by me as a sample game for the benefit of the less experienced programmers), but I consider only the first two to be sufficiently polished to be usable. I also wrote an interface library to communicate with the 3D sensor and translate its reading into an onscreen coordinate, as well as a number of utility libraries to simplify the programming task for the other programmers in the team.

• Cupcake Muncher
  
  Cupcake Muncher is a single motion game, that only requires one repetitive motion (eg. raising of arm) to play. At the starting page, the patient will select the type of exercise he wish to perform from a choice of "arm raise", "arm swing", "shoulder shrug", "arm thrust", "hand grip", and "leg raise". When the game starts, a "muncher" walks continuously towards the right of the screen, and the player carry out the selected motion to make the muncher jump to avoid rocks or to catch cupcakes.

• Photo Window
  
  Unlike Cupcake Muncher, Photo Window is a multi-motion game which requires the patient to swing, raise, and grip his hand/arm, in order to complete a stage. The game displays a scenery obscured by a dirty window, and tracks the patient's arm orientation, translating it into the onscreen movement of a sponge. Swinging your arm to the left,right,up, or down, will move the sponge in the corresponding direction. Gripping onto the rubber ball will make the sponge wipe the window. As the window gets wiped cleaned, the scenery behinds it gets revealed. When the window is sufficiently cleaned, the full scenery is exposed and the level is considered complete. A large number of publicly available images licensed under the creative commons improves replayability of the game.

  Due to the large amount of data operations that needs to be carried out in the game, Py Numeric was used to provide array operations. Numpy was used as a replacement on windows, but not flawlessly. With Numpy, the cleaning of the window does not follows the sponge pixmap, but it's bonding box instead. Not sure why this is happening, but maybe I'll try and fix it one day...

• 3D Sensor Interface Library
  The interface library provide a layer of abstraction between the game logic and the hardware, and also allow for easy testing of the game by replacing it with a fake interface library which uses the mouse to simulate the sensor. The sensor library works on a polling basis, which turns out to be a bad design decision. Turns out the bluetooth module introduces significant latency into the link, and as all the games poll the sensor once per frame, this resulted in a lot of slowdowns as the game waits for the sensor library to return. Attempts to segregate the sensor library into a separate thread was unsuccessful. It seems that python does not provide real threads, as blocking in one thread would affect the others. Eventually, I faked it by making the library do a non-blocking read whenever it is polled, and return the previous result while waiting for the new one.

  On hindsight, I probably should have made the sensor push out data continuously without polling, and made the sensor interface library read off the latest data in the serial buffer, but at the time of the project, there simply wasn't enough time to make the necessary changes.

• Utility Libraries
  Besides the sensor interface library, I also wrote a number of other libraries to simplify some common programming task for the other less experienced programmers. It includes a menu generation library, an animation library, and a text drawing library (could have used pygame for the last one, but I had a lot of bad experiences dealing with fonts and didn't want to risk it).