Sign Translation Interactive Systems labs

Large amounts of information are embedded in natural scenes. Signs are good examples of objects in natural environments which have high information content. Signs are everywhere in our lives. They make our lives easier when we understand them, but they can pose problems or even danger when we don't.

Automatic sign translation, in conjunction with spoken language translation, can help us to overcome language barriers. We are developing technologies for automatic sign translation. These technologies will help us to retrieve information in foreign languages using a wearable computer or a PDA. These technologies can also help visually handicapped people to increase environmental awareness

Challenges of Automatic Sign Translation

Automatic sign detection
- No motion information - signs move with the background
- No standard geometric information - signs can be different shapes
- No color reflectance information - signs are meant to have different colors
Automatic sign recognition
- Low quality of input images
- Image deformation due to a bad viewing angle
Automatic sign translation
- Sign has to be short and concise
- Shorter words/phrase are more likely to be ambiguous
- Abbreviations are widely used in signs

Research Goals

We are developing technologies that automatically detect, recognize, and translate signs or other forms of text embedded in natural scenes to increase information awareness, overcome language barriers, and reduce cognitive loads. These technologies will be able to:

Automatically detect signs embedded in natural scenes
Enhance character recognition technology
Robustly translate embedded information
Integrate Human-Human and Human-Machine Communication into a variety of applications