Systems Design Augmented Reality

Design team members: Bernice Chan, Michael Chan, Jamie Huynh, Nathan Ng

Supervisor: Justin Eichel, PhD student

Background

 With the recent emergence and domination of the Internet in everyday life, people all over the world have used it to share information and connect with each other [1]. Social networking, provided by services such as Facebook, has become an intrinsic part of many people's lives as a means to store and share personal information as well as to add, interact and stay connected with others. Unfortunately, as more and more people begin using the internet as their main method of interaction, there is the danger of people being lost in a virtual world and forgetting about the benefits and joy of real-life encounters.

Developers of mobile applications on devices such as the iPhone, Android and Windows Mobile phones have begun experimenting with Augmented Reality (AR) [2]. AR is the mixing and integration of real world elements and digital imagery to enhance a user's perception of the world. It is an excellent way to enhance the real life experiences of users with the added benefit of digital and online information.

The need to expand their network can be solved by integrating the benefits of social networking with reality through an application that utilizes AR. An Augmented Reality system will increase the interactivity of personal digital information as well as provide a fun way of meeting other new and familiar friends. An AR application would allow them to simply point the camera to get instant access to the other user's personal information. This system could provide benefits to numerous social gatherings such as conferences, parties and other social events.

Project description

This design project attempts to create a mobile phone application that seamlessly integrates the digital world with the real world by allowing users to get personal information regarding a person standing in front of them. This application requires the integration of many different components to produce an optimal user experience. The main components of the proposed design include geolocation, image processing, user interface design, software design and implementation on the iPhone, and backend server and database.

The main user will use their mobile device to retrieve social networking and personal information from the peripheral individual (with the same application and device) through the use of a camera. Both devices will retrieve geolocation information from various sources and periodically send the information to a backend server. Then an algorithm will refine their location and register it onto one coordinate system and return that data back to the main user. Once the phone has determined the relative location of the second device and the orientation of the camera (via a compass), the phone will determine the identity of the person in the camera image. The mobile device will then use image processing techniques to identify and track the person in the camera, and an easy to use user interface will overlay the user's social network information on the screen. The final output is an Augmented Reality application that bridges the gap between real friends and their online identities.

Design methodology

Geolocation
    The geolocation component enables the application to connect the real world to the digital world by identifying the specific user in the camera view through their relative location. The main objectives are to determine the long range location, short range proximity and angular direction of other devices with respect to the main user achieved with the use of location based services such as GPS, cell tower triangulation, Wi-Fi location and digital compass. In short, the geolocation component determines "who" the camera is pointed towards.

Augmented reality

Augmented Reality primarily utilizes image processing techniques to perform a fast and accurate facial detection on the environment it is surveying with the iPhone's camera. The facial detection will then be used to discover and track people as they enter the scene. This combined with geolocation will be able to uniquely identify people standing infront of a user.

User interface

The user interface focuses on a modal overlay for the camera view as the user looks at the environment using the device's built-in camera. The overlay is direction-oriented from first-person perspective, but remain unobtrusive to the observed view. The use of colours, highlighted areas and directional pointers must be carefully situated to adhere to small screen real estate principles and selective information (name, social network status) [3]. In addition to this, the user interface involves a more in-depth view of the target's online information. This allows the user to scroll through offered information organized for efficiency and importance such as profiling and contact information.

Software

The structure in which the software is developed will proceed according to the software life cycle so as to achieve sufficient algorithms to meet performance benchmarks in frame rate for user interface and object identification, application size, and compilation speeds sufficient for image processing. This process involves requirements and domain analysis, specification, implementation and development of software architecture [4]. The software component serves as an intermediary for implementation of the image processing algorithms for the user presented in a form of augmented reality. The programming will exploit geolocation technology to achieve communication via networking technology from peer to peer. The software design must also adhere to appropriate limits on bandwidth and data usage as presented by the device on which this tool is programmed for and fully utilizes the iPhone's native programming language: Objective-C.

Backend server

In order to develop a backend system that is secure, robust, reliable, and easy to develop and implement for, cloud computing is used as the network architecture from which to host cloud applications and store data. Cloud computing services have immense scalability, zero downtime, and relatively low maintenance costs [5]. Subsequently, a specific cloud computing vendor needs to be chosen. After a thorough analysis, Microsoft's Windows Azure Platform is used as the platform for which to develop for, because it provides the needed functionality, has abundant documentation, and uses familiar development languages and environments.

With Windows Azure acting as the platform for the backend system, the individual components of the system are planned out and defined. These components include storage solutions, compute services and communication protocols [6]. In terms of storage solutions, Windows Azure Tables are used to store relevant information about a specific user's current location and social networks. These are then used in the compute services, where the application actively searches for nearby users. Finally, for the communication protocols, Representation State Transfer (REST) protocols are used to facilitate the transfer of information between the client application and the backend system.

References

[1] Our Over-Reliance on Technology. McPherson, Susan. 2, s.l. : Associated Content, 2009, Vol. 1.
[2] Kharif, Olga. Social Networking Goes Mobile. Business Week. [Online] 31 May 2006. [Cited: 5 December 2009.]
[3] Enhancing User Interaction With First Person User Interface. Smashing Magazine. [Online] 7 Dec 2009. [Cited: 5 December 2009.] http://www.smashingmagazine.com/2009/09/21/enhancing-user-interaction-with-first-person-user-interface/.
[4] Measuring Commercial PL/I Programs Using Halstead's Criteria. Elshoff, J.L. 5, s.l. : ACM SIGPLAN Notices, 1976, Vol. 11.
[5] Villmann, Alex. How to Setup a Dedicated Web Server for Free. net tuts+. [Online] 4 December 2008. [Cited: 6 December 2009.] http://net.tutsplus.com/articles/news/how-to-setup-a-dedicated-web-server-for-free/.
[6] Windows Azure. Windows Azure Platform. [Online] 2009. [Cited: 6 December 2009.] http://www.microsoft.com/windowsazure/windowsazure.