How to Easily Develop an Augmented Reality Application? Software Development

The easiest answer – don’t try to fit reality into a virtual concept. Instead, paste objects into the displayed image of the real world, so it can feel as real as the reality right next to it. What methodologies can you use to Augmented Reality application development? What risks should you avoid or contain? What platform to choose and what are the matters you should remember about?

Different platforms and functionalities dedicated to Augmented Reality application development

ARCore from Google

AR Core is a SDK created by Google. It allows to build augmented reality applications so the virtual content can be integrated with real world thanks to the phone’s camera. It is one of the most popular AR technologies these days.

To integrate virtual and real-world AR Core uses:

• Motion tracking – to track a phone’s position relative to the surroundings.
• Environmental understanding – to detect the size/location of surfaces, from horizontal and vertical, to even angled surfaces.
• Light estimation – to estimate the real-life lighting conditions.

Software

OS: Android 7.0 (Nougat) or later, iOS (ARCore requires an ARKit compatible device running iOS 11.0 or later)

Hardware

ARCore is available for free for newer devices like Samsung Galaxy S7 Edge, Galaxy S8/S8+, Galaxy S9/S9+, LG G6, LG V30, Huawei P20, Google Pixel 2/2XL and many more. All available devices can be found via Google.

iOS devices are also supported but it requires the usage of ARKit anyway.

Unity support

ARCore supports Unity developers. It ensures:

• SDK for Unity with examples in the Unity
• Tutorials on ARCore and creates simply app
• Unity documentation and application programming interface (API) references

Surface detection feature

It is possible to detect horizontal and vertical surfaces with ARCore. It scans your environment with the device camera and recognizes surfaces. The developer receives the position and rotation of the detected plane. Having this information, the programmer is able to calculate angles between any detected surfaces. It could be very useful for placing objects in 3D space precisely.

This feature is useful but precision leaves a lot to be desired. Sometimes it works well, but sometimes it works not good enough. It is a very nice feature for placing a game arena on the office table. Then the whole real environment can be used as a game environment. On the other hand, for some precision measurement apps, it could be not precise enough without additional amendments and adjustments in the code.

ARKit from Apple

ARKit is an augmented reality framework created by Apple for devices with iOS. It can be used to create mobile apps and games where the living room can be turned into the environment of the app. ARKit is an integrated part of the iOS system. It is an alternative for the ARCore. It works very similarly in general.

Main functionalities that Apple includes in their AR technology are:

• Persistent AR Experiences Now, your mobile apps can provide AR experiences that persist between sessions, and can be resumed at a later time. Users can start an AR puzzle on a table and come back to it later in the same state, or tackle an interior decorating project over multiple days without having to start over. Shared AR Experiences AR apps are no longer limited to a single person or device participating in an AR experience. Now, multiple users can use their iOS device to simultaneously view AR experiences or play multiplayer games. Bystanders can even spectate AR games being played by multiple participants.
• Object Detection and Tracking ARKit 1.5 added support for 2D image detection, letting you trigger an AR experience based on 2D images like posters, artwork, or signs. ARKit 2 extends this support to offer full 2D image tracking, so you can incorporate movable objects like product boxes or magazines into your AR experiences. ARKit 2 also adds the ability to detect known 3D objects like sculptures, toys, or furniture.

ARKit is available for newer devices with Apple logo. Almost all current devices are supported. Full device compatibility list can be found on Apple’s website.

Unity support

ARKit similarly to ARCore supports many developer platforms including Unity. SDK for Unity as a plugin

Useful tutorials/articles can be found here and here.

ARKit, similarly to ARCore, can detect surfaces, both horizontal and vertical. It is implemented in a different way, but results are almost the same. The developer, thanks to devices camera scan, can access detected surface position and rotation. It is given in a different structure and in a different way than in ARCore, but the final result can be the same: position and rotation in Unity structures Vector3 and Quaternion. More about detecting surfaces and angles between them can be found on Knights of Unity’s blog.

Vuforia

OS: Android, iOS

Vuforia is developed by an American company called Qualcomma. This software development kit is used in many apps in over 130 countries. Image Database for Vuforia can be built and accessed in different ways, such as:

• creating it within the mobile device using the application and processing all of the data using devices processor,
• sending data onto an external cloud server, where all of the processes are done.

Vuforia and Image recognition

Vuforia SDK allows to track not only one but multiple image targets at the same time within the camera field of view. By analyzing connections between characteristic reference points, it is not limited to only 2D space but also 3D. A recognized image can be used to generate additional content in a few ways.

The most common one is displaying a text message about tracked objects, for example, name of the painting or piece of art by downloading data from a local or external database and displaying it onto the camera view. The source image here isn’t a QR code but an actual existing object. Another way to use image detection technology is to generate more advanced objects such as animated models. This can be used to show the way animals behave or display a battle scene in a book one is reading.

Vuforia also allows us to recognise text! In the standard version, we can access a dictionary with over 100 thousand English words available right away, but developers can also add their own words database.

Qualcomm gives developers a possibility to use special codes called “VuMark” which are quite similar to normal QR codes but give the freedom to put your own image inside of it, so as to make it more personalized. More information on how to create your own VuMark can be found on Vuforia’s website.

Other Functionalities:

Vuforia Smart Terrain – a feature that allows to reconstruct and augment physical objects in an environment, to create new types of gaming and visualization software. It is best suited for tabletop experiences where you can detect objects such as cereal boxes and cans to create an environment for your game or visualization.

• Vuforia Model Targets – introducing object recognition by shape, contrary to visual print media designs. This will help track images, scanned objects, special marks (encoded), text and surfaces
• Vuforia Ground Plane – a feature for Unity engine that allows to place the augmented content on the ground or other horizontal surfaces. It helps to create high-quality visual apps and designs.
• Vuforia User Defined Targets – a feature that allows creating image targets at runtime from a user’s camera. They share most of the capabilities of a standard Image Target with only one exception, they don’t support Virtual Buttons.
• Vuforia Fusion – a feature aimed at solving the fragmentation and enabling cameras, sensors, as well as external frameworks, e.g. ARKit.
• External Camera Usage – The External Camera feature allows the developer to define how Vuforia Engine acquires frame data. It must implement several APIs which the SDK uses to access the external system. At the current moment, it only supports vision-based features such as Image Targets, Multi Targets, VuMarks and Object Targets.
• Vuforia 7.1 Update – a whole bunch of improvements on v.7 release, including automatic initialization on scenes with Vuforia components, removal of Google Tango support, an option to clone objects, and more. In Vuforia 7.2, it is expected to expand to supporting ARCore, HoloLens and Universal Windows Platform devices.

Future of Vuforia

Qualcomm is currently working on other features which will be added to SDK, such as:

• Background Effects, which will allow adding virtual shader effects on the camera, for example, X-ray effect or night vision.
• Video Playback, this feature will allow smooth transitions between a static image into full-motion video directly on the target surface.
• Virtual Buttons, which can allow a new type of interaction with objects like magazines or posters by adding a virtual touch screen on them.
• Occlusion Management will allow recognizing an image hidden behind everyday barriers and obstacles like the human hand etc. Special occlusion handling allows displaying objects as if they were in real physical objects.

Summary

Augmented Reality application development is a very spacious field of science, which in the future, can bring a lot of good solutions and fun into our lives. AR allows us to soar into a new level of entertainment. It is a rapidly growing market where new solutions and devices are being developed faster and faster.

Read more here.