Prototyping Augmented Reality

graphics-6

And so it is, having zilch knowledge of Augmented Reality, I need to take a big leap into the unknown. For the purpose of this work-based project, to test my analyses, I will need to do a prototype of this technology. Well, the closest technical knowledge I possess is my 3D skills. It’s a good thing that my crew mate suggested Unity 3D as something I could tinker with. I have started on some lynda tutorials on this software. The Augment app is also something I could try my hands on. That app provides more limitations but is easier to use for the layman. Good luck to me.

unity-scnshot

With all the limitations of time, and the challenge of needing time to practice so as to have a basic grasp of the software, i would say I am progressing at an acceptable pace. How far can I go with regards to my mastery of this game engine? Can I create an augmented reality prototype? Will my skill level and research status only allow me to reach the point where I can design and visualise an augmented reality idea? Either way, I think I am okay. This research project has already spurred me to study so much of the augmented reality and mixed reality realm, and a little of virtual reality thrown into the mix. My ultimate goal is to know these technologies so in-depth that I can share the knowledge I have at my fingertips and with great accuracy for the purpose of giving helpful professional advice to my clients.

I do, however, still want to master my knowledge of augmented reality and I consider the mastery to be successful when I can create an application. It might take me a longer time than the deadline of this research project.

18 October 2016 – Checkpoint

The more research I did for AR, the more I realized that what I have researched on is just the tip of the iceberg for such a gargantuan subject matter. At this stage, I must prioritise and decide what do I really want to do? What do I really want to be? This will streamline the way I use my time for research and education. It gets me excited to know that AR is doable for ordinary folks like me. Yet, the question is how in-depth do I want to go in acquiring knowledge about AR? Besides Unity 3D, research has brought me to Vuforia, Blender, Arduino, xCode and Swift. That is a lot of software to learn for a month!

I decided that I might need to start planning my prototype and learn the essential things that I need for this prototype. So there are some key questions I need to answer at this stage. For the rest of the software which I would love to explore, but just do not have the time at this stage, I will still pursue post-AWPB.

So now, the crucial question. What do I want to do for my prototyping? What design issues do I want to solve? What jobs do I want to do? What customer experience do I want to create that will alleviate their pain points with reference to their previous museum visits?

30 October 2016 – Taking the plunge

Having talked to some industry players on augmented reality, I decided that I needed to have some surface knowledge of how it is done. It shouldn’t be too difficult, given my background knowledge in 3D modelling and animation. And after a week of tinkering with Unity, Vuforia and Xcode in little ways, i started the day determined to “do something”. I scoured the internet for tutorials and found a few that could help me to reach my goals.

My goal then, was to build a simple AR app using  Unity, Vuforia and Xcode. My project prototype was based on my Malaysia pavilion pitch, so I used the mascot as a tracker.  The first tutorial I referenced was from this post:

My first AR teacher – Youtube. Source: (Youtube.com, 2014)

(“The Art Of: 3D – How To Make An Augmented Reality App (Exclusive) – Ovation”)

There were some assumptions that I already know a little bit of Unity. Basically the idea behind an AR app is simple. Fix an AR camera which works like a virtual camera, attach a target image to this camera, and attach an animator controller to the intended animated object.

screen-shot-2016-11-12-at-5-39-51-pm
You can see the ARCamera and the ImageTarget in this scene.

 

vr-setup
Under the ARcamera, there is a whole list of attributes to control. The App License Key is the most important. In order to link my Vuforia asset to this camera, I need to copy and paste my vuforia license key as shown in the next screenshot.
screen-shot-2016-11-12-at-6-34-31-pm
This is the license key required under the ARcamera in Unity. This key will link my Vuforia asset to my ARcamera.

 

screen-shot-2016-11-12-at-6-18-06-pm
You can see the Inspector panel on the right. This shows properties of the action pack (The animated character which I downloaded from the asset store). Notice that under animator controller, it’s the “action pact”. This means that the animation is applied to this object.
screen-shot-2016-11-12-at-6-34-45-pm
I added 2 target managers into my database. I will just use one of them in my project – Tiger2.
screen-shot-2016-11-12-at-6-34-59-pm
Under Tiger2, I have two target images. Mascot is the one I will use for my prototype.
screen-shot-2016-11-12-at-6-35-11-pm
I have uploaded the target image into my Vuforia asset database. Once my ARcamera tracks this image, the animated object in my scene will be activated.

I have effectively succeeded in using the webcam of my macbook pro to do a test of this AR. Everything worked as expected. However, I wanted to take it further by testing on a mobile device since that is the ultimate goal of any app. For this, I searched for more tutorials. I took reference from this page:

https://unity3d.com/learn/tutorials/topics/mobile-touch/building-your-unity-game-ios-device-testing

(Unity – Building Your Unity Game To An Ios Device For Testing, 2016)

I followed the steps religiously, like linking my apple ID and choosing the IOS option and etc. The following screenshot shows my Xcode coding (which contains a lot of errors, by the way.).

screen-shot-2016-11-13-at-3-57-14-pm

The end result looks like this:

Insert video.

My peripheral programming practice – Arduino

The next stage of my AR programming adventures brought me to Arduino. I bought an Arduino starter kit and got to work immediately on a tutorial. Arduino is useful for AR as a programming tool for real objects interaction. Motion sensors (Or any other feedback sensors, for that matter) can be connected to an Arduino module that can then be programmed to replay an AR scene to a device. This video shows a very simple example of Arduino powered AR.

The furthest I got in my Arduino adventure is in understanding the circuit board, resistors and the switch, as shown in my video.

Video

Reflection

With this prototype aided by scraps of tutorials from everywhere, I don’t think I have reached even the intermediate level of Unity3D, Vuforia and Xcode, but this experiment helped me to have at least some surface understanding of how these multiple platforms work together. From researching different tutorials on the Unity3D game engine, I am intrigued by the incredible possibilities this software can offer. It is a single software with many deployments. I could use this to create a virtual reality or an augmented reality game, and deploy to IOS, android, HoloLens, VR format and many more. Is this the beginning of an exploration in Unity3D for me? Being trained in 3D animation provides an advantage for me. This looks to be an area where I can explore, whilst building on my previous foundation in 3D.

The future of augmented reality in Unity seems to be cemented in a strategic partnership announced by Unity and Vuforia on 1 Nov 2016. (Vuforia Developer Portal |, 2016) They have plans to integrate the Vuforia AR platform into Unity to deliver a seamless development experience to Unity’s global ecosystem. This is good news especially for me who had to learn three SDK to create an AR app. I had to log in to my Unity account and my Vuforia account while working on the app. It was tedious and confusing. The integration of these two SDK into one is a good news for AR developers.

The third part of my prototype – testing an AR app in a mobile device – was to try out what simple coding is like. All of these SDK have a foundation in the C++ and C# programming language. I tried some Swift programming using xCode by following an online tutorial called CodewithChris.com (How To Make An Iphone App, 2016). I got only as far as creating buttons. 

Depending on my career plans for the future, I might or might not pursue these coding knowledge, but it sure was fun while it lasted. This amount of prototyping afforded me a basic programmer’s lingo which I could use as I talk to developers and technical project managers in business setting.

 

 

References:

  • The Art Of: 3D – How To Make An Augmented Reality App (Exclusive) – Ovation. YouTube. N.p., 2016. Web.

  • Unity – Building Your Unity Game To An Ios Device For Testing. Unity. N.p., 2016. Web. 

  • Vuforia Developer Portal |. Developer.vuforia.com. N.p., 2016. Web. 
  • How To Make An Iphone App. Codewithchris.com. N.p., 2016. Web. 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s