AR Prototyping

By Marcus on 14. Dezember 2017 — 2 mins read

We have been playing around with Core ML and ARKit for a couple of days. Say Hello to Explainar.

It is one thing to read and think about stuff. And it is something totally different to actually work on something. So we, the people of Vragments, decided to spent a week on exploring the possibilities of currently available AR-technologies on smartphones.

There is a lot of talk circling around the soon-to-be status quo. Forget about the smartphone, welcome the age of wearables, smart HMDs, glasses etc. Yay. Sounds good. But if you start having a closer look at the possibilities because you want to build the future you will end up quite disappointed pretty soon. We did not find a really clever and convincing AR-app in the stores yet. It is december 2017. Have you?


After checking the technical specifications of ARKit – you will end up disappointed yet again. Scene Understanding right now means:  ARKit can detect horizontal planes like tables and floors, and can track and place objects on smaller feature points as well. Well. The horizontal limitation is – a limitation. That is for sure.

Whatever. The question is: What do you want to built? What problem needs to be solved? What do you want to use? We got inspired by Benedict Evans talk 10 Years Futures. Besides a lot of interesting stuff Evans tackles the whole topic Mixed Reality (MR) and breaks it down nicely. LATWATMAI became our guiding thought.

This is the point where Machine Learning enters the game. Apples Core ML and Vision allowed us to let the smartphone 1) see 2) interpret and 3) augment further information using ARKit . For example: What kind of object is that, how is the word written, how is it spelled in different languages and is there a definition (from Wikipedia).

Truth told. We were aiming for a very specific use case (ARBnB) and while exactly working on that we saw that AirBnb is actually working on that – right now: Developing the Next Realities for Travel. Obviously the idea was not that bad.

Great to see a smartphone doing stuff it could not do before. I had to think about Jaron Lanier talking about how the kids most people these days think that you can not change stuff that is already there or take another path or leave the path. Playing around with the very first prototype Tino had built in about 2 hours felt really cool, like being onto something.

We are not yet there but we will wrap up things and put them on a website soon. Then you will be able to play around with our litte app. And we will be able to way better reflect about what currently works and what does not using augmented reality technologies on your smartphone in the year 2017.