What are some of the skills and tools that you use to develop and test your AR solutions?

Yeah you must learn it all if you want to have an AR app, but I only use C# for all of it! How? With Unity Engine you can have a great power of making AR applications and games with a Powerful library and support of new features, with only use C# and Some unity packages like ARCore, ARKit and ... you can have a AR app for all platforms and it's really easy to make one

Since Unity was mentioned, it would be useful to mention directly AR Foundation that is an integral part of Unity and brings the support for a range of AR platforms and devices, including ARKit (iOS), ARCore (Android), OpenXR (used for HoloLens), Magic Leap support, and others. And it is great that you can develop one application and than build it for different platforms and devices

For those interested in developing applications for AR glasses, specifically for Hololens 2, I would argue that the easiest and fastest way to get started is by using Unity and Mixed Reality Toolkit (MRTK 2) from Microsoft. One would need C# skills (coming from another OOP language should be okay, e.g., Java), and have a good understanding of AR interaction and user interface (even UX).

You can use any language or/and engine if it supports these libraries. Yes, by default it is a usage of ARFoundation + Unity, but if you want to have a browser AR - WebXR with A-Frame can help you there. Right now the best approach of AR usage you can find on iOS devices with ARKit help, you can do it native as Swift iOS development, or cross-platform with Unity.

I agree that you would want to be proficient in at least one language but I always advise anyone not get scared of coding and give up on learning altogether. Coding gets better with experience and hands-on time. IDEs that are coupled with game engines are pretty good at showing you prompts and can help you learn along the way.

A good approach is to test a framework for a couple of days before getting really into it. There are additional factors to consider: max size of point clouds, workflow, other involved technology/companies, access for needed equipment (e.g. Matterport 3D camera), how easy is it to develop, and pricing of course. There are some big differences between frameworks, some are more top level then others. E.g. Vuforia and ARFoundation is more top level than ARCore and ARKit. Often you can also mix frameworks, so you are not bound to the advantages of one. I’ve been using Vuforia and Unity for large AR experiences over the last three years very successfully. For more simple things I prefer to use ARFoundation and Unity.

Mixed Reality Toolkit (MRTK) should be added to the list. It is an open source cross-platform toolkit that provides the basic (and much more) building blocks to allow one to develop AR/MR applications in Unity. I’ve been using it since 2018 to develop AR apps for Hololens (1&2). It drastically impacted the time and effort required to build these apps. I would rate it as a must-have tool when it comes to Hololens app development.

First and foremost you need to know what devices you are targetting. Mobile browser AR which requires compatibility with iOS and Android require a totally different approach than developing an AR app for Hololens for instance. Once you have defined this, you will probably notice that suddenly your choice of frameworks will become a lot smaller. And in my experience, the developer with Unity/AR Foundation skills are not the same people as the ones who can master an 8th Wall project.

AR frameworks can be really helpful in expediting your application development. I would suggest to start with simple use cases like image tracking in any SDK and move on to complex applications and use cases

An appealing visual composition is very necessary in creating AR experiences. Not only does good visual composition help make your creation look pretty, but it also helps create a balanced look for it to feel seamless and not out of place in the physical environment it's placed in. It helps arrange all the digital bits and real-world pieces in a way that's not just pleasing to the eye but also grabs your attention. Think of colour, balance, contrast, and focal points.

One thing I found helpful is to test design ideas with users very early and very often. Looking at the design guidelines from ARCore and ARKit have helped a lot when I started. It’s important to take into consideration that UI creation for AR is more time consuming than e.g. a web app because your canvas is three dimensional.

A consideration for design should also be to understand the ultimate purpose of why the user is utilizing the AR. There are many ways that AR can supplement the field of view or improve actions/time, but without a clear purpose or scope it will be hard to reduce clutter on the screen and identify what should or should not be included for the user. A way that I worked to improve the scope for my user was to ask better questions of the focus group during the build process to understand their experience and goals before broadly scaling the AR solution.

There are various detect ways of AR, and you have better make high relation between the target and the virtual AR materials. For example, when you decide to make an AR experience by scanning a target, make sure your video frame starts with the same image of the target. This makes a classical Harry Potter newspaper effects.

From my experience, I found that natural and implicit user interaction is key in building an AR app. Users must feel safe and at ease while in an immersive AR experience, reason why the AR interface should be as intuitive and simple as possible. An AR designer/developer should constantly keep in mind that most users come from 2D interaction mediums. Even though 3D space is natural for humans, interacting with digital 3D interfaces is not obvious for everyone. Do not clutter the UI with dozens of digital elements!

Debugging and analytics through Firebase, Bugsnag, Flurry, or Appsee are essential but should only be done in tandem with UX/UI testing - the most essential element is the human. Quality Assurance takes a lot of human testing time. It's worth it.

When it comes to AR, most of the testing phase needs to be spent on figuring out device compatibility, performance optimisation and basic user interaction. I’m sure you’ll come up with wide range of issues if your application use case is even more niche. But time spent on debugging and optimising can help you a long way in creating pipelines and guidelines for your next projects

Make sure you know which devices are being used by your target market. While the number of mobile devices supporting AR is growing, there are still certain parts of the world, like India, where smartphones used by a large population don't support AR features. We set out to build great AR marketing campaigns 18 months ago but had to pack up because many target consumers couldn't render AR experiences on their mobile.

First you need to define what problems are you solving and then start building the story board of the idea and work on the prototype. AR is the horizontal technology works on any vertical industries.

On every step in the process ask yourself if AR is adding value to the user over other media. Often I see AR apps that have no connection to the apparent context and therefore would be better suited to use VR. There needs to be a clearly readable connection: the trickiest part of AR is reality.

It is helpful to know that AR development is just part of the whole development process. While AR alone may represent the whole frontend part of a software system, it is also used in a mix with other standard frontend technologies. Besides, the frontend part most likely needs to connect to a backend. C# or even Unity knowledge alone will not be sufficient to develop the whole system. Consider the situation when a new user account has to be created on a server and the marketing department asks you to push the information to Pipedrive so that it can start sending onboarding and sales emails.

In the starting stages of my career the SDKs , when compared to their current states, were pretty basic. We were restricting to certain devices and certain features. But today, the possibilities are endless and most of the recent devices let you experiment with XR. So feel free to explore different frameworks and brainstorm on how XR can help in your particular use case. Start small, redesign your approach and keep scaling it further. I would also put an emphasis on optimisation and making sure to cover usability on a wide range of target devices.