cAR Implementation Notes

Table of Contents

Table of Contents

• 1 Introduction
• 1.1 Purpose
• 1.2 Scope
• 1.3 Definitions
• 2 ARCore
• 2.1 Development RoadBlocks
• 3 ARKit-Flutter Integration
• 4 UI/Database Integration Problems
• References

1 Introduction

The process of building real software is always arduous and time consuming and will often face roadblocks that cannot be anticipated in the design and analysis of an idea for an application. Implementation notes document all the roadblocks and challenges that were faced in the development of the cAR application. This dwells into detail the plethora of real world software development challenges faced by the developers in attempting to build cAR.

1.1 Purpose

Implementation notes are a thorough documentation of all the roadblocks and challenges that were faced in the development of the cAR application. The document describes all major issues with external software tools that permanently blocked the development of key features of the cAR application.

1.2 Scope

Implementation notes will only focus on external software tools used and the unsolvable roadblocks created by said tools. The document will not discuss major issues that were faced, but were eventually solved in the development of cAR.

1.3 Definitions

2 ARCore

During the design and analysis phase of cAR, it was discovered that the only way to render AR models in an Android app was through the use of Google's ARCore API. However, ARCore API required significant knowledge of the industry standard graphics library, OpenGL.

Using OpenGL, to load and render 3D models requires a developer to have a strong understanding of graphics engineering. None of the engineers on the team had the background to use OpenGL directly. A workaround to this roadblock was to use Google’s third party OpenGl wrapper Sceneform.

Sceneform abstracted the calls to OpenGL and removed the massive technical understanding of graphics. The entirety of the android application of cAR, depended on the usability of Sceneform.
Figure 1: A screenshot of Sceneform’s marketing material taken from https://developers.google.com/sceneform , in the top right corner the web page states, “Sceneform is a 3D framework that makes it easy for you to build ARCore apps without OpenGL.”

Based on the information presented, during the design and analysis phase, it was assumed that Sceneform was a functioning software tool that abstracted OpenGL.

2.1 Development RoadBlocks

While building the prototype of cAR for android, several tutorials [1,2] were consulted and sample code was looked at. It became obvious that Sceneform was the only viable option to rapidly build cAR on Android.

The very first glaring bug that was encountered was the incompatibility of Sceneform with the latest version of Android Studio. Android Studio is the only existing IDE that allows developers to build and test Android applications on an Android device or Android emulator.

This incompatibility prevented developers from correctly installing the Sceneform plugin, which further prevented the automatic loading of 3-D assets. The first viable workaround to this issue was to create the Sceneform configuration file manually and point them to the 3-D models. However, since the Sceneform library didn’t generate the required configuration lines in the build gradle file, the model’s textures would not load correctly.


Figure 2 : A screenshot of the textureless car model on Android.

In order to find the root cause of the missing textures, several sanity tests were performed using existing 3-D models that were sourced from existing sample applications. These 3-D models were originally sourced from Google’s 3-D model sharing website google.ar.poly.com, which contained 3-D models that were optimized for the purpose of AR. During these sanity checks, it was discovered that Google had indefinitely removed google.ar.poly.com.


Figure 3 : A screenshot of Google’s notice on google.ar.poly.com

After doing additional research, it was clear that development and maintenance of Sceneform had indefinitely stopped[3,4]. Google shifted their own priorities for AR apps and decided to suspend several projects that originally supported ARCore.

It is now clear that Google is discouraging developers from building AR related applications using Sceneform. This meant that any existing bugs in Sceneform have most certainly not been fixed and that the library is itself broken. Because of cARs dependencies on Sceneform, the only viable decision was to cancel the Android version of the application.

3 ARKit-Flutter Integration

Due to the time constraints for designing this project, the teams working on ARKit designed their app natively using Swift on Xcode 12. While designing the application, the entire AR portion of the iOS application was then required to be implemented into the main Flutter front end. The only possible approach to this project was to integrate the Flutter SDK into the ARKit project using Cocoapods, and then porting the Dart code from Flutter into the Xcode ARKit project.

This process however, was far from straightforward. The first major issue was the layout of the integrated Flutter engine, which was more of a ‘pop-up’ style view rather than an integrated AppView. This meant that the app felt like it was ‘glued’ on top of the existing AR implementation rather than one cohesive whole.

The second issue was the implementation of Firebase. Due to the Firebase calls inside Dart, the iOS AR base project needed a complete revamp of the Firebase trigger systems due to the system permission hierarchy. This meant that the entire development process of setting up Firebase on the Flutter project using FlutterFire ended up being redundant, which was a major issue as organizing that infrastructure was a significant portion of the development time.

The third issue was the call to trigger the Flutter engine into the iOS application. The way the Firebase-to-iOS framework is designed dictates that the base iOS application view must call the first Flutter designed view, which meant that the layout of the first pages (WelcomeScreen) of the app would be redundant, and be easily dismissed due to the ‘pop-up’ nature of the Flutter UI. This effectively destroyed the MVC (Model-View-Controller) work on the Flutter side by adding easily circumventable views that could break the flow of the app implementation.

4 UI/Database Integration Problems

Facebook sign in did not work for us, crashing the android side of our app. The AndroidManifest.xml file seemed to not handle the calls, and our team was unable to get through the process. Google Sign in returned an error with our sdk version, it also was reportedly overriding deprecated plugins we had so it was removed. This page posed a big error for us, as we had trouble connecting the authentication to the user object in our Realtime database. The user page was made but the subsequent pages we did not have enough intel to understand how to populate the remaining pages. The Change User Info page created multiple errors due to rendering issues and was removed from final submission. Favourites and favourite button features were removed due to problems updating the list view page with the new information. Search was removed due to the complexity it posed and the fact that not enough of the group were able to help with its implementation due to technical issues on their end.

References