A 1000 commits later...

Just today I reached the milestone of 1000 commits for the ShapeReality codebase. Because of this, I’m taking the opportunity to reflect on my how I have developed, both on a personal and professional level.

To summarise, I have become a way better programmer, but I still have so many things to learn on a professional level and in terms of domain expertise. Software cannot be created in isolation and requires one to deeply understand the domain of users or clients.

I stopped communicating with potential users or clients because I was afraid I wouldn’t be able to deliver the software. Now that I have almost proven to myself I am able to deliver a piece of software with real functionality and integration with an existing ecosystem of standards and file formats, instead of just a prototype that aims to test some UI/UX concepts, I need to throw myself more into the real world again. Talk to users and see where their issues lie.

The challenge #

Before, I used to pride myself on my achievements in design- and wacky art projects, but now this has shifted to software development. Not the boring CRUD kind, but the hard, unchartered forefront of what software can be.

This, because I think there still is huge untapped potential in AR and VR devices. Not in entertainment or social media, but for productivity. While there are some issues in terms of accessibility and usability with existing AR and VR headsets compared to more traditional computing devices such as mobile phones, laptops and desktop computers, these issues can be solved on the long term. That’s what I am designing for, the long term. I’m exploring what AR and VR software can be and in this way inspiring people to look at it as a professional computing device in its own right, instead of a mere accessory to sometimes pop onto your head to see whether the VR experience you’re creating does not have any issues.

After talking to a dozen VR companies in the Netherlands, all producing VR training software, and hearing that they don’t use VR at all in their professional workflow, I got demotivated. Was there really no place for VR in a professional workflow? Seeing ShapesXR struggle with their value proposition, attracting and keeping users, further emphasised this impending feeling.

The vision #

But I still see it, the vision of being able to design directly in the real world, instead of sitting behind a computer. Combining scanning technology, ever-advancing machine learning models, with the power of real-time rendering, could provide us with a technology so advanced it would feel like one was living in a science fiction movie.

This technology could disrupt – a disgusting, overused word, but useful nonetheless – many industries that have needs related to spatial planning, communicating spatial plans or integrating people’s opinions in these spatial plans.

Advantages of AR and VR over traditional 3D software are:

Intuitiveness of navigation #

Because you’re editing and viewing something that is inherently 3D, doesn’t it make sense to view it in 3D too? Humans are good at spatial navigation, so why rely on navigation through complicated keyboard-mouse combinations? In AR/VR one can simply walk and look around.

Manipulating and moving objects around can be done without learning compicated commands and

True-to-life Scale #

Someone viewing a design in AR/VR, can see the actual scale of an object. It is hard to miss problems with chairs or doorframes being too small.

Immersion #

One can get a feeling for what a space, environment or design is actually going to look like. With a car you can simply sit inside the newly produced vehicle to test out whether it fits your needs, but with the built environment, every design is unique (with the exception of prefab buildings), so there needs to be some way to validate whether the design is actually what is wanted before it gets produced.

That is where architectural renders, or architectural visualisation (commonly abbreviated as archviz) comes in. It allows the client or end-user of the space to feel and experience the space.

Architecture, interior design, urban design and planning are all fields that could greatly benefit from designing spatially. It reduces errors.

Solution in search of a problem #

One of the biggest issues with ShapeReality I’m experiencing is that it is a solution in search of a problem. Same with Microsoft Maquette, Tilt Brush, 123DC.io, Clara.io, there’s no real need for a 3D design tool that is not the industry standard. If it has less features than the industry standard it is most likely to be used for learning or education, which is a slow market with long sales cycles, thus not able to convert users (because all these tools did have users) to paying customers.

In the safe environment of academia, Innovation Space and a so called student team, one can simply ignore these hard lessons. But I don’t want to ignore them.

Another market that is interesting to focus on is 3D artists. These people work for game studios, VFX studios or do freelance work for a variety clients, same as what I did for the Ministry of Defence while creating AR / VR experiences.

Competition #

At first I was scared that someone else would have the same idea as me and implement it better. Now I see that even if there is an existing company, they are also probably struggling with the exact same questions of market viability, whether they’re solving a real need or how to price the software. When I saw Arkio (Arkio.is) launch their product with the slogan:

Design interiors, sketch buildings and craft environments with your hands. Mix realities and experience design options on-site. Collaborate anywhere using VR, desktop and mobile.

I was terrified. I genuinely thought: “oh no, it’s over, it’s a solved problem now”. Then, after some time acquainting myself with the software, I thought “This piece of software sucks, I can do it better, but the fact that they exist means there’s a real need here! And they have customers!” Then, again after some more time reflecting and thinking, I thought “Fuck, the reason their software is so bad is because there isn’t a real need and probably don’t have enough money to improve the product. They’re probably struggling to get more customers.”

That’s an interesting phenomenon I noticed in myself. Going from: To seeing them as an existential threat, to feeling relieved, to seeing their underperformance as an existential threat again.

On building XR software #

At first, I was only comfortable with using existing packages, libraries and frameworks, such as Unity, the XR Interaction Toolkit, MRTK, Unity UI, Apple’s UI frameworks such as SwiftUI, React.

But at a certain point this changed into wanting to start from scratch. Into reinventing the wheel and honoring the not-invented-here syndrome. I wanted to create my own UI library. How hard could it be?

At that point, too hard, I didn’t know the requirements and did not have the experience to actually set up a good software architecture. I didn’t know about how complicated the flexbox spec was for example.

So I let it go and focused on using everything the way it was intended from Unity and the XR Interaction Toolkit. I would only add abstractions once clearly presented as needed.

On building software in general #

Because I was just starting out, I thought, how hard can it be to write a backend, a front-end and a VR application all at once? As it turns out, this is also incredibly hard because the complexity increases by a lot when you start dealing with “the cloud”. At a certain point (after finishing the login screen with Amazon Cognito, a .NET backend, a React front-end and setting up CI/CD pipelines), I saw how much time it was going to take and said to myself: why create all this infrastructure and supporting software if the core interaction model for placing objects doesn’t even work?

I said to myself: “Let’s write a basic implementation so that I can continue with the cloud part of the application later”. And now, 5 months later. This “basic” implementation of a design app is still not finished.

Who would have guessed I had to reimplement the entire framework for creating XR interactions, just because I’m too perfectionistic and stubborn to simply use the MRTK? Who would have guessed that Unity’s own UI components are so bad I ended up writing my own? Even the sliders, toggles and buttons are written from scratch.

Code that is written and working is scary to change. But it’s important to change code or architectural decisions when it greatly improves the ability to add features later on (but only when you’ve clearly identified that those features are needed).

Lessons learned #

1. Never be comfortable. Don’t think anyone is going to buy what you’re creating.
2. Look at startups with a grain of salt. They can pour in money they didn’t make themselves to make it appear like they’re succesful, even though they might not be.
3. Technology, while initially appearing to be in this safe bubble of innovation, is also heavily influenced by larger societal trends, such as inflation, legislation and market demand.
4. I don’t want to be just a designer, I want to be a founder, software developer and designer in one.
5. Software takes a shit-ton of time to create. Especially if you have to implement your own VR interaction framework and UI framework.
6. Software is all about balance. With the end goal being code that is easily refactored, does not have different classes all being dependant on each other in circular dependencies, but also not too abstracted that there are too many layers of indirection. Clear and concise non-abstracted code without giant inheritance-trees is better. Almost everything can be solved in functions. Not everything needs OOP. And so on… It’s all about the art of knowing where to introduce new abstractions, when to introduce them and how to manage software that you’ve written in a time where you didn’t know enough about the framework to make the best decisions.

Conclusion #

I love writing software and I love creative software. The last years I have been continuously exposed to programming and creative software such as Adobe’s suite, Blender, Unity, and now everything feels like it’s coming together to allow me to create something truly novel, without being too constrained and limited by cynicism that comes with experience.