TREASURY SPATIAL COMPUTATION THESIS

Manoochehri Oct 23 v3

This document explores the spatial technology and computation context around the Treasury toolset and platform.

The aim of this whitepaper is firstly to put the specific opportunity of Treasury as a fingerprinting, licensing, archiving, distribution, and deployment platform for digital spatial assets into a technical and historical context.

Secondly, the aim is to contribute to the broader understanding of the technical, social and creative outlook for spatial tech and computation, including the so-called ‘metaverse’.

THE SPATIAL TECH CONVERGENCE

There are a range of spatial technologies that are technically maturing at roughly the same time, and starting to both overlap in their application and integrate into an infrastructure-quality technology layer on top of modern economy and society, in the way that internet tech has already done. This may be called the spatial technology convergence.

These spatial technologies leverage network, compute, device technology maturity - the internet technology layer that is now essentially mature and universal - but bring their own dynamics and opportunities into focus.

These technologies include

  • location and context-awareness:
    • mapping
    • geopositioning
    • GIS
    • geophysical tomography
    • computer vision
    • SLAM
    • pathfinding
  • reality capture:
    • lidar/time-of-flight depth-sensing
    • photogrammetry
    • stereo/time-offset photography
    • 360˙ photography/video
    • point-clouds
    • 3D semantic segmentation
    • neural radiance fields (nerfs)
    • medical/enterprise reality capture (MRI/CAT)
  • design & visualization:
    • 2D + 3D authoring
    • CAD, BIM, digital twins
    • generative and procedural design
    • topology and informational modeling
    • static render
    • animation
    • real-time render
    • agent-based models and environments
  • XR:
    • augmented reality
    • mixed reality
    • virtual reality

This convergence now has a name for its domain of applications: spatial computation.

The Apple Vision announcement of June 2023 offers a market centerpoint, and some kind of hardware vision for spatial computation, but it stopped short of pre-supposing the nature of the applications that will emerge from this new technology paradigm.

Below, we explore the application potential of spatial computing, and break it down into four grand spaces of application development.

SPATIAL COMPUTATION SPACES & THE METAVERSE

Prior to the Apple Vision announcement, the most prominent evidence of the spatial technology convergence was the so-called ‘metaverse’ concept and application marketplace.

The ‘metaverse’ concept springs from the same premise - that spatial tech is converging onto an infrastructural layer atop internet-enabled economy and society - but assumes a couple of very specific application end-points:

  • a lot of humans will experience deep technical immersion in digital spatial environments
  • those spatial environments, or some of them, will be ‘persistent’, and connected, in the way the physical world is made of persistent, connected spaces.

These two ideas, or rather presumptions, about spatial computation are what define much of the metaverse, and as such have dominated the spatial computation debate and practice fields until recently.

The enthusisasm for this framing of spatial computing arises perhaps less because of any particularly strong rationale that either is true or going to be true, but perhaps because it offers the promise of a simplified product and business pathway for technology development with the following steps

  1. create unified immersive environments for a mass of users,
  2. gatekeep their experience and capture the value,
  3. exploit the persistence of the environment and desire of user and groups (including business) to ‘own their identity’ persistently.

This is where much of the metaverse conversation in technical and popular conversation ends up.

There may be good reasons, other than slightly cynical suspicion that it is driven by grand commercial objectives, to believe in this two-fold - lots of people with deep immersion; persistent environments amounting to ‘another world’ - application of spatial computation, of course. There certainly are others - although usually speculative, if not creative (if not outright escapist).

And for these reasons, and given the spectacular amount of work put in to develop it, in particular by Meta/Facebook and Epic Games among others, we will have to take it seriously.

We might call this vision, this space of application development for spatial computation, the Persistent Spaces (or Metaverse) domain. But though this space is the most aggressively promoted, indeed the space that is claimed as ultimate destination of spatial computing, it is not, so far, the end, and may not even be the beginning, of what society does with its spatial technology convergence.

There seem to be a few, perhaps more credible - or at least more universally approachable - alternative application spaces for spatial technology emerging.

They are based, just as Persistent Spaces, on the convergence of the spatial tech convergence to a deployable, general infrastructure, but are end up in, or at least focussed on very different applications and application premises.

In part, in any case, if the Persistent Metaverse is to be real, only a few companies will ever lead it (because so vast in technical and business requirements, and so much winner-take-all character) - and so other companies should look for additional applications of the spatial tech layer that is emerging, if only to have business opportunities all of their own.

The first, and perhaps more practical and universalizable vision of highly spatialized experiences, is ad-hoc Temporary Spaces.

Rather than pointing to a somehow-inevitable set of legible, high-value highly contained, persistent spatial experiences and associate super-platform opportunities for vast user groups, it seems the spatial tech convergence is at least right now facilitates more readily temporary experiences of digital immersivity for (usually much) smaller groups.

These might grow into larger environments, but at least through their ephemerality will remain plural. Neither the environments, nor the user identities, persist for very long, and indeed new ones need to be created all the time - and so the ability of stable groups to form is also limited.

Currently, these Temporary Spaces are available in various contexts for various purposes: events, product launches, casual games, sales and much more. Technically, the opportunity is for developers to create digital-space-on-demand products, or even digital-space-as-a-service propositions. (We can easily and conveniently use the phrase ‘metaverse-on-demand’ and ‘metaverse-as-a-service’ here, but this is, in the logic of this whitepaper, to give too much or at least premature credence to the idea that temporary spaces will be /become/ permanent digital spaces.)

Fuller opportunity for the spatial tech convergence beyond Persistent and Temporary Spaces is somewhat immature.

What is known is that a massive new wave of innovation is unfolding: explore the experiences, platforms, products, form-factors, monetization models, real-world linkage, social and cultural frames, user psychologies and incentives, based party around the individual and combined capabilities of these techs themselves, but also on the ‘spatialization’ of existing technologies, industries, experiences and cultural forms.

Within this innovation explosion, there two other large terrains of spatial technology application seem most promising.

The first is Film Spaces. The main use case is more and more complete use of spatial technology in the CG workflow of live-action movies, above all Virtual Production (VP) in the pre-production and production stages, in addition to but also in preference to classic CG compositing in the post-production stage. In addition, there is the use of spatial technology for fully digital animated movies.

In VP (unlike most post-production CG) there is a detailed CG model used for some part of the live-action shot, either as a virtual shooting environment which is visible in some form to actors, director, crew, or as a fully rendered background to live-action shots. All of this relies on new tools, techniques, software that come from various different sectors - hence proving the spatial tech convergence premise.

In VP on a live soundstage, the size, shape, resolution and fidelity required to render quality live-action 2D images (from 3D models) suitable for capture in-camera is a result of huge advance in flat-panel screen tech; and the technology that renders the images filmable in real-time is directly derived from video game engines that have mastered high-resolution real-time rendering.

The second remaining area of application is a catch-all which we might call, for simplicitly, Technical Spaces.

This is a catch-all category for any spatial technology deployment at scale that is focussed on technical (rather than creative) problems and solutions.

One somewhat evolved centre of this strand of innovation is the development of so-called digital twins: 3D models of real-world environments embedded with rich data (including live data) and analytics, to guide various applications including management, energy and materials optimization, safety, business strategy and more.

It seems likely that Technical Spaces, while the least compelling, will be by far the most universal and arguably valuable, since the potential is there to transform it not entirely disrupt countless industry verticals and research domains. As a tiny slice of potential, beyond digital twins, the opportunity for spatial technology to revolutionize, say, health and medicine in countless subdomains is huge: context-specific health monitoring, remote surgery, medical education, and more.

In sum: as spatial technologies mature - becoming stable, affordable, small, low-powered, integratable - they appear to converge on a new infrastructural layer to the existing, mature internet technology layer that is transforming economy and society so profoundly.

But: it’s not clear they are converging, or could converge, on a single or unified set of use cases, products, experiences.

These four spaces of application of spatial compuation represent just a convenient, temporary shorthand for a grand transformation in means, ends, value and experience, society and culture, that has hardly begun, and will likely - as with the internet - go beyond any pre-emptive framing, technical or business or other, that we could generate now.

We certainly are not simply, if at all, going to apply spatial computation to developing the ‘metaverse’, engage with it as a global society together willingly and unthinkingly, and that will be that.

This assessment - of spatial tech converging on a pervasive, infrastructural layer like the internet, but not converging on specific uses or applications - is Treasury’s founding vision and may be called the spatial computation thesis.

Spatial computation is happening, everywhere, to everyone and will continue to do so. But this is isn’t (just) the metaverse.

AXES OF IMMERSIVITY

Over time, a lot of different ways will be developed to characterize the spatial computation domain, beyond simply the combination of technologies used for any particular experience, and certainly beyond the broad outline of application spaces presented above.

There is one vector of analysis, however, that seems universal and durable - relating to the immersivity of the spatial experience enabled by the spatial computation domain.

Looking at the convergence of spatial tech on an infrastructural layer, that is the emergence of spatial computing, through an immersivity lens is instructive, if only to create more differentiation than is possible under much contemporary ‘metaverse’ discourse.

One of the two foundational premises of metaverse-specific framing of spatial computing is, as mentioned, the idea that metaverses are deeply, experientially immersive, in a technical sense.

Metaverses, we are supposed to believe, do and must somehow ‘fully’ involve the users in the experience offered. ‘More immerseion’ somehow means a ‘better experience’.

Testing whether this is true or not, or what it even means, will help us understand the implications and opportunities of spatial computing, that is the application of spatial computation very broadly, and perhaps restrain or enhance the particular value of the metaverse, or Persistent Spaces area of application.

If we investigate what we really mean when saying or assuming that the metaverse/metaverses are based ‘immersion’ to achieve a valuable experience, two aspects of immersivity immediately become clear:

technical immersivity, and

narrative immersivity.

Technical immersivity is the physical and technical engagement of a user in a technologically-enabled experience - such as increased sensory (largely but not solely visual) immersion in a digitally-imaged environment.

Narrative immersivity implies non-technical engagement dynamics, measured in terms of time spent, emotional investment, social density and dynamics, non-leisure experiences, or simply ‘attention’, involving roles/identities, goals/incentives, and possibly new interaction mechanics with the environment and/or other agents.

With just this distinction in mind, it’s clear that much of what is offered as a metaverse experience is increasingly technically immersive - but it’s not so obvious that it is as equally, or even at all, narratively immersive.

A VR experience that is fully technically immersive - because it takes over your field of view with incredibly detailed views at all scales, offers surround audio, captures hand gesture, even body movements - is available to us, on many platforms.

But how much do these digital spatial environments capture your attention in a narratively organic way, rather than through distraction gimmicks?

Defining ‘narrative immersivity’ in a sophisticated way involves a claim that human attention, indeed value and preference, is primarily driven by ‘narratives’ relating to the self and others. While that’s a relatively established framework, to build that case fully is beyond the scope of this whitepaper,

If we need to supply a quantitative foundation for the role of narrative in ‘immersivity’, we could present this: how interesting is something in narrative terms, that is in terms of its narrative content, and for how long, whether in one session or across many sessions, does it sustain your interest?

Put more pointedly, do people play video games for scores of hours because of technical immersivity, gaining ever more value from the verisimilitude of the digital environments? Or do they continue to play video games because of their unquenchable thirst for narrative immersivity - where their roles, identities, groups, goals, failures, processes are all unfolding?

If technical immersivity were so important, so central, why would people play low-resolution true-3D games, with poor verisimilitude, or 2D games, such as mobile games, at all?

As technology achievements, the sensory immersivity in artificial environments is fascinating and profound - but if you want to know why VR has stubbornly refused to become popular let alone mainstream a solid suggestion would be that after the wow factor, it gets boring very fast, because the narrative immersion is low or non-existent.

We may say, besides, this so far has been the principal failing of the metaverse concept: deep technical immersivity, coupled with persistent spaces, is simply not enough to sustainably capture people’s attention at scale.

There’s many dimensions of immersivity one might add to these two - technical and narrative - to build out a more complete picture of how the spatial web may play out. But note that additional ones will likely be powerfully subject to these two.

If, say, we were to ask is ‘social immersivity’ present in a given metaverse, in many cases the answer is yes - there are other ‘people’ (eg avatars of other real people) there - but they are still not that interesting (narratively immersive) because of something about how these ‘people’ are supposed to manifest, or behave, interact. In other, words just adding other dimensions of immersivity, doesn’t necessary solve what may be the primary one: are they psychologically rewarding, on a sustained basis, are they fundamentally interesting and engaging.

In any case, in observing that spatial computation spaces have at least these two dimensions of immersivity - technical and narrative - and that there are different levels of immersivity for each, we may say that the spatial web has at least two axes of immersivity: from more to less immersive, which moreover may not be correlated to each other in any particular case.

This outlook on the spatial tech convergence:

  • a pervasive infrastructural layer sitting on top of the existing pervasive infrastructure layer, internet
  • at least four emerging application spaces - Persistent, Temporary, Film, Technical
  • at least two, non-correlated, axes of immersivity - technical, narrative

gives us a vastly bigger domain, with at least some more clarity, of the ‘metaverse’ or ‘metaverses’ might be - other than simply a widespread social shift towards technically immersive VR, which is where much of the public debate and tech product discourse has ended up.

Instead of tracking and critiquing the applicability and value of technically immersive metaverse with poor narrative immersivity, we can analyze the demand for spatial web experiences with much more nuance, and discover and innovate in a much more precise way. Let’s explore some of this across the various metaverse domains.

DIGITAL MAPS AND POKEMON GO: TWO STARTING POINTS

Defined as a above - a pervasive infrastructural layer of cheap, stable, integratable, spatial technologies, with varying levels and types of immersivity - the domain of spatial computation may turn out to be far more universal and subtle in potential already that we currently maybe realize.

Indeed, it may help us understand how omnipresent aspects of the spatial computing revolution already are. Below are a couple of starting points to explore spatial computing and its potential with a very different set of premises than metaverse-oriented, ie persistent immersive alternative digital spatial environments.

Digital mapping is one starting point. Digital location-finding and mapping is one of the most universal internet applications at the present time, present in some form for everyone with mobile computation - which is most people on this planet.

Using a handheld device, your position, or the position of something you are looking for, is presented on a digital map, very often with extreme degrees of precision. The extent to which this is technically immersive depends on what kind of device you are using, but most people would agree that a digital map - at least compared to any other kind of technical immersion - has low level of technical immersivity.

But the technical immersivity is not primarily what makes digital maps so valuable: it’s their narrative immersivity that brings them alive, and makes them literally universal to the modern human.

In the first instance, knowing where you are, or where something else is, is both fascinating and often very important - and if where we are and what we want are narratively important to humans, then geopositioning is a huge contributor to narrative immersivity, and thus interest and engagement. But even more powerful to the human narrative engagement than where we are and what we want is: where we are going, or have to go, which digital maps facilitate through turn-by-turn real-time directions.

When you have to get to work, or are late for a favorite concert, or to the hospital or the site of some other emergency, the narrative immersivity of a digital map goes from mild, in the case of a commute, to major in the case of the concert, to absolute in the case of a medical or other emergency. If it’s a matter of life and death, your entire available attention will be dedicated to navigation, inside the ‘metaverse’ of a digital map.

Perhaps ironically, the addition of more technically immersive features - such as satellite mapping, let alone 3D models - is likely to decrease rather than increase the narrative immersivity, since they don’t help with the essential goals of navigating, at least in many cases.

In addition to help us understand how universal spatial web applications already are, and how not just uncorrelated but inversely-correlated technical and narrative immersivity might be, this example of how mapping is deployed gives us an insight into the rest of the spatial tech convergence.

Most mapping tools are infrastructure elements in other products, not map products in their own right.

In fact, it is hard to launch a new map product, given the omnipresence of quality mapping in every domain. New mapping products are only really credible as infrastructure components. This validates the spatial web thesis as a convergence on infrastructure, not product per se.

To take another example of spatial technology convergence, and see how differential spatial computing is evolving from the metaverse narrative, let’s consider AR-based geolocated gaming - such as Pokemon Go.

In Pokemon Go, the AR viewer (usually phone) has to be able to render a character’s form in a specific 3D spatial environment (requiring dimensioned environmental capture, viewfield occlusion and more) AND position the individual user’s interaction in geolocation terms, as well as that of other players. Clearly, this is describing is a convergence of spatial technologies on infrastructures which allow the game experience to be built.

And yet: none of this may lead to a user being shown an actual, classic map or being invited to adopt new hardware technologies.

The AR views may include both geolocation and wayfinding without an explicit map reference. This means that ‘mapping’ has essentially become an abstracted, but no less essential, layer of the product, in the way that ‘servers’ are increasingly abstracted, and still essential, parts of general computation stacks. And the AR viewer may be simply (and currently already is for Apple + Android-based phones) a built-in part of the technology stack, not a new device to get and learn.

Here, we observe that while there is certainly far more, and far more powerful, technology involved in serving the overall experience to the enduser, it is still the power of the narrative experience - the user’s progress in game, their location in relation to their fellow players, their interaction with the real world around them with all it’s particular meaning to that user - that is driving the overall value, and not the fuller technically immersive promise of metaverse VR headsets and the rest.

These starting points are presented simply as seeds of reflection to further development on the basis that: the spatial computing domain is not the same as the metaverse, and narrative immersivity trumps technical immersivity.

OUTLOOK & THE TREASURY SPATIAL ASSET ECOSYSTEM

With these small framing cues - four spaces of spatial computation application, two axes of immersivity built on pervasive, infrastructural technology - we have a manageable way to explore and unfold the enormity of the spatial computation domain, with more optimism (and more precise ways to activate caution) then simply assuming that spatial tech, and thus the metaverse concept, is inevitably devolving to simply a productized, platformized super-technically-immersive VR experience.

This is an outlook of immense potential, as great and perhaps greater than the internet itself, given the centrality to human society, cognition, science and imagination of the visual, that is to say spatial, environment.

And within this outlook, the role of the Treasury, to provide a digital spatial asset ecosystem, is as clear as it is vast - and helps Treasury define the best ways to contribute.

Here’s some of the initial guidance that seems to be available:

  • Treasury’s focus of being an asset store for premium digital spatial assets is likely at least in the medium-term to find most value as an infrastructural play - not as a consumer offering, or even over-packaged commercial offering.
  • the market for digital spatial assets will be fragmented across use cases that differ widely in terms of technical and narrative immersivity, and being aware of these differences is crucial in setting a technically-efficient and scalable offering - not every asset needs to be a high-resolution render
  • value from the Treasury asset store and syndication model may exist beyond the IP-protection focus of asset use in creative metaverses - in Technical Metaverses, rather than simply licensing the 3D design IP of premium architectural assets, another value stream may come from abstracting average floorplan statistics, or other spatial or engineering (including environmental, such as daylighting/energy/fluid flow)

Many more pointers will be drawn from this thesis over time, to ensure that the Treasury spatial asset tools, platform and partner ecosystem has the most robust and relevant possible development path, and contributes the most value to everyone involved.

In short, as the spatial tech convergence unfolds, the opportunity for Treasury to be a trusted, infrastructural store and syndicator of not just 3D digital spatial asset IP - but all potential value streams from quality 3D digital spatial assets - is very significant at this time.

This analysis note is intended to show that while the spatial web is both very wide and deep, with vast and unknown application potential, it is also legible conceptually, technically and culturally - and as such, Treasury will share analysis on an ongoing basis to help make the opportunity widely addressable.