Into the Metaverse: Challenges in Hardware and Software

Image by Jan Vašek from Pixabay

Let’s take a look at one of the greatest phenomena in technology today, the metaverse, and in specific what drives it; its hardware and its software, what is lacking in both today, and how both can be improved to pave the way for a better tomorrow. 

If you’re just joining us now at Beyond Enterprizes, we invite you to see our previous article on the metaverse for a general understanding of this exciting technology and how it is already being used to make life better for so many in a variety of industries and applications, worldwide. 

About hardware and software, let us consider this important point: the two have a synergistic relationship and cannot truly be separated as far as their influence upon each other, hence the potential of hardware affecting new ideas in software and vice versa. Often, new ideas for use cases of VR have been necessary in garnering new funding, always essential for new developments in both hardware and software projects. 

Two steps forward, one back: History of the metaverse

The history of virtual reality (VR) is fascinating as it demonstrates the evolution of how people conceptualized the creation of parallel, inclusive and interactive new worlds. Let’s take a look at a few early entries as they include some inventive and ambitious structures, signalling the trajectory of what has become the metaverse, with a view to what it can be in the near future.

Stanley Weinbaum, American science fiction author, described a VR experience in his short story “Pygmalion's Spectacles” in 1935. In the story, a professor invents a pair of goggles–the representation of a Head Mounted Display (HMD)–enabling "a movie that gives one sight and sound...taste, smell, and touch. You are in the story, you speak to the shadows (characters) and they reply.”

Illustration from “Pygmalion’s Spectacles,” published in 1935

Considered one of the earliest attempts at an immersive VR experience, or "Experience Theater" as its creator referred to his invention, the Sensorama was presented in 1962 by Morton Heilig, along with 5 short films or what we might consider attempts at metaverses. Heilig’s multi-sensory simulator included a chair that could tilt and vibrate to mimic real life experiences, full-color, stereoscopic 3-D image projection in a wide-angle view, binaural sound, various scents and rushing air to simulate wind. Though the system had myriad unique features, ultimately it was not interactive, and the project failed to garner sufficient investment for production. The idea however, had been planted in the collective imagination of society and it would grow through the years.

Illustration of the Sensorama, U.S. Patent #3050870

Heilig even patented the “Telesphere Mask,” the very first head-mounted display (HMD) showing 3D images with stereo sound. There was, however, no motion tracking in the Telesphere Mask headset. 

Computer scientist Ivan Sutherland created the first augmented reality HMD system, named The Sword of Damocles in 1968. It was connected to a computer, not a camera like the Sensorama, but could only project virtual wire-frame shapes in 3D that changed perspective when the user moved their head due to the tracking system. They were never developed commercially since the model was too heavy for comfortable wear. The units were so heavy in fact, that they were suspended from the ceiling.

Comfortable, lightweight, breathable headwear in VR continues to be a challenge as most users simply cannot wear an HMD for more than 2 hours comfortably. This issue will continue to be improved upon as a necessity for mass adoption of the metaverse.

Photo of Ivan Sutherland’s “Sword of Damocles” structure

In 1972, General Electric Corporation contributed to aspirations for valuable use cases of AR by creating their computerised flight simulator–a popular and practical use case– featuring a 180º field of vision with three screens surrounding the cockpit. 

More realistic experiences would ensue. Simulation for the handling and finessing of large machinery has become one of the best and most in-demand use cases for augmented reality for both military as well as commercial vocational training. 

Foreshadowing Google’s Street View application, Aspen Movie Map was created by MIT in 1977. This program gave users the virtual experience of touring Aspen, Colorado in their choice of summer, winter or polygons mode. Though it did not utilize an HMD, it was interactive and showed VR’s potential for virtual teleportation, paving the way for future use cases even further.

The military has, and continues to play a vital role in VR development for work and training experienced remotely as well as for creating new scenarios for soldiers to navigate. In 1979, McDonnell-Douglas Corporation integrated VR into its VITAL HMD helmet, for military use. A head tracker followed the pilot's eye movements to match computer-generated images. Today, military supplier Baesystems boasts having the “world’s most advanced helmet-mounted display” with daylight color display and integrated night vision.

In 1985, VPL Research, Inc. founded by Jaron Lanier and Thomas Zimmerman, both originally from the gaming company Atari, was the first company to create consumer-grade VR devices. They developed groundbreaking equipment, including the DataGlove, EyePhone HMD and the Audio Sphere. Lanier was reportedly the first person to coin the term “virtual reality.”

Photo from a 1989 demonstration of the EyePhone system with DataGloves. Objects could be seen and moved in a computer created environment.

Lanier also developed a full body “DataSuit” with tracking sensors to detect movements of one’s arms, legs and torso. Lanier and his company VPL set the course for today’s VR, but seemed to be simply ahead of their time. Many of their fundamental principles remain, although the computing power available at the time was not sufficient to support their advanced concepts of a metaverse. 

Software in accordance with Moore’s law has now evolved to a much higher level for supporting speed, precision and the variety of options available to users of VR. Yet in accordance to a society with an ever-expanding appetite for realism and/or novelty–new worlds, scenarios, creatures, avatars, prizes, pitfalls and the like– new possibilities are imagined even faster than Moore’s law might accommodate software advances to bring them to life. 

VPL declared bankruptcy in 1990. Still, in 1992 the SciFi movie “The Lawnmower Man” featured the DataSuit and EyePhone. VPL eventually sold all their patents to Sun Microsystems in 1999.

VPL’s DataGlove evolved into the Z Glove which was eventually licensed to Mattel who sold 1.3 million Power Gloves, based on Z Glove technology. It was quickly declared one of the most coveted gifts of the 1989 holiday season, but disappointed users with its limited functionality and lack of precision. Users found its physical one-size-fits-all often clumsy, and in performance, its response time was too slow for the Nintendo games of the day, an issue of software as well as hardware.

In 2012, the 18-year-old “Palmer Luckey” prototyped the Oculus Rift $300 headset, and proceeded to raise $2.4 million with its Kickstarter campaign. Oculus was famously bought by Facebook in 2014, sparking much speculation on the future of VR, its scope and influence with such a social media behemoth invested.

Soon after, Sony announced Project Morpheus, for PlayStation, promising a high resolution and low frame rate visual display. Samsung followed with the Samsung Gear VR pairing with the Galaxy smartphone. Haptic interfaces were developed in 2016, as HTC VIVE announced their SteamVR headset. 

These various hardware and software advances all contributed in their own way to help facilitate the popularity of VR in 2019, and the high level of interest in the metaverse today. It is a rich and storied history of major advances peppered with minor setbacks. 

Many challenges remain today. Hardware is still not ready for full day use and therefore cannot be seen as a full corporate play. It is conceivable in the near future that specially equipped contact lenses as well as sound units as small as hearing aids could replace headsets. Going a step even further is the potential for neural implants to receive and transfer data with virtual worlds.

Image by Lars_Nissen from Pixabay 

Blockchain's Role in the Metaverse

Addressing issues of interoperability and viable rewards systems for games and other applications of VR, blockchain, crypto and NFTs will continue to play a greater role.  

As VR games will ultimately invite an endless number of creators to contribute to the metaverse for diversity as well as optimization, an open source, decentralized system is the logical choice. The centralized paradigm is broken, and the ability of Blockchain to grow in proportion in power, with the needs of various users, is a clear advantage.

Crypto or another kind of virtual money has been used in virtual worlds such as Second Life for over ten years. The advantages of this practice will continue and grow as phenomena like the substantial income gained by Axie Infinity players may attest to. 

NFTs will continue to be relevant. Any immersive, 3D environment, with its 3D objects, requires the ability to uniquely identify these objects and their interactions. The ability to provide that feature to users, so a unique object and its digital identity have the digital analogs of ownership, transferability, nonfungibility, provenance and new features that are harder to implement in the physical world, will be imperative to the new metaverse. 

For example, Second Life offers beautiful enhancements for digital objects, where a user could sell a new “skin” for one’s avatar, paid for in Linden Dollars, the currency of Second LIfe, which are convertible to fiat currencies. This incentivized market inspires creators of new skins. An interesting option was also introduced to this market: a creator could choose whether to allow a skin to be transferable or not, so the buyer may or may not be able to give or sell the skin. Another option to consider is whether or not copies of the skin are allowed. If copies are allowed, then purchasing the skin might be more like licensing a product for resale or redistribution. Lastly, if a digital item like a skin, is transferred from one user to another, would the original owner still be able to keep a digital copy or would the original owner surrender these rights? These options and much more are possible through smart contracts via Blockchain. 

One of the greatest lessons of the Blockchain is how interoperability persists at the forefront, due to economic incentives; the ability to buy and sell a token or an NFT. Following this economic incentive, no one wants to be stuck in a silo; we want our tokens, NFTs or any other new reward to be available everywhere, via interoperable systems. Any metaverse that is not decentralized, and is not based on NTFs will not sync with what modern software architectures enable today. 

Software issues will continue to improve but are currently not at pace with the expanding ideas of the metaverse. All of these, of course will continue to evolve. We invite you to follow with us at Beyond Enterprizes as we watch this exciting space and continue to report on its new achievements and possibilities for the future.

Image by Gerd Altmann from Pixabay