If your dreams occasionally involve spreadsheets that extend endlessly in all directions, good news: Researchers have developed a virtual reality spreadsheet interface that could expand Google Sheets and Microsoft Excel files from flat screens into 3D spaces, and rather than being nightmare-inducing, it could actually make spreadsheet apps more usable than before.
While traditional spreadsheets have been limited by the boundaries of 2D windows and displays, the research team envisions VR opening up adjacent 2D workspaces for related content, then using 3D for everything from floating menus to cell selection and repositioning. In one example, a VR headset mirrors the view of one spreadsheet page displayed on a physical tablet, while two virtual sheets sit to the left and right, permitting drag and drop access to their cells, and an overview hovers above the tablet.
Alternately, tablet-surrounding areas could display useful reference materials, expanded views of formulas, or the full collection of a spreadsheet’s pages, displayed as floating previews. Another possibility is a single spreadsheet page that stretches far further than the 30-degree diagonal field of view occupied by a typical tablet on a desk, utilizing more of the ~110-degree fields of view supported by VR headsets.
Fans of the film Minority Report and portrayals of similarly holographic future 3D interfaces will appreciate the team’s use of a floating pie menu — complete with a drop shadow on the spreadsheet — for selecting features and functions, as well as spherical rather than flat buttons, and other visual elements that appear to leap off the flat pages. The 3D spreadsheet workspace could also be extended with floating desktop objects, such as a virtual trash can, to make disposing of unwanted content more intuitive.
Interestingly, the project’s research team includes members of the Mixed Reality Lab at Germany’s Coburg University, as well as a professor from the University of Cambridge, and two principal researchers from Microsoft — mixed reality expert Eyal Ofek and UX engineer Michel Pahud. But their work isn’t limited to potential Microsoft applications: The paper’s author Dr. Jens Grubert tells VentureBeat that the cross-organizational team includes “long time collaborators,” and actually used Google Sheets rather than Excel for the backend.
In addition to an HTC Vive Pro VR headset and Microsoft Surface Pro 4 tablet, the researchers employed a spatially tracked stylus, enabling precision direct input for spreadsheet interactions while adding the freedom of in-air movement within a 3D space. Unsurprisingly, the VR spreadsheets can be used on existing commodity PC hardware, and the virtual UI was created with the Unity engine. Sheets pages are rendered within Chromium browser windows to match the resolution and size of the Surface Pro 4’s screen.
Full details on the project are available in the “Pen-based Interaction with Spreadsheets in Mobile Virtual Reality” research paper. If you’re interested in deeper dives, you can see a video of the project here, and a broader exploration of the team’s VR-tablet research here, ahead of their presentation at the IEEE’s International Symposium on Mixed and Augmented Reality from November 9 to 13.
