1. Introduction

1.1. Essentiality of Narrative in Architecture Design

Schematic design is the initial phase of the architectural design process, where this phase involves architects presenting the architectural and spatial design and the tectonic elements to their clients. Consequently, the delivery of ideas and messages and the connection formed with the clients during the architectural design presentation is crucial for architects. In fact, most architectural design has a narrative essence. A narrative describes a sequence of events involving characters and messages or themes [1]. Narrative in literary works reasonably clarifies the story by detailing the context and the characters’ living environment. On the other hand, narrative in architecture relies on the historical context, regional culture, and careful consideration of the morphological characteristics of the site. Architects also use figurative elements such as color, light, and water to build the heterogeneity of space, create a different spatial atmosphere, and link them in a specific spatial sequence. Emotion is also significantly involved in architecture design. Architects use an Artful space layout, sensible flow arrangement, in-depth deliberation, and integration of details and materials to stimulate emotion. Moreover, associative plot narrative is common in some current architecture designs [2]. The architects allow people to experience the events of different places in a sequential movement and feel the gradual development of the storyline.

1.2. Current Problem of Presenting Architectural Design

The typical medium to present architectural design is through drawings on paper and 3D models shown on plain computer surfaces with textual explanations. Architects use computers to illustrate the drawings and build up the models, adding descriptions and narrations explaining the whole thing before handing these presentations to clients. These media of architectural design presentation restrain the narrative aspects of the architecture and the emotional delivery. The designs are static, and the audience cannot interact with them, so it is hard for the audience to form any in-depth connection with them. Also, the narrative and visual aspects of architecture are usually separated, causing the design to form any resonance with the audience hardly. Moreover, the clients cannot have an intuitional understanding of the spatial arrangement and the impact of details and material design. Lastly, it is impractical for the audience to understand architectural design with an associative plot narrative without having an immersive experience.

1.3. Problem Statement

The problem is that many architects are not satisfied with the current architectural design presentation because they cannot successfully communicate the narration and artistic expression to their clients due to the disassociation between narration/emotion and the design. In addition, when present-day means are used, it is hard for architects to clearly express the details of their design ideas and the actual living experiences of the designed building.

1.4. Solution to the Problem in Brief

We proposed a kind of interactive VR film involving future technologies to help solve this problem and thus allow architects better exhibit their designs. The VR technologies in these films can simulate the virtual world via many senses instead of just through sight and sound. These films can also give people multiple degrees of freedom, which refers to letting people do any movements while feeling the presence of their bodies. When using the technology described above, the presentations of architectural designs will no longer be unreal and ever immersive. In the later parts of this essay, we will explain the fundamental theorems of this technology in detail [3].

1.5. Reasons for Using the form of Interactive Film in the Solution

Interactive film is defined as interactive media that has characteristics of a cinematic film. Interactivity is a necessary factor for the experience to be realistic, and it also elevates immersion. At the same time, the film is the most suitable medium for creating narrative immersion. The film can guide people to see all the essential parts of a building. If the clients have not been given any instructions, they would not know what a correct visiting order is and may miss specific things that the architect wants to show. The narrations in the architectural design presentations are essential because clients can best understand a design only by following the designer’s logic and narrative order. With the instructions given by the form of film, architects no longer need to worry about misunderstandings of the narrative sequence since the audience will likely perceive efficient narrations through an immersive and interactive experience. On the other hand, if the whole process is designed to be a traditional film without any interactive elements, the clients will be hindered from checking out some details they want to learn. Accordingly, an interactive film is a satisfying form to use.

2. Advantage of VR in Architectural Design Presentation

Before digging in to obtain more details about how the future interactive film will operate in theory, it is crucial to first look at the current situation of VR and its applications in architecture and films.2.1 Current state of VR technology

Virtual reality (VR) technologies have experienced rapid advancement in the past decade, strengthening the sense of presence in the immersive virtual environment. Compared to two-dimensional, three-dimensional, or 360˚ audio-visual contents, VR technologies intensify the spectators' perceptual immersion, creating a more impacting viewing experience [4]. This advantage is highly beneficial in architectural design presentation because it has the potential to solve the restraints of current presentation software, including the visualization of scale and medium and the presentation of dynamic design.

Studies have also shown that users have a better spatial perception in the immersive virtual space than in traditional methods. The greater accuracy in identifying distance and space enhances the users' understanding of the three-dimensional model and its spatial relationship in real-life [5].

Virtual reality technologies also have significant advantages in interactivity. The difference between interactivity in VR and traditional interactive multimedia is the disappearance of the interface, where the users can directly manipulate objects in the virtual environment. Therefore, the application of VR in architectural design presentations allows the clients to have close-to-reality interactions, possessing a deeper understanding of the architecture in the real world.

Moreover, VR has the potential to form a more intimate connection with viewers by evoking stronger emotions. A study from Fonseca suggests a positive correlation between the sense of presence and the involvement of emotions [6]. The correlation is crucial for architectural design presentation because there tends to be an emotional message behind each architect's work, and conveying human emotions is difficult to achieve through traditional methods of architectural design presentation. Consequently, VR can build an emotional connection between the architecture and the clients.

3. The Current State of VR in the Field of Films and Architecture

3.1. The Current State of VR in the Field of Film

The first immersive VR film was released in 1992 by Nicole Stenger, where Stenger achieved interaction through goggles and a data glove [7]. However, since VR has only been to the Trough of Disillusionment stage of the Gartner Hype Cycle for Emerging Tech [8], VR film is still at its early stage. 

The experience of current VR film can be separated into three major categories by different degree-of-freedom (DOF): 3DOF, 6DOF, and MDOF. As DOF elevates, the users can experience more immersion, interactivity, and a sense of presence. 

Three degree-of-freedom (3DOF): VR Film with 3DOF is a comparatively mature technology in the field. It allows the users to rotate their heads around X, Y, and Z axes to achieve a 360° panoramic view in virtual space; however, users have a fixed viewing position in the virtual space [9]. The usage of hardware for VR film with 3DOF can either be a mobile phone or a head-mounted display (HMD). In architectural design presentation, VR on mobile is most common and dominant due to its convenience, but immersion and interactivity are limited. HMD is also common for the user to visualize architecture, and it is more immersive than mobile VR, but 3DOF limits immersion as movement is restrained. 

Six degree-of-freedom (6DOF): 6DOF allows the displacement along the X, Y, and Z axes on top of 3DOF. In a VR film with 6DOF, the position tracking technology enables the user to become an avatar and walk freely in the virtual space [9], achieving a higher immersion level. However, interactive VR film with 6DOF is less common in architectural design presentations due to the technical difficulties of attaining locomotion. 

Multi-degree-of-freedom (MDOF): The progression from 6DOF to MDOF relies on motion capture technology [9]; therefore, users in VR interactive film with MDOF can perform non-displacement movements such as fist grasping, arm shaking, shoulder twisting, and body tilting in the virtual space. Due to VR interactive film with MDOF is still under development, architects have not yet adopted it in the field of architectural design presentation. 

The significant limitations of VR film used in architectural design presentation are the level of immersion and interactivity. 

3.2. The Current State of VR in the Field of Architecture

Because of its immaturity, VR has not yet been used widely in the architecture field. Currently, most architects mainly use 3D models and digital drawings to exhibit their works instead of VR [10]. However, VR is slowly thriving in the field and is gradually becoming familiar to more and more architects. At this point, the primary way for architects to use VR in their design presentations is through applications that transform 3D models or even 2D drawings into VR scenes. Such VR scenes usually create relatively immersive experiences but provide no more information than a 3D model. People cannot feel the actual living experience of a building from current VR but can only learn about the structure. Another problem of current VR which hinders it from quickly becoming popular in the architecture field is that the cost of VR is still high [11], and some architects believe that spending money to buy VR headsets and VR software is not worth it because current VR provides nothing but a more immersive view on the structures, which does not sound necessary and helpful.

4. Proposal of the Design Solution

4.1. The Detailed Concept and Blueprint of this System

The VR interactive film provides a sensory and narrative immersive way for architects to present their designs to the clients. The film will have a predetermined order of narration and predefined travel path for the audience. The immersive experience can solve the problem of ineffective display of narrative aspects in architectural design because it enhances the emotional connection between the user and architects, and it can project the storyline vividly and comprehensively. The VR in this interactive film not only provides visual and audio simulations but also generates the feeling of touch, forces, moving on stairs, wind, and temperature. It can also simulate different weather conditions to make the environment realistic and changeable. Smell and taste are less relevant in architecture design presentations, so they will not be included in this technology. In addition to simulations in senses, this VR technology gives people an MDOF experience, making the resulting experience immersive and close to reality. In the functions mentioned above, the feeling of wind and temperature will be carried out through fans and air-conditioners, the forces and touch perception will be realized mainly by using haptic feedback technologies, the movement problems will be dealt with locomotion technologies, and the MDOF experience will be generated through the cooperation of numerous hardware. Part of the technologies used here do not have prototypes yet but are based on present technologies and might be prototyped soon.

4.2. The Specific Technologies

4.2.1. Haptics Feedback

Realistic haptic feedback is a critical factor of high immersion in the virtual space, and it is achieved through a combination of diversified tactile feedback and real kinesthetic feedback. The tactile feedback is responsible for presenting the materials and texture, and kinesthetic feedback enhances the interactivity through displaying directional force. The haptics feedback allows the user to acquire realistic sensations while performing interactive activities in the VR film, including touching the furniture, lifting objects, opening doors, and walking on a particular surface.

Tactile feedback: The mainstream technology for tactile feedback is through force feedback or vibrotactile feedback [12], and they have a common feature that the physical movement of the hardware, such as vibration, compression, and thrust, generates the tactile feedback. The diversity of tactile feedback is limited, and it cannot simulate the multitudinous material and texture in architectural design. We solve this issue by adopting electrotactile feedback. The study from Kourtesis et al. [13] shows that electrotactile pulses can stimulate four types of mechanoreceptors to elicit tactile sensation: Meissner's Corpuscles—low-frequency vibration, Pacinian Corpuscle—high-frequency vibration, Ruffini Endings—skin stretch, and Merkel's Cells—pressure and texture. To stimulate electrotactile feedback, a stimulator produces an electrical current. The current flows from the cathode that passes through the skin layer, stimulates nerve endings (the mechanoreceptors), flows to the anode, and travels back to the cathode, forming a circuit. The stimulator can modulate the pulses' frequency, amplitude, and width and apply different stimulation to the mechanoreceptors, rendering different intensities and sensations. The different combination of stimulation onto the four types of mechanoreceptors allows the user to sense different materials and textures.

The two involved haptics hardware in the design for stimulating tactile feedback are haptics gloves and haptics shoes. For haptics gloves, the epidermal electrodes with the size of 10mm2 will be placed onto the inner side of the gloves [13]. This way, the palm and the fingers can stay closely in contact with the electrodes. The distance between the palm, finger and fingertip electrodes would be 7.73mm, 7.25mm, and 2mm, respectively [14]. For the haptics shoes, the electrodes will be placed on the insole of the shoes so that the sole of the feet and toes can receive haptic feedback.

Kinesthetic feedback: Force feedback system is ideal for kinesthetic feedback because it is most suitable for generating directional forces. The two types of systems for kinesthetic feedback in the current market are grounded and exoskeleton-based devices [15]. We have adopted exoskeleton-based devices because it allows the user to move around the space freely, and it is also more convenient.

In our design, kinesthetic feedback is realized by cable-based actuators. We place the motor on the upper back of the user, and the motor connects to a series of torque control. Each torque control is attached to a cable, manipulating its twisting force. The other end of the cable connects to fingertips, wrists, elbow joints, and shoulder joints to admit or impede their movement, stimulating different types of motion. The torque can also perform y-axis and z-axis rotation to display directional force from a different direction. The motor can modulate the torque control's rotation, rotational direction, and rotational speed. When the user performs actions like grabbing, pinching, or compressing in the interactive film, the torque control shortens the length of the cable and applies forces to the user's joints or fingertips. When the user is lifting objects in the virtual space, the rotational speed of the cable varies based on the mass of the objects, thus providing the user with realistic weight perception.

4.2.2. Locomotion Techniques

Users' perception of distance is an important benchmark for locomotion techniques that the user should perceive distance in the interactive VR film similarly to distance in the real world to create an immersive experience. Based on the study of Keil et al. [16] that the distance perception is an integration of three major types of sensory information: proprioception—information provided by proprioceptors that locate in joints, muscle, and skin; optic flow—visual information on the motion between the observer and the environment; and vestibular system—"information about angular and linear acceleration of the head." [16]

The locomotion technique that can highly satisfy all three types of sensory information is room-scale locomotion. However, the working area of room-scale locomotion is limited by the room's boundary, and it is unrealistic to build a room-size space, particularly for architectural design presentations. Therefore, we adopt the locomotion technique that is second to room-scale locomotion to satisfy the sensory information—omnidirectional treadmill (ODT). 

We designed an active treadmill to create the ODT with realistic haptics and friction. Based on Kennedy's study on the compact packing of the sphere [17], we arranged the sphere as shown in figure 1 to minimize the space between the spheres. Unlike the regular treadmill, ODT also requires sideways motion, so in our design, the blue spheres are responsible for forward and backward motion, and the red spheres are responsible for sideways motion. The blue and red spheres vary in size, and the axles for the red sphere are on a higher plane than the blue spheres, creating a plane with a flat surface. The actual treadmill will contain sixteen times as the sphere is shown in figure 1, the large sphere has a diameter of 7 centimeters, and all spheres are made of high friction materials. The space between the spheres is negligible when users wear haptic shoes.

To achieve locomotion, the user starts walking from the center of the treadmill. The shoes capture the direction and the distance between the shoes. Each architecture design has a preset scale, the distance and direction moved in the virtual space corresponding to the user's motion on the treadmill. The spheres then roll in the reverse direction by the recorded distance, moving the user back to the center of the treadmill, and the walking motion continues.

Figure 1 Design of the ODT (top view and side view).

Another critical feature in architectural presentation is stairs, but the user cannot experience immersive walking motion in a vertical direction solely through ODT. Inspired by the project "Infinite Stairs" by Nagao et al. [18], visuo- haptics interaction with the angled bar placed on the floor can generate the sense of walking up and down the stairs. In our design, another one-meter regular treadmill with angled bars on its belt is located at one end of the ODT (see figure 2). When the user approaches the staircase in the VR film, the spheres of the ODT will rotate and move the user to the regular treadmill. The angled bar on the belt corresponds to the edges of the staircase in the virtual space, and the haptic stimulus immerses the walking motion in a vertical direction. When the user reaches the end of the staircase, the treadmill moves them back to the origin when they step onto the next bar, and it carries the user back to the ODT.

4.3. Hardware Design

Head-mounted display (HMD): A wearable display that projects the user with virtual images. The HMD in the system has a high resolution and refresh rate to provide the user with a close-to-reality image. The head tracking is 6DOF, and the angle of field is large. The HMD is also light-weighted, so it would not limit the user's immersion due to its clunk size. It also enables eye tracking to enhance the immersion because, in real life, the angle of view is also related to the movement of the eyes.

Stereo: The system involves ten stereos. Eight large stereos are placed at the eight corners of the room to form a surround-sound system, responsible for sound from a further distance. Two small stereos are attached to the user's back head, responsible for sound from a closer distance.

Haptic gloves: Stimulating electrotactile feedback to provide tactile feedback to the user's palm and finger.

Wind simulator: Each side of the wall in the room is placed with a wind simulator to blow wind from various directions, simulating different weather conditions in the interactive film.

Haptic shoes: Stimulating electrotactile feedback to provide tactile feedback to the user's feet

Motion capture camera: Another critical feature for achieving high immersion is that users should be able to see themselves in the interactive film; for instance, they would see their body while lowering their head or their arms and hand while raising their arms. Six motion capture cameras are placed at the top of the room, filming the user's movement from a different direction, and the cameras synchronize the captured movement into the virtual space.

Cabe-based actuator: Generating force feedback to stimulate kinesthetic feedback

Air conditioner: Instead of putting another device onto the user to stimulate temperature, we used an air conditioner to lower the user's weight. The air conditioner can change the room's temperature, humidity, and pressure to simulate different weather conditions.

Omnidirectional treadmill: Taking charge of the user's movement on the flat surface.

Regular treadmill with angular bar: Taking charge of user's ascending and descending movement.

Figure 2 Hardware design for the VR interactive film system.

5. Challenges behind VR Interactive Film

The current challenges of VR interactive film systems can be categorized into two aspects: technical challenges and artistic challenges.

Technical challenges occur because VR technologies' hardware is not yet at a mature and ubiquitous stage, so the application of VR film, more advanced technology in the field of VR, would be restricted. The primary limitations are all related to the computing power. The HMD cannot attain high resolution and refresh rate due to the computing rate, impacting users' immersive experience. VR film's low resolution and fluidity can also read to motion sickness. The computing power also limited high precision tracking, so the tracking system on the haptic shoes (related to ODT) and motion capture camera also face technical challenges.

An interactive film allows the freedom of viewpoint, thereby causing some artistic challenges. In the interactive film, the composition techniques, scene arrangement, and landscape design used in the traditional film are no longer applicable. A study suggests that the narrative immersion of VR interactive film is comparatively weaker than traditional film because the entire film consists of 360-degree panoramic shots. Accordingly, to ensure the narrative aspect in architecture design, the creator must develop new cinematics techniques to keep the audience's focus.

6. Conclusion

In this paper, we have created a design solution for the problem that architects cannot present the narration and the emotion of their design through the current method of architectural design. We have also evaluated the state of film and architecture and explained the advantage of VR in solving this problem. In our final design solution, we came up with a VR interactive film-based system that provides a highly immersive experience to the users. We have also explained in detail the locomotion techniques and the haptic feedback involved in the system. However, since the technology is innovative, some technical and artistic challenges need to be solved.

Acknowledgement

All the graphics in the paper were created by Junhao Zeng.

[1] Archisoup. (n.d.) Architecture Narratives – The Storytelling of Design. Retrieved from archisoup: https://www.archisoup.com/studio-guide/architecture-narratives

[2] Zhen, J. (2018). ARCHITECTURAL NARRATIVE DESIGN ——INTERPRETATION OF BERNARD TSCHUMI’S DESIGN WORKS. Design works. (09)

[3] Portman, M.E.(2015). To go where no man has gone before: Virtual reality in architecture, landscape architecture and environmental planning. Retrieved 30 June 2022, from https://www.sciencedirect.com/science/article/pii/S019897151500054X

[4] Fonseca, D., & Kraus, M. (2016, October). A comparison of head-mounted and hand-held displays for 360 videos with focus on attitude and behavior change. In Proceedings of the 20th International Academic Mindtrek Conference (pp. 287-296). https://doi.org/10.1145/2994310.2994334

[5] Marques, T., Vairinhos, M., & Almeida, P. (2021). VR 360º and its impact on the immersion of viewers of suspense AV content. Multimedia Tools and Applications, 80(20), 31021-31038. https://doi.org/10.1007/s11042-021-11067-9

[6] Kim, A., Chang, M., Choi, Y., Jeon, S., & Lee, K. (2018, March). The effect of immersion on emotional responses to film viewing in a virtual environment. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)(pp. 601-602). IEEE.https://doi.org/10.1109/VR.2018.8446046

[7] Stenger, N. (1992). Angels. Retrieved from Archive of Digital Art: https://www.digitalartarchive.at/database/general/work/angels.html

[8] Dedehayir, O., & Steinert, M. (2016). The hype cycle model: A review and future directions. Technological Forecasting and Social Change, 108, 28-41. https://doi.org/10.1016/j.techfore.2016.04.005

[9] Dan, X. & Li, X. (2020) The Body and Its Theoretical Problems in Virtual Art Experience: An Investigation Based on Physical Practice in VR Films. Theoretical Studies of Literature and Art, (05), 66-77.

[10] Arango, B. (2022). Virtual Reality: Current State and Future Challenges. Retrieved 30 June 2022, from https://filmora.wondershare.com/virtual-reality/virtual-reality-technology-current-state-and-future-challenges.html

[11] Claypool, M. (2022). The Digital in Architecture: Then, Now and in the Future | SPACE10. Retrieved 30 June 2022, from https://space10.com/project/digital-in-architecture/

[12] Kishishita, Y., Das, S., Ramirez, A. V., Thakur, C., Tadayon, R., & Kurita, Y. (2019, March). Muscleblazer: force-feedback suit for immersive experience. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (pp. 1813-1818). IEEE. https://doi.org/10.1109/VR.2019.8797962

[13] Kourtesis, P., Argelaguet, F., & Vizcay, S. (2021). Electrotactile feedback applications for hand and arm interactions: A systematic review, meta-analysis, and future directions. arXiv e-prints, arXiv-2105.

[14] Solomonow, M., Lyman, J., & Freedy, A. (1977). Electrotactile two-point discrimination as a function of frequency, body site, laterality, and stimulation codes. Annals of biomedical engineering, 5(1), 47-60. https://doi.org/10.1007/BF02409338.

[15] See, A. R., Choco, J. A. G., & Chandramohan, K. (2022). Touch, Texture and Haptic Feedback: A Review on How We Feel the World around Us. Applied Sciences, 12(9), 4686. https://doi.org/10.3390/app12094686

[16] Keil, J., Edler, D., O’Meara, D., Korte, A., & Dickmann, F. (2021). Effects of virtual reality locomotion techniques on distance estimations. ISPRS International Journal of Geo-Information, 10(3), 150. https://doi.org/10.3390/ijgi10030150.

[17] Demaine, E. D., Fekete, S. P., & Lang, R. J. (2010). Circle packing for origami design is hard. arXiv preprint arXiv:1008.1224. https://doi.org/10.48550/arXiv.1008.1224.

[18] Nagao, R., Matsumoto, K., Narumi, T., Tanikawa, T., & Hirose, M. (2017). Walking up virtual stairs based on visuo-haptic interaction. In ACM SIGGRAPH 2017 Posters (pp. 1-2). https://doi.org/10.1145/3102163.3102231.