Virtual Reality (VR) is a powerful medium to design and experience architecture which researchers and practitioners are currently exploring with increasing interest. This paper investigates the impact of the use of VR in architectural design. We developed an interactive VR design tool by using the Unreal Engine 4 game engine, Dreamscape Bricks VR, in which the participants can design and build architectural models using virtual LEGO pieces via direct manipulation. We conducted design experiments comparing the physical medium and VR design processes with 14 subjects, including architects, graduate, and undergraduate design students. Each participant was asked to design a shelter and a pavilion, one with physical LEGO bricks (in situ) and another in Dreamscape Bricks VR (in virtuo). The design processes in both the physical medium and VR were analyzed by video recording of participants with retrospective think-aloud reporting and a post-experiment survey. The Function-Behavior-Structure (FBS) framework was used to analyze the participants’ cognitive design process, while the Embody- Experience-Manipulate (EEM) framework was developed and used to analyze the recorded design activities. The results showed similarly rich cognitive design processes in both media. Embodying and experiencing activities were significantly higher in VR. The analysis results and participant comments indicated that the ability to change the user scale in VR provides novel opportunities for the design process that are not available in the physical medium. This study offers insight into the impact of using VR in architectural design processes and the potential and limitations of VR design tools.