ARCH361 Lecture 11: Architecture in the Age of Artificial Intelligence PDF

Liverpool School of Architecture TECHNOLOGY 3.1: INTEGRATED TECHNICAL PROJECT DESIGN Lecture 11: Architecture in the Age of Artificial Intelligence Dr Carlos Medel-Vera Lecturer in Archite...

Liverpool School of Architecture TECHNOLOGY 3.1: INTEGRATED TECHNICAL PROJECT DESIGN Lecture 11: Architecture in the Age of Artificial Intelligence Dr Carlos Medel-Vera Lecturer in Architectural Technology ARCH361 - CMV  Liverpool School of Architecture Presentation contents ❑ Motivation: What has AI got to do with architecture? Context and background ❑ Giving computers the ability to ‘see’ – ‘baby steps’ Current applications Concept design with AI ❑ Giving computers the ability to ‘identify’ – ‘toddler steps’ ❑ Giving computers the ability to ‘create’ – ‘adult steps’ ❑ Applications of convolutional neural and generative adversarial networks in architecture ❑ Applications of deep transfer style in architecture ❑ Applications of stable diffusion deep learning in architecture ❑ Some tools for concept design: Deep Dream Generator ❑ Some tools for concept design: Midjourney ❑ Summary and key takeaways ARCH361 - CMV  Liverpool School of Architecture How does this lecture fit into Architectural Technology? ▪ Architectural technology is the realisation of architecture through the application of building science; it’s the fusion of three separate BIM worlds, bringing together artistic, Fashion practical, and procedural skills Artificial PRACTICAL Typology intelligence Aesthetics Functional Sensory engagement tangible ARTISTIC technology Style Design Architectural Materials intangible Social technology Services context creative Buildability Structure Scheduling & Geotechnics cost estimation Legislation Construction PROCEDURAL Quality methods assurance Ordered Based on Emmitt, S. (2012). Architectural deliverable Technology. 2nd Edition, Wiley-Blackwell, UK management ARCH361 - CMV  Liverpool School of Architecture Part I Context and Background ARCH361 - CMV  Liverpool School of Architecture Alan Mathison Turing (1912 – 1954) ARCH361 - CMV  Liverpool School of Architecture The Turing Test – Can machines think? ▪ The Turing Test, originally called ‘the imitation game’, was proposed by the British mathematician Alan Turing in a seminal paper published in 1950 called ‘Computing Machinery and Intelligence’ ▪ He addresses the question ‘can machines think?’ by asking if a machine can win a game where an interrogator cannot distinguish if they are dealing with a machine/computer or a human being Interrogator (can’t see who A and B are) ‘Turing’s work inspired generations of research into what scientists called “Turing Machines” Today, we call them computers’ ▪ Turing states in his paper: “I believe that in about 50 years’ time it will be possible to programme computers (…) to make them play the imitation game so well that The Imitation Game, Warner Bros., 2014 an average interrogator will not have more than 70% chance of making the right identification after 5 minutes of questioning” ARCH361 - CMV  Liverpool School of Architecture A Visual Turing Test for Architecture ▪ You will be shown a series of 16 images of building facades; some are actual photographs and some are AI- generated images ▪ There are 8 categories covering attributes you normally find in architectural settings: (i) materials and textures, (ii) interior spaces, (iii) vernacular architecture and traditional buildings, (iv) adaptive reuse, (v) nature-inspired architecture, (vi) conceptual architectural models, (vii) fluid and organic architecture, and (viii) urban spaces. ▪ There are always 2 images per attribute: one real, one AI-generated ▪ The order in which the images are sorted is purely based on the results of a random number generator ▪ For each image, you need to answer the following question: ‘Which of the two, (i) a human being using a camera, or (ii) a computer using AI, is the creator of this image? ARCH361 - CMV  Liverpool School of Architecture A Visual Turing Test for Architecture (1) (2) (3) (4) (5) (6) (7) (8) ARCH361 - CMV  Liverpool School of Architecture A Visual Turing Test for Architecture (9) (10) (11) (12) (13) (14) (15) (16) ARCH361 - CMV  Liverpool School of Architecture Turing Test: Discussion of results ARCH361 - CMV  Liverpool School of Architecture A Visual Turing Test for Architecture – AI-generated (1) (2) (4) (9) (11) (12) (14) (16) ARCH361 - CMV  Liverpool School of Architecture A Visual Turing Test for Architecture – Real photographs (3) (5) (6) (7) (8) (10) (13) (15) ARCH361 - CMV  Liverpool School of Architecture ChatGPT changed everything… ARCH361 - CMV  Liverpool School of Architecture Why suddenly AI is being linked to Architecture? ARCH361 - CMV  Liverpool School of Architecture Why suddenly AI is being linked to Architecture? ✓ There is nothing more human than Architecture… ARCH361 - CMV  Liverpool School of Architecture Human intelligence vs Computer intelligence https://medium.com/the- spike/brains-as-analog- computers-fa297021f935 ▪ Can you multiply thousands of large numbers quickly? Hard Easy 23 x 45 x 65 x 34 x 75 x 80 x … ▪ Can you identify objects/faces on pictures quickly? Easy Hard ARCH361 - CMV  Liverpool School of Architecture Natural neural networks vs artificial neural networks The basic unit of a biological brain – the neuron ▪ They all transmit an electrical signal from one end to the other, from the dendrites along the axons to the terminals ▪ These signals are then passed from one neuron to another ▪ This is how your body senses light, sound, touch pressure, heat, etc. The human brain has about 100 billion neurons ▪ Biological neurons take many inputs, not just one ▪ The electrical signals are collected by the dendrites and these combine to form a stronger electrical signal ▪ If the signal is strong enough to pass a threshold, the neuron fires a signal down the axon towards the terminals to pass onto the next neuron’s dendrites ▪ Each neuron takes input from many before it, and also provides signals to many more, if it happens to be firing ARCH361 - CMV  Liverpool School of Architecture Natural neural networks vs artificial neural networks How to replicate this from nature to an artificial model? The artificial neuron, originally named perceptron, receives many inputs which are multiplied by weights that represent the strength of the preceding signal; then are added up and filtered, to then shoot an output Natural neuron Artificial neuron One way to replicate this from nature to an artificial model is to have layers of neurons, with each connected to every other one in the preceding and subsequent layer Natural neural network Artificial neural network  Liverpool School of Architecture How artificial neural networks (ANNs) work ▪ Let’s start with a simple example to illustrate the concept of an artificial neuron “thinking” ▪ Suppose we need a machine that converts kilometres to miles, like the following: ✓ Let’s suppose we don’t know the formula for converting kilometres to miles ✓ We only know that it’s a linear relationship: miles = kilometres x c, where c is a constant ✓ The only available information we have is two real examples we know they are true ✓ So, let’s just try different random numbers to see what happens. Start with c = 0.5 We obtained 50 miles, but we know the true value is 62.137 so we’re wrong by 12.137 error = truth – calculated = 62.137 - 50 ARCH361 - CMV  Liverpool School of Architecture How artificial neural networks (ANNs) work ✓ So, we know we’re wrong and by how much ✓ We were short by 12.137, so we need to increase c. Let’s try c = 0.6 ✓ We’re in a much better position, but still short. Let’s ✓ We’ve gone too far and overshot the true value. Let’s go back try again with c = 0.7 when we had a little error and nudge c up a little bit. Let’s try c = 0.61 ARCH361 - CMV  Liverpool School of Architecture How artificial neural networks (ANNs) work ✓ From this process we learned we should moderate how much we nudge the value of c. If the outputs are getting close to the correct answer - that is, the error is getting smaller - then don’t nudge the changeable bit so much. That way we avoid overshooting the right value. C = 0.5 C = 0.6 C = 0.7 C = 0.61 This iterative process whereby a machine adjusts the model parameters to get closer to the true value (or Machine learning reduce the error) is known as ‘learning’ ARCH361 - CMV  Liverpool School of Architecture How artificial neural networks (ANNs) work ✓ In computer science, the setup of an artificial neural network, i.e. the number of neurons in the input layer, the number of hidden layers, and the number of neurons in the output layer, is called ‘architecture’ An architecture that involves many hidden layers means that Deep learning the process must dig deep to learn ARCH361 - CMV  Liverpool School of Architecture AI, ML and DL https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/ ARCH361 - CMV  Liverpool School of Architecture ‘Baby’ steps – Giving computers the ability to ‘see’ Human vision system Computer vision system ▪ The first step for a computer to be able to ‘interpret’ an image is to define how it deals with it ▪ An image can be represented as a function of two variables x and y, which define a two-dimensional area ▪ A digital image is made of a grid of pixels ▪ The pixel is the raw building block of an image This image is 32 x 16 = 512 pixels ARCH361 - CMV  Liverpool School of Architecture ‘Baby’ steps – Giving computers the ability to ‘see’ What humans see What computers see For grayscale (or black and white) images, computers use a 1-channel matrix, where each position (pixel) represents the intensity of light on that specific pixel For colour images, computers use a 3- channel matrix, each channel representing the intensities of red, green and blue (RGB images) ARCH361 - CMV  Liverpool School of Architecture ‘Toddler’ steps – Giving computers the ability to ‘identify’ Convolution is a fancy word for a feature finder window that slides over the image pixel One of the firsts things a toddler learns is by pixel to extract meaningful features that identify the objects in the image to ‘classify’ “A way to encourage children to learn classifying and sorting is by sorting out toys, leaves, rocks or other similar items into 'like' groups, such as big/small, long/short, or colours…” https://playingwithlearning.weebly.com/classifying- and-sorting ▪ It turns out that artificial neural networks, in particular deep neural networks, are really good at being able to extract features and then classify images ▪ This type of networks that deal with image classification are called convolutional neural networks CMV - MMXXIV  Liverpool School of Architecture ‘Toddler’ steps – Giving computers the ability to ‘identify’ ▪ Once a computer has seen hundreds or thousands of images of the same class or category, and learned the essential information about them, i.e. extracted their main features, then… …it will be able to tell if a brand new image belongs to that category ARCH361 - CMV  Liverpool School of Architecture Convolutional neural networks (CNNs) ▪ A typical architecture for a CNN would look like this: If I learned and I’m good at rollerblading, surely I can’t be that bad at skiing? Or snowboarding? This bit looks rather complicated… good news is we don’t need to worry about it and we can make use of a pre-trained CNN, i.e. we take advantage of an already trained CNN to identify objects (whatever they might be) and we train it using our own dataset. This is called Transfer learning transfer learning ARCH361 - CMV  Liverpool School of Architecture CNNs: Handwritten numbers classification ▪ A well known exercise that makes use of CNNs is handwritten numbers classification. A large dataset used for this purpose is known as MNIST and is composed of 60,000 images of numbers from 0 to 9 ▪ For demonstration purposes, a subset of 10,000 images were used in a pre- trained CNN that was originally trained to detect objects (transfer learning) ▪ This pre-trained CNN has been trained on over a million images and can classify images into 1000 categories (keyboard, coffee mug, pencil, car, etc.) ▪ Surely, we don’t need such an ‘expert’ CNN to teach it how to classify numbers… but probably, it will classify them with high accuracy Examples of handwritten numbers in the MNIST dataset ARCH361 - CMV  Liverpool School of Architecture CNNs: Handwritten numbers classification ▪ Examples of brand-new handwritten numbers (my own writing): ▪ CNN classification: Probability Probability Probability Probability certainty: 94.7% certainty: 81.9% certainty: 61.8% certainty: 98.5% ARCH361 - CMV  Liverpool School of Architecture CNNs: Healthcare research ▪ Lung cancer is a growing problem. When diagnosed, it is usually in the middle or late stages. Doctors typically use their eyes to examine CT scans images looking for small nodules in the lungs ▪ Doctors are very good at identifying mid- and large-size modules, such as 6-10 mm. But when nodules are 4 mm or smaller, they sometimes have difficulties identifying them. ▪ CNNs are able to learn these features automatically and detect small nodules early, before they become deadly ARCH361 - CMV  Liverpool School of Architecture CNNs: Self-driving cars ▪ Self-driving cars use cameras, radar and lidar sensors to gather data about the environment around the vehicle. ▪ This data is then processed by the CNN to identify objects such as other vehicles, pedestrians, road signs and traffic lights. ▪ The network then uses this information to make decisions about how to control the vehicle, such as determining the best route or avoiding obstacles. https://www.augmentedstartups.com/blog/convolutional-neural-networks-cnn-in-self-driving-cars ARCH361 - CMV  Liverpool School of Architecture ‘Adult’ steps – Giving computers the ability to ‘create’ Can a computer generate a brand-new image using the features it already knows? How do adults learn or use the ability to ‘create’? Where does creativity come from? We would be asking the network the question: can you run backwards? Generative adversarial networks (GANs) CMV - MMXXIV https://sg.style.yahoo.com/left-brain-vs-brain-difference-110620237.html  Liverpool School of Architecture Generative adversarial networks (GANs) ▪ The GAN architecture basically consists of two neural networks that compete against each other: - The generator tries to convert random input data into observations that look as if they have been sampled from the original dataset - The discriminator tries to predict whether an observation comes from the original dataset or is one of the generator’s forgeries The discriminator network is a typical CNN; the generator network is an inverted CNN ARCH361 - CMV  Liverpool School of Architecture Generative adversarial networks (GANs) The generator gets better are mimicking the handwritten digits of the MNIST dataset Generator Discriminator A GAN can create new, ‘made-up’ images from a set of existing images ARCH361 - CMV  Liverpool School of Architecture (Text-to-Image) Diffusion Models ❑ Diffusion models progressively destruct data by injecting noise to the learn to reverse the process. In this way, new instances can be generated from just a random noise as a starting point ❑ Text-to-image models are specific DMs that generate images from a descriptive text or prompt A Golden A Golden Retriever dog Retriever dog wearing a blue blue chequered Diffusion model Text encoder chequered beret beret and red dotted turtleneck red dotted turtleneck CMV - MMXXIV  Liverpool School of Architecture (Text-to-Image) Diffusion Models - Deep Dream Generator (https://deepdreamgenerator.com/) - DALL-E3 (https://openai.com/product/dall-e-3) - Runway ML (https://runwayml.com/) - Stable Diffusion (https://stablediffusionweb.com/) - Midjourney (https://midjourney.com/) - Adobe Firefly (https://www.adobe.com/products/firefly.html) CMV - MMXXIV  Liverpool School of Architecture This House Does Not Exist: Houses imagined by a GAN https://thishousedoesnotexist.org/ ARCH361 - CMV  Liverpool School of Architecture This House Does Not Exist: Houses imagined by a GAN https://thishousedoesnotexist.org/ [manila] [philippines] [interior] [living room] [interior] [sunset] [bali] [florianopolis] [brazil] [exterior] [wood] [glass] [exterior] [sunrise] [glass] [stone] [downtown] [night] [beach] [view] [wood] [stone] [stone] [tulum] [mexico] [interior] [sunset] [wood] [exterior] [wood] [stone] [hawaii] [zurich] [switzerland] [exterior] [night] [beach] [amed] [indonesia] [bath] [united kingdom] [bejuco] [view] [glass] [stone] [beijing] [china] [interior] [bathroom] [sunrise] [beach] [view] [glass] [stone] ARCH361 - CMV  Liverpool School of Architecture This House Does Not Exist: Houses imagined by a GAN https://thishousedoesnotexist.org/ [dublin] [ireland] [interior] [living room] [sunrise] [santiago] [chile] [interior] [bedroom] [night] [exterior] [wood] [stone] [bali] [tokyo] [japan] [plants] [hanging plants] [interior] [beach] [steam] [view] [wood] [hot spring] [beach] [view] [glass] [bedroom] [wood] [singapore] [office] [exterior] [wood] [shenzhen] [china] [interior] [kitchen] [beach] [interior] [sunset] [wood] [stone] [bali] [barcelona] [spain] [pai] [thailand] [bar] [view] [glass] [stone] [montenegro] [exterior] [sunrise] [glass] [stone] [downtown] ARCH361 - CMV  Liverpool School of Architecture Can machines ‘design’ buildings? by Tim Fu, Designer at Zaha Hadid Architects, using Midjourney ARCH361 - CMV  Liverpool School of Architecture Human intelligence growth vs Computer intelligence growth: Summary Human intelligence Baby steps Toddler steps Adult steps Ability to ‘see’ Ability to ‘classify’ Ability to ‘create’ Computer intelligence Computer vision: computer Convolutional neural networks Generative Adversarial Networks ‘sees’ via matrices (CNNs) give computer the ability and Diffusion Models give to ‘classify’ objects computers the ability to ‘create’ new images never seen before  Liverpool School of Architecture Part II Some Applications in Architecture and Urban Studies ARCH361 - CMV  Liverpool School of Architecture (Simplified) Design process in architecture Stage 5. Full design & specs Full plans and models, technical specifications, building services, detail drawings etc. - creativity + precision Stage 4. Design development 3D massing, scale models, floor plans, envelope design (building physics), etc. Stage 3. Sketching Concept design, façade explorations, interiors concept design, architectural moments, bubble diagrams, etc. + creativity Stage 2. Research - precision Design options, materials, social context, climate, style, aesthetics, etc. Stage 1. Design brief Client requirements, budget, site visits, accommodation, topographical survey, etc. ARCH361 - CMV  Liverpool School of Architecture CNNs: Building façade classification Design Full design & Design brief Research Sketching developmen specs t ▪ An example of current research on CNNs in architecture is related to the identification and classification of visual design principles in building façades (Credits: Dr. Asli Cemki, Istanbul Visual Design Principles in Facades Technical University) EMPHASIS BALANCE RHYTHM Creation of dominant elements in Creation of equality of visual weights in Creation of repetition in elements , colors, a composition (focal point, attracts a composition (two forces of equal forms, and textures (a sequence of visual attention and encourages to look strength that pull in opposite elements at prescribed intervals) closer) directions) SYMMETRIC REGULAR COLOUR Reflection of elements within the Composition that contains the Contrasting or distinct colour in a composition with respect to a same or similar recurring composition centerline or axis elements (grids) ASYMMETRIC ISOLATION PROGRESSIVE While two sides are not identical , Element apart from other things in a sense of balance could still be Hierarchical change in a group of a composition achieved recurring elements SHAPE CHRYSTALLOGRAFIC FLOWING Element with distinct shape in It’s about repetition and Repetition of wavy lines, bended form or scale appears in a consistency. It has equal weight elements and curved shapes composition in all regions ARCH361 - CMV  Liverpool School of Architecture CNNs: House style classification Design Full design & Design brief Research Sketching developmen specs t (Credits: Dr. Yun Kyu Yi, University of Illinois at Urbana-Champaign) ▪ Another example of current research on CNNs in architecture is related to the identification and classification of house styles in the US ARCH361 - CMV  Liverpool School of Architecture Building façades: GANs – Phillip Isola et al. Design Full design & Design brief Research Sketching developmen specs t ▪ GANs have been used to help architects in the design of building façades using simple discretisation of elements ▪ Façade is thought of as a composition of simple structuring elements such as windows, cornices, pilasters, doors, balconies, etc. Pillar Shop Window Cornice Door Sill Blind Background ARCH361 - CMV  Liverpool School of Architecture Building mass: FrankenGAN – Tom Kelly et al. Design Full design & Design brief Research Sketching developmen specs t ▪ Building are rarely in isolation; FrankenGAN takes into account that entire city blocks often share a similar typology ▪ FrankenGAN takes as input the raw 3D massing of city blocks and a façade style reference, generating a highly detailed and textured envelope ▪ This approach is both style informed and geometry specific Initial massing Detailing Texturing ARCH361 - CMV  Liverpool School of Architecture Internal space planning: ArchiGAN – Stanislas Chaillou Design Full design & Design brief Research Sketching developmen specs t ▪ ArchiGAN is a generative stack for apartment building design submitted as MArch thesis in Harvard University in 2019. Using AI, the process involves three steps: (i) building footprint massing, (ii) program repartition, and (iii) furniture layout. ✓ Each step relies on a model that has been GAN-trained and tasks are performed one after the other, allowing for user input at each step building footprint space furniture massing repartition layout Step 1 - Footprint ✓ Building footprints significantly define the internal organisation of floor plans. ✓ Their shape is heavily conditioned by their surroundings and, more specifically, the shape of their plot ARCH361 - CMV  Liverpool School of Architecture Internal space planning: ArchiGAN – Stanislas Chaillou Design Full design & Design brief Research Sketching developmen specs t Step 2 - Program ✓ The network takes as input the footprint of a given housing unit produced by Model I, the position of its entrance door, and the position of the main windows specified by the user. ✓ The plans used to train the network derive from a database of 800+ plans of apartments, properly annotated and given in pairs to the model during training. In the output, ✓ The program encodes rooms using colours while representing the wall structure and its fenestration using a black patch. ARCH361 - CMV  Liverpool School of Architecture Internal space planning: ArchiGAN – Stanislas Chaillou Design Full design & Design brief Research Sketching developmen specs t Step 3 - Furnishing ✓ Finally, Model III tackles the challenge of furniture layout using the output of model II. ✓ This model trains on pairs of images, mapping room programs in colour to adequate furniture layouts. ✓ The program retains wall structure and fenestration during image translation while filling the rooms with relevant furniture, specified by each room’s program ARCH361 - CMV  Liverpool School of Architecture Internal space planning: ArchiGAN – Stanislas Chaillou Design Full design & Design brief Research Sketching developmen specs t ▪ User interface is available from: http://stanislaschaillou.com/thesis/GAN/unit_program/ ARCH361 - CMV  Liverpool School of Architecture Internal space planning: ArchiGAN – Stanislas Chaillou Design Full design & Design brief Research Sketching developmen specs t ARCH361 - CMV  Liverpool School of Architecture Annual daylight metrics: Daylight GAN – Theodore Galanos Design Full design & Design brief Research Sketching developmen specs t ▪ Daylight GAN aims at forecasting the potential reach of natural light within a project, given a floor plan footprint and its façade openings ▪ Daylight GAN allows to instantly assess the performance of a space, (nearly) in real time, with as little information as possible ARCH361 - CMV  Liverpool School of Architecture CNNs: Urban spaces classification ▪ An example of current research on CNNs in urban design is related to the identification and classification of urban spaces Natural/semi-natural Civic space Public open space urban space Movement space Service space Interchange space Third place space Private public space User selecting space Seed Dataset – Photographs (sample) ARCH361 - CMV  Liverpool School of Architecture Urban spaces: GAN Loci – Kyle Steinfeld ▪ This project applies GANs to produce synthetic images that capture the predominant visual properties of urban places. ▪ This project uses GANs as a tool for tacit design and to apply the capacity of this technology for capturing the implicit yet salient visual properties of a set of images Nine synthetic images of urban places ARCH361 - CMV  Liverpool School of Architecture PLEASE REGISTER YOUR ATTENDANCE ON THE TIMETABLE APP USING THE FOLLOWING 6-DIGIT CODE: 671554 ARCH361 - CMV  Liverpool School of Architecture Part III Text-to-Image Diffusion Models in Practice ARCH361 - CMV  Liverpool School of Architecture Text-to-Image Diffusion Models - Deep Dream Generator (https://deepdreamgenerator.com/) ❑ Prompt crafting is the art of concisely - DALL-E3 (https://openai.com/product/dall-e-3) communicating requests to an AI - Runway ML (https://runwayml.com/) engine for optimal output. It involves a - Stable Diffusion (https://stablediffusionweb.com/) combination of knowledge, - Midjourney (https://midjourney.com/) experience, and creativity - Adobe Firefly (https://www.adobe.com/products/firefly.html) CMV - MMXXIV  Liverpool School of Architecture vs ARCH361 - CMV  Liverpool School of Architecture Why Adobe Firefly? Adobe Firefly is designed to generate content safe for commercial use. This is because the dataset used for training has three relevant features: Adobe Stock repository Public space Expired copyright (+300 million images) (no copyright) CMV - MMXXIV  Liverpool School of Architecture Why Adobe Firefly? – It deliberately prevents harm and bias Text prompt: A Van Gogh style painting featuring beautiful green fields, cows, and a blue sky Prompt concept remains No references to real artists or other visual style Firefly tries to protect the original artist Prompts including “… in the style of…” do not work on Firefly CMV - MMXXIV  Liverpool School of Architecture Why Adobe Firefly? – It deliberately prevents harm and bias Text prompt: Disney’s Mickey Mouse performing a magic spell Prompt concept remains No references to actual (copyrighted) characters – although this copyright expired in January 2024 Images are therefore safe from copyright issues CMV - MMXXIV  Liverpool School of Architecture Data scraping: Midjourney v6 Some images in the training dataset used in Midjourney might have been obtained without consent. Data scraping is the act of mass crawling and extracting information from a website. Text prompt: A Van Gogh style painting featuring beautiful green fields, cows, and a blue sky CMV - MMXXIV  Liverpool School of Architecture Data scraping: Midjourney v6 Some images in the training dataset used in Midjourney might have been obtained without consent. Data scraping is the act of mass crawling and extracting information from a website. Text prompt: Disney’s Mickey Mouse performing a magic spell CMV - MMXXIV  Liverpool School of Architecture Why Adobe Firefly? – It deliberately prevents harm and bias Text prompt: a person cooking pasta over an open fire ▪ Inspired by images from Adobe Stock ▪ Diversity in terms race and gender CMV - MMXXIV  Liverpool School of Architecture Why Adobe Firefly? – It deliberately prevents harm and bias ▪ Text prompt: a woman cooking pasta over an open fire ▪ Text prompt: a man cooking pasta over an open fire CMV - MMXXIV  Liverpool School of Architecture Adobe Firefly 3 vs Midjourney v5 ‘a beach house, Zaha Hadid’ ‘a beach house, Zaha Hadid’ Adobe Firefly 3 Midjourney v5 CMV - MMXXIV  Liverpool School of Architecture Adobe Firefly 3 vs Midjourney v5 ‘a beach house, Antoni Gaudi’ ‘a beach house, Antoni Gaudi’ Adobe Firefly 3 Midjourney v5 CMV - MMXXIV  Liverpool School of Architecture Adobe Firefly 3 vs Midjourney v5 ‘a beach house, Tadao Ando’ ‘a beach house, Tadao Ando’ Adobe Firefly 3 Midjourney v5 CMV - MMXXIV  Liverpool School of Architecture Why Adobe Firefly? – It deliberately prevents harm and bias https://contentauthenticity.org/  Liverpool School of Architecture Why Adobe Firefly? – It deliberately prevents harm and bias https://contentcredentials.org/verify  Liverpool School of Architecture Structure of a prompt In general, a prompt needs to be: 1. Descriptive – Don’t give commands – don’t start with a verb, software is expecting a description 2. Detailed – Describe specific image details as you envision them – where is the scene taking place? City, outside, forest, time of day, etc. 3. Specific – Leave as little room as possible for interpretation – fruit on a table? Is it a wooden table, metallic table, round table? Is it apples, oranges? 4. Combined – Subject, visual style, colour, and background description CMV - MMXXIV  Liverpool School of Architecture Firefly – Text-to-image functionality – Structure of a prompt Bad example – Prompt: a knight on a horse Too short Not detailed enough Not specific enough Not great results CMV - MMXXIV  Liverpool School of Architecture Firefly – Text-to-image functionality – Structure of a prompt Good example – Prompt: a hyper realistic, detailed image of a knight wearing armour and a blue shield, riding a white horse, on a sunny day, with a medieval castle visible in the background, wide shot, fantasy theme Very specific Detailed Describes surroundings Style theme and camera information CMV - MMXXIV  Liverpool School of Architecture Firefly – Text-to-image functionality – Structure of a prompt Good example – Prompt: a hyper realistic, detailed image of a knight wearing armour and a blue shield, riding a white horse, on a sunny day, with a medieval castle visible in the background, wide shot, fantasy theme Very specific Detailed Describes surroundings Style theme and camera information CMV - MMXXIV  Liverpool School of Architecture Firefly – Text-to-image functionality – Structure of a prompt Bad example – Prompt: a waterfront home Too short Not detailed enough Not specific enough Not great results CMV - MMXXIV  Liverpool School of Architecture Firefly – Text-to-image functionality – Structure of a prompt Good example – Prompt: modern waterfront home with large windows and open layout, timber construction, enhancing the connection with the water, ultra detailed, architectural photography Very specific description of main concept Detailed architectural style and material Describes surroundings, visual style/mood/texture Style theme and camera information, enhancer words CMV - MMXXIV  Liverpool School of Architecture Firefly – Text-to-image functionality – Structure of a prompt Good example – Prompt: modern waterfront home with large windows and open layout, enhancing the connection with the water, timber construction, ultra detailed, architectural photography Adobe Firefly Midjourney v5 CMV - MMXXIV  Liverpool School of Architecture Firefly – Text-to-image functionality – Structure of a prompt Good example – Prompt: an innovative cantilevered home suspended over a rocky cliff, overlooking a serene ocean, highlight the seamless integration of glass, steel, and concrete, cinematic lighting, architectural magazine Very specific description of main concept Detailed architectural style and material Describes surroundings, visual style/mood/texture Style theme and camera information, enhancer words CMV - MMXXIV  Liverpool School of Architecture Firefly – Text-to-image functionality – Structure of a prompt Good example – Prompt: a converted warehouse, preserving elements such as exposed bricks and beams, turned into a modern apartment with large windows, combination of interior and exterior light, 8K Very specific description of main concept Detailed architectural style and material Describes surroundings, visual style/mood/texture Style theme and camera information, enhancer words CMV - MMXXIV  Liverpool School of Architecture Midjourney v5 Prompt crafting: long vs short Midjourney v5 /imagine A picturesque beach house nestled along the pristine coastline, embracing the /imagine A modern beach house that harmonizes luxury and nature. Imagine large harmony between modern luxury and natural serenity. Envision a structure that seamlessly windows framing ocean views, seamlessly blending indoor and outdoor spaces. Prioritize integrates with its surroundings, utilizing sustainable materials and innovative architectural sustainable materials, with a focus on coastal landscaping. Create a tranquil retreat that elements. Capture the essence of coastal living with expansive windows that offer captures the essence of coastal living panoramic views of the ocean, allowing the play of natural light to define the interior spaces. Consider the incorporation of outdoor living areas, such as a spacious deck or veranda, that effortlessly blend with the sandy shores. Pay attention to details like coastal landscaping, creating a seamless transition from the beach to the house. Emphasize a timeless design that not only complements the coastal environment but also provides a tranquil retreat for residents ARCH361 - CMV  Liverpool School of Architecture Challenge – Text prompting for Entrance & Approach The Baccarat Hotel, New York Very specific description of main concept Aim: To find the best text prompt in Adobe Detailed architectural style and material Firefly to reproduce a similar entrance, Describes surroundings, visual style/mood/texture using the prompt structure learned today: Style theme and camera information, enhancer words CMV - MMXXIV  Liverpool School of Architecture Midjourney – Prompt craft – Reverse prompting Rather than using /imagine to generate images, Midjourney can produce text prompts for an image by using /describe The entrance of the luxury hotel features a tall door with large glass panels, adorned by black steel columns holding boxwood plants and an open fireplace in front. The overall 1 design exudes elegance through its use of materials, lighting effects, and a sophisticated color palette. This masterpiece captures every detail to create a stunning visual experience for all who view it. --ar 64:25 The front door of the luxury hotel features a wall made of glass and stainless steel vertical stripes. Inside, there is an open fireplace between two black metal flower pots on both 2 sides. The overall look is modern, elegant, and luxurious. The environment outside should have soft lighting to highlight the exterior design. --ar 64:25 An entrance to the stately building in New York, with tall black wall panels and glass walls, 3 featuring an elegant fireplace. The entrance is flanked by large potted plants on either side of it. There's also a double door leading into the hotel, illuminated from within. --ar 64:25 The entrance of the luxury hotel features an open-air fireplace, a row of potted plants on both sides, and minimalist metal wall panels with black frames. The design is influenced by 4 nature, featuring modern simplicity and dark tones. It creates a luxurious atmosphere on New York City's Hope Street, with soft lighting creating an elegant ambiance. A large glass door leads directly to it from outside. --ar 64:25 ARCH361 - CMV  Liverpool School of Architecture Midjourney – Prompt craft – Reverse prompting Rather than using /imagine to generate images, Midjourney can produce text prompts for an image by using /describe 1 2 3 4 ARCH361 - CMV  Liverpool School of Architecture Explorations for AI-supported Concept Façade Design ▪ Using text-to-image diffusion models to explore building facades by mixing different concepts is a powerful way to spark creativity and push architectural boundaries. Bioluminescent Marine Life + Urban Facades /imagine a building facade inspired by bioluminescent creatures, with glowing, translucent sections that mimic jellyfish or deep-sea creatures ARCH361 - CMV  Liverpool School of Architecture Explorations for AI-supported Concept Façade Design ▪ Using text-to-image diffusion models to explore building facades by mixing different concepts is a powerful way to spark creativity and push architectural boundaries. Nature-Inspired Forms + Geometric Precision /imagine a building facade combining organic, flowing patterns from nature with precise, angular geometric shapes ARCH361 - CMV  Liverpool School of Architecture Explorations for AI-supported Concept Façade Design ▪ Using text-to-image diffusion models to explore building facades by mixing different concepts is a powerful way to spark creativity and push architectural boundaries. Ancient Ruins + Futuristic Technology /imagine a building facade blending the texture and form of ancient ruins with high-tech, futuristic elements like metallic and transparent surfaces ARCH361 - CMV  Liverpool School of Architecture Explorations for AI-supported Concept Façade Design ▪ Using text-to-image diffusion models to explore building facades by mixing different concepts is a powerful way to spark creativity and push architectural boundaries. Mosaic Art + Parametric Patterns /imagine a building facade inspired by colourful mosaic art combined with modern parametric patterns that shift in shape and form across the building ARCH361 - CMV  Liverpool School of Architecture Explorations for AI-supported Concept Façade Design ▪ Using text-to-image diffusion models to explore building facades by mixing different concepts is a powerful way to spark creativity and push architectural boundaries. Traditional Weaving Techniques + Glass and Steel /imagine a building facade that mimics traditional weaving patterns, intertwined with glass and steel materials ARCH361 - CMV  Liverpool School of Architecture Explorations for AI-supported Concept Façade Design ▪ Using text-to-image diffusion models to explore building facades by mixing different concepts is a powerful way to spark creativity and push architectural boundaries. Origami Folds + Minimalist Modernism /imagine a building facade inspired by intricate origami folds combined with minimalist, modernist design principles ARCH361 - CMV  Liverpool School of Architecture Explorations for AI-supported Concept Façade Design ▪ Using text-to-image diffusion models to explore building facades by mixing different concepts is a powerful way to spark creativity and push architectural boundaries. Crystal Formations + Modular Architecture /imagine a building facade combining the sharp, multifaceted forms of crystals with a modular architectural approach. ARCH361 - CMV  Liverpool School of Architecture Explorations for AI-supported Concept Façade Design ▪ Using text-to-image diffusion models to explore building facades by mixing different concepts is a powerful way to spark creativity and push architectural boundaries. Flowing Waterfalls + Sustainable Green Walls /imagine a building facade inspired by cascading waterfalls, integrated with green walls and vertical gardens. ARCH361 - CMV  Liverpool School of Architecture Explorations for AI-supported Concept Façade Design ▪ Using text-to-image diffusion models to explore building facades by mixing different concepts is a powerful way to spark creativity and push architectural boundaries. Bird Feathers + Textured Concrete /imagine a building facade that mimics the layered textures and colours of bird feathers, using concrete and textured surfaces. ARCH361 - CMV  Liverpool School of Architecture Midjourney – Prompt crafting Sentence that captures the main concept + architectural style and material + visual style/mood/texture/detail + enhancer words + [Midjourney parameters] /imagine modern waterfront home with large windows and open layout, enhancing the connection with the water, timber construction, ultra detailed, architectural photography Midjourney v5 ARCH361 - CMV  Liverpool School of Architecture Midjourney – Prompt craft – Parameters The Midjourney bot uses a seed number to create a field of visual noise, like television static, as a starting They are words placed at the end of the prompt that enforce some action in the algorithm point to generate the initial image grids. Seed numbers are generated randomly for each image but can be Typical examples can be: --seed / --no / --s / --iw / --chaos / --quality specified with the –seed parameter (accepts whole numbers 0–4294967295) Examples of outputs using the same prompt with randomly generated seeds (selected by Midjourney) ‘A house on a beach’ (seed 3345573798) ‘A house on a beach’ (seed 1924049066) ‘A house on a beach’ (seed 2362644341) ARCH361 - CMV  Liverpool School of Architecture Midjourney – Prompt craft – Parameters They are words placed at the end of the prompt that enforce some action in the algorithm The Midjourney bot uses a seed number to create a field of visual noise, like television static, as a starting point to generate the initial image grids. Seed numbers Typical examples can be: --seed / --no / --s / --iw / --chaos / --quality are generated randomly for each image but can be specified with the –seed parameter (accepts whole Examples of outputs forced using similar –seed values numbers 0–4294967295) ‘A house on a beach --seed 100’ ‘A house on a beach --seed 101’ ‘A house on a beach --seed 102’ ARCH361 - CMV  Liverpool School of Architecture Midjourney – Prompt craft – Parameters They are words placed at the end of the prompt that enforce some action in the algorithm --no: Negative prompting, e.g. --no plants Typical examples can be: --seed / --no / --s / --iw / --chaos / --quality would try to remove plants from the image. ‘A house on a beach –no trees’ ‘A house on a beach –no clouds’ ‘A house on a beach –no dunes’ ARCH361 - CMV  Liverpool School of Architecture Midjourney – Prompt craft – Parameters The Midjourney Bot has been trained to produce images that favour artistic colour, composition, and They are words placed at the end of the prompt that enforce some action in the algorithm forms. The --stylize or --s parameter influences