GPT-3 Quiz

Building AI Apps with GPT: A McGill University and Steamship Tech Talk

• There is a high level of interest in the world of AI and open AI, specifically GPT, as evidenced by the rapid RSVPs for the tech talk.

• GPT is a large language model that uses a neural network to produce a probability distribution over a vocabulary of 50,000 words.

• GPT is trained on the internet to learn how words are likely to follow each other in a sequence and can generate new text based on this training.

• As the model scales up and is given more compute, it becomes more expressive and capable and gains some level of intelligence.

• Chat GPT is an example of how GPT can be used as a writing assistant, content generator, and chatbot.

• GPT3 was trained on a large dataset of questions and answers to operate in a Q&A form and gained 100 million users in one month.

• Instruction tuning and reinforcement alignment with human feedback are key to turning GPT into an agent capable of achieving ambiguous goals.

• Steamship is building AWS for AI apps and has seen a variety of apps built and deployed using language models, including companionship bots, question answering bots, utility functions, creativity, and wild experiments.

• Companionship bots can be built by wrapping GPT in an endpoint that injects a purpose for the friend.

• Question answering bots can assist with tasks such as conforming to style guidelines or helping with homework.

• Utility functions can automate tasks that humans can do with language computation and knowledge, such as reading tweets and recommending the ones to read.

• Wild experiments involve letting the AI decide what to do and be self-directed.Building AI Companions, Question Answering Systems, and Creative Tools

• AI companions can be built by engineering personalized prompts that consistently perform as desired.

• A Mandarin idiom coach was built in a hackathon to generate Chinese idioms based on a given problem and respond with examples and encouragement.

• A wrapper around GPT can be used to inject personality into prompts and add tools (e.g. web search, image generation) to interact with GPT.

• Question answering systems can be built by cutting up documents into fragments, turning them into embedding vectors, and storing them in a vector database.

• A sliding window can be used to extract fragments of text for the vector database, and the prompt can be engineered to instruct the AI to answer specific questions using only the provided source materials.

• Simple question answering systems can be built by loading a list of things the AI knows and having it respond to user questions based on that list.

• Utility functions that automate tasks requiring basic language understanding (e.g. generating unit tests, looking up documentation, brand checks) are low-hanging fruit for AI builders.

• AI can be used to assist in creative processes by generating possibilities that can be edited down by human editors.

• Domain knowledge is important in creative AI applications because it allows for pre-agreed editing and deletion of generated content.

• AI-generated content raises questions about intellectual property and artistic style.

• AI tools are accessible to CS 101 grads and industry professionals alike, and can be deployed easily through platforms like Replit and Telegram.

• Experimentation and creative tinkering with AI tools can lead to new applications and solutions in various fields.Building AI Applications with GPT-3

• GPT-3 is a language model that can generate human-like text.

• GPT-3 can be used to build a variety of AI applications, including chatbots, content generators, and recommendation systems.

• Domain knowledge is important when using GPT-3 to generate text, as it can help to provide more targeted and accurate results.

• One example of an AI application built with GPT-3 is a writing recommendation system called Writing Atlas, which suggests stories based on user input and domain knowledge.

• Prompt engineering is a key aspect of building AI applications with GPT-3, as it involves providing specific prompts to guide the language model's output.

• Baby AGI, or Auto GPT, is an emerging area of research that involves using GPT-3 to build multi-step planning bots that can self-direct their actions.

• Hallucinations are a common problem when using GPT-3, as the language model lacks a ground truth and relies on differentiation to generate text.

• Mitigating hallucinations can involve providing more examples, letting the model search the web, or using external databases for domain-specific knowledge.

• Over-engineering AI systems can help to reduce the risk of errors, as seen in spacecraft that use multiple computers that must agree before taking action.

• Collaborative AI, where multiple software agents work together, may become a more common approach to building AI applications in the future.

• Using specific prompts, such as "my best guess is," can help to guide GPT-3's output and reduce the risk of hallucinations.

• GPT-3 can simulate personalities and predict how conscious beings might react in a given situation, making it useful for building chatbots and other conversational AI applications.Discussion on GPT-3 and its potential business value

• GPT-3 is a language model capable of generating text that sounds like it was written by a human.

• It can complete a transcript or a story in which a character is present and interacting.

• The model is not capable of reasoning or thinking like humans, but it can pass some tests that demonstrate some sort of rationale or logic within the model.

• Prompting is a finicky process, and the model is very sensitive to the way it is prompted.

• GPT-4, which was released in March, passed the LSAT test, but it still requires the right approach to prompt it correctly.

• Companies are already using GPT-3 to develop AI apps that apply it to data and combine it with domain knowledge and interface.

• GPT-3 is a foundation model that will likely slide into the ether as more models emerge in the future.

• The model is useful but requires some art to find the right prompt and post-process it for checking.

• One can fine-tune instruction-tuning style models, which are more likely to respond with the computer parsable output.

• There are three versions of GPT-3: SAS version, Enterprise version, and the maximalist version where one runs their own machines and models.

• Companies like Amazon, Microsoft, and Google are offering SAS and VPC versions of GPT-3.

• GPT-3's privacy implications are significant, and Childcpt recently updated their privacy policy to not use prompts for the training process.