MM Biscuits V2 Multimodal Project Overview

MM Biscuits V2 Multimodal Project Overview

• This project focuses on improving large language models' (LLMs) ability to analyze images.
• Participants upload images, write prompts, rate model responses, and rewrite the best response.
• The goal is to enhance the model's understanding of images and provide more accurate and insightful responses.

Task Attempt Workflow

• Step 1: Image Selection and Upload
  • Choose an image that you are familiar with.
  • Ensure the image meets the image specifications outlined in the document.
• Step 2: Prompt Writing
  • Write a creative, unique, and complex prompt based on the image.
  • The prompt should be something you would ask a model about and challenge its reasoning abilities.
• Step 3: Rating Model Responses
  • Assess each model response based on four dimensions:
    • Image Grounding: How well the response connects to the image.
    • Truthfulness: Accuracy of the information provided.
    • Instruction Following: How well the model followed the prompt's instructions.
    • Writing Style: Clarity, fluency, and overall quality of writing.
  • Rate each dimension on a scale of 1 (major issues) to 3 (no issues).
• Step 4: Ranking Preferences
  • Rank your preference for the responses on a 1 to 5 scale.
  • Provide a justification for your ranking, highlighting the key reasons for choosing one response over the other.
• Step 5: Rewriting Responses
  • Rewrite the best model response to remove minor issues and improve writing style.
  • Focus on addressing any inaccuracies, inconsistencies, or unclear language.
• Step 6: Continuing Turns
  • Repeat steps 1-5 for multiple turns.
  • You can continue posing prompts that require the image and previous prompt/response pairs for up to 5 turns.

Image Specifications

• Images should be:
  • Relevant to a topic you are familiar with.
  • Capable of triggering thoughtful questions as prompts.
  • High enough resolution for legibility.
  • Less than 1 megabyte in size.
  • Free from PII (Personal Identifiable Information) and NSFW content.
  • Diverse in subject matter (e.g., charts, animals, flowers, etc.).
  • Aligned with the requested prompt type in the specific task.

Prompt Writing Specifications

• Prompts should:
  • Sound natural and authentic.
  • Reflect prompts that you or a friend would ask a chatbot.
  • Be uncontrived and avoid excessive complexity.
  • Incorporate an informal tone, abbreviations, minor spelling and grammar errors, and slang.

Project Rules

• Do not use ChatGPT or other AI tools to create images, write prompts, or evaluate responses.
• Violating these rules will result in account flags, removal from the project, and potential removal from the platform.