UNIT1-OSINT.pdf

SOCIAL ENGINEERING AND OSINT UNIT I Aditya Rajesh More OSINT OSINT, or Open Source Intelligence, refers to the process of collecting, analyzing, and utilizing information that is publicly available. OSINT is commonly used in various fields such as intelligence, security, law enforcement, jour...

SOCIAL ENGINEERING AND OSINT UNIT I Aditya Rajesh More OSINT OSINT, or Open Source Intelligence, refers to the process of collecting, analyzing, and utilizing information that is publicly available. OSINT is commonly used in various fields such as intelligence, security, law enforcement, journalism, and competitive business analysis. It helps organizations and individuals gather valuable insights, identify trends, and make informed decisions based on readily accessible data. Collecting open-source intelligence is achieved in a variety of different ways, such as: ▪ Social Media Intelligence, which is acquired from viewing or observing a subjects online social profile activity. ▪ Search engine data mining or scraping. ▪ Public records checking. ▪ Information matching and verification from data broker services. Social Engineering and OSINT Unit I- by Aditya More 2 History of OSINT Early Documentation and Practices Mid-19th Century: OSINT practices were documented as early as the mid-19th century in the United States. Early 20th Century: Similar practices were recorded in the United Kingdom during the early 20th century. Development in the United States 1941: The United States established the Foreign Broadcast Monitoring Service (FBMS), later renamed the Foreign Broadcast Information Service (FBIS), to monitor foreign broadcasts. An example of their work includes correlating changes in the price of oranges in Paris with successful bombings of railway bridges during World War II. Social Engineering and OSINT Unit I- by Aditya More 3 History of OSINT Recommendations and Reforms 1996: The Aspin-Brown Commission highlighted that U.S. access to open sources was "severely deficient" and recommended that improving this should be a "top priority" for both funding and the attention of the Director of Central Intelligence (DCI). Post-9/11: In July 2004, following the September 11 attacks, the 9/11 Commission recommended the creation of an open-source intelligence agency. March 2005: The Iraq Intelligence Commission further recommended the creation of an open-source directorate within the CIA. Social Engineering and OSINT Unit I- by Aditya More 4 History of OSINT Establishment of the DNI Open Source Center November 2005: The Director of National Intelligence (DNI) announced the creation of the DNI Open Source Center. The center was tasked with collecting information from various open sources, including the internet, databases, press, radio, television, video, geospatial data, photos, and commercial imagery. Involvement of the Private Sector In-Q-Tel: This Central Intelligence Agency-supported venture capital firm in Arlington, VA, has played a significant role in assisting companies to develop web-monitoring and predictive analysis tools. Social Engineering and OSINT Unit I- by Aditya More 5 Methods of OSINT Web Scraping: Description: Web scraping involves using automated tools or scripts to extract data from websites. This can include anything from simple text to complex datasets. Uses: It is commonly used to gather large volumes of data quickly, which can then be analyzed to identify trends, patterns, or specific information. Social Media Monitoring: Description: This method involves tracking activities, posts, and interactions on social media platforms like Twitter, Facebook, and Instagram. Uses: Social media monitoring can help analyze public sentiment, identify trending topics, and detect influential figures or potential threats. Social Engineering and OSINT Unit I- by Aditya More 6 Methods of OSINT Public Records Search: Description: Public records searches involve accessing government and official records that are available to the public. These can include property records, court documents, patents, and business filings. Uses: This method is useful for verifying information about individuals and organizations, such as ownership of assets, legal history, and corporate details. Database Searches: Description: Utilizing online databases and subscription services to find specific information. These databases can include academic journals, news archives, and industry-specific repositories. Uses: Database searches provide access to a wealth of detailed information that may not be easily found through general web searches. Social Engineering and OSINT Unit I- by Aditya More 7 Methods of OSINT Geospatial Intelligence (GEOINT): Description: This involves analyzing geographical data from maps, satellite images, and geographic information systems (GIS). Uses: GEOINT is particularly useful for tracking movements, planning routes, and understanding the physical characteristics of an area. Media Analysis: Description: Reviewing news articles, press releases, broadcasts, and other media sources to gather information. Uses: Media analysis helps understand public perception, identify emerging issues, and gather timely information on events and developments. Social Engineering and OSINT Unit I- by Aditya More 8 Basic Tools of OSINT Maltego: Description: Maltego is a data mining tool that performs link analysis and visualization of relationships between entities such as people, organizations, and websites. Uses: It helps uncover hidden connections and patterns that are not immediately apparent. Google Dorking: Description: Google Dorking uses advanced search operators to refine Google search queries, uncovering specific and often hidden information. Uses: This can reveal unsecured pages, confidential files, and other sensitive data that are accessible online. Social Engineering and OSINT Unit I- by Aditya More 9 Basic Tools of OSINT Shodan: Description: Shodan is a search engine for internet-connected devices. It indexes information about devices such as webcams, routers, servers, and more. Uses: Shodan can be used to find vulnerable devices, assess security risks, and gather information about the digital infrastructure. Recon-ng: Description: Recon-ng is a reconnaissance framework written in Python that automates information gathering tasks. Uses: It provides modules for domain searches, social media scraping, and other data collection activities, making the process more efficient. Social Engineering and OSINT Unit I- by Aditya More 10 Basic Tools of OSINT Social-Engineer Toolkit (SET): Description: SET is an open-source tool for simulating social engineering attacks and penetration testing. Uses: It helps in gathering information about targets by simulating real-world social engineering tactics. SpiderFoot: Description: SpiderFoot is an open-source intelligence automation tool that collects data from various sources. Uses: It aggregates data from over 100 sources, including social media, public records, and DNS, to provide comprehensive intelligence. WHOIS Lookup: Description: WHOIS is a query and response protocol that retrieves information about domain names. Uses: It provides details about the domain owner, registration dates, and contact information, useful for identifying the entities behind websites. Social Engineering and OSINT Unit I- by Aditya More 11 Legal Boundaries of OSINT in India Privacy Laws Constitutional Right to Privacy: Recognized as a fundamental right under Article 21 of the Indian Constitution following the Supreme Court's verdict in 2017. OSINT activities must respect this fundamental right by ensuring personal data is collected and used lawfully. Information Technology (IT) Act, 2000: ▪ Section 43A: Mandates compensation for failure to protect personal data. ▪ Section 72A: Punishes disclosure of personal information without consent. Personal Data Protection Bill (PDPB), 2019: Pending enactment, this bill aims to provide a comprehensive data protection framework, requiring data fiduciaries to handle personal data responsibly and securely. Social Engineering and OSINT Unit I- by Aditya More 12 Legal Boundaries of OSINT in India Intellectual Property Rights Copyright Act, 1957: Protects the rights of creators over their original works, such as texts, images, and software. OSINT practitioners must avoid unauthorized use of copyrighted material. Trademarks Act, 1999: Protects brand names and logos. Unauthorized use can lead to legal consequences. Patents Act, 1970: Protects inventions and innovations. OSINT activities must respect these protections. Information Technology Rules, 2021: These rules provide guidelines for intermediaries and digital media to ensure compliance with Indian laws, including the protection of user privacy and prohibition of certain types of content. Social Engineering and OSINT Unit I- by Aditya More 13 Legal Boundaries of OSINT in India Right to Information Act (RTI), 2005 Description: Provides the public with the right to access information held by public authorities to promote transparency. Usage: OSINT practitioners can file RTI requests to obtain information from government agencies, following the proper procedures and respecting exemptions related to national security and privacy. Terms of Service Agreements Description: Websites and online services have terms of service that users must comply with. Implications: Violating these agreements, such as by using automated tools to scrape data when prohibited, can result in legal actions and bans from the service. OSINT practitioners must read and adhere to these terms. Social Engineering and OSINT Unit I- by Aditya More 14 Ethical Considerations Respect for Privacy: Minimizing the collection of personal data and avoiding intrusive methods. Accuracy and Verification: Ensuring the reliability of collected information by verifying it through multiple sources. Avoiding Harm: Considering the potential consequences of their activities to prevent physical, financial, or reputational harm. Transparency: Disclosing sources and methodologies whenever possible while protecting sensitive information. Purpose and Necessity: Ensuring activities are necessary and justified by legitimate purposes. Respecting Human Rights: Aligning OSINT activities with broader human rights principles to avoid contributing to abuses or undermining individual dignity and rights. Social Engineering and OSINT Unit I- by Aditya More 15 OSINT Sources Internet Sources- Search Engines: Google, Bing, Yahoo Social Media Platforms: Twitter, Facebook, LinkedIn, Instagram, Reddit Public Websites: o Government Portals o Company Websites o News Websites Forums and Discussion Boards: Reddit, Quora, Stack Exchange Online Databases: PubMed, JSTOR, LexisNexis Multimedia Content: YouTube, Vimeo, podcasts, webinars Social Engineering and OSINT Unit I- by Aditya More 16 OSINT Sources Offline Sources Print Media: Newspapers, magazines, journals Books and Academic Publications: Books, theses, dissertations, academic conference papers Public Records and Documents: Birth and death certificates, marriage licenses, land records, court records Interviews and Personal Contacts: Expert interviews, eyewitness accounts, professional networking Social Engineering and OSINT Unit I- by Aditya More 17 OSINT Sources Deep Web Sources Academic Databases: Examples: JSTOR, IEEE Xplore, SpringerLink. Description: Access to scholarly articles and technical reports not indexed by standard search engines. Subscription-Based Services: Examples: LexisNexis, Factiva, ProQuest. Description: Comprehensive archives of news, legal documents, and market research reports. Government Databases: Examples: SEC’s EDGAR, national archives, census databases. Description: Official data, including financial filings and demographic information. Social Engineering and OSINT Unit I- by Aditya More 18 OSINT Sources Corporate and Financial Data: Examples: Bloomberg, Reuters, Dun & Bradstreet. Description: Business intelligence, market analysis, and financial records. Specialized Search Engines: Examples: Pipl, Spokeo, Intelius. Description: Find hidden or less accessible personal information. Medical and Scientific Research: Examples: ClinicalTrials.gov, PubMed, ResearchGate. Description: Access to clinical trial data, medical research papers, and scientific studies. Social Engineering and OSINT Unit I- by Aditya More 19 Using search engines effectively The Google Hacking Database (GHDB) is a compilation of Google search queries, known as "Google dorks" or "Google hacks," which can be used to find sensitive information or vulnerabilities on websites. These queries leverage Google's advanced search operators to locate specific types of data, such as exposed files, directories, error messages, and other information that may be inadvertently indexed by search engines. It is made up of carefully formulated search queries, known as “dorks,” that are designed to locate specific categories of information or potential security flaws. Social Engineering and OSINT Unit I- by Aditya More 20 Using search engines effectively 1. site: Limits search results to a specific website or domain. Example: site:example.com (Searches only within the website example.com) 2. intitle: Finds pages with specific words in the title. Example: intitle:"login page" (Finds pages with "login page" in the title) 3. inurl: Searches for URLs containing specific words. Example: inurl:admin (Finds URLs containing "admin") 4. filetype: Searches for specific file types. Example: filetype:pdf (Finds PDF files) 5. intext: Searches for specific words within the text of a page. Example: intext:"confidential" (Finds pages containing the word "confidential") Social Engineering and OSINT Unit I- by Aditya More 21 Investigating Social Media Profile Analysis: Description: Gather and analyze information from user profiles on platforms like Facebook, Twitter, LinkedIn, and Instagram. This can include personal details, employment history, education, interests, and connections. Use Cases: Identifying individuals, verifying identities, mapping relationships. Post and Activity Tracking: Description: Monitor and analyze posts, comments, likes, and other activities to understand behavior, interests, and affiliations. Use Cases: Detecting trends, gathering evidence, understanding sentiment. Hashtag and Keyword Tracking: Description: Monitor specific hashtags and keywords to follow conversations and trends. Use Cases: Identifying trending topics, understanding public opinion, tracking event discussions. Social Engineering and OSINT Unit I- by Aditya More 22 Investigating Social Media Image and Video Analysis: Description: Analyze images and videos posted on social media to extract information such as locations, faces, objects, and events. Use Cases: Identifying locations, verifying events, recognizing individuals. Geolocation: Description: Use geotagged posts to determine the physical locations of users or events. Use Cases: Tracking movements, verifying locations, gathering location-based insights. Network Analysis: Description: Map and analyze connections between users to understand relationships and influence networks. Use Cases: Identifying key influencers, mapping social networks, understanding group dynamics. Social Engineering and OSINT Unit I- by Aditya More 23