Introduction to Web Layers

Study Notes

The internet is divided into three main layers: Surface Web, Deep Web, and Dark Web.

The Surface Web is the normal, basic level of the internet and the primary entry point for most users.
It includes websites filled with data that are indexed by search engines.
Standard normal websites that users access daily reside here.
However, the Surface Web does not contain much information, which is instead contained deeper inside the internet.

The Deep Web contains most of the internet's data and is not a secret place.
It is accessible by knowing the URL, but most of the information here is related to personal information, such as medical and financial records.
Most of these sites require access control, and search engines do not index this information.
The Deep Web contains 400-500 times more information and has a 15% larger visit capacity than the Surface Web.
Deep Web pages are generated dynamically from a database and are difficult for standard crawler-based search engines to index.
Data in the Deep Web cannot be accessed directly through hyperlinks, but most useful web data is saved in the Deep Web.
The Deep Web is also called the hidden Web or invisible Web.

The information extraction task can be classified into three categories:
- Record-level: discovers record boundaries and divides them into separate attributes.
- Page-level: extracts all data embedded in one Web page.
- Site-level: populates a database from pages of a Web site.

The Deep Web can be accessed using special browsers like TOR (The Onion Router).
These browsers can find information not available through surface browsers, such as Hidden WIKI and Bitcoins.

Level S: ClearNet, includes social networks, search engines, wiki, e-mail services, and common internet content.
Level 1: Surface Web, includes closed social networks, e-commerce sites, blogs, university databases, hosting services, and forums.
Level 2: Bergie Web, includes web archives, torrents, streaming services, restricted access content, robots.txt, and anti-virus databases.
Level 3: Deep Web, includes hacking guides, spambots, virtual reality, computer security, and advanced artificial intelligence.
Level 4: Dark Web, includes hacking groups, hidden wiki, most .onion sites, drug dealers, and banned media.
Level 4B: Private Web, includes FPI mid-classified, supercomputing, closed systems, and geometric algorithms researches.
Level 5: Marianas Web, also called the final boss of the web, and is considered the most dangerous level.