Difference between revisions of "Deep web"
From Gender and Tech Resources
(2 intermediate revisions by one other user not shown) | |||
Line 1: | Line 1: | ||
== What is the deep web??== | == What is the deep web??== | ||
− | + | The deep web refers to all the information that is in the world wide web but is not indexed, so not accessible using search engines such as Google <ref>https://en.wikipedia.org/wiki/Deep_web</ref>. Web indexing (or Internet indexing) refers to various methods for indexing the contents of a website or of the entire Internet. Individual websites or intranets may use a "back-of-the-book" index, while search engines usually use keywords and metadata to provide a more useful vocabulary for Internet or on-site searching. With the increase in the number of periodicals that have articles online, web indexing is also becoming important for periodical websites <ref>https://en.wikipedia.org/wiki/Web_indexing</ref> | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | The deep web refers to all the information that is in the world wide web but is not indexed, so | + | |
[[File:deep_web.png|400px|center|alt=Web profunda]] | [[File:deep_web.png|400px|center|alt=Web profunda]] | ||
Line 179: | Line 9: | ||
A common confusions is to mix up the deep web and the dark web; the latter corresponds to a part of the deep web, which besides not being indexed, has been hidden using methods such as masking the IP address and/or requiring special permissions or the use of specific software to access it <ref>https://en.wikipedia.org/wiki/Dark_web</ref>. | A common confusions is to mix up the deep web and the dark web; the latter corresponds to a part of the deep web, which besides not being indexed, has been hidden using methods such as masking the IP address and/or requiring special permissions or the use of specific software to access it <ref>https://en.wikipedia.org/wiki/Dark_web</ref>. | ||
− | But if | + | But if the deep web is visible, how do we access it? Below we describe two ways you can immerse yourself in the deep web. |
== Navigating the deep web using Tor== | == Navigating the deep web using Tor== | ||
Line 189: | Line 19: | ||
Considering the wide variety of users and type of information that you can find inside the deep web, using the '''TOR browser''' is normally the easiest and safest method <Ref> https://www.torproject.org/projects/torbrowser.html.en </ ref>, and is available for various operating systems. Alternatively, for optimum security we recommend '''TAILS''' <ref>https://tails.boum.org/</ref>, which is a highly secure version of Linux you boot on your computer from a USB stick or other external device.. | Considering the wide variety of users and type of information that you can find inside the deep web, using the '''TOR browser''' is normally the easiest and safest method <Ref> https://www.torproject.org/projects/torbrowser.html.en </ ref>, and is available for various operating systems. Alternatively, for optimum security we recommend '''TAILS''' <ref>https://tails.boum.org/</ref>, which is a highly secure version of Linux you boot on your computer from a USB stick or other external device.. | ||
− | But if it is not indexed | + | But if it the deep web is not indexed how do navigate it? |
===Case A=== | ===Case A=== | ||
− | You | + | You find a website with a .onion address. “.onion" is a special-use top level domain suffix designating an anonymous hidden service only reachable via the Tor network. These addresses are not actual DNS names, and the .onion TLD is not in the Internet DNS root. But with the appropriate proxy software installed, Internet programs such as web browsers can access sites with .onion addresses by sending the request through the network of Tor servers. The purpose of using such a system is to make both the information provider and the person accessing the information more difficult to trace, whether by one another, by an intermediate network host, or by an outsider <ref>https://en.wikipedia.org/wiki/.onion</ref>. |
For instance, you receive: | For instance, you receive: | ||
Line 235: | Line 65: | ||
https://www.reddit.com//r/onions | https://www.reddit.com//r/onions | ||
− | You should consider that the availability of information in many cases is temporary, so you will need to spend some time to getting used to the deep web so you search more fruitfully. Also, language can be a limitation; before entering forums remember that they generally use specific jargon<ref>http://domekit.cc/survival-kit-in-the-deep-web/</ref>. | + | You should consider that the availability of information in many cases is temporary, so you will need to spend some time to getting used to the deep web so you can search more fruitfully. Also, language can be a limitation; before entering forums remember that they generally use specific jargon<ref>http://domekit.cc/survival-kit-in-the-deep-web/</ref>. |
===Case C=== | ===Case C=== | ||
Line 251: | Line 81: | ||
The second option is using zeronet.io. This tool allows you to publish open, free and uncensored blogs and it uses Bitcoin cryptography and the BitTorrent network. | The second option is using zeronet.io. This tool allows you to publish open, free and uncensored blogs and it uses Bitcoin cryptography and the BitTorrent network. | ||
− | Although you can use this tool with different operating systems, the option we recommend is '''zeronet + TAILS''' (https://tails.boum.org/) | + | Although you can use this tool with different operating systems, the safest method and the option we recommend is '''zeronet + TAILS''' (https://tails.boum.org/). However, with this approach you need to dedicate a little more time to its configuration since you must know how to enable ports on an internet modem. |
[[File:zeronet_logo.png|120px|none|alt=zeronet.io|ZeroNet]] | [[File:zeronet_logo.png|120px|none|alt=zeronet.io|ZeroNet]] | ||
Line 299: | Line 129: | ||
You should now have successfully started Zeronet | You should now have successfully started Zeronet | ||
− | 3) In the browser '''TOR''' you need to open the options of '''Configuration''' and select ''Advanced'' where you will find the tab '''Network''' and in the first point you need to go to '''Connection''' '''Options''' and add in '''No | + | 3) In the browser '''TOR''' you need to open the options of '''Configuration''' and select ''Advanced'' where you will find the tab '''Network''' and in the first point you need to go to '''Connection''' '''Options''' and add in '''No proxy''' 127.0.0.1 or your own inet_addr |
[[File:Proxy_conf_tor.png|400px|center|alt=Configuration options for the TOR browser]] | [[File:Proxy_conf_tor.png|400px|center|alt=Configuration options for the TOR browser]] |
Latest revision as of 09:17, 20 March 2017
Contents
What is the deep web??
The deep web refers to all the information that is in the world wide web but is not indexed, so not accessible using search engines such as Google [1]. Web indexing (or Internet indexing) refers to various methods for indexing the contents of a website or of the entire Internet. Individual websites or intranets may use a "back-of-the-book" index, while search engines usually use keywords and metadata to provide a more useful vocabulary for Internet or on-site searching. With the increase in the number of periodicals that have articles online, web indexing is also becoming important for periodical websites [2]
According to Wikipedia, "The portion of the web that is indexed by standard search engines is known as the surface web. As of 2001, the deep web was several orders of magnitude larger than the surface web. An analogy of an iceberg used by Denis Shestakov represents the division between surface web and deep web respectively: It is impossible to measure, and harsh to put estimates on, the size of the deep web because the majority of the information is hidden or locked inside databases. Early estimates suggested that the deep web is 400 to 550 times larger than the surface web. However, since more information and sites are always being added, it can be assumed that the deep web is growing exponentially at a rate that cannot be quantified.”
A common confusions is to mix up the deep web and the dark web; the latter corresponds to a part of the deep web, which besides not being indexed, has been hidden using methods such as masking the IP address and/or requiring special permissions or the use of specific software to access it [3].
But if the deep web is visible, how do we access it? Below we describe two ways you can immerse yourself in the deep web.
One of the easiest ways is to use the hidden service protocol TOR [4] that operates within the anonymous distributed TOR network [5].
Considering the wide variety of users and type of information that you can find inside the deep web, using the TOR browser is normally the easiest and safest method Cite error: Closing </ref>
missing for <ref>
tag, which is a highly secure version of Linux you boot on your computer from a USB stick or other external device..
But if it the deep web is not indexed how do navigate it?
Case A
You find a website with a .onion address. “.onion" is a special-use top level domain suffix designating an anonymous hidden service only reachable via the Tor network. These addresses are not actual DNS names, and the .onion TLD is not in the Internet DNS root. But with the appropriate proxy software installed, Internet programs such as web browsers can access sites with .onion addresses by sending the request through the network of Tor servers. The purpose of using such a system is to make both the information provider and the person accessing the information more difficult to trace, whether by one another, by an intermediate network host, or by an outsider [6].
For instance, you receive:
http://duskgytldkxiuqc6.onion
you need to change the .onion for something like:
.onion.to
.onion.city
.onion.cab
or any other domain made available by the network of volunteers of TOR2web [7].
and you need to open it up in the browser TOR.
Case B
You want to look for a page.
You can begin with: http://deepweb.pw/
This page can be opened in any browser, but you can only access .onion pages using the TOR browser.
Here you can find interesting sites like the Hidden wiki.
You can also look for information here:
https://ahmia.fi/search/
Or if you are interested in websites about leaks:
https://en.wikipedia.org/wiki/GlobaLeaks#Implementations
Other possible search places:
http://3g2upl4pq6kufc4m.onion/
https://www.reddit.com//r/onions
You should consider that the availability of information in many cases is temporary, so you will need to spend some time to getting used to the deep web so you can search more fruitfully. Also, language can be a limitation; before entering forums remember that they generally use specific jargon[8].
Case C
You can also use the hidden services of Riseup
https://help.riseup.net/en/tor
Adding data to the deep web
Now you can surf the deep web, but how do you add information?
If you need to add a file you can use the Riseup Share service that lets you upload a file of up to 50mb for no more than a week. You can find the .onion.to league at: https://help.riseup.net/en/tor
The second option is using zeronet.io. This tool allows you to publish open, free and uncensored blogs and it uses Bitcoin cryptography and the BitTorrent network.
Although you can use this tool with different operating systems, the safest method and the option we recommend is zeronet + TAILS (https://tails.boum.org/). However, with this approach you need to dedicate a little more time to its configuration since you must know how to enable ports on an internet modem.
What do you need?
One TAILS USB with persistence activated.
Enable the port 15441 of your modem
1) Login to a TAILS session using persistence and advanced settings to access the root terminal.
2) In the terminal you need to write the following commands:
- To update and install the required applications:
$ sudo apt-get update $ sudo apt-get upgrade $ sudo apt-get install python-msgpack python-gevent
For downloading zeronet:
$ wget https://github.com/HelloZeroNet/ZeroNet/archive/master.tar.gz
To unzip the downloaded file:
$ tar xvpfz master.tar.gz
To enter the file:
$ cd ZeroNet-master
For starting:
$ python zeronet.py
If this option does not work you need to check your IP
$ sudo ifconfig
This command will tell you your internet address (inet addr), for example in the image it is 10.14.241.166
$ python zeronet.py --proxy 127.0.0.1:9050 --tor disable --ui_ip inet_addr
In the previous command you must replace inet_addr with the one that corresponds to your computer
You should now have successfully started Zeronet
3) In the browser TOR you need to open the options of Configuration and select Advanced where you will find the tab Network and in the first point you need to go to Connection Options and add in No proxy 127.0.0.1 or your own inet_addr
After you can open/
http://inet_addr:43110/
for instance:
http://127.0.0.1:43110
And you will find the graphical interface
To create a new blog you only need to activate Zero blog and use the tool for cloning.
Note:
You will only be able to upload information to the network zeronet if your port 15441 is enabled You can see more information about set up and configuration here ZeroNet
References
- ↑ https://en.wikipedia.org/wiki/Deep_web
- ↑ https://en.wikipedia.org/wiki/Web_indexing
- ↑ https://en.wikipedia.org/wiki/Dark_web
- ↑ https://www.torproject.org/docs/hidden-services.html.en
- ↑ https://www.torproject.org/about/overview.html.en#thesolution
- ↑ https://en.wikipedia.org/wiki/.onion
- ↑ https://tor2web.org/
- ↑ http://domekit.cc/survival-kit-in-the-deep-web/