Difference between revisions of "Deep web"

From Gender and Tech Resources

(Created page with "== What is the deep web??== The deep web refers to all the information that is in the world wide we but is not indexed so it is not possible to access it using search engines...")
 
 
(7 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
== What is the deep web??==
 
== What is the deep web??==
  
The deep web refers to all the information that is in the world wide we but is not indexed so it is not possible to access it using search engines like Google <ref>https://en.wikipedia.org/wiki/Deep_web</ref>. Web indexing (or Internet indexing) refers to various methods for indexing the contents of a website or of the Internet as a whole. Individual websites or intranets may use a back-of-the-book index, while search engines usually use keywords and metadata to provide a more useful vocabulary for Internet or onsite searching. With the increase in the number of periodicals that have articles online, web indexing is also becoming important for periodical websites  <ref>https://en.wikipedia.org/wiki/Web_indexing</ref>
+
The deep web refers to all the information that is in the world wide web but is not indexed, so not accessible using search engines such as Google <ref>https://en.wikipedia.org/wiki/Deep_web</ref>. Web indexing (or Internet indexing) refers to various methods for indexing the contents of a website or of the entire Internet. Individual websites or intranets may use a "back-of-the-book" index, while search engines usually use keywords and metadata to provide a more useful vocabulary for Internet or on-site searching. With the increase in the number of periodicals that have articles online, web indexing is also becoming important for periodical websites  <ref>https://en.wikipedia.org/wiki/Web_indexing</ref>
  
 
[[File:deep_web.png|400px|center|alt=Web profunda]]
 
[[File:deep_web.png|400px|center|alt=Web profunda]]
  
According to the wikipedia "The portion of the web that is indexed by standard search engines is known as the surface web. As of 2001, the deep web was several orders of magnitude larger than the surface web. An analogy of an iceberg used by Denis Shestakov represents the division between surface web and deep web respectively: It is impossible to measure, and harsh to put estimates on, the size of the deep web because the majority of the information is hidden or locked inside databases. Early estimates suggested that the deep web is 400 to 550 times larger than the surface web. However, since more information and sites are always being added, it can be assumed that the deep web is growing exponentially at a rate that cannot be quantified.”
+
According to Wikipedia, "The portion of the web that is indexed by standard search engines is known as the surface web. As of 2001, the deep web was several orders of magnitude larger than the surface web. An analogy of an iceberg used by Denis Shestakov represents the division between surface web and deep web respectively: It is impossible to measure, and harsh to put estimates on, the size of the deep web because the majority of the information is hidden or locked inside databases. Early estimates suggested that the deep web is 400 to 550 times larger than the surface web. However, since more information and sites are always being added, it can be assumed that the deep web is growing exponentially at a rate that cannot be quantified.”
  
A common confusions is to mix up the deep web and the dark web. Latter corresponds to a part of the deep web, which besides not being indexed has been hidden using methods such as masking IP and/or requiring special permissions or the use of specific software to access it <ref>https://en.wikipedia.org/wiki/Dark_web</ref>.  
+
A common confusions is to mix up the deep web and the dark web; the latter corresponds to a part of the deep web, which besides not being indexed, has been hidden using methods such as masking the IP address and/or requiring special permissions or the use of specific software to access it <ref>https://en.wikipedia.org/wiki/Dark_web</ref>.  
  
But if it is not indexed, how do we access it? Below we describe two ways you can immerse yourself in the deep web.
+
But if the deep web is visible, how do we access it? Below we describe two ways you can immerse yourself in the deep web.
  
 
== Navigating the deep web using Tor==
 
== Navigating the deep web using Tor==
Line 15: Line 15:
 
[[File:Tor-logo-2011-flat.svg|120px|left|alt=TOR]]
 
[[File:Tor-logo-2011-flat.svg|120px|left|alt=TOR]]
  
One of the easiest ways is to use the  hidden service protocol TOR <ref>https://www.torproject.org/docs/hidden-services.html.en</ref> that operates within the anonymous distributed network of TOR <ref>https://www.torproject.org/about/overview.html.en#thesolution</ref>.  
+
One of the easiest ways is to use the  hidden service protocol TOR <ref>https://www.torproject.org/docs/hidden-services.html.en</ref> that operates within the anonymous distributed TOR network <ref>https://www.torproject.org/about/overview.html.en#thesolution</ref>.  
  
Considering the wide variety of users and type of information that you can find inside the deep web, it is initially easier and safer to access it using the  '''TOR browser''' <Ref> https://www.torproject.org/projects/torbrowser.html.en </ ref> that is available for various operating systems. Alternatively to maximize your security we recommend the use of '''TAILS''' <ref>https://tails.boum.org/</ref>.
+
Considering the wide variety of users and type of information that you can find inside the deep web, using the  '''TOR browser''' is normally the easiest and safest method <Ref> https://www.torproject.org/projects/torbrowser.html.en </ ref>, and is available for various operating systems. Alternatively, for optimum security we recommend '''TAILS''' <ref>https://tails.boum.org/</ref>, which is a highly secure version of Linux you boot on your computer from a USB stick or other external device..
  
Maybe you can ask yourself  if it is not indexed where to start?
+
But if it the deep web is not indexed how do navigate it?
  
 
===Case A===
 
===Case A===
You get an .onion address. Those are “.onion is a special-use top level domain suffix designating an anonymous hidden service reachable via the Tor network. Such addresses are not actual DNS names, and the .onion TLD is not in the Internet DNS root, but with the appropriate proxy software installed, Internet programs such as web browsers can access sites with .onion addresses by sending the request through the network of Tor servers. The purpose of using such a system is to make both the information provider and the person accessing the information more difficult to trace, whether by one another, by an intermediate network host, or by an outsider  <ref>https://en.wikipedia.org/wiki/.onion</ref>.  
+
You find a website with a .onion address. “.onion" is a special-use top level domain suffix designating an anonymous hidden service only reachable via the Tor network. These addresses are not actual DNS names, and the .onion TLD is not in the Internet DNS root. But with the appropriate proxy software installed, Internet programs such as web browsers can access sites with .onion addresses by sending the request through the network of Tor servers. The purpose of using such a system is to make both the information provider and the person accessing the information more difficult to trace, whether by one another, by an intermediate network host, or by an outsider  <ref>https://en.wikipedia.org/wiki/.onion</ref>.  
  
 
For instance, you receive:
 
For instance, you receive:
Line 47: Line 47:
 
You want to look for a page.
 
You want to look for a page.
  
You can begin in:
+
You can begin with:
 
http://deepweb.pw/
 
http://deepweb.pw/
  
This page could be opened in any browser but only using '''TOR''' browser you will be able to access  .onion pages.  
+
This page can be opened in any browser, but you can only access .onion pages using the '''TOR''' browser.  
  
There you can find interesting sites like the Hidden wiki.
+
Here you can find interesting sites like the Hidden wiki.
  
 
You can also look for information here:
 
You can also look for information here:
Line 65: Line 65:
 
  https://www.reddit.com//r/onions
 
  https://www.reddit.com//r/onions
  
You should consider that the availability of information in many cases is temporary, so you will need to spend considerable time to get used to the deep web and search more fruitfully. Also the language can be a limitation and before entering forums remember that they generally use specific jargon<ref>http://domekit.cc/survival-kit-in-the-deep-web/</ref>.  
+
You should consider that the availability of information in many cases is temporary, so you will need to spend some time to getting used to the deep web so you can search more fruitfully. Alsolanguage can be a limitation; before entering forums remember that they generally use specific jargon<ref>http://domekit.cc/survival-kit-in-the-deep-web/</ref>.  
  
 
===Case C===
 
===Case C===
Line 73: Line 73:
 
==Adding data to the deep web==
 
==Adding data to the deep web==
  
Well now you surf the web deep but how do you add information?
+
Now you can surf the deep web, but how do you add information?
  
If you need to add a file you can use the services '' Share '' of Riseup that let you upload a file up to 50mb for no more than a week. You can find the .onion.to league at:  https://help.riseup.net/en/tor
+
If you need to add a file you can use the Riseup '' Share '' service that lets you upload a file of up to 50mb for no more than a week. You can find the .onion.to league at:  https://help.riseup.net/en/tor
  
 
[[File:Deep_web4.png|400px|center|alt= Riseup Services]]
 
[[File:Deep_web4.png|400px|center|alt= Riseup Services]]
  
The second option is using zeronet.io. This tool allows you to publish open, free and uncensored blogs and it uses the Bitcoin cryptography and the BitTorrent network.
+
The second option is using zeronet.io. This tool allows you to publish open, free and uncensored blogs and it uses Bitcoin cryptography and the BitTorrent network.
  
Si bien puedes hacer uso de esta herramienta en diferentes sistemas operativos la opción que recomendamos es '''zeronet + TAILS'''  (https://tails.boum.org/) debido a que representa la forma más segura, sin embargo requiere que dediques un poco más de tiempo a su configuración ya que debes saber habilitar puertos en el modem de internet.
+
Although you can use this tool with different operating systems, the safest method and the option we recommend is '''zeronet + TAILS'''  (https://tails.boum.org/). However, with this approach you need to dedicate a little more time to its configuration since you must know how to enable ports on an internet modem.
  
 
[[File:zeronet_logo.png|120px|none|alt=zeronet.io|ZeroNet]]  
 
[[File:zeronet_logo.png|120px|none|alt=zeronet.io|ZeroNet]]  
  
¿Qué necesitas?
+
What do you need?
  
Una USB con TAILS que tenga persistencia
+
One TAILS USB with persistence activated.
  
Habilitar el puerto 15441 en tu modem
+
Enable the port  15441 of your modem
  
¿Cómo se hace?
+
1) Login to a TAILS session using persistence and advanced settings to access the ''root'' terminal.
  
1) Ingresas a una sesión de TAILS usando persistencia y configuraciones avanzadas para poder acceder a la terminal ''root''.
+
2) In the terminal you need to write the following commands:
  
2) Desde una Terminal tienes que escribir:
+
- To update and install the required applications:
 
+
Para actualizar e instalar las dependencias necesarias:
+
  
 
  $ sudo apt-get update
 
  $ sudo apt-get update
Line 103: Line 101:
 
  $ sudo apt-get install python-msgpack python-gevent
 
  $ sudo apt-get install python-msgpack python-gevent
  
Para descargar zeronet:
+
For downloading zeronet:
 +
 
 
  $ wget https://github.com/HelloZeroNet/ZeroNet/archive/master.tar.gz
 
  $ wget https://github.com/HelloZeroNet/ZeroNet/archive/master.tar.gz
Para descomprimir el archivo descargado:
+
 
 +
To unzip the downloaded file:
 +
 
 
  $ tar xvpfz master.tar.gz
 
  $ tar xvpfz master.tar.gz
Para entrar en el archivo:
+
 
 +
To enter the file:
 +
 
 
  $ cd ZeroNet-master
 
  $ cd ZeroNet-master
Para iniciar
+
 
 +
For starting:
 +
 
 
  $ python zeronet.py
 
  $ python zeronet.py
  
Si esta opción no te resulta necesitas:
+
If this option does not work you need to check your IP
 
+
Consultar tu ip
+
  
 
  $ sudo ifconfig
 
  $ sudo ifconfig
  
Este comando te indicará cual es tu dirección de internet (inet addr), por ejemplo en la imagen es 10.14.241.166  
+
This command will tell you your internet address (inet addr), for example in the image it is 10.14.241.166
 
[[File:ifconfig.png|400px|center|alt=Comando ifconfig]]  
 
[[File:ifconfig.png|400px|center|alt=Comando ifconfig]]  
  
 
  $ python zeronet.py --proxy 127.0.0.1:9050 --tor disable --ui_ip inet_addr
 
  $ python zeronet.py --proxy 127.0.0.1:9050 --tor disable --ui_ip inet_addr
En el comando anterior debes sustituir inet_addr con el que le corresponde a tu computadora
+
In the previous command you must replace inet_addr with the one that corresponds to your computer
  
Con esto habrás iniciado los servicios de zeronet
+
You should now have successfully started Zeronet
  
3) En el navegador '''TOR''' tienes que abrir las opciones de '''Configuración''' y dirigirte a '''Avanzado''' ahí encontrarás la pestaña '''Redes''' (Network) y en el primer punto '''Conexión''' debes ir a '''Opciones''' y añadir en '''No proxi''' a 127.0.0.1 o a tu inet_addr
+
3) In the browser '''TOR''' you need to open the options of '''Configuration''' and select ''Advanced'' where you will find the tab  '''Network''' and in the first point you need to go to '''Connection''' '''Options''' and add in '''No proxy''' 127.0.0.1 or your own inet_addr
  
[[File:Proxy_conf_tor.png|400px|center|alt=Comando Opciones de configuración en el navegador TOR]]
+
[[File:Proxy_conf_tor.png|400px|center|alt=Configuration options for the TOR browser]]
  
 
+
After you can open/
Después ya puedes abrir:
+
 
  http://inet_addr:43110/
 
  http://inet_addr:43110/
por ejemplo
+
for instance:
 
  http://127.0.0.1:43110
 
  http://127.0.0.1:43110
  
y encontrarás la interfase gráfica
+
And you will find the graphical interface
  
 
[[File:zeronet.png|400px]]
 
[[File:zeronet.png|400px]]
  
Para crear un nuevo blog sólo debes activar  '''Zero blog''' y usar la herramienta clonar.
+
To create a new blog you only need to activate '''Zero blog''' and use the tool for cloning.
 
[[File:Zeronet2.png|400px]]  
 
[[File:Zeronet2.png|400px]]  
  
'''Nota:'''
+
'''Note:'''
 
+
Sólo podrás subir información a la red de zeronet si tienes habilitado el puerto 15441
+
Puedes ver más detalles sobre la configuración en [https://github.com/HelloZeroNet/ZeroNet ZeroNet]
+
 
+
==Referencias==
+
 
+
[[Category:How_To]]
+
 
+
 
+
 
+
 
+
 
+
 
+
 
+
 
+
 
+
  
 +
You will only be able to upload information to the network zeronet if your port 15441 is enabled
 +
You can see more information about set up and configuration here [https://github.com/HelloZeroNet/ZeroNet ZeroNet]
  
 +
==References==
  
 
  [[Category:How_To]]
 
  [[Category:How_To]]

Latest revision as of 09:17, 20 March 2017

What is the deep web??

The deep web refers to all the information that is in the world wide web but is not indexed, so not accessible using search engines such as Google [1]. Web indexing (or Internet indexing) refers to various methods for indexing the contents of a website or of the entire Internet. Individual websites or intranets may use a "back-of-the-book" index, while search engines usually use keywords and metadata to provide a more useful vocabulary for Internet or on-site searching. With the increase in the number of periodicals that have articles online, web indexing is also becoming important for periodical websites [2]

Web profunda

According to Wikipedia, "The portion of the web that is indexed by standard search engines is known as the surface web. As of 2001, the deep web was several orders of magnitude larger than the surface web. An analogy of an iceberg used by Denis Shestakov represents the division between surface web and deep web respectively: It is impossible to measure, and harsh to put estimates on, the size of the deep web because the majority of the information is hidden or locked inside databases. Early estimates suggested that the deep web is 400 to 550 times larger than the surface web. However, since more information and sites are always being added, it can be assumed that the deep web is growing exponentially at a rate that cannot be quantified.”

A common confusions is to mix up the deep web and the dark web; the latter corresponds to a part of the deep web, which besides not being indexed, has been hidden using methods such as masking the IP address and/or requiring special permissions or the use of specific software to access it [3].

But if the deep web is visible, how do we access it? Below we describe two ways you can immerse yourself in the deep web.

Navigating the deep web using Tor

TOR

One of the easiest ways is to use the hidden service protocol TOR [4] that operates within the anonymous distributed TOR network [5].

Considering the wide variety of users and type of information that you can find inside the deep web, using the TOR browser is normally the easiest and safest method Cite error: Closing </ref> missing for <ref> tag, which is a highly secure version of Linux you boot on your computer from a USB stick or other external device..

But if it the deep web is not indexed how do navigate it?

Case A

You find a website with a .onion address. “.onion" is a special-use top level domain suffix designating an anonymous hidden service only reachable via the Tor network. These addresses are not actual DNS names, and the .onion TLD is not in the Internet DNS root. But with the appropriate proxy software installed, Internet programs such as web browsers can access sites with .onion addresses by sending the request through the network of Tor servers. The purpose of using such a system is to make both the information provider and the person accessing the information more difficult to trace, whether by one another, by an intermediate network host, or by an outsider [6].

For instance, you receive:

http://duskgytldkxiuqc6.onion

you need to change the .onion for something like:

.onion.to

.onion.city

.onion.cab

or any other domain made available by the network of volunteers of TOR2web [7].

and you need to open it up in the browser TOR.

Case B

You want to look for a page.

You can begin with: http://deepweb.pw/

This page can be opened in any browser, but you can only access .onion pages using the TOR browser.

Here you can find interesting sites like the Hidden wiki.

You can also look for information here:

https://ahmia.fi/search/

Or if you are interested in websites about leaks:

https://en.wikipedia.org/wiki/GlobaLeaks#Implementations

Other possible search places:

http://3g2upl4pq6kufc4m.onion/
https://www.reddit.com//r/onions

You should consider that the availability of information in many cases is temporary, so you will need to spend some time to getting used to the deep web so you can search more fruitfully. Also, language can be a limitation; before entering forums remember that they generally use specific jargon[8].

Case C

You can also use the hidden services of Riseup

https://help.riseup.net/en/tor

Adding data to the deep web

Now you can surf the deep web, but how do you add information?

If you need to add a file you can use the Riseup Share service that lets you upload a file of up to 50mb for no more than a week. You can find the .onion.to league at: https://help.riseup.net/en/tor

 Riseup Services

The second option is using zeronet.io. This tool allows you to publish open, free and uncensored blogs and it uses Bitcoin cryptography and the BitTorrent network.

Although you can use this tool with different operating systems, the safest method and the option we recommend is zeronet + TAILS (https://tails.boum.org/). However, with this approach you need to dedicate a little more time to its configuration since you must know how to enable ports on an internet modem.

zeronet.io

What do you need?

One TAILS USB with persistence activated.

Enable the port 15441 of your modem

1) Login to a TAILS session using persistence and advanced settings to access the root terminal.

2) In the terminal you need to write the following commands:

- To update and install the required applications:

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install python-msgpack python-gevent

For downloading zeronet:

$ wget https://github.com/HelloZeroNet/ZeroNet/archive/master.tar.gz

To unzip the downloaded file:

$ tar xvpfz master.tar.gz

To enter the file:

$ cd ZeroNet-master

For starting:

$ python zeronet.py

If this option does not work you need to check your IP

$ sudo ifconfig

This command will tell you your internet address (inet addr), for example in the image it is 10.14.241.166

Comando ifconfig
$ python zeronet.py --proxy 127.0.0.1:9050 --tor disable --ui_ip inet_addr

In the previous command you must replace inet_addr with the one that corresponds to your computer

You should now have successfully started Zeronet

3) In the browser TOR you need to open the options of Configuration and select Advanced where you will find the tab Network and in the first point you need to go to Connection Options and add in No proxy 127.0.0.1 or your own inet_addr

Configuration options for the TOR browser

After you can open/

http://inet_addr:43110/

for instance:

http://127.0.0.1:43110

And you will find the graphical interface

Zeronet.png

To create a new blog you only need to activate Zero blog and use the tool for cloning. Zeronet2.png

Note:

You will only be able to upload information to the network zeronet if your port 15441 is enabled You can see more information about set up and configuration here ZeroNet

References

  1. https://en.wikipedia.org/wiki/Deep_web
  2. https://en.wikipedia.org/wiki/Web_indexing
  3. https://en.wikipedia.org/wiki/Dark_web
  4. https://www.torproject.org/docs/hidden-services.html.en
  5. https://www.torproject.org/about/overview.html.en#thesolution
  6. https://en.wikipedia.org/wiki/.onion
  7. https://tor2web.org/
  8. http://domekit.cc/survival-kit-in-the-deep-web/