Gender and Tech Resources - User contributions [en]

Kinky linux command-line

2015-12-09T14:27:41Z

Lilith2: /* Configuring network interfaces */

Graphical user interfaces (GUIs) are helpful for many tasks, but they box you in in the tasks the designer designed the GUI for. This is true to a certain extent for the command-line too, as it relies on the commands available. Still, some commands are so basic (close to the kernel), and come with many flags and options, or can be built on easily and be combined with other commands in shell scripts, that knowing the command-line and [[shell scripting]] is well worth the effort.

''When I first discovered the power to delete the file in my OpenBSD terminal that the OSX finder could not trash I felt was no longer a prisoner inside my machine, only possessing knowledge of a GUI, I was formerly stuck in a holding pattern. Using *nix you keep moving all the time, discovering always new executable codes sensitive to commands.''
''In the shell I find a marvelous mess of constellations, nebulae, interstellar gaps, awesome gullies, that provokes in me an indescribable sense of vertigo, as if I am hanging from earth upside down on the brink of infinite space, with terrestrial gravity still holding me by the heels but about to release me any moment. An example is /dev/null – a special *nix file where you pipe your unwanted data flow through this output. ''<ref>Linux for Theatre Makers: Embodiment and *nix modus operandi http://networkcultures.org/blog/2007/04/23/linux-for-theatre-makers-embodiment-and-nix-modus-operandi/</ref>

[[File:Kinkypinguins.png|400px|thumb|right|Farce of the Pinguins: A mockumentary that illuminates penguin survival and mating rituals, as well as one bird's search for love while on a 70-mile trek with his hedonistic buddies http://www.imdb.com/title/tt0488539/]]

== Command Line Culture (CLC) ==
Some people use a Command Line Interface (CLI) extensively, and like it more than a GUI. After a ten-step program, they will admit something like, "I am a command line junkie, I like it far better than pointing and clicking. I have become addicted to the bash command, and the basic linux utilities. I find myself installing the basic GNU tools on any system I use. Heck I even installed cygnus-win on my windows gaming box. Mmmm... Command completion... Tasty!"

An only somewhat more sane version of that seems to be running a GUI and a command line at the same time and switching between the two depending on what needs doing. Usually things can be done faster with the command line, but there are situations, such as doing something with multiple directories, when a GUI is more efficient.

$ cd /insanely/long/directory/path/and/you/thought/you/were/there/yet/but/no/muhhahahaaa/aaaaah

Typing that tends to waste time, even when using that yummy command completion. When doing that same thing regularly with the GUI, that may get annoying too and scripting ensues. Goodbye ten step plan. :D Well, except for <code>geany</code>. My cup of tea.

== Getting started ==

* Case sensitivity is very important and a common source of problems for people new to Linux. Other systems such as M$ windows are case insensitive when it comes to referring to files. Linux is not like this. You can have two or more files and directories with the same name but letters of different case.
* A lof of commands in linux are named as an abbreviation of a word or words describing them. This makes it easier to remember them.

=== Man ===
Bash shells come with a very useful utility called <code>man</code>, short for manual files or manual pages. It gives a standardised format for documenting the purpose and usage of most of the utilities, libraries, and system calls https://www.kernel.org/doc/man-pages/. For documentation other than man pages, see the Linux Documentation Project site http://www.tldp.org/.

The manual pages are a set of pages that explain every command available on your system including what they do, the specifics of how you run them and what command line arguments they accept. They are fairly consistent in their structure so you can easily get the hang of it. Start up a console or terminal and invoke the manual pages with the following command:

$ man [command]

For example:

$ man grep

[[File:Man-grep.png|640px|thumb|center]]

== Everything is a file ==
Everything in linux can be viewed as a file:
* regular files are documents, images, archives, recordings, directories (just a file containing names of other files) …
* (character and block) device files give you access to hardware components
* named pipes and sockets give access points for processes to communicate with each other
* (hard and soft) links make a file accessible from different locations

=== Navigation ===
With <code>pwd</code> ('''p'''resent '''w'''orking '''d'''irectory) you can see your location in the file structure.

$ pwd
/home/user
$

With <code>ls</code> ('''l'''i'''s'''t) you can see what is in a location:

$ ls [options] [location]

For example:

$ ls -l /home/user
total 20
drwxr-xr-x 2 user user 4096 Jun 17 14:39 Desktop
drwxr-xr-x 2 user user 4096 Jul 2 00:45 Documents
drwxr-xr-x 4 user user 4096 Jul 2 00:46 Pictures

The result lines explained

* The first character on a result line indicates whether it is a normal file <code>-</code> or a directory <code>d</code>. In the above example all are directories.
* The next 9 characters are permissions for the file or directory. More on that in file permissions below.
* A character representing the number of blocks.
* The field following that is the owner of the file or directory (user in this case).
* The group the file or directory belongs to (user)
* File size
* File modification time
* Name of the file or directory

For more explanation on and examples of using <code>ls</code> do:

$ man ls
When referring to either a file or directory on the command line, like with <code>/home/user</code> in the <code>ls</code> example, we are referring to a path, a description of a route to get to a particular file or directory on the system. The linux file system is a hierarchical with at the very top of the structure a directory called the '''root''' directory denoted by a single slash <code>/</code>. It has subdirectories and the subdirectories have subdirectories and so on. Files may reside in any of these directories.

Paths can be '''absolute''' or '''relative''':
* Absolute paths specify a location (file or directory) in relation to the root directory and begin with a <code>/</code>
* Relative paths specify a location (file or directory) in relation to where you currently are in the system and do not begin with a <code>/</code>
More building blocks:
* <code>~</code> (tilde), a shortcut for your home directory. For example <code>/home/user/Pictures</code> and <code>~/Pictures</code> both refer to the <code>Pictures</code> folder in the home directory of user.
* <code>.</code> (dot), a reference to your current directory. For example, <code>./Pictures</code> refers to the same directory again.
* <code>..</code> (dotdot), a reference to the parent directory. You can use this several times in a path to keep going up the hierarchy. If you are in the location the path <code>/home/user</code> refers to, you could run the command <code>ls ../../</code> and this would return a listing of the root directory.
In order to move around in the system you can use a command called <code>cd</code> ('''c'''hange '''d'''irectory):

$ cd [location]

Typing out these paths can become tedious. Not to mention my typos. Yessss. Tab Completion. When you start typing a path and hit the Tab key on your keyboard at any time you will invoke an auto complete action. If nothing happens then that means there are several possibilities. If you hit Tab again it will show you those possibilities. You can continue typing and hit Tab again and it will again try to auto complete for you.

=== Expansion ===

When a tilde is used at the beginning of a word, it expands into the name of the home directory of the named user, or if no user is named, the home directory of the current user:

$ echo ~
/home/user

If user "foo" has an account, then:

$ echo ~foo
/home/foo

The shell allows arithmetic to be performed by expansion making using the shell prompt as a calculator easy. Arithmetic expansion only supports integers (whole numbers, no decimals), but can perform quite a number of different operations::

$ echo $((2 + 2))
4

It allows for nesting of expressions (5**2 means 52):

$ echo $(($((5**2)) * 3))
75

Single parentheses may be used to group multiple subexpressions:

$ echo $(((5**2) * 3))
75

Here is an example using the division and remainder operators (integer division):

$ echo Five divided by two equals $((5/2))
Five divided by two equals 2
$ echo with $((5%2)) left over.
with 1 left over.

Perhaps the strangest expansion is called ''brace expansion''. You can create multiple text strings from a pattern containing braces:

$ echo last{mce,boot,xorg}.log
lastmce.log lastboot.log lastxorg.log

Patterns to be brace expanded may contain a leading portion called a ''preamble'' and a trailing portion called a ''postscript''. The brace expression itself may contain either a comma-separated list of strings, or a range of integers or single characters. The pattern may not contain embedded whitespace.

Using a range of integers in reverse order:

$ echo Number_{5..1}
Number_5 Number_4 Number_3 Number_2 Number_1

Brace expansions may also be nested:

$ echo a{A{1,2},B{3,4}}b
aA1b aA2b aB3b aB4b

The most common application of brace expansion is to easily make files or directories:
$ mkdir {2011..2013}-0{1..9} {2011..2013}-{10..12}
$ ls
2011-01 2011-07 2012-01 2012-07 2013-01 2013-07
2011-02 2011-08 2012-02 2012-08 2013-02 2013-08
2011-03 2011-09 2012-03 2012-09 2013-03 2013-09
2011-04 2011-10 2012-04 2012-10 2013-04 2013-10
2011-05 2011-11 2012-05 2012-11 2013-05 2013-11
2011-06 2011-12 2012-06 2012-12 2013-06 2013-12

=== File manipulation ===
From the command line, there are many ways to create, find and list different types of files.

In systems such as M$ Windows the extension is important and the system uses it to determine what type of file it is. In linux the system ignores extensions and looks inside the file to determine what type of file it is. So sometimes it can be hard to know for certain what type of file a particular file is. You can determine the type of a file with the file command:
$ file privatelyinvestigating.wordpress.2015-05-02.xml
privatelyinvestigating.wordpress.2015-05-02.xml: XML document text

With <code>cp</code> ('''c'''o'''p'''y) you can copy files and directories:
$ cp [options] [filename] [filename]

For example:
$ cp -u *.png /home/user/Pictures/

Will copy all files in the current directory with extension .png to the Pictures directory in the home directory of user.

With <code>mv</code> ('''m'''o'''v'''e) you can move or rename files and directories. To rename a file, use like this:
$ mv [filename1] [filename2]

To move a file, use like this:
$ mv [filename1] [directory]

To move files, use like this:
$ mv [filename1] [filename2] [directory]

With <code>rm</code> ('''r'''e'''m'''ove) you can remove files and directories. Linux does not have an undelete command. Once you delete something with <tt>rm</tt>, it's gone. You can inflict horrifying damage on your system with <tt>rm</tt> if you are not careful, particularly with wildcards such as *.

To remove a file:
$ rm [filename]

To remove directories:
$ rm -r [filename]

And with <code>mkdir</code> ('''m'''a'''k'''e '''dir'''ectory) you can create directories:
$ mkdir [directory]

=== File permissions ===
Unix-like operating systems differ from other computing systems in that they are not only ''multitasking'' but also ''multi-user''. The multi-user capability of Unix-like systems is a feature that is deeply ingrained into the design of the *nix operating system. In the environment in which Unix was created, this makes perfect sense, and now, with the internet, this makes perfect sense again. In the beginning, computers were large, expensive, and centralised, access was by ''terminals ''and The Computer would support many users at the same time, as does the internet. A method had to be devised to protect users from each other.

In linux, each file and directory is assigned access rights for the owner of the file, the members of a group of related users, and everybody else. Rights can be assigned to read a file, to write a file, and to execute a file (run the file as a program). There are two ways to specify the permissions.

For the first, see the permission settings for a file or directory, use the <code>ls -l</code> command (see above in [[Kinky linux command-line#Navigation|Navigation]]). Taking one line of the results:

drwxr-xr-x 4 user user 4096 Jul 2 00:46 Pictures

The <code>ls -l</code> output line starts with a <code>d</code> indicating it is a directory, and the next nine characters are for permissions. These are three groups of three characters each. The first set of three characters <code>rwx</code> is for <code>owner</code>, the owner of the file. Owner has '''r'''ead <code>r</code>, '''w'''rite <code>w</code> and e'''x'''ecute <code>x</code> permissions on that directory. The second set of characters is for <code>group</code>. Users in the group have <code>r-x</code> permissions and can only read and execute the file. <code>Other</code> (the rest of the world) have those permissions too in this case.

The conversion to the other permissions representation goes like this:

1) Convert the three sets <code>rwx r-x r-x</code> to three groups of binary code using 1's for "turned on" indicated by <code>r</code>, <code>w</code> and <code>x</code> and 0's as "turned off" indicated by a <code>-</code>, like so:

rwx = (111)2

r-x = (101)2

r-- = (100)2

--x = (001)2

The example then looks like <code>111 101 101</code>2

2) Convert binary code. If not familiar with number conversions, a decent tutorial can be found in http://www.cstutoringcenter.com/tutorials/general/convert.php

(111)2 = 20 + 21 + 22 = 1 + 2 + 2*2 = 710

(101)2 = 20 + 22 = 1 + 2*2 = 510

(100)2 = 22 = 2*2 = 410

(001)2 = 20 = 1 10

The example then looks like <code>7 5 5</code>10. So the permissions for this <code>Pictures</code> directory are 755 10 and the base, 10, is often not mentioned.

'''Handy file permissions mental shortcuts:'''
* 777 ''(rwxrwxrwx)'' No restrictions on permissions. Anybody may do anything. Generally not a desirable setting.
* 755 ''(rwxr-xr-x)'' The file's owner may read, write, and execute the file. All others may read and execute the file. This setting is common for programs that are used by all users.
* 700 ''(rwx------)'' The file's owner may read, write, and execute the file. Nobody else has any rights. This setting is useful for programs that only the owner may use and must be kept private from others.
* 666 ''(rw-rw-rw-)'' All users may read and write the file. But not execute.
* 644 ''(rw-r--r--)'' The owner may read and write a file, while all others may only read the file. A common setting for data files that everybody may read, but only the owner may change.
* 600 ''(rw-------) ''The owner may read and write a file. All others have no rights. A common setting for data files that the owner wants to keep private.
'''Handy directory permissions mental shortcuts:'''
* 777 ''(rwxrwxrwx)'' No restrictions on permissions. Anybody may list files, create new files in the directory and delete files in the directory. Generally not a good setting.
* 755 ''(rwxr-xr-x)'' The directory owner has full access. All others may list the directory, but cannot create files nor delete them. This setting is common for directories that you wish to share with other users.
* 700 ''(rwx------)'' The directory owner has full access. Nobody else has any rights. This setting is useful for directories that only the owner may use and must be kept private from others.
With '''chmod''' you can modify access rights to a file:
$ chmod [permissions] [filename]
With <code>su</code> ('''s'''uper '''u'''ser) or <code>sudo</code> ('''as s'''uper'''u'''ser '''do''') you can temporarily become the superuser. Doing <code>su</code> on debian you will be asked for your root password.
$ su

A new shell owned by root is started, indicated by a <code>#</code> instead of a <code>$</code> as prompt. You can kill that shell and return to your previous user shell with <code>exit</code>:
# exit
In mint and ubuntu default you do not have <code>su</code> but you can use <code>sudo</code>. You will be asked for your user password.
$ sudo

In order to change the owner of a file, you must be the superuser. With <code>chown</code> ('''ch'''ange '''own'''ership) you can then change file ownership:
# chown [username] [filename]
For changing the group ownership of a file you do not need superuser, but you do need to be owner of the file. With <code>chgrp</code> ('''ch'''ange '''gr'''ou'''p''') you can change a file's group ownership:
$ chgrp [groupname] [filename]

== Regular expressions ==
Regular expressions are strings that describe a collection of strings using a for that purpose created language. That probably reads like garble, but a few examples can help. Regular expressions are useful for expansion, static code source analysis, reverse engineering, malware fingerprinting, vulnerability assessment, and exploit development. Many of the tools for working with text enable you to use ''regular expressions'', sometimes referred to as ''regex'', to identify the text you are looking for based on some pattern. You can use these strings to find text within a text editor or use them with search commands to scan multiple files for the strings of text you want.

=== Matching using regex ===

{|
|-
! Expression !! Matches
|-
| a* || a, ab, abc, abs, absolutely, ...
|-
| ^a || Any "a" appearing at the beginning of a line
|-
| *a$ || Any "a" appearing at the end of a line
|-
| a.c || Three character strings that begin with a and end with c
|-
| [bcf]at || bat, cat, or fat
|-
| [a-d]at || aat, bat, cat, dat (but not Aat, Bat, Cat or Dat)
|-
| [A-D]at || Aat, Bat, Cat, Dat (but not aat, bat, cat or dat)
|-
| 1[3-5]7 || 137, 147 and 157
|-
| \tHello || A tab character preceding the word Hello
|-
| \.[tT][xX][Tt] || .txt, .TXT, .TxT, and all other case combinations
|}

Regular expressions are not completely consistent from program to program. For example, the meaning of the asterisk <code>*</code> in the shell's filename expansion is different from that used by <code>grep</code> and other programs which support regular expressions. In addition, other versions of <code>grep</code> (like <code>fgrep</code> and <code>egrep</code>) support additional features. Programming languages also have many additional extensions to regular expressions. The online <code>man</code> pages can be consulted to resolve any discrepancies.

[[File:Guest-xkcd.png|369px|thumb|right|Try this terminal: http://uni.xkcd.com/]]

=== Searching and replacing text with regular expressions in vi ===

The most common editor is still <code>vi</code> and can be found on any *nix (unless it has been removed). Knowledge about how to make minor file edits is critical for administrators. On minimalist systems or when trying to bring a foreign server back online, <code>vi</code> will almost certainly be there. Vim is an enhanced vi editor that may be there (for vim regex see http://www.vimregex.com/ and for vim macros see http://vimdoc.sourceforge.net/htmldoc/usr_10.html#10.1).

You can use regular expressions to find patterns in files from inside editors like <code>vi</code>:

{|
|-
! Expression !! Matches
|-
| . (dot) || Any single character except newline
|-
| * || Zero or more occurances of any character
|-
| [...] || Any single character specified in the set
|-
| [^...] || Any single character not specified in the set
|-
| ^ || Anchor - beginning of the line
|-
| $ || Anchor - end of line
|-
| \< || Anchor - beginning of word
|-
| \> || Anchor - end of word
|-
| $...$ || Grouping - usually used to group conditions
|-
| \n || Contents of nth grouping
|}

'''Examples of sets:'''

{|
|-
! Expression !! Matches
|-
| [A-Z] || The ''set'' from Capital A to Capital Z
|-
| [a-z] || The ''set'' from lowercase a to lowercase z
|-
| [0-9] || The ''set'' from 0 to 9 (All numerals)
|-
| [./=+] || The ''set'' containing . (dot), / (slash), =, and +
|-
| [-A-F] || The ''set'' from Capital A to Capital F and the dash (dashes must be specified first)
|-
| [0-9 A-Z] || The ''set'' containing all capital letters and digits and a space
|-
| [A-Z][a-zA-Z] || In the first position the ''set'' from capital A to Z, in the second position the ''set'' of all letters
|}

'''Examples of expressions:'''

{|
|-
! Expression !! Matches
|-
| /Hello/ || Line containing the value Hello
|-
| /^TEST$/ || Line containing TEST by itself
|-
| /^[a-zA-Z]/ || Line starts with any letter
|-
| /^[a-z].*/ || First character of the line is a-z and there is at least one more of any character following it
|-
| /2134$/ || Line ends with 2134
|-
| /[0-9]*/ || Zero or more numbers in the line
|-
| \<00* || A number with leading zeroes
|-
| /^[^#]/ || The first character is not a # in the line
|}

The search and replace function in vi is done with the <code>:%s</code> command:

:%s/pattern/string/flags

This command replaces ''pattern'' with ''string'' according to ''flags'':

* Flags can be <code>g</code> for replacing all occurences of pattern globally (in the open file) and <code>c</code> for confirming replacements.
* With <code>&</code> you can repeat the last <code>:%s</code> command.

Suppose we have a text file with this content:

<pre>- I mount my soul at /dev/null
- Those who do not understand Unix are condemned to reinvent it, poorly.
- Unix was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
- Unix is sexy: who | grep -i blonde | date; cd ~; unzip; touch; strip; finger; mount; gasp; yes; uptime; umount; sleep
- Try this terminal: http://uni.xkcd.com/</pre>

Use the ''Escape'' key to get into command mode, type <code>:%s</code> (your cursor will jump to the bottom and show you what you are typing) and enclose your regular expression in either slashes <code>/ /</code> for a forward search or backslashes <code>\ \</code> for a backward search. In the above file (with the cursor on the first line) try:

:%s/U.i/linu/g

And see what happens:

<pre>- I mount my soul at /dev/null
- Those who do not understand linux are condemned to reinvent it, poorly.
- linux was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
- linux is sexy: who | grep -i blonde | date; cd ~; unzip; touch; strip; finger; mount; gasp; yes; uptime; umount; sleep
- Try this terminal: http://uni.xkcd.com/</pre>

=== Extracting columns of text with awk ===

Awk is a programming language which allows easy manipulation of structured data and the generation of formatted reports. Awk stands for the names of its authors "'''A'''ho, '''W'''einberger, and '''K'''ernighan". Awk is used for pattern scanning and processing. It searches one or more files to see if they contain lines that matches with the specified patterns and then perform associated actions.

Key features of <code>awk</code> are:

* Awk views a text file as records and fields.
* Like most programming language, awk has variables, conditionals and loops
* Awk has arithmetic and string operators.
* Awk can generate formatted reports.
* Awk reads from a file or standard input, and outputs to standard output.
* Awk does not get along with non-text files.

$ awk '/[pattern]/ [Actions] /[pattern]/ [Actions]' [inputfile]

''Pattern'' is a regular expression and the single quotes are used to make sure the shell does not interpret any of the enclosed special characters.

$ awk '{print;}' fun.txt

will print all the lines in fun.txt.

$ awk '/null/' fun.txt
- I mount my soul at /dev/null

You can continue on a next line. When you enter a return, the <code>></code> appears as prompt. When awk reads completion the return will be taken as "go".
$ awk '/sexy/
> /null/' fun.txt
- I mount my soul at /dev/null
- linux is sexy: who | grep -i blonde | date; cd ~; unzip; touch; strip; finger; mount; gasp; yes; uptime; umount; sleep

The examples below use a pipe <code>|</code>, see [[Kinky_linux_command-line#I/O redirection|I/O redirection]] for more on using pipes and [[Kinky_linux_command-line#Process management|Process management]] for more on <code>ps</code>.

Showing you the processes of user:

$ ps auwx | awk '/user/ {print $11}'

or:

$ ps auwx | grep user | awk '{print $11}'

Both display the contents of the 11th column (command name) from currently running processes output from the ps command <code>ps auwx</code>. In the first example <code>awk</code> is used and in the second <code>grep</code> to find all processes owned by the user named user. In each case, when processes owned by user are found, column 11 (command name) is displayed for each of those processes.

By default <code>awk</code> uses spaces as delimiter between columns. You can specify a different delimiter with the <code>-F</code> option:

$ awk -F: '{print $1,$5}' /etc/passwd

or:

$ cut -d: -f1,5 /etc/passwd

In both cases the colon <code>:</code> is specified as delimiter. Changing the comma to a dash prints columns 1 through 5:

$ cut -d: -f1-5 /etc/passwd

When there a varying number of spaces such as in the output of <code>ps</code>, <code>awk</code> is recommended. When there's files delimited by commas or colons as is the case in the <code>/etc/passwd</code> file, <code>cut</code> is recommended.

=== Searching for text with grep ===
The <code>grep</code> command comes in handy when performing more advanced string searches in a file. By now it's a verb. Grep's regex flavor is limited. An enhanced version of grep is called egrep. It uses a text-directed engine. Since neither grep nor egrep support any of the special features such as lazy repetition or lookaround, and because grep and egrep only indicate whether a match was found on a particular line or not, this distinction does not matter, except that the text-directed engine is faster. On POSIX systems, egrep uses POSIX Extended Regular Expressions http://www.regular-expressions.info/posix.html#bre. For more on POSIX see wikipedia https://en.wikipedia.org/wiki/POSIX. Despite the name "extended", egrep is almost the same as grep. It just uses a slightly different regex syntax and adds support for alternation, but loses support for backreferences.

The usual suspects with some differences, for example * is not a wildcard:

{|
|-
! Expression !! Matches
|-
| . (dot) || Any character except the end of the line character.
|-
| $ || The expression at the end of a line.
|-
| * || Zero or more occurrence of the previous character.
|}

Bracket expressions:

{|
|-
! Expression !! Matches
|-
| [:alnum:] || Alphanumeric characters.
|-
| [:alpha:] || Alphabetic characters.
|-
| [:blank:] || Blank characters: space and tab.
|-
| [:lower:] || Lower-case letters: 'a b c d e f g h i j k l m n o p q r s t u v w x y z'.
|-
| [:digit:] || Digits: '0 1 2 3 4 5 6 7 8 9'.
|-
| [:space:] || Space characters: tab, newline, vertical tab, form feed, carriage return, and space.
|-
| [:upper:] || Upper-case letters: 'A B C D E F G H I J K L M N O P Q R S T U V W X Y Z'.
|}

A regular expression for <code>grep</code> may be followed by one of several repetition operators:

{|
|-
! Expression !! Matches
|-
| ? || The preceding item is optional and matched at most once.
|-
| * || The preceding item will be matched zero or more times.
|-
| + || The preceding item will be matched one or more times.
|-
| {n} || The preceding item is matched exactly n times.
|-
| {n,} || The preceding item is matched n or more times.
|-
| {,m} || The preceding item is matched at most m times.
|-
| {n,m} || The preceding item is matched at least n times, but not more than m times.
|}

For showing lines containing ''linux'' in a file:
$ grep linux fun.txt
- Those who do not understand linux are condemned to reinvent it, poorly.
- linux was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
- linux is sexy: who | grep -i blonde | date; cd ~; unzip; touch; strip; finger; mount; gasp; yes; uptime; umount; sleep
$

For showing empty lines in a file:

$ grep -c "^$" [filename]

Searching for a pattern “kernel: *” i.e kernel: and zero or more occurrence of space character:
$ grep "kernel: *." *
grep: Desktop: Is a directory
grep: eepsite: Is a directory
grep: Music: Is a directory
grep: scripts: Is a directory
grep: Templates: Is a directory

Use of bracket expressions:
$ grep '[:upper:]' filename

Wildcards, matching all 3 character word starting with "b" and ending in "t":
$ grep '\<b.t\>' filename

Print all lines with exactly two characters:
$ grep '^..$' filename

The following regex to find an IP address 192.168.1.254 will not work:
$ grep '192.168.1.254' /etc/hosts

All three dots need to be escaped:
$ grep '192\.168\.1\.254' /etc/hosts

An IP address with egrep:

<pre>$ egrep '[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}' filename</pre>

The examples below use a pipe <code>|</code>, see [[Kinky_linux_command-line#I/O redirection|I/O redirection]] for more on using pipes, [[Kinky_linux_command-line#Process management|Process management]] for more on <code>ps</code> and [[Kinky_linux_command-line#Network_exploitation_and_monitoring|Network exploitation and monitoring]] for more on <code>tcpdump</code>.

For showing ''init'' lines from ps output:
$ ps auwx | grep init
root 1 0.0 0.0 29432 5416 ? Ss 10:14 0:01 /sbin/init
user 4999 0.0 0.0 12724 2092 pts/0 S+ 18:32 0:00 grep init

Using grep to search for specific network traffic with tcpdump:
$ sudo tcpdump -n -A | grep -e 'POST'
[sudo] password for user:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
E...=.@.@......e@.H..'.P(.o%~...P.9.PN..POST /blog/wp-admin/admin-ajax.php HTTP/1.1
E...c_@.@..=...e@.H..*.PfC&lt;....wP.9.PN..POST /blog/wp-admin/admin-ajax.php HTTP/1.1
E.....@.@......e@.H...."g;.(.-,WP.9.Nj..POST /login/?login_only=1 HTTP/1.1

Sniffing passwords using egrep:
$ tcpdump port http or port ftp or port smtp or port imap or port pop3 -l -A | egrep -i 'pass=|pwd=|log=|login=|user=|username=|pw=|passw=|passwd=|password=
|pass:|user:|username:|password:|login:|pass|user' --color=auto --line-buffered -B20

== Input/Output redirection ==
I/O redirection is one of the easiest things to master. It allows for combining different utilities effectively. For example, you may want to search through the output from <code>nmap</code> or <code>tcpdump</code> or a key-logger by feeding its output to another file or program for further analysis.

=== File descriptors ===
Every running program starts with three files (data streams) already opened:

* STDIN (0) - Standard input (data fed into the program, defaults to keyboard), $0
* STDOUT (1) - Standard output (data printed by the program, defaults to terminal/console), $1
* STDERR (2) - Standard error (for error messages, also defaults to the terminal/console), $2

That "open file"? The value returned by an <code>open</code> call is called a ''file descriptor'' and it is an index into an array of open files kept by the kernel, making the file-descriptor the gateway into the kernel's abstractions of underlying hardware.

[[File:File-descriptors.png|||center|]]

For more on device drivers see Linux Device Drivers, Third Edition By Jonathan Corbet, Alessandro Rubini, Greg Kroah-Hartman http://www.oreilly.com/openbook/linuxdrive3/book/index.html It's a bit outdated (2005) but a good reference.

Piping and redirection is the means by which we may connect these streams between programs and files to direct data in interesting and useful ways.

=== Redirecting to a file ===

Take all output from standard out (stdout) and place it into filename (Using <code>>></code> will append to the file, rather than overwrite it):

$ ls > filename

You do not have to create the file named filename in the example first. The way the mechanism works, the file ''filename'' is created first (if it did not exist already) and then the program is run and output saved into the file. If we save into a file which already exists, however, then it's contents will be cleared, and then the new output saved to it.

=== Reading from a file ===

Copy all data from the file to the standard input (stdin) of the program:

$ echo < filename

Most programs allow us to input a file. So why is this redirection handy? An example of that using <code>wc</code> ('''w'''ord '''c'''ount, the <code>-l</code> is for printing newline counts):
$ wc -l fun.txt
5 fun.txt
while:
$ wc -l < fun.txt
5

When <code>wc</code> is supplied with the file to process as a command line argument, the output from the program included the name of the file that was processed. When redirecting the contents of fun.txt into wc the file name was not printed. When using redirection or piping, the data is sent anonymously. This mechanism is useful for getting ancilliary data to not be printed.

We can used the <code>sort</code> command to process the contents of fun.txt. We can combine the two forms of redirection like this:
$ sort < fun.txt > sorted_fun.txt
$ cat sorted_fun.txt
- I mount my soul at /dev/null
- linux is sexy: who | grep -i blonde | date; cd ~; unzip; touch; strip; finger; mount; gasp; yes; uptime; umount; sleep
- linux was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
- Those who do not understand linux are condemned to reinvent it, poorly.
- Try this terminal: http://uni.xkcd.com/

The three streams have numbers associated with them (<code>$0</code>, <code>$1</code> and <code>$2</code>). STDERR is stream number 2 and we can use these numbers to identify the streams. If we place a number before the <code>></code> operator then it will redirect that stream (if we don't use a number, then it defaults to stream 1).
$ [command] 2> errors.txt

We can save both normal output and error messages into a single file by redirecting the STDERR stream to the STDOUT stream and redirecting STDOUT to a file. We redirect to a file first then redirect the error stream. We identify the redirection to a stream by placing an & in front of the stream number (otherwise it would redirect to a file called 1).
$ [command] > commandoutput 2>&1
$ cat commandoutput

=== Piping ===

Take everything from standard out (stdout) of program1 and pass it to standard input (stdin) of program2:
$ ls | more

We can pipe as many programs together as we like. In the below example the output of <code>ls</code> is piped to <code>head</code> to give us the first three lines of the output of the <code>ls</code> command, and that is piped to <code>tail</code> so as to get only the third file:

$ ls
commandoutput firstfile filename foo1 fun.txt funny.png
$ ls | head -3
commandoutput
firstfile
filename
$ ls | head -3 | tail -1
filename

To make debugging of huge piped commands easier, build your pipes up incrementally. Run the first program and make sure it provides the output you were expecting. Then add the second program and check again before adding the third and so on. This can save you a lot of frustration. :D

When piping and redirecting, the actual data will always be the same, but the formatting of that data may be slightly different to what is normally printed to the screen.

== Process management (job control) ==
One of the most powerful aspects of linux is its ability not only to keep many processes in memory at once but also to switch between them fast enough to make it appear as though they were all running at the same time, called multitasking. In much of the Linux code the references are to tasks, not to processes. Because the term process seems to be more common in *nix literature and I am used to that term, I will be using ''process''.

[[File:Figure2.png|541px|thumb|right|In a general purpose multi-user computing environment such as Linux, all programs must share the finite physical memory that is available. In the absence of virtual memory, each program would have to be aware of the activities of its neighbours. ... Understanding Memory (context 64-bit GNU/Linux on AMD Opteron hardware) http://www.ualberta.ca/CNS/RESEARCH/LinuxClusters/mem.html]]

=== Process management concepts ===

* A ''process'' is a single sequence of events utilizing memory and files. A process is ''created'' by ''forking'' a copy of the process being made. The two processes are only distinguished by the parent being able to wait for the child process to finish. A process may replace itself by another program to be executed.
* Control of the multitasking is maintained in a ''preemtive'' or ''timesliced'' way. In ''timesliced'', after a certain amount of time (in ms) the operating system passes operation over from one process to the next, more deserving process. It is the scheduler which chooses which is the most appropriate process to run next and linux uses a number of scheduling strategies to ensure fairness. In general on scheduling http://www.hugovanhove.net/cursussen/OpSys/ProcessScheduling/ProcessScheduling.html. Prior to version 2.5.4, the linux kernel was non-preemptive, which means a process running in kernel mode could not be moved out of processor until it left the processor of its own accord or it waited for some input output operation to complete. Generally a process in user mode can enter into kernel mode using system calls. Previously when the kernel was non-preemptive, a lower priority process could priority invert a higher priority process by denying it access to the processor by repeatedly calling system calls and remaining in kernel mode. Even if the lower priority process' timeslice expired, it would continue running until it completed its work in the kernel or voluntarily relinquished control. If the higher priority process waiting to run is a text editor in which the user is typing or an MP3 player ready to refill its audio buffer, the result is very poor interactive performance. The kernel is now preemptive http://www.informit.com/articles/article.aspx?p=414983&seqNum=2
* An ''image'' is a computer execution environment which includes the program, associated data, status of open files (ie. file descriptor table and system file table), and the default directory. Some image attributes such as the user-id are accessible directly but other attributes such as the list of child processes can only be accessed through system calls.
* A ''process'' is the execution of an image. During execution it has four parts to its execution space: program code segment (read only and sharable), program data segment (writable, non-sharable), runtime stack segment, and system segment (system data localized to process).

[[File:LinuxFlexibleAddressSpaceLayout.png|422px|thumb|right|Anatomy of a Program in Memory http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory/]]

* A ''system call'' is a standardized access method or ''hook'' from user scripts or programs. The process management system uses four main system calls:
** fork creates two copies (parent and child) of an image.
** wait allows a parent to pause until the child process completes.
** exec allows overlaying of the calling program with a new one.
** exit is a voluntary completion of the process.
* Processes intercommunicate with each other using ''signals''.
* A ''process table'' maintains records for each process on the system. These processes are owned in a tree type structure.
* Processes are normally but not necessarily associated with a terminal device. This is done automatically on creation.
* ''Daemons'' are processes that are NOT associated with a terminal. An example is the print spooler. These are identified in the process table as ? in the tty column.
* Processes may be run in the 'background' (often by using an ampersand (&) at the end of the shell script that initiates the process). Programs running in the ''background'' do not cause the system to ''wait'' for their completion.

When an executable program starts up, it runs as a process under management of the process table. The <code>ps</code> and <code>top</code> command can be used to look at running processes; <code>nice</code> and <code>renice</code> for raising and lowering priority of a process; processes can be moved to run in the background with <code>bg</code> or to the foreground with <code>fg</code>; <code>kill</code> and <code>killall</code> can be used to send signals to a process; <code>stop</code>, <code>start</code> and <code>restart</code> to manage the running of a process; and <code>cron</code> can run commands at a scheduled time.

=== Looking at processes ===
Listing processes of current user at current shell:

$ ps
PID TTY TIME CMD
2446 pts/1 00:00:00 bash
5348 pts/1 00:00:00 ps

<pre>PID = Process ID (number)
TTY = Controlling TTY (terminal)
TIME = Total CPU time in [DD-]HH:MM:SS format
CMD = Command</pre>

Show all user' running processes (with CPU/MEM):

$ ps -u user u
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
user 2040 0.0 0.1 367732 13540 ? Ssl 09:04 0:00 x-session-manag
and a long list ...

<pre>%CPU = CPU utilisation of process's lifetime in 00.0 format
%MEM = Percentage of process's machine's physical memory use (resident test size)
VSZ = Process's cirtual memory (1024-byte units)
RSS = Non-swapped physical memeory (resident set size) in Kb
START = Start time of command started in HH:MM
STAT = Multi-character state: One character "s" state plus other state characters </pre>

"plus other state characters" ... The example shows Ssl. For a list the different values that the s, stat and state output specifiers (header "STAT" or "S") will display to describe the state of a process on your machine:

$ man ps|grep -A 20 'output specifiers'

Grep again! :D And the command <code>pgrep</code> looks through the currently running processes and lists the process IDs matching the selection criteria to stdout. All criteria have to match.

For listing all processes named ssh AND owned by root:
$ pgrep -u root ssh
Listing processes owned by root OR daemon:
$ pgrep -u root,daemon

And more fun stuff like that. Some processes start up other processes. A webserver for example, will spin off multiple httpd daemons to wait for requests to your webserver. You can view the hierarchy of processes in a tree view with <code>ps -ejH</code>, or in BSD style <code>ps axjf</code>, forest format <code>ps -ef --forest</code> or with <code>pstree</code>.

=== Changing process priority ===

The kernel schedules processes and allocates CPU time accordingly for each of them. When one of your process requires higher priority to get more CPU time, you can use the <code>nice</code> and <code>renice</code> command. The process scheduling priority range is from -20 to 19. We call this as nice value. A nice value of -20 represents highest priority, and a nice value of 19 represent least priority for a process.

Launch a test program called test.sh ([[Shell_scripting#Testing|infinite loop testing script]]):

$ ./test.sh

Check with <code>ps</code>:

$ ps -u user fl|grep './test.sh'
0 1000 3940 3708 20 0 13248 2908 - S+ pts/0 0:00 | | \_ /bin/bash ./test.sh
0 1000 4087 3956 20 0 12720 2100 - S+ pts/1 0:00 | \_ grep ./test.sh

The sixth column is NI (nice) and it is set to 0 for <code>/bin/bash ./test.sh</code>. You can check what column is which by running that <code>ps</code> command without piping it to <code>grep</code>:

F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND

Instead of launching the program with the default priority, you can use <code>nice</code> command to launch the process with a specific priority (-10 in the above command sets the priority of a process to 10. The – in nice command stands for the dash, which we use to pass options to the command. with two dashes it would be -10, a higher priority):

$ nice -10 ./test.sh

And check again:

$ ps -u user fl|grep './test.sh'
0 1000 4100 3708 30 10 13248 2980 - SN+ pts/0 0:00 | | \_ /bin/bash ./test.sh
0 1000 4104 3956 20 0 12720 2204 - S+ pts/1 0:00 | \_ grep ./test.sh

The test script is now launched with a nice value of 10, which means it runs at a lower priority when compared to other programs that are launched by default.

The process priority can be adjusted with the help of -n option. Increase:

$ nice -n -5 ./test.sh

Decrease:

$ nice -n 5 .test.sh

You can also change the priority of a running process with <code>renice</code>. For that you will need the PID (4th column in the output of the grep command above):

$ renice -n -19 -p 3708

=== Running jobs, jobs, jobs! ===

The <code>at</code> command runs a command or script at the time you set and that you enter at the <code>at></code> prompt (leave with Ctr-D):

$ at now +1 min
at> backupdb
at> <EOT>
job 1 at Sat Jul 11 14:28:00 2015

With <code>atq</code> you can look at all with <code>at</code> queued jobs.

You can also use <code>cron</code> for running commands or scripts at a given date and time. You can schedule scripts to be executed periodically. It is usually used for sysadmin jobs such as backups or cleaning /tmp/ directories and more. The cron service (daemon) runs in the background and constantly checks the <code>/etc/crontab</code> file, and <code>/etc/cron.*/</code> directories. It also checks the <code>/var/spool/cron/</code> directory.

To create a personal crontab:

$ crontab -e
no crontab for user - using an empty one
Select an editor. To change later, run 'select-editor'.
1. /bin/nano <---- easiest
2. /usr/bin/vim.tiny

Likely you'll be given a choice as to which editor you wish to use, nano or vi(m), and then a new crontab file is opened for you in the chosen editor. Under the explanation you can enter your crontab line, in the given column format:

m h dom mon dow command

Crontab fields and allowed ranges (linux crontab syntax)

<pre>Field Description Allowed Value
MIN Minute field 0 to 59
HOUR Hour field 0 to 23
DOM Day of Month 1-31
MON Month field 1-12
DOW Day Of Week 0-6
CMD Command Any command to be executed.</pre>

Scheduling a job for a specific time (july 11th 8:30 pm) every week:

30 20 11 07 * /home/user/scripts/backupdb

And using incremental backups, I can do this twice a day, evening and morning (the three <code>*</code> expand to every day, every month and every week):

30 8,20 * * * /home/user/scripts/incremental-backupdb

To view your crontab entries use <code>crontab -l</code>. To edit, use <code>crontab -e</code>

== Network connections ==
Connecting to a network from a linux box is easy, and on occasion not. If a network interface does not come up or requires manual setup, there are many commands for configuring interfaces, checking network connections and setting up special routing. Once connection is up there are more commands for getting information about networks your machine is connected to.

See [[Networking concepts]].

=== Configuring network interfaces ===
'''TCP/IP network configuration files:'''

<pre>/etc/resolve.conf List DNS servers for internet domain name resolution.
/etc/hosts Lists hosts to be resolved locally (not by DNS).
/etc/nsswitch.conf List order of host name search. Typically look at local files, then NIS server, then DNS server.</pre>

For interfaces (see <code>/etc/network/interfaces</code>):

<pre>$ cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback</pre>

<pre>lo: Loopback interface (network within your system without slowing down for the real ethernet based network)
eth0: First ethernet interface card
eth1: Second ethernet interface card
eth2: Third ethernet interface card
wlan0: First wireless network interface</pre>

Machines may be assigned a static IP address or assigned one dynamically. A server will require a static IP address so that it's resources can easily be found. A PC will likely use an automated Dynamic Host Configuration Protocol (DHCP) for IP address assignment.

'''Configuring cards:'''

For viewing the network settings of an ethernet card (NIC):

<pre>$ sudo ifconfig eth0
[sudo] password for user:
eth0 Link encap:Ethernet HWaddr 00:10:4C:25:7A:3F
inet addr:XXX.XXX.XXX.XXX Bcast:XXX.XXX.XXX.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:113857 errors:0 dropped:0 overruns:0 frame:0
TX packets:122132 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:77715800 (74.1 MiB) TX bytes:73348568 (69.9 MiB)</pre>

To display details of all interfaces (including disabled interfaces):

$ sudo ifconfig -a

Disable an interface:

$ sudo ifconfig eth0 down

Enable an interface:

$ sudo ifconfig eth0 up

Assign 192.168.2.2 as the IP address for the interface eth0:

$ sudo ifconfig eth0 192.168.10.12

Change subnet mask of the interface eth0:

$ sudo ifconfig eth0 netmask 255.255.255.0

Assigning ip-address, netmask and broadcast at the same time to interface eth0:

<pre>$ sudo /sbin/ifconfig eth0 192.168.10.12 netmask 255.255.255.0 broadcast 192.168.10.255</pre>

By default when a network card receives a packet, it checks whether the packet belongs to itself. If not, the interface card drops the packet, unless it is running in promiscuous mode. Most network monitor tools use this mode to capture packets and analyse network traffic:

$ sudo ifconfig eth0 promisc

And back to normal mode:

$ sudo ifconfig eth0 -promisc

'''Adding a card:'''

Some systems, like server systems, are equipped with two network interfaces built-in to the motherboard. On most computers additional interfaces can be installed in extra PCI expansion slots. If your system has one and you wish to put it to use:

1. try the command <code>lspci -vv</code> to see if the hardware is detected properly, and which kernel module (if any) is being assigned:

$ lspci -vv | grep Ethernet

2. Configure the card with <code>ifconfig</code> or in <code>/etc/network/interfaces</code>

3. Restart networking services. Ah yes. Depends on your distribution and what is on your system:

$ sudo /etc/init.d/networking restart

or

$ sudo systemctl restart network.service

or

$ sudo systemctl restart network

or

$ sudo systemctl restart systemd-networkd

or reboot.

'''Changing host name:'''

1. Use <code>hostname</code> to change hostname into your newly chosen [hostname]:

# hostname [hostname]

2. Change the <code>HOSTNAME</code> variable in the network configuration file <code>/etc/sysconfig/network</code>:
HOSTNAME=[hostname]

3. Restart systems which relied on the hostname (or reboot).

'''Enabling and disabling forwarding:'''

Forwarding allows the network packets on one network interface card to be forwarded to another network interface, for example from <code>eth0</code> to <code>eth1</code>. This will allow the computer to connect ("ethernet bridge") or route network traffic. The bridge configuration will merge two (or several) networks into one single network topology. IpTables firewall rules can be used to filter traffic.

A router configuration can support multicast and basic IP routing using the "route" command. IP masquerading (NAT) can be used to connect private local area networks (LAN) to the internet or to load balance servers (see below).

To turn it on, alter the <code>net.ipv4.ip_forward</code> variable in the kernel config file <code>/etc/sysctl.conf</code>:

net.ipv4.ip_forward = 1

Test (must be 1 now):

$ cat /proc/sys/net/ipv4/ip_forward

'''Routing:'''

Display your current routing table:
$ sudo route -e
or
$ sudo route -n

To route all traffic via 192.168.1.254 gateway connected via eth0 network interface:

$ sudo route add default gw 192.168.1.254 eth0

You can also add a default gateway (route) using the <code>ip</code> command:

$ sudo ip route add 192.168.1.0/24 dev eth0

To verify a new route you can use the <code>ping</code> command:

<pre>$ sudo ping [router-ip-address]
$ sudo ping [ISPs-Gateway-ip-address]
$ sudo ping 192.168.1.254</pre>

For accessing ISP network identified by the network address and netmask using network interface card eth0:
$ route add -net 10.13.21.0 netmask 255.255.255.0 gw 192.168.10.254 eth0

And for deleting it:
$ route del -net 10.13.21.0 netmask 255.255.255.0 gw 192.168.10.254 eth0

You can specify two gateways for two network destinations, one external and one internal private network:

<pre>$ sudo route add default gw 201.51.31.1 eth0
$ sudo route add -net 10.0.0.0 netmask 255.0.0.0 gw 192.168.10.254 eth0</pre>

'''Network IP aliasing:'''

For assigning more than one IP address to one ethernet card (0 and 1 are aliases in addition to the regular eth0):

<pre># ifconfig eth0 XXX.XXX.XXX.XXX netmask 255.255.255.0 broadcast XXX.XXX.XXX.255
# ifconfig eth0:0 192.168.10.12 netmask 255.255.255.0 broadcast 192.168.10.255
# ifconfig eth0:1 192.168.10.14 netmask 255.255.255.0 broadcast 192.168.10.255</pre>

and add the routes:
<pre># route add -host XXX.XXX.XXX.XXX dev eth0
# route add -host 192.168.10.12 dev eth0
# route add -host 192.168.10.14 dev eth0</pre>

The result of the <code>ifconfig</code> command:

<pre>eth0 Link encap:Ethernet HWaddr 00:10:4C:25:7A:3F
inet addr:XXX.XXX.XXX.XXX Bcast:XXX.XXX.XXX.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:14218 errors:0 dropped:0 overruns:0 frame:0
TX packets:1362 errors:0 dropped:0 overruns:0 carrier:0
collisions:1 txqueuelen:100
Interrupt:5 Base address:0xe400

eth0:0 Link encap:Ethernet HWaddr 00:10:4C:25:7A:3F
inet addr:192.168.10.12 Bcast:192.168.10.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:5 Base address:0xe400

eth0:1 Link encap:Ethernet HWaddr 00:10:4C:25:7A:3F
inet addr:192.168.10.14 Bcast:192.168.10.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:5 Base address:0xe400</pre>

Aliases can be shut down independently:

# ifdown eth0:0

'''Tunneling:'''

For more on tunneling see [[Anonymising your traffic with linux]].

'''MacChanger:'''

With <code>macchanger</code> you can change the mac address http://whatismyipaddress.com/mac-address of any Ethernet network device:

1. Turn off your network interface:

$ sudo ifconfig eth0 down

2. Use macchanger to randomly generate new MAC address and assign it to eth0 network interface:

<pre>$ sudo macchanger -r eth0
Current MAC: 00:10:4C:25:7A:3F (unknown)
Faked MAC: 32:cf:cb:6c:63:cd (unknown)</pre>

3. Enable eth0 network interface and check new MAC address:

<pre>$ sudo ifconfig eth0 up
$ sudo ifconfig eth0</pre>

If a specific MAC address is required:

$ sudo macchanger -m 00:0a:95:9d:68:16 eth0

=== Monitoring network connections ===

You can use any one of the following command-line tools that displays network connections, routing tables, and a number of network interface statistics: the <code>netstat</code> command to see network status including open ports, tcp/udp connections, etc; the <code>sockstat</code> command to show open sockets (not default installed in debian and its branches but available from repository with no dependencies); and the <code>lsof</code> command to list open files such as network sockets and files on disks. With <code>tcpdump</code> you can capture packets from the network traffic.

'''netstat:'''

List all connections:
$ netstat -a

List only TCP connections:
$ netstat -at

List only UDP connections:
$ netstat -au

Disable reverse dns lookup for faster output:

$ netstat -ant

List only listening connections:

$ netstat -tnl

Get process name/pid and user id (use sudo to get more information):
$ sudo netstat -nlpt

When using the <code>p</code> option, <code>netstat</code> must be run with root privileges, otherwise it cannot detect the pids of processes running with root privileges and most services like <code>http</code> and <code>ftp</code> often run with root privileges.

Along with process name/pid its even more useful to get the username/uid owning that particular process. Use the e option along with the p option to get the username too.

$ sudo netstat -ltpe

For printing statistics:

$ netstat -s

Display kernel routing information:

$ netstat -rn

Get netstat output continuously:

$ netstat -ct

Display multicast group information (<code>g</code> option):

$ netstat -g

'''sockstat:'''

Show listening sockets:
$ sudo sockstat -l

Use the grep command to select ports. To find out if port 22 and 80 are open or not:

<pre>$ sudo sockstat -l | grep :22
$ sudo sockstat -l | grep :80</pre>

'''lsof:'''

Show all connections:

$ sudo lsof -i

Get only IPv6 traffic:

$ sudo lsof -i 6

Get only IPv4 traffic:

$ sudo lsof -i 4

Show only TCP connections (provide the protocol right after the <code>-i</code>):

$ sudo lsof -iTCP

Show networking related to a given port (22 in this case):

$ sudo lsof -i :22

Show connections to a specific host:

$ sudo lsof -i@XXX.XXX.XXX.XXX

Show connections based on host and port (22 in this case):

$ lsof -i@XXX.XXX.XXX.XXX:22

'''tcpdump'''

Some reuseful different combination of options to tcpdump:

* Basic communication: <code>-nS</code>
* Verbose communication: <code>-nnvvS</code> (verbosity and no name help)
* Deeper look: <code>-nnvvXS</code> (<code>-X</code> for payload but doesn’t grab any more of the packet)
* Rock bottom: <code>-nnvvXSs</code> (<code>-s</code> increases the snaplength, grabbing the whole packet)

Expressions allow for trimming out various types of traffic and finding exactly what you’re looking for. Mastering the expressions and learning to combine them creatively is what makes one truly powerful with tcpdump. There are three main types of expression: <code>type</code>, <code>dir</code>, and <code>proto</code>.

Type options are <code>host</code>, <code>net</code>, and <code>port</code>. Direction is indicated by <code>dir</code>, and there you can have <code>src</code>, <code>dst</code>, <code>src or dst</code>, and <code>src and dst</code>.

Look for traffic based on IP address (also works with hostname if you’re not using -n):

# tcpdump host XXX.XXX.XXX.XXX

Find traffic from only source or destination (eliminate one side of a host conversation)

# tcpdump src XXX.XXX.XXX.XXX

or
# tcpdump dst XXX.XXX.XXX.XXX

Capture an entire network using CIDR notation

# tcpdump net XXX.XXX.XXX.XXX/24

<code>proto</code> works for <code>tcp</code>, <code>udp</code>, and <code>icmp</code>:

# tcpdump [proto]

For example:

# tcpdump icmp

See traffic to or from a port:

# tcpdump port [portnumber]

Filter based on the source or destination port:

# tcpdump src port [portnumber]

or

# tcpdump dst port [portnumber]

Combine in something like:

# tcpdump src port 1025 and tcp

or

# tcpdump udp and src port 53

You can also filter by a range of ports:

# tcpdump portrange 21-23

And ask to be only shown packets that are above or below a certain size (in bytes):

# tcpdump less 32

or

# tcpdump greater 128

And instead of using words, you can use the symbols for less than, greater than, and less than or equal/greater than or equal signs:

# tcpdump <= 32
or
# tcpdump > 128

=== Serial line IP ===

* Point-to-Point Protocol (PPP) is the most common form of IP over serial line and is the most common technique used by telephone dial-up ISPs.
* Serial Line IP (SLIP) is older than PPP and less capable.
* Parallel Line IP (PLIP) is a computer networking protocol for direct computer-to-computer communications using the parallel port normally used for connections to a printer. It can be used for installing debian on machines without ethernet card https://www.debian.org/releases/squeeze/i386/apds04.html.en.

To display detected serial support run <code>dmesg</code>:

<pre>$ dmesg | grep tty
[ 0.000000] console [tty0] enabled</pre>

For configuring the I/O address, IRQ, and name of serial devices: http://rampex.ihep.su/Linux/linux_howto/html/tutorials/Serial-HOWTO-6.html

== Kernel and hardware ==

=== Kernel ===

To find out the name of the kernel release running on your box:

$ sudo uname -r

For displaying all kernel info:

$ sudo uname -a

When the kernel starts, messages about what occurs are placed in the kernel ring buffer. You can see it with <code>dmesg</code> (get out with <code>q</code>:

$ sudo dmesg | less

If that buffer fills up it may not contain the info you seek anymore, and you can use:

$ sudo less /var/log/dmesg

The <code>lsmod</code> command lets you view the names of the loaded modules:

$ sudo lsmod

If you need to add or remove a loadable module, for example to get some hardware item to work, you can use <code>modprobe</code>.

For listing all modules:

$ sudo modprobe -l

Loading a module:
$ sudo modprobe [modulename]
Removing a module:

$ sudo modprobe -r [modulename]

You can control kernel parameters (with the system running) with <code>sysctl</code>.
List all kernel parameters:

$ sudo sysctl -a | less

Also see [[Linux_security#Configuring_sysctl_.28.2A.29|Linux security: configuring sysctl]].

=== Hardware ===

List all pci hardware:

$ lspci

With more detail:

$ lspci -v

With even more detail:

$ lspci -vv

Display information about hardware components:

$ sudo dmidecode | less

View and change information related to your harddisk:

$ sudo hdparm /dev/sda

Detailed information:

$ sudo hdparm -I /dev/sda

== Resources ==

* Commandlinefu http://www.commandlinefu.com/commands/browse is a place to record those command-line gems that you return to again and again. Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too.
* Network Calculators http://www.subnetmask.info/
* IP subnet calculator http://www.subnet-calculator.com/

== Related ==
* [[Installing linux]]
* [[Linux applications]]
* [[Linux security]]
* [[Anonymising your traffic with linux]]
* [[Clean up all the things]]
* [[Shell scripting]]
* [[Encrypting everything]]

== References ==

[[Category:How_To]]

Surveillance

2015-11-25T13:56:52Z

Lilith2:

A citizenry that’s constantly on guard for secret, unaccountable surveillance is one that’s constantly being remade along the lines the state would prefer. Foucault illustrated this point by reference to a hypothetical prison called the Panopticon. Designed by utilitarian philosopher Jeremy Bentham, the Panopticon is a prison where all cells can be seen from a central tower shielded such that the guards can see out but the prisoners can’t see in.

The prisoners in the Panopticon could thus never know whether they were being surveilled, meaning that they have to, if they want to avoid running the risk of severe punishment, assume that they were being watched at all times. Thus, the Panopticon functioned as an effective tool of social control even when it wasn’t being staffed by a single guard.

This page is an ongoing effort and forever under construction.

[[File:Spy_vs_spy_by_xxweareanonymousxx-d6b9mkt.jpg|484px|thumb|right|http://xxweareanonymousxx.deviantart.com/art/Spy-Vs-Spy-381721997]]

== Technocracy ==
The logic of this system is, in a limited sense, highly logical: past elites were unable to maintain control due to human mistakes and unpredictabilities. The thinking behind a technocracy is that machines make no mistakes (this is a weakness of theirs). The large tech companies have all discussed this openly at large conferences. Money is a symbol of energy, and the erection of the mass surveillance grid is not for the purpose of security or preventing 'terrorism'. <ref>NSA Surveillance is about Control & Leverage, not Security http://www.juancole.com/2014/01/surveillance-leverage-security.html</ref>
== Surveillance ==
=== Dragnet surveillance ===
Every citizen on this planet is subject to mass surveillance. This includes data that does not, by itself, identify individuals, but sits in various databases until analysts do a search for a particular name. Then network analysts connect within two degrees and surveil others.

There are important details about these programs that need to be filled in. A large amount of information is supposedly already in the public domain but consists mainly of presentations with a kazillion names in 'em (that can easily be altered). This page attempts to make sense of the spaghetti.

=== Targeted surveillance ===
Journalists and observers working on reporting on projects like national security, leaks, environmental issues, activist coverage, demonstration coverage, waste and abuse in governments, human rights, crime, may be targeted by state and private actors. Scenarios where people are being followed require a bit more thinking than installing and using tools on our machines.

== Legalities ==

The Snowden leaks (see [[timeline masters of the internet]]) revealed a massive surveillance program including interception of email and other internet communications and phone call tapping. Some of it appears illegal, while other documents show the US spying on friendly nations during various international summits, and on its citizens. The programs are enabled by two US laws, the Patriot Act<ref>ACLU: Surveillance under Patriot Act https://www.aclu.org/infographic/surveillance-under-patriot-act</ref> and the FISA Amendments Act (FAA)<ref>FISA Amendments Act (FAA) https://www.aclu.org/faa-foia-documents</ref>, and a side dish called Executive Order 12.333<ref>ACLU: Executive order 12.333 https://www.aclu.org/cases/executive-order-12333-foia-lawsuit</ref>. Although the Foreign Intelligence Surveillance Court oversees such surveillance activities, it operates in secrecy through one-sided procedures that favour the US government.

==== Executive Order 12333 (EO12333) ====
Executive Order 12333 - 1.13 allows the FBI to provide the NSA with "technical assistance" in the United States, i.e. to collect metadata about American citizens by collaborating with "foreign intelligence" and law enforcement services. <ref>Meet the Spies Doing the NSA’s Dirty Work http://foreignpolicy.com/2013/11/21/meet-the-spies-doing-the-nsas-dirty-work/
</ref> By using telecom companies the US government can circumvent controls and accountability that don't apply to companies. All major telecom companies have worked closely and secretly with government spying since their inceptions.

==== Plausible Deniability ====

Plausible deniability began under Central Intelligence Agency (CIA) director Allen Dulles <ref>Spywriter: Plausible deniability http://spywriter.com/robots/plaus_denial.html</ref>.'' The CIA black ops division undertakes dangerous and usually what would be considered illegal missions that are not officially sanctioned by the US administration so that the administration, which usually benefits from such missions, can safely disavow any knowledge of them in the event of their publically uncovered success or failure. The administration is in the position of plausible deniability towards the CIA's actions'' <ref>Urban dictionary: http://www.urbandictionary.com/define.php?term=plausible%20deniability</ref>.

== Intelligence ==

There are six overarching categories that describe the various ways that intelligence is collected: SIGINT, HUMINT, MASINT, GEOINT, OSINT and IMINT. Of these, SIGINT and HUMINT are considered instrumental in the "war on terror".

SIGINT and security procedures are closely coordinated under what is called the UKUSA Community, or "five eyes": Australia, Canada, New Zealand, the United Kingdom, and the United States.

=== Signals Intelligence (SIGINT) ===
Signals intelligence, or SIGINT, is the process of intercepting signals between people (COMINT) or from electronic signals not directly used in communication (ELINT), or communications transmitted electronically such as through radars, radios, and weapon systems. One of the main tools that SIGINT employs is the use of cryptanalysis, which is the process of breaking codes in order to read encrypted information.

The National Security Agency (NSA) is the organisation that is responsible for SIGINT. The NSA SIGINT Strategy 2012-2016 covers all the comsec and crypto initiatives to covertly exploit people, cryptographers, anonymisers, informants, planted spies, security firms, networks, governments, nations <ref>NSA Global SIGINT Power to Generate Profits and Pay http://cryptome.org/2013/11/nsa-power-profit-pay.htm</ref>.

==== Communications Intelligence (COMINT) ====
Communications intelligence deals with voice communications commonly over radios. This is used to gather information such as location and identity. It covers all electronic communications in the US and the world "to ensure national security". The NSA at Ft Meade, Maryland has had the most advanced computers in the world since the 60s. Technology is developed and implemented in secret.

==== Electronic Intelligence (ELINT) ====
ELINT is divided into Technical ELINT (TechELINT) and Operational ELINT (OpELINT). Technical ELINT obtains signal structure, emission characteristics, modes of operation, emitter functions, and weapons systems associations of such emitters as radars, beacons, jammers, and navigational signals for determining its capabilities and role in a larger system as part of electronic warfare; Operational ELINT focuses on locating specific ELINT targets and determining operational patterns of the systems named Electronic Order of Battle (EOB) for supporting military operational planners and tactical military commanders on the battlefield. And ofcourse there are "counter" programs, existing and under development such as Electronic counter-countermeasures (ECCM) in the US and electronic protective measures (EPM) in Europe, Wild Weasel (alias Iron Hand or SEAD), Smart Dust and Wolfpack.

=== Human Intelligence (HUMINT) ===
This is the gathering of information from human sources. Interrogation and elicitation would be classified under this heading.

The Central Intelligence Agency (CIA) is responsible for overseeing the majority of HUMINT operations, although the military (DoD) is often involved in HUMINT as well. Both parties make use of two main tactics, gathering intelligence through interrogations and through conversations with key persons who have access to valuable information. HUMINT sources of information include diplomats, military attaches, prisoners of war, and espionage.

=== Foreign Instrumentation Signals Intelligence (FISINT) ===
TELINT (Telemetry Intelligence), later named FISINT (Foreign Instrumentation Signals Intelligence) used for intercepting, processing, and analysis of foreign telemetry is closely related to TechELINT and conducted by DoD.

== Operations ==
=== NSA ===
The Special Sources Operation (SSO) gathers information from internet communications through arrangements with internet companies under the program codenamed PRISM, and by Upstream collection (cable tapping).

==== PRISM ====
PRISM is part of the overall NSA surveillance effort - a program authorised in the united states under the FISA Amendments Act (FAA), now located in Section 702 of the FISA, that allows the NSA to collect communications of specifically identified foreign targets.

When people in the rest of the world use the net, they are effectively using US-based services, making them a legal target for US intelligence. Supposedly the NSA has no "direct access" to the servers of companies through PRISM - other agencies, like the FBI, collect the data. Emails, chat--video, voice, photos, stored data, VoIP, file transfers, video conferencing, notifications of target activity--logins, etc, online social networking details, and special requests are collected from nine major Silicon Valley technology companies: Facebook, Google, Yahoo, Microsoft, PalTalk, Skype, YouTube, Apple, and AOL. <ref>
What is known about NSA's PRISM program http://electrospaces.blogspot.nl/2014/04/what-is-known-about-nsas-prism-program.html
</ref>

==== Upstream collection ====
FAIRVIEW seems to be the main umbrella program, incorporating many different operations, such as BLARNEY, STORMBREW, and OAKSTAR (yet another umbrella for eight different programs used for collection outside the US), that intercept massive amounts of international Internet traffic by various means, and involves the collection of communications — both their metadata and their content — as they pass through undersea fiber-optic cables in:
* Cooperation with telecommunication companies
* Cooperation with foreign intelligence agencies
* Unilateral cable tapping operations
The five biggest collection programs of the Special Source Operations division are DANCINGOASIS, SPINNERET, MOONLIGHTPATH, INCENSER and AZUREPHOENIX. The largest cable tapping program appears to be DANCINGOASIS (DGO) <ref>NSA's largest cable tapping program: DANCINGOASIS http://electrospaces.blogspot.nl/2014/05/nsas-largest-cable-tapping-program.html
</ref>. It could be that DANCINGOASIS is another umbrella program encompassing various smaller sub-programs.

Upstream collection also includes data from MUSCULAR in a joint operation with the GCHQ.

Internet data collected by Upstream programs can be processed and searched though the XKEYSCORE indexing and analysing system. BOUNDLESS INFORMANT tracks the international intelligence gathering techniques of the NSA.

==== XKEYSCORE ====

XKEYSCORE is reported to receive unfiltered internet communication sessions ('full-take') from WEALTHYCLUSTER2. The session data is stored for a short period of time: content is buffered for 3 to 5 days (sometimes shorter or sometimes longer, depending on the amount of traffic), and metadata for up to 30 days. In other words, XKEYSCORE creates a rolling buffer that is continually being rewritten. This buffering enables analysts to perform federated queries using so-called "soft selectors", like keywords, against the body texts of e-mail and chat messages, digital documents, and spreadsheets in English, Arabic and Chinese. XKEYSCORE also allows analysts to look for the usage of encryption, the use of a VPN or the TOR network, and a number of other things that could lead to a target.

Analysts can create so-called 'fingerprints', rules that contain search terms (especially all the correlated identities of a certain target <ref>Fingerprints and the Phone Dragnet’s Secret “Correlations” Order https://www.emptywheel.net/2014/04/09/fingerprints-and-the-phone-dragnets-secret-correlations-order/
</ref>) that are automatically executed by the system. Some examples of XKeyscore fingerprints were disclosed by German regional television on July 3, 2014 <ref>Example of XKEYSCORE fingerprinting rules http://daserste.ndr.de/panorama/xkeyscorerules100.txt</ref>, who presented them as excerpts of source code.

==== Tailored Access Operations (TAO) ====
The epochal batch of leaks on TAO's hacking unit and ANT division catalog used is typical COMINT. <ref>Leaksource: NSA’s ANT Division Catalog of Exploits for Nearly Every Major Software/Hardware/Firmware http://leaksource.info/2013/12/30/nsas-ant-division-catalog-of-exploits-for-nearly-every-major-software-hardware-firmware/</ref>. See [[Digital threats, detection, protection and (counter) moves#Targeted_surveillance|targeted surveillance threats]] for some examples.

=== DEA ===

==== Hemisphere ====

The Hemisphere Project is coordinated from the Los Angeles Clearinghouse and is funded by the US Office of National Drug Control Policy (ONDCP) and DEA. Hemisphere provides electronic call detail records (CDRs) in response to federal, state, and local administrative/grand jury subpoenas. In effect, it is mass surveillance conducted by US telephone company AT&T in collaboration with the DEA (AT&T supplying DEA officials with metadata from a database of telephone calls dating back to 1987).

* AT&T & Verizon eager to get on their knees for the NSA https://www.propublica.org/article/nsa-spying-relies-on-atts-extreme-willingness-to-help
* How they connected the dots, supporting documents https://www.propublica.org/article/a-trail-of-evidence-leading-to-atts-partnership-with-the-nsa

=== CIA ===

The National Clandestine Service (NCS) is the branch of the CIA responsible for the collection of HUMINT.

=== FBI ===

==== PRISM ====
The FBI has its own capacity to monitor phone, email, chats, etc., in addition to whatever NSA shares with them and seems to be doing quite well obtaining what it needs by collecting all the data everywhere via subpoenas, though there are abundant reasons to worry about control functions in FBI’s bulky databases. PRISM, an NSA program, is apparently coordinated through the FBI. <ref>New insights into the PRISM program http://electrospaces.blogspot.nl/2013/07/new-insights-into-prism-program.html</ref>

==== Data Intercept Technology Unit (DITU) ====
The Data Intercept Technology Unit (DITU) has been described as NSA's Alter Ego in the FBI. It carries out its own signals intelligence operations and is trying to collect huge amounts of email and internet data from U.S. companies — an operation that the NSA once conducted, was reprimanded for, and says it abandoned.
==== Digital Collection System Network (DCSNet) ====
The Digital Collection System Network (DCSNet) is a suite of software that collects, sifts and stores phone numbers, phone calls and text messages. The system directly connects FBI wiretapping outposts around the US to a far-reaching private communications network. DCSNet includes at least three collection components, each running on Windows-based computers:
* DCS-3000 (alias Red Hook) handling pen-registers and trap-and-traces, a type of surveillance that collects signaling information -- primarily the numbers dialed from a telephone -- but no communications content. (Pen registers record outgoing calls; trap-and-traces record incoming calls.)
* DCS-6000 (alias Digital Storm) captures and collects the content of phone calls and text messages for full wiretap orders.
* DCS-5000 is used for targeted surveillance wiretaps (targeting spies or terrorists).

=== DoD ===
==== Defense Information Systems Agency (DISA) ====
The Defense Information Systems Agency (DISA), is a United States Department of Defense (DoD) combat support agency composed of military, federal civilians, and contractors. DISA provides information technology (IT) and communications support to the president, vice president, secretary of defense, the military services, the combatant commands, and any individual or system contributing to the defense of the United States.

=== GCHQ ===

During the 1950’s, the uk government became concerned that emanations could be captured and then reconstructed. Obviously, the emanations from a blender aren’t important (at least not then, now they may become important in the "internet of things"), but emanations from an electric encryption device would be. If the emanations were recorded, interpreted, and then played back on a similar device, it would be extremely easy to reveal the content of an encrypted message. Research showed it was possible to capture emanations from a distance, and as a response, the TEMPEST program was started. <ref>AllYouEverWantedToKnowAboutTempest http://auriea.org/index.pl/AllYouEverWantedToKnowAboutTempest
</ref>

==== MUSCULAR ====
Appears to be a back-door collection from Google and Yahoo private clouds.

== Sharing of data ==

=== PROTON/CRISSCROSS/ICREACH ===
PROTON is a storage and analysis system of telecommunications selectors at the TS/SI/FISA/ORCON/NOFORN level of classification and handling. PROTON is the program name as well as the name of the technology. It has been described as "SAP-like", and is suspected to have started life as a DEA special program.

CRISSCROSS/PROTON is a data sharing system launched in 1990 by the CIA and the DEA. By 1999, the NSA, the Defense Intelligence Agency, and the FBI had gained access to CRISSCROSS and were contributing information to it. As CRISSCROSS continued to expand, it was supplemented with a system called PROTON that enabled analysts to store and examine additional types of data. <ref>PROTON, CLEARWATER and Lexis-Nexis http://cryptome.org/2013/08/proton-clearwater-lexis-nexis.htm</ref>

After 9/11, the NSA came to view CRISSCROSS/PROTON as insufficient, and believed it was time to build a new and more advanced system to radically increase metadata sharing for HUMINT: ICREACH <ref>ICREACH: NSA’s Surveillance Search Engine http://leaksource.info/2014/08/25/icreach-nsa-surveillance-search-engine/
</ref>, now in use by CIA, DEA, DIA, FBI, and NSA. There seems to be no oversight.

=== MUSCULAR ===
MUSCULAR is one of at least four other similar programs that rely on a trusted 2nd party, programs which together are known as WINDSTOP. MUSCULAR is jointly run by the NSA and GCHQ. It operates via an access point outside the United States, and relies on an unnamed telecommunications operator to provide secret access for the NSA and the GCHQ.

== (Counter) moves ==

[[File:Operationtrollthensa.png|480px|thumb|right|http://trollthensa.com/]]

=== Communications privacy folly ===
In reply to a request for comments on Mexico law revisions – Warrantless Real-time Cell phone Geolocation Data Surveillance, Cryptome comments - This is the short version of why there can be no solution to preventing communication systems abuse <ref>Communications Privacy Folly http://cryptome.org/2012/06/comms-folly.htm</ref>.

Knowing that, what if we make spying on us as expensive as possible using '''lawful''' techniques and tools?

=== Understanding and minimising our digital shadows ===
The internet is a great space to explore, learn, speak up, listen and communicate with people across the world. Unfortunately, the internet has also become a space where people who challenge the dominant discourse often find themselves under attack. These attacks can be very personal - enabled by the fact that there is often a lot of personal information about us on the internet.

To strengthen our defences against these kinds of attacks, it’s a good idea to start by our assessing our digital shadows. These shadows - can tell a story about us: who we are, where we live and hang out, what we are interested in, and who our friends and colleagues are. For more on that see [[Step 1#Understanding_and_minimising_our_digital_shadows|Understanding and minimising our digital shadows]].
=== Controlled folly ===
And continued from there, what if we make spying on us as expensive as possible using lawful techniques and tools? And if need be we invent new (GNU and creative commons licensed) techniques and tools? <ref>Portal - 'Still Alive' https://www.youtube.com/watch?v=Y6ljFaKRTrI</ref>

As Cory Doctorow writes: ''There is no legal or technical mechanism by which code that is designed to be modified by its users can co-exist with a rule that says that code must treat its users as adversaries and seek to prevent them from running prohibited code. ''<ref>What David Cameron just proposed would endanger every Briton and destroy the IT industry http://boingboing.net/2015/01/13/what-david-cameron-just-propos.html
</ref>

See [[Scenario planning]] to prepare for some [[roleplay]], for adding a "red copper armour" for online roleplay see [[Digital threats, detection, protection and (counter) moves]], [[Installing linux]] and [[Linux security|securing]] it as best as you can and fits your context and purpose, and see [[Confusing surveillance systems|confusing of surveillance systems]] for teh lulz.

=== Political ecology ===

In my opinion, the predicament we’re in has been woven of threads that include our evolved neuropsychology, our innate cleverness, our general lack of wisdom, and the technological and cultural structures that we have created around us. We may be able to work our way free of this monkey trap, but not unless we gain insight into our own nature, lovingly nurture the wisdom we need to recognize what we’re doing, and the courage to say “No” to some of it. If we can do that, the problems of nuclear power and all the rest will become much less threatening. Can we accomplish such a radical transformation of human consciousness? We are getting a wake-up call. it’s time for us to heed it, and to do a bit of growing up. ~ Slaying The Hydra <ref>Slaying the hydra http://paulchefurka.ca/Hydra.html</ref>

Brian Holmes writes, ''I don’t think any alternative will be possible until certain realities are faced, not just by fringe figures like ourselves but by much broader swathes of society, and that seems unlikely'' <ref>The Californian Reality http://nettime.org/Lists-Archives/nettime-l-1401/msg00064.html</ref>: ''Sadly, I must report that up to now, almost no one has been interested. Left-leaning intellectuals are still preoccupied by individual liberation, minority and sectoral rights claims, the ghosts of working-class struggles, and anarchist longings for direct democracy. All of those have been very important, but none of the current oppositional discourses can marshall the sophistication, depth, durability and power to confront the transnational capitalist class. An alternative is not something that one fabricates on the fly, in a study or an artwork or or a hacklab or an affinity group or a church or a social center, even if all of those can be part of it. To make it real would require a large-scale articulation of theory and practice, extending into mainstream institutions even while outstripping and transforming them. Obviously it's easier said than done, but without saying it you can't get anywhere. The silence of the intellectuals is the new treason of the clercs. ''

''I think the keyword of systemic change already exists: political ecology. There are many people working in that direction. But the universities, cultural systems, professional association (including unions) and press/media apparatuses are still massively captured by the dream of belonging to the transnational capitalist class, or mired in some vague nostalgia for the klarion calls of yesterday's struggles. Meanwhile our old nettime nemesis, the Californian Ideology, has made tremendous forward strides.''

''Just how far will we let it go?'' It's that infamous gut/head conversation and it seems the gut is winning, isn't it?

=== Question the surveillance system ===
Examples that flew by:
* Why do we upgrade firmware? Many HW installations never add functionality.
* Why doesn't Brazil yank the cable splitter on their sovereign ground at Fortaleza and tell users they did?
* Why doesn't Germany seize the Einstein/Castanet RF illuminators on the roofs of the US, Canadian and British embassies?
* Why doesn't Orange file their threatened lawsuit over the SEA-ME-WE4 data theft?
* Why doesn't Google give us an explanation of where and how the Level Three data theft occurred?
* Why doesn't Apple tell us all the ways TAO has been hacking the iOS and if they're still ongoing?
* Why do only 8 of 24,000 attendees to RSA security conference cancel their attendance?

== Resources ==

=== News & watchdogs ===
* Bugged Planet http://buggedplanet.info/
* CanaryWatch https://www.canarywatch.org/
* Cryptome http://cryptome.org/
* AllYouEverWantedToKnowAboutTempest http://auriea.org/index.pl/AllYouEverWantedToKnowAboutTempest
* Lux ex Umbra - monitoring Canadian signals intelligence (SIGINT) activities past and present http://luxexumbra.blogspot.ca/
* NSA observer https://www.nsa-observer.net/
* Top Level Communications http://electrospaces.blogspot.fr/
* Watching Them, Watching Us http://www.spy.org.uk/
* WikiLeaks: The Spy Files https://wikileaks.org/the-spyfiles.html

=== Maps ===
* Mindmeister NSA map https://www.mindmeister.com/fr/308518551/the-national-security-agency-operates-more-than-500-separate-signals-intelligence-platforms-employs-

=== Books ===
* The Shadow Factory, 2008 http://www.bookarmor.com/_files/TSF.pdf

=== Documentaries ===
* All Watched Over by Machines of Loving Grace http://topdocumentaryfilms.com/all-watched-over-by-machines-of-loving-grace/

=== Presentations and articles ===

* Dear NSA, let me take care of your slides http://www.slideshare.net/EmilandDC/dear-nsa-let-me-take-care-ou
* Mind-blowing secrets of NSA's security exploit stockpile revealed at last http://www.theregister.co.uk/2015/09/04/nsa_explains_handling_zerodays/

=== Targeted surveillance ===

* Hacking Team: a zero-day market case study https://tsyrklevich.net/2015/07/22/hacking-team-0day-market/
* NSA QUANTUM Spies with Radio Signals https://cryptome.org/2014/01/nsa-quantum-radio.htm

=== Hardware implants ===
* Getting Terminal Access to a Cisco Linksys E-1000 https://www.trustwave.com/Resources/SpiderLabs-Blog/Getting-Terminal-Access-to-a-Cisco-Linksys-E-1000/
* Oops, I pwned your router https://www.trustwave.com/Resources/SpiderLabs-Blog/Oops,-I-pwned-your-router--Part-One/
* Hackers create spy plug inspired by the NSA's surveillance kit http://www.dailymail.co.uk/sciencetech/article-2920419/When-USBs-attack-Hackers-create-covert-spy-plug-inspired-NSA-s-Cottonmouth-surveillance-kit.html
* Infosec institute: NSA Backdoor Part 2, BULLDOZER: And, Learn How to DIY a NSA Hardware Implant http://resources.infosecinstitute.com/nsa-bios-backdoor-aka-god-mode-malware-part-2-bulldozer/

=== Radio frequency exfiltration ===
* Wireless Data Exfiltration - building and using low cost signal intelligence Tools http://www.slideshare.net/warezjoe/ccdc-2012-wireless-data-exfiltration-building-and-using-low-cost-signal-intelligence-devices-v2
* Getting Started with HackRF and GNU Radio https://github.com/mossmann/hackrf/wiki/Getting-Started-with-HackRF-and-GNU-Radio

=== Infected firmware ===
* Implementing and Detecting a PCI Rootkit http://www.blackhat.com/presentations/bh-dc-07/Heasman/Paper/bh-dc-07-Heasman-WP.pdf
* To Protect And Infect, Part 2 [30c3] https://www.youtube.com/watch?v=vILAlhwUgIU

=== Authorship analysis ===
* An Authorship Analysis: Francis Bacon as Shake-speare: http://fly.hiwaay.net/~paul/outline.html
* The jargon file: http://www.comedia.com/hot/jargon-4.4.6/html/

=== Wiretapping ===
* University of Pennsylvania Researcher Reports JitterBugs Could Turn Your Keyboard Against You, Steal Data http://www.upenn.edu/pennnews/news/university-pennsylvania-researcher-reports-jitterbugs-could-turn-your-keyboard-against-you-stea (2006)

== Related ==

* [[Digital threats, detection, protection and (counter) moves]]
* [[Timeline masters of the internet]]
* [[Covert operations]]
* [[Psychological warfare]]
* [[Game of drones]]
* [[Biometrics]]

== References ==

Linux security

2015-11-17T07:08:35Z

Lilith2: /* Using a firewall */

This page contains some basic security measures specifically for people getting started with linux. Knowing your way around a bit on the [[kinky linux command-line]] helps. If you run into trouble, find us on IRC.

[[File:Sparta.jpg|400px|thumb|right|Internet Security is based on a slew of old myths, which reach their most ‘refined’ form in RandomSource, which in turn of course derived from the original Rule of Least Power episode "Planet of Wobbly Rocks where the Security Guard Got Shot".]]

== Setting up root mail ==
* Exim uses the Sendmail design model where a single binary controls all the facilities of the MTA. This monolithic design is inherently less secure due to the lack of binary separation between the individual components of the system. Exim separates processes and has well-defined stages where it gains or loses privileges.
* Postfix has a modular design to improve security over Qmail. A master daemon (a background process) launches other smaller processes with limited privileges that do specific tasks related to the various stages of mail delivery. The modular approach limits the effects of attacks.
* Sendmail is the default MTA shipped with many Linux distribution sets and is the most well-known. It is easy to configure but had the most security loopholes, partly because it was designed long before hackers started attacking email systems. Developers fix most security issues quickly, but because it has the most number of users, it is still the biggest target for hackers.
=== Postfix ===
Debian with MATE comes with the <code>exim</code> mailserver installed. If you don't like <code>exim</code>, install <code>postfix</code> (if you install the one, the other is uninstalled).

==== Installing postfix ====
To install postfix:

$ sudo apt-get install postfix

=== Setting up local email account ===
Next is setting up an account in a mail client.

==== icedove ====
The wizard of icedove (thunderbird) does not allow directly for setting up local mail adresses. Choose "I think I'll configure my account later.":

[[File:Localmail1.png|640px|thumb|center]]

Make the menu appear, and choose the big Movemail link (all the way to the right of the blue email):

[[File:Localmail2.png|640px|thumb|center]]

Fill in "Identity" data like so:

[[File:Localmail3.png|640px|thumb|center]]

Outgoing Server Information like so (will be revisited):

[[File:Localmail4.png|640px|thumb|center]]

Your "Account Name":

[[File:Localmail5.png|640px|thumb|center]]

If all went well the account has appeared. Revisit SMTP settings: Edit > Account Settings -> Outgoing Server (SMTP) -> Edit

[[File:Localmail7.png|640px|thumb|center]]

Set Port 25:

[[File:Localmail6.png|640px|thumb|center]]

You may already have messages for user. Maybe not for root yet.

Open <code>/etc/aliases</code> with your favourite editor (vi, vim, nano, geany):

$ sudo geany /etc/aliases

And make sure there is a <code>root: username</code>, username being *your* username of course, and a <code>postmaster: root</code> line in the file.

[[File:Localmail8.png|640px|thumb|center]]

== Installing intrusion detection ==
You can use a firewall, keep your system softwares up to date, stop all unrequired services, use long and difficult passwords and password managers, and more (see below), there is still a chance that some intruder might get into your system.

When intruders penetrate your system there is a great chance that they will want to make their presence as quiet as possible. To do so, they are most likely to replace some common binaries such as <code>ls</code>, <code>netstat</code> or <code>ps</code> with versions that will not make their presence visible. For example, <code>ls</code> can be replaced with an <code>ls</code> version that won't show the files they created, <code>netstat</code> will not show connections that are used by the intruder, and <code>ps</code> will hide processes the intruder runs. To detect changes like that in your system, you can set up intrusion detection with for example <code>aide</code> or <code>tripwire</code>.

''Aide'' and ''tripwire'' are both File Integrity Agents (FIAs). A FIA monitors the integrity and state of the files and objects on your machine. If it detects changes to those files, then it alerts the administrator that an unauthorised access or change has taken place. FIAs usually take a hash of all files to be monitored using an algorithm like MD5. The snapshot is periodically checked against the current hash of the file and any variations alerted on.

For some "''aide'' vs ''tripwire''" notes, see [[Linux applications#Intrusion detection|Linux applications: Intrusion detection]], and for an example install of tripwire on a server, see [[Linux server security#Installing intrusion detection|Linux server security: Installing intrusion detection]].
=== aide ===
==== Installing aide ====
To install ''aide'' on a debian based system:

$ sudo apt-get install aide

Open up <code>/etc/default/aide</code> with your favourite editor to change who to send its reports to:

$ sudo geany /etc/default/aide

[[File:Aide1.png|640px|thumb|center]]

==== Initialising aide ====
Initialise (this can take some time):

$ sudo aideinit

==== Configuring aide ====
Most of ''aide'' configuration can be found in <code>/etc/aide/aide.conf</code>. This file is pretty well documented and default rules are quite decent. Aide aims at reporting files that changed since the last snapshot (stored in <code>/var/lib/aide/aide.db</code>). In hostile environments, a good security measure is to keep the database file on a floppy disk, cdrom or external disk. If you have such a device, you can copy the current snapshot to it and use the snapshot from that device during later runs of aide.

Before making modifications to the configuration file, make a backup first:

$ sudo cp /etc/aide/aide.conf /etc/aide/aide.conf.old

So let say that you have a copy of aide.db on an external disk. To use that snapshot, you can change ''database=file:/var/lib/aide/aide.db'' to ''database=file:/media/user/drive/aide/aide.db'':

[[File:Aide2.png|640px|thumb|center]]

Read through the file to acquaint yourself with aide. By default, changes in ''Binaries'' and ''Libraries'' directories are checked. Those are matched to the ''BinLib'' rule, which basically checks for any changes in permissions, ownership, modification, access and creation date, size change, md5 and sha1 signature, inode, number of links and block count. Then, it also checks for modifications in the log files against the rule ''Logs''. Because log files tend to grow, you cannot use a signature there and you also have to ask aide not to check for size modification (S).

==== Using aide ====
On debian alikes, aide is run on a daily basis through the script <code>/etc/cron.daily/aide</code>. Default settings in <code>/etc/default/aide</code> tells aide to update it's database. Using <code>database_out</code> value in <code>/etc/aide/aide.conf</code>, aide is going to output a new database any time it runs in <code>/var/lib/aide/aide.db.new</code> if you kept the default settings.

Any time you install new packages or change some configuration settings, run:

$ sudo /etc/cron.daily/aide

And check in the report that any modifications were only done to files you intended to modify and that added files are only coming from packages you have just installed and copy the new database to whatever place your database points to. Great way to learn about linux and non-intended changes too.

=== tripwire ===
==== Installing tripwire ====
To install <code>tripwire</code> on a debian-based system:

<pre>$ sudo apt-get install tripwire
[sudo] password for user:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
tripwire
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,385 kB of archives.
After this operation, 10.4 MB of additional disk space will be used.
Get:1 http://ftp.is.debian.org/debian/ jessie/main tripwire amd64 2.4.2.2-4 [1,385 kB]
Fetched 1,385 kB in 0s (1,669 kB/s)
Preconfiguring packages ...
Selecting previously unselected package tripwire.
(Reading database ... 142010 files and directories currently installed.)
Preparing to unpack .../tripwire_2.4.2.2-4_amd64.deb ...
Unpacking tripwire (2.4.2.2-4) ...
Processing triggers for man-db (2.7.0.2-5) ...
Setting up tripwire (2.4.2.2-4) ...</pre>

[[File:Tripwire1.png|660px|thumb|center]]

Use the arrow keys to navigate:

[[File:Tripwire2.png|656px|thumb|center]]

[[File:Tripwire3.png|654px|thumb|center]]

[[File:Tripwire4.png|660px|thumb|center]]

[[File:Tripwire5.png|663px|thumb|center]]

[[File:Tripwire6.png|657px|thumb|center]]

[[File:Tripwire7.png|658px|thumb|center]]

"Remember that passphrase", for example in [[#Managing_passwords|keepassx]] (I had keepassx generate the passphrase):

[[File:Tripwire-site-key-kpassx.png|499px|thumb|center]]

Repeat:

[[File:Tripwire8.png|658px|thumb|center]]

Generating site key (this may take several minutes)...

Do the same for local passphrase:

[[File:Tripwire-local-key-kpassx.png|495px|thumb|center]]

[[File:Tripwire9.png|659px|thumb|center]]

Generating local key (this may take several minutes)...

==== Configuring tripwire ====
Just after install the tripwire directory <code>/etc/tripwire</code> contains a number of files, two of which are useful for its configuration.

* <code>twcfg.txt</code> is for general configuration and can easily be the same for all the computers on the same local network. It contains “things” like the location of the tripwire database, instructions for minimising the amount of time the passphrases are kept in memory, and the number of redundant reports.
* <code>twpol.txt</code> contains the policy that declares all the objects that must be monitored and what to do when one of them is lost or altered. Unlike the configuration file, the policy could (and almost certainly will) vary across the several computers on the same network.

For security reasons (preventing a malicious intruder from spoofing tripwire into giving a false “okay” message) tripwire uses these cleartext files to create binary files for database checking rather than the cleartext files (that are best immediately removed after the deed). If you make incorrect edits to either of these files, you will have to restore from backup or tripwire will not be able to create its database. Before making modifications to the configuration files, make a backup first:

<pre>$ cd /etc/tripwire
$ sudo cp twcfg.txt twcfg.txt.old
$ sudo cp twpol.txt twpol.txt.old</pre>

==== Initialising tripwire ====
Install default policy as root (mind the capital P, and know the cursor does not move at all when copy/pasting the passphrase (right-click works on the command-line for copy/pasting)):

<pre># /usr/sbin/twadmin -m P /etc/tripwire/twpol.txt
Please enter your site passphrase:
Wrote policy file: /etc/tripwire/tw.pol</pre>

Still as root, generate the initial database (You will be asked for your local passphrase):

<pre># /usr/sbin/tripwire -m i
Please enter your local passphrase:
Parsing policy file: /etc/tripwire/tw.pol
Generating the database...
*** Processing Unix File System ***</pre>

It may take a while and then:

<pre>### Warning: File system error.
### Filename: /var/lib/tripwire/debian.twd
### No such file or directory
### Continuing...</pre>

Of course no such database exists (yet). This is the initial run to create it.

And a bunch of these appear with (hopefully) a success message at the end:

<pre>### Warning: File system error.
### Filename: /proc/5199/fd/4
### No such file or directory
...
The object: "/proc/sys/fs/binfmt_misc" is on a different file system...ignoring.
Wrote database file: /var/lib/tripwire/debian.twd
The database was successfully generated.</pre>

The <code>twpol.txt</code> file distributed with tripwire contains anything that could be on a UNIX system, so it is guaranteed to complain about programs that you never installed or placed in a different location. These false positives are created when there is a discrepancy between the default policy and the actual local configuration. ACTUALLY!!!!! :D

Some further customization is necessary to prevent these large numbers of false positives. The “thing” to do to create a good policy is to remove as many unneeded programs as possible before starting. Next, to make your usage as quick and effective as possible, your policy must cover everything you really need to monitor and nothing else. This includes, at least, all the system binary and library directories (minimally the contents of <code>/bin</code>, <code>/sbin</code>, <code>/usr/bin</code>, <code>/lib</code>) and the corresponding configuration files in <code>/etc</code>.

As of version 2.3.1.2-5, the tripwire package manages policy at a directory level: if a directory appears in the policy, tripwire will add the files in that directory. Not all directory policy entries are recursive and tripwire may not check the contents of those subdirectories.

To generate a listing of the discrepancies between the local system and the default policy (or just do the first bit without grep and search for occurrences of "Filename"):

# /usr/sbin/tripwire -m c | grep Filename >> twtest.txt

Waaaait for it! When this listing has been generated, edit <code>/etc/tripwire/twpol.txt</code> again, and comment out or delete each of the filenames listed in twtest.txt. Use your preferred editor (mine is <code>geany</code>):

# geany /etc/tripwire/twpol.txt

[[File:Tripwire11.png|800px|thumb|center]]

[[File:Tripwire12.png|800px|thumb|center]]

And there are other files in the default policy that may not make sense to monitor on your local system. These include lock files (which identify that some process is in use) and pid files (which identify the process ID of some daemons). Such files are likely to change often, if not at every system boot, and they can cause tripwire to generate false positives. You can comment out all of the <code>/var/lock/subsys</code> entries as well as the entry for <code>/var/run</code>.

After that you need to re-install default policy and generate the database again. This can result in:

<pre>*** Processing Unix File System ***
The object: "/dev/hugepages" is on a different file system...ignoring.
The object: "/dev/mqueue" is on a different file system...ignoring.
The object: "/dev/pts" is on a different file system...ignoring.
The object: "/dev/shm" is on a different file system...ignoring.
### Warning: File system error.
### Filename: /proc/22273/fd/4
### No such file or directory
### Continuing...
### Warning: File system error.
### Filename: /proc/22273/fdinfo/4
### No such file or directory
### Continuing...
### Warning: File system error.
### Filename: /proc/22273/task/22273/fd/4
### No such file or directory
### Continuing...
### Warning: File system error.
### Filename: /proc/22273/task/22273/fdinfo/4
### No such file or directory
### Continuing...
The object: "/proc/sys/fs/binfmt_misc" is on a different file system...ignoring.</pre>

Any <code>/dev</code> entries spell that the <code>/dev</code> file system also has other file systems mounted as <code>/dev/hugepages</code>, <code>/dev/mqueue</code>, <code>/dev/pts</code>, and <code>/dev/shm</code>. Similar objects in <code>proc</code> are also ignored. Directories in <code>/proc</code> change all the time and it is a pseudo-filesystem. My solution is to comment out both <code>/dev</code> and <code>/proc</code> and add the listed objects that have appeared and that appear.

<pre>The object: "/proc/sys/fs/binfmt_misc/register" is on a different file system...ignoring.
The object: "/proc/sys/fs/binfmt_misc/status" is on a different file system...ignoring.</pre>

<code>binfmt_misc</code> is a capability of the Linux kernel which allows arbitrary executable file formats to be recognised and passed to certain user space applications, such as emulators and virtual machines. The executable formats are registered through a special purpose file system interface (similar to <code>/proc</code>). Debian-based distributions provide the functionality through an extra binfmt-support package. Needs monitoring. So, added those two as well.

[[File:Tripwire14.png|800px|thumb|center]]

After that you need to re-install default policy and generate the database again.

==== Cleaning up ====
It is now safe to delete the clear text versions of the tripwire policy and configuration files:
# rm /etc/tripwire/twcfg.txt /etc/tripwire/twpol.txt

If they need to be restored, cleartext versions of these files can be created from the by tripwire encrypted versions (lowercase p):

# usr/sbin/twadmin -m p > /etc/tripwire/twpol.txt

==== Using tripwire ====
$ sudo /usr/sbin/tripwire -m c

[[File:Tripwire15.png|800px|thumb|center]]

The management at a directory level also means that addition of packages to a system will almost certainly require the updating or regeneration of the tripwire database.
==== External drive ====
If you wish to use an external drive for holding the binaries and/or database, on the external drive create a directory tripwire to hold both the tripwire binary in <code>/usr/bin/</code> and databases in <code>/var/lib/tripwire</code> so that if and when your system is compromised and the drive is mounted, the intruder would need root to change the files (-rwxr-xr-x). Also create an <code>/etc/tripwire</code> directory for file manipulation on the drive.

This setup has several problems. Even with the paths set to the drive directories and files, when running tripwire from the drive the resulting report is put in the <code>/var/lib/tripwire</code> directory on the local machine (not on the drive), and when re-initialising the database the database is also put in the directory on the local machine. Plus that running tripwire from the drive expects the database in <code>/var/lib/tripwire</code> on the local machine. Rather than hacking the code, I used a workaround and added two little zero version scripts to my [[shell scripting]] wardrobe, one for running tripwire from the drive and another one for updating the tripwire policy database.

The binaries to copy from <code>/usr/sbin</code> are:

* tripwire: The main file; used for initialising the database, checking the integrity of the file system, updating the database and updating the policy.
* twadmin: tripwire’s administrative and utility tool; used for creating and printing configuration files, replacing and printing a policy file, generating site and local keys and other encryption related functions.
* twprint: Used to print the reports and database in human-readable format.
* siggen: Generates the various hashes that tripwire supports for checking the integrity of files.

With drive being the name of your drive as root (<code>sudo -s</code>):

# cp -R /var/lib/tripwire /media/user/drive/tripwire/var/lib/

and copy the binaries tripwire, twadmin, siggen and twprint:

<pre># cp /usr/sbin/tripwire /media/user/drive/tripwire/usr/sbin/
# cp /usr/sbin/twadmin /media/user/drive/tripwire/usr/sbin/
# cp /usr/sbin/siggen /media/user/drive/tripwire/usr/sbin/
# cp /usr/sbin/twprint /media/user/drive/tripwire/usr/sbin/</pre>

Change <code>DBFILE</code> and <code>REPORTFILE</code> with your preferred text editor in <code>/etc/tripwire/twcfg.txt</code> to reflect the location on your external drive:

# geany /etc/tripwire/twcfg.txt

And in <code>twpol.txt</code> change <code>TWBIN</code> and <code>TWVAR</code> to reflect their locations on the external drive:

# geany /etc/tripwire/twpol.txt

== Installing rootkit scanners ==
A rootkit is a collection of tools a hacker installs on a victim's computer after gaining initial access. It generally consists of network sniffers, log-cleaning scripts, and trojaned replacements of core system utilities such as <code>ps</code>, <code>netstat</code>, <code>ifconfig</code>, and <code>killall</code>. Most times they are self-hiding toolkits to avoid the eye of the sysadmin. Applications that are used to detect rootkits are known as rootkit scanners. There are two rootkit scanners that are important to have on your linux box: <code>rkhunter</code> and <code>chkrootkit</code>. Both. They may not stop a sophisticated adversary, they do help us see mistakes we make.

=== rkhunter ===
Rootkit Hunter (<code>rkhunter</code>) is a shell script to detect known rootkits or malware. It also performs checks to see if commands have been modified, and various checks on the network interfaces, including checks for listening applications.

==== Installing rkhunter ====
You can install it from the Software Center or synaptic, or from the commandline:

$ sudo apt-get install rkhunter

Or you can get get it from http://sourceforge.net/projects/rkhunter/ or check current version number and use wget:

$ wget http://downloads.sourceforge.net/project/rkhunter/rkhunter/1.4.2/rkhunter-1.4.2.tar.gz

After download, extract the files and enter the resulting directory:

$ tar xzvf rkhunter*
$ cd rkhunter*

Run the install script:

$ sudo ./installer.sh --layout /usr --install

[[File:Rkhunter-running-install-script.png|599px|thumb|center]]

==== Updating database ====
Update the rkhunter database:

$ sudo rkhunter --update

This command has to be run on a regular basis to keep the database of known rootkits current. You can use <code>cron</code> to schedule running of this command at regular intervals. See [[Kinky_linux_command-line#Running_jobs.2C_jobs.2C_jobs.21|Running jobs, jobs, jobs!]] for more on running commands or scripts at a given date and time.

[[File:Update-rkhunter-db-and-baseline.png|788px|thumb|center]]

==== Baselining rkhunter ====
Set baseline file properties by checking the current values and storing them as known-good values:

$ sudo rkhunter --propupd

rkhunter is configured to be generic and each distro can (and most do) diverge from expected defaults. These appear during an initial run as false positives:

$ sudo rkhunter -c --enable all --disable none

You'll get groups of results, warnings, and at the end a summary of the results.

[[File:Rkhunter-first-run.png|786px|thumb|center]]

You can have a look at more details of the warnings in <code>/var/log/rkhunter.log</code>.

$ sudo geany /var/log/rkhunter.log

I immediately encounter these noticeable two:

[[File:No-mail-on-warning.png|983px|thumb|center]]

[[File:Package-manager.png|780px|thumb|center]]

Sometimes if you change something, more warnings vanish or appear, so I always work on a few unrelated ones, then run <code>--propupd</code> and <code>rkhunter -c</code> again as shown above, until I no longer have warnings. Open up rkhunter's configuration file:

$ sudo geany /etc/rkhunter.conf
Remove the <code>#</code> in front of MAIL-ON-WARNING and replace the fake email adresses with your local mail account (see above):
[[File:Mail-on-warning.png|705px|thumb|center]]
Likewise set the package manager:
[[File:Package-manager-set.png|672px|thumb|center]]

I am also getting warnings from the ''deleted_files'' test:

[[File:Deleted-files.png|990px|thumb|center]]

No problem. Mate and iceweasel need to be able to use tmp files. You can make the strings to allow processes using deleted files for in <code>/etc/rkhunter.conf</code> (on debian-based):

<pre>$ sudo awk '/Process: / {print "ALLOWPROCDELFILE="$3}' /var/log/rkhunter.log | sort -u
[sudo] password for user:
ALLOWPROCDELFILE=/usr/bin/mate-terminal
ALLOWPROCDELFILE=/usr/lib/iceweasel/iceweasel</pre>

Select and copy/paste the strings to the <code>/etc/rkhunter.conf</code> file (right-click mouse will give you the option to copy from command-line):

[[File:Allowprocdelfile.png|745px|thumb|center]]

And a warning from the ''packet_cap_apps'' test:

[[File:Dhcpclient.png|768px|thumb|center]]
Get the strings for allowing (current) packet capturing applications:
<pre>$ sudo awk -F"'" '/is listening on the network/ {print "ALLOWPROCLISTEN="$2}' /var/log/rkhunter.log
[sudo] password for user:
ALLOWPROCLISTEN=/sbin/dhclient</pre>
and I only needed to comment it out:
[[File:Allowproclisten.png|730px|thumb|center]]

A few suspicious files and a hidden dir from the ''filesystem'' test:

[[File:Suspicious-files-hidden-dirs.png|666px|thumb|center]]

<code>/dev/shm</code> is an implementation of traditional shared memory concept for passing data between programs. One program will create a memory portion, which other processes (if permitted) can access. Like a tmp file. A performance choice. <code>/dev/shm/pulse-shm-0123456789</code> or whatever number seems not dangerous for the system. It's a clean install, and I'll whitelist it.

If I were to just add a line <code>ALLOWDEVFILE=/dev/shm/pulse-shm-0123456789</code> to the rkhunter configuration file, this will stop the warning. Until I reboot. The numbers in the end of pulse-shm files change on every start and is unpredictable. This is where [[Kinky_linux_command-line#Regular_expressions|regular expressions and wildcards]] come in handy. As it is, it only needs commenting out:

[[File:Pulse-shm.png|744px|thumb|center]]

The presence of the hidden <code>/etc/.java</code> directory also gives me a warning. So often do <code>/dev/.udev</code>, <code>/dev/.static</code> and <code>/dev/.initramfs</code>. These are known hidden (sofar) non-malignent hidden directories and can simply be whitelisted (only needs commenting out):

[[File:Java-hidden-dir.png|747px|thumb|center]]

ALL GREEN. Purrrrfect baseline. For now. :) In the near future more applications may try to use deleted files, and after installation of applications more warnings may appear that I'll have to deal with to keep it green!

==== unhide ====
ACTUALLY, nearly all green and two purple:

[[File:Hidden-processes-skipped.png|640px|thumb|center]]

[[File:Hidden-ports-skipped.png|640px|thumb|center]]

* <code>unhide</code> (ps) is for detecting hidden processes and implements six techniques:
** Compare <code>/proc</code> vs <code>/bin/ps</code> output
** Compare info gathered from <code>/bin/ps</code> with info gathered by walking through the ''procfs''.
** Compare info gathered from <code>/bin/ps</code> with info gathered from ''syscalls'' (syscall scanning).
** Full PIDs space occupation (PIDs bruteforcing)
** Reverse search, verify that all thread seen by <code>ps</code> are also seen by the kernel (<code>/bin/ps</code> output vs <code>/proc</code>, ''procfs walking'' and ''syscall'')
** Quick compare <code>/proc</code>, ''procfs walking'' and ''syscall'' vs <code>/bin/ps</code> output.
* <code>unhide-tcp</code> is a forensic tool that identifies TCP/UDP ports that are listening but are not listed in /bin/netstat through brute forcing of all TCP/UDP ports available. It comes with unhide.

$ sudo apt-get install unhide

$ sudo rkhunter --propupd

$ sudo rkhunter -c --enable all --disable none

Now it's all green. :)

=== chkrootkit ===
<code>chkrootkit</code> is a shell script that checks system binaries for rootkit modification.

==== Installing chkrootkit ====
To install chkrootkit:
$ sudo apt-get install chkrootkit

==== Configuring chkrootkit ====
If you want an automatic daily run of chkrootkit, open <code>/etc/chkrootkit.conf</code> and replace <code>RUN_DAILY="false"</code> with <code>RUN_DAILY="true"</code>

$ sudo geany /etc/chkrootkit.conf

[[File:Chkrootkit-config.png|640px|thumb|center]]

==== Using chkrootkit ====
Using chkrootkit (perform all tests):
$ sudo chkrootkit

And you may get something like:
<pre>Checking `chkutmp'... The tty of the following user process(es) were not found in /var/run/utmp !
! RUID PID TTY CMD
! root 897 tty7 /usr/bin/X :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
chkutmp: nothing deleted
</pre>

The <code>man utmp</code> reads:
''The utmp file allows one to discover information about who is currently using the system. There may be more users currently using the system, because not all programs use utmp logging.''

''Warning: utmp must not be writable, because many system programs (foolishly) depend on its integrity. You risk faked system logfiles and modifications of system files if you leave utmp writable to any user.''

These messages are always worth checking out because chkrootkit compares all the user processes running in the system with what is registered in <code>/var/run/utmp</code>, reporting the fact that this particular process run by root is not registered (is hidden). It doesn't mean your system has cooties (rootkits or loggers), but it might.

In this case it is not the cooties, but xorg listening for remote desktop connection requests. :D I have tried turning it off. Not a good idea.

You may also get:

<pre>Searching for suspicious files and dirs, it may take a while... The following suspicious files and directories were found:
/usr/lib/icedove/.autoreg /usr/lib/pymodules/python2.7/.path /usr/lib/jvm/.java-1.7.0-openjdk-amd64.jinfo</pre>

These are false positives: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=789052

== Managing passwords ==
Using a password manager allows you to load passwords in your clipboard, making it hard to catch it with a keylogger (just in case). And all good password managers allow you to generate passwords, making the passwords independent from your personal preferences and harder to crack. One of the easiest password managers is a GUI tool called keepassx:

$ sudo apt-get install keepassx

It will appear in your menu. A "first steps" tutorial can be found here http://keepass.info/help/base/firststeps.html

Recommendations:
* Keep the database on a removable medium like an encrypted USB stick.
* Use a second encrypted stick to keep a backup of that database on.
* For added security, set up the database to use a password as well as a keyfile.
* Try to always close off the application. It uses a lockfile to prevent access by other processes. Quitting the application removes it. If not, and whenever you get a message that you cannot open your database, check on the USB in the directory where the database resides that there isn't a lockfile blocking access. If there is a lockfile while the database is not open for you, just delete the lockfile.
* On rare occasions a database is truly corrupted. That backup on that other stick, phew. :D

If you want to try your hand at cracking your own (current and new) passwords, try one of these http://sectools.org/tag/pass-audit/

== Blacklisting modules ==
=== Camera ===

[[File:Yao-ming-meme-turning-on-the-webcam.jpg|420px|thumb|right]]

Intruders can gain access to their victims’ computers with remote-access Trojans (RATs) – malware that gives an intruder administrative control over its targeted computers, including, in this case, the ability to remotely control webcams. It’s an invisibly-installed malware program spread via email attachment or by tricking victims into visiting a booby-trapped site.

Tape the camera. It may even be possible to turn it off in BIOS (depending on your version).

Better yet, if you know what driver module is controlling the webcam, you can disable the driver with <code>modprobe -r</code>.

Check with:

$ lsmod | grep "uvc"

If stuff like this appears, your webcam device uses an uvc driver http://www.ideasonboard.org/uvc/:
<pre>uvcvideo 79005 0
videobuf2_vmalloc 12816 1 uvcvideo
videobuf2_core 47787 1 uvcvideo
videodev 126451 3 uvcvideo,v4l2_common,videobuf2_core
media 18305 2 uvcvideo,videodev
usbcore 195340 7 uvcvideo,ums_realtek,usb_storage,ehci_hcd,ehci_pci,usbhid,xhci_hcd</pre>

To disable the cam:

$ sudo modprobe -r uvcvideo

To enable the cam:

$ sudo modprobe uvcvideo

If not an uvcvideo driver, try <code>$ sudo lsmod | grep "media"</code> and use the <code>modinfo</code> command to find out more about the modules listed behind it to locate your webcam driver module:

$ sudo modinfo [modulename]

Then replace the "uvcvideo" in the above modprobe command with the name of your driver module.

To disable the webcam at boot (if file not exists, create it):

$ sudo geany /etc/modprobe.d/blacklist.conf

At bottom of the file add this line: <code>blacklist uvcvideo</code> (or the name of your driver instead of "uvcvideo").

Save the file and reboot. Your webcam is no longer functioning.

=== Microphone ===
Open the alsa control panel for muting the microphone:

$ sudo alsamixer

You can also use the same <code>modprobe</code> enabling/disabling method as used above for webcam. This page may be helpful for finding your driver module: http://alsa.opensrc.org/ALSA_modules.
=== Firewire and thunderbolt ===
A '''d'''irect '''m'''emory '''a'''ccess (DMA) attack is a type of side channel attack in which an adversary penetrates a device by exploiting the presence of high-speed expansion ports that permit Direct Memory Access. Firewire, thunderbolt and ExpressCard allow (by design) any connecting device full direct memory access to your system. They can be disabled in /etc/modprobe.d/blacklist-dma.conf:

<pre>blacklist firewire-core
blacklist thunderbolt</pre>

The modules will be blacklisted upon reboot.

== Using a firewall ==
[[File:Gufw-mint.png|320px|thumb|right]]
With a firewall, you can:
* Log hosts scanning services that aren't running
* Limit the services that programs can connect to
* Segregate the local network into trust segments (LAN, DMZ, and Internet)
* Redirect ports to the hosts providing the service (NAT).

If this is for a personal machine at home, only the first two are of interest. These are also the two that are most likely to cause issues for a beginning user.
=== ufw and gufw ===
==== Enabling ====
Mint and ubuntu come with a firewall installed, but it is disabled. Usually it's better to turn it on. Especially on mobile devices like laptops, which sometimes connect to other networks than your own.

The firewall is called <code>ufw</code> ('''u'''ncomplicated '''f'''ire'''w'''all), and can be operated from the terminal (Menu > Accessories > Terminal):

$ sudo ufw enable

Type your password when prompted for it; this will remain entirely invisible, not even asterisks will show, which is normal.

Ufw has a sensible set of default settings (profile), which are fine for the vast majority of home users. So unless you have special wishes: you're done!
==== Checking status ====
You can check the status of the firewall with this command:

$ sudo ufw status verbose

If you wish, you can install a GUI frontend to ufw, called gufw:

$ sudo apt-get install gufw

And it'll be in your menu items (the icon in mint is a brick wall).

==== Making the rules (ufw) ====
Found a pretty good no-nonsense tutorial here https://help.ubuntu.com/community/UFW

As example, try your hand at <code>ICMP</code> (see [[Networking_concepts#Security_3|ICMP security notes]]). By default, UFW allows ping requests. You may find you wish to leave (icmp) ping requests enabled to diagnose networking problems. To disable ping (ICMP) requests, edit <code>/etc/ufw/before.rules</code> and remove the following lines:

<pre># ok icmp codes
-A ufw-before-input -p icmp --icmp-type destination-unreachable -j ACCEPT
-A ufw-before-input -p icmp --icmp-type source-quench -j ACCEPT
-A ufw-before-input -p icmp --icmp-type time-exceeded -j ACCEPT
-A ufw-before-input -p icmp --icmp-type parameter-problem -j ACCEPT
-A ufw-before-input -p icmp --icmp-type echo-request -j ACCEPT</pre>

or change the "ACCEPT" to "DROP":

<pre># ok icmp codes
-A ufw-before-input -p icmp --icmp-type destination-unreachable -j DROP
-A ufw-before-input -p icmp --icmp-type source-quench -j DROP
-A ufw-before-input -p icmp --icmp-type time-exceeded -j DROP
-A ufw-before-input -p icmp --icmp-type parameter-problem -j DROP
-A ufw-before-input -p icmp --icmp-type echo-request -j DROP</pre>

=== iptables and netfilter ===
==== Installing persistence ====
<code>iptables</code> is installed by default. For rules to be persistent (automatically loaded) so they work after reboot, install <code>iptables-persistent</code>. It relies on <code>netfilter-persistent</code>:

<pre>$ sudo apt-get install iptables-persistent
[sudo] password for user:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
netfilter-persistent
The following NEW packages will be installed:
iptables-persistent netfilter-persistent
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 18.9 kB of archives.
After this operation, 147 kB of additional disk space will be used.</pre>

[[File:Iptables-peristent-install.png|659px|thumb|center]]

==== Configuring iptables ====
To set it to deny everything except the bare minimum requirements that you need on your system(s) and only accept connections that have been explicitly allowed in the rules:

Accept localhost data:
<pre>$ sudo iptables -A INPUT -i lo -j ACCEPT</pre>

Accept data related to outbound connections your system has initiated (this includes <code>apt</code> mirrors):
<pre>$ sudo iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT</pre>

You can set up tables for rules for traffic you wish to accept on your system from external connections:

<pre># iptables -N accept_traffic
# iptables -A INPUT -j accept_traffic</pre>

You then can use it, for example on a server where you run SSH, with (change port to port you set):

<pre># iptables -A accept_traffic -p tcp --dport 22 -j ACCEPT</pre>

The two most common cracks posted on the Ubuntu forums are <code>ssh</code> and <code>vnc</code>, both running with password authentication. This probably is true for all debian-based distro's. You are recommended to secure <code>ssh</code> by using keys ('''and disabling password authentication''') and either configuring <code>iptables</code> (see http://bodhizazen.net/Tutorials/iptables) or using a service such as denyhosts or fail2ban (see [[Linux server security]]).

To reject all traffic for which there are no rules set (the first way sends a ''Host Unreachable'' ICMP packet then terminates the connection, the second way simply ignores):

<pre># iptables -A INPUT -j REJECT --reject-with icmp-host-unreachable</pre>

or:

<pre># iptables -A INPUT -j DROP</pre>

==== Configuring ip6tables ====
The rules for IPv6 are similar:

Accepting localhost data:
<pre>$ sudo ip6tables -A INPUT -i lo -j ACCEPT</pre>

Accepting data related to outbound connections your system has initiated:
<pre>$ sudo ip6tables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT</pre>

Setting up a table:

<pre># ip6tables -N accept_traffic
# ip6tables -A INPUT -j accept_traffic</pre>

Rejecting all traffic for which there are no rules set:

<pre># ip6tables -A INPUT -j REJECT --reject-with icmp6-addr-unreachable</pre>

or

<pre># ip6tables -A INPUT -j DROP</pre>

==== Making the rules (iptables) ====
For a readable tutorial on the basics see ''The Beginner’s Guide to iptables, the Linux Firewall'' http://www.howtogeek.com/177621/the-beginners-guide-to-iptables-the-linux-firewall or the IP Tables Primer http://bodhizazen.net/Tutorials/iptables/

As example, try your hand at managing IPv4 <code>ICMP</code> packets (see [[Networking_concepts#Security_3|ICMP security notes]]):

For accepting "Time Exceeded" (necessary for time-restricted connection setups):
<pre>$ sudo iptables -A INPUT -p icmp -m icmp --icmp-type 11 -j ACCEPT</pre>

For accepting "Destination Unreachable":
<pre>$ sudo iptables -A INPUT -p icmp -m icmp --icmp-type 3/4 -j ACCEPT</pre>

For accepting PING requests/responses ("Echo" ICMP, for keep-alive requests):
<pre>$ sudo iptables -A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT</pre>

For IPv6 there are a [[Networking_concepts#Internet_Control_Message_Protocol_.28ICMP.29|kazillion ICMP requests]] so if you are not blocking IPv6, it is recommended to not block ICMP packets:

<pre>$ sudo ip6tables -A INPUT -p icmpv6 -j ACCEPT</pre>

==== Saving the rules ====
For saving the rules so they can be loaded at every reboot (you need to have <code>iptables-persistent</code> installed)
<pre># iptables-save > /etc/iptables/rules.v4
# ip6tables-save > /etc/iptables/rules.v6</pre>

== Safer browsing ==
It is clear that browsing has the most low cost attack vectors for data theft by hackers, tracking by corporations and for governmental spying. Safer browsing is a good investment all around.

Always use a GNU free software browser. This ensures it can be freely audited. Doesn’t mean it is audited or reviewed. Check for that.

=== Settings ===
For disabling flash:

* Firefox: Add-ons > Plugins > Flash > Never Activate.
* Chromium: Settings > Show advanced settings… > Content settings > Do not run plugins by default.

For disabling java:

* Firefox: Add-ons > Plugins > Java > Never Activate.
* Chromium: Settings > Show advanced settings… > Content settings > Do not run plugins by default.

=== Extensions ===
==== Script blockers ====
The NoScript Firefox extension provides extra protection for Firefox, Seamonkey and other mozilla-based browsers: this add-on allows JavaScript, Java, Flash and other plugins to be executed only by trusted web sites of your choice (e.g. your online bank) http://noscript.net/. There is mozilla talk of “adding NoScript functionality into the core browser”.

The closest Extension to NoScript that I have found for Chromium is ScriptSafe, which has certain operational differences: Apparently ScriptSafe acts more stringently concerning components of Web pages compared to NoScript.

HTTP Switchboard is a scriptblocker AND an adblocker. It is no longer developed. The project has split into two distinct, more advanced extensions: uBlock Origin and uMatrix.
==== Ad blockers ====
Ghostery looks for third-party "trackers" on web pages. These can be social network widgets, advertisements, invisible pixels used for tracking and analytics, etc. It notifies of presence and companies operating the trackers and allows for blocking the trackers.

BUT how do they make money? Ghostery sells the tracking data that users (who volunteer for this) gather from sites they visit. Ghostery Inc. then sells this data to the companies that run the sites <ref>A Popular Ad Blocker Also Helps the Ad Industry http://www.technologyreview.com/news/516156/a-popular-ad-blocker-also-helps-the-ad-industry/</ref>.
==== Restricting access to browsing history ====
SafeHistory restricts the marking of visited links on the basis of the originating document, defending against web privacy attacks that remote sites can use to determine your browser history at other sites https://addons.mozilla.org/en-US/firefox/addon/safehistory/ (and cannot be installed in up-to-date iceweasels anymore)

== Using removable storage media ==
A simple but straightforward security method is to keep all sensitive information on removable storage media like portable flash memory drives or external hard drives that you only mount the device when you need access to it.

You can turn old harddrives into external harddrives by taking them out of an old PC and putting them in a portable HDD USB storage kit (alias harddrive enclosure). These "DIY" kits are available for 2.5 inch and 3.5 inch HDD using either IDE or SATA connections and cost way less than a manufactured USB HDD.

You can check state, and turn write protect of a drive (or USB stick) on and off (during initialization and updates) with <code>hdparm</code> http://linux.die.net/man/8/hdparm

You may have a permission problem accessing it. If so, change the group of <code>/media/[externaldisk]</code> to <code>storage</code>, change permissions to <code>775</code> and add users that need to be able to write to disk to the <code>storage</code> group (re-login required before changes are visible).

As root (or use sudo) do:

<pre># chgrp storage /media/[externaldisk]
# chmod 775 /media/[externaldisk]
# gpasswd -a [user] storage</pre>

This way everyone will be able to read the disk, but only root and users in storage group will be able to write to it. If you want everyone to be able to read/write, use chmod 777. If the operation isn't permitted while running it as root you probably need to add support for the NTFS file system first:

<pre># apt-get update
# apt-get install ntfs-3g</pre>

== Encryption ==
But the most popular forms of security all rely on encryption, the process of encoding information in such a way that only the person (or computer) with the key can decode it. Or maybe others can for some forms of encryption. Or can soon?

For an overview of encryption concepts see [[encrypting everything]] and for application of concepts in useful tools, see [[object encryption on linux]] and [[anonymising your traffic with linux]].

== Making backups ==
Data backups were traditionally done by commands to archive and compress files to backup, then writing that backup archive to tape. Tape archiving has been replaced by backing up data over the network, to other hard disks, CD's, DVD's and other low-cost removable media.

=== Archiving and compressing ===
Linux offers separate tools for gathering groups of files into a single archive and compressing that archive for efficient storage. Oh yuk, "efficient", oh well. It's about using less disk space and reducing transfer times over networks. And comes at a price. It can increase your CPU usage.
==== tar ====
You can combine archiving and compressing together by using additional options to the <code>tar</code> ('''t'''ape '''ar'''chiver) command. Tar can behave differently on different distros. The manual page contains descriptions of the myriad of features that come with <code>tar</code>. The basic operations of <code>tar</code>:

* Create a backup archive: <code>c</code>
* Extract files from an archive: <code>x</code>
* Compare differences between archives: <code>d</code>
* Update files in an archive: <code>u</code>
* Append files: <code>r</code> or <code>A</code>
* Delete files from an existing archive: <code>d</code>
* List contents of an archive: <code>t</code>

When creating a <code>tar</code> archive, you can add options that compress the resulting archive. With <code>j</code> the archive will be compressed in <code>bzip2</code> format, and with <code>z</code> in <code>gzip</code> format. For example, <code>tar</code> with <code>bzip2</code> compression:

$ tar xjvf funnybusiness.tar.bz2 *.txt

You can also use other compression mechanisms like <code>lzop</code> with <code>tar</code>. It is faster, but the compression is less than with <code>bzip2</code> which is around 10 times slower while only giving twice the amount of compression. You can also set the compression level in the compression commands themselves.

==== gzip ====
Gzip compress the size of the given files using Lempel-Ziv coding (LZ77). Whenever possible, each file is replaced by one with the extension .gz. For more on how it works, see https://en.wikipedia.org/wiki/DEFLATE

Compress a file:
$ gzip funnybusiness.txt
Compress all images with a <code>.jpg</code> extension in a directory:
$ gzip *.jpg
Compress all files in a directory:
$ gzip -rv [directory]

Test integrity of a gzip compressed file:
$ gzip -tv funnybusiness.gz

To uncompress a compressed file:
$ gunzip -v funnybusiness.gz

Fastest compression time, least compression:

$ gzip -1 funnybusiness.gz

Slowest compression time, most compression:

$ gzip -9 funnybusiness.gz

==== bzip2 ====
<code>bzip2</code> compresses files using the Burrows-Wheeler block sorting text compression algorithm and Huffman coding. Compression is generally considerably better than that achieved by bzip command (LZ77/LZ78-based compressors). Whenever possible, each file is replaced by one with the extension <code>.bz2</code>.

$ bzip2 [filename]

Uncompress:

$ bunzip [filename].bz2

==== lzop ====
<code>lzop</code> uses the LZO data compression library for compression services, and its main advantages over gzip are much higher compression and decompression speed (at the cost of some compression ratio). <code>lzop</code> is copyrighted under the terms of the GNU General Public License (GPL). More on http://www.lzop.org/

You will probably have to install it first:

$ sudo apt-get install lzop

Compress:

$ lzop [filename]

Unlike <code>gzip</code> and <code>bzip2</code>, <code>lzop</code> has no separate command for unlzopping. NO UNLZOPPING! Use the <code>d</code> option:

$ lzop -dv [filename].lzo

=== Backing up files ===
Data backup is something that everybody should do, but unfortunately, few people do it on a regular basis. You can make a backup of your entire system with rsync or with unison.

==== rsync ====
<code>rsync</code> ('''r'''emote '''sync'''), is a remote and local file synchronisation tool. It uses an algorithm that minimises the amount of data copied by only moving the portions of files that have changed. It operates in a way similar to <code>ssh</code>, <code>scp</code>, and <code>cp</code>, for example:

$ rsync -a [directoryname]/ /media/[pathtoexternaldisk]/[directoryname]

The <code>a</code> option is a combination flag. It stands for "archive" and syncs recursively and preserves symbolic links, special and device files, modification times, group, owner, and permissions. Rsync can be used for a lot more in many different ways http://www.thegeekstuff.com/2010/09/rsync-command-examples/.

==== unison ====
<code>unison</code> allows for accessing the same set of files from any computer (running Mac OS X, M$ Windows XP, or UNIX/Linux) and keeps these files up-to-date by always maintaining the most recently-modified version of each file during synchronization. The great benefit of using <code>unison</code> to replicate files across different computers (servers) is that your backups are alive http://www.cis.upenn.edu/~bcpierce/unison/.

== Traffic monitoring ==
=== wireshark ===
Wireshark lets you capture and interactively browse the traffic running on a computer network and can be used to debug network protocol implementations, examine security problems and inspect network protocol internals. See [[networking concepts]].
==== Installing wireshark ====

$ sudo apt-get install wireshark

Choose YES if you want to capture packets as user:

[[File:Configuring-wireshark-common.png|657px|thumb|center]]

You will also need to allow non-root users to capture packets. For that, follow the procedure described in <code>/usr/share/doc/wireshark-common/README.Debian</code>:

$ sudo adduser user wireshark

==== Capturing packets ====
You can launch it and click the name of an interface under ''Interface List'' to start capturing packets on that interface:

[[File:Interface-list.png|840px|thumb|center]]

You can configure advanced features by clicking ''Capture Options'':

[[File:Capture-options.png|840px|thumb|center]]

As soon as you click ''Start'', you’ll see the packets start to appear in real time. Wireshark captures each packet sent to or from your system. If you’re capturing on a wireless interface and have promiscuous mode enabled in your capture options, you’ll also see other the other packets on the network.

Click the stop capture button near the top left corner of the window when you want to stop capturing traffic.

[[File:Stop-capture.png|840px|thumb|center]]

For examples of its use, see [[Anonymising your traffic with linux]]. If there’s nothing interesting on your own network to inspect, Wireshark’s wiki has you covered. The wiki contains a page of sample capture files that you can load and inspect https://wiki.wireshark.org/SampleCaptures. To open a capture file, click ''Open'' on the main screen and browse for a file. You can also save your own captures and open them later.

==== Filtering packets ====
If you’re trying to inspect something specific, such as the traffic a program sends, it helps to close down all other applications using the network so you can narrow down the traffic. Still, you’ll likely have a large amount of packets to sift through. That’s where filtering come in.

The most basic way to apply a filter is by typing it into the filter box at the top of the window and clicking ''Apply''. For example, type ''dns'' and you’ll see only DNS packets.

[[File:Filtering.png|840px|thumb|center]]

Another interesting thing you can do is right-click a packet and select Follow TCP, UDP or SSL Stream.

==== Inspecting packets ====
Click a packet to select it and you can dig down to view its details. You can also create filters by using the ''Apply as Filter'' submenu to create a filter based on it (those ''and''s and ''or''s are boolean algebra):

[[File:Set-filter-based-on-packet.png|840px|thumb|center]]

== Configuring sysctl (*) ==
<code>sysctl</code> is a tool for examining and changing kernel parameters at runtime, a near-necessary tool in the defense from malware.

$ sudo geany /etc/sysctl.conf

For now it is naught more than removing a few comment-out chars and adding a few lines:

<pre>#
# /etc/sysctl.conf - Configuration file for setting system variables
# See /etc/sysctl.d/ for additonal system variables
# See sysctl.conf (5) for information.
#

#kernel.domainname = example.com

# Uncomment the following to stop low-level messages on console
#kernel.printk = 3 4 1 3

##############################################################3
# Functions previously found in netbase
#

# Uncomment the next two lines to enable Spoof protection (reverse-path filter)
# Turn on Source Address Verification in all interfaces to
# prevent some spoofing attacks
net.ipv4.conf.default.rp_filter=1
net.ipv4.conf.all.rp_filter=1

# Uncomment the next line to enable TCP/IP SYN cookies
# See http://lwn.net/Articles/277146/
# Note: This may impact IPv6 TCP sessions too
net.ipv4.tcp_syncookies=1

# Uncomment the next line to enable packet forwarding for IPv4
#net.ipv4.ip_forward=1

# Uncomment the next line to enable packet forwarding for IPv6
# Enabling this option disables Stateless Address Autoconfiguration
# based on Router Advertisements for this host
#net.ipv6.conf.all.forwarding=1

###################################################################
# Additional settings - these settings can improve the network
# security of the host and prevent against some network attacks
# including spoofing attacks and man in the middle attacks through
# redirection. Some network environments, however, require that these
# settings are disabled so review and enable them as needed.
#
# Do not accept ICMP redirects (prevent MITM attacks)
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
# _or_
# Accept ICMP redirects only for gateways listed in our default
# gateway list (enabled by default)
# net.ipv4.conf.all.secure_redirects = 1
#
# Do not send ICMP redirects (we are not a router)
net.ipv4.conf.all.send_redirects = 0
#
# Do not accept IP source route packets (we are not a router)
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0
#
# Log Martian Packets
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1</pre>

For more see resources below.

== Auditing your setup ==
[[File:Look-at-all-the-fucks-i-give.jpg|300px|thumb|right]]
=== Tiger ===
Tiger supports multiple UNIX platforms it is free and provided under a GPL license. Tiger is used as a security audit and is useful both for system analysis (security auditing) and for real-time, host-based intrusion detection. More on its history and resurrection here http://digital-era.net/setting-up-and-using-tiger-for-security-audit-and-intrusion-detection/.

It consists of Bourne Shell scripts, C code and data files used for checking for security problems on a UNIX system. It scans system configuration files, file systems, and user configuration files for possible security problems and creates a report on issues found.

==== Installing tiger ====
$ sudo apt-get install tiger

==== Configuring tiger ====
* You can mess with the tiger variables in the <code>/etc/tiger/tigerrc</code> configuration file. For each available module there is a corresponding variable in the file that determines whether the module is run. A variable can be set equal to Y to run, or N to skip. Other configuration variables will modify the behaviour of some modules, and can be adjusted based on the operating system.
* The <code>/etc/tiger/tiger.ignore</code> configuration file defines the set of messages that will not be presented in the report even if any of the modules generate them. All entries (per line) are used as extended regular expressions that are compared against each message (mind the overhead which grows with the size of the file).

==== Using tiger ====
The command tig‐exp(8) can be used to obtain explanations of the problems reported by tiger. Doing a first run can be shocking (not boring). For all listed fixes below the explanations are included.

Checking passwd files:

<pre>--WARN-- [pass014w] Login (name) is disabled, but has a valid shell.</pre>

The listed login ID is disabled in some manner (‘*’ in passwd field, etc), but the login shell for the login ID is a valid shell (from /etc/shells or the system equivalent). A valid shell can potentially enable the login ID to continue to be used. An attacker can hide programs within these shells by simply ‘su’ing into these accounts.

Debian comes with some predefined users (https://www.debian.org/doc/manuals/securing-debian-howto/ch12.en.html#s-faq-os-users) listed in <code>/etc/passwd</code>:

* The default behavior is that UID’s from 0 to 99 are reserved in Debian (https://www.debian.org/doc/debian-policy/) to ease the installation of some services that require that they run under an appropriate user/UID. If you do not intend to install these services, you can safely remove those users who do not own any files in your system and do not run any services.
* UID’s from 100 to 999 are created by packages on install (and deleted when the package is purged). Some of the accounts in this range are users linked to programs you’re running. In general, it is better to prune your system via the associated packages than try to manage them manually.
* Manually created users end up with a UID just above 1000 (unless you manually specified the uid).

For example <code>www-data</code> in most cases, does not need a shell, but some cgi could need it. Historically all users under UID 99 had a <code>/bin/sh</code> as login shell, and the ones found and listed here by tiger are historic or legacy users, for all “modern” linux services already come preconfigured to <code>/bin/false</code>. For those not preconfigured, you can dig into your system to know what each user does there:

* Do a <code>top</code> or a <code>ps aux</code> and check the users; if there’s a process started by “nobody” user you can’t delete it.
* Also check crontab and cron.d; if you delete a user required for starting a cron job, that job will never work again.
* Another useful check is to check the <code>/etc/shadow</code> file and see which second fields are X (crossed out) for which accounts.

If you conclude a process shouldn’t be there, then stop the daemons, uninstall its packages, and then delete the users. If you are unsure about deleting a user used as administrative user you can set its shell to <code>/bin/false</code> or <code>/sbin/nologin</code> (or something else that doesn’t exist). The user will remain there but nobody will get a shell from it.

Always leave the shell for your user and root user. Whatever you do, do NOT remove or change the shell for root.

<pre>--WARN-- [pass015w] Login ID sync does not have a valid shell (/bin/sync)</pre>

The listed login ID does not have a valid login program or shell. Usually these Shells are defined in <code>/etc/shells</code>.

This is a special case in which there is not a valid shell for sync. <code>/bin/sync</code> is a valid file with a valid purpose. You can ignore.

<pre>--WARN-- [pass006w] Integrity of password files questionable (/usr/sbin/pwck r).</pre>

The password files have integrity issues as found by <code>pwck -r</code> http://www.sanfoundry.com/pwck-command-usage/. This can lead to looping of password manipulation programs and to authentication or login issues if not corrected. Install the relevant packages or create the directory and set permissions for only limited access.

$ sudo /usr/sbin/pwck -r

Checking user accounts:

<pre>--WARN-- [acc022w] Login ID nobody home directory (/nonexistent) is not accessible.</pre>

But without a valid home directory, the user could end up with <code>/</code> as the home directory. You can disable his shell in <code>/etc/passwd</code> by setting it to <code>/bin/false</code> or <code>/sbin/nologin</code> and ignore the resulting error, or you can create the directory, change its ownership and persmissions.

<pre>$ sudo mkdir /nonexistent
$ chown nobody:nobody /nonexistent
$ chmod 400 /nonexistent</pre>

Checking cron entries:

Cron allows users to submit jobs for the system to do at a particular, possibly recurring time. It can be very useful, but also has a very real potential for abuse by either users or system crackers. Users can be restricted to use cron by creating a <code>/etc/cron.allow</code> (holding only system administrators) or a <code>/etc/cron.deny</code> file (listing which users are not allowed access).

<pre>--WARN-- [cron004w] Root crontab does not exist</pre>

Not a problem. Ignore.

<pre>--WARN-- [cron005w] Use of cron is not restricted</pre>

For standard Debian systems, all users may use this command. Use the fix on Linux/UNIX Restricting at/cron Usage To Authorized Users here http://www.cyberciti.biz/faq/howto-restrict-at-cron-command-to-authorized-users/ or ignore.

Checking services:

<pre>--WARN-- [inet003w] The port for service (x) is also assigned to service (y)</pre>

A program with a port assigned to TCP and a second entry for the same port with UDP. These service ports are defined in /etc/services and ''double entries'' is quite normal (it is about synonyms). You can just ignore or review these and make sure they are all synonyms and you don’t have two very different programs accessing the same port. Check the <code>/etc/services</code> file against the file the official assigning authority provides. Review open ports and if any, either uninstall or disable services you don’t use or want. Be careful.

Checking system file permissions:

<pre>--ALERT-- [perm023a] /bin/su is setuid to `root'</pre>

The explanation of tiger reads: The indicated file has the setuid bit set, but it should not have it. This should be changed by using <code>chmod u-s file</code> where ‘file’ is the indicated file. The system should be checked for signs of intrusion.

Restricting the su command to superuser only in Linux (http://www.debianadmin.com/howto-restrict-su-command-to-superuser-only-in-linux.html) can affect some apps and packages that link to suid rooted su binary. For example, on debian it leads to an authentication failure when ‘su’ing in console or terminal of a user. My recommendation is to ignore this and those like it. You also don’t, for example, want to remove the suid bit from <code>/usr/bin/passwd</code>, for if you do no user will be able to change password.

Everything from <code>/bin/</code>, <code>/sbin/</code>, <code>/usr/bin</code> and <code>/usr/sbin</code> should be safe, they are part of the base OS. The most common reason for any program to be setuid is to enable it to act as root (setuid root) for access to hardware, secure storage, devices …

AND any vulnerability in a program that is set up as setuid root can get an attacker root-level access to your box, by running the command as a regular user (privilege escalation). If you can avoid it (or doing so is inconvenient or opens up other holes or cans of worms and authentication failures), you don’t want things to run setuid root.

If you are mounting shares that have setuid root files on them that were created by a user on another box, they can be used to own your box. There are options to mount such shares no-setuid – so the setuid bit on files will be ignored on that filesystem.

In particular, you don’t want files that are setuid to be writable by non-root users, otherwise someone could replace them with malicious code that will then be executed as root.

If you find setuid binaries in non-binary filesystem locations (especially in for example <code>/tmp</code> or anywhere under <code>/var</code>) whose origins are unknown you may have been hacked. A setuid enabled binary for root under <code>/home/username/crack</code> smells definitely fishy.

You can find all files (in <code>/usr/local/bin</code>) with <code>S_ISUID</code> bit set with:

$ sudo find /usr/local/bin -perm +4000

And all files with <code>S_ISUID</code> and that are writable by owner with:

$ sudo find /usr/local/bin -perm -4200

Checking network configuration:

<pre>--WARN-- [lin017w] The system is not configured to log suspicious (martian) packets</pre>

Fix with:
$ sudo sysctl w net.ipv4.conf.all.log_martians=1

$ sudo sysctl w net.ipv4.conf.default.log_martians=1

Yes, yes, I had forgotten those default versions. Also corrected in sysctl (*).

== Resources ==

=== RandomSource ===
* Dual EC DRBG https://projectbullrun.org/dual-ec/

=== General ===
* Linux security http://www.linuxsecurity.com/

=== Debian ===
* Security information https://www.debian.org/security/
* Securing Debian Manual https://www.debian.org/doc/manuals/securing-debian-howto/index.en.html

=== Ubuntu ===
* Ubuntu security notices http://www.ubuntu.com/usn/
* Basic Ubuntu Security Guide, Desktop Edition https://wiki.ubuntu.com/BasicSecurity

=== Mint ===
* Forum http://forums.linuxmint.com/viewforum.php?f=143&sid=144ced36d297bed0a2330fda9abc9d40

=== Sysctl ===
* Linux Kernel /etc/sysctl.conf Security Hardening http://www.cyberciti.biz/faq/linux-kernel-etcsysctl-conf-security-hardening/
* Tuning the Kernel http://cromwell-intl.com/cybersecurity/stack-hardening.html

== Related ==
* [[Installing linux]]
* [[Linux applications]]
* [[Kinky linux command-line]]
* [[Object encryption on linux]]
* [[Anonymising your traffic with linux]]
* [[Clean up all the things]]
* [[Shell scripting]]

[[Category:How_To]]

Networking concepts

2015-10-09T19:53:04Z

Lilith2:

''Distant, abstract and idealised, principles have been drafted, ideas floated, and suggestions made about how best to use the enormous, networked tool. But the very idea of "neutrality" where "communications" and "investment" come together'' (these two not only get together in the FCC, they get together in lots of standardisation committees run by military and corporate interests), ''where information is key and the battle for access fundamental, suggest the fictional character of the effort.''

''Dandy it may be to speak about such "access" entitlements to internet power, till one realises the range of forces at work seeking to limit and restrict its operations. They come from governments and their agencies. They come from companies and their subsidiaries. The internet, in other words, is simply another territory of conflict, and one filled with fractious contenders vying for the shortest lived of primacies. Forget neutrality – it was never there to begin with. Just ask the lawyers getting their briefs ready for the next round of dragging litigation.'' ~ The FCC, the Internet and Net Neutrality <ref>The FCC, the Internet and Net Neutrality http://www.counterpunch.org/2015/03/05/the-fcc-the-internet-and-net-neutrality/</ref>
[[File:Fcc.jpg|480px|thumb|right|By John Trevor http://editorialcartoonists.com/cartoonist/profile.cfm/TreveJ/]]

Note: On this page I frequently use the word ''digital device''. A computer is a digital device. So is a router.

[[File:P2pfoundation.png|480px|thumb|right|http://p2pfoundation.net/Main_Page]]

[[File:Tor_project_banner.png|480px|thumb|right|https://www.torproject.org/]]

[[File:Take-Back-The-Tech.png|480px|thumb|right|What can you do to end (economic) violence against women? Take Back The Tech! http://filipinaimages.com/what-can-you-do-to-end-violence-against-women-take-back-the-tech/]]

[[File:Secret-tunnel.png|480px|thumb|right|https://geti2p.net/]]
== Network topology ==

A ''network'' consists of multiple digital devices connected using some type of interface, each having one or more interface devices such as a Network Interface Card (NIC) and/or a serial device for PPP networking. Each digital device is supported by network software that provides server and/or client functionality.
=== Centralised vs Distributed ===
We can make distinctions in type of networks according to centralisation vs distribution:
* In a server based network, some devices are set up to be primary providers of services. These devices are called servers and the devices that request and use the service are called clients.
* In a peer-to-peer (p2p) network, various devices on the network can act both as clients and servers. Like a network of switchers. :D

Social p2p processes are interactions with a peer-to-peer dynamic. Peers can both be a device or a human. The term comes from the P2P distributed computer application architecture which partitions tasks or workloads between peers. P2P has inspired new structures and philosophies in many areas of human interaction. Its human dynamic affords a critical look at current authoritarian and centralized social structures. Peer-to-peer is also a political and social program for those who believe that in many cases, peer-to-peer modes are a preferable option.

=== Flat vs Hierarchical ===

In general there are two fundamental design relationships that can be identified in the construction of a network infrastructure: flat networks versus hierarchical networks. In a flat network every device is directly reachable by every other device. In a hierarchical network the world is divided into separate locations and devices are assigned to a specific location. The advantage of hierarchical design is that devices interconnecting the parts of the infrastructure need only know how to reach intended destinations without having to keep track of individual devices at each location. Routers make forwarding decisions by looking at that part of the station address that identifies the location of the destination.

=== Physical wiring ===
The network topology describes the method used to do the physical wiring of the network. The main ones are:
* ''Bus'' networks (not to be confused with the system bus of a computer) use a common backbone to connect all devices. Both ends of the network must be terminated with a terminator. A barrel connector can be used to extend it. A device wanting to communicate with another device on the network sends a broadcast message onto the wire that all other devices see, but only the intended recipient actually accepts and processes the message. Bus networks are limited in the number of devices it can serve due to the broadcast traffic it generates.
* ''Ring'' networks connecting from one to another in a ring. Every device has exactly two neighbors. A data token is used to grant permission for each computer to communicate. All messages travel through a ring in the same direction, either "clockwise" or "counterclockwise". A failure in any cable or device breaks the loop and can take down the entire network, so there are also rings that have doubled up on networking hardware and information travels both "clockwise" and "counterclockwise".
* ''Star'' networks using a central connection point called a "hub node", a network ''hub'', ''switch'' or ''router'', that controls the network communications. Most home networks are of this type. Star networks are limited in number of hub connection points.
* ''Tree'' networks join multiple star topologies together onto a bus. In its simplest form, only hub devices connect directly to the tree bus, and each hub functions as the root of a tree of devices.
* ''Mesh'' networks use ''routes''. Unlike the previous topologies, messages sent on a mesh network can take any of several possible paths from source to destination. Most prominent example is the internet.

<pre> +-----+ +-----+ +-----+
+-----+ +-----+ +-----+ | |--------| | | |
| | | | | | +-----+ +-----+ +-----+
+-----+ +-----+ +-----+ / \ |
| | | +-----+ +-----+ +-----+ +-----+ +-----+
----------------------------- | | | | | |---------| |---------| |
| | | +-----+ +-----+ +-----+ +-----+ +-----+
+-----+ +-----+ +-----+ \ / / \
| | | | | | +-----+ +-----+ +-----+ +-----+
+-----+ +-----+ +-----+ | |--------| | | | | |
+-----+ +-----+ +-----+ +-----+
Bus topology Ring topology Star topology</pre>

== Hardware connections ==
=== Network interface card (NIC) ===
A Network Interface Card (NIC) is a circuit board or chip which allows the computer to communicate with other computers. This board when connected to a cable or other method of transferring data such as infrared or ISM bands can share resources, information and computer hardware. Using network cards to connect to a network allows users to share data such as collectives being able to have the capability of having a library, receive e-mail internally within the collective or share hardware devices such as printers.

Each network interface card (NIC) has a built in hardware address programmed by its manufacturer. This is a 48 bit address and should be unique for each card. This address is called a media access control (MAC) address.
=== Network cabling ===

[[File:Hub2.png|420px|thumb|right]]
You can connect two digital devices (computers) together with a ''cross-over cable'' between their network cards, not a straight network jumper cable (otherwise the transmit port would be sending to the transmit port on the other side).

Common network cable types:

* In ''Twisted Pair cables'', wire is twisted to minimize crosstalk interference. It may be shielded or unshielded.
** Unshielded Twisted Pair (UTP)
** Shielded twisted pair (STP)
* ''Coaxial cables'' are two conductors separated by insulation. Coax cable types of intrest:
** RG-58 A/U - 50 ohm, with a stranded wire core.
** RG-58 C/U - Military version of RG-58 A/U.
* With ''Fiber-optic cables'' data is transmitted using light rather than electrons. Usually there are two fibers, one for each direction. It is not subject to interference. Two types of cables are:
** Single mode cables for use with lasers.
** Multimode cables for use with Light Emitting Diode (LED) drivers.

=== Hubs and switches ===
A ''network hub'' is a hardware device to connect network devices together. The devices will all be on the same network and/or subnet. All network traffic is shared and can be sniffed by any other node connected to the same hub.

An ''uplink'' is a connection from a device or smaller local network to a larger network. ''Uplink'' does not have a crossover connection and is designed to fit into a crossover connection on the next hub. This way you can keep linking hubs to put more computers on a network. Because each hub introduces some delay onto the network signals, there is a limit to the number of hubs you can sequentially link. Also the computers that are connected to the two hubs are on the same network and can talk to each other. All network traffic including all broadcasts is passed through the hubs.

A ''network switch'' is like a hub but creates a private link between any two connected nodes when a network connection is established. This reduces the amount of network collisions and thus improves speed. Broadcast messages are still sent to all nodes.

If you have a machine (device, computer) with two network cards, eth0, connected to an outbound hub, and eth1, connected to another hub that only connects to local machines, and it is not configured as ''router'' or ''bridge'', the two networks are considered separated. If no other machines on the network that the eth0 card is connected to is an outbound device, all devices in that network are dependent.

[[File:Netconnections2.png|640px|thumb|center|Three hubs, A, B and C, with uplink connections. A can be connected to the internet. As depicted it is not, and as a result all computers are not.]]

=== Wireless media ===

Transmission of waves take place in the electromagnetic (EM) spectrum. The carrier frequency of the data is expressed in cycles per second called hertz(Hz). Low frequency signals can travel for long distances through many obstacles but can not carry a high bandwidth of data. High frequency signals can travel for shorter distances through few obstacles and carry a narrow bandwidth. Also the effect of noise on the signal is inversely proportional to the power of the radio transmitter, which is normal for all FM transmissions. The three broad categories of wireless media are:

''Radio frequency (RF)'' refers to frequencies of radio waves. RF is part of electromagnetic spectrum that ranges from 3 Hz - 300 GHz. Radio wave is radiated by an antenna and produced by alternating currents fed to the antenna. RF is used in many standard as well as proprietary wireless communication systems. RF has long been used for radio and TV broadcasting, wireless local loop, mobile communications, and amateur radio. It is broken into many bands including AM, FM, and VHF bands. The Federal communications Commission (FCC) regulates the assignment of these frequencies. Frequencies for unregulated use are:
* 902 - 928Mhz - Cordless phones, remote controls.
* 2.4 Ghz
* 5.72 - 5.85 Ghz
''Microwave'' is the upper part of RF spectrum, i.e. those frequencies above 1 GHz. Because of the availability of larger bandwidth in microwave spectrum, microwave is used in many applications such as wireless PAN (Bluetooth), wireless LAN (Wi-Fi), broadband wireless access or wireless MAN (WiMAX), wireless WAN (2G/3G cellular networks), satellite communications and radar. But it became a household name because of its use in microwave oven.
* Terrestrial - Used to link networks over long distances but the two microwave towers must have a line of sight between them. The signal is normally encrypted for privacy.
* Satellite - A satellite orbits at 22,300 miles above the earth which is an altitude that will cause it to stay in a fixed position relative to the rotation of the earth. This is called a geosynchronous orbit. A station on the ground will send and receive signals from the satellite. The signal can have propagation delays between 0.5 and 5 seconds due to the distances involved.

''Infrared'' light is part of electromagnetic spectrum that is shorter than radio waves but longer than visible light. Its frequency range is between 300 GHz and 400 THz, that corresponds to wavelength from 1mm to 750 nm. Infrared has long been used in night vision equipment and TV remote control. Infrared is also one of the physical media in the original wireless LAN standard, that's IEEE 802.11. Infrared use in communication and networking was defined by the Infrared Data Association (IrDA). Using IrDA specifications, infrared can be used in a wide range of applications, e.g. file transfer, synchronization, dial-up networking, and payment. However, IrDA is limited in range (up to about 1 meter). It also requires the communicating devices to be in LOS and within its 30-degree beam-cone. A light emitting diode (LED) or laser is used to transmit the signal. The signal cannot travel through objects. Light may interfere with the signal. Some types of infared are:
* Point to point - Transmission frequencies are 100GHz-1,000THz . Transmission is between two points and is limited to line of sight range. It is difficult to eavesdrop on the transmission.
* Broadcast - The signal is dispersed so several units may receive the signal. The unit used to disperse the signal may be reflective material or a transmitter that amplifies and retransmits the signal. Installation is easy and cost is relatively inexpensive for wireless.

=== LAN radio communications ===

* Low power, single frequency is susceptible to interference and eavesdropping.
* High power, single frequency requires FCC licensing and high power transmitters. It is susceptible to interference and eavesdropping.
* Spread spectrum uses several frequencies at the same time. Two main types are:
** In ''Direct sequence modulation'' the data is broken into parts and transmitted simultaneously on multiple frequencies. Decoy data may be transmitted for better security.
** In ''Frequency hopping'' the transmitter and receiver change predetermined frequencies at the same time (in a synchronized manner).

== TCP/IP ports and addresses ==

The part of the network that does the job of transporting and managing the data across the "normal" internet is called ''TCP/IP'' which stands for ''Transmission Control Protocol (TCP)'' and ''Internet Protocol (IP)''. The IP layer requires a 4 (''IPv4'') or 6 (''IPv6'') byte address to be assigned to each network interface card on each computer. This can be done automatically using network software such as ''Dynamic Host Configuration Protocol (DHCP)'' or by ''manually entering static addresses''.

=== Port numbers ===
The TCP layer requires what is called a ''port number'' to be assigned to each message. This way it can determine the type of service being provided. These are not ports that are used for serial and parallel devices or for computer hardware control, but reference numbers used to define a service (RFC 6335).

=== Addresses ===
Addresses are used to locate computers, almost like a house address. Each IP address is written in what is called ''dotted decimal notation''. This means there are four numbers, each separated by a dot. Each number represents a one byte value with a possible range of 0-255.

== Network protocol levels ==
Protocols are sets of standards that define all operations within a network and how devices outside the network can interact with the network. Protocols define everything: basic networking data structures, higher level application programs, services and utilities.

The ''International Standards Organization (ISO)'' has defined the ''Open Systems Interconnection (OSI)'' model for current networking protocols, commonly referred to as the ISO/OSI model (ISO standard 7498-1). It is a hierarchical structure of seven layers that defines the requirements for communications between two computers. It was conceived to allow interoperability across the various platforms offered by vendors. The model allows all network elements to operate together, regardless of who built them. By the late 1980's, ISO was recommending the implementation of the OSI model as a networking standard. By that time, TCP/IP had been in use for years. TCP/IP was fundamental to ARPANET and the other networks that evolved into the internet. For differences between TCP/IP and ARPANET, see RFC 871. Only a subset of the whole OSI model is used today.

=== RFCs ===
Protocols are outlined in Request for Comments (RFCs). The RFCs central to the TCP/IP protocol:
* RFC 1122 - Defines host requirements of the TCP/IP suite of protocols covering the link, network (IP), and transport (TCP, UDP) layers.
* RFC 1123 - The companion RFC to 1122 covering requirements for internet hosts at the application layer
* RFC 1812 - Defines requirements for internet gateways which are IPv4 routers

=== ISO/OSI model ===

[[File:Osi-model.png|404px|thumb|right]]
7. The '''Application layer''' provides a user interface by interacting with the running application. Examples of application layer protocols are ''Telnet'', ''File Transfer Protocol (FTP)'', ''Simple Mail Transfer Protocol (SMTP)'' and ''Hypertext Transfer Protocol (HTTP)''.

6. The '''Presentation layer''' transforms data it receives from and passes on to the ''Application layer'' and ''Session layer''. MIME encoding, data compression, data encryption and similar manipulations of the presentation are done at this layer. Examples: converting an EBCDIC-coded text file to an ASCII-coded file or from a .wav to .mp3 file, or serializing objects and other data structures into and out of XML. This layer makes the type of data transparent to the layers around it.

5. The '''Session layer''' establishes, manages and terminates the connections between local and remote applications. The OSI model made this layer responsible for "graceful close" of sessions (a property of TCP), and session checkpointing and recovery (usually not used in the internet protocol suite). It provides for duplex or half-duplex operation, dialog control (who transmits next), token management (who is allowed to attempt a critical action next) and establishes checkpointing of long transactions so they can continue after a crash, adjournment, termination, and restart procedures.
* ''Full Duplex'' allows the simultaneous sending and receiving of packets.
* ''Half Duplex'' allows the sending and receiving of packets in one direction at a time only.

4. The '''Transport layer''' provides end-to-end delivery of data between two nodes and is responsible for the delivery of a message from one process to another. It converts messages into Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Stream Control Transmission Protocol (SCTP), etc. Some protocols are state and connection oriented, allowing the transport layer to keep track of the packets and retransmit those that fail: It divides data into different segments before transmitting it. On receipt of these segments, the data is reassembled and forwarded to the next layer. If the data is lost in transmission or has errors, then this layer recovers the lost data and transmits the same.

3. The '''Network layer''' translates the network address into a physical MAC address. It performs network routing, flow control, segmentation/desegmentation, and error control functions. The best known example of a layer 3 protocol is the ''Internet Protocol (IP)''.

2. The '''Data Link layer''' is responsible for moving frames from one hop (node) to the next. The main function of this layer is to convert the data packets received from the upper layer(s) into frames, to establish a logical link between the nodes, and to transmit the frames sequentially. The addressing scheme is physical as MAC addresses are hard-coded into network cards at the time of their manufacture. The best known example is data transfer method (802x ethernet). IEEE divided this layer into the two following sublayers:
* ''Logical Link Control (LLC)'' maintains the link between two computers by establishing ''Service Access Points (SAPs)'' which are a series of interface points. See IEEE 802.2.
* ''Media Access Control (MAC)'' is used to coordinate the sending of data between computers. See the IEEE 802.3, 4, 5, and 12 standards.

1. The '''Physical layer''' coordinates the functions required to transmit a bit stream over a physical medium. It defines all the electrical and physical specifications for devices. This includes layout of pins, voltages, cable specifications, etc. Hubs, repeaters and network adapters are physical-layer devices. Popular protocols at this layer are Fast Ethernet, ATM, RS232, etc. The major functions and services performed by the physical layer are:
* Establishment and termination of a connection to a device.
* Participation in the process whereby resources are effectively shared among multiple users.
* Modulation, or conversion between the representation of digital data in user equipment and the corresponding signals transmitted over a channel. These are signals operating over the physical cabling or over a radio link.

=== TCP/IP model ===

4. The '''Application layer''' includes all the higher-level protocols such as TELNET, FTP, DNS SMTP, SSH, etc. The TCP/IP model has no session or presentation layer. Its functionalities are folded into its application layer, directly on top of the transport layer.

<pre>
+--------------------+
| Application data | Application packet
+--------------------+
</pre>

3. The '''Transport layer''' provides datagram services to the Application layer. This layer allows host and destination devices to communicate with each other for exchanging messages, irrespective of the underlying network type. Error control, congestion control, flow control, etc., are handled by the transport layer. The protocols used are the Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). TCP gives a reliable, end-to-end, connection-oriented data transfer, while UDP provides unreliable, connectionless data transfers.

The data necessary for these functions is added to the packet. This process of "wrapping" the application data is called data encapsulation.

<pre>
+-----------------------------------+
| TCP header | Application data | TCP packet
+-----------------------------------+
</pre>

2. The '''Internet layer''' (alias '''Network layer''') routes data to its destination. Data received by the link layer is made into data packets (IP datagrams), containing source and the destination IP address or logical address. These packets are sent and delivered independently (unordered). Protocols at this layer are Internet Protocol (IP), Internet Control Message Protocol (ICMP), etc.

<pre>
+-------------------------------------------------+
| IP header | TCP header | Application data | IP packet
+-------------------------------------------------+
</pre>

1. The '''Network Interface layer''' combines OSI's Physical and Data Link layers.

<pre>

+---------------------------------------------------------------------+
| Ethernet header | IP header | TCP header | Application data | Ethernet packet
+---------------------------------------------------------------------+
</pre>

== Data link layer ==

The IEEE 802 standards define the two lowest levels of the seven layer network model and primarily deal with the control of access to the network media <ref>IEEE 802 https://en.wikipedia.org/wiki/IEEE_802</ref>. The network media is the physical means of carrying the data such as network cable. The control of access to the media is called media access control (MAC).

=== Network access methods ===
All clients talking at once doesn't work. What ways have been developed sofar to avoid this?

* Contention
** Carrier-Sense Multiple Access with Collision Detection (CSMA/CD) used by ethernet
** Carrier-Sense Multiple Access with Collision Avoidance (CSMA/CA)
* Token Passing - A token is passed from one computer to another, which provides transmission permission.
* Demand Priority - Describes a method where intelligent hubs control data transmission. A computer will send a demand signal to the hub indicating that it wants to transmit. The hub sill respond with an acknowledgement that will allow the computer to transmit. The hub will allow computers to transmit in turn.
* Polling - A central controller, also called the primary device will poll computers, called secondary devices, to find out if they have data to transmit. Of so the central controller will allow them to transmit for a limited time, then the next device is polled.

Ethernet uses CSMA/CD, a method that allows network stations to transmit any time they want. Network stations sense the network line and detect if another station has transmitted at the same time they did. If such a collision happened, the stations involved will retransmit at a later, randomly set time in hopes of avoiding another collision.

=== Data encapsulation ===

1. One computer requests to send data to another over a network.

2. The data message flows through the ''Application layer'' by using a TCP or UDP port to pass onto the ''Internet layer''. The Transport layer may have one of two names, segment or datagram. If the TCP protocol is being used, it is called a ''segment''. If the UDP protocol is being used, it is called a ''datagram''.

3. The data segment obtains logical addressing at the ''Internet layer'' via the IP protocol, and the data is then encapsulated into a datagram. The requirements for IP to link layer encapsulation for hosts on a Ethernet network are:

* All hosts must be able to send and receive packets defined by RFC 894.
* All hosts should be able to receive a mix of packets defined by RFC 894 and RFC 1042.
* All hosts may be able to send RFC 1042 defined packets.

Hosts that support both must provide a means to configure the type of packet sent and the default must be packets defined by RFC 894.

4. The datagram enters the Network Access layer, where software will interface with the physical network. A data frame encapsulates the datagram for entry onto the physical network. At the end of the process, the frame is converted to a stream of bits that is then transmitted to the receiving computer.

5. The receiving computer removes the frame, and passes the packet onto the Internet layer. The Internet layer will then remove the header information and send the data to the Transport layer. Likewise, the Transport layer removes header information and passes data to the final layer. At this final layer the data is whole again, and can be read by the receiving computer if no errors are present.

=== Encapsulation formats ===

Ethernet (RFC 894) message format:

<pre>
+--------------------------------------------------------------------------------------------------------------------+
| preamble | destination address | source address | type | application, transport and network data | CRC | Ethernet packet
+--------------------------------------------------------------------------------------------------------------------+
</pre>
* 8 bytes for preamble
* 6 bytes for destination address
* 6 bytes for source address.
* 2 bytes of message type indicating the type of data being sent
* 46 to 1500 bytes of data. The maximum length of an Ethernet frame is 1526 bytes. This means a data field length of up to 1500 bytes.
* 4 bytes for cyclic redundancy check (CRC) information

IEEE 802 (RFC 1042) message format:
<pre>
+------------------------------------------------------------------------------------------------------------------------+
| preamble | SFD | destination address | source address | length | DSAP | SSAP | control | info | FCS | IEEE 802.3 packet
+------------------------------------------------------------------------------------------------------------------------+
</pre>
IEEE 802.3 Media Access Control section used to coordinate the sending of data between computers:
* 7 bytes for preamble
* 1 byte for the start frame delimiter (SFD)
* 6 bytes for destination address
* 6 bytes for source address
* 2 bytes for length - The number of bytes that follow not including the CRC.
Additionally, IEEE 802.2 Logical Link control establishes service access points (SAPs) between computers:
* 1 byte destination service access point (DSAP)
* 1 byte source service access point (SSAP)
* 1 byte for control
Followed by Sub Network Access Protocol (SNAP):
* 3 bytes for org code.
* 2 bytes for message type which indicates the type of data being sent
* 38 to 1492 bytes of data
* 4 bytes for cyclic redundancy check (CRC) information named frame check sequence (FCS)

=== Trailor encapsulation ===

This link layer encapsulation is described in RFC 1122 and RFC 892. It is not used very often today and may be very interesting for some further experimentation with.

== TCP/IP network protocols ==

''The Transmission Control Protocol/Internet Protocol (TCP/IP)'' uses a client - server model for communications. The protocol defines the data packets transmitted (packet header, data section), data integrity verification (error detection bytes), connection and acknowledgement protocol, and re-transmission.

''TCP/IP Time To Live (TTL)'' is a counting mechanism to determine how long a packet is valid before it reaches its destination. Each time a TCP/IP packet passes through a router it will decrement its TTL count. When the count reaches zero the packet is dropped by the router. This ensures that errant routing and looping aimless packets will not flood the network.

TDP/IP includes a wide range of protocols which are used for a variety of purposes on the network. The set of protocols that are a part of TCP/IP is called the ''TCP/IP protocol stack'' or the TCP/IP suite of protocols https://en.wikipedia.org/wiki/Internet_protocol_suite.

{| class="wikitable"
|-
! ISO/OSI !! TCP/IP !! TCP/IP protocol examples
|-
| Application, session, presentation || Application || NFS, NIS, DNS, RPC, LDAP, telnet, ftp, rlogin, rsh, rcp, RIP, RDISC, SNMP, and others
|-
| Transport || Transport || TCP, UDP, SCTP
|-
| Network || Internet || IPv4, IPv6, ARP, ICMP
|-
| Data link || Data link || PPP, IEEE 802.2, HDLC, DSL, Frames, Network Switching, MAC address
|-
| Physical || Physical network || Ethernet (IEEE 802.3), Token Ring, RS-232, FDDI, and others
|}

=== Application protocols ===
FTP, TFTP, SMTP, Telnet, NFS, ping, rlogin provide direct services to the user. DNS provides address to name translation for locations and network cards. RPC allows remote computer to perform functions on other computers. RARP, BOOTP, DHCP, IGMP, SNMP,RIP, OSPF, BGP, and CIDR enhance network management and increase functionality.
* ''Hypertext Transfer Protocol (HTTP)'' is the protocol that facilitates transfer of data. Typically, data is transferred in the form of pages, or HTML markup. HTTP operates on TCP port 80.
* ''Secure HTTP (HTTPS)'' uses TCP port 443 to securely transfer HTTP data via SSL, or Secure Socket Layer. [[Digital_threats,_detection,_protection_and_(counter)_moves#TLS.2FSSL|TLS]] is the newer SSL.
* ''File Transfer Protocol (FTP)'' operates on TCP ports 20 (data)/21(transmission control). It is used in simple file transfers from one node to another without any security (transferred in cleartext). Secure (SFTP) is a version of FTP that uses SSH to transfer data securely, using whichever port SSH uses (usually 22).
* ''Trivial FTP (TFTP)'' is a UDP version of FTP that uses UDP port 69. It is called "trivial" because it is relatively unreliable and inefficient and so is more often used for inter-network communication between routers.
* ''Telnet (Telecommunications Network)'' is an old protocol used to remotely connect to a node. All communications with telnet are in cleartext (even passwords for authentication). Telnet operates on TCP port 23. Except for in lab situations, no longer in use.
* ''Secure Shell (SSH)'' is a secure replacement of Telnet. SSH allows terminal emulation in cipher text, which equates to enhanced and increased security. SSH usually operates on TCP port 22.
* ''Network News Transfer Protocol (NNTP)'' is a protocol used by client and server software to carry USENET (newsgroup) postings back and forth over a TCP/IP network. NNTP operates on TCP port 119.
* ''Lightweight Directory Access Protocol (LDAP)'' is a "Directory Services" protocol allowing a server to act as a central directory for client nodes. LDAP operates on TCP and UDP port 389.
* ''Network Time Protocol (NTP)'' allows for synchronizing network time with a server. NTP operates on UDP port 123.
* ''Post Office Protocol (POP3)'' is the mailbox protocol allowing users to download mail from a mail server. Once you access it, your client software will download all of your incoming mail and wipe it from the server. POP3 operates on TCP port 110.
* The ''Internet Message Access Protocol (IMAP)'' allows for server-based repositories of sent mail and other specialized folders. When using IMAP4 instead of POP3 as your incoming mail protocol, you download very minimal information to your local machine and when you want to access actual incoming mail, you are pulling this directly from the mail server. This allows you to access your mail from virtually anywhere. IMAP4 operates on TCP port 143.
* ''Simple Mail Transfer Protocol (SMTP)'' used in conjunction with POP3 or IMAP4 allows for sending/receiving of email. Without it you will only be able to receive mail. SMTP operates on TCP port 25.
* ''Domain Name System (DNS)'' resolves easy to read domain names into computer readable IP addresses and operates on UDP port 53.
* ''Simple Network Management Protocol (SNMP)'' manages devices on IP networks, such as modems, switches, routers, or printers. Default works on UDP port 161.

=== Transport protocols ===

Controls the management of service between computers. Based on values in TCP and UDP messages a server knows what service is being requested.
* ''Transmission Control Protocol (TCP)'' is a reliable connection oriented protocol used to control the management of application level services between computers.
* ''User Datagram Protocol (UDP)'' is an unreliable connection less messaging protocol used to control the management of application level services between computers.
* The ''Stream Control Transmission Protocol (SCTP)'' is message-oriented like UDP and ensures reliable, in-sequence transport of messages with congestion control like TCP. In the absence of native SCTP support in operating systems it is possible to tunnel SCTP over UDP, as well as mapping TCP API calls to SCTP calls.

=== Internet protocols ===
ARP communicates between layers to allow one layer to get information to support another layer. This includes broadcasting. IP and ICMP manage movement of messages and report errors (including routing).
* ''Internet Protocol (IP)'' provides the mechanism to use software to address and manage data packets being sent to computers. Except for ARP and RARP all protocols' data packets will be packaged into an IP data packet.
* ''Address resolution protocol (ARP)'' enables packaging of IP data into ethernet packages. It is the system and messaging protocol that is used to find the ethernet (hardware) address from a specific IP number. Without it, the ethernet package can not be generated from the IP package, because the ethernet address can not be determined.
* ''Internet Control Message Protocol (ICMP)'' provides management and error reporting to help manage the process of sending data between computers.

=== Network interface protocols ===
Allows messages to be packaged and sent between physical locations.

* ''Serial line IP (SLIP)'' is a form of data encapsulation for serial lines.
* ''Point to point protocol (PPP)'' is a form of serial line data encapsulation that is an improvement over SLIP.
* ''Ethernet'' provides transport of information between physical locations on ethernet cable. Data is passed in ethernet packets.

== Network devices ==

=== Repeater ===

As signals travel along a network cable (or any other medium of transmission), they degrade and become distorted in a process that is called attenuation. If a cable is long enough, the attenuation will finally make a signal unrecognizable by the receiver. A repeater retimes and regenerates the signals to proper amplitudes and sends them to the other segments, enabling signals to travel longer distances over a network.

To pass data through the repeater in a usable fashion from one segment to the next, the packets and the Logical Link Control (LLC) protocols must be the same on the each segment. This means that a repeater will not enable communication, for example, between an 802.3 segment (Ethernet) and an 802.5 segment (Token Ring). Repeaters do not translate anything.

=== Bridge ===

Bridges work at the Data Link layer. This means that all information contained in the higher levels of the OSI model is unavailable to them, including IP addresses. Bridges read the outermost section of data on the data packet to tell where a message is going.

Bridges do not distinguish between one protocol and another and simply pass all protocols along the network. Because all protocols pass across the bridges, it is up to the individual computers to determine which protocols they can recognise. As traffic passes through the bridge, information about the computer addresses is then stored in the bridge's RAM. The bridge will then use this RAM to build a routing table based on source (MAC) addresses. To determine the network segment a MAC address belongs to, bridges use one of:

* In Transparent Bridging a table of addresses (bridging table) is built as they receive packets. If the address is not in the bridging table, the packet is forwarded to all segments other than the one it came from. This type of bridge is used on ethernet networks.
* In Source Route Bridging the source computer provides path information inside the packet. This is used on Token Rings.

Bridges can be used to:
* Expand the distance of a segment.
* Provide for an increased number of computers on the network.
* Reduce traffic bottlenecks resulting from an excessive number of attached computers.

=== Router ===

In an environment consisting of several network segments with different protocols and architecture, a bridge may not be adequate for ensuring fast communication among all of the segments. A complex network needs a device, which not only knows the address of each segment, but also can determine the best path for sending data and filtering broadcast traffic to the local segment. Such device is called a router. Routers work at the Network layer of the OSI model meaning that the Routers can switch and route packets across multiple networks.

A router is used to route data packets between two networks. It reads the information in each packet to tell where it is going. If it is destined for an immediate network it has access to, it will strip the outer packet, readdress the packet to the proper ethernet address, and transmit it on that network. If it is destined for another network and must be sent to another router, it will re-package the outer packet to be received by the next router and send it to the next router.

=== Gateway ===

Gateways make communication possible between different architectures and environments. They repackage and convert data going from one environment to another so that each environment can understand the other's environment data. Most gateways operate at the application layer, but can operate at the network or session layer of the OSI model.

Gateways strip information until getting to the required level, repackages the information to match the requirements of the destination system, and works its way back toward the hardware layer of the OSI model: It decapsulates incoming data through the networks complete protocol stack and encapsulates the outgoing data in the complete protocol stack of the other network to allow transmission. A gateway links two systems that do not use the same:

* Communication protocols
* Data formatting structures
* Languages
* Architecture

== ARP and RARP address translation ==

Address resolution refers to the determination of the address of a device from the address of that equipment to another protocol level, for example, an IP address in an Ethernet address.

The ''Address Resolution Protocol (ARP)'' provides a completely different function to the network than ''Reverse Address Resolution Protocol (RARP)''. ARP is used to resolve the ethernet address of a NIC from an IP address in order to construct an ethernet packet around an IP data packet. This must happen in order to send any data across the network. ''Reverse Address Resolution protocol (RARP)'' is used for diskless computers to determine their IP address using the network.

In IPv6, ARP and RARP are replaced by a neighbor discovery protocol called Neighbor Discovery (ND), which is a subset of the control protocol Internet Control Message Protocol (ICMP).

=== ARP ===
''A Media Access Control address (MAC Address)'' is the network card address used for communication between other network devices on the subnet. This information is not routable. The ARP table maps a (global internet) TCP/IP address to the local hardware on the local network. The MAC address uniquely identifies each node of a network and is used by the Ethernet protocol.

To determine a recipient's physical address, a device broadcasts an ARP request on the subnet that contains the IP address to be translated. The machine with the relevant IP address responds with its physical address.

To make ARP more efficient, each machine maintains in memory a table of addresses resolved and thus reduces the number of ''Broadcast'' emissions.

=== RARP ===

The RARP mechanism allows a device to be identified as a target on the network by broadcasting a RARP request. The servers receiving the message examine their table and meet. Once the IP address obtained, the machine stores it in memory and no longer uses RARP until it is reset.

== Network address translation (NAT) ==

IP-Masquerading translates internal IP addresses into external IP addresses. This is called Network Address Translation (NAT). From the outside world, all connections will seem to be originating from the one external address.

=== One to one NAT ===
1:1 NAT (Network Address Translation) is a mode of NAT that maps one internal address to one external address. For example, if a network has an internal server at 192.168.1.10, 1:1 NAT can map 192.168.1.10 to 1.2.3.4 where 1.2.3.4 is an additional external IP address provided by an ''internet service provider (ISP)''.

=== One to many NAT ===

The majority of NATs map multiple private hosts to one publicly exposed IP address. In a typical configuration, a local network uses one of the designated "private" IP address subnets (RFC 1918). A router on that network has a private address in that address space. The router is also connected to the internet with a "public" IP address assigned by an ''internet service provider (ISP)''. As traffic passes from the local network to the internet, the source address in each packet is translated on the fly from a private address to the public address. The router tracks basic data about each active connection (particularly the destination address and port). When a reply returns to the router, it uses the connection tracking data it stored during the outbound phase to determine the private address on the internal network to which to forward the reply.

=== Static NAT ===
Most NAT devices allow for configuring static translation table entries for connections from the external network to the internal masqueraded network. This feature is often referred to as static NAT. It may be implemented in two types: port forwarding which forwards traffic to a specific external port to an internal host on a specified port, and designation of a DMZ host which receives all traffic received on the external interface on any port number to an internal IP address, preserving the destination port. Both types may be available in the same NAT device.

== Basic TCP/IP addressing ==

An ''IP Address'' is a logical numeric address that is assigned to every single computer, printer, switch, router or any other device that is part of a TCP/IP-based network.

Until the introduction of ''Classless Inter-Domain Routing (CIDR)'' in 1993 to slow the growth of routing tables on routers across the internet, and to help slow the rapid exhaustion of IPv4 addresses, ''classful networks'' were used. You can still find it in tutorials, some networks, and in archeological artifacts such as ''default subnet mask''. In classful adresses, the first one or two bytes (depending on the class of network), generally will indicate the number of the network, the third byte indicates the number of the subnet, and the fourth number indicates the host number.

Most of servers and personal computers use ''Internet Protocol version 4 (IPv4)''. This uses 32 bits to assign a network address as defined by the four octets of an IP address, up to <code>255.255.255.255</code>. Each octet is converted to a decimal number (base 10) from 0–255 and separated by a period (a dot). This format is called dotted decimal notation. If not familiar with number conversions, a decent tutorial can be found in http://www.cstutoringcenter.com/tutorials/general/convert.php

For example the IPv4 address:

<code>11000000101010000000001100011000</code>

is segmented into 8-bit blocks:

<code>11000000</code> | <code>10101000</code> | <code>00000011</code> | <code>00011000</code>

Each block is converted to decimal:

<code>27 + 26</code> | <code>27 + 25 + 23</code> | <code>21 + 20</code> | <code>24 + 23</code>

<code>128 + 64 = 192</code> | <code>128 + 32 + 8 = 168</code> | <code>2 + 1 = 3</code> | <code>16 + 8 = 24</code>

The adjacent octets 192, 168, 3 and 24 are separated by a period:
<code>192.168.3.24</code>

''Internet Protocol version 6 (IPv6)'' was designed to answer the future exhaustion of the IPv4 address pool. IPv4 address space is 32 bits which translates to just above 4 billion addresses. IPv6 address space is 128 bits translating to billions and billions of potential addresses. The protocol has also been upgraded to include new quality of service features and security, but also has its vulnerabilities <ref>Routing Loop Attack using IPv6 Automatic Tunnels: Problem Statement and Proposed Mitigations http://tools.ietf.org/html/draft-ietf-v6ops-tunnel-loops-07</ref> <ref>When moving to IPv6, beware the risks http://gcn.com/articles/2013/03/20/risks-moving-to-ipv6.aspx</ref>. IPv6 addresses are represented as eight groups of four hexadecimal digits with the groups being separated by colons, for example 2805:F298:0004:0148:0000:0000:0740:F5E9, but methods to abbreviate this full notation exist http://www.vorteg.info/ipv6-abbreviation-rules/.

=== Routing tables ===

To minimise unnecessary traffic load and provide efficient movement of frames from one location to another, interconnected hosts are grouped into separate networks. As a result of this grouping (determined by network design and administration functions) it is possible for a router to determine the best path between two networks. A router forms the boundary between one network and another network. When a frame crosses a router it is in a different network. A frame that travels from source to destination without crossing a router has remained in the same network. A network is a group of communicating machines bounded by routers.

The router will use some of the bits in the IP address to identify the network location to which the frame is destined. The remaining bits in the address will uniquely identify the host on that network that will ultimately receive the frame. There are bits to identify the network and to identify the host. The sender of a frame must make this differentiation because it must decide whether it is on the same network as the destination or on a different network
* If the sender is on the same network as the destination, it will determine the data link address of the destination machine. Then it will send the frame directly to the destination machine.
* If the destination is on a different then the originator must send the frame to a router and let the next router in line forward the frame on to the ultimate destination network. At the ultimate destination network the last router must determine the data link address of the host and forward the frame directly to that host on that ultimate destination network.

When a router receives an incoming data frame, it masks the destination address to create a lookup key that is compared to the entries in its routing table. The routing table indicates how the frame should be processed. The frame might be delivered directly on a particular port on the router. The frame might have to be sent on to the next router in line for ultimate delivery to some remote network.

The routing table is created by the combination of direct configuration by an administrator or dynamically through the periodic broadcasting of router update frames. Protocol frames from ''Routing Information Protocol (RIP)'', ''Open Shortest Path First (OSPF)'' and ''Internet Gateway Routing Protocol (IGRP)'' are sent from all routers at periodic intervals. As a result, all routers become aware of how to reach all other networks.

The specific behavior that is expected from an IP router is discussed in RFC 1812, a TL;DR (lengthy, like this page) document providing a complete discussion of routing in the IPv4 network environment.

=== Subnet masking ===

The IP Address Mask is a configuration parameter used by a TCP/IP end-node and IP router to differentiate between that part of the IP address that represents the network and the part that represents the host.

A router uses the mask value to create a key value that is looked up in the router table to determine where to forward a frame. An end-node uses the mask value to create the same key value but the value is used to compare the destination address with the end-node address to determine whether the destination is directly reachable (on the same network) or remote (in which case the frame must be sent to a router and can not be sent directly to the destination).

The mask value can be assigned by default or it can be specified by the installer of the end-node or router software. The destination IP address and the mask value are combined with a Boolean AND operation to produce the resultant key value. Just in case, for a start in boolean algebra, see http://www.i-programmer.info/babbages-bag/235-logic-logic-everything-is-logic.html

For example:

An end-node is assigned the IP address 164.25.74.131 and a mask value of 255.255.0.0. This end-node wants to send a frame to 164.7.9.2.
* If 164.7.9.2 is on the same network as 164.25.74.131 then the end-node will broadcast an ARP (Address Resolution Protocol) frame to determine the data link address of the destination and it will then send the frame directly to the destination.
* If 164.7.9.2 is on a different network then the workstation must send the frame to a router for forwarding to the ultimate destination network.

All the dotted-decimal notation must be converted to the underlying 32-bit binary numbers to understand what is taking place:
<pre>
End node 164.25.74.131 = 10100100 00011001 01001010 10000011
Mask 255.255.0.0 = 11111111 11111111 00000000 00000000
Destination 164.7.9.2 = 10100100 00000111 00001001 00000010
</pre>
When the IP address of the end node is AND'ed with the mask we get:
<pre>
End node 164.25.74.131 = 10100100 00011001 01001010 10000011
Mask 255.255.0.0 = 11111111 11111111 00000000 00000000
-------------------------------------------------------------------------------------
Boolean AND = 10100100 00011001 00000000 00000000
In dotted-decimal = 164. 25. 0. 0
</pre>
When the destination IP address is AND'ed with the mask we get:
<pre>
Destination 164.7.9.2 = 10001100 00000111 00001001 00000010
Mask 255.255.0.0 = 11111111 11111111 00000000 00000000
-------------------------------------------------------------------------------------
Boolean AND = 10001100 00000111 00000000 00000000
(In dotted-decimal) = 164. 7. 0. 0
</pre>
Since the results (164.25.0.0 and 164.7.0.0) are not equal, the end node concludes that the destination must be on a different network and the frame is sent to a router. A router masks the destination address in an incoming frame and the result is used as a lookup key in the routing table.

=== Address classes ===

The origins of the current implementation of the Internet Protocol (IPv4) and its associated classes of IP addressing can be found in RFC 791: IP addresses were to be of fixed, 32-bit (4 octets) length comprised of a Network Number and a Local Address or Host Number. The resulting range of addresses were then divided into three broad groupings or “Classes”, based on the bit values within the first octet:

* Class A: high order bit is “0”, the remaining 7 bits are the network, and the last 24 bits are the host.
* Class B: high order two bits are “10”, the remaining 14 bits are the network, and the last 16 bits are the host.
* Class C: high order three bits are “110”, the remaining 21 bits are the network, and the last 8 bits are the host.

Two additional classes of IPv4 addressing, Class D & E that were specified in subsequent RFC’s:

* Class D: high order four bits are “1110”, the remaining 20 bits identify the multicast group.
* Class E: high order five bits are “11110”, the remaining bits are reserved for experimental use.

Implied within RFC 791, was the concept of ''Masking'', to be used by routers and hosts:

* Class A mask = 255.0.0.0
* Class B mask = 255.255.0.0
* Class C mask = 255.255.255.0

These masks were applied by default based on the value of the leading bits in the IP address. If an address started with a binary 0, then stations assumed Class A masking. The starting bits 10 indicated Class B, and 110 indicated Class C. Consequently, the class of addressing masking being used could be determined by looking at the first octet in the address:

* Class A starts with 0 and ends with 0111 1111 (the smallest value in the first octet is decimal 0 and the largest value is 127 yielding a potential range of 0-127).
* Class B starts with 10 and ends with 1011 1111 (the smallest value in the first octet is decimal 128 and the largest value is 191 yielding a potential range of 128-191).
* Class C starts with 110 and ends with 1101 1111 (the smallest value in the first octet is decimal 192 and the largest value is 223 yielding a potential range of 192-223).
* Class D starts with 1110 and ends with 1110 1111(the smallest value in the first octet is decimal 224 and the largest value is 239 yielding a potential range of 224-239).
* Class E starts with 1111 and ends with 1111 1111 (the smallest value in the first octet is decimal 240 and the largest value is 255 yielding a potential range of 240-255).

This division of addressing allows for the following potential number of addresses.

For example: Class A has 8 bits in the ''Network part'' and 24 bits in the ''Host part'', meaning 28 = 128 possible values in the Network part and 224 = 16777216 possible values in the Host part.

=== Creating subnets ===
To ''subnet'' a network is to create logical divisions of the network, for example arranged on one floor, building or geographical location. Each device on each subnet is to have an address that logically associates it with the others on the same subnet. This also prevents devices on one subnet from getting confused with hosts on another subnet. Subnetting applies to IP addresses because this is done by borrowing bits from the host portion of the IP address. In a sense, the IP address has three components - the network part, the subnet part and the host part. We can create a subnet by logically grabbing the last bit from the network component of the address and using it to determine the number of subnets required.

To make learning subnetting easier see http://www.subnetting.net/Tutorial.aspx (builds up from no knowledge) and http://www.9tut.com/subnetting-tutorial (starts from knowledge about adressess).

Also, http://www.subnet-calculator.com/ and http://www.subnetmask.info/ and these mental shortcuts:

{|
|-
! Mask !! # of subnets !! SlashFmt !! Class A hosts !! Class A mask !! Class B hosts !! Class B mask !! Class C hosts !! Class C mask !! Class C sub hosts !! Class C sub mask
|-
| 255 || 1 or 256 || /32 || 16,777,214 || 255.0.0.0 || 65,534 || 255.255.0.0 || 254 || 255.255.255.0 || Invalid, 1 address || 255.255.255.255
|-
| 254 || 128 || /31 || 33,554,430 || 254.0.0.0 || 131,070 || 255.254.0.0 || 510 || 255.255.254.0 || Invalid, 2 addresses || 255.255.255.254
|-
| 252 || 64 || /30 || 67,108,862 || 252.0.0.0 || 262,142 || 255.252.0.0 || 1,022 || 255.255.252.0 || 2 hosts, 4 addresses || 255.255.255.252
|-
| 248 || 32 || /29 || 134,217,726 || 248.0.0.0 || 524,286 || 255.248.0.0 || 2,046 || 255.255.248.0 || 6 hosts, 8 addresses || 255.255.255.248
|-
| 240 || 16 || /28 || 268,435,454 || 240.0.0.0 || 1,048,574 || 255.240.0.0 || 4,094 || 255.255.240.0 || 14 hosts, 16 addresses || 255.255.255.240
|-
| 224 || 8 || /27 || 536,870,910 || 224.0.0.0 || 2,097,150 || 255.224.0.0 || 8,190 || 255.255.224.0 || 30 hosts, 32 addresses || 255.255.255.224
|-
| 192 || 4 || /26 || 1,073,741,822 || 192.0.0.0 || 4,194,302 || 255.192.0.0 || 16,382 || 255.255.192.0 || 62 hosts, 64 addresses || 255.255.255.192
|-
| 128 || 2 || /25 || 2,147,483,646 || 128.0.0.0 || 8,388,606 || 255.128.0.0 || 32,766 || 255.255.128.0 || 126 hosts, 128 addresses || 255.255.255.128
|}

== Internet Protocol (IP) ==

Internet Protocol (IP) provides support at the network layer of the OSI model. All transport protocol data packets such as UDP or TCP packets are encapsulated in IP data packets to be carried from one host to another. IP is a connection-less unreliable service, meaning there is no guarantee that the data will reach the intended host. The datagrams may be damaged upon arrival, out of order, or not arrive at all. IP is defined by RFC 791. Therefore the layers above IP such as TCP are responsible for being sure correct data is delivered. IP provides for:

* Addressing
* Type of service specification
* Fragmentation and re-assembly
* Security

=== IP packet format ===
<pre>
0 4 8 16 31
+--------------------------------------------------------------------------------------+
| Version | Length | Type of Service | Total Length |
|------------------------------------------|-------------------------------------------|
| Identification | Flags | Fragmentation Offset |
|------------------------------------------|-------------------------------------------|
| Time to Live | Protocol | Header Checksum |
|--------------------------------------------------------------------------------------|
| Source Address |
|--------------------------------------------------------------------------------------|
| Destination Address |
|--------------------------------------------------------------------------------------|
| Options |
|--------------------------------------------------------------------------------------|
| Data |
+--------------------------------------------------------------------------------------+
</pre>

* Version (4 bits): The IP protocol version, currently 4 or 6.
* Header length (4 bits): The number of 32 bit words in the header
* Type of service (TOS) (8 bits): Only 4 bits are used which are minimize delay, maximize throughput, maximize reliability, and minimize monetary cost. Only one of these bits can be on. If all bits are off, the service is normal. Some networks allow a set precedences to control priority of messages the bits are as follows:
** Bits 0-2 - Precedence:
*** 111 - Network Control
*** 110 - Internetwork Control
*** 101 - CRITIC/ECP
*** 100 - Flash Override
*** 011 - Flash
*** 010 - Immediate
*** 001 - Priority
*** 000 - Routine
** Bit 3 - A value of 0 means normal delay. A value of 1 means low delay.
** Bit 4 - Sets throughput. A value of 0 means normal and a 1 means high throughput.
** Bit 5 - A value of 0 means normal reliability and a 1 means high reliability.
** Bit 6-7 are reserved for future use.
* Total length of the IP data message in bytes (16 bits)
* Identification (16 bits) - Uniquely identifies each datagram. This is used to re-assemble the datagram. Each fragment of the datagram contains this same unique number.
* Flags (3 bits): One bit is the more fragments bit
** Bit 0 - reserved.
** Bit 1 - The fragment bit. A value of 0 means the packet may be fragmented while a 1 means it cannot be fragmented. If this value is set and the packet needs further fragmentation, an ICMP error message is generated.
** Bit 2 - This value is set on all fragments except the last one since a value of 0 means this is the last fragment.
* Fragment offset (13 bits): The offset in 8 byte units of this fragment from the beginning of the original datagram.
* Time to live (TTL) (8 bits): Limits the number of routers the datagram can pass through. Usually set to 32 or 64. Every time the datagram passes through a router this value is decremented by a value of one or more. This is to keep the datagram from circulating in an infinite loop forever.
* Protocol (8 bits): Identifies which protocol is encapsulated in the next data area. This is may be one or more of TCP(6), UDP(17), ICMP(1), IGMP(2), or OSPF(89). A list of these protocols and their associated numbers may be found in the /etc/protocols file on Unix or Linux systems.
* Header checksum (16 bits): For the IP header, not including the options and data.
* Source IP address (32 bits): The IP address of the card sending the data.
* Destination IP address (32 bits): The IP address of the network card the data is intended for.
* Options:
** Security and handling restrictions
** Record route - Each router records its IP address
** Time stamp - Each router records its IP address and time
** Loose source routing - Specifies a set of IP addresses the datagram must go through.
** Strict source routing - The datagram can go through only the IP addresses specified.
* Data: Encapsulated hardware data such as ethernet data.
The message order of bits transmitted is 0-7, then 8-15, in network byte order. Fragmentation is handled at the IP network layer and the messages are reassembled when they reach their final destination. If one fragment of a datagram is lost, the entire datagram must be retransmitted. This is why fragmentation is avoided by TCP. The data on the last line is ethernet data, or data depending on the type of physical network.

=== Type of service specification ===

RFC 791 defined a field within the IP header called the ''Type Of Service (TOS)'' byte. This field is used to specify the quality of service desired for the datagram and is a mix of factors. These factors include fields such as Precedence, Speed, Throughput and Reliability. In normal conversations you would not use any such special alternatives, so the Type of Service byte typically would be set to zero. With the advent of multimedia transmission and emergence of protocols such as ''Session Initiation Protocol (SIP)'', this field is coming into use.

The IP Type of Service Byte:
<pre>
0 1 2 3 4 5 6 7
+------------------------------------------------+
| Precedence | D | T | R | 0 | 0 |
+------------------------------------------------+
</pre>
* Bits 0-2: Precedence.
* Bit 3: Delay (0 = Normal Delay, 1 = Low Delay)
* Bit 4: Throughput (0 = Normal Throughput, 1 = High Throughput)
* Bit 5: Reliability (0 = Normal Reliability, 1 = High Reliability)
* Bits 6-7: Reserved for Future Use.

The three bit Precedence field is defined as follows:

{| class="wikitable"
|-
! Precedence bits !! Definition
|-
| 111 || Network Control
|-
| 110 || Internetwork Control
|-
| 101 || CRITIC/ECP
|-
| 100 || Flash Overrride
|-
| 011 || Flash
|-
| 010 || Immediate
|-
| 001 || Priority
|-
| 000 || Routine
|}

'''Session Initiation Protocol (SIP)'''

This protocol is an application-layer control protocol used for creating, modifying and terminating sessions with one or more participants. Some examples of such activities include Internet multimedia conferences, Internet telephone calls and multimedia distribution. SIP is intended to support communications using ''Multicast'', a ''mesh of Unicast relations'', or a combination of both.

=== Fragmentation and reassembly ===

There are a number of deferring network transmission architectures, with each having a physical limit of the number of data bytes that may be contained within a given frame. This physical limit is described in numerous specifications and is referred to as the ''Maximum Transmission Unit (MTU)'' of the network. As a block of data is prepared for transmission, the sending or forwarding device examines the MTU for the network the data is to be sent or forwarded across. If the size of the block of data is less then the MTU for that Network, the data is transmitted in accordance with the rules for that particular network.

There are two situations in which MTU becomes important:
* The size of the block of data being transmitted is greater than the MTU.
* Data must traverse across multiple network architectures, each with a different MTU.

'''IPv4 Fragmentation Fields'''

The three fields concerned with ''IP Fragmentation'' are:

{| class="wikitable"
|-
! Field Name !! Function !! Offset Location !! Alias
|-
| Identification || 16-bit field containing a unique number used to identify the frame and any associated fragments for reassembly. || 18-19 || Identification Field
|-
| Flags || 3-bit field containing the flags that specify the function of the frame in terms of whether fragmentation has been employed, additional fragments are coming, or this is the final fragment. || 20 ||Fragmentation Flags
|-
| Fragmentation Offset || 13-bit field indicating the position of a particular fragment's data in relation to the first byte of data (offset 0). || 20-21 || Fragment Offset
|}

'''Identification'''

With increasing interconnection and complexity of networks, fragments from multiple blocks of data might travel along different paths to the destination, possibly arriving out of sequence in relation to one another. That is, a fragment from block number one might arrive intermixed with the data stream for block number 2 or vice versa. The function of the Fragment Offset Field is to identify the relative position of each fragment, and it is the Identification Field that serves to allow the receiving device to sort out which fragments comprise what block of data. Each fragment from a particular data stream will have the same Identification Field, uniquely identifying which block it belongs to. If one or more fragments are lost, the buffer of the device performing the reassembly process will time out and discard all of the fragments. In the event of such a time out, the data will then have to be retransmitted by the sending device.

'''Flags'''

{| class="wikitable"
|-
! Bit Indicator !! Definition
|-
| 0xx || Reserved
|-
| x0x || May fragment
|-
| x1x || Do not fragment
|-
| xx0 || Last fragment
|-
| xx1 || More fragments
|}

When a receiving station processes each frame, one of the operations it performs is to review the Flags field. Depending on the value indicated by this field, several possible actions are then initiated, including:

* (xx1) More Fragments - Indicates that there are additional IP Fragments that comprise the data associated with that specific Identification Field. The receiving device will allocate buffer resources for reassembly and pass all frames containing that unique Identification Field to the buffer.
* (xx0) Last Fragment - Indicates that this fragment is the final frame for the data block identified by the Identification Field. The receiving device will now attempt to reassemble the fragments in the order specified by the Fragment Offset field.

'''Fragment Offset'''

Because it is possible that the fragments that comprise a block of data might travel along different paths to the destination, it is possible they might arrive out of sequence. While the Identification Field serve to mark which IP fragments belong to which block of data, it is the Fragment Offset Field, sometimes referred to as the Fragmentation Offset Field, that tells the receiving device which order to reassemble them in.

During the IP Fragmentation Reassembly process, if a particular fragment is found to be missing, as indicated by the Fragmentation Offset count, the buffer will enter a wait state until either the missing piece(s) are received or a time out occurs. In the event of such a time out, the buffer simply discards the fragments.

'''IP Fragmentation'''

Regardless of what situation occurs that requires IP Fragmentation, the procedure followed by the device performing the fragmentation must be as follows:
# The device attempting to transmit the block of data will first examine the Flag field to see if the field is set to the value of (x0x or x1x). If the value is equal to (x1x) this indicates that the data may not be fragmented, forcing the transmitting device to discard that data. Depending on the specific configuration of the device, an Internet Control Message Protocol (ICMP) Destination Unreachable -> Fragmentation required and Do Not Fragment Bit Set message may be generated.
# Assuming the flag field is set to (x0x), the device computes the number of fragments required to transmit the amount of data in by dividing the amount of data by the MTU. This will result in "X" number of frames with all but the final frame being equal to the MTU for that network.
# It will then create the required number of IP packets and copies the IP header into each of these packets so that each packet will have the same identifying information, including the Identification Field.
# The Flag field in the first packet, and all subsequent packets except the final packet, will be set to "More Fragments." The final packets Flag Field will instead be set to "Last Fragment."
# The Fragment Offset will be set for each packet to record the relative position of the data contained within that packet.
# The packets will then be transmitted according to the rules for that network architecture.

'''IP Fragment Reassembly'''

If a receiving device detects that IP Fragmentation has been employed, the procedure followed by the device performing the Reassembly must be as follows:

# The device receiving the data detects the Flag Field set to "More Fragments".
# It will then examine all incoming packets for the same Identification number contained in the packet.
# It will store all of these identified fragments in a buffer in the sequence specified by the Fragment Offset Field.
# Once the final fragment, as indicated by the Flag Field, is set to "Last Fragment," the device will attempt to reassemble that data in offset order.
# If reassembly is successful, the packet is then sent to the ULP in accordance with the rules for that device.
# If reassembly is unsuccessful, perhaps due to one or more lost fragments, the device will eventually time out and all of the fragments will be discarded.
# The transmitting device will than have to attempt to retransmit the data in accordance with its own procedures.

'''Across networks'''

Imagine a block of data originating on a 16Mb Token Ring network (MTU = 17914B) that is connected to another 16Mb Token Ring network (MTU = 17914B) via an Ethernet network (MTU = 1500B). The data block met the MTU restriction for a 16Mb Token Ring Network, but the Router connecting the Token Ring to the Ethernet Network is faced with having to forward this large block onto a network with a smaller MTU. It will simply follow the rules for IP Fragmentation as if was transmitting the frame itself except that the Identification Field will be that of the original frame.

Once the data reaches the router on the other end of the Ethernet network, it will perform reassembly of the fragments exactly as previously described and pass the reassembled block of data onto the network with the new MTU.

=== Security ===

IP is responsible for the transmission of packets between network end points. IP includes some features which provide basic measures of fault-tolerance (time to live, checksum), traffic prioritization (type of service) and support for the fragmentation of larger packets into multiple smaller packets (ID field, fragment offset). The support for fragmentation of larger packets provides a protocol allowing routers to fragment a packet into smaller packets when the original packet is too large for the supporting datalink frames. ''IP fragmentation exploits'' (attacks) use the fragmentation protocol within IP as an attack vector <ref>IP fragmentation attack https://en.wikipedia.org/wiki/IP_fragmentation_attack</ref><ref>Rose Fragmentation Attack explained http://www.digital.net/~gandalf/Rose_Frag_Attack_Explained.htm</ref>.

The IPv4 ''Fragmentation and Reassembly'' can also be used to trigger a Denial of Service Attack (DOS). The receiving device will attempt reassembly following receipt of a frame containing a Flag field set to (xx1), indicating more fragments are to follow. Receipt of such a frame causes the receiving device to allocate buffer resources for reassembly. If a device is flooded with separate frames, each with the Flag field set to (xx1), but each has the Identification Field set to a different value, the device would attempt to allocate resources to each separate fragment in preparation for reassembly and would quickly exhaust its available resources while waiting for buffer time-outs to occur. To defend against such DOS attempts, many network security features include specific rules implemented at the Firewall that change the time-out value for how long they will hold incoming fragments before discarding them.

== Transmission Control Protocol (TCP) ==

Transmission Control Protocol (TCP) supports the network at the transport layer. Transmission Control Protocol (TCP) provides a reliable connection oriented service. Connection oriented means both the client and server must open the connection before data is sent. TCP is defined by RFC 793 and RFC 1122. TCP provides:
* End-to-end reliability
* Flow control
* Congestion control

TCP relies on the IP service at the network layer to deliver data to the host. Since IP is not reliable with regard to message quality or delivery, TCP must make provisions to make sure messages are delivered on time and correctly.

=== TCP segment format ===
<pre>
0 16 31
+-----------------------------------------------------------------------------------------------------------+
| Source port | Destination port |
|-----------------------------------------------------------------------------------------------------------|
| Sequence number |
|-----------------------------------------------------------------------------------------------------------|
| Acknowledgement number |
|-----------------------------------------------------------------------------------------------------------|
| hlen | Reserved | URG | ACK | PSH | RST | SYN | FIN | Window size |
|-----------------------------------------------------------------------------------------------------------|
| Checksum | Urgent pointer |
|-----------------------------------------------------------------------------------------------------------|
| Options | Padding |
|-----------------------------------------------------------------------------------------------------------|
| Data |
+-----------------------------------------------------------------------------------------------------------+
</pre>

* Source port number (16 bits)
* Destination port number (16 bits)
* Sequence number (32 bits): The byte in the data stream that the first byte of this packet represents.
* Acknowledgement number (32 bits): Contains the next sequence number that the sender of the acknowledgement expects to receive which is the sequence number plus 1 (plus the number of bytes received in the last message?). This number is used only if the ACK flag is on.
* Header length (4 bits): The length of the header in 32 bit words, required since the options field is variable in length.
* Reserved (6 bits)
* Flags:
** URG (1 bit) - The urgent pointer is valid.
** ACK (1 bit) - Makes the acknowledgement number valid.
** PSH (1 bit) - High priority data for the application.
** RST (1 bit) - Reset the connection.
** SYN (1 bit) - Turned on when a connection is being established and the sequence number field will contain the initial sequence number chosen by this host for this connection.
** FIN (1 bit) - The sender is done sending data.
* Window size (16 bits): The maximum number of bytes that the receiver will to accept.
* TCP checksum (16 bits): Calculated over the TCP header, data, and TCP pseudo header.
* Urgent pointer (16 bits): Only valid if the URG bit is set. The urgent mode is a way to transmit emergency data to the other side of the connection. It must be added to the sequence number field of the segment to generate the sequence number of the last byte of urgent data.
* Options (0 or more 32 bit words)
* Data (optional)

=== End-to-end reliability ===
In order for two hosts to communicate using TCP they must first establish a connection by exchanging messages in what is known as the three-way handshake:
<pre>
Host A Host B
In the network
Send SYN SEQ=x | --------------------> | Receive SYN
| |
Receive SYN + ACK | <-------------------- | Send SYN SEQ=y, ACK x+1
| |
Send ACK y+1 | --------------------> | Receive ACK
</pre>

# Host A initiates the connection by sending a TCP segment with the SYN control bit set and an initial sequence number (ISN) we represent as the variable x in the sequence number field.
# Host B receives this SYN segment at some point in time, processes it and responds with a TCP segment of its own. The response from Host B contains the SYN control bit set and its own ISN represented as variable y. Host B also sets the ACK control bit to indicate the next expected byte from Host A should contain data starting with sequence number x+1.
# When Host A receives Host B's ISN and ACK, it finishes the connection establishment phase by sending a final acknowledgement segment to Host B. In this case, Host A sets the ACK control bit and indicates the next expected byte from Host B by placing acknowledgement number y+1 in the acknowledgement field.

Once ISNs have been exchanged, communicating applications can transmit data between each other.

In order for a connection to be released, four segments are required to completely close a connection. Four segments are necessary due to the fact that TCP is a full-duplex protocol, meaning that each end must shut down independently:

<pre>
Host A Host B
In the network
Send FIN SEQ=x | --------------------> | Receive FIN
| |
Receive ACK | <-------------------- | Send ACK x+1
| |
Receive FIN + ACK | <-------------------- | Send FIN SEQ=y, ACK x+1
| |
Send ACK y+1 | --------------------> | Receive ACK
</pre>
# The application running on Host A signals TCP to close the connection. This generates the first FIN segment from Host A to Host B.
# When Host B receives the initial FIN segment, it immediately acknowledges the segment and notifies its destination application of the termination request.
# Once the application on Host B also decides to shut down the connection, it then sends its own FIN segment
# Host A receives the FIN segment and responds with an acknowledgement.

=== Flow control ===

Flow control is a technique whose primary purpose is to properly match the transmission rate of sender to that of the receiver and the network. It is important for the transmission to be at a high enough rate to ensure good performance, but also to protect against overwhelming the network or receiving host. Flow control is not the same as congestion control. Congestion control is primarily concerned with a sustained overload of network intermediate devices such as IP routers.

TCP uses the window field as the primary means for flow control. During the data transfer phase, the window field is used to adjust the rate of flow of the byte stream between communicating TCPs.

<pre>
window
+-------------------+
|-------------------|---------------------------------------+
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
|-------------------|---------------------------------------+
+-------------------+

Sent and Not yet
ACKed window sent
| +-------------------+ |
+---------|-------------------|-----------------------------+
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
+---------|-------------------|-----------------------------+
+-------------------+

window
+-------------------+
+---------------------------------------|-------------------|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
+---------------------------------------|-------------------|
+-------------------+
</pre>
The drawing shows a 4-byte sliding window. Moving from left to right, the window "slides" as bytes in the stream are sent and acknowledged. A simple TCP implementation will place segments into the network for a receiver as long as there is data to send and as long as the sender does not exceed the window advertised by the receiver. As the receiver accepts and processes TCP segments, it sends back positive acknowledgements, indicating where in the byte stream it is. These acknowledgements also contain the "window" which determines how many bytes the receiver is currently willing to accept. If data queued by the sender reaches a point where data sent will exceed the receiver's advertised window size, the sender must halt transmission and wait for further acknowledgements and an advertised window size that is greater than zero before resuming.

=== Congestion control ===

The data transfer strategy:
* The TCP host sends packets into the network without a reservation and then the host reacts to observable events.
* Each sender determines how much capacity is available to a given flow in the network.
* ACKs are used to "pace" the transmission of packets such that TCP is "self-clocking".

As an over simplified example, imagine a client wants to request the webpage http://www.cracked.com/ from a server. The requested page is 6 KB and we assume there is no overhead on the server to generate the page (it is static content cached in memory) or any other overhead:

# Client sends SYN to server - "Hey sexy, how are you? My receive window is 65,535 bytes."
# Server sends SYN, ACK - "Great! How are you? My receive window is 4,236 bytes"
# Client sends ACK, SEQ - "Great as well... Please send me the webpage http://www.cracked.com/"
# Server sends 3 data packets. Roughly 4 - 4.3 kb (3 * MSS1) of data
# Client acknowledges the segment (sends ACK)
# Server sends the remaining bytes to the client

After step 6 the connection can be ended (<code>FIN</code> - "Have a Nice Day") or kept alive, but that is irrelevant here, since at this point the browser has already received the data.

This transaction took 3 * RTT (Round Trip Time) to finish. If your RTT to a server is 200ms this transaction will take you at least 600ms to complete, no matter how big your bandwidth is. The bigger the file, the more round trips and the longer it takes to download.

'''AIMD:'''

In ''Additive Increase/Multiplicative Decrease (AIMD)'' a <code>CongestionWindow</code> variable is held by the TCP sender for each connection. The smallest of the two windows, the congestion window and the advertised window by the receiver is the maximum window size to start with for that receiver.

<pre>MaxWindow :: min (CongestionWindow , AdvertisedWindow)
EffectiveWindow = MaxWindow – (LastByteSent -LastByteAcked)</pre>

The variable <code>cwnd</code> is set based on the perceived level of congestion. The sender receives implicit (packet drop) or explicit (packet mark) indications of internal congestion.

* ''Additive Increase'' is the reaction to perceived available capacity (referred to as congestion avoidance stage): <code>cwnd</code> is incremented fractionally for each arriving <code>ACK</code>:
<pre>increment = MSS x (MSS /cwnd)
cwnd = cwnd + increment</pre>
* A dropped packet and resultant timeout are considered due to congestion at a router and TCP reacts with a ''Multiplicate Decrease'' by halving cwnd (but not below size of 1 MMS (packet).

Because this simple ''Congestion Control (CC)'' mechanism involves timeouts that cause retransmissions, it is important that hosts have an accurate timeout mechanism.

'''Slow Start'''

Linear additive increase takes too long to ramp up a new TCP connection from cold start and the slow start mechanism was added to provide an initial exponential increase in the size of <code>cwnd</code>. When a TCP connection begins, ''Slow Start'' initialises a congestion window to one segment, which is the ''Maximum Segment Size (MSS)'' initialised by the receiver during the connection establishment phase. When ACKs are returned by the receiver, the congestion window increases by one segment for each acknowledgement returned.

Every time an ACK arrives, <code>cwnd</code> is incremented (effectively doubling per RTT "epoch"): The first successful transmission and acknowledgement of a TCP segment increases the window to two segments. After successful transmission of these two segments and acknowledgements completes, the window is increased to four segments. Then eight segments, then sixteen segments and so on, doubling from there on out up to the maximum window size advertised by the receiver or until congestion occurs.

''Slow Start'' prevents a slow start, but is slower than sending a full advertised window’s worth of packets all at once. In the imagined example of requesting the cracked webpage, the client told the server it can receive a maximum of 65,535 bytes of un-acknowledged data (before ACK), but the server only sent about 4 KB and then waited for ACK. This is because the initial congestion window (initcwnd) on the server is set to 3. The server is being cautious. Rather than throw a burst of packets into a fresh connection, the server chooses to ease into it gradually, making sure that the entire network route is not congested. The more congested a network is, the higher is the chances for packet loss. Packet loss results in retransmissions which means more round trips, resulting in higher download times.

There are two slow start situations:
* At the very beginning of a connection {cold start}.
* When the connection goes dead waiting for a timeout to occur (the advertised window goes to zero). In this case the source has more information. The current value of <code>cwnd</code> can be saved as a congestion threshold. This is also known as the "slow start threshold" <code>ssthresh</code>:
** if cwnd <= ssthresh, do slow-start
** if cwnd > ssthresh, do congestion avoidance

If the network is forced to drop one or more packets due to overload or congestion, ''Congestion Avoidance'' is used to slow the transmission rate, in conjunction with ''Slow Start'' to get the data transfer going again so it doesn't slow down and stay slow. In the ''Congestion Avoidance'' algorithm a retransmission timer expiring or the reception of duplicate ACKs can implicitly signal the sender that a network congestion situation is occurring. The sender then sets its transmission window to one half of the current window size (the minimum of the congestion window and the receiver's advertised window size), but to at least two segments. If congestion was indicated by a timeout, the congestion window is reset to one segment, which automatically puts the sender into ''Slow Start'' mode.

'''Fast Retransmit'''

[[File:Fast-retransmit.png|264px|thumb|right]]

When a duplicate <code>ACK</code> is received, the sender does not know if it is because a TCP segment was lost or simply that a segment was delayed and received out of order at the receiver. If the receiver can re-order segments, it should not be long before the receiver sends the latest expected acknowledgement. Typically no more than one or two duplicate ACKs should be received when simple out of order conditions exist. If however more than two duplicate ACKs are received by the sender, it is a strong indication that at least one segment has been lost. The TCP sender will assume enough time has lapsed for all segments to be properly re-ordered by the fact that the receiver had enough time to send three duplicate ACKs.

When three or more duplicate ACKs are received, the sender does not even wait for a retransmission timer to expire before retransmitting the segment (as indicated by the position of the duplicate ACK in the byte stream). This process is called the ''Fast Retransmit'' algorithm.

''Fast Retransmit'' eliminates about half the coarse-grain timeouts (~ 20% improvement in throughput). Immediately following ''Fast Retransmit'' (which was added with TCP Tahoe) is the Fast Recovery algorithm (which was added with TCP Reno).

'''Fast Recovery'''

The TCP sender has implicit knowledge that there is data still flowing to the receiver because duplicate ACKs can only be generated when a segment is received. This is a strong indication that serious network congestion may not exist and that the lost segment was a rare event. So instead of reducing the flow of data abruptly by going all the way into ''Slow Start'', the sender only enters ''Congestion Avoidance'' mode: When ''Fast Retransmit'' detects three duplicate ACKs, the recovery process starts from the Congestion Avoidance region and uses ACKs in the pipe to pace the sending of packets (half <code>cwnd</code> and commence recovery from this point using linear additive increase "primed" by left over ACKs in pipe).

As a result, rather than start at a window of one segment as in ''Slow Start'' mode, the sender resumes transmission with a larger window, incrementing as if in ''Congestion Avoidance'' mode. This allows for higher throughput under the condition of only moderate congestion.

With ''Fast Recovery'', ''Slow Start'' only occurs:
* At cold start
* After a coarse-grain timeout

== User Datagram Protocol (UDP) ==

The User Datagram Protocol (UDP) is the other layer 4 protocol commonly used to support the network at the transport layer. While TCP is designed for reliable data delivery with built-in error checking, UDP aims for speed. Protocols relying on UDP typically have their own built-in reliability services, or use certain features of ICMP to make the connection somewhat more reliable. DNS, SNMP, RPC and RIP are examples of services using UDP.

UDP is a so-called unreliable connectionless protocol and is defined by RFC 768 and RFC 1122:
* A connectionless protocol does not formally establish and terminate a connection between hosts, unlike TCP with its handshake and teardown processes.
* It is a datagram service, suitable for modeling other protocols such as IP tunneling or Remote Procedure Call and the Network File System.
* There is no guarantee that the data will reach its destination. UDP is meant to provide serivce with very little transmission overhead.
* It is stateless, suitable for very large numbers of clients, such as in streaming media applications for example IPTV
* Works well in unicast and is suitable for broadcast information such as in many kinds of service discovery and shared information such as broadcast time or ''Routing Information Protocol (RIP)''

=== UDP datagram format ===
UDP adds very little to IP datapackets except for some error checking and port direction (UDP encapsulates IP packets):

<pre>
0 16 31
+-----------------------------------------------------------------------------------------------------------+
| Source port | Destination port |
|-----------------------------------------------------------------------------------------------------------|
| Length | Checksum |
|-----------------------------------------------------------------------------------------------------------|
| Data |
+-----------------------------------------------------------------------------------------------------------+
</pre>

* Source port number (16 bits)
* Destination port number (16 bits)
* UDP length (16 bits)
* UDP checksum (16 bits): The UDP checksum (for IPv4 optionally) includes UDP data, not just the header as with IP message formats.

=== Error checking ===

The UDP protocol has error checking but doesn't have any error-recovery:
* Error-detection: detect error occurs on the frame (FCS). The method used to compute the checksum is defined in RFC 768.
* Error-recovery: using sequence of bytes to detect error occurs, and if it happens, resend it.

Checksum is the 16-bit one's complement of the one's complement sum of a pseudo header of information from the IP header, the UDP header, and the data, padded with zero octets at the end (if necessary) to make a multiple of two octets http://mathforum.org/library/drmath/view/54379.html.

=== Port direction ===

Port numbers are used for addressing different functions at the source and destination of the datagram.

=== Multicasting ===

Regular network exchanges of data are peer to peer unicast transactions. A HTTP request to a web server (TCP/IP), email SNMP (TCP/IP), DNS (UDP), FTP (TCP/IP), etc, are all peer to peer unicast transactions. If you want to transmit a video, audio or data stream to multiple nodes with one transmission stream instead of multiple individual peer to peer connections, you can use ''multicasting''. Multicast and a network broadcast are different. Multicast is a UDP broadcast only and the messages are only "heard" by the nodes on the network that have "joined the multicast group", which are those that are interested in the information https://netref.soe.ucsc.edu/osnl/65.

=== Security ===

Depending on environment, UDP's built-in checksum may or may not be reliable enough.

UDP has a 16 bit checksum field starting at bit 40 of the packet header. This suffers from (at least) 2 weaknesses:
* Checksum is not mandatory, all bits set to 0 are defined as "No checksum".
* It is a 16 bit checksum in the strict sense of the word, so it is susceptible to undetected corruption.

An even more realistic threat than data courruption along the transport is packet loss reordering: UDP makes no guarantees about all packets to (eventually) arrive at all and packets to arrive in the same sequence as sent.

UDP has no built-in mechanism to deal with payloads bigger than a single packet. It wasn't built for that. If we choose to use UDP, we need to build those parts that are integral to TCP but not to UDP into the application. This will most likely result in a (possibly) inferior reimplementation of TCP. Or not. These protocols may even be an improvement.

== Internet Control Message Protocol (ICMP) ==

Compared to other IP protocols the Internet Control Message Protocol (ICMP) is fairly small and is defined by RFC 792 and RFC 1122. It belongs to the IP layer of TCP/IP but relies on IP for support at the network layer. ICMP messages are encapsulated inside IP datagrams. ICMP only reports errors involving fragment 0 of any fragmented datagrams. The IP, UDP or TCP layer will usually take action based on ICMP messages.

ICMP serves a large number of disparate functions. At its core ICMP was designed as the debugging, troubleshooting, and error reporting mechanism for IP. The errors reported by ICMP are generally related to datagram processing. ICMP will report the following network information:

* Timeouts
* Network congestion
* Network errors such as an unreachable host or network.
* The ping command is also supported by ICMP.

<pre>
0 8 16 31
+--------------------------------------------------------------------------------------+
| Message type | Code | Checksum |
|--------------------------------------------------------------------------------------|
| Unused |
|--------------------------------------------------------------------------------------|
| Data |
+--------------------------------------------------------------------------------------+
</pre>
The ICMP message consists of an 8 bit type, an 8 bit code, an 8 bit checksum, and contents which vary depending on code and type http://www.iana.org/assignments/icmp-parameters/icmp-parameters.xhtml.

=== Messages ===

'''Echo/Reply'''

ICMP echo messages (ICMP type 8) are sent to a remote computer and are returned in an echo-reply response. The primary use for these messages is to check the availability of the target computer.
# Computer A creates an ICMP ECHO datagram, using computer A's IP address as the source IP address, and computer B's IP address as destination.
# The ICMP ECHO datagram is transmitted via the network to destination B.
# The destination B copies the ECHO information into a new ECHO-REPLY message datagram.
# B destroys the original ICMP ECHO message.
# B now becomes the source of a new ECHO-REPLY datagram and places it's own address in the source IP address field of the IP header, and host A's IP address in the destination field of the IP header.
# The datagram is transmitted to the network and is routed to A.

'''Host unreachable'''

When a datagram is being forwarded, and reaches a gateway attached to the network a destination host is to be found on, and the route to the host is down, the host is not responding or does not respond on the service port in question, a Host Unreachable message is sent back.

This message is usually displayed as a ''U'' in a ping result, and as ''!H'' in a traceroute result.

'''Redirect'''

ICMP redirect messages can be used to redirect a source host to use a different gateway that may be closer to the destination. These redirect messages are sent by the receiving gateway and the source host should adapt it's forwarding accordingly when receiving this message. ICMP Redirects are most often used in source routing environments where the source host calculates routing paths to all destinations itself.

Even though the gateway has instructed the source to redirect it's traffic, it still forwards the original datagram that triggered the redirect message, however the source should no longer continue to direct packets to that gateway and should instead use the gateway specified in the response to the redirect.

'''Source quench'''

An ICMP source quench message is intended as a congestion control mechanism in IP. Source quench messages are used when a network gateway cannot forward a message because its message buffers are full. The gateway transmits a source quench message back to the source host machine to request that the source reduce it's transmission rate until it no longer receives source quench messages from the gateway. Thus, this effectively throttles back the source's transmission rate.

The gateway can transmit multiple source quench messages, one for each packet it receives from a source. The source machine is not required to respond to these source quench messages.

'''Time (To Live) Exceeded/Expired'''

Every IP datagram contains a field called "time to live" or TTL. On each hop along the path to the destination, the TTL field is decremented by one. When the value of the TTL field equals zero, the datagram is discarded to prevent the datagram from floating around the network forever. The gateway may also notify the source host via the ICMP time exceeded message.

Because hosts along a network path may not have the same ammount of memory for buffering data, it is sometimes necessary to fragment a packet into smaller pieces. These fragments must later be reassembled. If a host is missing fragments, and is unable to reassemble the datagram, and the TTL has expired, an ICMP message can be sent to the transmitting host.

Traceroute uses the TTL exceeded message to track the path through the network from source to destination. Traceroute sets the TTL on it's first set of packets to 1 and waits for the TTL exceeded response, which returns with the sender's IP address (both round trip time to that device, and its IP address are aquired).

'''Timestamp'''

The data received (a timestamp) in the message is returned in the reply together with an additional timestamp. The timestamp is 32 bits of milliseconds since midnight UT.

The ''Originate Timestamp'' is the time the sender last touched the message before sending it, the ''Receive Timestamp'' is the time the echoer first touched it on receipt, and the ''Transmit Timestamp'' is the time the echoer last touched the message on sending it. If the time is not available in miliseconds or cannot be provided with respect to midnight UT then any time can be inserted in a timestamp provided the high order bit of the timestamp is also set to indicate this non-standard value.

The identifier and sequence number may be used by the echo sender to aid in matching the replies with the requests. For example, the identifier can be used like a port in TCP or UDP to identify a session, and the sequence number can be incremented on each request sent. The destination returns these same values in the reply.

=== Ping ===

# Source host generates an ICMP ''Protocol Data Unit (PDU)''.
# The ICMP PDU is encapsulated in an IP datagram, with the source and destination IP addresses in the IP header. The datagram is an ICMP ECHO datagram, but is often called an IP datagram because that is what it looks like to the networks it is sent over.
# Source host notes the local time on it's clock as it transmits the IP datagram towards the destination. Each host that receives the IP datagram checks the destination address to see if it matches their own address or is the ''all hosts'' address (all 1's in the host field of the IP address).
# If the destination IP address in the IP datagram does not match the local host's address, the IP datagram is forwarded to the network where the IP address resides.
# The destination host receives the IP datagram, finds a match between itself and the destination address in the IP datagram.
# Destination host notes the ICMP ''Echo'' information in the IP datagram, performs any necessary work, then destroys the original IP/ICMP ''Echo'' datagram.
# The destination host creates an ICMP ''Echo Reply'', encapsulates it in an IP datagram placing it's own IP address in the source IP address field, and the original sender's IP address in the destination field of the IP datagram.
# The new IP datagram is routed back to the originator of the ping. The host receives it, notes the time on the clock and finally prints ping output information, including elapsed time.

This is repeated until all requested ICMP ''Echo'' packets have been sent and their responses have been received or the default 2-second timeout expired (The default 2-second timout is local to the host initiating the ping and is '''not''' the Time-To-Live value in the datagram).

The response times for ping are Round Trip and cumulative over the entire path out and back to that destination. ''Ping'' reveals nothing regarding the intermediate devices. It does not tell *where* a latency or packet loss occurs, nor if some sort of queuing stragegem is in place altering the results. It cannot be trusted for other purposes than to verify that a host is up and functioning.

=== Security ===

Most routers come with the option to set the router to ignore or drop ICMP redirects because they can be used to attack networks by confusing hosts as to where the correct default gateway is. ICMP redirects may also be used to set up Man-in-the-Middle attacks.

The variable size of the ICMP packet data section has been exploited a lot. In the well-known "Ping of death," large or fragmented ping packets are used for denial-of-service attacks. ICMP can also be used to create covert channels for communication (see LOKI exploit). When people talk about blocking ICMP they're really talking about <code>ping</code> and <code>traceroute</code>. ''Ping'' can be used to determine if a host is alive, ''Time Exceeded'' (as part of a traceroute) can be used to to map out network architectures, or void forbid a ''Redirect'' (type 5 code 0) to change the default route of a host.

Reasons why we may not want to restrict ICMP:
* Path MTU Discovery - We use a combination of the Don't Fragment flag and type 3 code 4 (Destination Unreachable - Fragmentation required, and DF flag set) to determine the smallest MTU on the path between the hosts. This way we avoid fragmentation during the transmission.
* Active Directory requires clients ping the domain controllers in order to pull down GPOs. They use ping to determine the "closest" controller and if none respond, then it is assumed that none are close enough. So the policy update doesn't happen.

Blocking ICMP in its entirety is probably not a good idea, picking and choosing what to block and to/from where, probably will get us what we want. See [[Linux_security#Making_the_rules|ufw rules]] and [[Linux_security#Making_the_rules_2|iptables rules]].

== Simple routing ==

Inbound traffic is captured based on ARP and IP address configuration. Outbound traffic is managed by ''routes''. Routing determines the path these packets take so that they are sent to their destinations. This is required for all IP traffic, local and remote, including when multiple network interfaces are available. Routes are held by the ''kernel routing table''.
* ''Direct routing'' table entries occur when the source and destination hosts are on the same physical network and packets are sent directly from the source to the destination.
* ''Indirect routing'' table entries occur when the source and destination hosts are on different physical networks. The destination host must be reached through one or more IP gateways. The first gateway is the only one which is known by the host system.
* ''Default routing'' defines a gateway to use when the direct network route and the indirect host routes are not defined for a given IP address.
For ''static routes'' IP uses the routing table to determine where packets should be sent. First the packet is examined to see if its destination is for the local or a remote network. If a remote network, the routing table is consulted to determine the path. If there is no information in the routing table then the packet is sent to the default gateway. Static routes are set with the <code>route</code> command http://linux.die.net/man/8/route.

For ''dynamic routes'' the ''Routing Information Protocol (RIP)'' is used https://en.wikipedia.org/wiki/Routing_Information_Protocol. If multiple routes are possible, RIP will choose the shortest route (fewest hops between routers not physical distance). Routers use RIP to broadcast the routing table over UDP port 520. The routers then add new or improved routes to their routing tables.

== Mesh network routing ==

Ad-Hoc is one of the modes of operation for an 802.11 radio at OSI layer 1, the physical layer, and it basically means that all devices can communicate directly to any other device that is within radio range. Normally, in "infrastructure mode", wireless devices can only communicate with a central Access Point (AP) or router and that device is responsible for re-transmitting packets from one client device to another client device (even if they are right next to each other). Ad-Hoc networks get rid of the middle-man that is the AP, however they don't have any inherent capability for multi-hop. That means, if device A can reach device B, and device B can reach device C, but A cannot reach C, then A and C cannot communicate because B will not re-transmit any packets.

Mesh networking, also know as Mesh Routing happens at OSI layer 3, the network layer. Mesh Routing allows each device on a network (also called nodes) to act as a router and re-transmit packets on behalf of any other devices. Mesh Routing provide the multi-hop facility that Ad-Hoc mode lacks. By combining Ad-Hoc mode at layer 1 and Mesh Routing at layer 3 we can create wireless mesh networks purely between client devices without any need for centralized Access Points or Routers. Both Ad-Hoc and Mesh Routing can be described as P2P as they are both instances of clients-to-client communication, just at different layers of the OSI model.

Routing protocols are organized as:

* Reactive or on-demand routing protocols where the route is discovered when needed. These protocols tend to decrease the control traffic messages overhead at the cost of increased latency in discovering a new routes. In reactive protocols there is no need of distribution of information. It consumes bandwidth when data is transferred from source to destination. Examples are AODV (ad-hoc on demand distance vector), DSR (distance vector routing) and ABR (associatively based routing) protocols.
* Proactive routing protocols where every node stores information in the form of tables and changes in network topology require an update to the tables. The nodes swap topology information. There is no route discovery delay associated with finding a new route. The fixed cost of proactive routing is greater than that of a reactive protocols. Examples are DSDV (destination sequenced demand vector) and OLSR (optimized link state routing protocols).
* Hybrid routing protocols are a combination of both reactive and proactive routing protocols. It was proposed to reduce the control overhead of proactive routing protocols and decrease the latency caused by route discovery in reactive routing protocols. Examples are ZRP (zone routing protocol) and TORA (temporarily ordered routing algorithm).

=== Reactive routing ===
* The ''Ad hoc On Demand Distance Vector (AODV)'' routing algorithm is an on demand routing protocol designed for ad hoc mobile networks. AODV is capable of both unicast and multicast routing. It is an on demand algorithm, meaning that it builds routes between nodes only as desired by source nodes. It maintains these routes as long as they are needed by the sources. Additionally, AODV forms trees which connect multicast group members. The trees are composed of the group members and the nodes needed to connect the members. AODV uses sequence numbers to ensure the freshness of routes. It is loop-free, self-starting, and scales to large numbers of mobile nodes.

=== Proactive routing ===
* ''The Optimized Link State Routing Protocol (OLSR)'' is a dynamic linkstate Protocol which collects link data and dynamically calculates the best routes within the network.
* The ''Better Approach To Mobile Ad-Hoc Networking (B.A.T.M.A.N.)'' is a protocol under development by the "Freifunk" community and intended to replace OLSR. No single node has all the data. Knowledge about the best route through the network is decentralised, eliminating the need to spread information concerning network changes to every node in the network. Individual nodes only save information about the “direction” it received data from. Data gets passed on from node to node and packets get individual, dynamically created routes. A network of collective intelligence is created.
* ''Caleb James DeLisle's Network Suite (CJDNS)'' is a table driven networking protocol designed to make every node equal; there is no hierarchy or edge routing. Rather than assigning addresses based on topology, all cjdns IPv6 addresses are within the FC00::/8 Unique local address space (keys which do not hash to addresses starting with 'FC' are discarded). Although nodes are identified with IPv6 addresses, Cjdns does not depend upon having IPv6. Each node connects to a couple other nodes by manually configured links over an IPv4 network (such as the Internet) or via the Ethernet Interface.
* ''olsrd'' and ''olsrd2'' are both table driven Link State Routing Protocol implementations optimized for Mobile ad hoc networks on embedded devices like commercial of the shelf routers, smartphones or normal computers. Sometimes these networks are called "mesh networks". olsrd and olsrd2 are the routing daemons which make up the mesh.
=== Hybrid routing ===
* The ''Zone Routing Protocol (ZRP)'' framework is a hybrid routing framework suitable for a wide variety of mobile ad-hoc networks, especially those with large network spans and diverse mobility patterns. Each node proactively maintains routes within a local region (referred to as the routing zone). Knowledge of the routing zone topology is leveraged by the ZRP to improve the efficiency of a globally reactive route query/reply mechanism. The proactive maintenance of routing zones also helps improve the quality of discovered routes, by making them more robust to changes in network topology. The ZRP can be configured for a particular network by proper selection of a single parameter, the routing zone radius.
=== Configuration ===
* The ''Ad-Hoc Configuration Protocol (AHCP)'' is an autoconfiguration protocol for IPv6 and dual-stack IPv6/IPv4 networks designed to be used in place of router discovery and DHCP on networks where it is difficult or impossible to configure a server within every link-layer broadcast domain. AHCP will automatically configure IPv4 and IPv6 addresses, name servers and NTP servers. It will not configure default routes, since it is designed to be run together with a routing protocol (such as Babel or OLSR).

== Anonimising proxies ==

An anonymising proxy server is a server whose only function is to be a node. It only reroutes requests from one location to another. If Cathy wants to make a connection to Heathcliff without him knowing that it is Cathy connecting to him, she would fill in Heathcliff's IP address at a proxy server. The proxy server would then make a connection to Heathcliff and relay all the information Heathcliff sends to it to Cathy.

This does not use expensive encryption techniques and is easy to understand and use.

=== Vulnerabilities ===
* If an unreliable third party controls a proxy server, a group of criminals who use the proxy server for phishing, users are no longer guaranteed secure and anonymous communication (over that route).

== Tunneling ==

Tunneling means that the complete IP packet to be sent from source to destination is encapsulated into another IP packet. This new packet has a legal internet IP address.

== SSH tunneling ==

Secure shell (SSH) is used to securely acquire and use a remote terminal session and has other uses as well. You can use SSH to tunnel your traffic, transfer files, mount remote file systems, and more. SSH also uses strong encryption and you can set your SSH client to act as a ''Socks proxy''. Once you have, you can configure applications on your computer – such as your web browser – to use the ''Socks proxy''. The traffic enters the ''Socks proxy'' running on your local system and the SSH client forwards it through the SSH connection – this is known as SSH tunneling. This works similar to browsing the web over a VPN. From a web server perspective, traffic appears to be coming from the SSH server. The traffic between source and the SSH server is encrypted, so you can browse over an encrypted connection as you could with a VPN. You must configure each application to use the SSH tunnel’s proxy.

Port forwarding or port mapping is a name given to the combined technique of:
* Translating the address and/or port number of a packet to a new destination.
* Possibly accepting such packet(s) in a packet filter (firewall).
* Forwarding the packet according to the routing table.

=== Socks proxy ===

A Socks proxy is different from a "normal" proxy in that they are application proxies. For example, when you use a HTTP proxy you are actually forwarding the HTTP request, and the HTTP proxy server then performs the request on your behalf. Socks provides authentication for protocols that cannot be authenticated and bypasses default routing in the internal network.

=== Socks protocol ===

The ''Socks protocol'' is roughly equivalent to setting up an IP tunnel with a firewall and the protocol requests are then initiated from the firewall.

# The client contacts the ''Socks proxy server'' and negotiates a proxy connection.
# When a connection is established, the client communicates with the Socks server using the Socks protocol.
# The external server communicates with the ''Socks server'' as if it were the actual client.

=== Secure shell ===

SSH tunnels can be created in several ways using different kinds of port forwarding mechanisms. Ports can be forwarded in three ways:

* [[Anonymising_your_traffic_with_linux#Tunneling_with_local_port_forwarding|Local port forwarding]]
* [[Anonymising_your_traffic_with_linux#Reverse_tunnelling_with_remote_port_forwarding|Remote port forwarding]]
* [[Anonymising_your_traffic_with_linux#Dynamic_port_forwarding|Dynamic port forwarding]]

== Virtual Private Network (VPN) ==
VPN is used for connecting to private networks over public networks (internet). A VPN client communicates over the internet and sends the computer’s network traffic through the encrypted connection to a VPN server. The encryption provides a secure connection, which means petty tyrants (adversaries) can not snoop on the connection and see sensitive information. Depending on the VPN service, all the network traffic may be sent over the VPN, or only some of it.

A VPN works more at the operating system level than the application level. In other words, when you set up a VPN connection, your operating system can route all network traffic through it from all applications (although this can vary from VPN to VPN, depending on how the VPN is configured). You don’t have to configure each individual application.

=== Point-to-Point Tunneling Protocol (PPTP) ===

PPTP defined in RFC 2637 allows multiprotocol traffic to be encrypted and then encapsulated in an IP header to be sent across an IP network or a public IP network. PPTP can be used for remote access and site-to-site VPN connections. When using the internet as the public network for VPN, the PPTP server is a PPTP-enabled VPN server with one interface on the internet and a second interface on the intranet.

PPTP encapsulates Point-to-Point Protocol (PPP) frames in IP datagrams for transmission over the network. PPTP uses a TCP connection for tunnel management and a modified version of Generic Routing Encapsulation (GRE) to encapsulate PPP frames for tunneled data. The payloads of the encapsulated PPP frames can be encrypted, compressed, or both. A PPTP packet containing an IP datagram:

<pre>
< Encrypted >
+-------------------------------------------------------------------------+
| IP header | GRE header | PPP header | PPP payload (IP datagram) |
+-------------------------------------------------------------------------+
< PPP Frame >

</pre>

PPTP is taking advantage of the underlying PPP encryption and encapsulating a previously encrypted PPP frame.

Nowadays usually only found using 128-bit encryption keys, in the years since it was first bundled with some Windows OS back in 1999, a number of security vulnerabilities have come to light, the most serious of which is the possibility of unencapsulated MS-CHAP v2 Authentication. Using this exploit, PPTP has been cracked within 2 days, and although Micro$oft has patched the flaw (through the use of PEAP authentication), it has itself issued a recommendation that VPN users should use L2TP/IPsec or SSTP instead <ref> Microsoft Security Advisory 2743314: Unencapsulated MS-CHAP v2 Authentication Could Allow Information Disclosure http://technet.microsoft.com/en-us/security/advisory/2743314</ref>. The vulnerable MS CHAPv2 authentication is still the most common in use.

Knowing that PPTP was insecure anyway, it came as no surprise to anybody that the NSA almost certainly decrypts PPTP encrypted communications as standard. Perhaps more worrying is that the NSA has (or is in the process of) almost certainly decrypted the vast amounts of older data it has stored, which was encrypted back when even security experts considered PPTP to be secure.

=== Layer 2 Tunneling Protocol (L2TP) ===

L2TP defined by RFC 2661, allows multiprotocol traffic to be encrypted and then sent over any medium that supports point-to-point datagram delivery, such as IP or ''Asynchronous Transfer Mode (ATM)''. L2TP is a combination of PPTP and Layer 2 Forwarding (L2F) with the best features of both.

Encapsulation for L2TP/IPsec packets consists of two layers. First L2TP encapsulation: A PPP frame (an IP datagram) is wrapped with an L2TP header and a UDP header:
<pre>
+-----------------------------------------------------------------------------------------+
| IP header | UDP header | L2TP header | PPP header | PPP payload (IP datagram) |
+-----------------------------------------------------------------------------------------+
< PPP Frame >
< L2TP Frame >
< UDP Frame >
</pre>
The resulting L2TP message is then wrapped with an IPsec ''Encapsulating Security Payload (ESP)'' header and trailer, an IPsec Authentication trailer that provides message integrity and authentication, and a final IP header. In the IP header is the source and destination IP address that corresponds to the VPN client and VPN server.

<pre>
+-----------------------------------------------------------------------------------------------------------------------------+
| IP header | IPSec | UDP header | L2TP header | PPP header | PPP payload | IPSec | IPSec |
| | ESP header | | | | (IP datagram) | ESP trailer | AUTH trailer |
+-----------------------------------------------------------------------------------------------------------------------------+
< Encrypted by IPSec >
</pre>

The L2TP message is encrypted with either [[Encrypting_everything#Data_Encryption_Standard_.28DES.29|Data Encryption Standard (DES) or Triple DES (3DES)]] by using encryption keys generated from the negotiation process.

IPsec encryption has no major known vulnerabilities, and if properly implemented may still be secure. However, Edward Snowden’s revelations have strongly hinted at the standard being compromised by the NSA, and as John Gilmore (security specialist and founding member of the Electronic Frontier Foundation) explains in this post, it is likely that it has been been deliberately weakened during its design phase <ref>Re: [Cryptography] Opening Discussion: Speculation on "BULLRUN" http://www.mail-archive.com/cryptography@metzdowd.com/msg12325.html</ref>.

=== Secure Socket Tunneling Protocol (SSTP) ===

Secure Socket Tunneling Protocol (SSTP) is a tunneling protocol that uses the HTTPS protocol over TCP port 443 to pass traffic through firewalls and Web proxies that might block PPTP and L2TP/IPsec traffic. SSTP provides a mechanism to encapsulate PPP traffic over the Secure Sockets Layer (SSL/TLS v3) channel of the HTTPS protocol. The use of PPP allows support for strong authentication methods, such as EAP-TLS. SSL/TLS provides transport-level security with enhanced key negotiation, encryption, and integrity checking.

SSTP encapsulates PPP frames in IP datagrams for transmission over the network. SSTP uses a TCP connection (over port 443) for tunnel management as well as PPP data frames. The SSTP message is encrypted with the SSL/TLS channel of the HTTPS protocol.

When a client tries to establish a SSTP-based VPN connection, SSTP first establishes a bidirectional HTTPS layer with the SSTP server. Over this HTTPS layer, the protocol packets flow as the data payload.

SSTP is a proprietary standard owned by Micro$oft. This means that the code is not open to public scrutiny, and Microsoft’s history of co-operating with the NSA, and on-going speculation about possible backdoors built-in to the Windows operating system, does not inspire trust and confidence in the standard.

=== Open Virtual Private Network (OpenVPN) ===

OpenVPN is a fairly new open source technology that uses the OpenSSL library and SSLv3/TLSv1 protocols, along with a combination of other technologies, to provide a strong and reliable VPN solution. One of its major strengths is that it is highly configurable, and although it runs best on a UDP port, it can be set to run on any port, including TCP port 443. This makes traffic on it impossible to tell apart from traffic using standard HTTPS over SSL/TLS, and it is therefore extremely difficult to block.

Another advantage of OpenVPN is that the OpenSSL library used to provide encryption supports a number of cryptographic algorithms (e.g. AES, Blowfish, 3DES, CAST-128, Camellia and more), although VPN providers almost exclusively use either AES or Blowfish. 128-bit Blowfish is the default cypher built in to OpenVPN, and although it is generally considered secure, it does have known weaknesses. Blowfish is known to be susceptible to attacks on reflectively weak keys. This means Blowfish users must carefully select keys as there is a class of keys known to be weak, or switch to more modern alternatives like Blowfish's successors Twofish and Threefish.

OpenVPN has become the default VPN connection type, and while natively supported by no platform, is widely supported on most through third party software (including iOS and Android).

It seems OpenVPN has not been compromised or weakened by the NSA. Although no-one knows the full capabilities of the NSA for certain, the mathematics strongly points to OpenVPN, and if used in conjunction with a strong cipher, could be the best choice.

=== Internet Key Exchange (IKEv2) ===
Internet Key Exchange (version 2) is an IPSec based tunneling protocol jointly developed by Micro$oft and Cisco, and baked into Windows versions 7 and above. The standard is supported by Blackberry devices, and independently developed (and compatible) open source implementations are available for Linux and other operating systems. IOW, the code can be inspected and if that implementation is used, we can perhaps be a little bit less wary.

It is not as ubiquitous as IPSec but is considered at least as good as, if not superior to, L2TP/IPsec in terms of security, performance (speed), and stability. Mobile users in particular benefit the most from using IKEv2 because of its support for the Mobility and Multihoming (MOBIKE) protocol, which also makes it highly resilient to changing networks.

== DNS leaks ==

When using an anonymity or privacy service, it is extremely important that all traffic originating from your computer is routed through the anonymity network. If any traffic leaks outside of the secure connection to the network, any adversary monitoring your traffic will be able to log your activity.

== Mix networks ==

In the eighties, digital mixes (sometimes called mix networks or mixnets) to achieve a higher level of anonymity with personal communication appeared. Digital mixing uses a similar system as routing but it adds several layers in the connection between the sender and receiver of the communication. The layers are created using [[Encrypting_everything#Asymmetric_key_encryption|public key cryptography]]. Using digital mixing is comparable to sending a letter encased in four envelopes pre-addressed and pre-stamped with a small message reading, "please remove this envelope and repost".

Note: ''Mixnets are not designed to disguise the fact that you are using a mix network. If an adversary can simply lock you up for using anonymity tools, you need to disguise your use of anonymity tools.''

=== Sending mixed messages ===

<pre>
_____________ +---------+ _____________ _____________ _____________ _____________
| | +----------+ /| | | +----------+ | | | | | |
| | +----------+ /|__/ | | | +----------+ /| | | +----------+ | | | |
| | |\ /|__/ |_\__| | | |\ /|__/ | | | |\ /| | | | |
|___________| | \______/ |_\__| |___________| | \______/ |_\__| |___________| | \______/ | |___________| |___________|
_|_____|_ |__/____\__| _|_____|_ |__/____\__| _|_____|_ |__/____\__| _|_____|_ _|_____|_
/ ******* \ ............................ / ******* \ ....................... / ******* \ .................. / ******* \ ................ / ******* \
/ ********* \ / ********* \ / ********* \ / ********* \ / ********* \
--------------- --------------- --------------- --------------- ---------------
</pre>
If Cathy wants to send a message to Heathcliff, without a third person being able to find out who the sender or recipient is, she would encrypt her message three times with the aid of public key cryptography. She would then send her message to a proxy server who would remove the first layer of encryption and send it to a second proxy server through the use of permutation. This second server would then decrypt and also permute the message and the third server would decrypt and send the message to the intended recipient.
=== Threshold batching ===
A mix node must collect more than one message before sending any out - otherwise the node is behaving as an onion router node with a time delay. The more messages collected, the more uncertainty is introduced as to which message went where. Using this threshold batching strategy to solve a lack of messages can make the period between the sending and the eventual receiving of the message long, like several hours, depending on the amount of messages deemed critical.

This system is thought effective because as long as the three successive recipients, the re-senders, send enough messages to different mixnodes it is impossible for a third person like an ISP (and government (policing) agencies) to find out what message was originally sent by whom and to whom. Mixing is specifically designed to provide security even if an adversary can see the entire path. See [[Simulation: Mixnets]].

=== Vulnerabilities ===
* Only works if the resenders send enough messages (at any given moment and during a set amount of time). Because (most) nodes, the resending servers, do not send enough messages at the same time, digital mixing could be vulnerable to statistical analysis such as data mining by governments or government policing and intelligence agencies.
* The use of [[Encrypting_everything#Asymmetric_key_encryption|public key cryptography]] in itself is not very fast, and has its own vulnerabilities.

== Tor onion routing ==

Tor combines aspects of [[#Mix_networks|digital mixing]] and [[#Anonimising_proxies|anonymising proxies]].

''Mix networks get their security from the mixing done by their component mixes, and may or may not use route unpredictability to enhance security. Onion routing networks primarily get their security from choosing routes that are difficult for the adversary to observe, which for designs deployed to date has meant choosing unpredictable routes through a network. And onion routers typically employ no mixing at all. This gets at the essence of the two even if it is a bit too quick on both sides. Mixes are also usually intended to resist an adversary that can observe all traffic everywhere and, in some threat models, to actively change traffic. Onion routing assumes that an adversary who observes both ends of a communication path will completely break the anonymity of its traffic. Thus, onion routing networks are designed to resist a local adversary, one that can only see a subset of the network and the traffic on it.'' - Paul Syverson - Why I'm not an Entropist <ref>Why I'm not an Entropist http://freehaven.net/anonbib/cache/entropist.pdf</ref>

=== Connecting ===

* If Cathy wants to make a connection to Heathcliff through the Tor network, she makes an unencrypted connection to a centralised directory server containing the addresses of Tor nodes.
* After receiving the address list from the directory server the Tor client software connects to a random node (the entry node), through an encrypted connection.
* The entry node makes an encrypted connection to a random second node which in turn does the same to connect to a random third Tor node.
* The third node (the exit node) connects to Heathcliff.

Every Tor node is chosen at random (the same node cannot be used twice in one connection and depending on data congestion some nodes will not be used) from the address list received from the centralised directory server, both by the client and by the nodes, to enhance the level of anonymity as much as possible.

=== Changing routes ===

If the same connection (the same set of nodes) were to be used for a longer period of time a Tor connection would be vulnerable to statistical analysis, which is why the client software changes the entry node every ten minutes.

=== Be a node ===

If Cathy uses the Tor network to connect to Heathcliff and also functions as node for Jane she also connects to a Tor node for that. An ill-willing third party will find it extremely hard to know which connection is initiated as a user and which as a node.

=== Vulnerabilities ===

* If an adversary is able to see the entire path, onion routing loses its security. If a government makes their own national internet, running Tor would not provide security because the government would be able to see the entire path.
* Not only that. If an attacker can see you, and can see the website you're visiting, even if you create a path outside the adversary's control - they will still be able to correlate the traffic and learn you are visiting the website. This clearly raises concerns about using onion routing to visit a website or websites related to your own government.

== I2P garlic routing ==

Garlic routing is a variant of onion routing that encrypts multiple messages together to make it more difficult for attackers to perform traffic analysis https://geti2p.net/en/docs/how/garlic-routing. I2P implements a ''packet switched routing'' instead of ''circuit switched'' (like Tor). Tunnels are unidirectional, and Tor's circuits are bidirectional.

I2P uses garlic routing, bundling and encryption in three places:

* For building and routing through tunnels (layered encryption)
* For determining the success or failure of end to end message delivery (bundling)
* For publishing some network database entries (dampening the probability of a successful traffic analysis attack) ([[Encrypting_everything#ElGamal|ElGamal]]/[[Encrypting_everything#Advanced_Encryption_Standard_.28AES.29|AES]])

=== Tunnels ===
An ''i2p tunnel'' is a directed path through an explicitly selected list of routers. The first router that belongs to a tunnel is named ''gateway''. The communication within a tunnel in unidirectional, this means that it is impossible to send back data without using another separated tunnel:
* ''outbound tunnels'' are those tunnels used to send messages away from the tunnel creator.
* ''inbound tunnels'' are those tunnels used to bring messages to the tunnel creator.

=== Routers ===
There is no rigid distinction between a server and a pure client like there is in the Tor architecture.

Information transits on network routers that are able to decrypt only the respective layer. The information managed by each single node is composed by:
* IP address of the next router
* Encrypted data to transfer.

=== The network database ===
''netDb'' is a pair of algorithms used to share the following metadata with the network:
* ''routerInfo'' is a data structure to provide routers the information necessary for contacting a specific router (public keys, transport addresses, etc). Each router send its ''routerInfo'' to the ''netDb'' directly, that will collect info on the entire network.
* ''leaseSets'' is a data structure to give routers the information necessary for contacting a particular destination. A ''leaseSet'' is a collection of ''leases''. Each specifies a tunnel gateway to reach a specific destination. It is sent through outbound tunnels anonymously, to avoid correlating a router with its ''leaseSets''. A lease contains the following info:
** Inbound gateway for a tunnel that allows reaching a specific destination.
** Expiration time of a tunnel.
** A pair of public keys to encrypt messages (to send through the tunnel and reach the destination).

=== Routing ===

* When Cathy wants to send a message to Heathcliff, she does a lookup in the netDb to find Heathcliff’s leaseSet, giving her his current inbound tunnel gateways.
* Cathy's router aggregates multiple messages into a single ''garlic message'', encrypting it using a public key.
* The garlic is encrypted using the public key published in Heathcliff's ''leaseSet'', allowing the message to be encrypted without giving out the public key to Heathcliff's router.
* Cathy's router selects one of her outbound tunnels and sends the data, including instructions for the outbound tunnel's endpoint to forward the message on to one of Heathcliff's inbound tunnel gateways.
* When the outbound tunnel endpoint receives those instructions, it forwards the message according the instructions provided, and when Heathcliff’s inbound tunnel gateway receives it, it is forwarded to his router.
* If Cathy wants Heathcliff to be able to reply to the message, she needs to transmit her own destination explicitly as part of the message itself.

I2P is end-to-end encrypted. No information is sent in clear or decrypted. Each node has an internal network address different from the network IP address (and is not used).

=== Layered encryption ===

I2P uses cryptographic ID's to identify routers and end point services. Naming identifiers use ''Base 32 Names'': a SHA256 digest is attributed to the base64 representation of the destination. The hash is base 32 encoded and <code>.b32.i2p</code> is concatenated onto the end of the hash.

* During ''connecting'' (build up of tunnel) only the routing instructions for the next hop are exposed to each peer.
* During data transfer, messages are passed through the tunnel. Message and its routing instructions are only exposed to the endpoint of the tunnel.
* An additional end to end layer of encryption hides the data from the outbound tunnel endpoint and the inbound tunnel gateway.
* Each tunnel has an encryption layer to avoid unauthorized disclosure to peers inside the network.

== Resources ==
[[File:Standards.png|500px|thumb|right|Standardisation committees are where "communications" and "investment" meet ... ]]
=== Standardisation ===
* RFC http://www.ietf.org/rfc.html
* IEEE http://standards.ieee.org/about/get/
* The Internet Engineering Task Force (IETF) https://www.ietf.org/
* An Analysis of Internet Standardization by Marcus Maher (1998) http://www.vjolt.net/vol3/issue/vol3_art5.html

=== Protocols ===
* TCP Slow Start - Whiteboard Series https://www.youtube.com/watch?v=JiyyQTJnlTs
* TCP slow start http://packetlife.net/blog/2011/jul/5/tcp-slow-start/

=== Security ===
* Security Assessment of the Internet Protocol Version 4 (2011) https://tools.ietf.org/html/rfc6274

=== Pirate boxes ===
* For Free Information and Open Internet Independent journalists, community media and hacktivists take action” (pdf): PirateBox or How to Escape the Big Brothers of the Internet (mathieu lapprand) – starts on page 143 http://www.coredem.info/IMG/pdf/pass11_an-2.pdf

=== Low-latency onion routing ===
* HORNET: High-speed Onion Routing at the Network Layer http://arxiv.org/pdf/1507.05724v1.pdf

== Related ==

* [[Anonymising your traffic with linux]]
* [[Take control of the tech]]
* [[Encrypting everything]]

== References ==

[[Category:Resources]]

Threat modeling the quick and dirty way

2015-10-09T14:40:11Z

Lilith2:

[[File:Dystopia.gif|500px|thumb|right]]

== Basic choreography ==
Steward Brand: Build a room
=== Set up a table ===
The first column contains a short description of the threat, the second the likelihood of it occurring, the third what impact it would have if it did happen, and the fourth an assessment (grade) of the time and energy you would need to protect yourself from the threat (for instance you can have no stars denote that there is no protection from that threat, hence it will cost nothing (except for the cost of the impact if it did happen).

=== Fill in the table ===
I recommend doing iterative brainstorming on "known and experienced threats" as initial filling of the first column in the table, before thinking about the other columns.

=== Reorder the list according to your set of priorities ===
Choose your ordering strategy carefully. Several strategies are possible.
* If this is a learning experience or you are a fan of "only time for putting out fires" cultures, no need for ordering.
* In a low risk environment (no immediate death threats) an "on demand" strategy works well. In this strategy you can use "low hanging fruit" and set up protection for items with a big impact and/or high likelihood of occurrence first.
* In a high risk environment or if any of the items in the list of possible impacts reads " loss of life" or some life-altering experience or you have turned procrastination into an art, best choose an "anticipating strategy", meaning do more research and detailed [[scenario planning]] (food for thought) to find possible [[Threats, detection, protection and (counter) moves|threats and solutions]] overlooked (food for gut).

== Examples ==

=== Silicon Valley first world problems ===

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Having to drink || medium || Developer will drop dead within a week, deadline will not be made.

The human body needs water to survive. The maximum time an individual can go without water seems to be a week — an estimate that would certainly be shorter in difficult conditions, like broiling heat.
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Having to eat food || high || Developer will drop dead within three weeks, deadline will not be made.

A human can go for more than three weeks without food (Mahatma Gandhi survived 21 days of complete starvation)
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Needing sleep || high || Like breathing, sleep is essential to humans. It has even been said that one could survive three times longer without food than one could without sleep. One of the better known experiments on this subject, found that depriving rats entirely of sleep resulted in their death, or near-dying state, within 11-32 days (Everson et al. 1989). || ** No long term alternatives known, but the deadline is still four weeks off. We tried adding coffee and coke to the Schmoylent in the previous development cycle.
|-
| Bathroom breaks || high || How many times do people go to the bathroom per day? Loads, and all those little breaks can add up to an hour or two per developer per day.
|| ***** Implanting stomata. Costly. Management has suggested we need more surveillance equipment to study the problem of the breaks.
|}

=== Journalist, observer or sousveillant in europe ===
A general counter strategy against police misconduct, abuse and brutality has been recording what happens during a protest. This of course, has gotten the police to target (citizen) journalists and observers overtly and covertly. And it is not just the police and government that are interested in us and our data.

The below threats, if happens, make for loss of trust and reputation and that translates to loss of effectiveness as independent observer.
{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Metanoia & Paranoia || low || Metanoia (safety delusion) makes us sitting ducks and paranoia renders us ineffective. || *
|-
| Physical attacks || medium || Physical damage/intimidation
||
|-
| Arrest: That might include direct approaches such as intimidation and asking for information on sources. || medium || Physical danger to those sources.||
|-
| Some contacts are named in reports, and some wish to remain anonymous. If we’re not careful, we might unintentionally disclose contact details or their location http://thenextweb.com/insider/2012/12/03/vice-leaves-metadata-in-photo-of-john-mcafee-pinpointing-him-to-a-location-in-guatemala/
* Phishing
* Guess/hack password
* Metadata
* Mobile trail
|| high || Intimidation/attack/imprisonment of sources. ||*
|-
| Correspondence with contacts includes all sorts of information that we do not want others to know about.
* Phishing
* Guess/hack password
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.
|| **
|-
| Email, social media accounts, websites, communication tools, a pile of passwords. Our passwords may be a used to attack others in our "organisation", and access information we don’t directly hold.
* Phishing emails can be sent from our email or social media accounts to others.
* Passwords can be guessed.
|| high || Followers on social media and people on mailinglists can get spammed.
|| ****
|-
| All work-in-progress, files and documents which we couldn’t publish for all sorts of reasons (space, legality, protection of others).
* Where are they? What forms do they have? Is there a security related documentation policy? Hey who is that man walking out the door with those files?
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.

Intimidation/attack/imprisonment of sources.
|| *
|-
| If we use computers in any way (including phones and even photocopiers) we can safely assume we are being recorded. Can lead to identification of sources through other data, our location, phone or email records.
* Metadata
* Computer forensics
|| high || Intimidation/attack/imprisonment of sources.
|| *
|-
| It may not be our activities they are interested in, but our reach. An example of this was the Syrian Electronic Army targeting E! Online http://ohnotheydidnt.livejournal.com/77479774.html || medium || Publication of hoax information on content management systems (the news website) and social media accounts
|| ***
|-
| Legal attacks include direct approaches such as subpoenas demanding that we reveal a source, or court orders to pass over footage, and increasingly might also include indirect approaches, to companies and organisations holding your information.
|| medium || Physical, social or economic danger to those sources, e.g. losing employment, losing privacy, criminal proceedings, assault.
|| **
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline masters of the internet]]
* [[Timeline merchants of death]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

Scenario planning

2015-10-09T14:07:11Z

Lilith2:

''A group of analysts generate scenario planning [[Simulations|simulation games]] for policy makers. The games combine known facts about the future, such as in environmental, demographics, geography, military, political, social, and science issues, industrial information, and (limiting) resources such as mineral reserves, with plausible alternative trends which are key driving forces of the games.''

It isn’t real. It is make-believe, pretend. But ... the scenario planning process/choreography can reveal anticipatory thinking elements that can be difficult to formalise, such as subjective experiences during its sessions, shifts in values, new regulations, guides, and/or sudden insights.
* 80% or more of the creativity comes during the first few hours that scenario planning teams brainstorm their story elements, implications, and responses.
* If a simulation is made from a set of likely scenarios, improved response times to real threats can be expected for years after.
* Can be used for decision making when organisations are facing a critical issue and implicitly look to scenarios for help in making a decision now, immediately, and is very effective at discovering 80% of the likely effects of our decisions.
* This choreography comes at a price: The games are likely to conjure up that which is tacitly denied.
Scenarios are a way of understanding the dynamics shaping the future. It can be likened to doing katas in martial arts: people are removed from "reality", placed in some "future" as if it is real, and then brainstorm and practice possible responses to potential threats and challenges and explore possible deviating paths.

By imagining the worst and best cases we come better prepared for either, and for "facing" that which we don’t like to see or hear or tend to be in denial about. So it is not so much a predictive tool and more like a training in decision making, threat modeling and risk management. We increase our self-confidence, an essential ingredient for being action-able if and when needed.

If scenario planning is done with collectives for the individuals in the collective as "policy makers" and we also pass on the choreography itself, we can distribute a way to distribute power. Now to find a way to make it a more informal and fun experience that includes all senses and common sense.

== Basic choreography of a focused scenario planning ==

=== Decision focus ===
Scenario planning isn't rocket science.

=== Identify the primary "driving forces" at work in the present ===
These fall roughly into four categories: social, economic, political and technological issues. Of course, categories are only parts of a mindset, and you can set it up completely different. Real issues entail a bit of all forces, and have reflections in the other categories. The point of listing the driving forces is to look past the everyday crises that typically occupy our minds and to explore long-term forces that ordinarily work well outside our concerns. It is these powerful forces that will usually catch us unaware.

=== Identify predetermined elements ===
Once these forces are enumerated, we can see that from our perspective, some forces can be called "predetermined" (meaning they are completely outside our control and will play out in any story we tell about the future). Predetermined elements are those that can be predicted accurately because they change very slowly. Not all forces are so evident, or so easy to calculate, but when we build our stories, predetermined elements figure in each one.

For example, I expect to find changing weather patterns in all recently published scenarios, in many different forms. In some it will be described as a worsening state, in others as a cause of great famine that was overcome by a few, or in the form of someone mentioning the great opportunity of wine making in an area that was previously unfit for growing grapes.

=== Develop scenario logics ===
After identifying and removing predetermined elements from the list of driving forces, we are left with a number of uncertainties. Discuss and sort these to make sure they are all critical uncertainties. A critical uncertainty is an uncertainty that is key to our focal issue. Our goals are to better understand all the uncertainties and their relationships with each other and we want the few that we believe are most important to the focal issue and most impossible to predict to float up to the surface.

Here we have a fork in the road: you can take both paths (recommended) or one of the two paths. If only taking the deductive road, you make yourself totally dependent on the moves of your adversaries. If only doing the inductive path you are likely to end up a sitting duck.

''Note: The deductive approach is easier with larger groups and for people untrained in reaching consensus. The inductive path is more unsystematic and calls for degrees of creativity and imagination and making it a multiple days process to include night time dreaming (How many times did I not wake up with new insights?) And it requires a lot of patience with an open ended debate. To make it easier, there is also a more guided inductive path.''

==== Deductive scenario logics ====
On the deductive path, prioritise the 'key factors' in order to find the two most critical uncertainties. Those then are placed, for example, in a 2×2 scenario matrix: we can reduce affinity groups of uncertainties that have some commonality to a single spectrum, an axis of uncertainty. Simplify the list of related uncertainties into two orthogonal axes, creating a matrix that allows us to define four very different, but plausible, quadrants of uncertainty. Each of these far corners is a logical future that we can explore.

Don't throw away the original list that we have taken predetermined elements and critical uncertainties from. The rest of the key factors and issues come back when fleshing out the scenarios in "rich compelling plots".

==== Inductive scenario logics ====
What if ...

By asking and discussing answers to these questions we can build a scenario that will have future consequences that may call for some strategic decisions in the present.

==== Official future deviations ====

This is a slightly more systematic variant of the inductive approach: The "official future" is what we believe, either explicitly or implicitly, will happen. Usually we make that a plausible and relatively non-threatening scenario, featuring no surprising changes to the current environment and continued stable growth. And in some circumstances the "official future" can reflect our fears, for example that the world is a mess, or we in trouble.

In this approach we start by describing radically different and optimistic futures and then work backwards, exploring the 'key factors' that would enable such a future to unfold. Deductive scenario logics can be used as an addition.

=== Fleshing out the scenarios ===
The driving forces that we generated now become "characters" in the stories that we develop. The "real" future will not be any of the scenarios, but will likely contain elements of all of our scenarios. Our goal is to pin down the "hilariously overdone" corners of the plausible futures. Make "hilariously overdone" be the outer limits of plausibility and as a result, our scenarios will have a caricature quality.

=== Implications ===
Some factors and decisions will make sense to all of us across all of the futures. Others will make sense only in one or two. The first can be used for making robust plans and for the latter we want to know the "early warning signs" that tell us if and when those scenarios are beginning to unfold. Leading indicators for a given scenario can be obvious, but more often they are subtle and could go by unnoticed. It may be some legislation, a seemingly small technical breakthrough, or a gradual social trend. When we have those identified, we can set up (watchdog and data mining) operations to monitor these critical signs.

=== Name of the game ===

Then focus on what the name of our game is and address the inverse question. Beef up the skeletal scenarios to discover the insights we need.

=== Data mining for key factors ===

==== Bayesian logic ====
For its vulnerabilities see [[Confusing surveillance systems]].

=== Diagramming of effects ===
Normally called causal-loop diagrams. In 'diagramming of effects' observables were added, next to measurables, allowing for inclusion of "[[Ominous_dots#Phenomenal_consciousness|phenomenal consciousness]]".

== Example of a short-term scenario planning ==
Short-term scenario planning is useful for threat modelling. In this example, ideas for new tools for "Journalist, observer or sousveillant roles in europe" that cut across digital and analogue space can emerge.

=== Decision focus ===
What are the threats? What can we do to protect ourselves, our sources and our data? In what order do we work on that?

=== Key factors ===

* What information do we not want other people to know? (This can be anything from passwords to contacts’ details, data and documents)
* Why might someone want that information? Who?
* What can they do to get it?
* What might happen if they do?

== Example of a long-term scenario planning ==

This example is unseriously seriously meant for designing a real world (analog) roleplay simulation game with (much more fun than workshops and serious scenario planning). And since we have great facilitators everywhere, a worthy challenge. And those have a hacker attitude (no problem should ever be solved twice http://www.catb.org/esr/faqs/hacker-howto.html#attitude): The Alpha Complex is an excellent base to build simulation games as this one on. <ref>Paranoia (roleplaying game) https://en.wikipedia.org/wiki/Paranoia_%28role-playing_game%29</ref>

We can include information already gathered by traditional-for-opportunity-and-profit scenario planners. Claimed "market" research data may not be that important to know, but what markets they are aiming to "penetrate" in the near future could be. For example, the so-called "Gen Z" is already a target in trend watching reports by some big corporations <ref>MAKE WAY FOR GEN Z https://media.ford.com/content/fordmedia/fna/us/en/news/2014/12/29/2015-ford-trend-report-explores-generation-z.html</ref>. And while some corporations benefit from pillage and plunder of earth's resources, other corporations are into "market penetration" games because any locals that do benefit from such pillage and plunder are "emerging markets". Have a hamburger! <ref>Anticipating the rise of junk food and soda taxes in emerging markets http://www.pwc.com/gx/en/retail-consumer/publications/assets/pwc-r-and-c-trendwatch-anticipating.pdf</ref>

Besides including key factors found by others for purposes of further fucking us over in the future, we can include sources these "futurists" have no access to, namely lots of stories and realities grubby grabbers (and their enablers) are in denial about.

=== Decision focus ===

Is it time yet?!? <ref>Anonymiss(tress) Operation ENOUGH! https://vimeo.com/33208014</ref> Yes, this is our time <ref>Underminers: A Practical Guide for Radical Change https://www.youtube.com/watch?v=GEMcVfswXMo</ref>

=== Key factors ===
Imagine a world designed by Kafka, Stalin, Orwell, Huxley, Sartre and the Marx Brothers … We’re talking about a virtual reality of course. A Role-Playing Game. The game is set in Alpha Complex, an immense and futuristic domed or underground city controlled by The Computer. The Computer runs everything within Alpha Complex ...
* What are the possible futures for the Alpha Complex?
* What are our own possible futures if and when the Alpha Complex collapses?
* What do we see in the future (time frame: the next ten years)?

==== Social forces ====
''While I'm still confused and uncertain, it's on a much higher plane, d'you see, and at least I know I'm bewildered about what you want ...''

Quantitative, demographic issues, softer issues of values, lifestyle, demand, political energy:
* IRONYGuard - Removes All The Painful Irony In Your News https://www.youtube.com/watch?v=BFPdnIwP-kw
* Will people get bored with fiddling with their phones?
* Toy Stories: photographic proof that childhood has been commodified http://www.theguardian.com/artanddesign/jonathanjonesblog/2014/apr/01/toy-stories-childhood-commodified
* Libraries Make Room For High-Tech 'Hackerspaces' (US) http://www.npr.org/2011/12/10/143401182/libraries-make-room-for-high-tech-hackerspaces
* Overstimulation and Desensitization — How Civilization Affects Your Brain http://jdmoyer.com/2011/11/28/overstimulation-and-desensitization-how-civilization-affects-your-brain/
* Push red button to add drama https://www.youtube.com/watch?v=316AzLYfAzw
* Wii Fit https://www.youtube.com/watch?v=_iYBmAVuBns

==== Technological forces ====
''A triumph of the silicon chunk, a miracle of modern magical technology ...''

Direct, enabling, and indirect factors:
* 2020 Media futures http://2020mediafutures.ca/Strategic+Foresight
* Big Brother is WWWatching You https://www.youtube.com/watch?v=o66FUc61MvU
* Our drone future https://www.youtube.com/watch?v=CgLkWT246qU
* New Paper on Digital Intelligence https://www.schneier.com/blog/archives/2015/03/new_paper_on_di.html
* Hydraulic Fracturing 101 http://www.earthworksaction.org/issues/detail/hydraulic_fracturing_101
* The Heretic's Guide to Global Finance: Hacking the Future of Money http://suitpossum.blogspot.co.uk/2015/06/high-frequency-trading-guide.html
* DNA and Mutations http://evolution.berkeley.edu/evolibrary/article/mutations_01
* DNA Replication and Causes of Mutation http://www.nature.com/scitable/nated/article?action=showContentInPopup&contentPK=409
* One Reason Why Chromosomes Break, Often Leading To Cancer http://www.sciencedaily.com/releases/2007/08/070803091034.htm
* Mutational landscape and significance across 12 major cancer types http://www.nature.com/nature/journal/v502/n7471/full/nature12634.html

==== Economic forces ====
''See Captain Samuel Vimes 'Boots' theory of socioeconomic unfairness''.

Macroeconomic trends and forces shaping the economy as a whole, microeconomic dynamics and internal forces:
* Clarke and Dawe - Quantitative Easing https://www.youtube.com/watch?v=j2AvU2cfXRk
* The Economy: Ron Paul vs Zeitgeist https://www.youtube.com/watch?v=ELEwjVRxxGE
* Bitcoin – finally, fair money? http://www.metamute.org/editorial/articles/bitcoin-%E2%80%93-finally-fair-money
* Which Corporations Control the World? http://www.internationalbusinessguide.org/corporations/
* A Bit Rich http://www.neweconomics.org/publications/entry/a-bit-rich
* Primavera De Filippi on Ethereum: Freenet or Skynet? https://www.youtube.com/watch?v=slhuidzccpI
* A Bitcoin Battle Is Brewing http://techcrunch.com/2014/12/06/a-bitcoin-battle-is-brewing/
* The Second Wave of Blockchain Innovation https://medium.com/@Swarm/the-second-wave-of-blockchain-innovation-270e6daff3f5
* Can Bitcoin Conquer Argentina? http://www.nytimes.com/2015/05/03/magazine/how-bitcoin-is-disrupting-argentinas-economy.html?_r=0
* The Revolution will (not) be decentralised: Blockchains http://commonstransition.org/the-revolution-will-not-be-decentralised-blockchains/
* Natural Resources and Conflict in Africa http://the-beacon.info/countries/africa/natural-resources-and-conflict-in-africa/
* Drilling into Debt: An Investigation into the Relationship Between Debt and Oil http://priceofoil.org/2005/07/01/drilling-into-debt-an-investigation-into-the-relationship-between-debt-and-oil/
* Foreclosing the Future: The World Bank and the Politics of Environmental Destruction http://triplecrisis.com/foreclosing-the-future-the-world-bank-and-the-politics-of-environmental-destruction/
* Wealth: Having it all and wanting more http://policy-practice.oxfam.org.uk/publications/wealth-having-it-all-and-wanting-more-338125
* Why Does Sweden Have So Many Billionaires? http://www.slate.com/articles/business/billion_to_one/2013/10/sweden_s_billionaires_they_have_more_per_capita_than_the_united_states.html
* Top 10 Policies for a Steady-State Economy http://steadystate.org/top-10-policies-for-a-steady-state-economy/
* Seventeen Contradictions and the End of Capitalism http://www.thewhitereview.org/features/seventeen-contradictions-and-the-end-of-capitalism/
* Ponzi Scheme Capitalism: An Interview with David Harvey http://review31.co.uk/interview/view/16/ponzi-scheme-capitalism-an-interview-with-david-harvey
* Why Government Hates Cash http://www.zerohedge.com/news/2015-08-24/why-government-hates-cash
* Germans begin the looting of Greece (Bailout paves the way for the fire-sale privatization of the public’s assets) http://www.marketwatch.com/story/germans-begin-the-looting-of-greece-2015-08-21 The plundering that has now begun unmasks the whole euro charade for what it really is — a war of conquest by money rather than by arms.

==== Environmental forces ====
''The truth may be out there, but the lies are inside your head. Environments are not just containers, but are processes that change the content totally.''
* The Fallacy of the Tragedy of the Commons http://steadystate.org/the-fallacy-of-the-tragedy-of-the-commons/ (there is an option beyond the privatisation or nationalisation of "the commons").
* The Dark Mountain Manifesto http://dark-mountain.net/about/manifesto/ And a reply: Re-civ not un-civ https://freelab2014.wordpress.com/2015/05/05/re-civ-not-un-civ/
* Urban Gardening with Vegetables http://www.gardeners.com/how-to/urban-gardening-with-vegetables/5491.html
* Nutrient-Dense "Green Food" In Just 10 Days http://www.growmicrogreens.com/
* Heaven belongs to us all – the new papal encyclical http://www.realclimate.org/index.php/archives/2015/06/heaven-belongs-to-us-all-the-new-papal-encyclical/#more-18619
* The heat is online http://www.heatisonline.org/news.cfm
* Geopiracy: The Case Against Geoengineering https://www.cbd.int/doc/emerging-issues/etcgroup-geopiracy-2011-013-en.pdf
* Health of U.S. Refinery Communities At Risk from Tar Sands http://www.forestethics.org/news/refineries-report-press-release
* Into eternity http://www.intoeternitythemovie.com/
* Accelerated modern human–induced species losses: Entering the sixth mass extinction http://advances.sciencemag.org/content/advances/1/5/e1400253.full.pdf

==== Political forces ====
''hAS aNYONE sEEN MY cAPSLOCK kEY?''

Electoral, legislative and regulatory factors:
* Capitalism and Climate Change Redux http://www.counterpunch.org/2014/09/19/capitalism-and-climate-change-redux/
* The EuroDiVision Contest - feat. Merkel, Žižek & IMF https://www.youtube.com/watch?v=y1OnDgBNlRU
* On the Commodification of Human Discovery http://bollier.org/blog/commodification-human-discovery
* TTIP, TISA and TTP – world trade revolution? http://www.alliancesud.ch/en/policy/trade/ttip-tisa-and-ttp-2013-world-trade-revolution
* The Arms Trade Treaty: A historic and momentous failure https://ceasefiremagazine.co.uk/failure-arms-trade-treaty/
* Behind a mining monopoly http://stream.aljazeera.com/story/201208272028-0022324
* The price of oil per barrel doesn’t include all of oil’s hidden costs http://priceofoil.org/thepriceofoil/
* Dirty energy money http://dirtyenergymoney.com/
* Corruption and fraud in the oil, gas and mining industries keeps poor countries poor and props up brutal regimes. Companies and governments must end the secrecy and bring deals and profits into the open https://www.globalwitness.org/campaigns/oil-gas-and-mining/
* U.S. Advises Oil Companies How to Break The Law, Approves LNG Despite Fracking Fallacy Debate http://petroleumtruthreport.blogspot.nl/2015/01/us-advises-oil-companies-how-to-break.html
* Ursula K Le Guin's speech at National Book Awards: 'Books aren't just commodities' http://www.theguardian.com/books/2014/nov/20/ursula-k-le-guin-national-book-awards-speech
* Davos 2014: For the real news, don’t listen ... watch http://www.independent.co.uk/voices/comment/davos-2014-for-the-real-news-dont-listen--watch-9069420.html

==== Military forces ====
''The consensus seemed to be that if really large numbers of men (or the weapons replacing them) were sent to storm the mountain, then enough might survive the rocks (defenses) to take the citadel. This is essentially the basis of all military thinking. ''
* [[Covert operations#War games|Covert operations war games]]
* Die Schweizer Armee probt den Ernstfall – in Basel http://www.basellandschaftlichezeitung.ch/basel/basel-stadt/die-schweizer-armee-probt-den-ernstfall-in-basel-129136790

==== Other key factors ====
* The 2nd Heliocentric Revolution https://www.youtube.com/watch?v=Q_xI_8aLjds
* One Tree Island http://polyp.org.uk/onetreeisland/onetreeisland_1.html
[[File:Deductive-logics-matrix.jpg|320px|thumb|right]]

=== Pre-determined elements ===
* Squirmishes, conflicts, wars
* Changing climate and weather

=== Scenario logics ===

==== Deductive logics ====
We can spin many scenarios combining the key factors and many axes in the matrix, but choosing just two axes provides already enough of a framework for exploring more forces. What hathanger we use is not as important as the clothes hanging on them.

For example, the horizontal axis of uncertainty chosen here is desire (and hence of military might as all wars start in spirit, in the Alpha Complex). Will it be individual or collective? Will the energy of individual "liberation" and "freedom" continue to prevail? Or will we root our social organization and self-definition in a collective? What will become the prevailing influence in our local culture?

The vertical axis represents social structure: Will it be distributed or centralised? Will it bring us a "grassroots" anarchy that wurks? Or will there be some "body" to impose order?

==== Inductive logics ====
A resistance fighter could easily ask "What if a change in socio-political institutions does take place? What if Brian Holmes' "political ecology" is possible? What if 1% of the people in various locations of the complex join us for this change? What might lead up to such a change? What would be a plausible chain of consequences following from such a change?"

=== Fleshing out the scenarios ===

==== Scenario 1: Everything Inc. ====

[[File:Rat race.jpg|480px|thumb|right|http://polyp.org.uk/index.html]]
The Alpha Complex has been taken over by corporations. There are no citizens anymore, not a single one in the entire Dome. There are only employees and consumers. There's even "mass tailored" services for dissenters and rebels.

Their phones may be built-into their bodies, but people still fiddle with their phones, so that looks like people are constantly fiddling with themselves. All corporations and companies are agile, nimble, have all their feedback loops effectively closed and anticipate the market's whims aggressively. Computers, robots and drones have replaced all manual labour, police and military functions, and are now replacing the technocracy class.

There's still democracy, in name. Politics means electronic voting. Our biometrics are tracked our entire lives, down to the littlest details. Not that people care about any of that. And I don't think there's any DIY done anywhere anymore. It's against the law anyway, and the corporate police drones see everything, really everything.

Though the word is still in use, we haven't seen any real money in years. We get paid in spending vouchers. Everything in The Dome (and 5%) is owned by corporations. All the land and all the water. All goods are made in specific areas of The Dome, automated of course. Another area manages everything. All other areas focus on tourism and entertainment, also run by corporations. Happiness is always just around the corner.

The underground movements flourish. We keep developing new spyware detection tools from what little that ends up on the scrapyard. Did I mention yet all feedback loops are effectively closed?

Temperatures are very hot, and seem to be getting hotter by the day where we live. The running around for food and water is taking more and more time and energy. Other areas are sealed off. They were flooded or hit by severe storms. We heard all people in those areas died.

==== Scenario 2: Ecotopia ====
[[File:Eco-dalek.jpg|480px|thumb|right|http://polyp.org.uk/index.html]]

GDP fetishism is finally finished. In response to centuries of pillage and plunder, collective values triumphed over strictly individualistic values.

Hackerspaces developed and spread a political ecology for the Dome that resulted in a small digitised distributed Alpha Complex: Direct taxation on property and capital (not on income) for funding health care and public works that aid survival of the greatest scope of life to a greater degree than any associated destruction.

The transformation was not easy and is still ongoing. Corporations, or what's left of them, are struggling. A lot of less developed areas in The Dome didn't make it either. Having been plundered and pillaged for centuries they could not fund the transformations necessary to meet the new environmental regulations. We may have been in the clutches of the "power of scarcity" but their scarcity experience was real and as grassroots movements go, at the time, we didn't have the resources nor the required infrastructure (yet) to send them what they needed. We asked those that did have the infrastructure and resources, but they understood only "no profit in it" and were unhelpful. We did what we could.

Ye always have underground. The current underground movements feature names like "free market", "free zones" and other "free" and "freedom" labeled combinations. I keep wondering why, because they are free, just not free to take someone elses freedom away or to arrogate and appropriate common resources (and rape the planet).

Sex is now considered normal (as long as it is consensual). That's a big relief for everyone. I don't know exactly what happened but can take an educated guess. Pr0n died the minute it was accepted and women and other genders started sharing what they liked, what their sexual fantasies were, and trying it out. Oh, and there's a lot of torrents on The Computer on hero related movies. Forget pr0n. Not interested. I am watching all old episodes of Dr. Who. He always gave me a good laugh.

==== Scenario 3: Bohemians ====

[[File:Jobs profits.jpg|480px|thumb|right|http://polyp.org.uk/index.html]]

The Alpha Complex has collapsed in the face of privatisation and is replaced by a largely electronic marketplace that connects and clears transactions of every type, bitcoin-like. The Dome organises itself by job via The Computer.

Communication focuses on personal empowerment. The Computer is the chief exchange medium for decentralised work, personal gratification, and domal commerce. Physical infrastructure stagnates.

Art and attention have turned inward and personal expressions flourish in new media on The Computer. Strangely enough, all physical hackerspaces have been abandoned while technology is the leading domal culture. A homogenous patchwork of unbridled bohemianism.

Civil wars regularly break out all over The Dome as a legacy of previously imposed borders unravels. In some locations the history of underground movements repeats itself and revolutions come around again, and again, including their co-options by "leaders" (surrounding themselves with followers as a personal survival strategy) and attempts at creating new borders.

There is a large "black market" and "informal circuit" of exchanges of services and goods not reported to The Computer, not the least of which is a network of people gifting their time and energy to keep a semblance of infrastructure going for an underground economy of goods and services between people that have no jobs or are doing by The Computer undervalued jobs.

==== Scenario 4: United Nations ====
[[File:World order.jpg|480px|thumb|right|http://polyp.org.uk/index.html]]
Surveillance of civilians has stopped, officially. The Computer is owned by corporations and contains no civilian traffic, officially. There is a separate civilian internet, a loosely connected network of local networks with local services. Some networks are only local. Groups in locally developed social media are usually closed groups. Rurals have their own local private computer networks, mostly as a warning system for when "others" come to "visit" the area.

There are no nations. The Dome is organised in tribes, clans, (extended) families, networks, and roving bands and warlords. Nations crumbled into city-states that consider rurals second-class people. Oddly enough, there is still a United Nations serving as Alpha Complex. Who is serving in the UN board remains a mystery.

Motor clubs and gangs in developing areas and old inner cities (now small city-states) transform into political law-and-order machines paid for by corporations, and they have quota to meet. So does the justice system. And the prison-industrial complex needs to meet their quota too. In the cities and in rural areas, everybody watches over everybody else really, and this has paid off. We're all very healthy. Corporations do extremely well, but are severely regulated by the United Nations and fund all kinds of United Nations programs. And just like in the days of the pharao's, large public works are undertaken, some even spanning the globe, serving corporations mostly because the "war-work-machine" (alias military-industrial-complex) constantly needs to gobble up new resources to grow. The military has globalised as well and has a lot of UN projects to protect for they are sought after targets by the underground.

Corporate UN universities (there are no others) are "free" in exchange for twenty years of work in the by you chosen industry. You can go into research if you can get a UN grant with a proposal or work directly in the field. There's many interesting corporate faculties to choose from of which geoengineering is very popular among young folk. Our new heroes, "protecting us from climate change". And if you sign up for another twenty years, you can perhaps build a modest pension fund for you and your (extended) family.

Both online and offline skirmishes and conflicts between groups are quite normal. The underground is everywhere if you believe the news. UN projects seem to be a favourite target of some underground movements.

=== Name of the inverse game ===
A resilient (distributed, partly underground when and where necessary) solidarity network.

== Examples of diagramming of effects ==
=== Gut, head, and decision making ===
Visualisation of Decision Architecture: Helping Users Make Better Decisions http://www.uxmatters.com/mt/archives/2010/11/decision-architecture-helping-users-make-better-decisions.php

[[File:Gut-and-head.png|640px|thumb|center|]]

* Gut processing is sophisticated, intuitive, and quick. Gut enables an efficient way of navigating a complex world.
* Head processing, on the other hand, is analytical, slow, and rational.
* According to Gardner, "Gut decides, Head reviews: This process is how most of our thoughts and decisions are made." Essentially, we are of two minds, each of which works semi-independently of the other.
* Gut processing always precedes Head processing. The "||" in the connector means a delay.
* Even if Head steps in and overrules Gut, this does not negate the impact of Gut on a decision. It is our initial impressions, our initial reactions to things, that shape and color the thoughts and judgments that follow.
* Sometimes, Head doesn’t bother to monitor Gut. Sometimes, Head doesn’t step in at all. When this happens, decision making occurs automatically, under the radar of our conscious attention.
* Gut can sometimes apply mental shortcuts that really aren’t appropriate to the situation at hand or are simply incorrect.
* Head can’t look inside Gut to figure out how or why Gut operates the way it does. Our subconscious is much like a black box with no access doors. It’s strictly off limits to the conscious mind. All Head can do is monitor and override Gut; it can’t change or negate the influence of Gut.
* One reason Head may fail to actively take part in the process of decision making is because conscious thought takes effort. People are remarkably sensitive to the effort of conscious decision making—the mental processing that occurs primarily within the prefrontal cortex.
* In a 1999 experiment, Shiv and Fedorikhin tested the effect of cognitive load on decision making: Distracting Head with a memory task made people more susceptible to temptation.
* A person’s objective in decision making is to arrive at the best possible decision outcome with the least possible effort. Better decision outcomes typically require more effort. So, decision makers must make tradeoffs between the two, adjusting their strategy according to the importance of the decision they’re making.
Speaking of not becoming a petty tyrant and inverse game scenarios:
* Insights and gained knowledge from "Gut decides, Head reviews" were unseriously seriously applied in [[Installing linux#Shopping for a linux distro|shopping for a linux distro]].
* Simulation games in safe spaces can create experiences with which a person can "soften" or "harden" emotional tags and/or set mental shortcuts. Their own. Make sure the facilitators are very experienced in "not being present" in that regard. Playful alpha state is a very vulnerable state. Setting direction for someone in that state is a "no no".
''"The trouble with having an open mind, of course, is that people will insist on coming along and trying to put things in it." Terry Pratchett, Diggers ''

=== Hidden effects of grubby grabbers ===

Visualisation of findings in Drilling into Debt (PDF) http://priceofoil.org/content/uploads/2011/01/DrillingIntoDebt.pdf of 2005 showing 4 reinforcing feedback loops:

[[File:Grubby-grabbers.png|640px|thumb|center|Contains 4 reinforcing feedback loops]]

Key findings of the report from 2005:
* Increasing oil production leads to increasing debt. There is a strong and positive relationship between oil production and debt burdens. The more oil a country produces, regardless of oil’s share of the country’s total economy, the more debt it tends to generate
* Increasing oil exports leads to increasing debt. There is a strong and positive relationship between oil export dependence and debt burdens. The more dependent on oil exports a country is, the deeper in debt it tends to be.
* Increasing oil exports improves the ability of developing countries to service their debts. There is a strong and positive relationship between oil exports and debt service. The global oil economy improves the ability of countries to make debt payments, while at the same time increasing their total debt.
* Increases in oil production predict increases in debt size. Doubling a country’s annual production of crude oil is predicted to increase the size of its total external debt as a share of GDP by 43.2 per cent. Likewise, the same change is predicted to increase a country’s debt service burden by 31 per cent. For example, the Nigerian government currently plans to increase oil production by 160% by 2010. Past trends indicate that Nigeria’s debt can thus be expected to increase by 69%, or $21 billion over the next six years.
* World Bank programs designed to increase Northern private investment in Southern oil production have instead drastically increased debt. Northern multilateral and bilateral "aid" for oil exporting projects in the South has exacerbated, rather than alleviated debt. Specifically, an examination of those countries where the World Bank Group conducted "Petroleum Exploration Promotion Programs" (PEPPs) reveals debt levels (debt-GDP ratios) in those countries that are 19% higher than those countries that did not undergo this form of structural adjustment.
* The relationship between debt & oil is most likely caused by the interplay in between three factors:
** Structural incentives for and direct investments in the oil industry by multilateral and bilateral institutions, such as the World Bank Group and export credit agencies.
** Oil fueled fiscal folly – both in the North by creditors over eager to lend to nations perceived as oil rich, and in the South by unwise fiscal policies.
** The volatility of the oil market ("power of scarcity").
A previous report, published in 2004 by the Institute for Policy Studies, demonstrates how multilateral support for oil is consistent with an agenda to diversify oil supplies for Northern consumption, and open Southern reserves to Northern corporate investment. It also noted that 82 percent of all oil extractive projects funded by the World Bank Group since 1992 are export-oriented, and primarily serve the energy needs of the North, not the South.

== Scenarios ==

=== An opportunistic scenario ===

''It was so much easier to blame it on Them. It was bleakly depressing to think that They were Us. If it was Them, then nothing was anyone’s fault. If it was us, what did that make Me? After all, I’m one of Us. I must be. I’ve certainly never thought of myself as one of Them. No one ever thinks of themselves as one of Them. We’re always one of Us. It’s Them that do the bad things.'' — (Terry Pratchett, Jingo)

10. Globalisation seems to indicate a “multitude of interconnected fatal consequences, social disintegration, a breakdown of democracy, more rapid and extensive deterioration of the environment, the spread of new diseases, and increasing poverty and alienation”.

9. Evasion of real civilian control is made possible by military-to-military and political-to-political relations. Building military-to-military relations with an unreformed and un-reforming military undermines efforts by citizens to rein in such institutions.

8. Economic ties between first and second and third world countries relies too heavily on cheap labour, with inadequate attention given to the rights of workers performing such labour.

7. Investments are too often focused on extractive industries exploiting local natural resources. While profitable to first world investors and their elite local second and third world partners, these investments too often devastate environments with ruinous implications. Let’s name this for what it is: engaging in environmentally criminal behavior such as resource greed. Forests are destroyed for timber or establishing plantations, with no thought to the consequences for local people for whom those same forests created homes and a source of livelihood. Mining operations poison rivers, bays and groundwater.

6. Whole communities are displaced and their subsequent intrusion elsewhere can cause local wars in such elsewheres.

5. Of course, this creates opportunities for a focus on sale or transfer of weapons and entire military bases, joint military training, etc.

4. Repeating colonialist mistakes. Believing we know better than local people. Intervening in a system we do not really know. Ignoring effects of our interventions, especially when staring us in the face. Denial.

3. Distracting people by creating a focus on irrelevant threats. Dangerous move, what with the internet and more and more people becoming aware of the usual and commonly used political patterns. The internet may put enough of a stop that. Telephone had a great social impact. Let’s hope the internet makes a similar difference. How can we believe “others than us” are a threat if I have exchanged with “others” and found them to be very human, and not so “other” at all? When we refuse to believe authorities and conjured up “common enemies”, and we think for ourselves? :)

2. Climate change conferences, as if we can stop the climate changing. We can’t. Too late. And even if we could, is that the way to go? <ref>Geopiracy: The Case Against Geoengineering http://www.etcgroup.org/content/geopiracy-case-against-geoengineering</ref> Intervening in yet another system we do not fully understand. And is the Copenhagen climate change treaty maybe just another global government power grab? ~500-700 new bureaucracies? New taxes? 2-5 percent levies on GDP? If cap-and-trade is used & “buying allowance” is done with inflatory currencies, does that mean that pollution inflates further too? Is this the new bubble? How many displacements will result due to continued pillaging and plundering?

1. We are learning. Are we? [[Wordsmithing#Sperm_are_Dying.21_Females_to_Outnumber_Males.21|Sperm are Dying! Females to Outnumber Males!]]

=== Overshoot loops and collapses ===

==== Time scales ====
Werner analyzes the environmental challenges that we face over short , intermediate and long-term time scales, noting the increasingly "two-way, nonlinear interactions" (i.e., chaotic or disproportionate interactions) between global (resource exploiting) markets and the "Earth System". These challenges — driven by increasing resource demand — result in a "mismatch between short-time-scale market and political forces driving resource extraction/use and longer-time-scale accommodations of the Earth system to these changes." Human markets and the Earth System, observes Werner, operate along different time scales. <ref>Is Earth F**ked? Dynamical Futility of Global Environmental Management and
Possibilities for Sustainability via Direct Action Activism https://environmentaldynamics.files.wordpress.com/2013/03/werner_agu2012.pdf (abstract)</ref><ref>AGU Scientist Asks, ‘Is Earth F**ked?’ Surprising Answer: Resistance is NOT Futile! http://thinkprogress.org/climate/2012/12/09/1306051/agu-scientist-asks-is-earth-fked-surprising-answer-resistance-is-not-futile/</ref>
==== BAU ====
''The Limits to Growth “standard run” (or business-as-usual, BAU) scenario produced about forty years ago aligns well with historical data that has been updated in this paper. The BAU scenario results in collapse of the global economy and environment (where standards of living fall at rates faster than they have historically risen due to disruption of normal economic functions), subsequently forcing population down. Although the modelled fall in population occurs after about 2030—with death rates rising from 2020 onward, reversing contemporary trends—the general onset of collapse first appears at about 2015 when per capita industrial output begins a sharp decline. Given this imminent timing, a further issue this paper raises is whether the current economic difficulties of the global financial crisis are potentially related to mechanisms of breakdown in the Limits to Growth BAU scenario. In particular, contemporary peak oil issues and analysis of net energy, or energy return on (energy) invested, support the Limits to Growth modelling of resource constraints underlying the collapse.'' <ref>Is global collapse imminent? http://sustainable.unimelb.edu.au/sites/default/files/docs/MSSI-ResearchPaper-4_Turner_2014.pdf</ref>
==== HANDY ====
On the HANDY report (2014) http://climateandcapitalism.com/wp-content/uploads/sites/2/2014/03/Motesharrei-Human-and-Nature-Dynamics.pdf:

''In this paper, we show that two factors can independently lead to collapse: Ecological Strain and Economic Stratification. We also show that there are two routes to such collapses, which we call Type-L and Type-N collapses. Type-N starts with exhaustion of Nature but Type-L results from the disappearance of Labor because Elite consumption does not leave sufficient resources to meet the needs of Commoners. These two distinct routes to a collapse show over-exploitation of Labor, not just of Nature, can cause a societal collapse.''

''We also show that a sustainable steady state, with the population equal to the Carrying Capacity, is reachable in different types of societies. This requires making rational choices for population, depletion, consumption, and stratification. Very importantly, the experiments show that if population does not overshoot carrying capacity by too much, it would still be possible to eventually converge to it. However, if the overshoot is too large, a full collapse would be hard to avoid.''

''The HANDY model shows us that Carrying Capacity (the long-term sustainable level of population at a given level of consumption) is the level of population at the time Accumulated Wealth starts to decline, which makes it easy to estimate it in the model. This is because, if the population at a given level of consumption is below the Carrying Capacity, the total consumption is lower than the level of depletion that Nature can maintain, and therefore Wealth can continue accumulating. However, if the population (at a given level of consumption) is above the Carrying Capacity, the total consumption is not covered by the level of depletion that Nature can maintain, and the Accumulated Wealth decreases.''

''Since we live in an era where we are primarily relying on non-renewable resources, which are by definition unsustainable, the population and its consumption can grow beyond the Carrying Capacity level while still being able to accumulate wealth, because we are rapidly drawing down the natural resources accumulated by Nature over hundreds of millions of years (i.e., fossil fuels).''

''In further modeling we have done, where we model the use of non-renewables, we find that these non-renewables allow population to rise by an order of magnitude higher than with renewables alone. Thus, the use of non-renewables postpones the collapse, but then when the collapse happens, it is much deeper. (Rivas, Motesharrei and Kalnay, 2014, in preparation).''

''Furthermore, as we point out in the article, the consumption of natural resources has both depletion and pollution effects, both of which have to be taken into account on the carrying capacity of the natural system, as the HANDY model does. When non-renewable resources (e.g., fossil fuels) are consumed rapidly over just a short time, as we are doing now, this can introduce a quantity of pollution (e.g., the stored carbon accumulated over hundreds of millions of years) large enough to affect the entire global system (e.g., causing climate change). Thus, the rapid release of vast quantities of previously stored carbon is creating changes in the world's climate system that are in turn impacting back on the human system. In addition, these changes to the global climate can create additional positive feedbacks leading to greater climate change, such as the release of previously stored methane in the now melting permafrost, or the decreasing albedo due to the melting of polar sea ice. So the consumption of non-renewables does not occur without significant effects on the entire system.''

''Carrying Capacity depends on two sets of parameters encompassing properties of both the human system and the natural system. On the socio-economic side, depletion rate, consumption rate, rate of change of population, and degree of inequality can influence the Carrying Capacity. All of these factors can be adjusted by various policies that societies may choose to implement. On the natural system side, Nature Capacity (maximum size of a natural resource) and Nature regeneration rate (rate at which a renewable natural resource can be replenished) can influence the Carrying Capacity. Although these parameters are properties of the natural system, humans can, and do, change them. For example, by replacing parts of a forest with a city, or by over-depleting rivers flowing into a lake and subsequently drying out the lake, we reduce the Nature Capacity.'' <ref>Q&A: when a theoretical article is misinterpreted
Before a paper on the HANDY model was published, findings were taken out of context in some press accounts; here, the authors explain their research http://www.elsevier.com/connect/q-and-a-when-a-theoretical-article-is-misinterpreted</ref>

== Resources ==
=== Original scenario planning concepts ===
* Visioning in planning: is the practice based on sound theory? (The resulting analysis shows that while there is a basis to support some of the assumptions about visioning there are also profound weaknesses in parts of the underlying theory.) http://www.envplan.com/abstract.cgi?id=a3461

=== Systems thinking approach to scenario planning ===
* The Importance of “Wild Card” Scenarios (Discussion paper -- does not represent the views of the US Government) http://www.au.af.mil/au/awc/awcgate/cia/nic2020/dewar_nov6.pdf
* Using Scenarios for Strategic Planning http://www.au.af.mil/au/awc/awcgate/usda/using_scenarios.htm

=== Diagramming of effects ===
* AYWQuartet: How did this happen? http://www.ayequartet.com/articles/how-did-this-happen/ with a short explanation of causal loop diagramming http://www.developerdotstar.com/mag/articles/gray_diagram_of_effects.html

=== Bayesian logic ===
* Bayes’ For Beginners https://www.ualberta.ca/~chrisw/BayesForBeginners.pdf

=== Underminers ===
* Chapter Four – The Principles of Undermining http://underminers.org/the-book/chapter-4/
* Chapter Five – Ground Rules http://underminers.org/the-book/chapter-5/

== Related ==
* [[Timeline merchants of death]]
* [[Timeline masters of the internet]]
* [[Timeline that is soooo 1984 ...]]
* [[Psychological warfare]]
* [[Roleplay]]

== References ==

Surveillance

2015-10-09T14:05:35Z

Lilith2:

A citizenry that’s constantly on guard for secret, unaccountable surveillance is one that’s constantly being remade along the lines the state would prefer. Foucault illustrated this point by reference to a hypothetical prison called the Panopticon. Designed by utilitarian philosopher Jeremy Bentham, the Panopticon is a prison where all cells can be seen from a central tower shielded such that the guards can see out but the prisoners can’t see in.

The prisoners in the Panopticon could thus never know whether they were being surveilled, meaning that they have to, if they want to avoid running the risk of severe punishment, assume that they were being watched at all times. Thus, the Panopticon functioned as an effective tool of social control even when it wasn’t being staffed by a single guard.

This page is an ongoing effort and forever under construction.

[[File:Spy_vs_spy_by_xxweareanonymousxx-d6b9mkt.jpg|484px|thumb|right|http://xxweareanonymousxx.deviantart.com/art/Spy-Vs-Spy-381721997]]

== Technocracy ==
The logic of this system is, in a limited sense, highly logical: past elites were unable to maintain control due to human mistakes and unpredictabilities. The thinking behind a technocracy is that machines make no mistakes (this is a weakness of theirs). The large tech companies have all discussed this openly at large conferences. Money is a symbol of energy, and the erection of the mass surveillance grid is not for the purpose of security or preventing 'terrorism'. <ref>NSA Surveillance is about Control & Leverage, not Security http://www.juancole.com/2014/01/surveillance-leverage-security.html</ref>

=== Targeted surveillance ===

=== Mass surveillance ===

== Legalities ==

The Snowden leaks (see [[timeline masters of the internet]]) revealed a massive surveillance program including interception of email and other internet communications and phone call tapping. Some of it appears illegal, while other documents show the US spying on friendly nations during various international summits, and on its citizens. The programs are enabled by two US laws, the Patriot Act<ref>ACLU: Surveillance under Patriot Act https://www.aclu.org/infographic/surveillance-under-patriot-act</ref> and the FISA Amendments Act (FAA)<ref>FISA Amendments Act (FAA) https://www.aclu.org/faa-foia-documents</ref>, and a side dish called Executive Order 12.333<ref>ACLU: Executive order 12.333 https://www.aclu.org/cases/executive-order-12333-foia-lawsuit</ref>. Although the Foreign Intelligence Surveillance Court oversees such surveillance activities, it operates in secrecy through one-sided procedures that favour the US government.

==== Executive Order 12333 (EO12333) ====
Executive Order 12333 - 1.13 allows the FBI to provide the NSA with "technical assistance" in the United States, i.e. to collect metadata about American citizens by collaborating with "foreign intelligence" and law enforcement services. <ref>Meet the Spies Doing the NSA’s Dirty Work http://foreignpolicy.com/2013/11/21/meet-the-spies-doing-the-nsas-dirty-work/
</ref> By using telecom companies the US government can circumvent controls and accountability that don't apply to companies. All major telecom companies have worked closely and secretly with government spying since their inceptions.

==== Plausible Deniability ====

Plausible deniability began under Central Intelligence Agency (CIA) director Allen Dulles <ref>Spywriter: Plausible deniability http://spywriter.com/robots/plaus_denial.html</ref>.'' The CIA black ops division undertakes dangerous and usually what would be considered illegal missions that are not officially sanctioned by the US administration so that the administration, which usually benefits from such missions, can safely disavow any knowledge of them in the event of their publically uncovered success or failure. The administration is in the position of plausible deniability towards the CIA's actions'' <ref>Urban dictionary: http://www.urbandictionary.com/define.php?term=plausible%20deniability</ref>.

== Intelligence ==

There are six overarching categories that describe the various ways that intelligence is collected: SIGINT, HUMINT, MASINT, GEOINT, OSINT and IMINT. Of these, SIGINT and HUMINT are considered instrumental in the "war on terror".

SIGINT and security procedures are closely coordinated under what is called the UKUSA Community, or "five eyes": Australia, Canada, New Zealand, the United Kingdom, and the United States.

=== Signals Intelligence (SIGINT) ===
Signals intelligence, or SIGINT, is the process of intercepting signals between people (COMINT) or from electronic signals not directly used in communication (ELINT), or communications transmitted electronically such as through radars, radios, and weapon systems. One of the main tools that SIGINT employs is the use of cryptanalysis, which is the process of breaking codes in order to read encrypted information.

The National Security Agency (NSA) is the organisation that is responsible for SIGINT. The NSA SIGINT Strategy 2012-2016 covers all the comsec and crypto initiatives to covertly exploit people, cryptographers, anonymisers, informants, planted spies, security firms, networks, governments, nations <ref>NSA Global SIGINT Power to Generate Profits and Pay http://cryptome.org/2013/11/nsa-power-profit-pay.htm</ref>.

==== Communications Intelligence (COMINT) ====
Communications intelligence deals with voice communications commonly over radios. This is used to gather information such as location and identity. It covers all electronic communications in the US and the world "to ensure national security". The NSA at Ft Meade, Maryland has had the most advanced computers in the world since the 60s. Technology is developed and implemented in secret.

==== Electronic Intelligence (ELINT) ====
ELINT is divided into Technical ELINT (TechELINT) and Operational ELINT (OpELINT). Technical ELINT obtains signal structure, emission characteristics, modes of operation, emitter functions, and weapons systems associations of such emitters as radars, beacons, jammers, and navigational signals for determining its capabilities and role in a larger system as part of electronic warfare; Operational ELINT focuses on locating specific ELINT targets and determining operational patterns of the systems named Electronic Order of Battle (EOB) for supporting military operational planners and tactical military commanders on the battlefield. And ofcourse there are "counter" programs, existing and under development such as Electronic counter-countermeasures (ECCM) in the US and electronic protective measures (EPM) in Europe, Wild Weasel (alias Iron Hand or SEAD), Smart Dust and Wolfpack.

=== Human Intelligence (HUMINT) ===
This is the gathering of information from human sources. Interrogation and elicitation would be classified under this heading.

The Central Intelligence Agency (CIA) is responsible for overseeing the majority of HUMINT operations, although the military (DoD) is often involved in HUMINT as well. Both parties make use of two main tactics, gathering intelligence through interrogations and through conversations with key persons who have access to valuable information. HUMINT sources of information include diplomats, military attaches, prisoners of war, and espionage.

=== Foreign Instrumentation Signals Intelligence (FISINT) ===
TELINT (Telemetry Intelligence), later named FISINT (Foreign Instrumentation Signals Intelligence) used for intercepting, processing, and analysis of foreign telemetry is closely related to TechELINT and conducted by DoD.

== Operations ==
=== NSA ===
The Special Sources Operation (SSO) gathers information from internet communications through arrangements with internet companies under the program codenamed PRISM, and by Upstream collection (cable tapping).

==== PRISM ====
PRISM is part of the overall NSA surveillance effort - a program authorised in the united states under the FISA Amendments Act (FAA), now located in Section 702 of the FISA, that allows the NSA to collect communications of specifically identified foreign targets.

When people in the rest of the world use the net, they are effectively using US-based services, making them a legal target for US intelligence. Supposedly the NSA has no "direct access" to the servers of companies through PRISM - other agencies, like the FBI, collect the data. Emails, chat--video, voice, photos, stored data, VoIP, file transfers, video conferencing, notifications of target activity--logins, etc, online social networking details, and special requests are collected from nine major Silicon Valley technology companies: Facebook, Google, Yahoo, Microsoft, PalTalk, Skype, YouTube, Apple, and AOL. <ref>
What is known about NSA's PRISM program http://electrospaces.blogspot.nl/2014/04/what-is-known-about-nsas-prism-program.html
</ref>

==== Upstream collection ====
FAIRVIEW seems to be the main umbrella program, incorporating many different operations, such as BLARNEY, STORMBREW, and OAKSTAR (yet another umbrella for eight different programs used for collection outside the US), that intercept massive amounts of international Internet traffic by various means, and involves the collection of communications — both their metadata and their content — as they pass through undersea fiber-optic cables in:
* Cooperation with telecommunication companies
* Cooperation with foreign intelligence agencies
* Unilateral cable tapping operations
The five biggest collection programs of the Special Source Operations division are DANCINGOASIS, SPINNERET, MOONLIGHTPATH, INCENSER and AZUREPHOENIX. The largest cable tapping program appears to be DANCINGOASIS (DGO) <ref>NSA's largest cable tapping program: DANCINGOASIS http://electrospaces.blogspot.nl/2014/05/nsas-largest-cable-tapping-program.html
</ref>. It could be that DANCINGOASIS is another umbrella program encompassing various smaller sub-programs.

Upstream collection also includes data from MUSCULAR in a joint operation with the GCHQ.

Internet data collected by Upstream programs can be processed and searched though the XKEYSCORE indexing and analysing system. BOUNDLESS INFORMANT tracks the international intelligence gathering techniques of the NSA.

==== XKEYSCORE ====

XKEYSCORE is reported to receive unfiltered internet communication sessions ('full-take') from WEALTHYCLUSTER2. The session data is stored for a short period of time: content is buffered for 3 to 5 days (sometimes shorter or sometimes longer, depending on the amount of traffic), and metadata for up to 30 days. In other words, XKEYSCORE creates a rolling buffer that is continually being rewritten. This buffering enables analysts to perform federated queries using so-called "soft selectors", like keywords, against the body texts of e-mail and chat messages, digital documents, and spreadsheets in English, Arabic and Chinese. XKEYSCORE also allows analysts to look for the usage of encryption, the use of a VPN or the TOR network, and a number of other things that could lead to a target.

Analysts can create so-called 'fingerprints', rules that contain search terms (especially all the correlated identities of a certain target <ref>Fingerprints and the Phone Dragnet’s Secret “Correlations” Order https://www.emptywheel.net/2014/04/09/fingerprints-and-the-phone-dragnets-secret-correlations-order/
</ref>) that are automatically executed by the system. Some examples of XKeyscore fingerprints were disclosed by German regional television on July 3, 2014 <ref>Example of XKEYSCORE fingerprinting rules http://daserste.ndr.de/panorama/xkeyscorerules100.txt</ref>, who presented them as excerpts of source code.

==== Tailored Access Operations (TAO) ====
The epochal batch of leaks on TAO's hacking unit and ANT division catalog used is typical COMINT. <ref>Leaksource: NSA’s ANT Division Catalog of Exploits for Nearly Every Major Software/Hardware/Firmware http://leaksource.info/2013/12/30/nsas-ant-division-catalog-of-exploits-for-nearly-every-major-software-hardware-firmware/</ref>. See [[Digital threats, detection, protection and (counter) moves#Targeted_surveillance|targeted surveillance threats]] for some examples.

=== DEA ===

==== Hemisphere ====

The Hemisphere Project is coordinated from the Los Angeles Clearinghouse and is funded by the US Office of National Drug Control Policy (ONDCP) and DEA. Hemisphere provides electronic call detail records (CDRs) in response to federal, state, and local administrative/grand jury subpoenas. In effect, it is mass surveillance conducted by US telephone company AT&T in collaboration with the DEA (AT&T supplying DEA officials with metadata from a database of telephone calls dating back to 1987).

* AT&T & Verizon eager to get on their knees for the NSA https://www.propublica.org/article/nsa-spying-relies-on-atts-extreme-willingness-to-help
* How they connected the dots, supporting documents https://www.propublica.org/article/a-trail-of-evidence-leading-to-atts-partnership-with-the-nsa

=== CIA ===

The National Clandestine Service (NCS) is the branch of the CIA responsible for the collection of HUMINT.

=== FBI ===

==== PRISM ====
The FBI has its own capacity to monitor phone, email, chats, etc., in addition to whatever NSA shares with them and seems to be doing quite well obtaining what it needs by collecting all the data everywhere via subpoenas, though there are abundant reasons to worry about control functions in FBI’s bulky databases. PRISM, an NSA program, is apparently coordinated through the FBI. <ref>New insights into the PRISM program http://electrospaces.blogspot.nl/2013/07/new-insights-into-prism-program.html</ref>

==== Data Intercept Technology Unit (DITU) ====
The Data Intercept Technology Unit (DITU) has been described as NSA's Alter Ego in the FBI. It carries out its own signals intelligence operations and is trying to collect huge amounts of email and internet data from U.S. companies — an operation that the NSA once conducted, was reprimanded for, and says it abandoned.
==== Digital Collection System Network (DCSNet) ====
The Digital Collection System Network (DCSNet) is a suite of software that collects, sifts and stores phone numbers, phone calls and text messages. The system directly connects FBI wiretapping outposts around the US to a far-reaching private communications network. DCSNet includes at least three collection components, each running on Windows-based computers:
* DCS-3000 (alias Red Hook) handling pen-registers and trap-and-traces, a type of surveillance that collects signaling information -- primarily the numbers dialed from a telephone -- but no communications content. (Pen registers record outgoing calls; trap-and-traces record incoming calls.)
* DCS-6000 (alias Digital Storm) captures and collects the content of phone calls and text messages for full wiretap orders.
* DCS-5000 is used for targeted surveillance wiretaps (targeting spies or terrorists).

=== DoD ===
==== Defense Information Systems Agency (DISA) ====
The Defense Information Systems Agency (DISA), is a United States Department of Defense (DoD) combat support agency composed of military, federal civilians, and contractors. DISA provides information technology (IT) and communications support to the president, vice president, secretary of defense, the military services, the combatant commands, and any individual or system contributing to the defense of the United States.

=== GCHQ ===

During the 1950’s, the uk government became concerned that emanations could be captured and then reconstructed. Obviously, the emanations from a blender aren’t important (at least not then, now they may become important in the "internet of things"), but emanations from an electric encryption device would be. If the emanations were recorded, interpreted, and then played back on a similar device, it would be extremely easy to reveal the content of an encrypted message. Research showed it was possible to capture emanations from a distance, and as a response, the TEMPEST program was started. <ref>AllYouEverWantedToKnowAboutTempest http://auriea.org/index.pl/AllYouEverWantedToKnowAboutTempest
</ref>

==== MUSCULAR ====
Appears to be a back-door collection from Google and Yahoo private clouds.

== Sharing of data ==

=== PROTON/CRISSCROSS/ICREACH ===
PROTON is a storage and analysis system of telecommunications selectors at the TS/SI/FISA/ORCON/NOFORN level of classification and handling. PROTON is the program name as well as the name of the technology. It has been described as "SAP-like", and is suspected to have started life as a DEA special program.

CRISSCROSS/PROTON is a data sharing system launched in 1990 by the CIA and the DEA. By 1999, the NSA, the Defense Intelligence Agency, and the FBI had gained access to CRISSCROSS and were contributing information to it. As CRISSCROSS continued to expand, it was supplemented with a system called PROTON that enabled analysts to store and examine additional types of data. <ref>PROTON, CLEARWATER and Lexis-Nexis http://cryptome.org/2013/08/proton-clearwater-lexis-nexis.htm</ref>

After 9/11, the NSA came to view CRISSCROSS/PROTON as insufficient, and believed it was time to build a new and more advanced system to radically increase metadata sharing for HUMINT: ICREACH <ref>ICREACH: NSA’s Surveillance Search Engine http://leaksource.info/2014/08/25/icreach-nsa-surveillance-search-engine/
</ref>, now in use by CIA, DEA, DIA, FBI, and NSA. There seems to be no oversight.

=== MUSCULAR ===
MUSCULAR is one of at least four other similar programs that rely on a trusted 2nd party, programs which together are known as WINDSTOP. MUSCULAR is jointly run by the NSA and GCHQ. It operates via an access point outside the United States, and relies on an unnamed telecommunications operator to provide secret access for the NSA and the GCHQ.

== (Counter) moves ==

[[File:Operationtrollthensa.png|480px|thumb|right|http://trollthensa.com/]]

=== Communications privacy folly ===
In reply to a request for comments on Mexico law revisions – Warrantless Real-time Cell phone Geolocation Data Surveillance, Cryptome comments - This is the short version of why there can be no solution to preventing communication systems abuse <ref>Communications Privacy Folly http://cryptome.org/2012/06/comms-folly.htm</ref>.

Knowing that, what if we make spying on us as expensive as possible using '''lawful''' techniques and tools?

=== Understanding and minimising our digital shadows ===
The internet is a great space to explore, learn, speak up, listen and communicate with people across the world. Unfortunately, the internet has also become a space where people who challenge the dominant discourse often find themselves under attack. These attacks can be very personal - enabled by the fact that there is often a lot of personal information about us on the internet.

To strengthen our defences against these kinds of attacks, it’s a good idea to start by our assessing our digital shadows. These shadows - can tell a story about us: who we are, where we live and hang out, what we are interested in, and who our friends and colleagues are. For more on that see [[Step 1#Understanding_and_minimising_our_digital_shadows|Understanding and minimising our digital shadows]].
=== Controlled folly ===
And continued from there, what if we make spying on us as expensive as possible using lawful techniques and tools? And if need be we invent new (GNU and creative commons licensed) techniques and tools? <ref>Portal - 'Still Alive' https://www.youtube.com/watch?v=Y6ljFaKRTrI</ref>

As Cory Doctorow writes: ''There is no legal or technical mechanism by which code that is designed to be modified by its users can co-exist with a rule that says that code must treat its users as adversaries and seek to prevent them from running prohibited code. ''<ref>What David Cameron just proposed would endanger every Briton and destroy the IT industry http://boingboing.net/2015/01/13/what-david-cameron-just-propos.html
</ref>

See [[Scenario planning]] to prepare for some [[roleplay]], for adding a "red copper armour" for online roleplay see [[Digital threats, detection, protection and (counter) moves]], [[Installing linux]] and [[Linux security|securing]] it as best as you can and fits your context and purpose, and see [[Confusing surveillance systems|confusing of surveillance systems]] for teh lulz.

=== Political ecology ===

In my opinion, the predicament we’re in has been woven of threads that include our evolved neuropsychology, our innate cleverness, our general lack of wisdom, and the technological and cultural structures that we have created around us. We may be able to work our way free of this monkey trap, but not unless we gain insight into our own nature, lovingly nurture the wisdom we need to recognize what we’re doing, and the courage to say “No” to some of it. If we can do that, the problems of nuclear power and all the rest will become much less threatening. Can we accomplish such a radical transformation of human consciousness? We are getting a wake-up call. it’s time for us to heed it, and to do a bit of growing up. ~ Slaying The Hydra <ref>Slaying the hydra http://paulchefurka.ca/Hydra.html</ref>

Brian Holmes writes, ''I don’t think any alternative will be possible until certain realities are faced, not just by fringe figures like ourselves but by much broader swathes of society, and that seems unlikely'' <ref>The Californian Reality http://nettime.org/Lists-Archives/nettime-l-1401/msg00064.html</ref>: ''Sadly, I must report that up to now, almost no one has been interested. Left-leaning intellectuals are still preoccupied by individual liberation, minority and sectoral rights claims, the ghosts of working-class struggles, and anarchist longings for direct democracy. All of those have been very important, but none of the current oppositional discourses can marshall the sophistication, depth, durability and power to confront the transnational capitalist class. An alternative is not something that one fabricates on the fly, in a study or an artwork or or a hacklab or an affinity group or a church or a social center, even if all of those can be part of it. To make it real would require a large-scale articulation of theory and practice, extending into mainstream institutions even while outstripping and transforming them. Obviously it's easier said than done, but without saying it you can't get anywhere. The silence of the intellectuals is the new treason of the clercs. ''

''I think the keyword of systemic change already exists: political ecology. There are many people working in that direction. But the universities, cultural systems, professional association (including unions) and press/media apparatuses are still massively captured by the dream of belonging to the transnational capitalist class, or mired in some vague nostalgia for the klarion calls of yesterday's struggles. Meanwhile our old nettime nemesis, the Californian Ideology, has made tremendous forward strides.''

''Just how far will we let it go?'' It's that infamous gut/head conversation and it seems the gut is winning, isn't it?

=== Question the surveillance system ===
Examples that flew by:
* Why do we upgrade firmware? Many HW installations never add functionality.
* Why doesn't Brazil yank the cable splitter on their sovereign ground at Fortaleza and tell users they did?
* Why doesn't Germany seize the Einstein/Castanet RF illuminators on the roofs of the US, Canadian and British embassies?
* Why doesn't Orange file their threatened lawsuit over the SEA-ME-WE4 data theft?
* Why doesn't Google give us an explanation of where and how the Level Three data theft occurred?
* Why doesn't Apple tell us all the ways TAO has been hacking the iOS and if they're still ongoing?
* Why do only 8 of 24,000 attendees to RSA security conference cancel their attendance?

== Resources ==

=== News & watchdogs ===
* Bugged Planet http://buggedplanet.info/
* CanaryWatch https://www.canarywatch.org/
* Cryptome http://cryptome.org/
* AllYouEverWantedToKnowAboutTempest http://auriea.org/index.pl/AllYouEverWantedToKnowAboutTempest
* Lux ex Umbra - monitoring Canadian signals intelligence (SIGINT) activities past and present http://luxexumbra.blogspot.ca/
* NSA observer https://www.nsa-observer.net/
* Top Level Communications http://electrospaces.blogspot.fr/
* Watching Them, Watching Us http://www.spy.org.uk/
* WikiLeaks: The Spy Files https://wikileaks.org/the-spyfiles.html

=== Maps ===
* Mindmeister NSA map https://www.mindmeister.com/fr/308518551/the-national-security-agency-operates-more-than-500-separate-signals-intelligence-platforms-employs-

=== Books ===
* The Shadow Factory, 2008 http://www.bookarmor.com/_files/TSF.pdf

=== Documentaries ===
* All Watched Over by Machines of Loving Grace http://topdocumentaryfilms.com/all-watched-over-by-machines-of-loving-grace/

=== Presentations and articles ===

* Dear NSA, let me take care of your slides http://www.slideshare.net/EmilandDC/dear-nsa-let-me-take-care-ou
* Mind-blowing secrets of NSA's security exploit stockpile revealed at last http://www.theregister.co.uk/2015/09/04/nsa_explains_handling_zerodays/

=== Targeted surveillance ===

* Hacking Team: a zero-day market case study https://tsyrklevich.net/2015/07/22/hacking-team-0day-market/
* NSA QUANTUM Spies with Radio Signals https://cryptome.org/2014/01/nsa-quantum-radio.htm

=== Hardware implants ===
* Getting Terminal Access to a Cisco Linksys E-1000 https://www.trustwave.com/Resources/SpiderLabs-Blog/Getting-Terminal-Access-to-a-Cisco-Linksys-E-1000/
* Oops, I pwned your router https://www.trustwave.com/Resources/SpiderLabs-Blog/Oops,-I-pwned-your-router--Part-One/
* Hackers create spy plug inspired by the NSA's surveillance kit http://www.dailymail.co.uk/sciencetech/article-2920419/When-USBs-attack-Hackers-create-covert-spy-plug-inspired-NSA-s-Cottonmouth-surveillance-kit.html
* Infosec institute: NSA Backdoor Part 2, BULLDOZER: And, Learn How to DIY a NSA Hardware Implant http://resources.infosecinstitute.com/nsa-bios-backdoor-aka-god-mode-malware-part-2-bulldozer/

=== Radio frequency exfiltration ===
* Wireless Data Exfiltration - building and using low cost signal intelligence Tools http://www.slideshare.net/warezjoe/ccdc-2012-wireless-data-exfiltration-building-and-using-low-cost-signal-intelligence-devices-v2
* Getting Started with HackRF and GNU Radio https://github.com/mossmann/hackrf/wiki/Getting-Started-with-HackRF-and-GNU-Radio

=== Infected firmware ===
* Implementing and Detecting a PCI Rootkit http://www.blackhat.com/presentations/bh-dc-07/Heasman/Paper/bh-dc-07-Heasman-WP.pdf
* To Protect And Infect, Part 2 [30c3] https://www.youtube.com/watch?v=vILAlhwUgIU

=== Authorship analysis ===
* An Authorship Analysis: Francis Bacon as Shake-speare: http://fly.hiwaay.net/~paul/outline.html
* The jargon file: http://www.comedia.com/hot/jargon-4.4.6/html/

=== Wiretapping ===
* University of Pennsylvania Researcher Reports JitterBugs Could Turn Your Keyboard Against You, Steal Data http://www.upenn.edu/pennnews/news/university-pennsylvania-researcher-reports-jitterbugs-could-turn-your-keyboard-against-you-stea (2006)

== Related ==

* [[Digital threats, detection, protection and (counter) moves]]
* [[Timeline masters of the internet]]
* [[Covert operations]]
* [[Psychological warfare]]
* [[Game of drones]]
* [[Biometrics]]

== References ==

Linux virtualisation

2015-10-09T14:04:19Z

Lilith2:

Over the years, computers have become sufficiently powerful to use virtualisation to create the illusion of many smaller virtual machines, each running a separate operating system instance. There are several kinds of Virtual Machines (VMs) which provide similar features, but differ in the degree of abstraction and the techniques used for virtualisation.

''OK, so virtualization is the latest hype. Why should I care? Computers are getting cheaper every day...''

== Why use virtualisation? ==
* Consolidation means combining multiple software workloads on one computer system. You can run various virtual machines in order to save money and power (electricity).
* Making moving workloads to a new server easy by installing a host OS onto your new server, then copying over a virtual machine. With virtual machine migration like xen live migration you can even move a virtual machine from one system to another while it is running.
* Testing configurations of new linux distros by creating less resource hungry and low priority virtual machines (VM). Afterwards you can throw the whole VM away instead of having a cluttered system.
* Learning new operating systems and programming languages/database without having to make changes to the base environment.
* Supposedly, if a mail server or any other app gets hacked, only that VM will be under control of the adversary.
* Misbehaving apps (e.g. memory leaks) cannot easily bring down the whole machine (handy for on servers).
* Magically running that old operating system on new hardware. No more critical applications tying us down to old hardware because it's running on a legacy operating system.

== Types of virtualisation ==

The techniques used differ in complexity of implementation, breadth of OS support, performance in comparison with standalone server, and level of access to common resources. For example, VMs have wider scope of usage, but poorer performance. Para-VMs have better performance, but can support fewer OSs because one has to modify the original OS.

Virtualisation on the OS level provides the best performance and scalability compared to other approaches. Virtual environments are usually also much simpler to administer as all of them can be accessed and administered from the host system. Generally, such systems are the best choice for server consolidation.

=== Emulation ===

A software emulator allows computer programs to run on a platform (computer architecture and / or operating system) other than the one for which they were originally written. Unlike simulation, which only attempts to reproduce a program's behavior, emulation attempts to model to various degrees the state of the device being emulated. The virtual machine simulates the complete hardware, allowing an unmodified OS for a completely different CPU to be run. This is also known as Dynamic Recompilation.

=== Paravirtualisation ===

Paravirtualisation is a technique that presents a software interface to virtual machines that is similar but not identical to that of the underlying hardware. This requires operating systems to be explicitly ported to run on top of the virtual machine monitor (VMM) but may enable the VMM itself to be simpler and for the virtual machines that run on it to achieve higher performance. Meaning, this technique requires a VMM, but most of its work is performed in the guest OS code, which in turn is modified (ported) to support this VMM and avoid unnecessary use of privileged instructions. The paravirtualisation technique also enables running different OSs on a single server, but requires them to be ported to «know» they are running under the hypervisor. The paravirtualisation approach is used by Xen and UML (see below in virtualisation software).

=== Native Virtualisation ===

Native virtualisation is a technique where the virtual machine monitor or hypervisor only partially simulates enough hardware to allow an unmodified Operating System to be run in isolation, but the guest Operating System must be designed for the same type of CPU.

=== Operating System-Level Virtualisation ===

Most applications running on a server can easily share a machine with others, if they could be isolated and secured. In most server situations, different operating systems are not required, merely multiple instances of a single operating system. OS-level virtualisation systems have been designed to provide the required isolation and security to run multiple applications or copies of the same OS (but different distributions of the OS) on the same server.

Containers, also known as operating-system level virtualization, do not run virtual machines at all, but simply segregate multiple user space environments from each other, while everything runs under one kernel. Each such partition looks and feels like a real server, from the point of view of its owner. On Unix systems, this technology can be thought of as an advanced extension of the standard chroot mechanism. OpenVZ, Virtuozzo, Linux-VServer, Solaris Zones and FreeBSD Jails are examples of OS-level virtualisation.

Container technologies like Linux VServer, Virtuozzo/OpenVZ and Solaris Zones, typically have a lower degree of isolation, in exchange for lower overhead and more flexible resource use.

== Virtualisation software ==

=== freebsd jail subsystem ===

2001: On most UNIX® systems, <code>root</code> has omnipotent power. This promotes insecurity. If an attacker gained root on a system, he would have every function at his fingertips. In FreeBSD there are sysctls which dilute the power of root, in order to minimize the damage caused by an attacker. Specifically, one of these functions is called secure levels. Similarly, another function which is present from FreeBSD 4.0 and onward, is a utility called jail(8). Jail chroots an environment and sets certain restrictions on processes which are forked within the jail. For example, a jailed process cannot affect processes outside the jail, utilize certain system calls, or inflict any damage on the host environment https://www.freebsd.org/doc/en_US.ISO8859-1/books/arch-handbook/jail.html.

=== uml ===

User Mode Linux (UML) was the first virtualization technology for Linux. User-mode Linux is generally considered to have lower performance than some competing technologies, such as Xen and OpenVZ.

=== openvz ===

OpenVZ is an operating system-level virtualization technology based on the Linux kernel and operating system. OpenVZ is container-based virtualization for Linux. OpenVZ creates multiple secure, isolated Linux containers (otherwise known as VEs or VPSs) on a single physical server enabling better server utilization and ensuring that applications do not conflict http://openvz.org/Main_Page.

=== xen ===

Xen is a virtual machine monitor for 32 / 64 bit Intel / AMD (IA 64) and PowerPC 970 architectures. It allows several guest operating systems to be executed on the same computer hardware concurrently. XEN is included with most popular Linux distributions such as Debian, Ubuntu, CentOS, RHEL, Fedora and many others http://www.xenproject.org/.

=== kvm ===

Kernel-based Virtual Machine (KVM) is a Linux kernel virtualisation infrastructure. KVM currently supports native virtualization using Intel VT or AMD-V. A wide variety of guest operating systems work with KVM, including many flavours of Linux, BSD, Solaris, and Windows etc. KVM is included with Debian, OpenSuse and other Linux distributions http://www.linux-kvm.org/page/Main_Page.

=== qemu ===

QEMU is a generic and open source machine emulator and virtualiser. When used as a machine emulator, QEMU can run OSes and programs made for one machine (e.g. an ARM board) on a different machine (e.g. your own PC). By using dynamic translation, it achieves very good performance. When used as a virtualizer, QEMU achieves near native performances by executing the guest code directly on the host CPU. QEMU supports virtualisation when executing under the Xen hypervisor or using the KVM kernel module in Linux. When using KVM, QEMU can virtualize x86, server and embedded PowerPC, and S390 guests http://wiki.qemu.org/Index.html.

Qemu (GPL v2) emulates a full computer without relying on trap & emulate by scanning the instructions that the guest is about to run, making sure there are no privileged instructions on the page and replacing those instructions with alternatives if necessary. Because there may be a debugger running inside the guest, unchanged code pages need to be marked read-only and changed code pages need to be duplicated.

=== linux-vserver ===
Linux-VServer is a virtual private server implementation done by adding operating system-level virtualization capabilities to the Linux kernel. The Linux-VServer technology is a soft partitioning concept based on Security Contexts which permits the creation of many independent Virtual Private Servers (VPS) that run simultaneously on a single physical server at full speed, efficiently sharing hardware resources http://linux-vserver.org.

=== virtualbox ===

VirtualBox is an x86 virtualization software package, developed by Sun Microsystems as part of its Sun xVM virtualization platform. Supported host operating systems include Linux, Mac OS X, OS/2 Warp, Windows XP or Vista, and Solaris, while supported guest operating systems include FreeBSD, Linux, OpenBSD, OS/2 Warp, Windows and Solaris https://www.virtualbox.org/.

==== Installing ====

==== Usage ====

* Setting up a virtualbox VM for chaining for [[Anonymising_your_traffic_with_linux#Chaining|anonymising your traffic with linux]].
* Testing devuan and voidlinux suitability for [[making our own linux images]].

=== bochs ===

Bochs is a portable x86 and AMD64 PC emulator and debugger. Many guest operating systems can be run using the emulator including DOS, several versions of Microsoft Windows, BSDs, Linux, AmigaOS, Rhapsody and MorphOS. Bochs can run on many host operating systems, like Windows, Windows Mobile, Linux and Mac OS X.

== Resources ==

* Table comparing the features and performance of the various virtualisation technologies available for Linux: http://virt.kernelnewbies.org/TechComparison

== Related ==

* [[Captivating capital and copyfighting]]
* [[A typical linux distribution]]
* [[Linux development process]]
* [[Making our own linux images]]
* [[Installing linux]]
* [[Linux applications]]

[[Category:Resources]]

Protest threats, detection, protection and (counter) moves

2015-10-09T14:03:12Z

Lilith2: /* Related */

This page lists theoretical defenses for selected groups of weapons used by police or military on civilians. This is just a thought experiment covering (theoretical) defenses against these attacks and not intended to spread fear, uncertainty or doubt about dictator and police states. The tables were initially filled with items from excited delirium (a protestor’s guide to “less-lethal” police weaponry) of 2008, then further investigated and added to.

The government just loves technology. They always assume that every problem —whether social, martial, economic, or ecological— is just waiting for some magic device or operation to solve it. This is another weakness of theirs. These tables are intended for educational purposes such as learning magic device threat modeling. And while the information is accurate to our knowledge (it was in 2008) we don't have any first hand experience with more than a handful of these weapons. Do your own research and don't ever assume you are invulnerable.

Ideally we discuss these attacks and counter moves without the police being privy to those discussion (as in, not on a public wiki), but that would make spreading possible counter moves too slow for the less-than-lethal development cycle (arms race) and effectively exclude non-activists from being able to defend themselves to some extent, so the next best solution in this context is to discuss everything in the open, rather than not discussing them at all.

== Electrical weapons ==

{| class="wikitable"
|-
! Weapon !! More information !! Attacks/Impact !! Protection
|-
| Stun guns (tasers)
|| Stun guns are less-lethal weapons that use electricity to hurt or incapacitate their victims. Both stinger and taser have two modes: 'projectile' and 'drive stun'.
|| In projectile mode, both use Electro-Muscular Disruption (EMD) technology to cause Neuromuscular Incapacitation (NMI) in their targets. After a half-second, the pain is nearly unbearable and causes muscle contractions. After 2-3 seconds, the subject is dazed and dropped to the ground. More than 3 seconds completely disorients a person and causes them to stay on the ground, risking breathing impairment and heart damage.

In drive stun mode, the stun gun runs electricity between two points in the gun itself and becomes, essentially, a cattle prod. This can cause burning, intense pain, and scarring, but is a pain-compliance weapon (as compared to the projectile mode, which will take anyone down, regardless of their pain tolerance).
|| With the newer tasers, you can’t just allow the prongs to not penetrate into your skin because of their “shaped pulses” that somehow shock you through clothing. It’s been suggested that you can’t just complete the prongs circuit, because if the prongs are in you, you’ll get shocked and/or burned.

While we’ve found no information on DIY methods being tested, we did run across the commercial product announced early in 2008, only available to law enforcement, military, and manufacturers of products for those two categories. But their patent #7284280 (http://www.patentgenius.com/patent/7284280.html) is online and we can learn from that.
|-
| Stun shield || Stinger Systems makes a device called "The Ice Shield", an electrically charged riot shield. They come both concave and convex, angled towards or away from the bearer.

''They may have become less common since a malfunctioning one allegedly killed a prison guard.''
|| It is unclear if they operate on a purely pain-compliance principle or if they will actually lock up the victims muscular system. || Short contacts on shield.
|-
| Taser XRep
|| These are basically wireless TASER darts that are fired out of a 12ga shotgun. It has the same NMI as a handheld taser. As soon as the shot leaves the shell, a ripcord activates it and it has 20 seconds of life. Its velocity is 260’ per second and it is intended to be used at ranges of up to 65’. It has 3 fins that deploy as fletching after firing and 4 prongs that stick into you on impact.
|| After impact, part of the body of the thing falls away, still attached by a cord, and 6 more electric barbs spread the shock out further on your body. If you grab at the projectile while it’s in you, the "reflex engagement electrodes" will send the electricity through your hand and arm as well, making it all that much the worse for you. It even has a damn computer in it to make decisions about how to spread out the shock between all of its nasty points to make your life as crappy as possible.
|| Same as for stun guns.
|}

== Chemical weapons ==
OC, CS, and CN - the three main chemicals of choice for law-enforcement use - are very different chemically, but affect people in similar ways http://emedicine.medscape.com/article/833315-overview

Prevention: The obvious thing is to protect your eyes and respiratory system against particulates and "organic vapors", and there are two basic ways to achieve this: A full-face respirator or gas mask, and a half-face respirator paired with goggles. See Protection From Riot Control Agents http://buggrit.com/gas-masks.html.
{| class="wikitable"
|-
! Weapon !! More information !! Effect !! Treatment
|-
| CS

2-chlorobenzalmalononitrile - C10H5ClN2
|| Odds are, if a canister starts spewing white gas, it’s CS gas. This is by far the most common "teargas" for controlling large crowds. CS gas is a solid (and not a gas), suspended in the air and eventually settling to white powder.

''Banned in warfare by the 1997 Chemical Weapons Convention, although that is mostly to keep the other side from reacting with more deadly chemical weapons. (This didn’t keep Blackwater from deploying it in Iraq a few years ago, but that’s a different story, or is it?)''
|| It reacts with water on the skin and within 20 seconds causes a burning sensation (particularly in the mucus membranes, like the eyes) as well as uncontrollable shutting of the eyes. It can make your throat burn and tears pour out of your ducts, it slows your heart and ups your blood pressure, slows your breathing, and it can cut off circulation to your outer body. In some people it causes dizziness. In some people it causes contact dermatitis, with blisters and crustiness that can last for months (though it usually doesn’t). Mostly, the ill effects go away pretty shortly after reaching fresh air, and you’re pretty much fine after 20 minutes.

Recent studies have shown CS gas to mutate chromosomes, cause miscarriages, and mess up your liver and heart
|| CS contamination can be removed by washing with soap and water, or an alkaline solution of water and 5% sodium bisulfite. Fill a bottle and take with you.
|-
| CN

chloroacetophenone - C8H7ClO
|| More toxic but less potent than CS gas, this is the active ingredient in the older “mace” self-defense sprays. (Mace, a brand name, now usually uses pepper spray instead).

''CN is used far less frequently than CS. This is the original "teargas," but has mostly fallen out of favor. ''
|| Like CS gas, this compound irritates the mucous membranes (oral, nasal, conjunctival and tracheobronchial). Sometimes it can give rise to more generalized reactions such as syncope, temporary loss of balance and orientation. More rarely, cutaneous irritating outbreaks have been observed and allergic contact permanent dermatitis.
|| Pepper spray and teargas http://medic.wikia.com/wiki/Pepper_spray_and_tear_gas
|-
| CR

dibenzoxazepine - C13H9NO
|| The nastiest teargas, CR gas is a lachrymatory agent 6-10 times stronger than CS. Normally a solid (melting point 73 ̊c), it is pale yellow and has a slight odor of pepper.

''The US claims to not use it in protest situations. It saw quite a bit of use in South Africa in the 1980s and, having been developed by the British in the first place, was used on Irish dissidents in the 1970s.''
|| In enclosed spaces, it can be lethal (from asphyxiation and swelling of the lungs). Causes temporary blindness and, occasionally, complete incapacitation. In addition to normal teargas crap, CR gas causes severe skin irritation, persists on surfaces for up to around 60 days, and is thought to be carcinogenic. Areas of skin affected by CR gas, even after thorough cleaning, can continue to hurt if exposed to air for 24 hours or longer.
||
|-
| OC
Oleoresin Capsicum
|| Alias pepper spray. OC is a compound of capsaicin suspended in some kind of agent, usually propylene glycol. Capsaicin, the active ingredient, is derived from various chili peppers. This solution is then pressurized for use in aerosol sprays.

''As a chemical weapon, it is banned in warfare. Cops love it, of course.''
|| Pepper spray is an inflammatory and will generally fuck your shit up. If it gets in your eyes, it will burn like hell for 30-45 minutes (untreated) and continue to sting for hours. If it gets into your mouth or nose it will get into your lungs and make breathing painful and hard. If it gets on your skin, your skin will begin to hurt. While there’s no evidence of anyone dying directly from pepper spray, the “difficulty breathing” bit has aided in the death of dozens of arrestees who were further restrained in such a way as to keep them from breathing properly.
|| Pepper spray and tear gas http://medic.wikia.com/wiki/Pepper_spray_and_tear_gas
OC Trials http://medic.wikia.com/wiki/OC_trials
|-
| PAVA
Pelargonic acid vanillylamide (desmethyldihydrocapsaicin)
|| PAVA is a synthetic pepper spray also derived from capsicum.

''The police in England apparently use it instead of OC, and the company PepperBall uses it in their pepperballs.''
|| It’s supposedly even more badass and hot and whatnot than OC.
|| Pepper spray and tear gas http://medic.wikia.com/wiki/Pepper_spray_and_tear_gas
|-
| WP

White phosphorus
|| White phosphorus is a material made from a common allotrope of the chemical element phosphorus that is used in smoke, tracer, illumination and incendiary munitions.

More on https://opgazalab.wordpress.com/2014/09/16/what-is-white-phosphorus/

''It has been reported used in Iraq (2004), the Israel–Lebanon conflict (2006), Gaza War (2008–2009), Afghanistan (2009), Yemen (2009), Libya (2011) and recently in the Ukraine (2014–2015)''
|| White phosphorus can cause injuries and death in three ways: by burning deep into tissue, by being inhaled as a smoke, and by being ingested. Extensive exposure by burning and ingestion is fatal.
|| White Phosphorus Exposure Treatment & Management http://emedicine.medscape.com/article/833585-treatment
|}

== Impact weapons ==
Build armour protecting your head, collar and shoulder, and kidneys and knees. Motocross armor is pretty amazing stuff, light and fairly low-profile under other clothing. Reinforce a banner to double as lightweight shield.

=== Launchers ===

{| class="wikitable"
|-
! Weapon !! More information !!
|-
| Riot guns
|| The 37mm grenade launchers are most often smoothbore (not rifled: they don’t make their projectile spin, which means they are less accurate). The 40mm grenade launchers look the same, but most are rifled, making them more accurate and also illegal for civilian use. Often at protests you will see what look like gigantic revolvers: these are called “multi-launchers” and come in either 37 or 40mm. They can fire all five or six shots in about three seconds. These grenade launchers can fire chemical weapon canisters, baton rounds, muzzle-blasts, rubber balls, or even flares and, of course, grenades. See the "Projectiles" section for more information.

Then there are 12 gauge shotguns. The short (14”-20”) barrel shotguns that police carry are called riot shotguns and they differ from military shotguns mostly in name, barrel-length, and stated purpose. A large number of less-lethal rounds have been designed to fit into a 12ga, although most prevalent are rubber balls and bean bag rounds. Because these shotguns can also fire traditional ammunition, some police agencies have color-coded their shotguns in neon colors to distinguish that they are intended to be loaded with less-lethal ammunition. And finally (of the common varieties) are pepperball guns.
|-
| Sprays
|| Chemical weapon sprays are used to spit out CS, CN, OC, and PAVA gasses at crowds. These come in all varieties, from the concealable "undercover" models to ones that look like small fire extinguishers, to converted flame-throwers with backpack fuel supplies.

More important than the actual sprayer is its spray pattern. There are five major spray patterns: stream (a thin stream aimed at individuals' eyes), foam (pepper spray encapsulated in a surfactant and turned into a foam), cone (a "fire extinguisher" spray that spits out in a widening cone and is used to soak entire lines of protestors, a portland police favorite), fog (the logical conclusion of this progression being a fog that gets pretty much everyone), and microspin (a rifled stream spreading out into an oval that is intended to envelop the face).
|}

=== Projectiles ===

{| class="wikitable"
|-
! Weapon !! More information
|-
| Chemical weapon canisters
|| There are three basic types of gas canisters: Regular old pyrotechnic gas canisters, which are gun-fired grenades that expel a gas (usually CS, but CN and OC are available as well) in the same manner as a smoke grenade. These are cylinders of sheet steel with emission holes at the top and bottom; Scattershot canisters that split into 3-5 separate canisters upon firing; and ferret rounds, which are unlikely to be encountered in protest situations as they can easily kill a person.
|-
| Muzzle blasts
|| A muzzle-blast is a short-range blast of CS, CN, or OC gas directly from the barrel of riot gun (12ga, 37mm, or 40mm). They appear to have a maximum effective range of between 10 and 30 feet.
|-
| Rubber balls
|| Rubber balls are packed into shells and canisters and then shot out in the way that buckshot is fired from a shotgun. These are fired from 12ga, 37mm, or 40mm, and come in multiple calibers (representing the size of the balls). They are not intended for single targets but instead for pain compliance on an entire crowd. They are supposed to be either skip-fired (fired at the ground and intended to bounce) or fired at a low trajectory.
|-
| Baton rounds
|| Baton rounds are big thick discs or cylinders of wood, foam, rubber, or plastic and can cause fairly grievous bodily harm. They are often fired at joints to disable protestors. It doesn’t matter how tough you are, a point-blank wooden dowel fired at your knee is going to keep you from running. Usually, however, long-term effects are limited to severe bruising and pain.
|-
| Bean bags
|| Used at short to medium range (20-30’) and fired directly, bean bags are nylon bags filled with silica sand. Occasional deaths have resulted when subjects are shot in the chest or neck.
|-
| Pepperballs
|| Pepperballs are essentially paintballs filled with PAVA and fired from specialized launchers (see above). The live ammunition balls are red. Green balls are paintballs and are used for identification purposes. Solid white balls are intended to shatter (and shatter themselves upon impact with) glass, intended for use against suspects in cars.
|-
| Bolo
|| These are 3 rubber balls connected by 12” of cord out of a 12ga shotgun. It has an effective range of 20-40 yards and is probably pretty rare. Used for capturing a running target.
|-
| Flash-bang
|| Flash-bangs are distraction grenades that are either thrown or launched from a riot gun. They spit out a 175db bang (plenty loud enough to damage your hearing) and flash so brightly that your retinas don’t take in anything else for about 5 seconds.
|}

=== Melee ===

{| class="wikitable"
|-
! Weapon !! More information
|-
| Batons
|| Batons come in straight, solid batons made of wood, plastic, and metal and in collapsible straight batons (often called asps). Main targets are head, collarbones, hands, wrists, elbows, knees.
|-
| Close quarter weapons
|| There are a few fist weapons that police occasionally carry. Back in the day they used yawara sticks, which were basically thick pieces of wood, plastic or metal, about the size and shape of a tagging marker. These protruded slightly from the fist. Nowadays a slightly thinner variety, the kubotan, is more common. Another model of a similar concept is the plastic Talon, which has a clip and is worn on the collar. All of these are used to deliver pain compliance holds on resisting subjects.
|-
| Orcutt police nunchaku
|| Apparently, about 200 police departments in the USA (including the Corrections Dept. of Denver) use the OPN against unarmed, resistant subjects. It’s basically a nunchaku that is used for holds and control rather than for impact. All of the parts are made to have some give, and the handles actually bend a bit to avoid breaking bones.
|}

== Foray into a weapons future ==

{| class="wikitable"
|-
! Weapon !! More information
|-
| ADS (Pain ray)
|| The ADS is a dish mounted to the top of a humvee that shoots out a focused, invisible ray of 95ghz millimeter waves (microwaves) that penetrates the first 1/64” of your skin and causes unbearable pain within two seconds by heating the surface of your skin to around 130 ̊F. The military claims that it takes about 250 seconds before it would actually burn the surface of your skin. The effective range is around 500 yards.
|-
| Water cannon
|| The modern riot-control water cannon has about a 2000-gallon tank and delivers around 250 gallons of water a minute. Modern water cannon trucks are controlled from within the cab with a joystick so that the operator is not exposed. The force is enough to knock a person over and they have been known to cause internal injuries.

''Water cannon have fallen out of vogue with police in the USA, primarily due to the negative media attention they received in the civil rights era. English and Indonesian police have, in the past, added dye to the water to identify protestors, but this isn’t common practice.''
|-
| LRAD
|| LRAD, can shoot loud, directed sound out to about 300 meters. Supposedly, they are intended for conveying messages and such, but ofcourse they are now anti-demonstrator devices. They emit a loud (120db in normal use, up to 151db to those nearby when safeties are overridden) high-pitched screech that disorients and upsets and hurts the listener. Anyone too close to one of these can certainly have their hearing permanently impaired. The beam goes out covering about a 30 degree angle. Can be counteracted, to a certain degree, with earplugs.

''These were used by the NYPD during the 2004 RNC.''
|-
| Dazzler
|| The only laser guns in production are meant to blind people and they are banned by international law. Of course, the US uses them in Iraq. They operate by shooting lasers of specific or varying frequencies into the target’s face, blinding them.
|-
| Pulsed energy projectile
|| The military is developing a flash-bang infrared laser: this thing plasmifies the first thing it touches, which in this case will be the top of your skin or perhaps your clothing, resulting in a loud bang and a shockwave that will knock you over. Lethal varieties have been crafted as well.
|-
| Vortex ring gun
|| In 2006 they began to develop the vortex ring gun, the science of which seems to be above my head, but basically it shoots a ring of high-pressure air (a vortex ring) that can knock you over at, so far, 10 meters with a single shot. They are also experimenting with having it fire at specific rates so as to cause resonance in your body to fuck your shit up even more. It will be fired with an adaptor for a 40mm grenade launcher, most likely.
|-
| Malodorant
|| Stink bombs have been used in warfare since at least WWII and the US government is currently researching their use for riot control, but I’ve found no evidence of their use. For the most part, they are banned in the way that most chemical weapons are banned.
|-
| Netgun
|| There are netguns in the world that shoot nets. You get entangled in the net. They’ve been rumored to be in use by the cops but I have no proof of the matter. The one I could find the most information on was the NET-2000 Shooting Net Rod that looks a hell of a lot like a big old clanky flashlight. It shoots out a 52sq. ft. nylon net with compressed air, with an effective range of about 50’.
Electrical and sticky nets are under development for military and law enforcement use, but little information is available.
|-
| Modular crowd control munition
|| What the world needed, the government realized, was a less-lethal version of the claymore mine. So they made this thing, which so far is only in military hands. Basically, it shoots out 600
rubber balls, 60 degrees of coverage with an effective range of 5-30 meters.
|-
| LED incapacitator
|| Intelligent Optical Systems is designing these multi-color flashlights that work by flashing colors so fast that it makes your head spin. In fact, it makes you throw up. It’s been dubbed the “puke saber” by some and the “sick light” by others. They have a built-in rangefinder that detects the nearest set of eyeballs and determines frequency.
|}

== Covert operations ==

If an officer is preparing to go undercover in an activist group, there is nothing more convincing than news footage of the undercover officer throwing rocks at uniformed cops or cussing at them before getting arrested. Found that on a great article: How to Deal with Agent Provocateurs and Undercover Cops at Protests http://theantimedia.org/dealing-with-undercover-cops-agent-provocateurs-protests/ (valid for US at least) and https://www.youtube.com/watch?v=H8yfDX0Myvw (Europe).

{| class="wikitable"
|-
! Covert operation !! Purpose !! (Counter) move(s)
|-
| Surveillance || Even though spying on peaceful protesters is a wasteful use of resources, it is common practice. When gathering intelligence, officers are attempting to determine exactly what the protest’s next move will be, who the leaders are, and whether or not the group or crowd has people prepared for violence. || Don’t “burn” the officer. Don’t let the officer know you are aware that they are law enforcement. When nations discover foreign intelligence officers at work, they typically don’t arrest them. They begin to feed them incorrect information. Remember what the officer’s intent is. Intent is the key element in intelligence and counter-intelligence operations.

The officer’s intent is determining who the leaders are. Talk about a nonexistent person. Describe where he lives and give them all the information they need to find this person. The department will waste tons of resources trying to find out about a person that doesn’t exist. Make sure to include that he lives entirely “off the grid” or that he is “really paranoid.” When they can’t find out about the person, it confirms the intelligence they already gathered and they become certain they are on the right track. You have successfully exploited the department’s confirmation bias, and will wear down their intelligence apparatus as they track false leads.

If you are really committed, you could become “friends” with the officer after the protest and continue to feed the department bogus information about nonexistent protests or groups. Eventually the officer’s superiors will grow tired of the cop’s bad intel and pull him off of the assignment.
|-
| Incitement || Officers will often incite the crowd in an attempt to justify arrest and justify their surveillance and anti-terrorism budgets. They will quietly encourage protesters to break the law and then be ready to testify once those protesters are arrested for the activity they encouraged.
<nowiki> </nowiki>Some law enforcement and intelligence agencies have made a habit of this. See for example the Anatomy of an FBI Terror Plot http://www.defendingdissent.org/now/issues/dismantling-the-police-state/manufactured-crimes-how-the-fbi-creates-terror-plots/anatomy-of-an-fbi-terror-plot/
|| One advice (for the US) has been: Don’t “burn” the officer (but maybe do warn a few people you trust to investigate together quietly, else you are doing their work for them in dividing the movement). Don’t let the officer know you are aware of their profession. The goal of these operations is to get an activist to commit a crime in the presence of law enforcement. They are allowed to do this as long as they don’t entice someone into doing something they wouldn’t do otherwise. If you believe an officer is doing this, photograph him and send it to the press immediately with an explanation. Then (where legal), record the officer secretly and allow him to encourage you to break windows, or whatever he is trying to encourage. Explain that you would never do that, if he continues to attempt to encourage you; walk away, distribute the video to friends for safe keeping, and then submit it to the FBI Civil Rights Division as well as the local press. You have proven attempted entrapment. The test here is whether the officer was inciting you to do something you wouldn’t have done without the officer’s encouragement.
|-
| Developing a cover
|| If an officer is preparing to go undercover in an activist group, there is nothing more convincing than news footage of the undercover officer throwing rocks at uniformed cops or cussing at them before getting arrested. || The officer is only at the protest to get arrested so that later when he shows up at an organization’s meetings, he has established credibility. There isn’t much that can be done to stop the operation, the key is to know how to spot it when it happens. Generally speaking, the wildest people at a protest are the youngest. If a person in their mid-twenties or older instigates violence with the police or is acting in a way that will overtly cause their arrest, they might be attempting to establish a cover. If the person “just moved to town,” the likelihood is even greater. If at later meetings the person that was out of control and got arrested reveals themselves to be relatively calm and very attentive to details, another warning flag should be raised. It is very rare for somebody in their mid-twenties or later that is just suddenly become wild and violent, if the person has no previous footage of them engaging in similar behavior or lacks an arrest record, the likelihood of it actually being an activist decreases immensely. Outspoken people are well known; use social media to see if the person is known within the movement. If the person is unknown within the movement and their only claim to legitimacy is an arrest at a protest, you are most likely dealing with an undercover officer sent to gather intelligence on peaceful protesters.
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline merchants of death]]
* [[Timeline masters of the internet]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

Digital threats, detection, protection and (counter) moves

2015-10-09T14:02:45Z

Lilith2: /* Related */

== Social engineering ==
These tables were started off from http://www.itsecurity.be/social-engineering-what-is-it-and-how-to-defend-yourself and then added to. I started off from a to gendersec external source to increase chances of finding that which we may be overlooking.

=== Internal threats ===

{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Power of scarcity
|| Scarcity is when people are told something they need or want has limited availability and to get it they must comply with a certain attitude or action. Many times the desired behavior is not even spoken, but the way it is conveyed is by showing people who are acting "properly" getting rewards.

Mental shortcuts are often beneficial because they enable us to more efficiently navigate the complexity of life. But the power of scarcity can also mislead us. If some people get something not everyone else has (energy, food, water), it makes them feel "special", "unique", "high status", and gives a "position" to defend. And fear of loss of that is an attachment.

An example is a government, in this case South Africa, taking something necessary to life, and making it "scarce" and available only to supporters — a malicious, but very effective, manipulation tactic: http://www.social-engineer.org/wiki/archives/Governments/Governments-FoodElectionWeapon.html See [[timeline merchants of death]] for more examples.

|| Scarcity complicates Gut/Head decision processes because, often, there is only a brief window of opportunity in which we can select something that is in scarce supply. When we can see the supply diminishing before our eyes, we feel particularly compelled to take action.

Each context and scarcity is different, but mind not to rule out the head too quickly in [[Scenario planning#Gut.2C head.2C and decision making|"Gut versus Head"]] decisions.
|-
| Fear of authority || Many people are apprehensive in the presence of someone they perceive to be an authority figure, it is not that person they are apprehensive about but most likely the position and power of the person that intimidates them.
<nowiki> </nowiki>The attackers take on roles of authority figures such as law enforcement officers or high-ranking company officials to extract sensitive information from the victims.
|| Protect yourself with learning about [[Self-dox#Elicitation|elicitation]] (in roleplay settings) and then, time for some controlled folly!
|-
| Desire to be helpful
|| People in their desire to be helpful and to solve other peoples queries, give out a lot of information that otherwise should not be. || Do not disclose information to an outsider as it could give an attacker a chance to get unauthorised access.
|-
| Laziness || All of us have come across some job that requires us to do only a specified set of activities and nothing more. This causes boredom to the person who performs the same task repeatedly on a daily basis and over time the "bored" learn "shortcuts" to do the tasks using minimal effort while meeting targets. This leads to a laid back attitude and becoming susceptible to attackers who target such individuals knowing they can get the required information with much ease. || Find ways of keeping yourself alert. Go do something else. You are becoming a sitting duck.
|-
| Ego || Many a times, the attacker makes a person more emotionally sure of himself/herself, thus removing the logical awareness of the security breach that is occurring. The result is that the person being hacked senses no harm in providing whatever it is that the attacker is requesting. The reason that such an attack succeeds is that the attacker is a receptive audience for victims to display how much knowledge they have. || The ego isn’t going to go away and whether it makes you vulnerable depends on the context. A bit of swagger based on achievements can be quite okay now and then! In some other contexts allowing your ego to call the shots can be a problem.

The simplest detector is asking yourself: Do you feel superior (or inferior) to others? If yes, then your ego is in control. Depending on context choose "not now, not here, not with this" or "do some counter-intelligence" or "Arrrrr, and a bottle of rum!"
|-
| Excitement of victory & fear of loss || Yes! An opportunity! Easy money!!! Out of excitement security is switched off, links are clicked and documents downloaded that turn out to be corrupted (and contained malware that allows the email sender to gain remote access to the machine, or someone's funds, or data, or ...).

Or, ''"You have won 1 Million Dollars and to claim the winning amount, deposit $75,000 in Account number: XXXXXX in 10 days from receiving this e-mail, failing to which the winning amount would be declared unclaimed and there would be a nee lucky-draw to decide the next winner"''

An example is the 419 scams many people fell for http://www.social-engineer.org/wiki/archives/ConMen/ConMen-Scam-NigerianFee.html
|| It is another ego thing isn't it?
|-
| Insufficient knowledge || People with insufficient knowledge can easily be exploited by creating a sense of urgency and not allowing much time to think and understanding the fact that they are under attack. || Gather (and spread) knowledge in a relaxed way. Stay with your own timing. Do the best you can.
|}

=== External threats ===
{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Shoulder surfing || Shoulder surfing is a security attack where-in, the attacker uses observational techniques, such as looking over someone's shoulder, to get information while they are performing some action that involves explicit usage of sensitive, visible information. This can be performed at a close range as well as at a long range using binoculars or other vision-enhancing devices. || Don't work on anything important in public.
|-
| Dumpster diving || Going through the trash can yield one of the most lucrative payoffs for information gathering. People often throw away invoices, notices, letters, CDs, computers, USB keys, and a plethora of other devices and reports that can truly give amazing amounts of information. The attacker can use these items to get a huge amount of information about people, organisations they participate in, and network structure.

Some people shred documents but some types of shredding can be thwarted with a little time and patience and some tape.
|| Mind what you throw away where and how.

Using a shredder that shreds both directions into a fine minced mess makes taping documents back together nearly impossible. Or incinerate. Or both.
|-
| Role playing || Role playing is one of the key weapons for a social engineer. It involves persuading or gathering information through the use of online chat sessions, emails, phones or any other method that you use to interact online with others, and in which the social engineer plays the role of a helpdesk or technician, helplessness, or whatever may work in that context to get targets to divulge confidential information. || Don't "burn" the social engineer. Apply controlled folly instead, and then let's see who is the cat here and who the mouse: Feed them incorrect information. Remember what the intent is. Intent is the key element in intelligence and counter-intelligence operations.
|-
| Trojan horses || It is one of the most predominant methods currently used by online criminals and intelligence agencies that involve tricking victims into downloading a malicious file to their machine, which on execution creates a backdoor in the machine that can be used by the attacker any time in the future and thus having complete access of the victim's machine. Compromising a browser is relatively easy and it is cross-platform, hence an often chosen attack vector. <ref>Meet The Hackers Who Sell Spies The Tools To Crack Your PC (And Get Paid Six-Figure Fees) http://www.forbes.com/sites/andygreenberg/2012/03/21/meet-the-hackers-who-sell-spies-the-tools-to-crack-your-pc-and-get-paid-six-figure-fees/</ref>.
<nowiki> </nowiki>Among all of the NSA hacking operations exposed by whistleblower Edward Snowden over the last two years, one in particular has stood out for its sophistication and stealthiness. Known as Quantum Insert, the man-on-the-side hacking technique has been used to great effect since 2005 by the NSA and its partner spy agency, Britain’s GCHQ, to hack into high-value, hard-to-reach systems and implant malware.

|| Detection focuses on identifying anomalies in the data packets that get sent to a victim’s browser client when the browser attempts to access web pages http://blog.fox-it.com/2015/04/20/deep-dive-into-quantum-insert/

For more such tools run by intelligence agency for targeted surveillance see [[#Infected_firmware|infected firmware]] below.
|-
| Phishing || Phishing is the act of creating and using websites and e-mails designed to look like those of well-known legitimate businesses, financial institutions and government agencies to deceive receivers into disclosing their personal information.
|| These are semantic attacks.
* Be suspicious of any email with urgent requests for personal (financial) information.
* Set your email client to receive email in plaintext, never html format.
* Check any url in the email carefully. It can be just one letter different or missing.
* If phishy, report it.
|-
| Information on websites, online forums and social media|| Huge amounts of information regarding organisational structures (formal and informal), email adresses, phone numbers, are all available publicly on websites, forums and social media. This information can be used by the attacker to refine his approach and create a plan on whom to target and methods most likely to succeed. || Several moves are possible (not necessarily exclusive):
* Delete your social media presence
* Use different identities for your various social contexts
|-
| Reverse social engineering || A reverse social engineering attack is an attack in which an attacker convinces a target that he or she has a problem or might have a certain problem in the future and that the attacker, is ready to help solve the problem.

Reverse social engineering involves three parts:
* Sabotage: After the attacker gains a simple access to the system, he corrupts the system or gives it an appearance of being corrupted. When the user sees the system in the corrupted state, he starts looking for help so as to solve the problem.
* Marketing: In order to make sure that the user approaches the attacker with the problem, the attacker advertises himself as the only person who can solve the problem.
* Support: In this step, the attacker gains the trust of the target and obtains access to sensitive information.
|| If your alarm didn't go off in the sabotage step, when your system appearing corrupt coincided with this remarkable good fortune of someone being right there at the right time ready to help, then let your alarm go off at the [[Propaganda#False_Dilemma|false dilemma]] you are presented with in the marketing step.
|}

== Targeted surveillance ==

=== Hardware implants ===
Unless you are targeted by a government intelligence agency, there seems to be no need to worry about installing commodity hardware from reputable vendors.

Detection: Looking for physical devices will always be the easiest solution to detect them. The links to NSA exploits of the day were added for the comments. :)

{| class="wikitable"
|-
! !! More information !! Possible types of attacks !! Detection
|-
| Godsurge || Godsurge is a physical device plugged-in to the Joint Test Action Group or JTAG headers on a system's motherboard. JTAG headers can be found on many systems and are notoriously common in embedded devices. These headers are used during the development process for debugging purposes: they give you a direct interface with the CPU and are extremely helpful. They are commonly left on the production boards, so finding them on a device is normal and not a security concern. However, if there is a chip or board wired in to a device's JTAG headers that you did not wire in yourself, then something fishy may be going on. || The JTAG debugging interface can be used to reflash the BIOS from scratch, for example loading a compromised version of the software.

|| Look for the JTAG connecter on the motherboard. The location of the JTAG headers may differ, but they tend to be near the CPU and may have exposed pins (or not). See the Wikipedia page on JTAG for more information and to see what they look like <ref>Wikipedia JTAG http://en.wikipedia.org/wiki/Joint_Test_Action_Group</ref>
|-
| Ginsu and Bulldozer
|| Ginsu provides software application persistence on target systems with the PCI bus hardware implant, Bulldozer
http://leaksource.files.wordpress.com/2013/12/nsa-ant-ginsu.jpg

https://www.schneier.com/blog/archives/2014/01/ginsu_nsa_explo.html
|| Exploit persistence from a PCI card ROM
|| Open up the computer's case and look for a PCI card that does not belong. For example, if you find a PCI card that appears to serve no purpose (e.g.. not your network card or it has no external ports), then perhaps try removing it and see how things work. If a few black SUVs roll up to your house after unplugging the PCI card, it's probably not because your domicile is the set for a rap superstar's new music video. Or maybe it is.
|-
| Cottonmouth I, Cottonmouth II and Cottonmouth III
|| These bugs are embedded somewhere along the USB bus and function as an air gap bridge to assist in exfiltration of data as well as allow persistent compromise. It can be embedded directly in the USB headers in an existing USB peripheral (USB hub or keyboard). These devices allow for exfiltration of data over unknown radio frequencies to listening devices in the area, even on a system that is not connected to the internet.
http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-i.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-i_n.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-ii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-ii.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonomouth-iii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-iii.html
|| USB host attack
|| Open up the keyboard or USB hub and identify a board that serves no purpose to the device. The malicious USB device would also likely show up on your computer's list of USB devices, so just checking there would be a good place to start.
|}

=== Radio frequency exfiltration ===
The devices used can vary, but they all employ a similar method of communication via an unknown radio/radar protocol. The frequencies are not known and based on the information leaked the devices are passive (only power on when data is being extracted) making identification of (type of) signal (and intelligence) hard.

Detection: First check for evidence of a device which has been wired in to an existing device such as a keyboard or other peripheral. The SpiderBlog describes the following possibility: You can check if there are RF transmitters in a device by monitoring the spectrum (using an amateur RF listening bug detector) while the device is off (to get a baseline for ambient RF background noise) and then monitoring it again after the suspect device is turned on and transmitting data via radio frequency. The detection device would pick up the signal and alert the user. <ref>Detecting A Surveillance State - Part 2 Radio Frequency Exfiltration https://www.trustwave.com/Resources/SpiderLabs-Blog/Detecting-A-Surveillance-State---Part-2-Radio-Frequency-Exfiltration/
</ref> Further analysis of the signal and its intelligence is still hard, as the exact protocols are (still) unknown.
{| class="wikitable"
|-
! !! More information !! Possible types of attack
|-
| Howlermonkey
|| Short to medium range Radio Transceiver

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/howlermonkey_ns.html
|| HOWLERMONKEY is a COTS-based transceiver deigned to be compatible with CONJECTURE/SPECULATION networks and STRIKEZONE devices running a HOWLERMONKEY personality. PCB layouts are tailored to individual implant space requirements and can vary greatly in form factor.
|-
| Ragemaster
|| Hardware implant in a VGA cable that sends video data over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/03/ragemaster_nsa.html
|| The RAGEMASTER taps the red video line between the video card within the desktop unit and the computer monitor, typically an LCD. When the RAGEMASTER is illuminated by a radar unit, the illuminating signal is modulated with the red video information. This information is re-radiated, where it is picked up at the radar, demodulated, and passed onto the processing unit, such as a LFS-2 and an external monitor, NIGHTWATCH, GOTHAM, or (in the future) VIEWPLATE. The processor recreates the horizontal and vertical sync of the targeted monitor, thus allowing TAO personnel to see what is displayed on the targeted monitor.
|-
| Loudauto
|| Hardware device that sends amplified audio over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/loudauto_nsa_ex.html

|| Room audio is picked up by the microphone and converted into an analog electrical signal. This signal is used to pulse position modulate (PPM) a square wave signal running at a pre-set frequency. This square wave is used to turn a FET (field effect transistor) on and off. When the unit is illuminated with a CW signal from a nearby radar unit, the illuminating signal is amplitude-modulated with the PPM square wave. This signal is re-radiated, where it is picked up by the radar, then processed to recover the room audio. Processing is currently performed by COTS equipment with FM demodulation capability (Rohde & Schwarz FSH-series portable spectrum analyzers, etc.) LOUDAUTO is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|-
| Surleyspawn
|| Hardware implant in a keyboard that emits keystrokes over RF

http://leaksource.files.wordpress.com/2013/12/nsa-ant-surlyspawn.jpg

https://www.schneier.com/blog/archives/2014/02/surlyspawn_nsa.html
|| The board taps into the data line from the keyboard to the processor. The board generates a square wave oscillating at a preset frequency. The data-line signal is used to shift the square wave frequency higher or lower, depending on the level of the data-line signal. The square wave, in essence, becomes frequency shift keyed (FSK). When the unit is illuminated by a CW signal from a nearby radar, the illuminating signal is amplitude-modulated (AM) with this square wave. The signal is re-radiated, where it is received by the radar, demodulated, and the demodulated signal is processed to recover the keystrokes. SURLYSPAWN is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|}

=== Infected firmware ===
Detection: Dump the BIOS to a bin file and compare the hash with a clean BIOS hash. See flashrom for identifying, reading, writing, erasing, and verifying BIOS/ROM/flash chips <ref>Debian packages: flashrom https://packages.debian.org/jessie/flashrom</ref>.

Removal: For each of these infections, where applicable, pulling the chip and replacing it with a new freshly burned BIOS chip or compact flash card would be sufficient. When dealing with built-in firmware it's a bit more difficult than pulling and replacing. You will need to re-flash the device using an operating system that is not at risk of being attacked by the infected firmware. You could boot the device into a low level OS in hopes that the firmware infection isn't able to protect itself. Or, you could wire in a debugging header to the device (such as JTAG , if available) to read or write the firmware on the device to clean things up for good.

{| class="wikitable"
|-
! !! More information !! Possible types of attack || Protection
|-
| Dietybounce
|| Motherboard BIOS Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-deitybounce.jpg?w=1208&h=1562

https://www.schneier.com/blog/archives/2014/01/nsa_exploit_of.html

|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS on a target machine to implant DEITYBOUNCE and its payload (the implant installer). Implantation via interdiction may be accomplished by nontechnical operator through use of a USB thumb drive. Once implanted, DEITYBOUNCE's frequency of execution (dropping the payload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Swap
|| Hard Drive Firmware Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-swap.jpg

https://www.schneier.com/blog/archives/2014/02/swap_nsa_exploi.html
|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS and TWISTEDKILT to write the Host Protected Area on the hard drive on a target machine in order to implant SWAP and its payload (the implant installer). Once implanted, SWAP's frequency of execution (dropping the playload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Headwater, sierramontana, and jetplow
|| Firmware backdoors that target popular networking hardware

https://leaksource.files.wordpress.com/2013/12/nsa-ant-headwater.jpg

https://www.schneier.com/blog/archives/2014/01/headwater_nsa_e.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-sierramontana.jpg

https://www.schneier.com/blog/archives/2014/01/sierramontana_n.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-jetplow.jpg

https://www.schneier.com/blog/archives/2014/01/jetplow_nsa_exp.html
|| HEADWATER PBD implant will be transferred remotely over the Internet to the selected target router by Remote Operations Center (ROC) personnel. After the transfer, the PBD will be installed in the router's boot ROM via an upgrade command. The PBD will then be activated after a system reboot. Once activated, the ROC operators will be able to use DNT's HAMMERMILL Insertion Tool (HIT) to control the PBD as it captures and examines all IP packets passing through the host router.

Currently, the intended DNT Implant to persist is VALIDATOR, which must be run as a user process on the target operating system. The vector of attack is the modification of the target's BIOS. The modification will add the necessary software to the BIOS and modify its software to execute the SIERRAMONTANA implant at the end of its native System Management Mode (SMM) handler.

JETPLOW persists DNT's BANANAGLEE software implant and modifies the Cisco firewall's operating system (OS) at boot time. If BANANAGLEE support is not available for the booting operating system, it can install a Persistent Backdoor (PDB) designed to work with BANANAGLEE'S communications structure, so that full access can be reacquired at a later time.
|| Most routers can be flashed with openWRT http://wiki.openwrt.org/toh/start
|}

== Dragnet surveillance ==

This table lists theoretical defenses and detection methods for selected groups of leaked [[surveillance]] programs and services. This is just a thought experiment covering (theoretical) defenses against these attacks and not intended to spread fear, uncertainty or doubt about surveillance states.

Due to the age and limited scope of the leaked documents and what we are up against <ref>Cryptome: Communications privacy folly http://cryptome.org/2012/06/comms-folly.htm
</ref>, the defenses mentioned in these tables should not be relied upon for protection and I make no guarantees to their accuracy. Do your own research and make informed decisions, knowing it is impossible to be completely safe. Things move incredibly fast in this arena and I will update these tables when more is found and/or theorised.

Ideally we discuss these exploits and counter moves without the NSA being privy to those discussion, but that would make spreading possible counter moves too slow for the surveillance development cycle (security arms race) and effectively exclude non-techie activists from being able to defend themselves to some extent, so the next best solution is to discuss everything in the open, rather than not discussing them at all. May we not have to rely on a new class of technocracy.

Best protection: Smash your PC to bits, distribute the pieces randomly among a dozen scrap heaps, and [https://undisconnect.miraheze.org/wiki/Rock_bottom move into the woods, deep in the woods ... ]
{| class="wikitable"
|-
! !! More information !! !! Protection
|-
| PRISM || PRISM is part of the overall NSA surveillance effort - a program authorised in the united states under the FISA Amendments Act (FAA), now located in Section 702 of the FISA, that allows the NSA to collect communications of specifically identified foreign targets.

When people in the rest of the world use the net, they are effectively using US-based services, making them a legal target for US intelligence. NSA has no "direct access" to the servers of companies like Microsoft, Facebook and Google - other agencies collect the data. More [[Surveillance#PRISM|here]]
|| Gathers emails, chat--video, voice, photos, stored data, VoIP, file transfers, video conferencing, notifications of target activity--logins, etc, online social networking details, and special requests collected by other programs from nine major Silicon Valley technology companies: Facebook, Google, Yahoo, Microsoft, PalTalk, Skype, YouTube, Apple, and AOL.
||
* Put political pressure on the united states - economical and political dependencies are making that unlikely, see the power of scarcity above.

* Avoid using popular web services.
* FOIA's, court cases on legitimacy
|-
| Upstream collection
|| It appears that through Upstream collection, the NSA gains access to data by forming partnerships with both foreign intelligence agencies and foreign telecommunication companies, by partnering with united states telecommunications companies that then make agreements with international companies for their internet data. More [[Surveillance#Upstream_collection|here]]
|| BLARNEY seems to focus specifically on metadata information about communications traffic and network devices (where and when they were sent).

FAIRVIEW collects just about everything on packet and session levels.
||
* Research which countries and telecommunication companies participate
* Avoid using services associated with these programs
* [[Step 1#Understanding_and_minimising_our_digital_shadows|Understanding and minimising our digital shadows]]
* Court cases on legitimacy
|}

== Censorship ==
Table initially filled with data from How to effectively argue against Internet Censorship ideas http://rys.io/en/94.txt.

Proxy servers, especially anonymous ones, located outside the area where a censorship solution is deployed can be used quite easily to circumvent any blocking method; users can modify their operating system or browser settings, or install browser additions that make using this circumvention method trivial. It is possible to block the proxy servers themselves (via IP-blocking, keyword blocking, etc.), however it is infeasible to block them all, as they are easy to set-up.

Virtual Private Networks (including “poor man’s VPNs” like SSH tunnels) require more technical prowess and usually a (usually commercial) VPN service (or SSH server) outside the area with blocking deployed. Blocking all VPN/SSH traffic is possible, but requires deep packet inspection and is a serious problem for many legitimate businesses using VPNs (and SSH) as their daily tools of trade, to allow their employees access to corporate networks from outside physical premises, via a secured link on the Internet.

TOR, or The Onion Router, is a very effective (if a bit slow) circumvention method. It is quite easy to set-up — users can simply download the TOR Browser Bundle and use it to access the Internet. Due to the way it works it is nigh-impossible to block TOR traffic (as it looks just like vanilla HTTPS traffic), to the point that it is known to allow access to the uncensored Internet to those living in areas with most aggressive Internet censorship policies — namely China, North Korea and Iran. See Tor threats.

None of the censorship solutions is able to block content on darknets — virtual networks accessible anonymously only via specialised software (for instance TOR, I2P, FreeNet), and guaranteeing high resilience to censorship through technical composition of the networks themselves. Because darknets are both practically impossible to block entirely and not allowing for any content blocking within them, they are effectively the ultimate circumvention methods. The downside to using darknets is their lower bandwidth. Deploying Internet censorship pushes the to-be-blocked content into darknets, making it ever-harder for law enforcement gather evidence and researchers gather data on the popularity of a given type of censored content.
{| class="wikitable"

! Blocking type
! How it works
! Circumvention
! DPI
|-
|DNS-based blocking
|DNS-based blocking requires ISPs (who usually run their own DNS servers, being default for their clients) to de-list certain domains (so that they are not resolvable when using these DNS servers). This means that the costs of implementing it are small.
|Custom DNS server settings can be used to easily circumvent DNS-based blocking. It does not require almost any technical prowess and can be used by anybody. There is a number of publicly available DNS servers that can be used for this purpose. There is no way to easily block the use of this method without deploying censorship methods other than pure DNS-blocking.
|no
|-
|IP address-based blocking
|IP-based blocking requires the ISPs to either block certain IP addresses
<nowiki> </nowiki>internally or route all the outgoing connections via a central, government-mandated censoring entity. It is only superficially harder to circumvent, while retaining most if not all problems of DNS-based blocking.
|
|no
|-
|URL-based blocking
|Because this method blocks only certain, URL-identified content, not whole websites or servers (as do DNS-based and IP-based methods), it has much lower potential for accidental over-blocking. This also entails it has a higher potential for under-blocking, as the content can be available on the same server under many different URLs, and changing just a small part of the name defeats the filter.
|
|yes
|-
|Dynamic blocking
|This method uses deep packet inspection to read the contents of data being transmitted, and compares it with a list of keywords, or with image samples or video (depending on the content type).
|
|yes
|-
|Hash-based blocking
|Hash-based blocking uses deep packet inspection to inspect the contents of data-streams, hashes them with cryptographic hash functions and compares to a known database of hashes to be blocked.
|
|yes
|-
|Hybrid solutions
|In order to compromise between high-resource, low-over-blocking hash-based blocking and low-resource, high-over-blocking IP- or DNS-based solutions, a hybrid solution might be proposed. Usually it means that there is a list of IP addresses or domain names for which the hash-based blocking is enabled, hence only operating for a small part of content. This method does employ deep packet inspection.
|
|yes
|}

== Server attacks ==

{| class="wikitable"
|-
! !! !! Significance !! Prevention/Protection
|-
| XXS || Cross-site scripting is a vulnerability in php web applications. || Attackers can exploit it to steal user information. || Configure web server and write better secured PHP scripts (validating all user input)
|-
| SQL injection || SQL injection is a vulnerability in the database layer of a php application. || Any SQL statements can be executed by the application. || Configure the web server and write secure code (validating and escaping all user input)
|-
| File uploads || Allowing users to place files on the server. || Can be used to delete files, databases, get user details and much more || Validate user input and only allow image file type such as png or gif.
|-
| Including local and remote files || An attacker can open files from remote server and execute any PHP code. || Can be used to upload files, delete files and install backdoors. || Configure php to disable remote file execution.
|-
| eval() || Evaluate a string as PHP code. || Can be used by an attacker to hide their code and tools on the server itself. || Configure php to disable eval().
|-
| CSRF || A Cross-Site Request Forgery forces an authenticated end user to execute unwanted actions on a web application. || Can compromise end user data and operation in case of normal user. If the targeted end user is the administrator account, this can compromise the entire web application. || See the Cross-Site Request Forgery (CSRF) Prevention Cheat Sheet https://www.owasp.org/index.php/Cross-Site_Request_Forgery_%28CSRF%29_Prevention_Cheat_Sheet
|}

== Related ==
* [[Timeline masters of the internet]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

[[Category:How_To]]

User:Lilith2

2015-10-09T14:02:00Z

Lilith2: /* Solidarity network */

[[File:fiddling.jpg|300px|thumb|right|Keep calm & Keep on fiddling]]
== Narratives ==
All of us, without exception, believe a mix of truth and misinformation, and often enough, disinformation. We strive to understand the world as it is, and not how it looks only according to our preconceptions, which are shaped by a multitude of forces, embedded as we are in our cultural matrix. Sometimes, the most unlikely seeming explanation turns out to be the correct one. In a warrior mindset we consider alternative views, but question everything.

Asking for details from ones own cultural matrix is not enough. Not even close. One needs to eat the local food, drink the water, breathe the air, listen to the stories, smell the earth, feel the bark of the trees, see local peoples, and experience local ways to catch what can only be described as a mere glimpse of the answer 42, the knowledge hidden in the trees and wheels in that specific locality on this beautiful planet.

People and communities use stories to understand the world and our place in it. These stories are embedded with power - the power to explain and justify the status quo as well as the power to make change imaginable and urgent.
* [[Timeline merchants of death]]
* [[Timeline masters of the internet]]
* [[Timeline that is soooo 1984 ...]]
A narrative analysis of power encourages us to ask: Which stories define cultural norms? Where did these stories come from? Whose stories were ignored or erased to create these norms? And, most urgently, what new stories can we tell to help create the world we desire? <ref>Harnessing the power of narrative for social change https://www.newtactics.org/conversation/change-story-harnessing-power-narrative-social-change</ref><ref>Underminers Chapter Ten – Reclaiming Ourselves http://underminers.org/the-book/chapter-10/</ref>
== Mindmaps and mindsets ==
If a mindmap is a cognitive "hathanger" then a mindset are the clothes hanging on the hathanger. Most traditions have mindmaps that have been and are evolving locally from the experiences of the previous generations on what worked and what didn't for them. Afaik, the oldest mindmaps use trees and wheels. And all mindmaps are generalisations in the wind without grounding details if not from there. The adversary grouping of the petty tyrants map is useful for gathering and interpreting data from narratives, questioning and timelines into a framework of possible threats.

* [[Petty tyrants]]
* [[Captivating capital and copyfighting]]
* [[Encrypting everything]]
* [[Arcane mixing techniques]]
* [[Networking concepts]]
* [[A typical linux distribution]]

== Methodologies, processes and choreographies ==
Most processes are adapted to allow for minimalist approaches and to minimalise the risk of unintentionally becoming the petty tyrants we fight <ref>Portal 2 Soundtrack Cara Mia (Turret Opera) One Hour https://www.youtube.com/watch?v=wFKfkfCeFj4</ref>.

* [[Shortest retrospective ever]]
* [[Scenario planning]]
* [[Linux development process]]

== Edge ==
* [[Confusing surveillance systems]]
* [[Simulations]]
** [[Simulation: Game of Goose]]
** [[Simulation: Mixnets]]
* [[Linux virtualisation]]

== Getting started with linux ==
''Standing on a hilltop in a thunderstorm on bare feet, wearing wet copper armour, holding a lightning rod and shouting ... bring it on, ye gods and godesses!''

We use lawful techniques and tools. And if need be we invent new (GNU and creative commons licensed) techniques and tools <ref>Portal - 'Still Alive' https://www.youtube.com/watch?v=Y6ljFaKRTrI</ref>.

* [[Installing linux]]
* [[Linux applications]]
* [[Linux security]]
* [[Kinky linux command-line]]
* [[Object encryption on linux]]
* [[Anonymising your traffic with linux]]
* [[Clean up all the things]]
* [[Shell scripting]]

== Teacher inside ==

* [[Self-dox]]
* [[Reconnaissance]]
* [[Network exploitation and monitoring]]
* [[Reverse engineering]]
* [[Roleplay]]
* [[Trolling]]
* [[Elicitation]]
* [[Wordsmithing]]
* [[Take control of the tech]]
* [[Making our own linux images]]

== Autonomy shaping infrastructure ==

* For setting up a home server see the [https://undisconnect.miraheze.org/wiki/Main_Page Servers Everywhere! section on UnDisConnect]
* For setting up a server see the documentation in the [http://anarchaserver.org/mediawiki/index.php/Anarcha_section Anarcha section on Alexandria]
* [[Linux server security]]
* Radio
* Libraries
* Meshnets
* Clusternets

== Solidarity network ==

* [[Threats, detection, protection and (counter) moves]]

== References ==
<references />.

User:Lilith2

2015-10-09T14:01:35Z

Lilith2: /* Edge */

User:Lilith2

2015-10-09T14:01:09Z

Lilith2:

[[File:fiddling.jpg|300px|thumb|right|Keep calm & Keep on fiddling]]
== Narratives ==
All of us, without exception, believe a mix of truth and misinformation, and often enough, disinformation. We strive to understand the world as it is, and not how it looks only according to our preconceptions, which are shaped by a multitude of forces, embedded as we are in our cultural matrix. Sometimes, the most unlikely seeming explanation turns out to be the correct one. In a warrior mindset we consider alternative views, but question everything.

Asking for details from ones own cultural matrix is not enough. Not even close. One needs to eat the local food, drink the water, breathe the air, listen to the stories, smell the earth, feel the bark of the trees, see local peoples, and experience local ways to catch what can only be described as a mere glimpse of the answer 42, the knowledge hidden in the trees and wheels in that specific locality on this beautiful planet.

People and communities use stories to understand the world and our place in it. These stories are embedded with power - the power to explain and justify the status quo as well as the power to make change imaginable and urgent.
* [[Timeline merchants of death]]
* [[Timeline masters of the internet]]
* [[Timeline that is soooo 1984 ...]]
A narrative analysis of power encourages us to ask: Which stories define cultural norms? Where did these stories come from? Whose stories were ignored or erased to create these norms? And, most urgently, what new stories can we tell to help create the world we desire? <ref>Harnessing the power of narrative for social change https://www.newtactics.org/conversation/change-story-harnessing-power-narrative-social-change</ref><ref>Underminers Chapter Ten – Reclaiming Ourselves http://underminers.org/the-book/chapter-10/</ref>
== Mindmaps and mindsets ==
If a mindmap is a cognitive "hathanger" then a mindset are the clothes hanging on the hathanger. Most traditions have mindmaps that have been and are evolving locally from the experiences of the previous generations on what worked and what didn't for them. Afaik, the oldest mindmaps use trees and wheels. And all mindmaps are generalisations in the wind without grounding details if not from there. The adversary grouping of the petty tyrants map is useful for gathering and interpreting data from narratives, questioning and timelines into a framework of possible threats.

* [[Petty tyrants]]
* [[Captivating capital and copyfighting]]
* [[Encrypting everything]]
* [[Arcane mixing techniques]]
* [[Networking concepts]]
* [[A typical linux distribution]]

== Methodologies, processes and choreographies ==
Most processes are adapted to allow for minimalist approaches and to minimalise the risk of unintentionally becoming the petty tyrants we fight <ref>Portal 2 Soundtrack Cara Mia (Turret Opera) One Hour https://www.youtube.com/watch?v=wFKfkfCeFj4</ref>.

* [[Shortest retrospective ever]]
* [[Scenario planning]]
* [[Linux development process]]

== Edge ==

* [[Autonomous collectives]]
* [[Controlled folly]]
* [[Confusing surveillance systems]]
* [[Simulations]]
** [[Simulation: Game of Goose]]
** [[Simulation: Mixnets]]
* [[Linux virtualisation]]

== Getting started with linux ==
''Standing on a hilltop in a thunderstorm on bare feet, wearing wet copper armour, holding a lightning rod and shouting ... bring it on, ye gods and godesses!''

We use lawful techniques and tools. And if need be we invent new (GNU and creative commons licensed) techniques and tools <ref>Portal - 'Still Alive' https://www.youtube.com/watch?v=Y6ljFaKRTrI</ref>.

* [[Installing linux]]
* [[Linux applications]]
* [[Linux security]]
* [[Kinky linux command-line]]
* [[Object encryption on linux]]
* [[Anonymising your traffic with linux]]
* [[Clean up all the things]]
* [[Shell scripting]]

== Teacher inside ==

* [[Self-dox]]
* [[Reconnaissance]]
* [[Network exploitation and monitoring]]
* [[Reverse engineering]]
* [[Roleplay]]
* [[Trolling]]
* [[Elicitation]]
* [[Wordsmithing]]
* [[Take control of the tech]]
* [[Making our own linux images]]

== Autonomy shaping infrastructure ==

* For setting up a home server see the [https://undisconnect.miraheze.org/wiki/Main_Page Servers Everywhere! section on UnDisConnect]
* For setting up a server see the documentation in the [http://anarchaserver.org/mediawiki/index.php/Anarcha_section Anarcha section on Alexandria]
* [[Linux server security]]
* Radio
* Libraries
* Meshnets
* Clusternets

== Solidarity network ==

* [[Threats, detection, protection and (counter) moves]]
* [[Autonomy patterns]]
* [[Communication patterns]]
* Mirroring servers
* Load balancing servers
* Reframing and refactoring operational accounts

== References ==
<references />.

User:Lilith2

2015-10-09T13:59:33Z

Lilith2: /* Methodologies, processes and choreographies */

[[File:fiddling.jpg|300px|thumb|right|Keep calm & Keep on fiddling]]
== Narratives ==
All of us, without exception, believe a mix of truth and misinformation, and often enough, disinformation. We strive to understand the world as it is, and not how it looks only according to our preconceptions, which are shaped by a multitude of forces, embedded as we are in our cultural matrix. Sometimes, the most unlikely seeming explanation turns out to be the correct one. In a warrior mindset we consider alternative views, but question everything.
* [[News & Watchdogs]]
Asking for details from ones own cultural matrix is not enough. Not even close. One needs to eat the local food, drink the water, breathe the air, listen to the stories, smell the earth, feel the bark of the trees, see local peoples, and experience local ways to catch what can only be described as a mere glimpse of the answer 42, the knowledge hidden in the trees and wheels in that specific locality on this beautiful planet.

People and communities use stories to understand the world and our place in it. These stories are embedded with power - the power to explain and justify the status quo as well as the power to make change imaginable and urgent.
* [[Timeline merchants of death]]
* [[Timeline masters of the internet]]
* [[Timeline that is soooo 1984 ...]]
A narrative analysis of power encourages us to ask: Which stories define cultural norms? Where did these stories come from? Whose stories were ignored or erased to create these norms? And, most urgently, what new stories can we tell to help create the world we desire? <ref>Harnessing the power of narrative for social change https://www.newtactics.org/conversation/change-story-harnessing-power-narrative-social-change</ref><ref>Underminers Chapter Ten – Reclaiming Ourselves http://underminers.org/the-book/chapter-10/</ref>
== Mindmaps and mindsets ==
If a mindmap is a cognitive "hathanger" then a mindset are the clothes hanging on the hathanger. Most traditions have mindmaps that have been and are evolving locally from the experiences of the previous generations on what worked and what didn't for them. Afaik, the oldest mindmaps use trees and wheels. And all mindmaps are generalisations in the wind without grounding details if not from there. The adversary grouping of the petty tyrants map is useful for gathering and interpreting data from narratives, questioning and timelines into a framework of possible threats.

* [[Petty tyrants]]
* [[Captivating capital and copyfighting]]
* [[Encrypting everything]]
* [[Arcane mixing techniques]]
* [[Networking concepts]]
* [[A typical linux distribution]]
* [[Memory matters]]
* [[Traffic analysis]]

== Methodologies, processes and choreographies ==
Most processes are adapted to allow for minimalist approaches and to minimalise the risk of unintentionally becoming the petty tyrants we fight <ref>Portal 2 Soundtrack Cara Mia (Turret Opera) One Hour https://www.youtube.com/watch?v=wFKfkfCeFj4</ref>.

* [[Shortest retrospective ever]]
* [[Scenario planning]]
* [[Linux development process]]

== Edge ==

* [[Autonomous collectives]]
* [[Controlled folly]]
* [[Confusing surveillance systems]]
* [[Simulations]]
** [[Simulation: Game of Goose]]
** [[Simulation: Mixnets]]
* [[Linux virtualisation]]

== Getting started with linux ==
''Standing on a hilltop in a thunderstorm on bare feet, wearing wet copper armour, holding a lightning rod and shouting ... bring it on, ye gods and godesses!''

We use lawful techniques and tools. And if need be we invent new (GNU and creative commons licensed) techniques and tools <ref>Portal - 'Still Alive' https://www.youtube.com/watch?v=Y6ljFaKRTrI</ref>.

* [[Installing linux]]
* [[Linux applications]]
* [[Linux security]]
* [[Kinky linux command-line]]
* [[Object encryption on linux]]
* [[Anonymising your traffic with linux]]
* [[Clean up all the things]]
* [[Shell scripting]]

== Teacher inside ==

* [[Self-dox]]
* [[Reconnaissance]]
* [[Network exploitation and monitoring]]
* [[Reverse engineering]]
* [[Roleplay]]
* [[Trolling]]
* [[Elicitation]]
* [[Wordsmithing]]
* [[Take control of the tech]]
* [[Making our own linux images]]

== Autonomy shaping infrastructure ==

* For setting up a home server see the [https://undisconnect.miraheze.org/wiki/Main_Page Servers Everywhere! section on UnDisConnect]
* For setting up a server see the documentation in the [http://anarchaserver.org/mediawiki/index.php/Anarcha_section Anarcha section on Alexandria]
* [[Linux server security]]
* Radio
* Libraries
* Meshnets
* Clusternets

== Solidarity network ==

* [[Threats, detection, protection and (counter) moves]]
* [[Autonomy patterns]]
* [[Communication patterns]]
* Mirroring servers
* Load balancing servers
* Reframing and refactoring operational accounts

== References ==
<references />.

Wordsmithing

2015-10-09T13:55:19Z

Lilith2: /* Jump on the emo bandwagon! */

Let grammar, syntax, semantics, punctuation, and spelling into your life! Even the most energetic and wonderful mess has to be turned into sentences...
[[File:Virus.jpg|480px|thumb|right]]

== And, But and However ==

Do not start sentences with conjunctions such as ''And, But and However''. According to English teachers everywhere, these should appear ONLY in the middle of a sentence, not shamefully whipped out at its beginning. That says no-thing about flaunting it as a title.

And, whenever someone says anything and you wish to add your 2 cents to the discussion, by all means, always start your sentence with "But ..." to make sure you wipe out everything anybody said before your "But ..." sentence entered the scene. "However ..." is a good alternative but not as effective as "But ..." at making others feel unheard (and therefore increasingly unwilling to listen to your additional information).

== Once Upon A Time ==

It was the Best of Times, it was the Worst of Times, it was the Age of Wisdom, it was the Era of Foolishness, it was the Epoch of Belief, it was the Time of Incredulity, it was the Season of Light, it was the Aeon of Darkness, it was the Spring of Decolonisation, it was the Winter of Despair ... In short, the period was so far like the present period, that some of its uproariest authorities insisted on its being received, for good or for evil, in the inflated degree of comparison only.

An important feature of a sentence is the reaction of the writer to some reality. In declarative sentences, this active part of the writer might be said to be manifesting a certain assertiveness.

A closer examination of assertive sentences shows an overwhelming majority to contain two basic content elements: a statement and an element about which a statement is made. The element about which something is stated may be said to be the ''basis of the utterance'' or the ''theme'', and what is stated about the basis is the ''nucleus of the utterance'' or the ''theme''. Old news, true. That a declarative sentence can be perceived to contain these two basic elements was recognised long ago.

However, less known is that these elements are given a different meaning in psychology. The theme is often called the ''psychological subject'' and the rheme the ''psychological predicate''. And in a narration stream the order is usually simple: the theme of the next sentence usually being the rheme of the sentence before it.

"Once upon a time there was a Queen. And the Queen had three daughters. The daughters ..."

"And the Queen had three daughters" has two parts: "the Queen" (basis) and "had three daughters" (nucleus).

But what is the situation at the beginning of a narration?

When we start to write about something which cannot yet be referred to as a known fact, then from the complex of notions included in the statement we anticipate one as given, as a notion that naturally presents itself and we make it the starting point.

It may be interesting to examine the beginnings of different narrations from this point of view, for further examination of our [[Scenario_planning#Gut.2C_head.2C_and_decision_making|gut level processing]].

== Making Total Sense ==

Through the fathomless deeps of space swims the star turtle Great A’Tuin, bearing on its back the four giant elephants who carry on their shoulders the mass of the Discworld. A tiny sun and moon spin around them, on a complicated orbit to induce seasons, so probably nowhere else in the multiverse is it sometimes necessary for an elephant to cock a leg to allow the sun to go past ...

=== Choices ===

The underminers guide mentions the dangers of using words like ''progress'' and ''choice'' because they are often ''fake choices'' while re-inforcing disconnection predicates. Many words have been arrogated and appropriated by the machine. Words like ''sustainability'', ''resilience'' and ''eco'' are all in use for greenwashing.

And just like the in the guide mentioned ''sharing real knowledge'', there seem to be ''real choices'': Do we now stop using these words altogether, or reinforce more connected predicates by using these words more congruently (depending on context and who we communicate with for what purpose)? That could be like ''take back the words''?

Note: Make that singular, ''take back the word'', and expect trolls believing religiously in ''the word'' (whatever the word is).

=== Predicates and implication by intonation ===

From Are Your Lights On?: How to Figure Out What the Problem Really is https://www.smashwords.com/extreader/read/53704/1/are-your-lights-on:

* ''Mary'' had a little lamb
* Mary ''had'' a little lamb
* Mary had ''a'' little lamb
* Mary had a ''little'' lamb
* Mary had a little ''lamb''

=== Where did words start? ===

Goodun. We have the written word, preceded by spoken word. And apparently the origin of spoken language has stumped linguistics. While some write things like ''"While it is widely understood that our ability to communicate through speech sets us apart from other animals, language experts, historians and scientists can only hypothesize how, where and when it all began. Some new findings may provide some real insight into this conundrum"'' <ref>Where and when did language begin? A remarkable new study may have the answer http://blog.dictionary.com/origin/</ref>, others not using the same definition of "language" and using more fluid perceptions might say that that is not "widely understood" but more a case of "widely hypothesised" <ref>Which language is the oldest? http://www.languagesoftheworld.info/historical-linguistics/which-language-is-the-oldest.html</ref> and that clearly, animal people, plant people and rock people also communicate <ref>Ainimal communication https://en.wikipedia.org/wiki/Animal_communication</ref><ref>Plants talk to each other using an internet of fungus http://www.bbc.com/earth/story/20141111-plants-have-a-hidden-internet and https://www.newscientist.com/article/mg20827864.600-fungal-threads-are-the-internet-of-the-plant-world/</ref><ref>Mycorrhiza-Induced Resistance and Priming of Plant Defenses http://www.researchgate.net/publication/225059105_Mycorrhiza-Induced_Resistance_and_Priming_of_Plant_Defenses</ref>, some even seem to "talk".

If someone asks the more fluid "when did the spoken words start?", these people might reply that ''spoken word'' started with communication by making sound waves <ref>Can any animals talk and use language like humans http://www.bbc.com/earth/story/20150216-can-any-animals-talk-like-humans</ref>. If then someone asks when and where communication started, they might communicate they think it started with chemistry <ref>Animal Behavior: Signaling and Communication http://www.sparknotes.com/biology/animalbehavior/signalingandcommunication/section2.rhtml</ref> and it appears just as true for humans <ref>Neurobiology of Chemical Communication Chapter 19 Human Pheromones http://www.ncbi.nlm.nih.gov/books/NBK200980/</ref><ref>Chemical communication and mother-infant recognition http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2717541/</ref>.

And some when presented with that question, will not reply because they do not understand the question because they speak another language. Or they may reply (possibly asking "what?") and we may or may not understand the reply because we do not speak their language. We haven't built those "receptors" and "embedded structures" (yet).

Words account for only 5%? <ref>Communication: 80% Body Language; 15% Tone & 5% Actual Words http://www.democraticunderground.com/10021960187</ref> Really?!? <ref>Is Nonverbal Communication a Numbers Game? Is body language really over 90% of how we communicate? https://www.psychologytoday.com/blog/beyond-words/201109/is-nonverbal-communication-numbers-game</ref>

=== The last word ===

Some people seem to need to have it. Sometimes they are argumentative, but often they simply will not let a chat end for whatever reason. Enough is enough!

''have the final/last word '':
# to say the last statement in a discussion or argument: ''He can't bear to lose an argument. He always has to have the last word. That's why he's so annoyed with other people wanting to have the last word too.''
# to make the final decision about something (usually + on ) ''Our bus driver has the final word on turning the wheel and pushing and pulling all kinds of levers and pedals in the bus. Do NOT interfere.''

For expanding the universe of possible solutions, we need to understand the "for whatever reason". It can be:
* know-it-all
* dense
* stubborn
* intoxicated
* always needing to be right
* conversation hogging
* excited
* consensus-driven

In short, the last word has no value in itself. No need for battles. Or?

=== Looking for universality? ===

Try emotions.

=== Jump on the emo bandwagon! ===

Are emoji the same thing as emoticons?

Answer 1: For all intents and purposes, yes they are.

Answer 2: ''Emoticons (from “emotion” plus “icon”) are specifically intended to depict facial expression or body posture as a way of conveying emotion or attitude in e-mail and text messages. They originated as ASCII character combinations such as :-) to indicate a smile—and by extension, a joke—and :-( to indicate a frown. In East Asia, a number of more elaborate sequences have been developed, such as (")(-_-)(") showing an upset face with hands raised. Over time, many systems began replacing such sequences with images, and also began providing ways to input emoticon images directly, such as a menu or palette. The emoji sets used by Japanese cell phone carriers contain a large number of characters for emoticon images, along with many other non-emoticon emoji.''

Don't get me started on dingbats.

To jump the bandwagon, first of all, keep calm and keep fiddling. Keep your towel close, and your IronyGuard at hand <ref>IRONYGuard - Removes All The Painful Irony In Your News https://www.youtube.com/watch?v=BFPdnIwP-kw</ref>. Then, jump the wagon.
[[File:fiddling.jpg|300px|thumb|right|Keep calm & Keep on fiddling]]
* Too fast. Before jumping, you should have informed yourself properly with the documentary: http://www.dissolve.com/showreels/emoji After allowing four scripts I still wasn't able to see it. It seems to be riding that magic carpet in the cloud.
* Inside information: If you wish to obfuscate your emoticon traffic, use hearts. Hearts are among the most used emoticons.
* For unpredictable emoji traffic get the newest of the newest. The Unicode Consortium regularly releases new innovations in emoticons. Visit emojipedia for more on that: http://emojipedia.org/new/
* Turn tracking on for (adjustments in) decision-making on your obfuscation/unpredictability ratio: http://www.emojitracker.com/ After allowing six! scripts I still get nothing.
* Social media: The first one (http://emoj.li/) was already closed but, don't weep, Emojicate is available now for iOS devices (https://itunes.apple.com/en/app/emojicate-emoji-only-chat/id902547983). Simply switch device and your happiness is right around the corner again!
* Do It Yourself: Turn your selfies, pets and favourite celebrities into emojis. The about page states "We believe in your freedom of expression". Download the app NOW from http://www.imojiapp.com/about
=== Read This! ===
(゜゜)～ employs the finite ~>゜)～～～ analysis method to (..) (._.) physical ( ..)φメモメモ φ(..)メモメモ of a =^_^=. The (V)o￥o(V) process consists of .....φ(・∀・＊) all systems into individual ((d[-_-]b)) or "＼(◎o◎)／" whose (-_-)zzz is easily //0-0\\ and then reconstructing the #-) from these o/\o. This is a O_o way of performing ┐('～`；)┌ and even in other（´∀｀）, such as ( ﾟヮﾟ). For example, ｷﾀﾜァ*･゜ﾟ･*:.｡..｡.:*･゜(n‘∀‘)ηﾟ･*:.｡. .｡.:*･゜ﾟ･* !!!!!

== man rtfm ==
[[File:RTFM.jpg|500px|thumb|right]]
For a recombination like this there is no need to make it convincing as a real man page, and the entertainment relies on making fun of overly reduced (context-less) statements and overly structured manual pages.

'''NAME'''

RTFM – tool to stimulate learning and improve workflow.

'''SYNOPSIS'''

RTFM is used to hit others with a lightning bolt.

'''DESCRIPTION'''

RTFM refers to the manual pages (or “man” pages, after the command used to display them) contained in *nix variants and has mutated to refer to reading any help pages or FAQs.

We lament over and over about how no one reads anymore, but we are also often just as guilty. Victims of our own hubris and overconfidence. In our own minds, we’re somehow annoyed by the very job we are doing. Help people solve problems. But sometimes it just touches the nerve, especially in situations where you’ve already explained how to fix it, and even more especially when you’ve already explained how to fix it to the same person.

It’s a slippery slope. Of course you always want to help, but at the same time, if you’re always there and available, it will lead to a condition where the user/junior admin will always skip figuring things out for themselves and just come straight to you.

There is no shame in reading. However, roughly 75% of IT is comprised of men. And it’s a pretty common perception that men don’t read (or ask for) directions.

''noun vs verb''

While the word “read” is a verb, the entire acronym is often used as a verb as well. Ex:

How do I set up pf to do ALTQ?

Dumbass, man it.

omg, *cry*

You’re lucky all I did was RTFM you.

*Theo sets mode : +b *!n00b@*

A common alternative to RTFM is to stop using Linux and move out of your parents’ basement.

Some critics assert that frequent users of the term are simply expressing elitism against the newcomers, thus driving them away without providing any tips or helpful suggestions. This is especially true when RTFM is used without even specifying which manual their correspondent should be reading. In the absence of a manual, one may be advised to Lurk Moar, see lurkmoar(8)

'''FILES'''

In its normal application RTFM uses no files and is words spoken (or written) only. If applied via email may contain a link to the manual in the environment and the odd meme image. Such meme images are also in use on twitter ever since the update_with_media function was added to the twitter API. If applied on a forum, many users and files may be affected; see butthurt(8).

http://readthefuckingmanual.com/

http://www.pophangover.com/4641/the-official-internet-butthurt-complaint-form/

https://duckduckgo.com/?q=writing+effective+manuals

'''ENVIRONMENT VARIABLES'''

Manual

'''SEE ALSO'''

lurkmoar(8), butthurt(8), tldr(8)

'''BUGS'''

You may be required to look things up and be able to fix things your self eventually.

'''EXAMPLES'''

It’s 10:30AM. The black themed firefox browser is about to stare back as a ping announces an incoming message over jabber to our fearless gurl.

“Hey! Sorry to bother you, but how do I …”

Our fearless gurl’s eyes roll deep into the back of her eye sockets as she emits a loud, exasperated sigh. She knows it’s in the manual – she helped write it. But no one seems to read it. She feels her efforts have been wasted.

But today, our fearless gurl has a plan. Through the miracles of modern science and some quick searches online, she has found a secret formula that would merge her mind into anyone that emails or tweets or pokes over jabber to ask questions. They would be under her control and would instantly think to look for the answers themselves. All she needs is a bolt of lightning to complete the transformation … the surrounding environment is already crackling …

“RTFM !!!!!”, she said.

'''RESTRICTIONS'''

Authoritarians and exceptionalists considering themselves very important and too high up to ever having to lift a finger may take a dim view of overenthusiastic application of this tool.

'''AUTHOR'''

While overcoming the RTFM syndrome remains a mystery, this man page was assembled from various sources by anonymii. She disclaims responsibility for any actions inspired by this man page.

'''HISTORY'''

Although largely unsubstantiated, usage of RTFM may have begun as a military lingo during World War II, when the phrase “Read The Field Manual” became a staple expression among American soldiers in response to basic questions asked by new enlistees. First introduced in 1939, US Army Field Manuals instructed all the vital, necessary skills for a soldier, from how to fold your clothes properly to how to toss a grenade inside the enemy’s tank. By the early 1950s the phrase RTFM was in common use by radio and radar technicians in the US Armed Forces. Operators frequently did not check for simple faults before asking questions; for example, checking whether a power switch was on, a fuse had blown or a power cord had become disconnected. A common response would be, “Did you check the oh en oh ef ef switch (On/Off). RTFM appeared in print in 1979 on the Table of Contents page of the LINPACK Users’ Guide in the form “R.T.F.M.” — Anonymous, suggesting that it was already well established. Cleve Moler has since revealed that a visit to Argonne National Laboratory by Tektronix Software Manager Ned Thanhouser (grandson of Edwin Thanhouser) during the development of MATLAB led to the anonymous quote. The first Urban Dictionary entry for RTFM was submitted on November 13th, 2003, although its usage has been documented in Slashdot comments as early as in March 1999. Just yonder, Manual was struggling to make his way through the foxy forest behind Fucking’s place. Manual had severely hurt his love handle during the window incident, and was starting to lose strength. Another pack of feral Manuals suddenly appeared, having caught wind of the The. One by one they latched on to Manual. Already weakened from his injury, Manual yielded to the furry onslaught and collapsed. The last thing he saw before losing consciousness was a buzzing horde of Manuals running off with his The. But then God came down with His charismatic smile and restored Manual’s The. Feeling pleased, God smote the Manuals for their injustice. Then He got in His best-in-its-so-called-‘class’ sedan and jetted away with the fortitude of half a million man-eating capybaras running from a oversized pack of 3-legged wallabies. Manual danced with joy when he saw this. His The was safe. It was a good thing, too, because in four minutes his favorite TV show, Fucking, was going to come on (followed immediately by ‘When 3-legged wallabies meet hand grenade’). Manual was excited. And so, everyone except Fucking and a few rusty razor blade-toting long-haired sea monkeys lived blissfully happy, forever after.

'''ANTITHESIS'''

The antithesis of RTFM is TL;DR which stands for “too long; didn’t read.”, see tldr(8)

If some asshole presents to you a long text and you still need to read it, you can incrementally read it with SuperMemo. What this entails is you mince and flashcardize the shit out of the text until it gets obliterated and you can’t recognize it. It’s like throwing the piece of e-paper in acid. The TL;DR is no moar.

'''FUTURE DIRECTIONS'''

None or a new type of hominids in around 5 million years (or sooner by self-destruction).

'''SEE ALSO'''

STFW (“Search The Fucking Web”), GIYF (“Google Is Your Friend”), LMGTFY (“Let Me Google That For You”), RTBM (“Read The Bloody Manual”—In some countries, e.g., the UK and Australia, this is a fractionally more polite alternative with identical meaning), RTFA (“Read The Fucking/Featured Article”—common on news forums such as Fark.com and Slashdot, where using “TFA” instead of “the article” has become a meme). Also see [[#Palindromes|palindrome]] and [[#Underminers|underminers]] resources below.

== Creating a fake press release ==
For press releases the entertainment relies on it (at least initially) being accepted as a civilised press release by receivers, IOW in how convincing it looks at first glance. Study a few local real press releases.
=== Anatomy of a press release ===
Keep it to one page (400-600 words):
* '''Logo''': Often found on releases distributed via mail, e-mail or a newswire service. Otherwise, omit.
* '''Headline''': Usually designed to be short and grab attention.
* '''Subheader''': Not always present. A descriptive expansion on the headline.
* '''Place line''': Identifies where the story is coming from.
* '''Date line''': Date of the release.
* '''Lead paragraph''': Often includes who, what, when, where, why. Can include url's.
* '''Body''': Typically 3 to 5 paragraphs, most important information first, making it a so-called ''inverted pyramid''.
* '''Boilerplate''': Organisational who and what.
* '''Contact information''': Newswire requirement.
* '''End mark''': ### or -end- (newswire)
=== Sperm are Dying! Females to Outnumber Males! ===
Amsterdam, 26 august 2015, GP
[[File:Triangle-of-power.jpg|480px|thumb|right|Undermining the Machine part 2 http://underminers.org/the-book/chapter-7/chapter-7-part-2/, may be freely reproduced but not altered]]
We may already be seeing the initial impact on men of rising temperatures and if this rise continues, sperm may fail completely. Will there be only females walking around on this planet? Does this spell the end of humans?

The seemingly most likely explanation for scrotal descent, and one that has been around for some time, is that sperm production and storage are maximised at typically 2.5 to 3 oC below the average healthy human body temperature of 37 oC. Supporting phenomena to that theory are skin of the scrotal sac being thin (promoting heat dissipation), and arteries supplying blood to the scrotum being located next to the veins that are taking blood away from it (one can easily recognise a cooling/heating exchange mechanism happening right there). Plus that spermatogenesis is most efficient at 34 oC. If sperm were kept at normal body temperature, sperm production would fail completely.

As temperatures rise (sperm with a Y survives three days, with an X five days, and higher temperatures slow down sperm) women may birth more girls than boys. If this were true, one would expect slightly more women conceived in tropical regions than at the poles (in historical records). This indeed appears to be the case <ref>Hot male: can sex in humans be modified by temperature? http://aerg.canberra.edu.au/library/sex_general/2003_McLachlan_Storey_TSD_in_humans.pdf</ref>. In Japan, rising temperatures are reported to have lead to a higher proportion of girls being born than boys, suggesting that climate change could indeed alter the mix of men and women <ref>Climate change is associated with male:female ratios of fetal deaths and newborn infants in Japan http://www.fertstert.org/article/S0015-0282%2814%2901840-8/abstract</ref>.

With the current rise in temperature we'd also expect to see a decline of male populations in other species. This too has been found. Male turtle populations are apparently already crashing in the heat <ref>Male turtle populations crashing in the heat https://www.newscientist.com/article/dn11227-male-turtle-populations-crashing-in-the-heat/</ref>.

Male turtles may be saved by female turtles digging deeper (hence cooler) birth pits on beaches, but with a direct threat to human balls, the chances of survival of just about every species on this planet went up as the powers-that-claim-to-be, as mired in self-interest as they are <ref>How the rich are destroying the earth http://makewealthhistory.org/2009/06/02/how-the-rich-are-destroying-the-earth-by-herve-kempf/</ref>, may take causes of rising temperatures more seriously as a threat to respond to with impeccable interdependent non-delusional energy dancing <ref>ZooKeys biodiversity research http://zookeys.pensoft.net/browse_articles (beautiful pictures of newly discovered species)</ref><ref>So you say you don't want a revolution? http://cluborlov.blogspot.nl/2015/07/so-you-say-you-dont-want-revolution.html (with some great comments too!)</ref><ref>The Logic of Nonviolent Revolutions http://privatewww.essex.ac.uk/~ksg/nvd2012/Dahletal.pdf</ref>, than as an opportunity for cap-and-trade on CO2 (or shifted to other types of) emissions ("regulating" farting) and continuing Business As Usual <ref>Can Christiana Figueres persuade humanity to save itself? http://www.newyorker.com/magazine/2015/08/24/the-weight-of-the-world (most likely not, as the attempt seems to assume economic growth & emissions can be uncoupled)</ref><ref>Fossil Fuel Execs Made Twice As Much As U.S. Pledged To Help Fight Climate Change Overseas http://www.huffingtonpost.com/entry/fossil-fuel-execs-climate-change_55e62913e4b0b7a9633ac41a</ref>, because their own gonads may be next.

'''About GenderSec Press''': ''GenderSec Press is a one-time only but really non-existent organisation, called into being for making this press release look fake.''

=== Smarter Cities ===

== Resources ==

=== Senses ===

* Vision verbs dominate in conversation across cultures, but the ranking of non-visual verbs varies http://pubman.mpdl.mpg.de/pubman/item/escidoc:2045058:10/component/escidoc:2109915/SanRoque_etal_2015.3.pdf
* English speakers, you stink at identifying smells https://www.newscientist.com/article/mg22530140.300-english-speakers-you-stink-at-identifying-smells

=== Making total sense ===

* The Science of Discworld https://www.youtube.com/watch?v=3CMdTlbGhXQ
* Hogfather https://www.youtube.com/watch?v=BaOHaBaKq-8&list=PLxUpNnImM9yL3AlTRei67FjowADRoBTF7
* The Color Of Magic I https://www.youtube.com/watch?v=V5Z5pFuXkWk (movie)
* The Color Of Magic II https://www.youtube.com/watch?v=fUyuIwtqduw (movie)
* Going postal https://www.youtube.com/watch?v=ZYYl3I1YKBE&list=PLB6U86TbV7A9baQOVHa_rE71WhTeIJ6pQ !!!!!
* Postmodern Parody In The Discworld Novels of Terry Pratchett http://www.lspace.org/books/analysis/christopher-bryant.html
* Blott on the landscape https://www.youtube.com/watch?v=w3TUU6313Xs&list=PLrIZy_2-uULPh7bbNxCwBVqz2_OyWVAKh
* Totalism http://totalism.org/
* Mind-blowing secrets of NSA's security exploit stockpile revealed at last http://www.theregister.co.uk/2015/09/04/nsa_explains_handling_zerodays/

=== Changing perceptions ===
* Mystery surrounds hallucinatory chaos at German homeopathy conference http://www.theguardian.com/science/2015/sep/10/homeopathy-conference-in-germany-goes-awry-as-delegates-take-lsd-like-drug
* The Net (documentary) https://www.youtube.com/watch?v=xLqrVCi3l6E and https://brianholmes.wordpress.com/2008/10/13/filming-the-world-laboratory/

=== Sound waves ===
* The Scream (Change the world without taking power), John Holloway https://libcom.org/library/change-world-without-taking-power-john-holloway

=== Inflationary language ===
* And punctuation https://www.youtube.com/watch?v=bcGA4alhPas

=== Palindromes ===
* A Chronotope of Revolution: The Palindrome from the Perspective of Cultural Semiotics http://www.realchange.org/pal/semiotic.htm
* Evolutionary history of novel genes on the tammar wallaby Y chromosome: Implications for sex chromosome evolution http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3290785/
* Lost Y Chromosome Genes Found on Autosomes http://www.the-scientist.com/?articles.view/articleNo/43078/title/Lost-Y-Chromosome-Genes-Found-on-Autosomes/ (WTF? mentioning Old World monkeys, New World monkeys, and apes :D)
* Male Bullshit Stories: Porn (Part V) https://trustyourperceptions.wordpress.com/ (And I know a lot of men who do not take pleasure in hurting women and animals. KEEP MEN LIKE THAT!!!!! and do not take the abusive kind seriously. AT ALL!!!!! See [[#Underminers|underminers]] resources Chapter 9))

=== Other times ===

* For the Aymara people living in the Andes, the past lies ahead and the future lies behind http://www.cogsci.ucsd.edu/~nunez/web/articles/Taipei%20Times%20-%20archives.pdf
* In Australia, the timeline of the Pormpuraaw, a remote Aboriginal community, runs along the east-west axis. The past is east. Time for the Pormpuraaw flows from left to right if they are facing south, right to left if they are facing north, towards the body if they are facing east, and away from the body if they are facing west. https://tylertretsven.wordpress.com/2012/03/05/time-and-space-in-pormpuraaw/
* In China, Mandarin speakers sometimes represent time along a vertical axis, with the past above and the future below http://psych.stanford.edu/~lera/papers/mandarin-time-2010.pdf
* Time for the Yupno flows uphill and is not even linear http://anthropology.net/2012/06/01/the-uphill-climb-of-time-for-the-yupno-of-papua-new-guinea/
* How Languages Construct Time http://www-psych.stanford.edu/~lera/papers/language-time.pdf

=== Language creation ===

* Language Creation Kit v2 http://www.zompist.com/kit.html

=== Language examples ===

* Scripts of the Ancient World http://www.ancientscripts.com
* Unifon Alphabet http://www.omniglot.com/writing/unifon.htm
* Click away http://www.languagesoftheworld.info/phonetics-and-phonology/click-away.html
* Snow Flower and the Secret Fan http://www.lisasee.com/snow-flower-and-the-secret-fan/

=== Measurement conversion ===

* Online Unit Conversion http://www.onlineconversion.com/ (converting modern units of measurement into measurement units no longer in general use)

=== Underminers ===
* Chapter Three – Who Are The Underminers? http://underminers.org/the-book/chapter-3/ (Propagating the message)
* Chapter Nine – Recreating Community (Part 1) http://underminers.org/the-book/chapter-9/chapter-9-part-1/
* Chapter Nine – Recreating Community (Part 2) http://underminers.org/the-book/chapter-9/chapter-9-part-2/

== Related ==

* [[Controlled folly]]

== References ==

[[Category:How_To]]

Threat modeling the quick and dirty way

2015-10-08T09:59:58Z

Lilith2: /* Fill in the table */

The traditional way of doing threat modelling, if it was done at all, was to sit down and think up attacks until bored (often ones that applications defended against anyway) and then declare victory.

* If we have non-security geeks doing the threat modelling then many attacks get missed or mis-identified. An expert facilitator can catch those.
* If we have security geeks involved then there tends to be a focus on attacks like ''sending a server custom-crafted messages that take advantage of the unusual mathematical properties of specially-formatted PKCS #1 message padding in RSA-encrypted data blocks and ignore the fact that the server’s private-key file is world-readable and indexed by Google.'' <ref>Peter Gutmann https://www.cs.auckland.ac.nz/~pgut001/ (yes, the professional paranoid guy, inventor of the Gutmann method, an algorithm for securely erasing the contents of computer hard drives by writing a series of 35 patterns over the region to be erased, and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996.)</ref>
* The problem with checklist-based approaches is that they only work when the attacker is using the same checklist as we are, and isn’t aware that a particular type of attack isn’t supposed to work, then they can walk right past the checklist-standardised defences.
* A variation of checklist-based threat modelling is risk mitigation: documenting every risk we can think of and then getting sign-off from someone in authority on the document. As a defense strategy, this is even less effective. But, more job-security oriented.
* Even more job-security oriented is an obfuscated checklist in a notation called Common Criteria, that was recovered from an UFO crash site . Don't allow people understanding it near sharp objects. Better yet, shoot on sight.

In the past this has delivered vulnerabilities and measures that provide the best (theoretical) security but very little effective security:

* The Internet Threat Model: I’m OK, you’re OK, and eavesdropping on credit card information sent over the Internet is the threat. Then we build something that people without a phone can not use (while ignoring all other threats).
* Inside-out Threat Model: A wonderful piece of circular reasoning which states that the threat model is whatever the security design is capable of defending against (anything that’s hard to defend against is excluded from the threat model).
* Provable Security for cryptographic algorithms: algorithms being proven secure against the threats that are defined by the provers (the attacker is transformed into some theoretical bogey man capable of doing anything that we know how to protect against).

Then how?

[[File:Dystopia.gif|500px|thumb|right]]

== Basic choreography ==
Steward Brand: Build a room
=== Set up a table ===
The first column contains a short description of the threat, the second the likelihood of it occurring, the third what impact it would have if it did happen, and the fourth an assessment (grade) of the time and energy you would need to protect yourself from the threat (for instance you can have no stars denote that there is no protection from that threat, hence it will cost nothing (except for the cost of the impact if it did happen).

=== Fill in the table ===
I recommend doing iterative brainstorming on "known and experienced threats" as initial filling of the first column in the table, before thinking about the other columns.

=== Reorder the list according to your set of priorities ===
Choose your ordering strategy carefully. Several strategies are possible.
* If this is a learning experience or you are a fan of "only time for putting out fires" cultures, no need for ordering.
* In a low risk environment (no immediate death threats) an "on demand" strategy works well. In this strategy you can use "low hanging fruit" and set up protection for items with a big impact and/or high likelihood of occurrence first.
* In a high risk environment or if any of the items in the list of possible impacts reads " loss of life" or some life-altering experience or you have turned procrastination into an art, best choose an "anticipating strategy", meaning do more research and detailed [[scenario planning]] (food for thought) to find possible [[Threats, detection, protection and (counter) moves|threats and solutions]] overlooked (food for gut).

== Examples ==

=== Silicon Valley first world problems ===

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Having to drink || medium || Developer will drop dead within a week, deadline will not be made.

The human body needs water to survive. The maximum time an individual can go without water seems to be a week — an estimate that would certainly be shorter in difficult conditions, like broiling heat.
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Having to eat food || high || Developer will drop dead within three weeks, deadline will not be made.

A human can go for more than three weeks without food (Mahatma Gandhi survived 21 days of complete starvation)
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Needing sleep || high || Like breathing, sleep is essential to humans. It has even been said that one could survive three times longer without food than one could without sleep. One of the better known experiments on this subject, found that depriving rats entirely of sleep resulted in their death, or near-dying state, within 11-32 days (Everson et al. 1989). || ** No long term alternatives known, but the deadline is still four weeks off. We tried adding coffee and coke to the Schmoylent in the previous development cycle.
|-
| Bathroom breaks || high || How many times do people go to the bathroom per day? Loads, and all those little breaks can add up to an hour or two per developer per day.
|| ***** Implanting stomata. Costly. Management has suggested we need more surveillance equipment to study the problem of the breaks.
|}

=== Journalist, observer or sousveillant in europe ===
A general counter strategy against police misconduct, abuse and brutality has been recording what happens during a protest. This of course, has gotten the police to target (citizen) journalists and observers overtly and covertly. And it is not just the police and government that are interested in us and our data.

The below threats, if happens, make for loss of trust and reputation and that translates to loss of effectiveness as independent observer.
{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Metanoia & Paranoia || low || Metanoia (safety delusion) makes us sitting ducks and paranoia renders us ineffective. || *
|-
| Physical attacks || medium || Physical damage/intimidation
||
|-
| Arrest: That might include direct approaches such as intimidation and asking for information on sources. || medium || Physical danger to those sources.||
|-
| Some contacts are named in reports, and some wish to remain anonymous. If we’re not careful, we might unintentionally disclose contact details or their location http://thenextweb.com/insider/2012/12/03/vice-leaves-metadata-in-photo-of-john-mcafee-pinpointing-him-to-a-location-in-guatemala/
* Phishing
* Guess/hack password
* Metadata
* Mobile trail
|| high || Intimidation/attack/imprisonment of sources. ||*
|-
| Correspondence with contacts includes all sorts of information that we do not want others to know about.
* Phishing
* Guess/hack password
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.
|| **
|-
| Email, social media accounts, websites, communication tools, a pile of passwords. Our passwords may be a used to attack others in our "organisation", and access information we don’t directly hold.
* Phishing emails can be sent from our email or social media accounts to others.
* Passwords can be guessed.
|| high || Followers on social media and people on mailinglists can get spammed.
|| ****
|-
| All work-in-progress, files and documents which we couldn’t publish for all sorts of reasons (space, legality, protection of others).
* Where are they? What forms do they have? Is there a security related documentation policy? Hey who is that man walking out the door with those files?
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.

Intimidation/attack/imprisonment of sources.
|| *
|-
| If we use computers in any way (including phones and even photocopiers) we can safely assume we are being recorded. Can lead to identification of sources through other data, our location, phone or email records.
* Metadata
* Computer forensics
|| high || Intimidation/attack/imprisonment of sources.
|| *
|-
| It may not be our activities they are interested in, but our reach. An example of this was the Syrian Electronic Army targeting E! Online http://ohnotheydidnt.livejournal.com/77479774.html || medium || Publication of hoax information on content management systems (the news website) and social media accounts
|| ***
|-
| Legal attacks include direct approaches such as subpoenas demanding that we reveal a source, or court orders to pass over footage, and increasingly might also include indirect approaches, to companies and organisations holding your information.
|| medium || Physical, social or economic danger to those sources, e.g. losing employment, losing privacy, criminal proceedings, assault.
|| **
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline masters of the internet]]
* [[Timeline merchants of death]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

Threat modeling the quick and dirty way

2015-10-08T09:39:23Z

Lilith2:

The traditional way of doing threat modelling, if it was done at all, was to sit down and think up attacks until bored (often ones that applications defended against anyway) and then declare victory.

* If we have non-security geeks doing the threat modelling then many attacks get missed or mis-identified. An expert facilitator can catch those.
* If we have security geeks involved then there tends to be a focus on attacks like ''sending a server custom-crafted messages that take advantage of the unusual mathematical properties of specially-formatted PKCS #1 message padding in RSA-encrypted data blocks and ignore the fact that the server’s private-key file is world-readable and indexed by Google.'' <ref>Peter Gutmann https://www.cs.auckland.ac.nz/~pgut001/ (yes, the professional paranoid guy, inventor of the Gutmann method, an algorithm for securely erasing the contents of computer hard drives by writing a series of 35 patterns over the region to be erased, and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996.)</ref>
* The problem with checklist-based approaches is that they only work when the attacker is using the same checklist as we are, and isn’t aware that a particular type of attack isn’t supposed to work, then they can walk right past the checklist-standardised defences.
* A variation of checklist-based threat modelling is risk mitigation: documenting every risk we can think of and then getting sign-off from someone in authority on the document. As a defense strategy, this is even less effective. But, more job-security oriented.
* Even more job-security oriented is an obfuscated checklist in a notation called Common Criteria, that was recovered from an UFO crash site . Don't allow people understanding it near sharp objects. Better yet, shoot on sight.

In the past this has delivered vulnerabilities and measures that provide the best (theoretical) security but very little effective security:

* The Internet Threat Model: I’m OK, you’re OK, and eavesdropping on credit card information sent over the Internet is the threat. Then we build something that people without a phone can not use (while ignoring all other threats).
* Inside-out Threat Model: A wonderful piece of circular reasoning which states that the threat model is whatever the security design is capable of defending against (anything that’s hard to defend against is excluded from the threat model).
* Provable Security for cryptographic algorithms: algorithms being proven secure against the threats that are defined by the provers (the attacker is transformed into some theoretical bogey man capable of doing anything that we know how to protect against).

Then how?

[[File:Dystopia.gif|500px|thumb|right]]

== Basic choreography ==
Steward Brand: Build a room
=== Set up a table ===
The first column contains a short description of the threat, the second the likelihood of it occurring, the third what impact it would have if it did happen, and the fourth an assessment (grade) of the time and energy you would need to protect yourself from the threat (for instance you can have no stars denote that there is no protection from that threat, hence it will cost nothing (except for the cost of the impact if it did happen).

=== Fill in the table ===
I recommend doing a brainstorm on threats as initial filling of the threat columns in the table, before thinking about the other columns.

=== Reorder the list according to your set of priorities ===
Choose your ordering strategy carefully. Several strategies are possible.
* If this is a learning experience or you are a fan of "only time for putting out fires" cultures, no need for ordering.
* In a low risk environment (no immediate death threats) an "on demand" strategy works well. In this strategy you can use "low hanging fruit" and set up protection for items with a big impact and/or high likelihood of occurrence first.
* In a high risk environment or if any of the items in the list of possible impacts reads " loss of life" or some life-altering experience or you have turned procrastination into an art, best choose an "anticipating strategy", meaning do more research and detailed [[scenario planning]] (food for thought) to find possible [[Threats, detection, protection and (counter) moves|threats and solutions]] overlooked (food for gut).

== Examples ==

=== Silicon Valley first world problems ===

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Having to drink || medium || Developer will drop dead within a week, deadline will not be made.

The human body needs water to survive. The maximum time an individual can go without water seems to be a week — an estimate that would certainly be shorter in difficult conditions, like broiling heat.
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Having to eat food || high || Developer will drop dead within three weeks, deadline will not be made.

A human can go for more than three weeks without food (Mahatma Gandhi survived 21 days of complete starvation)
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Needing sleep || high || Like breathing, sleep is essential to humans. It has even been said that one could survive three times longer without food than one could without sleep. One of the better known experiments on this subject, found that depriving rats entirely of sleep resulted in their death, or near-dying state, within 11-32 days (Everson et al. 1989). || ** No long term alternatives known, but the deadline is still four weeks off. We tried adding coffee and coke to the Schmoylent in the previous development cycle.
|-
| Bathroom breaks || high || How many times do people go to the bathroom per day? Loads, and all those little breaks can add up to an hour or two per developer per day.
|| ***** Implanting stomata. Costly. Management has suggested we need more surveillance equipment to study the problem of the breaks.
|}

=== Journalist, observer or sousveillant in europe ===
A general counter strategy against police misconduct, abuse and brutality has been recording what happens during a protest. This of course, has gotten the police to target (citizen) journalists and observers overtly and covertly. And it is not just the police and government that are interested in us and our data.

The below threats, if happens, make for loss of trust and reputation and that translates to loss of effectiveness as independent observer.
{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Metanoia & Paranoia || low || Metanoia (safety delusion) makes us sitting ducks and paranoia renders us ineffective. || *
|-
| Physical attacks || medium || Physical damage/intimidation
||
|-
| Arrest: That might include direct approaches such as intimidation and asking for information on sources. || medium || Physical danger to those sources.||
|-
| Some contacts are named in reports, and some wish to remain anonymous. If we’re not careful, we might unintentionally disclose contact details or their location http://thenextweb.com/insider/2012/12/03/vice-leaves-metadata-in-photo-of-john-mcafee-pinpointing-him-to-a-location-in-guatemala/
* Phishing
* Guess/hack password
* Metadata
* Mobile trail
|| high || Intimidation/attack/imprisonment of sources. ||*
|-
| Correspondence with contacts includes all sorts of information that we do not want others to know about.
* Phishing
* Guess/hack password
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.
|| **
|-
| Email, social media accounts, websites, communication tools, a pile of passwords. Our passwords may be a used to attack others in our "organisation", and access information we don’t directly hold.
* Phishing emails can be sent from our email or social media accounts to others.
* Passwords can be guessed.
|| high || Followers on social media and people on mailinglists can get spammed.
|| ****
|-
| All work-in-progress, files and documents which we couldn’t publish for all sorts of reasons (space, legality, protection of others).
* Where are they? What forms do they have? Is there a security related documentation policy? Hey who is that man walking out the door with those files?
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.

Intimidation/attack/imprisonment of sources.
|| *
|-
| If we use computers in any way (including phones and even photocopiers) we can safely assume we are being recorded. Can lead to identification of sources through other data, our location, phone or email records.
* Metadata
* Computer forensics
|| high || Intimidation/attack/imprisonment of sources.
|| *
|-
| It may not be our activities they are interested in, but our reach. An example of this was the Syrian Electronic Army targeting E! Online http://ohnotheydidnt.livejournal.com/77479774.html || medium || Publication of hoax information on content management systems (the news website) and social media accounts
|| ***
|-
| Legal attacks include direct approaches such as subpoenas demanding that we reveal a source, or court orders to pass over footage, and increasingly might also include indirect approaches, to companies and organisations holding your information.
|| medium || Physical, social or economic danger to those sources, e.g. losing employment, losing privacy, criminal proceedings, assault.
|| **
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline masters of the internet]]
* [[Timeline merchants of death]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

Threat modeling the quick and dirty way

2015-10-08T09:38:11Z

Lilith2:

The traditional way of doing threat modelling, if it was done at all, was to sit down and think up attacks until bored (often ones that applications defended against anyway) and then declare victory.

* If we have non-security geeks doing the threat modelling then many attacks get missed or mis-identified. An expert facilitator can catch those.
* If we have security geeks involved then there tends to be a focus on attacks like ''sending a server custom-crafted messages that take advantage of the unusual mathematical properties of specially-formatted PKCS #1 message padding in RSA-encrypted data blocks and ignore the fact that the server’s private-key file is world-readable and indexed by Google.'' <ref>Peter Gutmann https://www.cs.auckland.ac.nz/~pgut001/ (yes, the professional paranoid guy, inventor of the Gutmann method, an algorithm for securely erasing the contents of computer hard drives by writing a series of 35 patterns over the region to be erased, and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996.)</ref>
* The problem with checklist-based approaches is that they only work when the attacker is using the same checklist as we are, and isn’t aware that a particular type of attack isn’t supposed to work, then they can walk right past the checklist-standardised defences.
* A variation of checklist-based threat modelling is risk mitigation: documenting every risk we can think of and then getting sign-off from someone in authority on the document. As a defense strategy, this is even less effective. But, more job-security oriented.
* Even more job-security oriented is an obfuscated checklist in a notation called Common Criteria, that was recovered from an UFO crash site . Don't allow people understanding it near sharp objects. Better yet, shoot on sight.

In the past this has delivered vulnerabilities and measures that provide the best (theoretical) security but very little effective security:

* The Internet Threat Model: I’m OK, you’re OK, and eavesdropping on credit card information sent over the Internet is the threat. Then we build something that people without a phone can not use (while ignoring all other threats).
* Inside-out Threat Model: A wonderful piece of circular reasoning which states that the threat model is whatever the security design is capable of defending against (anything that’s hard to defend against is excluded from the threat model).
* Provable Security for cryptographic algorithms: algorithms being proven secure against the threats that are defined by the provers (the attacker is transformed into some theoretical bogey man capable of doing anything that we know how to protect against).

Then how?

[[File:Dystopia.gif|500px|thumb|right]]

== Basic choreography ==

=== Step 1. Set up a table ===
The first column contains a short description of the threat, the second the likelihood of it occurring, the third what impact it would have if it did happen, and the fourth an assessment (grade) of the time and energy you would need to protect yourself from the threat (for instance you can have no stars denote that there is no protection from that threat, hence it will cost nothing (except for the cost of the impact if it did happen).

=== Step 2. Fill in the table ===
I recommend doing a brainstorm on threats as initial filling of the threat columns in the table, before thinking about the other columns.

=== Step 3. Reorder the list according to your set of priorities ===
Choose your ordering strategy carefully. Several strategies are possible.
* If this is a learning experience or you are a fan of "only time for putting out fires" cultures, no need for ordering.
* In a low risk environment (no immediate death threats) an "on demand" strategy works well. In this strategy you can use "low hanging fruit" and set up protection for items with a big impact and/or high likelihood of occurrence first.
* In a high risk environment or if any of the items in the list of possible impacts reads " loss of life" or some life-altering experience or you have turned procrastination into an art, best choose an "anticipating strategy", meaning do more research and detailed [[scenario planning]] (food for thought) to find possible [[Threats, detection, protection and (counter) moves|threats and solutions]] overlooked (food for gut).

== Examples ==

=== Silicon Valley first world problems ===

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Having to drink || medium || Developer will drop dead within a week, deadline will not be made.

The human body needs water to survive. The maximum time an individual can go without water seems to be a week — an estimate that would certainly be shorter in difficult conditions, like broiling heat.
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Having to eat food || high || Developer will drop dead within three weeks, deadline will not be made.

A human can go for more than three weeks without food (Mahatma Gandhi survived 21 days of complete starvation)
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Needing sleep || high || Like breathing, sleep is essential to humans. It has even been said that one could survive three times longer without food than one could without sleep. One of the better known experiments on this subject, found that depriving rats entirely of sleep resulted in their death, or near-dying state, within 11-32 days (Everson et al. 1989). || ** No long term alternatives known, but the deadline is still four weeks off. We tried adding coffee and coke to the Schmoylent in the previous development cycle.
|-
| Bathroom breaks || high || How many times do people go to the bathroom per day? Loads, and all those little breaks can add up to an hour or two per developer per day.
|| ***** Implanting stomata. Costly. Management has suggested we need more surveillance equipment to study the problem of the breaks.
|}

=== Journalist, observer or sousveillant in europe ===
A general counter strategy against police misconduct, abuse and brutality has been recording what happens during a protest. This of course, has gotten the police to target (citizen) journalists and observers overtly and covertly. And it is not just the police and government that are interested in us and our data.

The below threats, if happens, make for loss of trust and reputation and that translates to loss of effectiveness as independent observer.
{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Metanoia & Paranoia || low || Metanoia (safety delusion) makes us sitting ducks and paranoia renders us ineffective. || *
|-
| Physical attacks || medium || Physical damage/intimidation
||
|-
| Arrest: That might include direct approaches such as intimidation and asking for information on sources. || medium || Physical danger to those sources.||
|-
| Some contacts are named in reports, and some wish to remain anonymous. If we’re not careful, we might unintentionally disclose contact details or their location http://thenextweb.com/insider/2012/12/03/vice-leaves-metadata-in-photo-of-john-mcafee-pinpointing-him-to-a-location-in-guatemala/
* Phishing
* Guess/hack password
* Metadata
* Mobile trail
|| high || Intimidation/attack/imprisonment of sources. ||*
|-
| Correspondence with contacts includes all sorts of information that we do not want others to know about.
* Phishing
* Guess/hack password
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.
|| **
|-
| Email, social media accounts, websites, communication tools, a pile of passwords. Our passwords may be a used to attack others in our "organisation", and access information we don’t directly hold.
* Phishing emails can be sent from our email or social media accounts to others.
* Passwords can be guessed.
|| high || Followers on social media and people on mailinglists can get spammed.
|| ****
|-
| All work-in-progress, files and documents which we couldn’t publish for all sorts of reasons (space, legality, protection of others).
* Where are they? What forms do they have? Is there a security related documentation policy? Hey who is that man walking out the door with those files?
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.

Intimidation/attack/imprisonment of sources.
|| *
|-
| If we use computers in any way (including phones and even photocopiers) we can safely assume we are being recorded. Can lead to identification of sources through other data, our location, phone or email records.
* Metadata
* Computer forensics
|| high || Intimidation/attack/imprisonment of sources.
|| *
|-
| It may not be our activities they are interested in, but our reach. An example of this was the Syrian Electronic Army targeting E! Online http://ohnotheydidnt.livejournal.com/77479774.html || medium || Publication of hoax information on content management systems (the news website) and social media accounts
|| ***
|-
| Legal attacks include direct approaches such as subpoenas demanding that we reveal a source, or court orders to pass over footage, and increasingly might also include indirect approaches, to companies and organisations holding your information.
|| medium || Physical, social or economic danger to those sources, e.g. losing employment, losing privacy, criminal proceedings, assault.
|| **
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline masters of the internet]]
* [[Timeline merchants of death]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

Threat modeling the quick and dirty way

2015-10-08T09:36:30Z

Lilith2:

The traditional way of doing threat modelling, if it was done at all, was to sit down and think up attacks until bored (often ones that applications defended against anyway) and then declare victory.

* If we have non-security geeks doing the threat modelling then many attacks get missed or mis-identified. An expert facilitator can catch those.
* If we have security geeks involved then there tends to be a focus on attacks like ''sending a server custom-crafted messages that take advantage of the unusual mathematical properties of specially-formatted PKCS #1 message padding in RSA-encrypted data blocks and ignore the fact that the server’s private-key file is world-readable and indexed by Google.'' <ref>Peter Gutmann https://www.cs.auckland.ac.nz/~pgut001/ (yes, the professional paranoid guy, inventor of the Gutmann method, an algorithm for securely erasing the contents of computer hard drives by writing a series of 35 patterns over the region to be erased, and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996.)</ref>
* The problem with checklist-based approaches is that they only work when the attacker is using the same checklist as we are, and isn’t aware that a particular type of attack isn’t supposed to work, then they can walk right past the checklist-standardised defences.
* A variation of checklist-based threat modelling is risk mitigation: documenting every risk we can think of and then getting sign-off from someone in authority on the document. As a defense strategy, this is even less effective. But, more job-security oriented.
* Even more job-security oriented is an obfuscated checklist in a notation called Common Criteria, that was recovered from an UFO crash site . Don't allow people understanding it near sharp objects. Better yet, shoot on sight.

In the past this has delivered vulnerabilities and measures that provide the best (theoretical) security but very little effective security:

* The Internet Threat Model: I’m OK, you’re OK, and eavesdropping on credit card information sent over the Internet is the threat. Then we build something that people without a phone can not use (while ignoring all other threats).
* Inside-out Threat Model: A wonderful piece of circular reasoning which states that the threat model is whatever the security design is capable of defending against (anything that’s hard to defend against is excluded from the threat model).
* Provable Security for cryptographic algorithms: algorithms being proven secure against the threats that are defined by the provers (the attacker is transformed into some theoretical bogey man capable of doing anything that we know how to protect against).

Then how?

[[File:Dystopia.gif|500px|thumb|right]]

== Basic choreography ==

=== Step 1. Set up a table ===
The first column contains a short description of the threat, the second the likelihood of it occurring, the third what impact it would have if it did happen, and the fourth an assessment (grade) of the time and energy you would need to protect yourself from the threat (for instance you can have no stars denote that there is no protection from that threat, hence it will cost nothing (except for the cost of the impact if it did happen).

=== Step 2. Fill in the table ===
I recommend doing a brainstorm on threats as initial filling of the threat columns in the table, before thinking about the other columns.

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Having to drink || medium || Developer will drop dead within a week, deadline will not be made.

The human body needs water to survive. The maximum time an individual can go without water seems to be a week — an estimate that would certainly be shorter in difficult conditions, like broiling heat.
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Having to eat food || high || Developer will drop dead within three weeks, deadline will not be made.

A human can go for more than three weeks without food (Mahatma Gandhi survived 21 days of complete starvation)
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Needing sleep || high || Like breathing, sleep is essential to humans. It has even been said that one could survive three times longer without food than one could without sleep. One of the better known experiments on this subject, found that depriving rats entirely of sleep resulted in their death, or near-dying state, within 11-32 days (Everson et al. 1989). || ** No long term alternatives known, but the deadline is still four weeks off. We tried adding coffee and coke to the Schmoylent in the previous development cycle.
|-
| Bathroom breaks || high || How many times do people go to the bathroom per day? Loads, and all those little breaks can add up to an hour or two per developer per day.
|| ***** Implanting stomata. Costly. Management has suggested we need more surveillance equipment to study the problem of the breaks.
|}

=== Step 3. Reorder the list according to your set of priorities ===
Choose your ordering strategy carefully. Several strategies are possible.
* If this is a learning experience or you are a fan of "only time for putting out fires" cultures, no need for ordering.
* In a low risk environment (no immediate death threats) an "on demand" strategy works well. In this strategy you can use "low hanging fruit" and set up protection for items with a big impact and/or high likelihood of occurrence first.
* In a high risk environment or if any of the items in the list of possible impacts reads " loss of life" or some life-altering experience or you have turned procrastination into an art, best choose an "anticipating strategy", meaning do more research and detailed [[scenario planning]] (food for thought) to find possible [[Threats, detection, protection and (counter) moves|threats and solutions]] overlooked (food for gut).

== Examples ==

=== Journalist, observer or sousveillant in europe ===
A general counter strategy against police misconduct, abuse and brutality has been recording what happens during a protest. This of course, has gotten the police to target (citizen) journalists and observers overtly and covertly. And it is not just the police and government that are interested in us and our data.

The below threats, if happens, make for loss of trust and reputation and that translates to loss of effectiveness as independent observer.
{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Metanoia & Paranoia || low || Metanoia (safety delusion) makes us sitting ducks and paranoia renders us ineffective. || *
|-
| Physical attacks || medium || Physical damage/intimidation
||
|-
| Arrest: That might include direct approaches such as intimidation and asking for information on sources. || medium || Physical danger to those sources.||
|-
| Some contacts are named in reports, and some wish to remain anonymous. If we’re not careful, we might unintentionally disclose contact details or their location http://thenextweb.com/insider/2012/12/03/vice-leaves-metadata-in-photo-of-john-mcafee-pinpointing-him-to-a-location-in-guatemala/
* Phishing
* Guess/hack password
* Metadata
* Mobile trail
|| high || Intimidation/attack/imprisonment of sources. ||*
|-
| Correspondence with contacts includes all sorts of information that we do not want others to know about.
* Phishing
* Guess/hack password
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.
|| **
|-
| Email, social media accounts, websites, communication tools, a pile of passwords. Our passwords may be a used to attack others in our "organisation", and access information we don’t directly hold.
* Phishing emails can be sent from our email or social media accounts to others.
* Passwords can be guessed.
|| high || Followers on social media and people on mailinglists can get spammed.
|| ****
|-
| All work-in-progress, files and documents which we couldn’t publish for all sorts of reasons (space, legality, protection of others).
* Where are they? What forms do they have? Is there a security related documentation policy? Hey who is that man walking out the door with those files?
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.

Intimidation/attack/imprisonment of sources.
|| *
|-
| If we use computers in any way (including phones and even photocopiers) we can safely assume we are being recorded. Can lead to identification of sources through other data, our location, phone or email records.
* Metadata
* Computer forensics
|| high || Intimidation/attack/imprisonment of sources.
|| *
|-
| It may not be our activities they are interested in, but our reach. An example of this was the Syrian Electronic Army targeting E! Online http://ohnotheydidnt.livejournal.com/77479774.html || medium || Publication of hoax information on content management systems (the news website) and social media accounts
|| ***
|-
| Legal attacks include direct approaches such as subpoenas demanding that we reveal a source, or court orders to pass over footage, and increasingly might also include indirect approaches, to companies and organisations holding your information.
|| medium || Physical, social or economic danger to those sources, e.g. losing employment, losing privacy, criminal proceedings, assault.
|| **
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline masters of the internet]]
* [[Timeline merchants of death]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

Threat modeling the quick and dirty way

2015-10-08T09:29:26Z

Lilith2:

The traditional way of doing threat modelling, if it was done at all, was to sit down and think up attacks until bored (often ones that applications defended against anyway) and then declare victory.

* If we have non-security geeks doing the threat modelling then many attacks get missed or mis-identified. An expert facilitator can catch those.
* If we have security geeks involved then there tends to be a focus on attacks like ''sending a server custom-crafted messages that take advantage of the unusual mathematical properties of specially-formatted PKCS #1 message padding in RSA-encrypted data blocks and ignore the fact that the server’s private-key file is world-readable and indexed by Google.'' <ref>Peter Gutmann https://www.cs.auckland.ac.nz/~pgut001/ (yes, the professional paranoid guy, inventor of the Gutmann method, an algorithm for securely erasing the contents of computer hard drives by writing a series of 35 patterns over the region to be erased, and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996.)</ref>
* The problem with checklist-based approaches is that they only work when the attacker is using the same checklist as we are, and isn’t aware that a particular type of attack isn’t supposed to work, then they can walk right past the checklist-standardised defences.
* A variation of checklist-based threat modelling is risk mitigation: documenting every risk we can think of and then getting sign-off from someone in authority on the document. As a defense strategy, this is even less effective. But, more job-security oriented.
* Even more job-security oriented is an obfuscated checklist in a notation recovered from an UFO crash site called Common Criteria. Don't allow people understanding it near sharp objects. Shoot on sight.

In the past this has delivered vulnerabilities and measures that provide the best (theoretical) security but very little effective security:

* The Internet Threat Model: I’m OK, you’re OK, and eavesdropping on credit card information sent over the Internet is the threat. Then we build something that people without a phone can not use (while ignoring all other threats).
* Inside-out Threat Model: A wonderful piece of circular reasoning which states that the threat model is whatever the security design is capable of defending against (anything that’s hard to defend against is excluded from the threat model).
* Provable Security for cryptographic algorithms: algorithms being proven secure against the threats that are defined by the provers (the attacker is transformed into some theoretical bogey man capable of doing anything that we know how to protect against).

Then how?

[[File:Dystopia.gif|500px|thumb|right]]

== Basic choreography ==

=== Step 1. Set up a table ===
The first column contains a short description of the threat, the second the likelihood of it occurring, the third what impact it would have if it did happen, and the fourth an assessment (grade) of the time and energy you would need to protect yourself from the threat (for instance you can have no stars denote that there is no protection from that threat, hence it will cost nothing (except for the cost of the impact if it did happen).

=== Step 2. Fill in the table ===
I recommend doing a brainstorm on threats as initial filling of the threat columns in the table, before thinking about the other columns.

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Having to drink || medium || Developer will drop dead within a week, deadline will not be made.

The human body needs water to survive. The maximum time an individual can go without water seems to be a week — an estimate that would certainly be shorter in difficult conditions, like broiling heat.
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Having to eat food || high || Developer will drop dead within three weeks, deadline will not be made.

A human can go for more than three weeks without food (Mahatma Gandhi survived 21 days of complete starvation)
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Needing sleep || high || Like breathing, sleep is essential to humans. It has even been said that one could survive three times longer without food than one could without sleep. One of the better known experiments on this subject, found that depriving rats entirely of sleep resulted in their death, or near-dying state, within 11-32 days (Everson et al. 1989). || ** No long term alternatives known, but the deadline is still four weeks off. We tried adding coffee and coke to the Schmoylent in the previous development cycle.
|-
| Bathroom breaks || high || How many times do people go to the bathroom per day? Loads, and all those little breaks can add up to an hour or two per developer per day.
|| ***** Implanting stomata. Costly. Management has suggested we need more surveillance equipment to study the problem of the breaks.
|}

=== Step 3. Reorder the list according to your set of priorities ===
Choose your ordering strategy carefully. Several strategies are possible.
* If this is a learning experience or you are a fan of "only time for putting out fires" cultures, no need for ordering.
* In a low risk environment (no immediate death threats) an "on demand" strategy works well. In this strategy you can use "low hanging fruit" and set up protection for items with a big impact and/or high likelihood of occurrence first.
* In a high risk environment or if any of the items in the list of possible impacts reads " loss of life" or some life-altering experience or you have turned procrastination into an art, best choose an "anticipating strategy", meaning do more research and detailed [[scenario planning]] (food for thought) to find possible [[Threats, detection, protection and (counter) moves|threats and solutions]] overlooked (food for gut).

== Examples ==

=== Journalist, observer or sousveillant in europe ===
A general counter strategy against police misconduct, abuse and brutality has been recording what happens during a protest. This of course, has gotten the police to target (citizen) journalists and observers overtly and covertly. And it is not just the police and government that are interested in us and our data.

The below threats, if happens, make for loss of trust and reputation and that translates to loss of effectiveness as independent observer.
{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Metanoia & Paranoia || low || Metanoia (safety delusion) makes us sitting ducks and paranoia renders us ineffective. || *
|-
| Physical attacks || medium || Physical damage/intimidation
||
|-
| Arrest: That might include direct approaches such as intimidation and asking for information on sources. || medium || Physical danger to those sources.||
|-
| Some contacts are named in reports, and some wish to remain anonymous. If we’re not careful, we might unintentionally disclose contact details or their location http://thenextweb.com/insider/2012/12/03/vice-leaves-metadata-in-photo-of-john-mcafee-pinpointing-him-to-a-location-in-guatemala/
* Phishing
* Guess/hack password
* Metadata
* Mobile trail
|| high || Intimidation/attack/imprisonment of sources. ||*
|-
| Correspondence with contacts includes all sorts of information that we do not want others to know about.
* Phishing
* Guess/hack password
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.
|| **
|-
| Email, social media accounts, websites, communication tools, a pile of passwords. Our passwords may be a used to attack others in our "organisation", and access information we don’t directly hold.
* Phishing emails can be sent from our email or social media accounts to others.
* Passwords can be guessed.
|| high || Followers on social media and people on mailinglists can get spammed.
|| ****
|-
| All work-in-progress, files and documents which we couldn’t publish for all sorts of reasons (space, legality, protection of others).
* Where are they? What forms do they have? Is there a security related documentation policy? Hey who is that man walking out the door with those files?
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.

Intimidation/attack/imprisonment of sources.
|| *
|-
| If we use computers in any way (including phones and even photocopiers) we can safely assume we are being recorded. Can lead to identification of sources through other data, our location, phone or email records.
* Metadata
* Computer forensics
|| high || Intimidation/attack/imprisonment of sources.
|| *
|-
| It may not be our activities they are interested in, but our reach. An example of this was the Syrian Electronic Army targeting E! Online http://ohnotheydidnt.livejournal.com/77479774.html || medium || Publication of hoax information on content management systems (the news website) and social media accounts
|| ***
|-
| Legal attacks include direct approaches such as subpoenas demanding that we reveal a source, or court orders to pass over footage, and increasingly might also include indirect approaches, to companies and organisations holding your information.
|| medium || Physical, social or economic danger to those sources, e.g. losing employment, losing privacy, criminal proceedings, assault.
|| **
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline masters of the internet]]
* [[Timeline merchants of death]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

Threat modeling the quick and dirty way

2015-10-08T09:28:55Z

Lilith2:

The traditional way of doing threat modelling, if it was done at all, was to sit down and think up attacks until bored (often ones that applications defended against anyway) and then declare victory.

* If we have non-security geeks doing the threat modelling then many attacks get missed or mis-identified. An expert facilitator can catch those.
* If we have security geeks involved then there tends to be a focus on attacks like ''sending a server custom-crafted messages that take advantage of the unusual mathematical properties of specially-formatted PKCS #1 message padding in RSA-encrypted data blocks and ignore the fact that the server’s private-key file is world-readable and indexed by Google.'' <ref>Peter Gutmann https://www.cs.auckland.ac.nz/~pgut001/ (yes, the professional paranoid guy, inventor of the Gutmann method, an algorithm for securely erasing the contents of computer hard drives by writing a series of 35 patterns over the region to be erased, and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996.)</ref>
* The problem with checklist-based approaches is that they only work when the attacker is using the same checklist as we are, and isn’t aware that a particular type of attack isn’t supposed to work, then they can walk right past the checklist-standardised defences.
* A variation of checklist-based threat modelling is risk mitigation: documenting every risk we can think of and then getting sign-off from someone in authority on the document. As a defense strategy, this is even less effective. But, more job-security oriented.
* Even more job-security oriented is an obfuscated checklist in a notation recovered from a UFO crash site called Common Criteria. Don't allow people understanding it near sharp objects. Shoot on sight.

In the past this has delivered vulnerabilities and measures that provide the best (theoretical) security but very little effective security:

* The Internet Threat Model: I’m OK, you’re OK, and eavesdropping on credit card information sent over the Internet is the threat. Then we build something that people without a phone can not use (while ignoring all other threats).
* Inside-out Threat Model: A wonderful piece of circular reasoning which states that the threat model is whatever the security design is capable of defending against (anything that’s hard to defend against is excluded from the threat model).
* Provable Security for cryptographic algorithms: algorithms being proven secure against the threats that are defined by the provers (the attacker is transformed into some theoretical bogey man capable of doing anything that we know how to protect against).

Then how?

[[File:Dystopia.gif|500px|thumb|right]]

== Basic choreography ==

=== Step 1. Set up a table ===
The first column contains a short description of the threat, the second the likelihood of it occurring, the third what impact it would have if it did happen, and the fourth an assessment (grade) of the time and energy you would need to protect yourself from the threat (for instance you can have no stars denote that there is no protection from that threat, hence it will cost nothing (except for the cost of the impact if it did happen).

=== Step 2. Fill in the table ===
I recommend doing a brainstorm on threats as initial filling of the threat columns in the table, before thinking about the other columns.

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Having to drink || medium || Developer will drop dead within a week, deadline will not be made.

The human body needs water to survive. The maximum time an individual can go without water seems to be a week — an estimate that would certainly be shorter in difficult conditions, like broiling heat.
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Having to eat food || high || Developer will drop dead within three weeks, deadline will not be made.

A human can go for more than three weeks without food (Mahatma Gandhi survived 21 days of complete starvation)
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Needing sleep || high || Like breathing, sleep is essential to humans. It has even been said that one could survive three times longer without food than one could without sleep. One of the better known experiments on this subject, found that depriving rats entirely of sleep resulted in their death, or near-dying state, within 11-32 days (Everson et al. 1989). || ** No long term alternatives known, but the deadline is still four weeks off. We tried adding coffee and coke to the Schmoylent in the previous development cycle.
|-
| Bathroom breaks || high || How many times do people go to the bathroom per day? Loads, and all those little breaks can add up to an hour or two per developer per day.
|| ***** Implanting stomata. Costly. Management has suggested we need more surveillance equipment to study the problem of the breaks.
|}

=== Step 3. Reorder the list according to your set of priorities ===
Choose your ordering strategy carefully. Several strategies are possible.
* If this is a learning experience or you are a fan of "only time for putting out fires" cultures, no need for ordering.
* In a low risk environment (no immediate death threats) an "on demand" strategy works well. In this strategy you can use "low hanging fruit" and set up protection for items with a big impact and/or high likelihood of occurrence first.
* In a high risk environment or if any of the items in the list of possible impacts reads " loss of life" or some life-altering experience or you have turned procrastination into an art, best choose an "anticipating strategy", meaning do more research and detailed [[scenario planning]] (food for thought) to find possible [[Threats, detection, protection and (counter) moves|threats and solutions]] overlooked (food for gut).

== Examples ==

=== Journalist, observer or sousveillant in europe ===
A general counter strategy against police misconduct, abuse and brutality has been recording what happens during a protest. This of course, has gotten the police to target (citizen) journalists and observers overtly and covertly. And it is not just the police and government that are interested in us and our data.

The below threats, if happens, make for loss of trust and reputation and that translates to loss of effectiveness as independent observer.
{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Metanoia & Paranoia || low || Metanoia (safety delusion) makes us sitting ducks and paranoia renders us ineffective. || *
|-
| Physical attacks || medium || Physical damage/intimidation
||
|-
| Arrest: That might include direct approaches such as intimidation and asking for information on sources. || medium || Physical danger to those sources.||
|-
| Some contacts are named in reports, and some wish to remain anonymous. If we’re not careful, we might unintentionally disclose contact details or their location http://thenextweb.com/insider/2012/12/03/vice-leaves-metadata-in-photo-of-john-mcafee-pinpointing-him-to-a-location-in-guatemala/
* Phishing
* Guess/hack password
* Metadata
* Mobile trail
|| high || Intimidation/attack/imprisonment of sources. ||*
|-
| Correspondence with contacts includes all sorts of information that we do not want others to know about.
* Phishing
* Guess/hack password
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.
|| **
|-
| Email, social media accounts, websites, communication tools, a pile of passwords. Our passwords may be a used to attack others in our "organisation", and access information we don’t directly hold.
* Phishing emails can be sent from our email or social media accounts to others.
* Passwords can be guessed.
|| high || Followers on social media and people on mailinglists can get spammed.
|| ****
|-
| All work-in-progress, files and documents which we couldn’t publish for all sorts of reasons (space, legality, protection of others).
* Where are they? What forms do they have? Is there a security related documentation policy? Hey who is that man walking out the door with those files?
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.

Intimidation/attack/imprisonment of sources.
|| *
|-
| If we use computers in any way (including phones and even photocopiers) we can safely assume we are being recorded. Can lead to identification of sources through other data, our location, phone or email records.
* Metadata
* Computer forensics
|| high || Intimidation/attack/imprisonment of sources.
|| *
|-
| It may not be our activities they are interested in, but our reach. An example of this was the Syrian Electronic Army targeting E! Online http://ohnotheydidnt.livejournal.com/77479774.html || medium || Publication of hoax information on content management systems (the news website) and social media accounts
|| ***
|-
| Legal attacks include direct approaches such as subpoenas demanding that we reveal a source, or court orders to pass over footage, and increasingly might also include indirect approaches, to companies and organisations holding your information.
|| medium || Physical, social or economic danger to those sources, e.g. losing employment, losing privacy, criminal proceedings, assault.
|| **
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline masters of the internet]]
* [[Timeline merchants of death]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

Threat modeling the quick and dirty way

2015-10-08T09:27:22Z

Lilith2:

The traditional way of doing threat modelling, if it was done at all, was to sit down and think up attacks until bored (often ones that applications defended against anyway) and then declare victory.

* If we have non-security geeks doing the threat modelling then many attacks get missed or mis-identified. An expert facilitator can catch those.
* If we have security geeks involved then there tends to be a focus on attacks like ''sending a server custom-crafted messages that take advantage of the unusual mathematical properties of specially-formatted PKCS #1 message padding in RSA-encrypted data blocks and ignore the fact that the server’s private-key file is world-readable and indexed by Google.'' <ref>Peter Gutmann https://www.cs.auckland.ac.nz/~pgut001/ (yes, the professional paranoid guy, inventor of the Gutmann method, an algorithm for securely erasing the contents of computer hard drives by writing a series of 35 patterns over the region to be erased, and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996.)</ref>
* The problem with checklist-based approaches is that they only work when the attacker is using the same checklist as we are, and isn’t aware that a particular type of attack isn’t supposed to work, then they can walk right past the checklist-standardised defences.
* A variation of checklist-based threat modelling is risk mitigation: documenting every risk we can think of and then getting sign-off from someone in authority on the document. As a defense strategy, this is even less effective. But, more job-security oriented.
* Even more job-security oriented is an obfuscated checklist in a notation recovered from a UFO crash site called Common Criteria. Don't allow people understanding it near sharp objects. Shoot on sight.

In the past this has delivered vulnerabilities and measures that provide the best (theoretical) security but very little effective security:

* The Internet Threat Model: I’m OK, you’re OK, and eavesdropping on credit card information sent over the Internet is the threat. Then we build something that people without a phone can not use.
* Inside-out Threat Model: A wonderful piece of circular reasoning which states that the threat model is whatever the security design is capable of defending against (anything that’s hard to defend against is excluded from the threat model).
* Provable Security for cryptographic algorithms: algorithms being proven secure against the threats that are defined by the provers (the attacker is transformed into some theoretical bogey man capable of doing anything that we know how to protect against).

Then how?

[[File:Dystopia.gif|500px|thumb|right]]

== Basic choreography ==

=== Step 1. Set up a table ===
The first column contains a short description of the threat, the second the likelihood of it occurring, the third what impact it would have if it did happen, and the fourth an assessment (grade) of the time and energy you would need to protect yourself from the threat (for instance you can have no stars denote that there is no protection from that threat, hence it will cost nothing (except for the cost of the impact if it did happen).

=== Step 2. Fill in the table ===
I recommend doing a brainstorm on threats as initial filling of the threat columns in the table, before thinking about the other columns.

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Having to drink || medium || Developer will drop dead within a week, deadline will not be made.

The human body needs water to survive. The maximum time an individual can go without water seems to be a week — an estimate that would certainly be shorter in difficult conditions, like broiling heat.
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Having to eat food || high || Developer will drop dead within three weeks, deadline will not be made.

A human can go for more than three weeks without food (Mahatma Gandhi survived 21 days of complete starvation)
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Needing sleep || high || Like breathing, sleep is essential to humans. It has even been said that one could survive three times longer without food than one could without sleep. One of the better known experiments on this subject, found that depriving rats entirely of sleep resulted in their death, or near-dying state, within 11-32 days (Everson et al. 1989). || ** No long term alternatives known, but the deadline is still four weeks off. We tried adding coffee and coke to the Schmoylent in the previous development cycle.
|-
| Bathroom breaks || high || How many times do people go to the bathroom per day? Loads, and all those little breaks can add up to an hour or two per developer per day.
|| ***** Implanting stomata. Costly. Management has suggested we need more surveillance equipment to study the problem of the breaks.
|}

=== Step 3. Reorder the list according to your set of priorities ===
Choose your ordering strategy carefully. Several strategies are possible.
* If this is a learning experience or you are a fan of "only time for putting out fires" cultures, no need for ordering.
* In a low risk environment (no immediate death threats) an "on demand" strategy works well. In this strategy you can use "low hanging fruit" and set up protection for items with a big impact and/or high likelihood of occurrence first.
* In a high risk environment or if any of the items in the list of possible impacts reads " loss of life" or some life-altering experience or you have turned procrastination into an art, best choose an "anticipating strategy", meaning do more research and detailed [[scenario planning]] (food for thought) to find possible [[Threats, detection, protection and (counter) moves|threats and solutions]] overlooked (food for gut).

== Examples ==

=== Journalist, observer or sousveillant in europe ===
A general counter strategy against police misconduct, abuse and brutality has been recording what happens during a protest. This of course, has gotten the police to target (citizen) journalists and observers overtly and covertly. And it is not just the police and government that are interested in us and our data.

The below threats, if happens, make for loss of trust and reputation and that translates to loss of effectiveness as independent observer.
{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Metanoia & Paranoia || low || Metanoia (safety delusion) makes us sitting ducks and paranoia renders us ineffective. || *
|-
| Physical attacks || medium || Physical damage/intimidation
||
|-
| Arrest: That might include direct approaches such as intimidation and asking for information on sources. || medium || Physical danger to those sources.||
|-
| Some contacts are named in reports, and some wish to remain anonymous. If we’re not careful, we might unintentionally disclose contact details or their location http://thenextweb.com/insider/2012/12/03/vice-leaves-metadata-in-photo-of-john-mcafee-pinpointing-him-to-a-location-in-guatemala/
* Phishing
* Guess/hack password
* Metadata
* Mobile trail
|| high || Intimidation/attack/imprisonment of sources. ||*
|-
| Correspondence with contacts includes all sorts of information that we do not want others to know about.
* Phishing
* Guess/hack password
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.
|| **
|-
| Email, social media accounts, websites, communication tools, a pile of passwords. Our passwords may be a used to attack others in our "organisation", and access information we don’t directly hold.
* Phishing emails can be sent from our email or social media accounts to others.
* Passwords can be guessed.
|| high || Followers on social media and people on mailinglists can get spammed.
|| ****
|-
| All work-in-progress, files and documents which we couldn’t publish for all sorts of reasons (space, legality, protection of others).
* Where are they? What forms do they have? Is there a security related documentation policy? Hey who is that man walking out the door with those files?
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.

Intimidation/attack/imprisonment of sources.
|| *
|-
| If we use computers in any way (including phones and even photocopiers) we can safely assume we are being recorded. Can lead to identification of sources through other data, our location, phone or email records.
* Metadata
* Computer forensics
|| high || Intimidation/attack/imprisonment of sources.
|| *
|-
| It may not be our activities they are interested in, but our reach. An example of this was the Syrian Electronic Army targeting E! Online http://ohnotheydidnt.livejournal.com/77479774.html || medium || Publication of hoax information on content management systems (the news website) and social media accounts
|| ***
|-
| Legal attacks include direct approaches such as subpoenas demanding that we reveal a source, or court orders to pass over footage, and increasingly might also include indirect approaches, to companies and organisations holding your information.
|| medium || Physical, social or economic danger to those sources, e.g. losing employment, losing privacy, criminal proceedings, assault.
|| **
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline masters of the internet]]
* [[Timeline merchants of death]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

Threat modeling the quick and dirty way

2015-10-08T09:26:40Z

Lilith2: /* Examples */

The traditional way of doing threat modelling, if it was done at all, was to sit down and think up attacks until bored (often ones that applications defended against anyway) and then declare victory.

* If we have non-security geeks doing the threat modelling then many attacks get missed or mis-identified. An expert facilitator can catch those.
* If we have security geeks involved then there tends to be a focus on attacks like ''sending a server custom-crafted messages that take advantage of the unusual mathematical properties of specially-formatted PKCS #1 message padding in RSA-encrypted data blocks and ignore the fact that the server’s private-key file is world-readable and indexed by Google.'' <ref>Peter Gutmann https://www.cs.auckland.ac.nz/~pgut001/ (yes, the professional paranoid guy, inventor of the Gutmann method, an algorithm for securely erasing the contents of computer hard drives by writing a series of 35 patterns over the region to be erased, and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996.)</ref>
* The problem with checklist-based approaches is that they only work when the attacker is using the same checklist as we are, and isn’t aware that a particular type of attack isn’t supposed to work, then they can walk right past the checklist-standardised defences.
* A variation of checklist-based threat modelling is risk mitigation: documenting every risk we can think of and then getting sign-off from someone in authority on the document. As a defense strategy, this is even less effective. But, more job-security oriented.
* Even more job-security oriented is an obfuscated checklist in a notation recovered from a UFO crash site called Common Criteria. Don't allow people understanding it near sharp objects. Shoot on sight.

In the past this has delivered vulnerabilities and measures that provide the best (theoretical) security but very little effective security:

* The Internet Threat Model: I’m OK, you’re OK, and eavesdropping on credit card information sent over the Internet is the threat. Then we build something that people without a phone can not use.
* Inside-out Threat Model: A wonderful piece of circular reasoning which states that the threat model is whatever the security design is capable of defending against (anything that’s hard to defend against is excluded from the threat model).
* Provable Security for cryptographic algorithms: algorithms being proven secure against the threats that are defined by the provers (the attacker is transformed into some theoretical bogey man capable of doing anything that we know how to protect against).

Then how?

[[File:Dystopia.gif|500px|thumb|right]]

== Basic choreography ==

=== Step 1. Set up a table ===
The first column contains a short description of the threat, the second the likelihood of it occurring, the third what impact it would have if it did happen, and the fourth an assessment (grade) of the time and energy you would need to protect yourself from the threat (for instance you can have no stars denote that there is no protection from that threat, hence it will cost nothing (except for the cost of the impact if it did happen).

=== Step 2. Fill in the table ===
I recommend doing a brainstorm on threats as initial filling of the threat columns in the table, before thinking about the other columns.

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Having to drink || medium || Developer will drop dead within a week, deadline will not be made.

The human body needs water to survive. The maximum time an individual can go without water seems to be a week — an estimate that would certainly be shorter in difficult conditions, like broiling heat.
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Having to eat food || high || Developer will drop dead within three weeks, deadline will not be made.

A human can go for more than three weeks without food (Mahatma Gandhi survived 21 days of complete starvation)
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Needing sleep || high || Like breathing, sleep is essential to humans. It has even been said that one could survive three times longer without food than one could without sleep. One of the better known experiments on this subject, found that depriving rats entirely of sleep resulted in their death, or near-dying state, within 11-32 days (Everson et al. 1989). || ** No long term alternatives known, but the deadline is still four weeks off. We tried adding coffee and coke to the Schmoylent in the previous development cycle.
|-
| Bathroom breaks || high || How many times do people go to the bathroom per day? Loads, and all those little breaks can add up to an hour or two per developer per day.
|| ***** Implanting stomata. Costly. Management has suggested we need more surveillance equipment to study the problem of the breaks.
|}

=== Step 3. Reorder the list according to your set of priorities ===
Choose your ordering strategy carefully. Several strategies are possible.
* If this is a learning experience or you are a fan of "only time for putting out fires" cultures, no need for ordering.
* In a low risk environment (no immediate death threats) an "on demand" strategy works well. In this strategy you can use "low hanging fruit" and set up protection for items with a big impact and/or high likelihood of occurrence first.
* In a high risk environment or if any of the items in the list of possible impacts reads " loss of life" or some life-altering experience or you have turned procrastination into an art, best choose an "anticipating strategy", meaning do more research and detailed [[scenario planning]] (food for thought) to find possible [[Threats, detection, protection and (counter) moves|threats and solutions]] overlooked (food for gut).

== Examples ==

=== Journalist, observer or sousveillant in europe ===
A general counter strategy against police misconduct, abuse and brutality has been recording what happens during a protest. This of course, has gotten the police to target (citizen) journalists and observers overtly and covertly. And it is not just the police and government that are interested in us and our data.

The below threats, if happens, make for loss of trust and reputation and that translates to loss of effectiveness as independent observer.
{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Metanoia & Paranoia || low || Metanoia (safety delusion) makes us sitting ducks and paranoia renders us ineffective. || *
|-
| Physical attacks || medium || Physical damage/intimidation
||
|-
| Arrest: That might include direct approaches such as intimidation and asking for information on sources. || medium || Physical danger to those sources.||
|-
| Some contacts are named in reports, and some wish to remain anonymous. If we’re not careful, we might unintentionally disclose contact details or their location http://thenextweb.com/insider/2012/12/03/vice-leaves-metadata-in-photo-of-john-mcafee-pinpointing-him-to-a-location-in-guatemala/
* Phishing
* Guess/hack password
* Metadata
* Mobile trail
|| high || Intimidation/attack/imprisonment of sources. ||*
|-
| Correspondence with contacts includes all sorts of information that we do not want others to know about.
* Phishing
* Guess/hack password
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.
|| **
|-
| Email, social media accounts, websites, communication tools, a pile of passwords. Our passwords may be a used to attack others in our "organisation", and access information we don’t directly hold.
* Phishing emails can be sent from our email or social media accounts to others.
* Passwords can be guessed.
|| high || Followers on social media and people on mailinglists can get spammed.
|| ****
|-
| All work-in-progress, files and documents which we couldn’t publish for all sorts of reasons (space, legality, protection of others).
* Where are they? What forms do they have? Is there a security related documentation policy? Hey who is that man walking out the door with those files?
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.

Intimidation/attack/imprisonment of sources.
|| *
|-
| If we use computers in any way (including phones and even photocopiers) we can safely assume we are being recorded. Can lead to identification of sources through other data, our location, phone or email records.
* Metadata
* Computer forensics
|| high || Intimidation/attack/imprisonment of sources.
|| *
|-
| It may not be our activities they are interested in, but our reach. An example of this was the Syrian Electronic Army targeting E! Online http://ohnotheydidnt.livejournal.com/77479774.html || medium || Publication of hoax information on content management systems (the news website) and social media accounts
|| ***
|-
| Legal attacks include direct approaches such as subpoenas demanding that we reveal a source, or court orders to pass over footage, and increasingly might also include indirect approaches, to companies and organisations holding your information.
|| medium || Physical, social or economic danger to those sources, e.g. losing employment, losing privacy, criminal proceedings, assault.
|| **
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline masters of the internet]]
* [[Timeline merchants of death]]
* [[Threats, detection, protection and (counter) moves]]

Threat modeling the quick and dirty way

2015-10-08T09:25:42Z

Lilith2:

The traditional way of doing threat modelling, if it was done at all, was to sit down and think up attacks until bored (often ones that applications defended against anyway) and then declare victory.

* If we have non-security geeks doing the threat modelling then many attacks get missed or mis-identified. An expert facilitator can catch those.
* If we have security geeks involved then there tends to be a focus on attacks like ''sending a server custom-crafted messages that take advantage of the unusual mathematical properties of specially-formatted PKCS #1 message padding in RSA-encrypted data blocks and ignore the fact that the server’s private-key file is world-readable and indexed by Google.'' <ref>Peter Gutmann https://www.cs.auckland.ac.nz/~pgut001/ (yes, the professional paranoid guy, inventor of the Gutmann method, an algorithm for securely erasing the contents of computer hard drives by writing a series of 35 patterns over the region to be erased, and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996.)</ref>
* The problem with checklist-based approaches is that they only work when the attacker is using the same checklist as we are, and isn’t aware that a particular type of attack isn’t supposed to work, then they can walk right past the checklist-standardised defences.
* A variation of checklist-based threat modelling is risk mitigation: documenting every risk we can think of and then getting sign-off from someone in authority on the document. As a defense strategy, this is even less effective. But, more job-security oriented.
* Even more job-security oriented is an obfuscated checklist in a notation recovered from a UFO crash site called Common Criteria. Don't allow people understanding it near sharp objects. Shoot on sight.

In the past this has delivered vulnerabilities and measures that provide the best (theoretical) security but very little effective security:

* The Internet Threat Model: I’m OK, you’re OK, and eavesdropping on credit card information sent over the Internet is the threat. Then we build something that people without a phone can not use.
* Inside-out Threat Model: A wonderful piece of circular reasoning which states that the threat model is whatever the security design is capable of defending against (anything that’s hard to defend against is excluded from the threat model).
* Provable Security for cryptographic algorithms: algorithms being proven secure against the threats that are defined by the provers (the attacker is transformed into some theoretical bogey man capable of doing anything that we know how to protect against).

Then how?

[[File:Dystopia.gif|500px|thumb|right]]

== Basic choreography ==

=== Step 1. Set up a table ===
The first column contains a short description of the threat, the second the likelihood of it occurring, the third what impact it would have if it did happen, and the fourth an assessment (grade) of the time and energy you would need to protect yourself from the threat (for instance you can have no stars denote that there is no protection from that threat, hence it will cost nothing (except for the cost of the impact if it did happen).

=== Step 2. Fill in the table ===
I recommend doing a brainstorm on threats as initial filling of the threat columns in the table, before thinking about the other columns.

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Having to drink || medium || Developer will drop dead within a week, deadline will not be made.

The human body needs water to survive. The maximum time an individual can go without water seems to be a week — an estimate that would certainly be shorter in difficult conditions, like broiling heat.
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Having to eat food || high || Developer will drop dead within three weeks, deadline will not be made.

A human can go for more than three weeks without food (Mahatma Gandhi survived 21 days of complete starvation)
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Needing sleep || high || Like breathing, sleep is essential to humans. It has even been said that one could survive three times longer without food than one could without sleep. One of the better known experiments on this subject, found that depriving rats entirely of sleep resulted in their death, or near-dying state, within 11-32 days (Everson et al. 1989). || ** No long term alternatives known, but the deadline is still four weeks off. We tried adding coffee and coke to the Schmoylent in the previous development cycle.
|-
| Bathroom breaks || high || How many times do people go to the bathroom per day? Loads, and all those little breaks can add up to an hour or two per developer per day.
|| ***** Implanting stomata. Costly. Management has suggested we need more surveillance equipment to study the problem of the breaks.
|}

=== Step 3. Reorder the list according to your set of priorities ===
Choose your ordering strategy carefully. Several strategies are possible.
* If this is a learning experience or you are a fan of "only time for putting out fires" cultures, no need for ordering.
* In a low risk environment (no immediate death threats) an "on demand" strategy works well. In this strategy you can use "low hanging fruit" and set up protection for items with a big impact and/or high likelihood of occurrence first.
* In a high risk environment or if any of the items in the list of possible impacts reads " loss of life" or some life-altering experience or you have turned procrastination into an art, best choose an "anticipating strategy", meaning do more research and detailed [[scenario planning]] (food for thought) to find possible [[Threats, detection, protection and (counter) moves|threats and solutions]] overlooked (food for gut).

== Examples ==

=== Protesting in the united states ===

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| || || ||
|}

=== Protesting in the westbank and gaza strip ===

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| || || ||
|}

=== Blogging from egypt ===

=== Journalist, observer or sousveillant in europe ===
A general counter strategy against police misconduct, abuse and brutality has been recording what happens during a protest. This of course, has gotten the police to target (citizen) journalists and observers overtly and covertly. And it is not just the police and government that are interested in us and our data.

The below threats, if happens, make for loss of trust and reputation and that translates to loss of effectiveness as independent observer.
{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Metanoia & Paranoia || low || Metanoia (safety delusion) makes us sitting ducks and paranoia renders us ineffective. || *
|-
| Physical attacks || medium || Physical damage/intimidation
||
|-
| Arrest: That might include direct approaches such as intimidation and asking for information on sources. || medium || Physical danger to those sources.||
|-
| Some contacts are named in reports, and some wish to remain anonymous. If we’re not careful, we might unintentionally disclose contact details or their location http://thenextweb.com/insider/2012/12/03/vice-leaves-metadata-in-photo-of-john-mcafee-pinpointing-him-to-a-location-in-guatemala/
* Phishing
* Guess/hack password
* Metadata
* Mobile trail
|| high || Intimidation/attack/imprisonment of sources. ||*
|-
| Correspondence with contacts includes all sorts of information that we do not want others to know about.
* Phishing
* Guess/hack password
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.
|| **
|-
| Email, social media accounts, websites, communication tools, a pile of passwords. Our passwords may be a used to attack others in our "organisation", and access information we don’t directly hold.
* Phishing emails can be sent from our email or social media accounts to others.
* Passwords can be guessed.
|| high || Followers on social media and people on mailinglists can get spammed.
|| ****
|-
| All work-in-progress, files and documents which we couldn’t publish for all sorts of reasons (space, legality, protection of others).
* Where are they? What forms do they have? Is there a security related documentation policy? Hey who is that man walking out the door with those files?
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.

Intimidation/attack/imprisonment of sources.
|| *
|-
| If we use computers in any way (including phones and even photocopiers) we can safely assume we are being recorded. Can lead to identification of sources through other data, our location, phone or email records.
* Metadata
* Computer forensics
|| high || Intimidation/attack/imprisonment of sources.
|| *
|-
| It may not be our activities they are interested in, but our reach. An example of this was the Syrian Electronic Army targeting E! Online http://ohnotheydidnt.livejournal.com/77479774.html || medium || Publication of hoax information on content management systems (the news website) and social media accounts
|| ***
|-
| Legal attacks include direct approaches such as subpoenas demanding that we reveal a source, or court orders to pass over footage, and increasingly might also include indirect approaches, to companies and organisations holding your information.
|| medium || Physical, social or economic danger to those sources, e.g. losing employment, losing privacy, criminal proceedings, assault.
|| **
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline masters of the internet]]
* [[Timeline merchants of death]]
* [[Threats, detection, protection and (counter) moves]]

Threat modeling the quick and dirty way

2015-10-08T09:20:22Z

Lilith2:

The usual pentesting seems to focus mostly on vulnerabilities and measures that provide the best (theoretical) security but very little effective security.

* The Internet Threat Model: I’m OK, you’re OK, and eavesdropping on credit card information sent over the Internet is the threat. Then we build something that people without a phone can not use.
* Inside-out Threat Model: A wonderful piece of circular reasoning which states that the threat model is whatever the security design is capable of defending against (anything that’s hard to defend against is excluded from the threat model).
* Provable Security for cryptographic algorithms: algorithms being proven secure against the threats that are defined by the provers (the attacker is transformed into some theoretical bogey man capable of doing anything that we know how to protect against).

[[File:Dystopia.gif|500px|thumb|right]]

== So how are we going to build a realistic threat model for applications? ==

The traditional way of doing this, if it was done at all, was to sit down and think up attacks until bored (often ones that applications defended against anyway) and then declare victory.

* If we have non-security geeks doing the threat modelling then many attacks get missed or mis-identified. An excellent tech facilitator catches those.
* If we have security geeks involved then they tend to focus on attacks like ''sending a server custom-crafted messages that take advantage of the unusual mathematical properties of specially-formatted PKCS #1 message padding in RSA-encrypted data blocks and ignore the fact that the server’s private-key file is world-readable and indexed by Google.'' <ref>Peter Gutmann https://www.cs.auckland.ac.nz/~pgut001/ (yes, the professional paranoid guy, inventor of the Gutmann method, an algorithm for securely erasing the contents of computer hard drives by writing a series of 35 patterns over the region to be erased, and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996.)</ref>
* The problem with checklist-based approaches is that they only work when the attacker is using the same checklist as we are, and isn’t aware that a particular type of attack isn’t supposed to work, then they can walk right past the checklist-standardised defences.
* A variation of checklist-based threat modelling is risk mitigation: documenting every risk we can think of and then getting sign-off from someone in authority on the document. As a defense strategy, this is even less effective. But, more job-security oriented.
* Even more job-security oriented is an obfuscated checklist in a notation recovered from a UFO crash site called Common Criteria. Don't allow people understanding it near sharp objects. Shoot on sight.

== Basic choreography ==

=== Step 1. Set up a table ===
The first column contains a short description of the threat, the second the likelihood of it occurring, the third what impact it would have if it did happen, and the fourth an assessment (grade) of the time and energy you would need to protect yourself from the threat (for instance you can have no stars denote that there is no protection from that threat, hence it will cost nothing (except for the cost of the impact if it did happen).

=== Step 2. Fill in the table ===
I recommend doing a brainstorm on threats as initial filling of the threat columns in the table, before thinking about the other columns.

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Having to drink || medium || Developer will drop dead within a week, deadline will not be made.

The human body needs water to survive. The maximum time an individual can go without water seems to be a week — an estimate that would certainly be shorter in difficult conditions, like broiling heat.
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Having to eat food || high || Developer will drop dead within three weeks, deadline will not be made.

A human can go for more than three weeks without food (Mahatma Gandhi survived 21 days of complete starvation)
|| * Blend together half a gallon of water, three and a half tablespoons of macadamia nut oil and a 16-ounce bag of powder called Schmoylent. Then pour the beige beverage into jars and chill them before bringing the containers to work the next day http://timesofindia.indiatimes.com/tech/jobs/No-time-to-eat-Silicon-Valley-drinks-its-meals/articleshow/47424226.cms
|-
| Needing sleep || high || Like breathing, sleep is essential to humans. It has even been said that one could survive three times longer without food than one could without sleep. One of the better known experiments on this subject, found that depriving rats entirely of sleep resulted in their death, or near-dying state, within 11-32 days (Everson et al. 1989). || ** No long term alternatives known, but the deadline is still four weeks off. We tried adding coffee and coke to the Schmoylent in the previous development cycle.
|-
| Bathroom breaks || high || How many times do people go to the bathroom per day? Loads, and all those little breaks can add up to an hour or two per developer per day.
|| ***** Implanting stomata. Costly. Management has suggested we need more surveillance equipment to study the problem of the breaks.
|}

=== Step 3. Reorder the list according to your set of priorities ===
Choose your ordering strategy carefully. Several strategies are possible.
* If this is a learning experience or you are a fan of "only time for putting out fires" cultures, no need for ordering.
* In a low risk environment (no immediate death threats) an "on demand" strategy works well. In this strategy you can use "low hanging fruit" and set up protection for items with a big impact and/or high likelihood of occurrence first.
* In a high risk environment or if any of the items in the list of possible impacts reads " loss of life" or some life-altering experience or you have turned procrastination into an art, best choose an "anticipating strategy", meaning do more research and detailed [[scenario planning]] (food for thought) to find possible [[Threats, detection, protection and (counter) moves|threats and solutions]] overlooked (food for gut).

== Examples ==

=== Protesting in the united states ===

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| || || ||
|}

=== Protesting in the westbank and gaza strip ===

{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| || || ||
|}

=== Blogging from egypt ===

=== Journalist, observer or sousveillant in europe ===
A general counter strategy against police misconduct, abuse and brutality has been recording what happens during a protest. This of course, has gotten the police to target (citizen) journalists and observers overtly and covertly. And it is not just the police and government that are interested in us and our data.

The below threats, if happens, make for loss of trust and reputation and that translates to loss of effectiveness as independent observer.
{| class="wikitable sortable"
|-
! Threat !! Likelihood !! Impact !! Protection
|-
| Metanoia & Paranoia || low || Metanoia (safety delusion) makes us sitting ducks and paranoia renders us ineffective. || *
|-
| Physical attacks || medium || Physical damage/intimidation
||
|-
| Arrest: That might include direct approaches such as intimidation and asking for information on sources. || medium || Physical danger to those sources.||
|-
| Some contacts are named in reports, and some wish to remain anonymous. If we’re not careful, we might unintentionally disclose contact details or their location http://thenextweb.com/insider/2012/12/03/vice-leaves-metadata-in-photo-of-john-mcafee-pinpointing-him-to-a-location-in-guatemala/
* Phishing
* Guess/hack password
* Metadata
* Mobile trail
|| high || Intimidation/attack/imprisonment of sources. ||*
|-
| Correspondence with contacts includes all sorts of information that we do not want others to know about.
* Phishing
* Guess/hack password
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.
|| **
|-
| Email, social media accounts, websites, communication tools, a pile of passwords. Our passwords may be a used to attack others in our "organisation", and access information we don’t directly hold.
* Phishing emails can be sent from our email or social media accounts to others.
* Passwords can be guessed.
|| high || Followers on social media and people on mailinglists can get spammed.
|| ****
|-
| All work-in-progress, files and documents which we couldn’t publish for all sorts of reasons (space, legality, protection of others).
* Where are they? What forms do they have? Is there a security related documentation policy? Hey who is that man walking out the door with those files?
|| medium || Sensitive information can be accessed. This information can be used to try to coerce us into not releasing and/or publishing information. Can also be used for smear campaigns and to sue us or our sources.

Intimidation/attack/imprisonment of sources.
|| *
|-
| If we use computers in any way (including phones and even photocopiers) we can safely assume we are being recorded. Can lead to identification of sources through other data, our location, phone or email records.
* Metadata
* Computer forensics
|| high || Intimidation/attack/imprisonment of sources.
|| *
|-
| It may not be our activities they are interested in, but our reach. An example of this was the Syrian Electronic Army targeting E! Online http://ohnotheydidnt.livejournal.com/77479774.html || medium || Publication of hoax information on content management systems (the news website) and social media accounts
|| ***
|-
| Legal attacks include direct approaches such as subpoenas demanding that we reveal a source, or court orders to pass over footage, and increasingly might also include indirect approaches, to companies and organisations holding your information.
|| medium || Physical, social or economic danger to those sources, e.g. losing employment, losing privacy, criminal proceedings, assault.
|| **
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline masters of the internet]]
* [[Timeline merchants of death]]
* [[Threats, detection, protection and (counter) moves]]

Digital threats, detection, protection and (counter) moves

2015-10-08T09:12:49Z

Lilith2:

== Social engineering ==
These tables were started off from http://www.itsecurity.be/social-engineering-what-is-it-and-how-to-defend-yourself and then added to. I started off from a to gendersec external source to increase chances of finding that which we may be overlooking.

=== Internal threats ===

{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Power of scarcity
|| Scarcity is when people are told something they need or want has limited availability and to get it they must comply with a certain attitude or action. Many times the desired behavior is not even spoken, but the way it is conveyed is by showing people who are acting "properly" getting rewards.

Mental shortcuts are often beneficial because they enable us to more efficiently navigate the complexity of life. But the power of scarcity can also mislead us. If some people get something not everyone else has (energy, food, water), it makes them feel "special", "unique", "high status", and gives a "position" to defend. And fear of loss of that is an attachment.

An example is a government, in this case South Africa, taking something necessary to life, and making it "scarce" and available only to supporters — a malicious, but very effective, manipulation tactic: http://www.social-engineer.org/wiki/archives/Governments/Governments-FoodElectionWeapon.html See [[timeline merchants of death]] for more examples.

|| Scarcity complicates Gut/Head decision processes because, often, there is only a brief window of opportunity in which we can select something that is in scarce supply. When we can see the supply diminishing before our eyes, we feel particularly compelled to take action.

Each context and scarcity is different, but mind not to rule out the head too quickly in [[Scenario planning#Gut.2C head.2C and decision making|"Gut versus Head"]] decisions.
|-
| Fear of authority || Many people are apprehensive in the presence of someone they perceive to be an authority figure, it is not that person they are apprehensive about but most likely the position and power of the person that intimidates them.
<nowiki> </nowiki>The attackers take on roles of authority figures such as law enforcement officers or high-ranking company officials to extract sensitive information from the victims.
|| Protect yourself with learning about [[Self-dox#Elicitation|elicitation]] (in roleplay settings) and then, time for some controlled folly!
|-
| Desire to be helpful
|| People in their desire to be helpful and to solve other peoples queries, give out a lot of information that otherwise should not be. || Do not disclose information to an outsider as it could give an attacker a chance to get unauthorised access.
|-
| Laziness || All of us have come across some job that requires us to do only a specified set of activities and nothing more. This causes boredom to the person who performs the same task repeatedly on a daily basis and over time the "bored" learn "shortcuts" to do the tasks using minimal effort while meeting targets. This leads to a laid back attitude and becoming susceptible to attackers who target such individuals knowing they can get the required information with much ease. || Find ways of keeping yourself alert. Go do something else. You are becoming a sitting duck.
|-
| Ego || Many a times, the attacker makes a person more emotionally sure of himself/herself, thus removing the logical awareness of the security breach that is occurring. The result is that the person being hacked senses no harm in providing whatever it is that the attacker is requesting. The reason that such an attack succeeds is that the attacker is a receptive audience for victims to display how much knowledge they have. || The ego isn’t going to go away and whether it makes you vulnerable depends on the context. A bit of swagger based on achievements can be quite okay now and then! In some other contexts allowing your ego to call the shots can be a problem.

The simplest detector is asking yourself: Do you feel superior (or inferior) to others? If yes, then your ego is in control. Depending on context choose "not now, not here, not with this" or "do some counter-intelligence" or "Arrrrr, and a bottle of rum!"
|-
| Excitement of victory & fear of loss || Yes! An opportunity! Easy money!!! Out of excitement security is switched off, links are clicked and documents downloaded that turn out to be corrupted (and contained malware that allows the email sender to gain remote access to the machine, or someone's funds, or data, or ...).

Or, ''"You have won 1 Million Dollars and to claim the winning amount, deposit $75,000 in Account number: XXXXXX in 10 days from receiving this e-mail, failing to which the winning amount would be declared unclaimed and there would be a nee lucky-draw to decide the next winner"''

An example is the 419 scams many people fell for http://www.social-engineer.org/wiki/archives/ConMen/ConMen-Scam-NigerianFee.html
|| It is another ego thing isn't it?
|-
| Insufficient knowledge || People with insufficient knowledge can easily be exploited by creating a sense of urgency and not allowing much time to think and understanding the fact that they are under attack. || Gather (and spread) knowledge in a relaxed way. Stay with your own timing. Do the best you can.
|}

=== External threats ===
{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Shoulder surfing || Shoulder surfing is a security attack where-in, the attacker uses observational techniques, such as looking over someone's shoulder, to get information while they are performing some action that involves explicit usage of sensitive, visible information. This can be performed at a close range as well as at a long range using binoculars or other vision-enhancing devices. || Don't work on anything important in public.
|-
| Dumpster diving || Going through the trash can yield one of the most lucrative payoffs for information gathering. People often throw away invoices, notices, letters, CDs, computers, USB keys, and a plethora of other devices and reports that can truly give amazing amounts of information. The attacker can use these items to get a huge amount of information about people, organisations they participate in, and network structure.

Some people shred documents but some types of shredding can be thwarted with a little time and patience and some tape.
|| Mind what you throw away where and how.

Using a shredder that shreds both directions into a fine minced mess makes taping documents back together nearly impossible. Or incinerate. Or both.
|-
| Role playing || Role playing is one of the key weapons for a social engineer. It involves persuading or gathering information through the use of online chat sessions, emails, phones or any other method that you use to interact online with others, and in which the social engineer plays the role of a helpdesk or technician, helplessness, or whatever may work in that context to get targets to divulge confidential information. || Don't "burn" the social engineer. Apply controlled folly instead, and then let's see who is the cat here and who the mouse: Feed them incorrect information. Remember what the intent is. Intent is the key element in intelligence and counter-intelligence operations.
|-
| Trojan horses || It is one of the most predominant methods currently used by online criminals and intelligence agencies that involve tricking victims into downloading a malicious file to their machine, which on execution creates a backdoor in the machine that can be used by the attacker any time in the future and thus having complete access of the victim's machine. Compromising a browser is relatively easy and it is cross-platform, hence an often chosen attack vector. <ref>Meet The Hackers Who Sell Spies The Tools To Crack Your PC (And Get Paid Six-Figure Fees) http://www.forbes.com/sites/andygreenberg/2012/03/21/meet-the-hackers-who-sell-spies-the-tools-to-crack-your-pc-and-get-paid-six-figure-fees/</ref>.
<nowiki> </nowiki>Among all of the NSA hacking operations exposed by whistleblower Edward Snowden over the last two years, one in particular has stood out for its sophistication and stealthiness. Known as Quantum Insert, the man-on-the-side hacking technique has been used to great effect since 2005 by the NSA and its partner spy agency, Britain’s GCHQ, to hack into high-value, hard-to-reach systems and implant malware.

|| Detection focuses on identifying anomalies in the data packets that get sent to a victim’s browser client when the browser attempts to access web pages http://blog.fox-it.com/2015/04/20/deep-dive-into-quantum-insert/

For more such tools run by intelligence agency for targeted surveillance see [[#Infected_firmware|infected firmware]] below.
|-
| Phishing || Phishing is the act of creating and using websites and e-mails designed to look like those of well-known legitimate businesses, financial institutions and government agencies to deceive receivers into disclosing their personal information.
|| These are semantic attacks.
* Be suspicious of any email with urgent requests for personal (financial) information.
* Set your email client to receive email in plaintext, never html format.
* Check any url in the email carefully. It can be just one letter different or missing.
* If phishy, report it.
|-
| Information on websites, online forums and social media|| Huge amounts of information regarding organisational structures (formal and informal), email adresses, phone numbers, are all available publicly on websites, forums and social media. This information can be used by the attacker to refine his approach and create a plan on whom to target and methods most likely to succeed. || Several moves are possible (not necessarily exclusive):
* Delete your social media presence
* Use different identities for your various social contexts
|-
| Reverse social engineering || A reverse social engineering attack is an attack in which an attacker convinces a target that he or she has a problem or might have a certain problem in the future and that the attacker, is ready to help solve the problem.

Reverse social engineering involves three parts:
* Sabotage: After the attacker gains a simple access to the system, he corrupts the system or gives it an appearance of being corrupted. When the user sees the system in the corrupted state, he starts looking for help so as to solve the problem.
* Marketing: In order to make sure that the user approaches the attacker with the problem, the attacker advertises himself as the only person who can solve the problem.
* Support: In this step, the attacker gains the trust of the target and obtains access to sensitive information.
|| If your alarm didn't go off in the sabotage step, when your system appearing corrupt coincided with this remarkable good fortune of someone being right there at the right time ready to help, then let your alarm go off at the [[Propaganda#False_Dilemma|false dilemma]] you are presented with in the marketing step.
|}

== Targeted surveillance ==

=== Hardware implants ===
Unless you are targeted by a government intelligence agency, there seems to be no need to worry about installing commodity hardware from reputable vendors.

Detection: Looking for physical devices will always be the easiest solution to detect them. The links to NSA exploits of the day were added for the comments. :)

{| class="wikitable"
|-
! !! More information !! Possible types of attacks !! Detection
|-
| Godsurge || Godsurge is a physical device plugged-in to the Joint Test Action Group or JTAG headers on a system's motherboard. JTAG headers can be found on many systems and are notoriously common in embedded devices. These headers are used during the development process for debugging purposes: they give you a direct interface with the CPU and are extremely helpful. They are commonly left on the production boards, so finding them on a device is normal and not a security concern. However, if there is a chip or board wired in to a device's JTAG headers that you did not wire in yourself, then something fishy may be going on. || The JTAG debugging interface can be used to reflash the BIOS from scratch, for example loading a compromised version of the software.

|| Look for the JTAG connecter on the motherboard. The location of the JTAG headers may differ, but they tend to be near the CPU and may have exposed pins (or not). See the Wikipedia page on JTAG for more information and to see what they look like <ref>Wikipedia JTAG http://en.wikipedia.org/wiki/Joint_Test_Action_Group</ref>
|-
| Ginsu and Bulldozer
|| Ginsu provides software application persistence on target systems with the PCI bus hardware implant, Bulldozer
http://leaksource.files.wordpress.com/2013/12/nsa-ant-ginsu.jpg

https://www.schneier.com/blog/archives/2014/01/ginsu_nsa_explo.html
|| Exploit persistence from a PCI card ROM
|| Open up the computer's case and look for a PCI card that does not belong. For example, if you find a PCI card that appears to serve no purpose (e.g.. not your network card or it has no external ports), then perhaps try removing it and see how things work. If a few black SUVs roll up to your house after unplugging the PCI card, it's probably not because your domicile is the set for a rap superstar's new music video. Or maybe it is.
|-
| Cottonmouth I, Cottonmouth II and Cottonmouth III
|| These bugs are embedded somewhere along the USB bus and function as an air gap bridge to assist in exfiltration of data as well as allow persistent compromise. It can be embedded directly in the USB headers in an existing USB peripheral (USB hub or keyboard). These devices allow for exfiltration of data over unknown radio frequencies to listening devices in the area, even on a system that is not connected to the internet.
http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-i.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-i_n.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-ii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-ii.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonomouth-iii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-iii.html
|| USB host attack
|| Open up the keyboard or USB hub and identify a board that serves no purpose to the device. The malicious USB device would also likely show up on your computer's list of USB devices, so just checking there would be a good place to start.
|}

=== Radio frequency exfiltration ===
The devices used can vary, but they all employ a similar method of communication via an unknown radio/radar protocol. The frequencies are not known and based on the information leaked the devices are passive (only power on when data is being extracted) making identification of (type of) signal (and intelligence) hard.

Detection: First check for evidence of a device which has been wired in to an existing device such as a keyboard or other peripheral. The SpiderBlog describes the following possibility: You can check if there are RF transmitters in a device by monitoring the spectrum (using an amateur RF listening bug detector) while the device is off (to get a baseline for ambient RF background noise) and then monitoring it again after the suspect device is turned on and transmitting data via radio frequency. The detection device would pick up the signal and alert the user. <ref>Detecting A Surveillance State - Part 2 Radio Frequency Exfiltration https://www.trustwave.com/Resources/SpiderLabs-Blog/Detecting-A-Surveillance-State---Part-2-Radio-Frequency-Exfiltration/
</ref> Further analysis of the signal and its intelligence is still hard, as the exact protocols are (still) unknown.
{| class="wikitable"
|-
! !! More information !! Possible types of attack
|-
| Howlermonkey
|| Short to medium range Radio Transceiver

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/howlermonkey_ns.html
|| HOWLERMONKEY is a COTS-based transceiver deigned to be compatible with CONJECTURE/SPECULATION networks and STRIKEZONE devices running a HOWLERMONKEY personality. PCB layouts are tailored to individual implant space requirements and can vary greatly in form factor.
|-
| Ragemaster
|| Hardware implant in a VGA cable that sends video data over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/03/ragemaster_nsa.html
|| The RAGEMASTER taps the red video line between the video card within the desktop unit and the computer monitor, typically an LCD. When the RAGEMASTER is illuminated by a radar unit, the illuminating signal is modulated with the red video information. This information is re-radiated, where it is picked up at the radar, demodulated, and passed onto the processing unit, such as a LFS-2 and an external monitor, NIGHTWATCH, GOTHAM, or (in the future) VIEWPLATE. The processor recreates the horizontal and vertical sync of the targeted monitor, thus allowing TAO personnel to see what is displayed on the targeted monitor.
|-
| Loudauto
|| Hardware device that sends amplified audio over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/loudauto_nsa_ex.html

|| Room audio is picked up by the microphone and converted into an analog electrical signal. This signal is used to pulse position modulate (PPM) a square wave signal running at a pre-set frequency. This square wave is used to turn a FET (field effect transistor) on and off. When the unit is illuminated with a CW signal from a nearby radar unit, the illuminating signal is amplitude-modulated with the PPM square wave. This signal is re-radiated, where it is picked up by the radar, then processed to recover the room audio. Processing is currently performed by COTS equipment with FM demodulation capability (Rohde & Schwarz FSH-series portable spectrum analyzers, etc.) LOUDAUTO is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|-
| Surleyspawn
|| Hardware implant in a keyboard that emits keystrokes over RF

http://leaksource.files.wordpress.com/2013/12/nsa-ant-surlyspawn.jpg

https://www.schneier.com/blog/archives/2014/02/surlyspawn_nsa.html
|| The board taps into the data line from the keyboard to the processor. The board generates a square wave oscillating at a preset frequency. The data-line signal is used to shift the square wave frequency higher or lower, depending on the level of the data-line signal. The square wave, in essence, becomes frequency shift keyed (FSK). When the unit is illuminated by a CW signal from a nearby radar, the illuminating signal is amplitude-modulated (AM) with this square wave. The signal is re-radiated, where it is received by the radar, demodulated, and the demodulated signal is processed to recover the keystrokes. SURLYSPAWN is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|}

=== Infected firmware ===
Detection: Dump the BIOS to a bin file and compare the hash with a clean BIOS hash. See flashrom for identifying, reading, writing, erasing, and verifying BIOS/ROM/flash chips <ref>Debian packages: flashrom https://packages.debian.org/jessie/flashrom</ref>.

Removal: For each of these infections, where applicable, pulling the chip and replacing it with a new freshly burned BIOS chip or compact flash card would be sufficient. When dealing with built-in firmware it's a bit more difficult than pulling and replacing. You will need to re-flash the device using an operating system that is not at risk of being attacked by the infected firmware. You could boot the device into a low level OS in hopes that the firmware infection isn't able to protect itself. Or, you could wire in a debugging header to the device (such as JTAG , if available) to read or write the firmware on the device to clean things up for good.

{| class="wikitable"
|-
! !! More information !! Possible types of attack || Protection
|-
| Dietybounce
|| Motherboard BIOS Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-deitybounce.jpg?w=1208&h=1562

https://www.schneier.com/blog/archives/2014/01/nsa_exploit_of.html

|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS on a target machine to implant DEITYBOUNCE and its payload (the implant installer). Implantation via interdiction may be accomplished by nontechnical operator through use of a USB thumb drive. Once implanted, DEITYBOUNCE's frequency of execution (dropping the payload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Swap
|| Hard Drive Firmware Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-swap.jpg

https://www.schneier.com/blog/archives/2014/02/swap_nsa_exploi.html
|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS and TWISTEDKILT to write the Host Protected Area on the hard drive on a target machine in order to implant SWAP and its payload (the implant installer). Once implanted, SWAP's frequency of execution (dropping the playload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Headwater, sierramontana, and jetplow
|| Firmware backdoors that target popular networking hardware

https://leaksource.files.wordpress.com/2013/12/nsa-ant-headwater.jpg

https://www.schneier.com/blog/archives/2014/01/headwater_nsa_e.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-sierramontana.jpg

https://www.schneier.com/blog/archives/2014/01/sierramontana_n.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-jetplow.jpg

https://www.schneier.com/blog/archives/2014/01/jetplow_nsa_exp.html
|| HEADWATER PBD implant will be transferred remotely over the Internet to the selected target router by Remote Operations Center (ROC) personnel. After the transfer, the PBD will be installed in the router's boot ROM via an upgrade command. The PBD will then be activated after a system reboot. Once activated, the ROC operators will be able to use DNT's HAMMERMILL Insertion Tool (HIT) to control the PBD as it captures and examines all IP packets passing through the host router.

Currently, the intended DNT Implant to persist is VALIDATOR, which must be run as a user process on the target operating system. The vector of attack is the modification of the target's BIOS. The modification will add the necessary software to the BIOS and modify its software to execute the SIERRAMONTANA implant at the end of its native System Management Mode (SMM) handler.

JETPLOW persists DNT's BANANAGLEE software implant and modifies the Cisco firewall's operating system (OS) at boot time. If BANANAGLEE support is not available for the booting operating system, it can install a Persistent Backdoor (PDB) designed to work with BANANAGLEE'S communications structure, so that full access can be reacquired at a later time.
|| Most routers can be flashed with openWRT http://wiki.openwrt.org/toh/start
|}

== Dragnet surveillance ==

This table lists theoretical defenses and detection methods for selected groups of leaked [[surveillance]] programs and services. This is just a thought experiment covering (theoretical) defenses against these attacks and not intended to spread fear, uncertainty or doubt about surveillance states.

Due to the age and limited scope of the leaked documents and what we are up against <ref>Cryptome: Communications privacy folly http://cryptome.org/2012/06/comms-folly.htm
</ref>, the defenses mentioned in these tables should not be relied upon for protection and I make no guarantees to their accuracy. Do your own research and make informed decisions, knowing it is impossible to be completely safe. Things move incredibly fast in this arena and I will update these tables when more is found and/or theorised.

Ideally we discuss these exploits and counter moves without the NSA being privy to those discussion, but that would make spreading possible counter moves too slow for the surveillance development cycle (security arms race) and effectively exclude non-techie activists from being able to defend themselves to some extent, so the next best solution is to discuss everything in the open, rather than not discussing them at all. May we not have to rely on a new class of technocracy.

Best protection: Smash your PC to bits, distribute the pieces randomly among a dozen scrap heaps, and [https://undisconnect.miraheze.org/wiki/Rock_bottom move into the woods, deep in the woods ... ]
{| class="wikitable"
|-
! !! More information !! !! Protection
|-
| PRISM || PRISM is part of the overall NSA surveillance effort - a program authorised in the united states under the FISA Amendments Act (FAA), now located in Section 702 of the FISA, that allows the NSA to collect communications of specifically identified foreign targets.

When people in the rest of the world use the net, they are effectively using US-based services, making them a legal target for US intelligence. NSA has no "direct access" to the servers of companies like Microsoft, Facebook and Google - other agencies collect the data. More [[Surveillance#PRISM|here]]
|| Gathers emails, chat--video, voice, photos, stored data, VoIP, file transfers, video conferencing, notifications of target activity--logins, etc, online social networking details, and special requests collected by other programs from nine major Silicon Valley technology companies: Facebook, Google, Yahoo, Microsoft, PalTalk, Skype, YouTube, Apple, and AOL.
||
* Put political pressure on the united states - economical and political dependencies are making that unlikely, see the power of scarcity above.

* Avoid using popular web services.
* FOIA's, court cases on legitimacy
|-
| Upstream collection
|| It appears that through Upstream collection, the NSA gains access to data by forming partnerships with both foreign intelligence agencies and foreign telecommunication companies, by partnering with united states telecommunications companies that then make agreements with international companies for their internet data. More [[Surveillance#Upstream_collection|here]]
|| BLARNEY seems to focus specifically on metadata information about communications traffic and network devices (where and when they were sent).

FAIRVIEW collects just about everything on packet and session levels.
||
* Research which countries and telecommunication companies participate
* Avoid using services associated with these programs
* [[Step 1#Understanding_and_minimising_our_digital_shadows|Understanding and minimising our digital shadows]]
* Court cases on legitimacy
|}

== Censorship ==
Table initially filled with data from How to effectively argue against Internet Censorship ideas http://rys.io/en/94.txt.

Proxy servers, especially anonymous ones, located outside the area where a censorship solution is deployed can be used quite easily to circumvent any blocking method; users can modify their operating system or browser settings, or install browser additions that make using this circumvention method trivial. It is possible to block the proxy servers themselves (via IP-blocking, keyword blocking, etc.), however it is infeasible to block them all, as they are easy to set-up.

Virtual Private Networks (including “poor man’s VPNs” like SSH tunnels) require more technical prowess and usually a (usually commercial) VPN service (or SSH server) outside the area with blocking deployed. Blocking all VPN/SSH traffic is possible, but requires deep packet inspection and is a serious problem for many legitimate businesses using VPNs (and SSH) as their daily tools of trade, to allow their employees access to corporate networks from outside physical premises, via a secured link on the Internet.

TOR, or The Onion Router, is a very effective (if a bit slow) circumvention method. It is quite easy to set-up — users can simply download the TOR Browser Bundle and use it to access the Internet. Due to the way it works it is nigh-impossible to block TOR traffic (as it looks just like vanilla HTTPS traffic), to the point that it is known to allow access to the uncensored Internet to those living in areas with most aggressive Internet censorship policies — namely China, North Korea and Iran. See Tor threats.

None of the censorship solutions is able to block content on darknets — virtual networks accessible anonymously only via specialised software (for instance TOR, I2P, FreeNet), and guaranteeing high resilience to censorship through technical composition of the networks themselves. Because darknets are both practically impossible to block entirely and not allowing for any content blocking within them, they are effectively the ultimate circumvention methods. The downside to using darknets is their lower bandwidth. Deploying Internet censorship pushes the to-be-blocked content into darknets, making it ever-harder for law enforcement gather evidence and researchers gather data on the popularity of a given type of censored content.
{| class="wikitable"

! Blocking type
! How it works
! Circumvention
! DPI
|-
|DNS-based blocking
|DNS-based blocking requires ISPs (who usually run their own DNS servers, being default for their clients) to de-list certain domains (so that they are not resolvable when using these DNS servers). This means that the costs of implementing it are small.
|Custom DNS server settings can be used to easily circumvent DNS-based blocking. It does not require almost any technical prowess and can be used by anybody. There is a number of publicly available DNS servers that can be used for this purpose. There is no way to easily block the use of this method without deploying censorship methods other than pure DNS-blocking.
|no
|-
|IP address-based blocking
|IP-based blocking requires the ISPs to either block certain IP addresses
<nowiki> </nowiki>internally or route all the outgoing connections via a central, government-mandated censoring entity. It is only superficially harder to circumvent, while retaining most if not all problems of DNS-based blocking.
|
|no
|-
|URL-based blocking
|Because this method blocks only certain, URL-identified content, not whole websites or servers (as do DNS-based and IP-based methods), it has much lower potential for accidental over-blocking. This also entails it has a higher potential for under-blocking, as the content can be available on the same server under many different URLs, and changing just a small part of the name defeats the filter.
|
|yes
|-
|Dynamic blocking
|This method uses deep packet inspection to read the contents of data being transmitted, and compares it with a list of keywords, or with image samples or video (depending on the content type).
|
|yes
|-
|Hash-based blocking
|Hash-based blocking uses deep packet inspection to inspect the contents of data-streams, hashes them with cryptographic hash functions and compares to a known database of hashes to be blocked.
|
|yes
|-
|Hybrid solutions
|In order to compromise between high-resource, low-over-blocking hash-based blocking and low-resource, high-over-blocking IP- or DNS-based solutions, a hybrid solution might be proposed. Usually it means that there is a list of IP addresses or domain names for which the hash-based blocking is enabled, hence only operating for a small part of content. This method does employ deep packet inspection.
|
|yes
|}

== Server attacks ==

{| class="wikitable"
|-
! !! !! Significance !! Prevention/Protection
|-
| XXS || Cross-site scripting is a vulnerability in php web applications. || Attackers can exploit it to steal user information. || Configure web server and write better secured PHP scripts (validating all user input)
|-
| SQL injection || SQL injection is a vulnerability in the database layer of a php application. || Any SQL statements can be executed by the application. || Configure the web server and write secure code (validating and escaping all user input)
|-
| File uploads || Allowing users to place files on the server. || Can be used to delete files, databases, get user details and much more || Validate user input and only allow image file type such as png or gif.
|-
| Including local and remote files || An attacker can open files from remote server and execute any PHP code. || Can be used to upload files, delete files and install backdoors. || Configure php to disable remote file execution.
|-
| eval() || Evaluate a string as PHP code. || Can be used by an attacker to hide their code and tools on the server itself. || Configure php to disable eval().
|-
| CSRF || A Cross-Site Request Forgery forces an authenticated end user to execute unwanted actions on a web application. || Can compromise end user data and operation in case of normal user. If the targeted end user is the administrator account, this can compromise the entire web application. || See the Cross-Site Request Forgery (CSRF) Prevention Cheat Sheet https://www.owasp.org/index.php/Cross-Site_Request_Forgery_%28CSRF%29_Prevention_Cheat_Sheet
|}

== Related ==
* [[Timeline masters of the internet]]
* [[Threat modeling the quick and dirty way]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

[[Category:How_To]]

Digital threats, detection, protection and (counter) moves

2015-10-08T09:04:55Z

Lilith2: /* Dragnet surveillance */

This page lists theoretical defenses and detection methods for selected groups of leaked surveillance programs and services. This is just a thought experiment covering (theoretical) defenses against these attacks and not intended to spread fear, uncertainty or doubt about surveillance states.

Due to the age and limited scope of the leaked documents and what we are up against <ref>Cryptome: Communications privacy folly http://cryptome.org/2012/06/comms-folly.htm
</ref>, the defenses mentioned in these tables should not be relied upon for protection and I make no guarantees to their accuracy. Do your own research and make informed decisions, knowing it is impossible to be completely safe. Things move incredibly fast in this arena and I will update these tables when more is found and/or theorised.

Ideally we discuss these exploits and counter moves without the NSA being privy to those discussion, but that would make spreading possible counter moves too slow for the surveillance development cycle (security arms race) and effectively exclude non-techie activists from being able to defend themselves to some extent, so the next best solution is to discuss everything in the open, rather than not discussing them at all. May we not have to rely on a new class of technocracy.

== Social engineering ==
These tables were started off from http://www.itsecurity.be/social-engineering-what-is-it-and-how-to-defend-yourself and then added to. I started off from a to gendersec external source to increase chances of finding that which we may be overlooking.

=== Internal threats ===

{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Power of scarcity
|| Scarcity is when people are told something they need or want has limited availability and to get it they must comply with a certain attitude or action. Many times the desired behavior is not even spoken, but the way it is conveyed is by showing people who are acting "properly" getting rewards.

Mental shortcuts are often beneficial because they enable us to more efficiently navigate the complexity of life. But the power of scarcity can also mislead us. If some people get something not everyone else has (energy, food, water), it makes them feel "special", "unique", "high status", and gives a "position" to defend. And fear of loss of that is an attachment.

An example is a government, in this case South Africa, taking something necessary to life, and making it "scarce" and available only to supporters — a malicious, but very effective, manipulation tactic: http://www.social-engineer.org/wiki/archives/Governments/Governments-FoodElectionWeapon.html See [[timeline merchants of death]] for more examples.

|| Scarcity complicates Gut/Head decision processes because, often, there is only a brief window of opportunity in which we can select something that is in scarce supply. When we can see the supply diminishing before our eyes, we feel particularly compelled to take action.

Each context and scarcity is different, but mind not to rule out the head too quickly in [[Scenario planning#Gut.2C head.2C and decision making|"Gut versus Head"]] decisions.
|-
| Fear of authority || Many people are apprehensive in the presence of someone they perceive to be an authority figure, it is not that person they are apprehensive about but most likely the position and power of the person that intimidates them.
<nowiki> </nowiki>The attackers take on roles of authority figures such as law enforcement officers or high-ranking company officials to extract sensitive information from the victims.
|| Protect yourself with learning about [[Self-dox#Elicitation|elicitation]] (in roleplay settings) and then, time for some controlled folly!
|-
| Desire to be helpful
|| People in their desire to be helpful and to solve other peoples queries, give out a lot of information that otherwise should not be. || Do not disclose information to an outsider as it could give an attacker a chance to get unauthorised access.
|-
| Laziness || All of us have come across some job that requires us to do only a specified set of activities and nothing more. This causes boredom to the person who performs the same task repeatedly on a daily basis and over time the "bored" learn "shortcuts" to do the tasks using minimal effort while meeting targets. This leads to a laid back attitude and becoming susceptible to attackers who target such individuals knowing they can get the required information with much ease. || Find ways of keeping yourself alert. Go do something else. You are becoming a sitting duck.
|-
| Ego || Many a times, the attacker makes a person more emotionally sure of himself/herself, thus removing the logical awareness of the security breach that is occurring. The result is that the person being hacked senses no harm in providing whatever it is that the attacker is requesting. The reason that such an attack succeeds is that the attacker is a receptive audience for victims to display how much knowledge they have. || The ego isn’t going to go away and whether it makes you vulnerable depends on the context. A bit of swagger based on achievements can be quite okay now and then! In some other contexts allowing your ego to call the shots can be a problem.

The simplest detector is asking yourself: Do you feel superior (or inferior) to others? If yes, then your ego is in control. Depending on context choose "not now, not here, not with this" or "do some counter-intelligence" or "Arrrrr, and a bottle of rum!"
|-
| Excitement of victory & fear of loss || Yes! An opportunity! Easy money!!! Out of excitement security is switched off, links are clicked and documents downloaded that turn out to be corrupted (and contained malware that allows the email sender to gain remote access to the machine, or someone's funds, or data, or ...).

Or, ''"You have won 1 Million Dollars and to claim the winning amount, deposit $75,000 in Account number: XXXXXX in 10 days from receiving this e-mail, failing to which the winning amount would be declared unclaimed and there would be a nee lucky-draw to decide the next winner"''

An example is the 419 scams many people fell for http://www.social-engineer.org/wiki/archives/ConMen/ConMen-Scam-NigerianFee.html
|| It is another ego thing isn't it?
|-
| Insufficient knowledge || People with insufficient knowledge can easily be exploited by creating a sense of urgency and not allowing much time to think and understanding the fact that they are under attack. || Gather (and spread) knowledge in a relaxed way. Stay with your own timing. Do the best you can.
|}

=== External threats ===
{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Shoulder surfing || Shoulder surfing is a security attack where-in, the attacker uses observational techniques, such as looking over someone's shoulder, to get information while they are performing some action that involves explicit usage of sensitive, visible information. This can be performed at a close range as well as at a long range using binoculars or other vision-enhancing devices. || Don't work on anything important in public.
|-
| Dumpster diving || Going through the trash can yield one of the most lucrative payoffs for information gathering. People often throw away invoices, notices, letters, CDs, computers, USB keys, and a plethora of other devices and reports that can truly give amazing amounts of information. The attacker can use these items to get a huge amount of information about people, organisations they participate in, and network structure.

Some people shred documents but some types of shredding can be thwarted with a little time and patience and some tape.
|| Mind what you throw away where and how.

Using a shredder that shreds both directions into a fine minced mess makes taping documents back together nearly impossible. Or incinerate. Or both.
|-
| Role playing || Role playing is one of the key weapons for a social engineer. It involves persuading or gathering information through the use of online chat sessions, emails, phones or any other method that you use to interact online with others, and in which the social engineer plays the role of a helpdesk or technician, helplessness, or whatever may work in that context to get targets to divulge confidential information. || Don't "burn" the social engineer. Apply controlled folly instead, and then let's see who is the cat here and who the mouse: Feed them incorrect information. Remember what the intent is. Intent is the key element in intelligence and counter-intelligence operations.
|-
| Trojan horses || It is one of the most predominant methods currently used by online criminals and intelligence agencies that involve tricking victims into downloading a malicious file to their machine, which on execution creates a backdoor in the machine that can be used by the attacker any time in the future and thus having complete access of the victim's machine. Compromising a browser is relatively easy and it is cross-platform, hence an often chosen attack vector. <ref>Meet The Hackers Who Sell Spies The Tools To Crack Your PC (And Get Paid Six-Figure Fees) http://www.forbes.com/sites/andygreenberg/2012/03/21/meet-the-hackers-who-sell-spies-the-tools-to-crack-your-pc-and-get-paid-six-figure-fees/</ref>.
<nowiki> </nowiki>Among all of the NSA hacking operations exposed by whistleblower Edward Snowden over the last two years, one in particular has stood out for its sophistication and stealthiness. Known as Quantum Insert, the man-on-the-side hacking technique has been used to great effect since 2005 by the NSA and its partner spy agency, Britain’s GCHQ, to hack into high-value, hard-to-reach systems and implant malware.

|| Detection focuses on identifying anomalies in the data packets that get sent to a victim’s browser client when the browser attempts to access web pages http://blog.fox-it.com/2015/04/20/deep-dive-into-quantum-insert/

For more such tools run by intelligence agency for targeted surveillance see [[#Infected_firmware|infected firmware]] below.
|-
| Phishing || Phishing is the act of creating and using websites and e-mails designed to look like those of well-known legitimate businesses, financial institutions and government agencies to deceive receivers into disclosing their personal information.
|| These are semantic attacks.
* Be suspicious of any email with urgent requests for personal (financial) information.
* Set your email client to receive email in plaintext, never html format.
* Check any url in the email carefully. It can be just one letter different or missing.
* If phishy, report it.
|-
| Information on websites, online forums and social media|| Huge amounts of information regarding organisational structures (formal and informal), email adresses, phone numbers, are all available publicly on websites, forums and social media. This information can be used by the attacker to refine his approach and create a plan on whom to target and methods most likely to succeed. || Several moves are possible (not necessarily exclusive):
* Delete your social media presence
* Use different identities for your various social contexts
|-
| Reverse social engineering || A reverse social engineering attack is an attack in which an attacker convinces a target that he or she has a problem or might have a certain problem in the future and that the attacker, is ready to help solve the problem.

Reverse social engineering involves three parts:
* Sabotage: After the attacker gains a simple access to the system, he corrupts the system or gives it an appearance of being corrupted. When the user sees the system in the corrupted state, he starts looking for help so as to solve the problem.
* Marketing: In order to make sure that the user approaches the attacker with the problem, the attacker advertises himself as the only person who can solve the problem.
* Support: In this step, the attacker gains the trust of the target and obtains access to sensitive information.
|| If your alarm didn't go off in the sabotage step, when your system appearing corrupt coincided with this remarkable good fortune of someone being right there at the right time ready to help, then let your alarm go off at the [[Propaganda#False_Dilemma|false dilemma]] you are presented with in the marketing step.
|}

== Targeted surveillance ==

=== Hardware implants ===
Unless you are targeted by a government intelligence agency, there seems to be no need to worry about installing commodity hardware from reputable vendors.

Detection: Looking for physical devices will always be the easiest solution to detect them. The links to NSA exploits of the day were added for the comments. :)

{| class="wikitable"
|-
! !! More information !! Possible types of attacks !! Detection
|-
| Godsurge || Godsurge is a physical device plugged-in to the Joint Test Action Group or JTAG headers on a system's motherboard. JTAG headers can be found on many systems and are notoriously common in embedded devices. These headers are used during the development process for debugging purposes: they give you a direct interface with the CPU and are extremely helpful. They are commonly left on the production boards, so finding them on a device is normal and not a security concern. However, if there is a chip or board wired in to a device's JTAG headers that you did not wire in yourself, then something fishy may be going on. || The JTAG debugging interface can be used to reflash the BIOS from scratch, for example loading a compromised version of the software.

|| Look for the JTAG connecter on the motherboard. The location of the JTAG headers may differ, but they tend to be near the CPU and may have exposed pins (or not). See the Wikipedia page on JTAG for more information and to see what they look like <ref>Wikipedia JTAG http://en.wikipedia.org/wiki/Joint_Test_Action_Group</ref>
|-
| Ginsu and Bulldozer
|| Ginsu provides software application persistence on target systems with the PCI bus hardware implant, Bulldozer
http://leaksource.files.wordpress.com/2013/12/nsa-ant-ginsu.jpg

https://www.schneier.com/blog/archives/2014/01/ginsu_nsa_explo.html
|| Exploit persistence from a PCI card ROM
|| Open up the computer's case and look for a PCI card that does not belong. For example, if you find a PCI card that appears to serve no purpose (e.g.. not your network card or it has no external ports), then perhaps try removing it and see how things work. If a few black SUVs roll up to your house after unplugging the PCI card, it's probably not because your domicile is the set for a rap superstar's new music video. Or maybe it is.
|-
| Cottonmouth I, Cottonmouth II and Cottonmouth III
|| These bugs are embedded somewhere along the USB bus and function as an air gap bridge to assist in exfiltration of data as well as allow persistent compromise. It can be embedded directly in the USB headers in an existing USB peripheral (USB hub or keyboard). These devices allow for exfiltration of data over unknown radio frequencies to listening devices in the area, even on a system that is not connected to the internet.
http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-i.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-i_n.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-ii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-ii.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonomouth-iii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-iii.html
|| USB host attack
|| Open up the keyboard or USB hub and identify a board that serves no purpose to the device. The malicious USB device would also likely show up on your computer's list of USB devices, so just checking there would be a good place to start.
|}

=== Radio frequency exfiltration ===
The devices used can vary, but they all employ a similar method of communication via an unknown radio/radar protocol. The frequencies are not known and based on the information leaked the devices are passive (only power on when data is being extracted) making identification of (type of) signal (and intelligence) hard.

Detection: First check for evidence of a device which has been wired in to an existing device such as a keyboard or other peripheral. The SpiderBlog describes the following possibility: You can check if there are RF transmitters in a device by monitoring the spectrum (using an amateur RF listening bug detector) while the device is off (to get a baseline for ambient RF background noise) and then monitoring it again after the suspect device is turned on and transmitting data via radio frequency. The detection device would pick up the signal and alert the user. <ref>Detecting A Surveillance State - Part 2 Radio Frequency Exfiltration https://www.trustwave.com/Resources/SpiderLabs-Blog/Detecting-A-Surveillance-State---Part-2-Radio-Frequency-Exfiltration/
</ref> Further analysis of the signal and its intelligence is still hard, as the exact protocols are (still) unknown.
{| class="wikitable"
|-
! !! More information !! Possible types of attack
|-
| Howlermonkey
|| Short to medium range Radio Transceiver

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/howlermonkey_ns.html
|| HOWLERMONKEY is a COTS-based transceiver deigned to be compatible with CONJECTURE/SPECULATION networks and STRIKEZONE devices running a HOWLERMONKEY personality. PCB layouts are tailored to individual implant space requirements and can vary greatly in form factor.
|-
| Ragemaster
|| Hardware implant in a VGA cable that sends video data over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/03/ragemaster_nsa.html
|| The RAGEMASTER taps the red video line between the video card within the desktop unit and the computer monitor, typically an LCD. When the RAGEMASTER is illuminated by a radar unit, the illuminating signal is modulated with the red video information. This information is re-radiated, where it is picked up at the radar, demodulated, and passed onto the processing unit, such as a LFS-2 and an external monitor, NIGHTWATCH, GOTHAM, or (in the future) VIEWPLATE. The processor recreates the horizontal and vertical sync of the targeted monitor, thus allowing TAO personnel to see what is displayed on the targeted monitor.
|-
| Loudauto
|| Hardware device that sends amplified audio over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/loudauto_nsa_ex.html

|| Room audio is picked up by the microphone and converted into an analog electrical signal. This signal is used to pulse position modulate (PPM) a square wave signal running at a pre-set frequency. This square wave is used to turn a FET (field effect transistor) on and off. When the unit is illuminated with a CW signal from a nearby radar unit, the illuminating signal is amplitude-modulated with the PPM square wave. This signal is re-radiated, where it is picked up by the radar, then processed to recover the room audio. Processing is currently performed by COTS equipment with FM demodulation capability (Rohde & Schwarz FSH-series portable spectrum analyzers, etc.) LOUDAUTO is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|-
| Surleyspawn
|| Hardware implant in a keyboard that emits keystrokes over RF

http://leaksource.files.wordpress.com/2013/12/nsa-ant-surlyspawn.jpg

https://www.schneier.com/blog/archives/2014/02/surlyspawn_nsa.html
|| The board taps into the data line from the keyboard to the processor. The board generates a square wave oscillating at a preset frequency. The data-line signal is used to shift the square wave frequency higher or lower, depending on the level of the data-line signal. The square wave, in essence, becomes frequency shift keyed (FSK). When the unit is illuminated by a CW signal from a nearby radar, the illuminating signal is amplitude-modulated (AM) with this square wave. The signal is re-radiated, where it is received by the radar, demodulated, and the demodulated signal is processed to recover the keystrokes. SURLYSPAWN is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|}

=== Infected firmware ===
Detection: Dump the BIOS to a bin file and compare the hash with a clean BIOS hash. See flashrom for identifying, reading, writing, erasing, and verifying BIOS/ROM/flash chips <ref>Debian packages: flashrom https://packages.debian.org/jessie/flashrom</ref>.

Removal: For each of these infections, where applicable, pulling the chip and replacing it with a new freshly burned BIOS chip or compact flash card would be sufficient. When dealing with built-in firmware it's a bit more difficult than pulling and replacing. You will need to re-flash the device using an operating system that is not at risk of being attacked by the infected firmware. You could boot the device into a low level OS in hopes that the firmware infection isn't able to protect itself. Or, you could wire in a debugging header to the device (such as JTAG , if available) to read or write the firmware on the device to clean things up for good.

{| class="wikitable"
|-
! !! More information !! Possible types of attack || Protection
|-
| Dietybounce
|| Motherboard BIOS Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-deitybounce.jpg?w=1208&h=1562

https://www.schneier.com/blog/archives/2014/01/nsa_exploit_of.html

|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS on a target machine to implant DEITYBOUNCE and its payload (the implant installer). Implantation via interdiction may be accomplished by nontechnical operator through use of a USB thumb drive. Once implanted, DEITYBOUNCE's frequency of execution (dropping the payload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Swap
|| Hard Drive Firmware Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-swap.jpg

https://www.schneier.com/blog/archives/2014/02/swap_nsa_exploi.html
|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS and TWISTEDKILT to write the Host Protected Area on the hard drive on a target machine in order to implant SWAP and its payload (the implant installer). Once implanted, SWAP's frequency of execution (dropping the playload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Headwater, sierramontana, and jetplow
|| Firmware backdoors that target popular networking hardware

https://leaksource.files.wordpress.com/2013/12/nsa-ant-headwater.jpg

https://www.schneier.com/blog/archives/2014/01/headwater_nsa_e.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-sierramontana.jpg

https://www.schneier.com/blog/archives/2014/01/sierramontana_n.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-jetplow.jpg

https://www.schneier.com/blog/archives/2014/01/jetplow_nsa_exp.html
|| HEADWATER PBD implant will be transferred remotely over the Internet to the selected target router by Remote Operations Center (ROC) personnel. After the transfer, the PBD will be installed in the router's boot ROM via an upgrade command. The PBD will then be activated after a system reboot. Once activated, the ROC operators will be able to use DNT's HAMMERMILL Insertion Tool (HIT) to control the PBD as it captures and examines all IP packets passing through the host router.

Currently, the intended DNT Implant to persist is VALIDATOR, which must be run as a user process on the target operating system. The vector of attack is the modification of the target's BIOS. The modification will add the necessary software to the BIOS and modify its software to execute the SIERRAMONTANA implant at the end of its native System Management Mode (SMM) handler.

JETPLOW persists DNT's BANANAGLEE software implant and modifies the Cisco firewall's operating system (OS) at boot time. If BANANAGLEE support is not available for the booting operating system, it can install a Persistent Backdoor (PDB) designed to work with BANANAGLEE'S communications structure, so that full access can be reacquired at a later time.
|| Most routers can be flashed with openWRT http://wiki.openwrt.org/toh/start
|}

== Dragnet surveillance ==

Best protection: Smash your PC to bits, distribute the pieces randomly among a dozen scrap heaps, and [https://undisconnect.miraheze.org/wiki/Rock_bottom move into the woods, deep in the woods ... ]
{| class="wikitable"
|-
! !! More information !! !! Protection
|-
| PRISM || PRISM is part of the overall NSA surveillance effort - a program authorised in the united states under the FISA Amendments Act (FAA), now located in Section 702 of the FISA, that allows the NSA to collect communications of specifically identified foreign targets.

When people in the rest of the world use the net, they are effectively using US-based services, making them a legal target for US intelligence. NSA has no "direct access" to the servers of companies like Microsoft, Facebook and Google - other agencies collect the data. More [[Surveillance#PRISM|here]]
|| Gathers emails, chat--video, voice, photos, stored data, VoIP, file transfers, video conferencing, notifications of target activity--logins, etc, online social networking details, and special requests collected by other programs from nine major Silicon Valley technology companies: Facebook, Google, Yahoo, Microsoft, PalTalk, Skype, YouTube, Apple, and AOL.
||
* Put political pressure on the united states - economical and political dependencies are making that unlikely, see the power of scarcity above.

* Avoid using popular web services.
* FOIA's, court cases on legitimacy
|-
| Upstream collection
|| It appears that through Upstream collection, the NSA gains access to data by forming partnerships with both foreign intelligence agencies and foreign telecommunication companies, by partnering with united states telecommunications companies that then make agreements with international companies for their internet data. More [[Surveillance#Upstream_collection|here]]
|| BLARNEY seems to focus specifically on metadata information about communications traffic and network devices (where and when they were sent).

FAIRVIEW collects just about everything on packet and session levels.
||
* Research which countries and telecommunication companies participate
* Avoid using services associated with these programs
* [[Step 1#Understanding_and_minimising_our_digital_shadows|Understanding and minimising our digital shadows]]
* Court cases on legitimacy
|}

== Censorship ==
Table initially filled with data from How to effectively argue against Internet Censorship ideas http://rys.io/en/94.txt.

Proxy servers, especially anonymous ones, located outside the area where a censorship solution is deployed can be used quite easily to circumvent any blocking method; users can modify their operating system or browser settings, or install browser additions that make using this circumvention method trivial. It is possible to block the proxy servers themselves (via IP-blocking, keyword blocking, etc.), however it is infeasible to block them all, as they are easy to set-up.

Virtual Private Networks (including “poor man’s VPNs” like SSH tunnels) require more technical prowess and usually a (usually commercial) VPN service (or SSH server) outside the area with blocking deployed. Blocking all VPN/SSH traffic is possible, but requires deep packet inspection and is a serious problem for many legitimate businesses using VPNs (and SSH) as their daily tools of trade, to allow their employees access to corporate networks from outside physical premises, via a secured link on the Internet.

TOR, or The Onion Router, is a very effective (if a bit slow) circumvention method. It is quite easy to set-up — users can simply download the TOR Browser Bundle and use it to access the Internet. Due to the way it works it is nigh-impossible to block TOR traffic (as it looks just like vanilla HTTPS traffic), to the point that it is known to allow access to the uncensored Internet to those living in areas with most aggressive Internet censorship policies — namely China, North Korea and Iran. See Tor threats.

None of the censorship solutions is able to block content on darknets — virtual networks accessible anonymously only via specialised software (for instance TOR, I2P, FreeNet), and guaranteeing high resilience to censorship through technical composition of the networks themselves. Because darknets are both practically impossible to block entirely and not allowing for any content blocking within them, they are effectively the ultimate circumvention methods. The downside to using darknets is their lower bandwidth. Deploying Internet censorship pushes the to-be-blocked content into darknets, making it ever-harder for law enforcement gather evidence and researchers gather data on the popularity of a given type of censored content.
{| class="wikitable"

! Blocking type
! How it works
! Circumvention
! DPI
|-
|DNS-based blocking
|DNS-based blocking requires ISPs (who usually run their own DNS servers, being default for their clients) to de-list certain domains (so that they are not resolvable when using these DNS servers). This means that the costs of implementing it are small.
|Custom DNS server settings can be used to easily circumvent DNS-based blocking. It does not require almost any technical prowess and can be used by anybody. There is a number of publicly available DNS servers that can be used for this purpose. There is no way to easily block the use of this method without deploying censorship methods other than pure DNS-blocking.
|no
|-
|IP address-based blocking
|IP-based blocking requires the ISPs to either block certain IP addresses
<nowiki> </nowiki>internally or route all the outgoing connections via a central, government-mandated censoring entity. It is only superficially harder to circumvent, while retaining most if not all problems of DNS-based blocking.
|
|no
|-
|URL-based blocking
|Because this method blocks only certain, URL-identified content, not whole websites or servers (as do DNS-based and IP-based methods), it has much lower potential for accidental over-blocking. This also entails it has a higher potential for under-blocking, as the content can be available on the same server under many different URLs, and changing just a small part of the name defeats the filter.
|
|yes
|-
|Dynamic blocking
|This method uses deep packet inspection to read the contents of data being transmitted, and compares it with a list of keywords, or with image samples or video (depending on the content type).
|
|yes
|-
|Hash-based blocking
|Hash-based blocking uses deep packet inspection to inspect the contents of data-streams, hashes them with cryptographic hash functions and compares to a known database of hashes to be blocked.
|
|yes
|-
|Hybrid solutions
|In order to compromise between high-resource, low-over-blocking hash-based blocking and low-resource, high-over-blocking IP- or DNS-based solutions, a hybrid solution might be proposed. Usually it means that there is a list of IP addresses or domain names for which the hash-based blocking is enabled, hence only operating for a small part of content. This method does employ deep packet inspection.
|
|yes
|}

== Server attacks ==

{| class="wikitable"
|-
! !! !! Significance !! Prevention/Protection
|-
| XXS || Cross-site scripting is a vulnerability in php web applications. || Attackers can exploit it to steal user information. || Configure web server and write better secured PHP scripts (validating all user input)
|-
| SQL injection || SQL injection is a vulnerability in the database layer of a php application. || Any SQL statements can be executed by the application. || Configure the web server and write secure code (validating and escaping all user input)
|-
| File uploads || Allowing users to place files on the server. || Can be used to delete files, databases, get user details and much more || Validate user input and only allow image file type such as png or gif.
|-
| Including local and remote files || An attacker can open files from remote server and execute any PHP code. || Can be used to upload files, delete files and install backdoors. || Configure php to disable remote file execution.
|-
| eval() || Evaluate a string as PHP code. || Can be used by an attacker to hide their code and tools on the server itself. || Configure php to disable eval().
|-
| CSRF || A Cross-Site Request Forgery forces an authenticated end user to execute unwanted actions on a web application. || Can compromise end user data and operation in case of normal user. If the targeted end user is the administrator account, this can compromise the entire web application. || See the Cross-Site Request Forgery (CSRF) Prevention Cheat Sheet https://www.owasp.org/index.php/Cross-Site_Request_Forgery_%28CSRF%29_Prevention_Cheat_Sheet
|}

== Related ==
* [[Timeline masters of the internet]]
* [[Threat modeling the quick and dirty way]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

[[Category:How_To]]

Digital threats, detection, protection and (counter) moves

2015-09-30T21:31:41Z

Lilith2:

This page lists theoretical defenses and detection methods for selected groups of leaked surveillance programs and services. This is just a thought experiment covering (theoretical) defenses against these attacks and not intended to spread fear, uncertainty or doubt about surveillance states.

Due to the age and limited scope of the leaked documents and what we are up against <ref>Cryptome: Communications privacy folly http://cryptome.org/2012/06/comms-folly.htm
</ref>, the defenses mentioned in these tables should not be relied upon for protection and I make no guarantees to their accuracy. Do your own research and make informed decisions, knowing it is impossible to be completely safe. Things move incredibly fast in this arena and I will update these tables when more is found and/or theorised.

Ideally we discuss these exploits and counter moves without the NSA being privy to those discussion, but that would make spreading possible counter moves too slow for the surveillance development cycle (security arms race) and effectively exclude non-techie activists from being able to defend themselves to some extent, so the next best solution is to discuss everything in the open, rather than not discussing them at all. May we not have to rely on a new class of technocracy.

== Social engineering ==
These tables were started off from http://www.itsecurity.be/social-engineering-what-is-it-and-how-to-defend-yourself and then added to. I started off from a to gendersec external source to increase chances of finding that which we may be overlooking.

=== Internal threats ===

{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Power of scarcity
|| Scarcity is when people are told something they need or want has limited availability and to get it they must comply with a certain attitude or action. Many times the desired behavior is not even spoken, but the way it is conveyed is by showing people who are acting "properly" getting rewards.

Mental shortcuts are often beneficial because they enable us to more efficiently navigate the complexity of life. But the power of scarcity can also mislead us. If some people get something not everyone else has (energy, food, water), it makes them feel "special", "unique", "high status", and gives a "position" to defend. And fear of loss of that is an attachment.

An example is a government, in this case South Africa, taking something necessary to life, and making it "scarce" and available only to supporters — a malicious, but very effective, manipulation tactic: http://www.social-engineer.org/wiki/archives/Governments/Governments-FoodElectionWeapon.html See [[timeline merchants of death]] for more examples.

|| Scarcity complicates Gut/Head decision processes because, often, there is only a brief window of opportunity in which we can select something that is in scarce supply. When we can see the supply diminishing before our eyes, we feel particularly compelled to take action.

Each context and scarcity is different, but mind not to rule out the head too quickly in [[Scenario planning#Gut.2C head.2C and decision making|"Gut versus Head"]] decisions.
|-
| Fear of authority || Many people are apprehensive in the presence of someone they perceive to be an authority figure, it is not that person they are apprehensive about but most likely the position and power of the person that intimidates them.
<nowiki> </nowiki>The attackers take on roles of authority figures such as law enforcement officers or high-ranking company officials to extract sensitive information from the victims.
|| Protect yourself with learning about [[Self-dox#Elicitation|elicitation]] (in roleplay settings) and then, time for some controlled folly!
|-
| Desire to be helpful
|| People in their desire to be helpful and to solve other peoples queries, give out a lot of information that otherwise should not be. || Do not disclose information to an outsider as it could give an attacker a chance to get unauthorised access.
|-
| Laziness || All of us have come across some job that requires us to do only a specified set of activities and nothing more. This causes boredom to the person who performs the same task repeatedly on a daily basis and over time the "bored" learn "shortcuts" to do the tasks using minimal effort while meeting targets. This leads to a laid back attitude and becoming susceptible to attackers who target such individuals knowing they can get the required information with much ease. || Find ways of keeping yourself alert. Go do something else. You are becoming a sitting duck.
|-
| Ego || Many a times, the attacker makes a person more emotionally sure of himself/herself, thus removing the logical awareness of the security breach that is occurring. The result is that the person being hacked senses no harm in providing whatever it is that the attacker is requesting. The reason that such an attack succeeds is that the attacker is a receptive audience for victims to display how much knowledge they have. || The ego isn’t going to go away and whether it makes you vulnerable depends on the context. A bit of swagger based on achievements can be quite okay now and then! In some other contexts allowing your ego to call the shots can be a problem.

The simplest detector is asking yourself: Do you feel superior (or inferior) to others? If yes, then your ego is in control. Depending on context choose "not now, not here, not with this" or "do some counter-intelligence" or "Arrrrr, and a bottle of rum!"
|-
| Excitement of victory & fear of loss || Yes! An opportunity! Easy money!!! Out of excitement security is switched off, links are clicked and documents downloaded that turn out to be corrupted (and contained malware that allows the email sender to gain remote access to the machine, or someone's funds, or data, or ...).

Or, ''"You have won 1 Million Dollars and to claim the winning amount, deposit $75,000 in Account number: XXXXXX in 10 days from receiving this e-mail, failing to which the winning amount would be declared unclaimed and there would be a nee lucky-draw to decide the next winner"''

An example is the 419 scams many people fell for http://www.social-engineer.org/wiki/archives/ConMen/ConMen-Scam-NigerianFee.html
|| It is another ego thing isn't it?
|-
| Insufficient knowledge || People with insufficient knowledge can easily be exploited by creating a sense of urgency and not allowing much time to think and understanding the fact that they are under attack. || Gather (and spread) knowledge in a relaxed way. Stay with your own timing. Do the best you can.
|}

=== External threats ===
{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Shoulder surfing || Shoulder surfing is a security attack where-in, the attacker uses observational techniques, such as looking over someone's shoulder, to get information while they are performing some action that involves explicit usage of sensitive, visible information. This can be performed at a close range as well as at a long range using binoculars or other vision-enhancing devices. || Don't work on anything important in public.
|-
| Dumpster diving || Going through the trash can yield one of the most lucrative payoffs for information gathering. People often throw away invoices, notices, letters, CDs, computers, USB keys, and a plethora of other devices and reports that can truly give amazing amounts of information. The attacker can use these items to get a huge amount of information about people, organisations they participate in, and network structure.

Some people shred documents but some types of shredding can be thwarted with a little time and patience and some tape.
|| Mind what you throw away where and how.

Using a shredder that shreds both directions into a fine minced mess makes taping documents back together nearly impossible. Or incinerate. Or both.
|-
| Role playing || Role playing is one of the key weapons for a social engineer. It involves persuading or gathering information through the use of online chat sessions, emails, phones or any other method that you use to interact online with others, and in which the social engineer plays the role of a helpdesk or technician, helplessness, or whatever may work in that context to get targets to divulge confidential information. || Don't "burn" the social engineer. Apply controlled folly instead, and then let's see who is the cat here and who the mouse: Feed them incorrect information. Remember what the intent is. Intent is the key element in intelligence and counter-intelligence operations.
|-
| Trojan horses || It is one of the most predominant methods currently used by online criminals and intelligence agencies that involve tricking victims into downloading a malicious file to their machine, which on execution creates a backdoor in the machine that can be used by the attacker any time in the future and thus having complete access of the victim's machine. Compromising a browser is relatively easy and it is cross-platform, hence an often chosen attack vector. <ref>Meet The Hackers Who Sell Spies The Tools To Crack Your PC (And Get Paid Six-Figure Fees) http://www.forbes.com/sites/andygreenberg/2012/03/21/meet-the-hackers-who-sell-spies-the-tools-to-crack-your-pc-and-get-paid-six-figure-fees/</ref>.
<nowiki> </nowiki>Among all of the NSA hacking operations exposed by whistleblower Edward Snowden over the last two years, one in particular has stood out for its sophistication and stealthiness. Known as Quantum Insert, the man-on-the-side hacking technique has been used to great effect since 2005 by the NSA and its partner spy agency, Britain’s GCHQ, to hack into high-value, hard-to-reach systems and implant malware.

|| Detection focuses on identifying anomalies in the data packets that get sent to a victim’s browser client when the browser attempts to access web pages http://blog.fox-it.com/2015/04/20/deep-dive-into-quantum-insert/

For more such tools run by intelligence agency for targeted surveillance see [[#Infected_firmware|infected firmware]] below.
|-
| Phishing || Phishing is the act of creating and using websites and e-mails designed to look like those of well-known legitimate businesses, financial institutions and government agencies to deceive receivers into disclosing their personal information.
|| These are semantic attacks.
* Be suspicious of any email with urgent requests for personal (financial) information.
* Set your email client to receive email in plaintext, never html format.
* Check any url in the email carefully. It can be just one letter different or missing.
* If phishy, report it.
|-
| Information on websites, online forums and social media|| Huge amounts of information regarding organisational structures (formal and informal), email adresses, phone numbers, are all available publicly on websites, forums and social media. This information can be used by the attacker to refine his approach and create a plan on whom to target and methods most likely to succeed. || Several moves are possible (not necessarily exclusive):
* Delete your social media presence
* Use different identities for your various social contexts
|-
| Reverse social engineering || A reverse social engineering attack is an attack in which an attacker convinces a target that he or she has a problem or might have a certain problem in the future and that the attacker, is ready to help solve the problem.

Reverse social engineering involves three parts:
* Sabotage: After the attacker gains a simple access to the system, he corrupts the system or gives it an appearance of being corrupted. When the user sees the system in the corrupted state, he starts looking for help so as to solve the problem.
* Marketing: In order to make sure that the user approaches the attacker with the problem, the attacker advertises himself as the only person who can solve the problem.
* Support: In this step, the attacker gains the trust of the target and obtains access to sensitive information.
|| If your alarm didn't go off in the sabotage step, when your system appearing corrupt coincided with this remarkable good fortune of someone being right there at the right time ready to help, then let your alarm go off at the [[Propaganda#False_Dilemma|false dilemma]] you are presented with in the marketing step.
|}

== Targeted surveillance ==

=== Hardware implants ===
Unless you are targeted by a government intelligence agency, there seems to be no need to worry about installing commodity hardware from reputable vendors.

Detection: Looking for physical devices will always be the easiest solution to detect them. The links to NSA exploits of the day were added for the comments. :)

{| class="wikitable"
|-
! !! More information !! Possible types of attacks !! Detection
|-
| Godsurge || Godsurge is a physical device plugged-in to the Joint Test Action Group or JTAG headers on a system's motherboard. JTAG headers can be found on many systems and are notoriously common in embedded devices. These headers are used during the development process for debugging purposes: they give you a direct interface with the CPU and are extremely helpful. They are commonly left on the production boards, so finding them on a device is normal and not a security concern. However, if there is a chip or board wired in to a device's JTAG headers that you did not wire in yourself, then something fishy may be going on. || The JTAG debugging interface can be used to reflash the BIOS from scratch, for example loading a compromised version of the software.

|| Look for the JTAG connecter on the motherboard. The location of the JTAG headers may differ, but they tend to be near the CPU and may have exposed pins (or not). See the Wikipedia page on JTAG for more information and to see what they look like <ref>Wikipedia JTAG http://en.wikipedia.org/wiki/Joint_Test_Action_Group</ref>
|-
| Ginsu and Bulldozer
|| Ginsu provides software application persistence on target systems with the PCI bus hardware implant, Bulldozer
http://leaksource.files.wordpress.com/2013/12/nsa-ant-ginsu.jpg

https://www.schneier.com/blog/archives/2014/01/ginsu_nsa_explo.html
|| Exploit persistence from a PCI card ROM
|| Open up the computer's case and look for a PCI card that does not belong. For example, if you find a PCI card that appears to serve no purpose (e.g.. not your network card or it has no external ports), then perhaps try removing it and see how things work. If a few black SUVs roll up to your house after unplugging the PCI card, it's probably not because your domicile is the set for a rap superstar's new music video. Or maybe it is.
|-
| Cottonmouth I, Cottonmouth II and Cottonmouth III
|| These bugs are embedded somewhere along the USB bus and function as an air gap bridge to assist in exfiltration of data as well as allow persistent compromise. It can be embedded directly in the USB headers in an existing USB peripheral (USB hub or keyboard). These devices allow for exfiltration of data over unknown radio frequencies to listening devices in the area, even on a system that is not connected to the internet.
http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-i.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-i_n.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-ii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-ii.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonomouth-iii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-iii.html
|| USB host attack
|| Open up the keyboard or USB hub and identify a board that serves no purpose to the device. The malicious USB device would also likely show up on your computer's list of USB devices, so just checking there would be a good place to start.
|}

=== Radio frequency exfiltration ===
The devices used can vary, but they all employ a similar method of communication via an unknown radio/radar protocol. The frequencies are not known and based on the information leaked the devices are passive (only power on when data is being extracted) making identification of (type of) signal (and intelligence) hard.

Detection: First check for evidence of a device which has been wired in to an existing device such as a keyboard or other peripheral. The SpiderBlog describes the following possibility: You can check if there are RF transmitters in a device by monitoring the spectrum (using an amateur RF listening bug detector) while the device is off (to get a baseline for ambient RF background noise) and then monitoring it again after the suspect device is turned on and transmitting data via radio frequency. The detection device would pick up the signal and alert the user. <ref>Detecting A Surveillance State - Part 2 Radio Frequency Exfiltration https://www.trustwave.com/Resources/SpiderLabs-Blog/Detecting-A-Surveillance-State---Part-2-Radio-Frequency-Exfiltration/
</ref> Further analysis of the signal and its intelligence is still hard, as the exact protocols are (still) unknown.
{| class="wikitable"
|-
! !! More information !! Possible types of attack
|-
| Howlermonkey
|| Short to medium range Radio Transceiver

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/howlermonkey_ns.html
|| HOWLERMONKEY is a COTS-based transceiver deigned to be compatible with CONJECTURE/SPECULATION networks and STRIKEZONE devices running a HOWLERMONKEY personality. PCB layouts are tailored to individual implant space requirements and can vary greatly in form factor.
|-
| Ragemaster
|| Hardware implant in a VGA cable that sends video data over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/03/ragemaster_nsa.html
|| The RAGEMASTER taps the red video line between the video card within the desktop unit and the computer monitor, typically an LCD. When the RAGEMASTER is illuminated by a radar unit, the illuminating signal is modulated with the red video information. This information is re-radiated, where it is picked up at the radar, demodulated, and passed onto the processing unit, such as a LFS-2 and an external monitor, NIGHTWATCH, GOTHAM, or (in the future) VIEWPLATE. The processor recreates the horizontal and vertical sync of the targeted monitor, thus allowing TAO personnel to see what is displayed on the targeted monitor.
|-
| Loudauto
|| Hardware device that sends amplified audio over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/loudauto_nsa_ex.html

|| Room audio is picked up by the microphone and converted into an analog electrical signal. This signal is used to pulse position modulate (PPM) a square wave signal running at a pre-set frequency. This square wave is used to turn a FET (field effect transistor) on and off. When the unit is illuminated with a CW signal from a nearby radar unit, the illuminating signal is amplitude-modulated with the PPM square wave. This signal is re-radiated, where it is picked up by the radar, then processed to recover the room audio. Processing is currently performed by COTS equipment with FM demodulation capability (Rohde & Schwarz FSH-series portable spectrum analyzers, etc.) LOUDAUTO is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|-
| Surleyspawn
|| Hardware implant in a keyboard that emits keystrokes over RF

http://leaksource.files.wordpress.com/2013/12/nsa-ant-surlyspawn.jpg

https://www.schneier.com/blog/archives/2014/02/surlyspawn_nsa.html
|| The board taps into the data line from the keyboard to the processor. The board generates a square wave oscillating at a preset frequency. The data-line signal is used to shift the square wave frequency higher or lower, depending on the level of the data-line signal. The square wave, in essence, becomes frequency shift keyed (FSK). When the unit is illuminated by a CW signal from a nearby radar, the illuminating signal is amplitude-modulated (AM) with this square wave. The signal is re-radiated, where it is received by the radar, demodulated, and the demodulated signal is processed to recover the keystrokes. SURLYSPAWN is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|}

=== Infected firmware ===
Detection: Dump the BIOS to a bin file and compare the hash with a clean BIOS hash. See flashrom for identifying, reading, writing, erasing, and verifying BIOS/ROM/flash chips <ref>Debian packages: flashrom https://packages.debian.org/jessie/flashrom</ref>.

Removal: For each of these infections, where applicable, pulling the chip and replacing it with a new freshly burned BIOS chip or compact flash card would be sufficient. When dealing with built-in firmware it's a bit more difficult than pulling and replacing. You will need to re-flash the device using an operating system that is not at risk of being attacked by the infected firmware. You could boot the device into a low level OS in hopes that the firmware infection isn't able to protect itself. Or, you could wire in a debugging header to the device (such as JTAG , if available) to read or write the firmware on the device to clean things up for good.

{| class="wikitable"
|-
! !! More information !! Possible types of attack || Protection
|-
| Dietybounce
|| Motherboard BIOS Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-deitybounce.jpg?w=1208&h=1562

https://www.schneier.com/blog/archives/2014/01/nsa_exploit_of.html

|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS on a target machine to implant DEITYBOUNCE and its payload (the implant installer). Implantation via interdiction may be accomplished by nontechnical operator through use of a USB thumb drive. Once implanted, DEITYBOUNCE's frequency of execution (dropping the payload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Swap
|| Hard Drive Firmware Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-swap.jpg

https://www.schneier.com/blog/archives/2014/02/swap_nsa_exploi.html
|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS and TWISTEDKILT to write the Host Protected Area on the hard drive on a target machine in order to implant SWAP and its payload (the implant installer). Once implanted, SWAP's frequency of execution (dropping the playload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Headwater, sierramontana, and jetplow
|| Firmware backdoors that target popular networking hardware

https://leaksource.files.wordpress.com/2013/12/nsa-ant-headwater.jpg

https://www.schneier.com/blog/archives/2014/01/headwater_nsa_e.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-sierramontana.jpg

https://www.schneier.com/blog/archives/2014/01/sierramontana_n.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-jetplow.jpg

https://www.schneier.com/blog/archives/2014/01/jetplow_nsa_exp.html
|| HEADWATER PBD implant will be transferred remotely over the Internet to the selected target router by Remote Operations Center (ROC) personnel. After the transfer, the PBD will be installed in the router's boot ROM via an upgrade command. The PBD will then be activated after a system reboot. Once activated, the ROC operators will be able to use DNT's HAMMERMILL Insertion Tool (HIT) to control the PBD as it captures and examines all IP packets passing through the host router.

Currently, the intended DNT Implant to persist is VALIDATOR, which must be run as a user process on the target operating system. The vector of attack is the modification of the target's BIOS. The modification will add the necessary software to the BIOS and modify its software to execute the SIERRAMONTANA implant at the end of its native System Management Mode (SMM) handler.

JETPLOW persists DNT's BANANAGLEE software implant and modifies the Cisco firewall's operating system (OS) at boot time. If BANANAGLEE support is not available for the booting operating system, it can install a Persistent Backdoor (PDB) designed to work with BANANAGLEE'S communications structure, so that full access can be reacquired at a later time.
|| Most routers can be flashed with openWRT http://wiki.openwrt.org/toh/start
|}

== Dragnet surveillance ==

Best protection: Smash your PC to bits, distribute the pieces randomly among a dozen scrap heaps, and move into the woods, deep in the woods ...
{| class="wikitable"
|-
! !! More information !! !! Protection
|-
| PRISM || PRISM is part of the overall NSA surveillance effort - a program authorised in the united states under the FISA Amendments Act (FAA), now located in Section 702 of the FISA, that allows the NSA to collect communications of specifically identified foreign targets.

When people in the rest of the world use the net, they are effectively using US-based services, making them a legal target for US intelligence. NSA has no "direct access" to the servers of companies like Microsoft, Facebook and Google - other agencies collect the data. More [[Surveillance#PRISM|here]]
|| Gathers emails, chat--video, voice, photos, stored data, VoIP, file transfers, video conferencing, notifications of target activity--logins, etc, online social networking details, and special requests collected by other programs from nine major Silicon Valley technology companies: Facebook, Google, Yahoo, Microsoft, PalTalk, Skype, YouTube, Apple, and AOL.
||
* Put political pressure on the united states - economical and political dependencies are making that unlikely, see the power of scarcity above.

* Avoid using popular web services.
* FOIA's, court cases on legitimacy
|-
| Upstream collection
|| It appears that through Upstream collection, the NSA gains access to data by forming partnerships with both foreign intelligence agencies and foreign telecommunication companies, by partnering with united states telecommunications companies that then make agreements with international companies for their internet data. More [[Surveillance#Upstream_collection|here]]
|| BLARNEY seems to focus specifically on metadata information about communications traffic and network devices (where and when they were sent).

FAIRVIEW collects just about everything on packet and session levels.
||
* Research which countries and telecommunication companies participate
* Avoid using services associated with these programs
* [[Step 1#Understanding_and_minimising_our_digital_shadows|Understanding and minimising our digital shadows]]
* Court cases on legitimacy
|}

== Censorship ==
Table initially filled with data from How to effectively argue against Internet Censorship ideas http://rys.io/en/94.txt.

Proxy servers, especially anonymous ones, located outside the area where a censorship solution is deployed can be used quite easily to circumvent any blocking method; users can modify their operating system or browser settings, or install browser additions that make using this circumvention method trivial. It is possible to block the proxy servers themselves (via IP-blocking, keyword blocking, etc.), however it is infeasible to block them all, as they are easy to set-up.

Virtual Private Networks (including “poor man’s VPNs” like SSH tunnels) require more technical prowess and usually a (usually commercial) VPN service (or SSH server) outside the area with blocking deployed. Blocking all VPN/SSH traffic is possible, but requires deep packet inspection and is a serious problem for many legitimate businesses using VPNs (and SSH) as their daily tools of trade, to allow their employees access to corporate networks from outside physical premises, via a secured link on the Internet.

TOR, or The Onion Router, is a very effective (if a bit slow) circumvention method. It is quite easy to set-up — users can simply download the TOR Browser Bundle and use it to access the Internet. Due to the way it works it is nigh-impossible to block TOR traffic (as it looks just like vanilla HTTPS traffic), to the point that it is known to allow access to the uncensored Internet to those living in areas with most aggressive Internet censorship policies — namely China, North Korea and Iran. See Tor threats.

None of the censorship solutions is able to block content on darknets — virtual networks accessible anonymously only via specialised software (for instance TOR, I2P, FreeNet), and guaranteeing high resilience to censorship through technical composition of the networks themselves. Because darknets are both practically impossible to block entirely and not allowing for any content blocking within them, they are effectively the ultimate circumvention methods. The downside to using darknets is their lower bandwidth. Deploying Internet censorship pushes the to-be-blocked content into darknets, making it ever-harder for law enforcement gather evidence and researchers gather data on the popularity of a given type of censored content.
{| class="wikitable"

! Blocking type
! How it works
! Circumvention
! DPI
|-
|DNS-based blocking
|DNS-based blocking requires ISPs (who usually run their own DNS servers, being default for their clients) to de-list certain domains (so that they are not resolvable when using these DNS servers). This means that the costs of implementing it are small.
|Custom DNS server settings can be used to easily circumvent DNS-based blocking. It does not require almost any technical prowess and can be used by anybody. There is a number of publicly available DNS servers that can be used for this purpose. There is no way to easily block the use of this method without deploying censorship methods other than pure DNS-blocking.
|no
|-
|IP address-based blocking
|IP-based blocking requires the ISPs to either block certain IP addresses
<nowiki> </nowiki>internally or route all the outgoing connections via a central, government-mandated censoring entity. It is only superficially harder to circumvent, while retaining most if not all problems of DNS-based blocking.
|
|no
|-
|URL-based blocking
|Because this method blocks only certain, URL-identified content, not whole websites or servers (as do DNS-based and IP-based methods), it has much lower potential for accidental over-blocking. This also entails it has a higher potential for under-blocking, as the content can be available on the same server under many different URLs, and changing just a small part of the name defeats the filter.
|
|yes
|-
|Dynamic blocking
|This method uses deep packet inspection to read the contents of data being transmitted, and compares it with a list of keywords, or with image samples or video (depending on the content type).
|
|yes
|-
|Hash-based blocking
|Hash-based blocking uses deep packet inspection to inspect the contents of data-streams, hashes them with cryptographic hash functions and compares to a known database of hashes to be blocked.
|
|yes
|-
|Hybrid solutions
|In order to compromise between high-resource, low-over-blocking hash-based blocking and low-resource, high-over-blocking IP- or DNS-based solutions, a hybrid solution might be proposed. Usually it means that there is a list of IP addresses or domain names for which the hash-based blocking is enabled, hence only operating for a small part of content. This method does employ deep packet inspection.
|
|yes
|}

== Server attacks ==

{| class="wikitable"
|-
! !! !! Significance !! Prevention/Protection
|-
| XXS || Cross-site scripting is a vulnerability in php web applications. || Attackers can exploit it to steal user information. || Configure web server and write better secured PHP scripts (validating all user input)
|-
| SQL injection || SQL injection is a vulnerability in the database layer of a php application. || Any SQL statements can be executed by the application. || Configure the web server and write secure code (validating and escaping all user input)
|-
| File uploads || Allowing users to place files on the server. || Can be used to delete files, databases, get user details and much more || Validate user input and only allow image file type such as png or gif.
|-
| Including local and remote files || An attacker can open files from remote server and execute any PHP code. || Can be used to upload files, delete files and install backdoors. || Configure php to disable remote file execution.
|-
| eval() || Evaluate a string as PHP code. || Can be used by an attacker to hide their code and tools on the server itself. || Configure php to disable eval().
|-
| CSRF || A Cross-Site Request Forgery forces an authenticated end user to execute unwanted actions on a web application. || Can compromise end user data and operation in case of normal user. If the targeted end user is the administrator account, this can compromise the entire web application. || See the Cross-Site Request Forgery (CSRF) Prevention Cheat Sheet https://www.owasp.org/index.php/Cross-Site_Request_Forgery_%28CSRF%29_Prevention_Cheat_Sheet
|}

== Related ==
* [[Timeline masters of the internet]]
* [[Threat modeling the quick and dirty way]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

[[Category:How_To]]

Digital threats, detection, protection and (counter) moves

2015-09-30T21:29:30Z

Lilith2: /* Server attacks */

This page lists theoretical defenses and detection methods for selected groups of leaked surveillance programs and services. This is just a thought experiment covering (theoretical) defenses against these attacks and not intended to spread fear, uncertainty or doubt about surveillance states.

Due to the age and limited scope of the leaked documents and what we are up against <ref>Cryptome: Communications privacy folly http://cryptome.org/2012/06/comms-folly.htm
</ref>, the defenses mentioned in these tables should not be relied upon for protection and I make no guarantees to their accuracy. Do your own research and make informed decisions, knowing it is impossible to be completely safe. Things move incredibly fast in this arena and I will update these tables when more is found and/or theorised.

Ideally we discuss these exploits and counter moves without the NSA being privy to those discussion, but that would make spreading possible counter moves too slow for the surveillance development cycle (security arms race) and effectively exclude non-techie activists from being able to defend themselves to some extent, so the next best solution is to discuss everything in the open, rather than not discussing them at all. May we not have to rely on a new class of technocracy.

== Social engineering ==
These tables were started off from http://www.itsecurity.be/social-engineering-what-is-it-and-how-to-defend-yourself and then added to. I started off from a to gendersec external source to increase chances of finding that which we may be overlooking.

=== Internal threats ===

{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Power of scarcity
|| Scarcity is when people are told something they need or want has limited availability and to get it they must comply with a certain attitude or action. Many times the desired behavior is not even spoken, but the way it is conveyed is by showing people who are acting "properly" getting rewards.

Mental shortcuts are often beneficial because they enable us to more efficiently navigate the complexity of life. But the power of scarcity can also mislead us. If some people get something not everyone else has (energy, food, water), it makes them feel "special", "unique", "high status", and gives a "position" to defend. And fear of loss of that is an attachment.

An example is a government, in this case South Africa, taking something necessary to life, and making it "scarce" and available only to supporters — a malicious, but very effective, manipulation tactic: http://www.social-engineer.org/wiki/archives/Governments/Governments-FoodElectionWeapon.html See [[timeline merchants of death]] for more examples.

|| Scarcity complicates Gut/Head decision processes because, often, there is only a brief window of opportunity in which we can select something that is in scarce supply. When we can see the supply diminishing before our eyes, we feel particularly compelled to take action.

Each context and scarcity is different, but mind not to rule out the head too quickly in [[Scenario planning#Gut.2C head.2C and decision making|"Gut versus Head"]] decisions.
|-
| Fear of authority || Many people are apprehensive in the presence of someone they perceive to be an authority figure, it is not that person they are apprehensive about but most likely the position and power of the person that intimidates them.
<nowiki> </nowiki>The attackers take on roles of authority figures such as law enforcement officers or high-ranking company officials to extract sensitive information from the victims.
|| Protect yourself with learning about [[Self-dox#Elicitation|elicitation]] (in roleplay settings) and then, time for some controlled folly!
|-
| Desire to be helpful
|| People in their desire to be helpful and to solve other peoples queries, give out a lot of information that otherwise should not be. || Do not disclose information to an outsider as it could give an attacker a chance to get unauthorised access.
|-
| Laziness || All of us have come across some job that requires us to do only a specified set of activities and nothing more. This causes boredom to the person who performs the same task repeatedly on a daily basis and over time the "bored" learn "shortcuts" to do the tasks using minimal effort while meeting targets. This leads to a laid back attitude and becoming susceptible to attackers who target such individuals knowing they can get the required information with much ease. || Find ways of keeping yourself alert. Go do something else. You are becoming a sitting duck.
|-
| Ego || Many a times, the attacker makes a person more emotionally sure of himself/herself, thus removing the logical awareness of the security breach that is occurring. The result is that the person being hacked senses no harm in providing whatever it is that the attacker is requesting. The reason that such an attack succeeds is that the attacker is a receptive audience for victims to display how much knowledge they have. || The ego isn’t going to go away and whether it makes you vulnerable depends on the context. A bit of swagger based on achievements can be quite okay now and then! In some other contexts allowing your ego to call the shots can be a problem.

The simplest detector is asking yourself: Do you feel superior (or inferior) to others? If yes, then your ego is in control. Depending on context choose "not now, not here, not with this" or "do some counter-intelligence" or "Arrrrr, and a bottle of rum!"
|-
| Excitement of victory & fear of loss || Yes! An opportunity! Easy money!!! Out of excitement security is switched off, links are clicked and documents downloaded that turn out to be corrupted (and contained malware that allows the email sender to gain remote access to the machine, or someone's funds, or data, or ...).

Or, ''"You have won 1 Million Dollars and to claim the winning amount, deposit $75,000 in Account number: XXXXXX in 10 days from receiving this e-mail, failing to which the winning amount would be declared unclaimed and there would be a nee lucky-draw to decide the next winner"''

An example is the 419 scams many people fell for http://www.social-engineer.org/wiki/archives/ConMen/ConMen-Scam-NigerianFee.html
|| It is another ego thing isn't it?
|-
| Insufficient knowledge || People with insufficient knowledge can easily be exploited by creating a sense of urgency and not allowing much time to think and understanding the fact that they are under attack. || Gather (and spread) knowledge in a relaxed way. Stay with your own timing. Do the best you can.
|}

=== External threats ===
{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Shoulder surfing || Shoulder surfing is a security attack where-in, the attacker uses observational techniques, such as looking over someone's shoulder, to get information while they are performing some action that involves explicit usage of sensitive, visible information. This can be performed at a close range as well as at a long range using binoculars or other vision-enhancing devices. || Don't work on anything important in public.
|-
| Dumpster diving || Going through the trash can yield one of the most lucrative payoffs for information gathering. People often throw away invoices, notices, letters, CDs, computers, USB keys, and a plethora of other devices and reports that can truly give amazing amounts of information. The attacker can use these items to get a huge amount of information about people, organisations they participate in, and network structure.

Some people shred documents but some types of shredding can be thwarted with a little time and patience and some tape.
|| Mind what you throw away where and how.

Using a shredder that shreds both directions into a fine minced mess makes taping documents back together nearly impossible. Or incinerate. Or both.
|-
| Role playing || Role playing is one of the key weapons for a social engineer. It involves persuading or gathering information through the use of online chat sessions, emails, phones or any other method that you use to interact online with others, and in which the social engineer plays the role of a helpdesk or technician, helplessness, or whatever may work in that context to get targets to divulge confidential information. || Don't "burn" the social engineer. Apply controlled folly instead, and then let's see who is the cat here and who the mouse: Feed them incorrect information. Remember what the intent is. Intent is the key element in intelligence and counter-intelligence operations.
|-
| Trojan horses || It is one of the most predominant methods currently used by online criminals and intelligence agencies that involve tricking victims into downloading a malicious file to their machine, which on execution creates a backdoor in the machine that can be used by the attacker any time in the future and thus having complete access of the victim's machine. Compromising a browser is relatively easy and it is cross-platform, hence an often chosen attack vector. <ref>Meet The Hackers Who Sell Spies The Tools To Crack Your PC (And Get Paid Six-Figure Fees) http://www.forbes.com/sites/andygreenberg/2012/03/21/meet-the-hackers-who-sell-spies-the-tools-to-crack-your-pc-and-get-paid-six-figure-fees/</ref>.
<nowiki> </nowiki>Among all of the NSA hacking operations exposed by whistleblower Edward Snowden over the last two years, one in particular has stood out for its sophistication and stealthiness. Known as Quantum Insert, the man-on-the-side hacking technique has been used to great effect since 2005 by the NSA and its partner spy agency, Britain’s GCHQ, to hack into high-value, hard-to-reach systems and implant malware.

|| Detection focuses on identifying anomalies in the data packets that get sent to a victim’s browser client when the browser attempts to access web pages http://blog.fox-it.com/2015/04/20/deep-dive-into-quantum-insert/

For more such tools run by intelligence agency for targeted surveillance see [[#Infected_firmware|infected firmware]] below.
|-
| Phishing || Phishing is the act of creating and using websites and e-mails designed to look like those of well-known legitimate businesses, financial institutions and government agencies to deceive receivers into disclosing their personal information.
|| These are semantic attacks.
* Be suspicious of any email with urgent requests for personal (financial) information.
* Set your email client to receive email in plaintext, never html format.
* Check any url in the email carefully. It can be just one letter different or missing.
* If phishy, report it.
|-
| Information on websites, online forums and social media|| Huge amounts of information regarding organisational structures (formal and informal), email adresses, phone numbers, are all available publicly on websites, forums and social media. This information can be used by the attacker to refine his approach and create a plan on whom to target and methods most likely to succeed. || Several moves are possible (not necessarily exclusive):
* Delete your social media presence
* Use different identities for your various social contexts
|-
| Reverse social engineering || A reverse social engineering attack is an attack in which an attacker convinces a target that he or she has a problem or might have a certain problem in the future and that the attacker, is ready to help solve the problem.

Reverse social engineering involves three parts:
* Sabotage: After the attacker gains a simple access to the system, he corrupts the system or gives it an appearance of being corrupted. When the user sees the system in the corrupted state, he starts looking for help so as to solve the problem.
* Marketing: In order to make sure that the user approaches the attacker with the problem, the attacker advertises himself as the only person who can solve the problem.
* Support: In this step, the attacker gains the trust of the target and obtains access to sensitive information.
|| If your alarm didn't go off in the sabotage step, when your system appearing corrupt coincided with this remarkable good fortune of someone being right there at the right time ready to help, then let your alarm go off at the [[Propaganda#False_Dilemma|false dilemma]] you are presented with in the marketing step.
|}

== Targeted surveillance ==

=== Hardware implants ===
Unless you are targeted by a government intelligence agency, there seems to be no need to worry about installing commodity hardware from reputable vendors.

Detection: Looking for physical devices will always be the easiest solution to detect them. The links to NSA exploits of the day were added for the comments. :)

{| class="wikitable"
|-
! !! More information !! Possible types of attacks !! Detection
|-
| Godsurge || Godsurge is a physical device plugged-in to the Joint Test Action Group or JTAG headers on a system's motherboard. JTAG headers can be found on many systems and are notoriously common in embedded devices. These headers are used during the development process for debugging purposes: they give you a direct interface with the CPU and are extremely helpful. They are commonly left on the production boards, so finding them on a device is normal and not a security concern. However, if there is a chip or board wired in to a device's JTAG headers that you did not wire in yourself, then something fishy may be going on. || The JTAG debugging interface can be used to reflash the BIOS from scratch, for example loading a compromised version of the software.

|| Look for the JTAG connecter on the motherboard. The location of the JTAG headers may differ, but they tend to be near the CPU and may have exposed pins (or not). See the Wikipedia page on JTAG for more information and to see what they look like <ref>Wikipedia JTAG http://en.wikipedia.org/wiki/Joint_Test_Action_Group</ref>
|-
| Ginsu and Bulldozer
|| Ginsu provides software application persistence on target systems with the PCI bus hardware implant, Bulldozer
http://leaksource.files.wordpress.com/2013/12/nsa-ant-ginsu.jpg

https://www.schneier.com/blog/archives/2014/01/ginsu_nsa_explo.html
|| Exploit persistence from a PCI card ROM
|| Open up the computer's case and look for a PCI card that does not belong. For example, if you find a PCI card that appears to serve no purpose (e.g.. not your network card or it has no external ports), then perhaps try removing it and see how things work. If a few black SUVs roll up to your house after unplugging the PCI card, it's probably not because your domicile is the set for a rap superstar's new music video. Or maybe it is.
|-
| Cottonmouth I, Cottonmouth II and Cottonmouth III
|| These bugs are embedded somewhere along the USB bus and function as an air gap bridge to assist in exfiltration of data as well as allow persistent compromise. It can be embedded directly in the USB headers in an existing USB peripheral (USB hub or keyboard). These devices allow for exfiltration of data over unknown radio frequencies to listening devices in the area, even on a system that is not connected to the internet.
http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-i.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-i_n.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-ii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-ii.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonomouth-iii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-iii.html
|| USB host attack
|| Open up the keyboard or USB hub and identify a board that serves no purpose to the device. The malicious USB device would also likely show up on your computer's list of USB devices, so just checking there would be a good place to start.
|}

=== Radio frequency exfiltration ===
The devices used can vary, but they all employ a similar method of communication via an unknown radio/radar protocol. The frequencies are not known and based on the information leaked the devices are passive (only power on when data is being extracted) making identification of (type of) signal (and intelligence) hard.

Detection: First check for evidence of a device which has been wired in to an existing device such as a keyboard or other peripheral. The SpiderBlog describes the following possibility: You can check if there are RF transmitters in a device by monitoring the spectrum (using an amateur RF listening bug detector) while the device is off (to get a baseline for ambient RF background noise) and then monitoring it again after the suspect device is turned on and transmitting data via radio frequency. The detection device would pick up the signal and alert the user. <ref>Detecting A Surveillance State - Part 2 Radio Frequency Exfiltration https://www.trustwave.com/Resources/SpiderLabs-Blog/Detecting-A-Surveillance-State---Part-2-Radio-Frequency-Exfiltration/
</ref> Further analysis of the signal and its intelligence is still hard, as the exact protocols are (still) unknown.
{| class="wikitable"
|-
! !! More information !! Possible types of attack
|-
| Howlermonkey
|| Short to medium range Radio Transceiver

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/howlermonkey_ns.html
|| HOWLERMONKEY is a COTS-based transceiver deigned to be compatible with CONJECTURE/SPECULATION networks and STRIKEZONE devices running a HOWLERMONKEY personality. PCB layouts are tailored to individual implant space requirements and can vary greatly in form factor.
|-
| Ragemaster
|| Hardware implant in a VGA cable that sends video data over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/03/ragemaster_nsa.html
|| The RAGEMASTER taps the red video line between the video card within the desktop unit and the computer monitor, typically an LCD. When the RAGEMASTER is illuminated by a radar unit, the illuminating signal is modulated with the red video information. This information is re-radiated, where it is picked up at the radar, demodulated, and passed onto the processing unit, such as a LFS-2 and an external monitor, NIGHTWATCH, GOTHAM, or (in the future) VIEWPLATE. The processor recreates the horizontal and vertical sync of the targeted monitor, thus allowing TAO personnel to see what is displayed on the targeted monitor.
|-
| Loudauto
|| Hardware device that sends amplified audio over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/loudauto_nsa_ex.html

|| Room audio is picked up by the microphone and converted into an analog electrical signal. This signal is used to pulse position modulate (PPM) a square wave signal running at a pre-set frequency. This square wave is used to turn a FET (field effect transistor) on and off. When the unit is illuminated with a CW signal from a nearby radar unit, the illuminating signal is amplitude-modulated with the PPM square wave. This signal is re-radiated, where it is picked up by the radar, then processed to recover the room audio. Processing is currently performed by COTS equipment with FM demodulation capability (Rohde & Schwarz FSH-series portable spectrum analyzers, etc.) LOUDAUTO is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|-
| Surleyspawn
|| Hardware implant in a keyboard that emits keystrokes over RF

http://leaksource.files.wordpress.com/2013/12/nsa-ant-surlyspawn.jpg

https://www.schneier.com/blog/archives/2014/02/surlyspawn_nsa.html
|| The board taps into the data line from the keyboard to the processor. The board generates a square wave oscillating at a preset frequency. The data-line signal is used to shift the square wave frequency higher or lower, depending on the level of the data-line signal. The square wave, in essence, becomes frequency shift keyed (FSK). When the unit is illuminated by a CW signal from a nearby radar, the illuminating signal is amplitude-modulated (AM) with this square wave. The signal is re-radiated, where it is received by the radar, demodulated, and the demodulated signal is processed to recover the keystrokes. SURLYSPAWN is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|}

=== Infected firmware ===
Detection: Dump the BIOS to a bin file and compare the hash with a clean BIOS hash. See flashrom for identifying, reading, writing, erasing, and verifying BIOS/ROM/flash chips <ref>Debian packages: flashrom https://packages.debian.org/jessie/flashrom</ref>.

Removal: For each of these infections, where applicable, pulling the chip and replacing it with a new freshly burned BIOS chip or compact flash card would be sufficient. When dealing with built-in firmware it's a bit more difficult than pulling and replacing. You will need to re-flash the device using an operating system that is not at risk of being attacked by the infected firmware. You could boot the device into a low level OS in hopes that the firmware infection isn't able to protect itself. Or, you could wire in a debugging header to the device (such as JTAG , if available) to read or write the firmware on the device to clean things up for good.

{| class="wikitable"
|-
! !! More information !! Possible types of attack || Protection
|-
| Dietybounce
|| Motherboard BIOS Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-deitybounce.jpg?w=1208&h=1562

https://www.schneier.com/blog/archives/2014/01/nsa_exploit_of.html

|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS on a target machine to implant DEITYBOUNCE and its payload (the implant installer). Implantation via interdiction may be accomplished by nontechnical operator through use of a USB thumb drive. Once implanted, DEITYBOUNCE's frequency of execution (dropping the payload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Swap
|| Hard Drive Firmware Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-swap.jpg

https://www.schneier.com/blog/archives/2014/02/swap_nsa_exploi.html
|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS and TWISTEDKILT to write the Host Protected Area on the hard drive on a target machine in order to implant SWAP and its payload (the implant installer). Once implanted, SWAP's frequency of execution (dropping the playload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Headwater, sierramontana, and jetplow
|| Firmware backdoors that target popular networking hardware

https://leaksource.files.wordpress.com/2013/12/nsa-ant-headwater.jpg

https://www.schneier.com/blog/archives/2014/01/headwater_nsa_e.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-sierramontana.jpg

https://www.schneier.com/blog/archives/2014/01/sierramontana_n.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-jetplow.jpg

https://www.schneier.com/blog/archives/2014/01/jetplow_nsa_exp.html
|| HEADWATER PBD implant will be transferred remotely over the Internet to the selected target router by Remote Operations Center (ROC) personnel. After the transfer, the PBD will be installed in the router's boot ROM via an upgrade command. The PBD will then be activated after a system reboot. Once activated, the ROC operators will be able to use DNT's HAMMERMILL Insertion Tool (HIT) to control the PBD as it captures and examines all IP packets passing through the host router.

Currently, the intended DNT Implant to persist is VALIDATOR, which must be run as a user process on the target operating system. The vector of attack is the modification of the target's BIOS. The modification will add the necessary software to the BIOS and modify its software to execute the SIERRAMONTANA implant at the end of its native System Management Mode (SMM) handler.

JETPLOW persists DNT's BANANAGLEE software implant and modifies the Cisco firewall's operating system (OS) at boot time. If BANANAGLEE support is not available for the booting operating system, it can install a Persistent Backdoor (PDB) designed to work with BANANAGLEE'S communications structure, so that full access can be reacquired at a later time.
|| Most routers can be flashed with openWRT http://wiki.openwrt.org/toh/start
|}

== Dragnet surveillance ==

Best protection: Smash your PC to bits, distribute the pieces randomly among a dozen scrap heaps, and move into the woods, deep in the woods ...
{| class="wikitable"
|-
! !! More information !! !! Protection
|-
| PRISM || PRISM is part of the overall NSA surveillance effort - a program authorised in the united states under the FISA Amendments Act (FAA), now located in Section 702 of the FISA, that allows the NSA to collect communications of specifically identified foreign targets.

When people in the rest of the world use the net, they are effectively using US-based services, making them a legal target for US intelligence. NSA has no "direct access" to the servers of companies like Microsoft, Facebook and Google - other agencies collect the data. More [[Surveillance#PRISM|here]]
|| Gathers emails, chat--video, voice, photos, stored data, VoIP, file transfers, video conferencing, notifications of target activity--logins, etc, online social networking details, and special requests collected by other programs from nine major Silicon Valley technology companies: Facebook, Google, Yahoo, Microsoft, PalTalk, Skype, YouTube, Apple, and AOL.
||
* Put political pressure on the united states - economical and political dependencies are making that unlikely, see the power of scarcity above.

* Avoid using popular web services.
* FOIA's, court cases on legitimacy
|-
| Upstream collection
|| It appears that through Upstream collection, the NSA gains access to data by forming partnerships with both foreign intelligence agencies and foreign telecommunication companies, by partnering with united states telecommunications companies that then make agreements with international companies for their internet data. More [[Surveillance#Upstream_collection|here]]
|| BLARNEY seems to focus specifically on metadata information about communications traffic and network devices (where and when they were sent).

FAIRVIEW collects just about everything on packet and session levels.
||
* Research which countries and telecommunication companies participate
* Avoid using services associated with these programs
* [[Step 1#Understanding_and_minimising_our_digital_shadows|Understanding and minimising our digital shadows]]
* Court cases on legitimacy
|}

== Server attacks ==

{| class="wikitable"
|-
! !! !! Significance !! Prevention/Protection
|-
| XXS || Cross-site scripting is a vulnerability in php web applications. || Attackers can exploit it to steal user information. || Configure web server and write better secured PHP scripts (validating all user input)
|-
| SQL injection || SQL injection is a vulnerability in the database layer of a php application. || Any SQL statements can be executed by the application. || Configure the web server and write secure code (validating and escaping all user input)
|-
| File uploads || Allowing users to place files on the server. || Can be used to delete files, databases, get user details and much more || Validate user input and only allow image file type such as png or gif.
|-
| Including local and remote files || An attacker can open files from remote server and execute any PHP code. || Can be used to upload files, delete files and install backdoors. || Configure php to disable remote file execution.
|-
| eval() || Evaluate a string as PHP code. || Can be used by an attacker to hide their code and tools on the server itself. || Configure php to disable eval().
|-
| CSRF || A Cross-Site Request Forgery forces an authenticated end user to execute unwanted actions on a web application. || Can compromise end user data and operation in case of normal user. If the targeted end user is the administrator account, this can compromise the entire web application. || See the Cross-Site Request Forgery (CSRF) Prevention Cheat Sheet https://www.owasp.org/index.php/Cross-Site_Request_Forgery_%28CSRF%29_Prevention_Cheat_Sheet
|}

== Censorship ==
Table initially filled with data from How to effectively argue against Internet Censorship ideas http://rys.io/en/94.txt.

Proxy servers, especially anonymous ones, located outside the area where a censorship solution is deployed can be used quite easily to circumvent any blocking method; users can modify their operating system or browser settings, or install browser additions that make using this circumvention method trivial. It is possible to block the proxy servers themselves (via IP-blocking, keyword blocking, etc.), however it is infeasible to block them all, as they are easy to set-up.

Virtual Private Networks (including “poor man’s VPNs” like SSH tunnels) require more technical prowess and usually a (usually commercial) VPN service (or SSH server) outside the area with blocking deployed. Blocking all VPN/SSH traffic is possible, but requires deep packet inspection and is a serious problem for many legitimate businesses using VPNs (and SSH) as their daily tools of trade, to allow their employees access to corporate networks from outside physical premises, via a secured link on the Internet.

TOR, or The Onion Router, is a very effective (if a bit slow) circumvention method. It is quite easy to set-up — users can simply download the TOR Browser Bundle and use it to access the Internet. Due to the way it works it is nigh-impossible to block TOR traffic (as it looks just like vanilla HTTPS traffic), to the point that it is known to allow access to the uncensored Internet to those living in areas with most aggressive Internet censorship policies — namely China, North Korea and Iran. See Tor threats.

None of the censorship solutions is able to block content on darknets — virtual networks accessible anonymously only via specialised software (for instance TOR, I2P, FreeNet), and guaranteeing high resilience to censorship through technical composition of the networks themselves. Because darknets are both practically impossible to block entirely and not allowing for any content blocking within them, they are effectively the ultimate circumvention methods. The downside to using darknets is their lower bandwidth. Deploying Internet censorship pushes the to-be-blocked content into darknets, making it ever-harder for law enforcement gather evidence and researchers gather data on the popularity of a given type of censored content.
{| class="wikitable"

! Blocking type
! How it works
! Circumvention
! DPI
|-
|DNS-based blocking
|DNS-based blocking requires ISPs (who usually run their own DNS servers, being default for their clients) to de-list certain domains (so that they are not resolvable when using these DNS servers). This means that the costs of implementing it are small.
|Custom DNS server settings can be used to easily circumvent DNS-based blocking. It does not require almost any technical prowess and can be used by anybody. There is a number of publicly available DNS servers that can be used for this purpose. There is no way to easily block the use of this method without deploying censorship methods other than pure DNS-blocking.
|no
|-
|IP address-based blocking
|IP-based blocking requires the ISPs to either block certain IP addresses
<nowiki> </nowiki>internally or route all the outgoing connections via a central, government-mandated censoring entity. It is only superficially harder to circumvent, while retaining most if not all problems of DNS-based blocking.
|
|no
|-
|URL-based blocking
|Because this method blocks only certain, URL-identified content, not whole websites or servers (as do DNS-based and IP-based methods), it has much lower potential for accidental over-blocking. This also entails it has a higher potential for under-blocking, as the content can be available on the same server under many different URLs, and changing just a small part of the name defeats the filter.
|
|yes
|-
|Dynamic blocking
|This method uses deep packet inspection to read the contents of data being transmitted, and compares it with a list of keywords, or with image samples or video (depending on the content type).
|
|yes
|-
|Hash-based blocking
|Hash-based blocking uses deep packet inspection to inspect the contents of data-streams, hashes them with cryptographic hash functions and compares to a known database of hashes to be blocked.
|
|yes
|-
|Hybrid solutions
|In order to compromise between high-resource, low-over-blocking hash-based blocking and low-resource, high-over-blocking IP- or DNS-based solutions, a hybrid solution might be proposed. Usually it means that there is a list of IP addresses or domain names for which the hash-based blocking is enabled, hence only operating for a small part of content. This method does employ deep packet inspection.
|
|yes
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline merchants of death]]
* [[Timeline masters of the internet]]
* [[Threat modeling the quick and dirty way]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

[[Category:How_To]]

Digital threats, detection, protection and (counter) moves

2015-09-30T21:21:16Z

Lilith2:

This page lists theoretical defenses and detection methods for selected groups of leaked surveillance programs and services. This is just a thought experiment covering (theoretical) defenses against these attacks and not intended to spread fear, uncertainty or doubt about surveillance states.

Due to the age and limited scope of the leaked documents and what we are up against <ref>Cryptome: Communications privacy folly http://cryptome.org/2012/06/comms-folly.htm
</ref>, the defenses mentioned in these tables should not be relied upon for protection and I make no guarantees to their accuracy. Do your own research and make informed decisions, knowing it is impossible to be completely safe. Things move incredibly fast in this arena and I will update these tables when more is found and/or theorised.

Ideally we discuss these exploits and counter moves without the NSA being privy to those discussion, but that would make spreading possible counter moves too slow for the surveillance development cycle (security arms race) and effectively exclude non-techie activists from being able to defend themselves to some extent, so the next best solution is to discuss everything in the open, rather than not discussing them at all. May we not have to rely on a new class of technocracy.

== Social engineering ==
These tables were started off from http://www.itsecurity.be/social-engineering-what-is-it-and-how-to-defend-yourself and then added to. I started off from a to gendersec external source to increase chances of finding that which we may be overlooking.

=== Internal threats ===

{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Power of scarcity
|| Scarcity is when people are told something they need or want has limited availability and to get it they must comply with a certain attitude or action. Many times the desired behavior is not even spoken, but the way it is conveyed is by showing people who are acting "properly" getting rewards.

Mental shortcuts are often beneficial because they enable us to more efficiently navigate the complexity of life. But the power of scarcity can also mislead us. If some people get something not everyone else has (energy, food, water), it makes them feel "special", "unique", "high status", and gives a "position" to defend. And fear of loss of that is an attachment.

An example is a government, in this case South Africa, taking something necessary to life, and making it "scarce" and available only to supporters — a malicious, but very effective, manipulation tactic: http://www.social-engineer.org/wiki/archives/Governments/Governments-FoodElectionWeapon.html See [[timeline merchants of death]] for more examples.

|| Scarcity complicates Gut/Head decision processes because, often, there is only a brief window of opportunity in which we can select something that is in scarce supply. When we can see the supply diminishing before our eyes, we feel particularly compelled to take action.

Each context and scarcity is different, but mind not to rule out the head too quickly in [[Scenario planning#Gut.2C head.2C and decision making|"Gut versus Head"]] decisions.
|-
| Fear of authority || Many people are apprehensive in the presence of someone they perceive to be an authority figure, it is not that person they are apprehensive about but most likely the position and power of the person that intimidates them.
<nowiki> </nowiki>The attackers take on roles of authority figures such as law enforcement officers or high-ranking company officials to extract sensitive information from the victims.
|| Protect yourself with learning about [[Self-dox#Elicitation|elicitation]] (in roleplay settings) and then, time for some controlled folly!
|-
| Desire to be helpful
|| People in their desire to be helpful and to solve other peoples queries, give out a lot of information that otherwise should not be. || Do not disclose information to an outsider as it could give an attacker a chance to get unauthorised access.
|-
| Laziness || All of us have come across some job that requires us to do only a specified set of activities and nothing more. This causes boredom to the person who performs the same task repeatedly on a daily basis and over time the "bored" learn "shortcuts" to do the tasks using minimal effort while meeting targets. This leads to a laid back attitude and becoming susceptible to attackers who target such individuals knowing they can get the required information with much ease. || Find ways of keeping yourself alert. Go do something else. You are becoming a sitting duck.
|-
| Ego || Many a times, the attacker makes a person more emotionally sure of himself/herself, thus removing the logical awareness of the security breach that is occurring. The result is that the person being hacked senses no harm in providing whatever it is that the attacker is requesting. The reason that such an attack succeeds is that the attacker is a receptive audience for victims to display how much knowledge they have. || The ego isn’t going to go away and whether it makes you vulnerable depends on the context. A bit of swagger based on achievements can be quite okay now and then! In some other contexts allowing your ego to call the shots can be a problem.

The simplest detector is asking yourself: Do you feel superior (or inferior) to others? If yes, then your ego is in control. Depending on context choose "not now, not here, not with this" or "do some counter-intelligence" or "Arrrrr, and a bottle of rum!"
|-
| Excitement of victory & fear of loss || Yes! An opportunity! Easy money!!! Out of excitement security is switched off, links are clicked and documents downloaded that turn out to be corrupted (and contained malware that allows the email sender to gain remote access to the machine, or someone's funds, or data, or ...).

Or, ''"You have won 1 Million Dollars and to claim the winning amount, deposit $75,000 in Account number: XXXXXX in 10 days from receiving this e-mail, failing to which the winning amount would be declared unclaimed and there would be a nee lucky-draw to decide the next winner"''

An example is the 419 scams many people fell for http://www.social-engineer.org/wiki/archives/ConMen/ConMen-Scam-NigerianFee.html
|| It is another ego thing isn't it?
|-
| Insufficient knowledge || People with insufficient knowledge can easily be exploited by creating a sense of urgency and not allowing much time to think and understanding the fact that they are under attack. || Gather (and spread) knowledge in a relaxed way. Stay with your own timing. Do the best you can.
|}

=== External threats ===
{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Shoulder surfing || Shoulder surfing is a security attack where-in, the attacker uses observational techniques, such as looking over someone's shoulder, to get information while they are performing some action that involves explicit usage of sensitive, visible information. This can be performed at a close range as well as at a long range using binoculars or other vision-enhancing devices. || Don't work on anything important in public.
|-
| Dumpster diving || Going through the trash can yield one of the most lucrative payoffs for information gathering. People often throw away invoices, notices, letters, CDs, computers, USB keys, and a plethora of other devices and reports that can truly give amazing amounts of information. The attacker can use these items to get a huge amount of information about people, organisations they participate in, and network structure.

Some people shred documents but some types of shredding can be thwarted with a little time and patience and some tape.
|| Mind what you throw away where and how.

Using a shredder that shreds both directions into a fine minced mess makes taping documents back together nearly impossible. Or incinerate. Or both.
|-
| Role playing || Role playing is one of the key weapons for a social engineer. It involves persuading or gathering information through the use of online chat sessions, emails, phones or any other method that you use to interact online with others, and in which the social engineer plays the role of a helpdesk or technician, helplessness, or whatever may work in that context to get targets to divulge confidential information. || Don't "burn" the social engineer. Apply controlled folly instead, and then let's see who is the cat here and who the mouse: Feed them incorrect information. Remember what the intent is. Intent is the key element in intelligence and counter-intelligence operations.
|-
| Trojan horses || It is one of the most predominant methods currently used by online criminals and intelligence agencies that involve tricking victims into downloading a malicious file to their machine, which on execution creates a backdoor in the machine that can be used by the attacker any time in the future and thus having complete access of the victim's machine. Compromising a browser is relatively easy and it is cross-platform, hence an often chosen attack vector. <ref>Meet The Hackers Who Sell Spies The Tools To Crack Your PC (And Get Paid Six-Figure Fees) http://www.forbes.com/sites/andygreenberg/2012/03/21/meet-the-hackers-who-sell-spies-the-tools-to-crack-your-pc-and-get-paid-six-figure-fees/</ref>.
<nowiki> </nowiki>Among all of the NSA hacking operations exposed by whistleblower Edward Snowden over the last two years, one in particular has stood out for its sophistication and stealthiness. Known as Quantum Insert, the man-on-the-side hacking technique has been used to great effect since 2005 by the NSA and its partner spy agency, Britain’s GCHQ, to hack into high-value, hard-to-reach systems and implant malware.

|| Detection focuses on identifying anomalies in the data packets that get sent to a victim’s browser client when the browser attempts to access web pages http://blog.fox-it.com/2015/04/20/deep-dive-into-quantum-insert/

For more such tools run by intelligence agency for targeted surveillance see [[#Infected_firmware|infected firmware]] below.
|-
| Phishing || Phishing is the act of creating and using websites and e-mails designed to look like those of well-known legitimate businesses, financial institutions and government agencies to deceive receivers into disclosing their personal information.
|| These are semantic attacks.
* Be suspicious of any email with urgent requests for personal (financial) information.
* Set your email client to receive email in plaintext, never html format.
* Check any url in the email carefully. It can be just one letter different or missing.
* If phishy, report it.
|-
| Information on websites, online forums and social media|| Huge amounts of information regarding organisational structures (formal and informal), email adresses, phone numbers, are all available publicly on websites, forums and social media. This information can be used by the attacker to refine his approach and create a plan on whom to target and methods most likely to succeed. || Several moves are possible (not necessarily exclusive):
* Delete your social media presence
* Use different identities for your various social contexts
|-
| Reverse social engineering || A reverse social engineering attack is an attack in which an attacker convinces a target that he or she has a problem or might have a certain problem in the future and that the attacker, is ready to help solve the problem.

Reverse social engineering involves three parts:
* Sabotage: After the attacker gains a simple access to the system, he corrupts the system or gives it an appearance of being corrupted. When the user sees the system in the corrupted state, he starts looking for help so as to solve the problem.
* Marketing: In order to make sure that the user approaches the attacker with the problem, the attacker advertises himself as the only person who can solve the problem.
* Support: In this step, the attacker gains the trust of the target and obtains access to sensitive information.
|| If your alarm didn't go off in the sabotage step, when your system appearing corrupt coincided with this remarkable good fortune of someone being right there at the right time ready to help, then let your alarm go off at the [[Propaganda#False_Dilemma|false dilemma]] you are presented with in the marketing step.
|}

== Targeted surveillance ==

=== Hardware implants ===
Unless you are targeted by a government intelligence agency, there seems to be no need to worry about installing commodity hardware from reputable vendors.

Detection: Looking for physical devices will always be the easiest solution to detect them. The links to NSA exploits of the day were added for the comments. :)

{| class="wikitable"
|-
! !! More information !! Possible types of attacks !! Detection
|-
| Godsurge || Godsurge is a physical device plugged-in to the Joint Test Action Group or JTAG headers on a system's motherboard. JTAG headers can be found on many systems and are notoriously common in embedded devices. These headers are used during the development process for debugging purposes: they give you a direct interface with the CPU and are extremely helpful. They are commonly left on the production boards, so finding them on a device is normal and not a security concern. However, if there is a chip or board wired in to a device's JTAG headers that you did not wire in yourself, then something fishy may be going on. || The JTAG debugging interface can be used to reflash the BIOS from scratch, for example loading a compromised version of the software.

|| Look for the JTAG connecter on the motherboard. The location of the JTAG headers may differ, but they tend to be near the CPU and may have exposed pins (or not). See the Wikipedia page on JTAG for more information and to see what they look like <ref>Wikipedia JTAG http://en.wikipedia.org/wiki/Joint_Test_Action_Group</ref>
|-
| Ginsu and Bulldozer
|| Ginsu provides software application persistence on target systems with the PCI bus hardware implant, Bulldozer
http://leaksource.files.wordpress.com/2013/12/nsa-ant-ginsu.jpg

https://www.schneier.com/blog/archives/2014/01/ginsu_nsa_explo.html
|| Exploit persistence from a PCI card ROM
|| Open up the computer's case and look for a PCI card that does not belong. For example, if you find a PCI card that appears to serve no purpose (e.g.. not your network card or it has no external ports), then perhaps try removing it and see how things work. If a few black SUVs roll up to your house after unplugging the PCI card, it's probably not because your domicile is the set for a rap superstar's new music video. Or maybe it is.
|-
| Cottonmouth I, Cottonmouth II and Cottonmouth III
|| These bugs are embedded somewhere along the USB bus and function as an air gap bridge to assist in exfiltration of data as well as allow persistent compromise. It can be embedded directly in the USB headers in an existing USB peripheral (USB hub or keyboard). These devices allow for exfiltration of data over unknown radio frequencies to listening devices in the area, even on a system that is not connected to the internet.
http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-i.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-i_n.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-ii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-ii.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonomouth-iii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-iii.html
|| USB host attack
|| Open up the keyboard or USB hub and identify a board that serves no purpose to the device. The malicious USB device would also likely show up on your computer's list of USB devices, so just checking there would be a good place to start.
|}

=== Radio frequency exfiltration ===
The devices used can vary, but they all employ a similar method of communication via an unknown radio/radar protocol. The frequencies are not known and based on the information leaked the devices are passive (only power on when data is being extracted) making identification of (type of) signal (and intelligence) hard.

Detection: First check for evidence of a device which has been wired in to an existing device such as a keyboard or other peripheral. The SpiderBlog describes the following possibility: You can check if there are RF transmitters in a device by monitoring the spectrum (using an amateur RF listening bug detector) while the device is off (to get a baseline for ambient RF background noise) and then monitoring it again after the suspect device is turned on and transmitting data via radio frequency. The detection device would pick up the signal and alert the user. <ref>Detecting A Surveillance State - Part 2 Radio Frequency Exfiltration https://www.trustwave.com/Resources/SpiderLabs-Blog/Detecting-A-Surveillance-State---Part-2-Radio-Frequency-Exfiltration/
</ref> Further analysis of the signal and its intelligence is still hard, as the exact protocols are (still) unknown.
{| class="wikitable"
|-
! !! More information !! Possible types of attack
|-
| Howlermonkey
|| Short to medium range Radio Transceiver

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/howlermonkey_ns.html
|| HOWLERMONKEY is a COTS-based transceiver deigned to be compatible with CONJECTURE/SPECULATION networks and STRIKEZONE devices running a HOWLERMONKEY personality. PCB layouts are tailored to individual implant space requirements and can vary greatly in form factor.
|-
| Ragemaster
|| Hardware implant in a VGA cable that sends video data over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/03/ragemaster_nsa.html
|| The RAGEMASTER taps the red video line between the video card within the desktop unit and the computer monitor, typically an LCD. When the RAGEMASTER is illuminated by a radar unit, the illuminating signal is modulated with the red video information. This information is re-radiated, where it is picked up at the radar, demodulated, and passed onto the processing unit, such as a LFS-2 and an external monitor, NIGHTWATCH, GOTHAM, or (in the future) VIEWPLATE. The processor recreates the horizontal and vertical sync of the targeted monitor, thus allowing TAO personnel to see what is displayed on the targeted monitor.
|-
| Loudauto
|| Hardware device that sends amplified audio over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/loudauto_nsa_ex.html

|| Room audio is picked up by the microphone and converted into an analog electrical signal. This signal is used to pulse position modulate (PPM) a square wave signal running at a pre-set frequency. This square wave is used to turn a FET (field effect transistor) on and off. When the unit is illuminated with a CW signal from a nearby radar unit, the illuminating signal is amplitude-modulated with the PPM square wave. This signal is re-radiated, where it is picked up by the radar, then processed to recover the room audio. Processing is currently performed by COTS equipment with FM demodulation capability (Rohde & Schwarz FSH-series portable spectrum analyzers, etc.) LOUDAUTO is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|-
| Surleyspawn
|| Hardware implant in a keyboard that emits keystrokes over RF

http://leaksource.files.wordpress.com/2013/12/nsa-ant-surlyspawn.jpg

https://www.schneier.com/blog/archives/2014/02/surlyspawn_nsa.html
|| The board taps into the data line from the keyboard to the processor. The board generates a square wave oscillating at a preset frequency. The data-line signal is used to shift the square wave frequency higher or lower, depending on the level of the data-line signal. The square wave, in essence, becomes frequency shift keyed (FSK). When the unit is illuminated by a CW signal from a nearby radar, the illuminating signal is amplitude-modulated (AM) with this square wave. The signal is re-radiated, where it is received by the radar, demodulated, and the demodulated signal is processed to recover the keystrokes. SURLYSPAWN is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|}

=== Infected firmware ===
Detection: Dump the BIOS to a bin file and compare the hash with a clean BIOS hash. See flashrom for identifying, reading, writing, erasing, and verifying BIOS/ROM/flash chips <ref>Debian packages: flashrom https://packages.debian.org/jessie/flashrom</ref>.

Removal: For each of these infections, where applicable, pulling the chip and replacing it with a new freshly burned BIOS chip or compact flash card would be sufficient. When dealing with built-in firmware it's a bit more difficult than pulling and replacing. You will need to re-flash the device using an operating system that is not at risk of being attacked by the infected firmware. You could boot the device into a low level OS in hopes that the firmware infection isn't able to protect itself. Or, you could wire in a debugging header to the device (such as JTAG , if available) to read or write the firmware on the device to clean things up for good.

{| class="wikitable"
|-
! !! More information !! Possible types of attack || Protection
|-
| Dietybounce
|| Motherboard BIOS Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-deitybounce.jpg?w=1208&h=1562

https://www.schneier.com/blog/archives/2014/01/nsa_exploit_of.html

|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS on a target machine to implant DEITYBOUNCE and its payload (the implant installer). Implantation via interdiction may be accomplished by nontechnical operator through use of a USB thumb drive. Once implanted, DEITYBOUNCE's frequency of execution (dropping the payload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Swap
|| Hard Drive Firmware Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-swap.jpg

https://www.schneier.com/blog/archives/2014/02/swap_nsa_exploi.html
|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS and TWISTEDKILT to write the Host Protected Area on the hard drive on a target machine in order to implant SWAP and its payload (the implant installer). Once implanted, SWAP's frequency of execution (dropping the playload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Headwater, sierramontana, and jetplow
|| Firmware backdoors that target popular networking hardware

https://leaksource.files.wordpress.com/2013/12/nsa-ant-headwater.jpg

https://www.schneier.com/blog/archives/2014/01/headwater_nsa_e.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-sierramontana.jpg

https://www.schneier.com/blog/archives/2014/01/sierramontana_n.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-jetplow.jpg

https://www.schneier.com/blog/archives/2014/01/jetplow_nsa_exp.html
|| HEADWATER PBD implant will be transferred remotely over the Internet to the selected target router by Remote Operations Center (ROC) personnel. After the transfer, the PBD will be installed in the router's boot ROM via an upgrade command. The PBD will then be activated after a system reboot. Once activated, the ROC operators will be able to use DNT's HAMMERMILL Insertion Tool (HIT) to control the PBD as it captures and examines all IP packets passing through the host router.

Currently, the intended DNT Implant to persist is VALIDATOR, which must be run as a user process on the target operating system. The vector of attack is the modification of the target's BIOS. The modification will add the necessary software to the BIOS and modify its software to execute the SIERRAMONTANA implant at the end of its native System Management Mode (SMM) handler.

JETPLOW persists DNT's BANANAGLEE software implant and modifies the Cisco firewall's operating system (OS) at boot time. If BANANAGLEE support is not available for the booting operating system, it can install a Persistent Backdoor (PDB) designed to work with BANANAGLEE'S communications structure, so that full access can be reacquired at a later time.
|| Most routers can be flashed with openWRT http://wiki.openwrt.org/toh/start
|}

== Dragnet surveillance ==

Best protection: Smash your PC to bits, distribute the pieces randomly among a dozen scrap heaps, and move into the woods, deep in the woods ...
{| class="wikitable"
|-
! !! More information !! !! Protection
|-
| PRISM || PRISM is part of the overall NSA surveillance effort - a program authorised in the united states under the FISA Amendments Act (FAA), now located in Section 702 of the FISA, that allows the NSA to collect communications of specifically identified foreign targets.

When people in the rest of the world use the net, they are effectively using US-based services, making them a legal target for US intelligence. NSA has no "direct access" to the servers of companies like Microsoft, Facebook and Google - other agencies collect the data. More [[Surveillance#PRISM|here]]
|| Gathers emails, chat--video, voice, photos, stored data, VoIP, file transfers, video conferencing, notifications of target activity--logins, etc, online social networking details, and special requests collected by other programs from nine major Silicon Valley technology companies: Facebook, Google, Yahoo, Microsoft, PalTalk, Skype, YouTube, Apple, and AOL.
||
* Put political pressure on the united states - economical and political dependencies are making that unlikely, see the power of scarcity above.

* Avoid using popular web services.
* FOIA's, court cases on legitimacy
|-
| Upstream collection
|| It appears that through Upstream collection, the NSA gains access to data by forming partnerships with both foreign intelligence agencies and foreign telecommunication companies, by partnering with united states telecommunications companies that then make agreements with international companies for their internet data. More [[Surveillance#Upstream_collection|here]]
|| BLARNEY seems to focus specifically on metadata information about communications traffic and network devices (where and when they were sent).

FAIRVIEW collects just about everything on packet and session levels.
||
* Research which countries and telecommunication companies participate
* Avoid using services associated with these programs
* [[Step 1#Understanding_and_minimising_our_digital_shadows|Understanding and minimising our digital shadows]]
* Court cases on legitimacy
|}

== Server attacks ==

{| class="wikitable"
|-
! !! !! Significance !! Prevention/Protection
|-
| XXS || Cross-site scripting is a vulnerability in php web applications. || Attackers can exploit it to steal user information. || Configure web server and write better secured PHP scripts (validating all user input)
|-
| SQL injection || SQL injection is a vulnerability in the database layer of a php application. || Any SQL statements can be executed by the application. || Configure the web server and write secure code (validating and escaping all user input)
|-
| File uploads || Allowing users to place files on the server. || Can be used to delete files, databases, get user details and much more || Validate user input and only allow image file type such as png or gif.
|-
| Including local and remote files || An attacker can open files from remote server and execute any PHP code. || Can be used to upload files, delete files and install backdoors. || Configure php to disable remote file execution.
|-
| eval() || Evaluate a string as PHP code. || Can be used by an attacker to hide their code and tools on the server itself. || Configure php to disable eval().
|-
| CSRF || A Cross-Site Request Forgery forces an authenticated end user to execute unwanted actions on a web application. || An compromise end user data and operation in case of normal user. If the targeted end user is the administrator account, this can compromise the entire web application. || See the Cross-Site Request Forgery (CSRF) Prevention Cheat Sheet https://www.owasp.org/index.php/Cross-Site_Request_Forgery_%28CSRF%29_Prevention_Cheat_Sheet
|}

== Censorship ==
Table initially filled with data from How to effectively argue against Internet Censorship ideas http://rys.io/en/94.txt.

Proxy servers, especially anonymous ones, located outside the area where a censorship solution is deployed can be used quite easily to circumvent any blocking method; users can modify their operating system or browser settings, or install browser additions that make using this circumvention method trivial. It is possible to block the proxy servers themselves (via IP-blocking, keyword blocking, etc.), however it is infeasible to block them all, as they are easy to set-up.

Virtual Private Networks (including “poor man’s VPNs” like SSH tunnels) require more technical prowess and usually a (usually commercial) VPN service (or SSH server) outside the area with blocking deployed. Blocking all VPN/SSH traffic is possible, but requires deep packet inspection and is a serious problem for many legitimate businesses using VPNs (and SSH) as their daily tools of trade, to allow their employees access to corporate networks from outside physical premises, via a secured link on the Internet.

TOR, or The Onion Router, is a very effective (if a bit slow) circumvention method. It is quite easy to set-up — users can simply download the TOR Browser Bundle and use it to access the Internet. Due to the way it works it is nigh-impossible to block TOR traffic (as it looks just like vanilla HTTPS traffic), to the point that it is known to allow access to the uncensored Internet to those living in areas with most aggressive Internet censorship policies — namely China, North Korea and Iran. See Tor threats.

None of the censorship solutions is able to block content on darknets — virtual networks accessible anonymously only via specialised software (for instance TOR, I2P, FreeNet), and guaranteeing high resilience to censorship through technical composition of the networks themselves. Because darknets are both practically impossible to block entirely and not allowing for any content blocking within them, they are effectively the ultimate circumvention methods. The downside to using darknets is their lower bandwidth. Deploying Internet censorship pushes the to-be-blocked content into darknets, making it ever-harder for law enforcement gather evidence and researchers gather data on the popularity of a given type of censored content.
{| class="wikitable"

! Blocking type
! How it works
! Circumvention
! DPI
|-
|DNS-based blocking
|DNS-based blocking requires ISPs (who usually run their own DNS servers, being default for their clients) to de-list certain domains (so that they are not resolvable when using these DNS servers). This means that the costs of implementing it are small.
|Custom DNS server settings can be used to easily circumvent DNS-based blocking. It does not require almost any technical prowess and can be used by anybody. There is a number of publicly available DNS servers that can be used for this purpose. There is no way to easily block the use of this method without deploying censorship methods other than pure DNS-blocking.
|no
|-
|IP address-based blocking
|IP-based blocking requires the ISPs to either block certain IP addresses
<nowiki> </nowiki>internally or route all the outgoing connections via a central, government-mandated censoring entity. It is only superficially harder to circumvent, while retaining most if not all problems of DNS-based blocking.
|
|no
|-
|URL-based blocking
|Because this method blocks only certain, URL-identified content, not whole websites or servers (as do DNS-based and IP-based methods), it has much lower potential for accidental over-blocking. This also entails it has a higher potential for under-blocking, as the content can be available on the same server under many different URLs, and changing just a small part of the name defeats the filter.
|
|yes
|-
|Dynamic blocking
|This method uses deep packet inspection to read the contents of data being transmitted, and compares it with a list of keywords, or with image samples or video (depending on the content type).
|
|yes
|-
|Hash-based blocking
|Hash-based blocking uses deep packet inspection to inspect the contents of data-streams, hashes them with cryptographic hash functions and compares to a known database of hashes to be blocked.
|
|yes
|-
|Hybrid solutions
|In order to compromise between high-resource, low-over-blocking hash-based blocking and low-resource, high-over-blocking IP- or DNS-based solutions, a hybrid solution might be proposed. Usually it means that there is a list of IP addresses or domain names for which the hash-based blocking is enabled, hence only operating for a small part of content. This method does employ deep packet inspection.
|
|yes
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline merchants of death]]
* [[Timeline masters of the internet]]
* [[Threat modeling the quick and dirty way]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

[[Category:How_To]]

Timeline that is soooo 1984 ...

2015-09-30T08:22:20Z

Lilith2:

This page contains links gathered, arrogated, appropriated, pirated, stolen, fenced, forwarded, and shared in general. Add your news item or story, if need be (in case of freedom of speech impediments in your locality) anonymously. You can also contact us on IRC.

== Indefinite solitary confinement ==

Whistleblower Chelsea Manning is now facing the serious threat of "indefinite solitary confinement" in prison at Fort Leavenworth, where she is serving a 35-year sentence for providing WikiLeaks with documentation of a vast array of war crimes and deception by the U.S. government <ref>Chelsea Manning Faces Solitary Confinement http://www.newsweek.com/chelsea-manning-faces-indefinite-solitary-confinement-having-caitlyn-jenner-362471</ref><ref>Trevor Timm: Chelsea Manning Threatened with ‘Indefinite Solitary Confinement’ http://boingboing.net/2015/08/12/chelsea-manning-threatened-wit.html</ref><ref>BuzzFeed News: Chelsea Manning Faces Solitary Confinement http://www.buzzfeed.com/chrisgeidner/chelsea-manning-faces-solitary-confinement-under-new-charges</ref><ref>Urgent Petition in Support of Chelsea Manning's Human Rights http://act.rootsaction.org/p/dia/action3/common/public/?action_KEY=11573</ref>.

== Fined for posting picture of police car ==

Unnamed woman from Alicante ordered to pay €800 under controversial gagging law for posting photo on her Facebook page. A Spanish woman has been fined €800 (£570) under the country’s controversial new gagging law for posting a photograph of a police car parked illegally in a disabled bay. The unnamed woman, a resident of Petrer in Alicante, south-east Spain, posted the photo on her Facebook page with the comment “Park where you bloody well please and you won’t even be fined” <ref>Spanish woman fined for posting picture of police parked in disabled bay http://www.theguardian.com/world/2015/aug/16/spanish-woman-fined-gagging-law-photographing-police</ref>.

== Irrational, unpredictable authorities ==

US, 09/02/2015, on Barrett Brown vs. the Dept of Justice – Defining the Right To Link: ''It is clear from the indictment and the transcript that prosecutor Candina Heath is willing to go after people on flimsy theories, and catching a case alone can be punishment enough. If defendants are denied bail, they can be detained for years—prior to his final sentencing, Brown spent more than 850 days behind bars. If they hire lawyers, they must pay legal costs. No matter what the accused try to do, the odds are incredibly stacked against them: 97 percent of federal cases end in plea bargains <ref>Stronger Hand for Judges in the ‘Bazaar’ of Plea Deals http://www.nytimes.com/2012/03/23/us/stronger-hand-for-judges-after-rulings-on-plea-deals.html</ref>.''

''The authorities’ arguments do not even have to make sense. As the defense’s motion to dismiss the indictment points out, the first count alleges Brown trafficked in “authentication features”—but the law defines those as springing only from governmental “issuing authorities,” not credit card companies <ref>18 U.S. Code § 1028 - Fraud and related activity in connection with identification documents, authentication features, and information https://www.law.cornell.edu/uscode/text/18/1028</ref>. The flat-out error is reminiscent of the prosecution’s earlier indictment, regarding Brown’s threats, crazily interpreting the writer’s disapproving retweet of Fox News analyst Bob Beckel advocating the murder of WikiLeaks editor-in-chief Julian Assange—“illegally shoot the son of a bitch”—as Brown somehow threatening the FBI agent. These irrational accusations are the Department of Justice wildly swinging punches wherever it can, aiming to take down the journalist, but not in any accordance with reason and only allowed to box because we let it.''<ref>Barrett Brown vs. the Dept of Justice – Defining the Right To Link http://revolution-news.com/barrett-brown-vs-the-dept-of-justice-defining-the-right-to-link/</ref>''

== Smart Cities ==

''"They will lead to unprecedented improvements in the quality of life."'' You have *got* to be kidding me. And knowing current practices of "giving people a choice" there is no opt-in (by default we are all in) and we may be allowed to opt-out (but only for a part of it). Sensors everywhere. Access points everywhere. Smaaaart. Not. http://www.smart-cities.eu/

== Fictional internet policy is bad for India, good only for Facebook ==

''Zuckerberg's "Free Basics" is a scam against its supposed beneficiaries for several reasons. First, rather than offering "the Internet," his service requires its users to route all their traffic to "free websites" through his servers, where the users' identities are logged so that their traffic can be paid for by the spy, rather than by them. So the first actual charge is that the poor will be comprehensively surveilled by Facebook, losing any shred of personal privacy, while the rich using the real Internet do not route all their traffic through Facebook.''

''Second, Zuckerberg destroys the security of his users, the benefited poor. As announced, Zuckerberg's service prohibited all use of the secure web protocol HTTPS (the one that lights up the little lock image on the status bar of your browser). HTTPS, and its authentication mechanism, are the only reasons that online banking and e-commerce are safe for consumers. So not only were the Indian poor to lose all chance of anonymity on the Net with respect to Zuckerberg, but they were also to abandon any possibility of common safety in the Net.'' http://tech.firstpost.com/news-analysis/fictional-internet-policy-is-bad-for-india-good-only-for-facebook-282664.html

== Resources ==

* Chapter Six – Removing The Veil (Part 1) http://underminers.org/the-book/chapter-6/chapter-6-part-1/
* Chapter Six – Removing The Veil (Part 2) http://underminers.org/the-book/chapter-6/chapter-6-part-2/
* Chapter Six – Removing The Veil (Part 3) http://underminers.org/the-book/chapter-6/chapter-6-part-3/

== Related ==

* [[Roleplay]]
* [[Elicitation]]
* [[Wordsmithing]]

== References ==

User:Lilith2

2015-09-29T22:08:00Z

Lilith2: /* Autonomy shaping infrastructure */

[[File:fiddling.jpg|300px|thumb|right|Keep calm & Keep on fiddling]]
== Narratives ==
All of us, without exception, believe a mix of truth and misinformation, and often enough, disinformation. We strive to understand the world as it is, and not how it looks only according to our preconceptions, which are shaped by a multitude of forces, embedded as we are in our cultural matrix. Sometimes, the most unlikely seeming explanation turns out to be the correct one. In a warrior mindset we consider alternative views, but question everything.
* [[News & Watchdogs]]
Asking for details from ones own cultural matrix is not enough. Not even close. One needs to eat the local food, drink the water, breathe the air, listen to the stories, smell the earth, feel the bark of the trees, see local peoples, and experience local ways to catch what can only be described as a mere glimpse of the answer 42, the knowledge hidden in the trees and wheels in that specific locality on this beautiful planet.

People and communities use stories to understand the world and our place in it. These stories are embedded with power - the power to explain and justify the status quo as well as the power to make change imaginable and urgent.
* [[Timeline merchants of death]]
* [[Timeline masters of the internet]]
* [[Timeline that is soooo 1984 ...]]
A narrative analysis of power encourages us to ask: Which stories define cultural norms? Where did these stories come from? Whose stories were ignored or erased to create these norms? And, most urgently, what new stories can we tell to help create the world we desire? <ref>Harnessing the power of narrative for social change https://www.newtactics.org/conversation/change-story-harnessing-power-narrative-social-change</ref><ref>Underminers Chapter Ten – Reclaiming Ourselves http://underminers.org/the-book/chapter-10/</ref>
== Mindmaps and mindsets ==
If a mindmap is a cognitive "hathanger" then a mindset are the clothes hanging on the hathanger. Most traditions have mindmaps that have been and are evolving locally from the experiences of the previous generations on what worked and what didn't for them. Afaik, the oldest mindmaps use trees and wheels. And all mindmaps are generalisations in the wind without grounding details if not from there. The adversary grouping of the petty tyrants map is useful for gathering and interpreting data from narratives, questioning and timelines into a framework of possible threats.

* [[Petty tyrants]]
* [[Captivating capital and copyfighting]]
* [[Encrypting everything]]
* [[Arcane mixing techniques]]
* [[Networking concepts]]
* [[A typical linux distribution]]
* [[Memory matters]]
* [[Traffic analysis]]

== Methodologies, processes and choreographies ==
Most processes are adapted to allow for minimalist approaches and to minimalise the risk of unintentionally becoming the petty tyrants we fight <ref>Portal 2 Soundtrack Cara Mia (Turret Opera) One Hour https://www.youtube.com/watch?v=wFKfkfCeFj4</ref>.

* [[Shortest retrospective ever]]
* [[Threat modeling the quick and dirty way]] (food for gut, looking backward)
* [[Scenario planning]] (food for thought, looking forward)
* [[Linux development process]]

== Edge ==

* [[Autonomous collectives]]
* [[Controlled folly]]
* [[Confusing surveillance systems]]
* [[Simulations]]
** [[Simulation: Game of Goose]]
** [[Simulation: Mixnets]]
* [[Linux virtualisation]]

== Getting started with linux ==
''Standing on a hilltop in a thunderstorm on bare feet, wearing wet copper armour, holding a lightning rod and shouting ... bring it on, ye gods and godesses!''

We use lawful techniques and tools. And if need be we invent new (GNU and creative commons licensed) techniques and tools <ref>Portal - 'Still Alive' https://www.youtube.com/watch?v=Y6ljFaKRTrI</ref>.

* [[Installing linux]]
* [[Linux applications]]
* [[Linux security]]
* [[Kinky linux command-line]]
* [[Object encryption on linux]]
* [[Anonymising your traffic with linux]]
* [[Clean up all the things]]
* [[Shell scripting]]

== Teacher inside ==

* [[Self-dox]]
* [[Reconnaissance]]
* [[Network exploitation and monitoring]]
* [[Reverse engineering]]
* [[Roleplay]]
* [[Trolling]]
* [[Elicitation]]
* [[Wordsmithing]]
* [[Take control of the tech]]
* [[Making our own linux images]]

== Autonomy shaping infrastructure ==

* For setting up a home server see the [https://undisconnect.miraheze.org/wiki/Main_Page Servers Everywhere! section on UnDisConnect]
* For setting up a server see the documentation in the [http://anarchaserver.org/mediawiki/index.php/Anarcha_section Anarcha section on Alexandria]
* [[Linux server security]]
* Radio
* Libraries
* Meshnets
* Clusternets

== Solidarity network ==

* [[Threats, detection, protection and (counter) moves]]
* [[Autonomy patterns]]
* [[Communication patterns]]
* Mirroring servers
* Load balancing servers
* Reframing and refactoring operational accounts

== References ==
<references />.

User:Lilith2

2015-09-29T22:04:10Z

Lilith2: /* Autonomy shaping infrastructure */

[[File:fiddling.jpg|300px|thumb|right|Keep calm & Keep on fiddling]]
== Narratives ==
All of us, without exception, believe a mix of truth and misinformation, and often enough, disinformation. We strive to understand the world as it is, and not how it looks only according to our preconceptions, which are shaped by a multitude of forces, embedded as we are in our cultural matrix. Sometimes, the most unlikely seeming explanation turns out to be the correct one. In a warrior mindset we consider alternative views, but question everything.
* [[News & Watchdogs]]
Asking for details from ones own cultural matrix is not enough. Not even close. One needs to eat the local food, drink the water, breathe the air, listen to the stories, smell the earth, feel the bark of the trees, see local peoples, and experience local ways to catch what can only be described as a mere glimpse of the answer 42, the knowledge hidden in the trees and wheels in that specific locality on this beautiful planet.

People and communities use stories to understand the world and our place in it. These stories are embedded with power - the power to explain and justify the status quo as well as the power to make change imaginable and urgent.
* [[Timeline merchants of death]]
* [[Timeline masters of the internet]]
* [[Timeline that is soooo 1984 ...]]
A narrative analysis of power encourages us to ask: Which stories define cultural norms? Where did these stories come from? Whose stories were ignored or erased to create these norms? And, most urgently, what new stories can we tell to help create the world we desire? <ref>Harnessing the power of narrative for social change https://www.newtactics.org/conversation/change-story-harnessing-power-narrative-social-change</ref><ref>Underminers Chapter Ten – Reclaiming Ourselves http://underminers.org/the-book/chapter-10/</ref>
== Mindmaps and mindsets ==
If a mindmap is a cognitive "hathanger" then a mindset are the clothes hanging on the hathanger. Most traditions have mindmaps that have been and are evolving locally from the experiences of the previous generations on what worked and what didn't for them. Afaik, the oldest mindmaps use trees and wheels. And all mindmaps are generalisations in the wind without grounding details if not from there. The adversary grouping of the petty tyrants map is useful for gathering and interpreting data from narratives, questioning and timelines into a framework of possible threats.

* [[Petty tyrants]]
* [[Captivating capital and copyfighting]]
* [[Encrypting everything]]
* [[Arcane mixing techniques]]
* [[Networking concepts]]
* [[A typical linux distribution]]
* [[Memory matters]]
* [[Traffic analysis]]

== Methodologies, processes and choreographies ==
Most processes are adapted to allow for minimalist approaches and to minimalise the risk of unintentionally becoming the petty tyrants we fight <ref>Portal 2 Soundtrack Cara Mia (Turret Opera) One Hour https://www.youtube.com/watch?v=wFKfkfCeFj4</ref>.

* [[Shortest retrospective ever]]
* [[Threat modeling the quick and dirty way]] (food for gut, looking backward)
* [[Scenario planning]] (food for thought, looking forward)
* [[Linux development process]]

== Edge ==

* [[Autonomous collectives]]
* [[Controlled folly]]
* [[Confusing surveillance systems]]
* [[Simulations]]
** [[Simulation: Game of Goose]]
** [[Simulation: Mixnets]]
* [[Linux virtualisation]]

== Getting started with linux ==
''Standing on a hilltop in a thunderstorm on bare feet, wearing wet copper armour, holding a lightning rod and shouting ... bring it on, ye gods and godesses!''

We use lawful techniques and tools. And if need be we invent new (GNU and creative commons licensed) techniques and tools <ref>Portal - 'Still Alive' https://www.youtube.com/watch?v=Y6ljFaKRTrI</ref>.

* [[Installing linux]]
* [[Linux applications]]
* [[Linux security]]
* [[Kinky linux command-line]]
* [[Object encryption on linux]]
* [[Anonymising your traffic with linux]]
* [[Clean up all the things]]
* [[Shell scripting]]

== Teacher inside ==

* [[Self-dox]]
* [[Reconnaissance]]
* [[Network exploitation and monitoring]]
* [[Reverse engineering]]
* [[Roleplay]]
* [[Trolling]]
* [[Elicitation]]
* [[Wordsmithing]]
* [[Take control of the tech]]
* [[Making our own linux images]]

== Autonomy shaping infrastructure ==

* For setting up a home server see the [https://undisconnect.miraheze.org/wiki/Main_Page Servers Everywhere section on UnDisConnect]
* For setting up a server see the documentation in the [http://anarchaserver.org/mediawiki/index.php/Anarcha_section Anarcha section on Alexandria]
* [[Linux server security]]
* Radio
* Libraries
* Meshnets
* Clusternets

== Solidarity network ==

* [[Threats, detection, protection and (counter) moves]]
* [[Autonomy patterns]]
* [[Communication patterns]]
* Mirroring servers
* Load balancing servers
* Reframing and refactoring operational accounts

== References ==
<references />.

User:Lilith2

2015-09-29T22:03:36Z

Lilith2: /* Autonomy shaping infrastructure */

[[File:fiddling.jpg|300px|thumb|right|Keep calm & Keep on fiddling]]
== Narratives ==
All of us, without exception, believe a mix of truth and misinformation, and often enough, disinformation. We strive to understand the world as it is, and not how it looks only according to our preconceptions, which are shaped by a multitude of forces, embedded as we are in our cultural matrix. Sometimes, the most unlikely seeming explanation turns out to be the correct one. In a warrior mindset we consider alternative views, but question everything.
* [[News & Watchdogs]]
Asking for details from ones own cultural matrix is not enough. Not even close. One needs to eat the local food, drink the water, breathe the air, listen to the stories, smell the earth, feel the bark of the trees, see local peoples, and experience local ways to catch what can only be described as a mere glimpse of the answer 42, the knowledge hidden in the trees and wheels in that specific locality on this beautiful planet.

People and communities use stories to understand the world and our place in it. These stories are embedded with power - the power to explain and justify the status quo as well as the power to make change imaginable and urgent.
* [[Timeline merchants of death]]
* [[Timeline masters of the internet]]
* [[Timeline that is soooo 1984 ...]]
A narrative analysis of power encourages us to ask: Which stories define cultural norms? Where did these stories come from? Whose stories were ignored or erased to create these norms? And, most urgently, what new stories can we tell to help create the world we desire? <ref>Harnessing the power of narrative for social change https://www.newtactics.org/conversation/change-story-harnessing-power-narrative-social-change</ref><ref>Underminers Chapter Ten – Reclaiming Ourselves http://underminers.org/the-book/chapter-10/</ref>
== Mindmaps and mindsets ==
If a mindmap is a cognitive "hathanger" then a mindset are the clothes hanging on the hathanger. Most traditions have mindmaps that have been and are evolving locally from the experiences of the previous generations on what worked and what didn't for them. Afaik, the oldest mindmaps use trees and wheels. And all mindmaps are generalisations in the wind without grounding details if not from there. The adversary grouping of the petty tyrants map is useful for gathering and interpreting data from narratives, questioning and timelines into a framework of possible threats.

* [[Petty tyrants]]
* [[Captivating capital and copyfighting]]
* [[Encrypting everything]]
* [[Arcane mixing techniques]]
* [[Networking concepts]]
* [[A typical linux distribution]]
* [[Memory matters]]
* [[Traffic analysis]]

== Methodologies, processes and choreographies ==
Most processes are adapted to allow for minimalist approaches and to minimalise the risk of unintentionally becoming the petty tyrants we fight <ref>Portal 2 Soundtrack Cara Mia (Turret Opera) One Hour https://www.youtube.com/watch?v=wFKfkfCeFj4</ref>.

* [[Shortest retrospective ever]]
* [[Threat modeling the quick and dirty way]] (food for gut, looking backward)
* [[Scenario planning]] (food for thought, looking forward)
* [[Linux development process]]

== Edge ==

* [[Autonomous collectives]]
* [[Controlled folly]]
* [[Confusing surveillance systems]]
* [[Simulations]]
** [[Simulation: Game of Goose]]
** [[Simulation: Mixnets]]
* [[Linux virtualisation]]

== Getting started with linux ==
''Standing on a hilltop in a thunderstorm on bare feet, wearing wet copper armour, holding a lightning rod and shouting ... bring it on, ye gods and godesses!''

We use lawful techniques and tools. And if need be we invent new (GNU and creative commons licensed) techniques and tools <ref>Portal - 'Still Alive' https://www.youtube.com/watch?v=Y6ljFaKRTrI</ref>.

* [[Installing linux]]
* [[Linux applications]]
* [[Linux security]]
* [[Kinky linux command-line]]
* [[Object encryption on linux]]
* [[Anonymising your traffic with linux]]
* [[Clean up all the things]]
* [[Shell scripting]]

== Teacher inside ==

* [[Self-dox]]
* [[Reconnaissance]]
* [[Network exploitation and monitoring]]
* [[Reverse engineering]]
* [[Roleplay]]
* [[Trolling]]
* [[Elicitation]]
* [[Wordsmithing]]
* [[Take control of the tech]]
* [[Making our own linux images]]

== Autonomy shaping infrastructure ==

* For setting up a home server see the https://undisconnect.miraheze.org/wiki/Main_Page Servers Everywhere section on UnDisConnect]
* For setting up a server see the documentation in the [http://anarchaserver.org/mediawiki/index.php/Anarcha_section Anarcha section on Alexandria]
* [[Linux server security]]
* Radio
* Libraries
* Meshnets
* Clusternets

== Solidarity network ==

* [[Threats, detection, protection and (counter) moves]]
* [[Autonomy patterns]]
* [[Communication patterns]]
* Mirroring servers
* Load balancing servers
* Reframing and refactoring operational accounts

== References ==
<references />.

Arcane mixing techniques

2015-09-29T21:55:17Z

Lilith2: /* Projects */

== Digital techniques basics ==
For conceptual basics from the atomic level up see ''Semiconductors, Diodes and Transistors'' https://www.youtube.com/watch?v=wPHG0DCWcC0 AND ''An Introduction to Logic Gates'' https://www.youtube.com/watch?t=10&v=95kv5BF2Z9E.

=== Digital gates ===
==== AND ====
The <code>AND</code> gate behaves in the same way as the logical ''and'' operator: Output is ''true'' when both inputs are ''true'' and any otherwise ''false'':
[[File:And.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 1
|}

==== OR ====

The <code>OR</code> gate behaves like a logical inclusive ''or'': Output is ''true'' if either or both of the inputs are ''true'' and ''false'' if both inputs are ''false'':

[[File:Or.png|140px|thumb|left]]

{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 1
|}
==== XOR ====

The <code>XOR</code> (e'''x'''clusive-'''OR''') gate acts as a logical ''either/or'': the output is ''true'' if the inputs are different, and ''false'' if the inputs are the same:

[[File:Xor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 0
|}

==== NOT ====

A logical inverter, alias <code>NOT</code> gate, has only one input and reverses logic state:

[[File:Not.png|140px|thumb|left]]

{| class="wikitable"
|-
! Input 1 !! Output
|-
| 0 || 1
|-
| 1 || 0
|}
 

==== NAND ====
A <code>NAND</code> gate works like an <code>AND</code> gate followed by a <code>NOT</code> gate:
[[File:Nand.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 0
|}

==== NOR ====

The <code>NOR</code> gate is a combination <code>OR</code> gate followed by an inverter:
[[File:Nor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 0
|}

==== XNOR ====
<code>XNOR</code> (e'''x'''clusive-'''NOR''') gate is a combination of an <code>XOR</code> gate followed by an inverter:
[[File:Xnor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 1
|}

=== Universal gates ===

The <code>NAND</code> and the <code>NOR</code> gate can be considered to be "universal" because you can make any of the basic operations out of its combinations: an inverter, an <code>OR</code> gate or an <code>AND</code> gate. Non-inverting gates do not have this versatility since they can not make an invert.

=== Labeling diagrams ===

Integrated circuits have many individual circuit components in a single package. For example, the 7400 chip has 4 NAND gates. See resources for datasheets.

In the below circuit an <code>XOR</code> gate is created with NAND, NOR and NOT gates. Three chips are used: the 7400, 7402 and 7404. All inverters are on one chip. Both NAND's are on the same chip too. The chips are IC1, IC2 and IC3 in the diagram, the order in which they will be placed on the breadboard. Using gates located on one chip is also why the pins are numbered as they are (no need to look up datasheets while wiring).

These diagrams were made with SmartSim.

[[File:Labeling.png|800px|thumb|center]]

Simulated:

[[File:Xor00.png|700px|thumb|center]]

[[File:Xor01.png|700px|thumb|center]]

[[File:Xor11.png|700px|thumb|center]]

== Wiring notes ==
* Create (or have at hand) a circuit diagram before starting any wiring. Sketchy is okay.
* Use the longer outer rows for +5V on one side and ground on the other side.
* First wire power to the circuit using a common colour (red) for +5V and another (black) for ground.
* Power off while wiring circuits on the board.
* Maintain a clean household. In general designs tend to get complex and can become difficult to debug. The gate circuits are not but serpent definitely qualifies and good practice can not start soon enough (see Morgan's laws).
* Strip insulation off wires no more than necessary to prevent wires from accidentally touching each other.
* Do not push wires too far into holes to prevent causing open circuits or short circuits.

== Building gate circuits ==

The resistors are sized to insure that the inverter circuit operates in digital mode.

For building the gate circuits, get:
* 2 pn2222 or 2 2n2222 transistors
* 1 ''1 KΩ'' resistor
* 2 ''33 kΩ'' resistor
* 2 LEDs
* 5V power

=== Inverter circuit ===

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
|--------------- LED
|
/
--------- |/
SW 1 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
GND ---------------------------+---------------------------->></pre>

=== NAND circuit ===
If another transistor is added in series with the transistor in the inverter circuit, a <code>NAND</code> gate is created.

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
|--------------- LED
|
/
--------- |/
SW 1 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
/
--------- |/
SW 2 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
GND ---------------------------+---------------------------->></pre>
=== NOR circuit ===

If another transistor is added in parallel with the transistor in the inverter circuit a <code>NOR</code> gate is the result. IOW, a <code>NOR</code> looks like two <code>NOT</code>s sharing a single pull-up transistor of 1 KΩ.

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
| +------------ LED
___|__|_
/ \
--------- |/ pn2222 \| ---------
SW 1 -----| 33 KΩ |-----| |-----| 33 KΩ |----- SW 2
--------- |\ pn2222 /| ---------
\_________/
|
GND ---------------------------+---------------------------->></pre>

== Boolean algebra ==

=== In plain english ===

If "today is Monday'" is ''true'' then "Not(today is Monday)" is ''false''. This logical expression translates into English as "today is Not Monday" and this makes it easier to see that it is ''false'' if today is indeed Monday. Explanations like that rapidly become convoluted and difficult to follow and this is part of the power of Boolean logic. You can write down arguments in symbolic form. To simplify the handling of complex binary connectives, the mathematician George Boole developed Boolean Algebra, using ordinary algebraic notations AND, OR and NOT. Alternatives that you may encounter are ^ for AND, v for OR, and either over-score or prefix for NOT.

And ... The Boolean <code>AND</code> is the same as the English use of the term, but the Boolean <code>OR</code> is a little different: If you are asked "coffee OR tea?", in most western households you are not expected to say "yes" to both, while in the Boolean case <code>OR</code> includes both. The Boolean operator that corresponds to the English use of the term "OR" is the <code>XOR</code>.

The above tables next to the digital gates are called ''truth tables''.
=== Burglary in progress ===
Suppose you want to build a security system which only works at night and responds to a window or the front door being opened. If you have light sensors you can use these for giving off a signal indicating the truth of the statement:

{| class="wikitable"
|-
! Front door open !! Window open !! Burglary in progress
|-
| False || False || False
|-
| False || True || True
|-
| True || False || True
|-
| True || True || True
|}

We can make the propositions as complex as we want:

<pre>(Burglary in progress) = (NOT(It is daytime)) AND ((Window open) OR (Front door open))

---------
Front door open -----> | |
| OR |-----
Window open -----> | | | ---------
--------- +-----> | |
| AND |-----> Burglary in progress
--------- +-----> | |
| | | ---------
It is day time -----> | NOT |-----
| |
---------
</pre>

=== Precedence and maintaining order ===

Boolean operations are carried out in a well defined order or "precedence":

{| class="wikitable"
|-
! Operator !! Symbol !! Precedence
|-
| NOT || ' || Highest
|-
| AND || . || Middle
|-
| OR || + || Lowest
|}

Expressions inside brackets are always evaluated first, overriding the default precedence order. The Boolean equation of the ''Burglary in progress'' block diagram in fully bracketed form:
<pre>B = ((T'). ((W) + (F)))</pre>
Using the precedence rules this can be simplified to:

<pre>B = T'. (W + F)</pre>

=== Manipulating boolean expressions ===

==== Unary NOT ====

And there are rules to manipulate Boolean expressions. The most simple are the rules for the unary operator NOT:

<pre>(A')' = A
A' . A = 0
A' + A = 1</pre>

==== General rules for AND and OR ====
For more see Group theory Boolean Algebra http://mathworld.wolfram.com/BooleanAlgebra.html

Associative:
<pre>(A . B) .C = A . (B . C)
(A + B) + C = A + (B + C)</pre>
Commutative:
<pre>A . B = B . A
A + B = B + A</pre>
Distributive:
<pre>A . (B + C) = A . B + A . C
A + (B . C) = (A + B) . (A + C)</pre>

==== Simplification rules ====

The first group of simplification rules uses one variable:

<pre>A . A = A
A + A = A</pre>

The second group uses Boolean constants:

<pre>A . 0 = 0
A . 1 = A
A + 0 = A
A + 1 = 1</pre>

==== Example ====
=== Minimisation with Morgan's laws ===

== K-map groupings ==

== Don't care conditions ==

== Debugging notes ==

* If the errors show patterns of consistency, it's somewhere in the logic.
* If inconsistent, it's probably a loose connection.
* Test each component separately.
* Don't kill friends.
* Check power and GND for all appropriate pins.
* It can be frustrating. Don't kill family members.
* Check pin connections if done as specified in (sketchy) circuit diagram and that they aren't tied together.
* It can be very frustrating. Don't kill any acquaintances either.
* Check minimisations and pinouts.
* On rare occasions it is not me/you/us or what I/you/we did, and it is one of the components (a bent and curled pin, faulty chip, split wire (inside the insulation), faulty test equipment). If that turns out to be the case, don't kill component producer associated people. Or anybody else for that matter.
== Projects ==
Notes on these projects are hosted on the [https://undisconnect.miraheze.org/wiki/Arcane_mixing_techniques UnDisConnect wiki]:
* A [https://undisconnect.miraheze.org/wiki/Greenhouse_sprinkler_system greenhouse sprinkler system]? Sure. Definitely not needs an arduino. That would be a tremendous overkill of resources.
* A [https://undisconnect.miraheze.org/wiki/4-bit_processor 4-bit processor]? Why not? If only to understand processor concepts and applied principles better.

== Resources ==
=== Simulators ===

* SmartSim http://smartsim.org.uk/downloads/manual/smartsim_user_manual.pdf (contains installation and getting started instructions. Because of glibc, I installed the stand-alone version)

=== Breadboards ===
[[File:Cable-bus.png|300px|thumb|right]]

You can buy solderless breadboards in a shop or online. Mind that you pick scalable like the SD-12 so you can hook up breadboards together and they become reuseful for other projects such as prototyping our own embedded water management and short range radio (repeater) systems.

Alternatively, start gathering and collecting IDC ribbon connectors (Floppy drive cables, old HD cables, SCSI cables) to build your own breadboards with. To do this, you will also need a solder iron, hot glue, wire (for example telephone wire will do), a small screw driver, and a clamp. This board will also be scalable. Fine grained scalable. :D

I'll do both. Buy a few, and make a few.

=== Power supply ===

=== Switches for input ===

=== Light emitting diodes (LEDs) ===

=== Transistors ===
Transistors are three pin devices. The amount of current that can flow between the collector and emitter is a function of the current flowing through the base of the transistor. If no current is flowing through the base of the transistor, no current will flow through the collector and emitter.
* Transistors http://hyperphysics.phy-astr.gsu.edu/hbase/solids/trans.html#c1

With the transistor operating in digital mode, it will be configured to carry the maximum (on) or minimum (off) current from the collector to the emitter that the circuit will allow.

=== Resistors ===

=== Capacitors ===

=== Chips ===

==== TTL Logic ====

TTL ('''t'''ransistor-'''t'''ransistor-'''l'''ogic) chips were developed with use of transistor switches for logical operations and defines the binary values as:
* 0 V to 0.8 V = logic 0
* 2 V to 5 V = logic 1

The 7400 series logic chips have been manufactured since the 1960s and were used to design and build computers but they are no longer used for that. They still have many uses in teaching digital logic. They are easy to obtain and fairly inexpensive. They draw a lot of power and must be supplied with +5 volts. Individual gates may draw 3 to 4 mA. The low power Schottky versions of TTL chips draw only 20% of the power, but are more expensive. These chips have LS in the middle of their model number.
* 7400 (Quad 2-Input NAND gate) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nand.html#c3
* 7402 (NOR, haven't found it yet) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nor.html#c3
* 7404 (Triple voltage regulator) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/buffer.html#c5
* 7410 (Triple 3-Input NAND gate) http://pdf1.alldatasheet.com/datasheet-pdf/view/7823/NSC/7410.html
* 7420 (Dual 4-Input NAND Gate) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nand3.html#c1
* 74151 (Data Selectors/Multiplexers) http://pdf1.alldatasheet.com/datasheet-pdf/view/7831/NSC/74151.html
* 74175 (Hex/Quad D Flip-Flops with Clear) http://pdf1.alldatasheet.com/datasheet-pdf/view/27396/TI/SN74175N.html
* Timer (haven't found one yet) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/t555.html#c1

==== CMOS Logic ====

CMOS ('''c'''omplementary '''m'''etal '''o'''xide '''s'''emiconductor) chips are lower in power requirements (drawing about 1 mA) and operate with a wide range of supply voltages (typically 3 to 18 volts). The CMOS model number will have a C in the middle of it. A bigh drawback is extreme sensitivity to static electricity - they must be carefully protected from static discharges.

==== NMOS and PMOS Logic ====

PMOS and NMOS ('''p'''- and '''n'''-channel '''m'''etal '''o'''xide '''s'''emiconductors) offer the advantage of higher component density than TTL chips. They too are sensitive to damage from electrical discharge.

=== Sensors ===

Sensors can be video, sonar, radar, inductive, magnetic, capacitive, PVDF wire, pneumatic treadle. These use use significant electrical power, so each sensor must be connected to the power distribution network. New types of sensors can use magnetic technology. Clusters of such sensors transmit data to a receiver mounted on an electrical pole up to 300 meters away, which
relays the data to a processing station. The nodes can operate from a small internal lithium battery for at least 10 years and are vulnerable to heat and shock.

== Related ==
* [[Encrypting everything]]

== References ==

Arcane mixing techniques

2015-09-29T21:54:32Z

Lilith2: /* Projects */

== Digital techniques basics ==
For conceptual basics from the atomic level up see ''Semiconductors, Diodes and Transistors'' https://www.youtube.com/watch?v=wPHG0DCWcC0 AND ''An Introduction to Logic Gates'' https://www.youtube.com/watch?t=10&v=95kv5BF2Z9E.

=== Digital gates ===
==== AND ====
The <code>AND</code> gate behaves in the same way as the logical ''and'' operator: Output is ''true'' when both inputs are ''true'' and any otherwise ''false'':
[[File:And.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 1
|}

==== OR ====

The <code>OR</code> gate behaves like a logical inclusive ''or'': Output is ''true'' if either or both of the inputs are ''true'' and ''false'' if both inputs are ''false'':

[[File:Or.png|140px|thumb|left]]

{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 1
|}
==== XOR ====

The <code>XOR</code> (e'''x'''clusive-'''OR''') gate acts as a logical ''either/or'': the output is ''true'' if the inputs are different, and ''false'' if the inputs are the same:

[[File:Xor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 0
|}

==== NOT ====

A logical inverter, alias <code>NOT</code> gate, has only one input and reverses logic state:

[[File:Not.png|140px|thumb|left]]

{| class="wikitable"
|-
! Input 1 !! Output
|-
| 0 || 1
|-
| 1 || 0
|}
 

==== NAND ====
A <code>NAND</code> gate works like an <code>AND</code> gate followed by a <code>NOT</code> gate:
[[File:Nand.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 0
|}

==== NOR ====

The <code>NOR</code> gate is a combination <code>OR</code> gate followed by an inverter:
[[File:Nor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 0
|}

==== XNOR ====
<code>XNOR</code> (e'''x'''clusive-'''NOR''') gate is a combination of an <code>XOR</code> gate followed by an inverter:
[[File:Xnor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 1
|}

=== Universal gates ===

The <code>NAND</code> and the <code>NOR</code> gate can be considered to be "universal" because you can make any of the basic operations out of its combinations: an inverter, an <code>OR</code> gate or an <code>AND</code> gate. Non-inverting gates do not have this versatility since they can not make an invert.

=== Labeling diagrams ===

Integrated circuits have many individual circuit components in a single package. For example, the 7400 chip has 4 NAND gates. See resources for datasheets.

In the below circuit an <code>XOR</code> gate is created with NAND, NOR and NOT gates. Three chips are used: the 7400, 7402 and 7404. All inverters are on one chip. Both NAND's are on the same chip too. The chips are IC1, IC2 and IC3 in the diagram, the order in which they will be placed on the breadboard. Using gates located on one chip is also why the pins are numbered as they are (no need to look up datasheets while wiring).

These diagrams were made with SmartSim.

[[File:Labeling.png|800px|thumb|center]]

Simulated:

[[File:Xor00.png|700px|thumb|center]]

[[File:Xor01.png|700px|thumb|center]]

[[File:Xor11.png|700px|thumb|center]]

== Wiring notes ==
* Create (or have at hand) a circuit diagram before starting any wiring. Sketchy is okay.
* Use the longer outer rows for +5V on one side and ground on the other side.
* First wire power to the circuit using a common colour (red) for +5V and another (black) for ground.
* Power off while wiring circuits on the board.
* Maintain a clean household. In general designs tend to get complex and can become difficult to debug. The gate circuits are not but serpent definitely qualifies and good practice can not start soon enough (see Morgan's laws).
* Strip insulation off wires no more than necessary to prevent wires from accidentally touching each other.
* Do not push wires too far into holes to prevent causing open circuits or short circuits.

== Building gate circuits ==

The resistors are sized to insure that the inverter circuit operates in digital mode.

For building the gate circuits, get:
* 2 pn2222 or 2 2n2222 transistors
* 1 ''1 KΩ'' resistor
* 2 ''33 kΩ'' resistor
* 2 LEDs
* 5V power

=== Inverter circuit ===

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
|--------------- LED
|
/
--------- |/
SW 1 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
GND ---------------------------+---------------------------->></pre>

=== NAND circuit ===
If another transistor is added in series with the transistor in the inverter circuit, a <code>NAND</code> gate is created.

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
|--------------- LED
|
/
--------- |/
SW 1 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
/
--------- |/
SW 2 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
GND ---------------------------+---------------------------->></pre>
=== NOR circuit ===

If another transistor is added in parallel with the transistor in the inverter circuit a <code>NOR</code> gate is the result. IOW, a <code>NOR</code> looks like two <code>NOT</code>s sharing a single pull-up transistor of 1 KΩ.

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
| +------------ LED
___|__|_
/ \
--------- |/ pn2222 \| ---------
SW 1 -----| 33 KΩ |-----| |-----| 33 KΩ |----- SW 2
--------- |\ pn2222 /| ---------
\_________/
|
GND ---------------------------+---------------------------->></pre>

== Boolean algebra ==

=== In plain english ===

If "today is Monday'" is ''true'' then "Not(today is Monday)" is ''false''. This logical expression translates into English as "today is Not Monday" and this makes it easier to see that it is ''false'' if today is indeed Monday. Explanations like that rapidly become convoluted and difficult to follow and this is part of the power of Boolean logic. You can write down arguments in symbolic form. To simplify the handling of complex binary connectives, the mathematician George Boole developed Boolean Algebra, using ordinary algebraic notations AND, OR and NOT. Alternatives that you may encounter are ^ for AND, v for OR, and either over-score or prefix for NOT.

And ... The Boolean <code>AND</code> is the same as the English use of the term, but the Boolean <code>OR</code> is a little different: If you are asked "coffee OR tea?", in most western households you are not expected to say "yes" to both, while in the Boolean case <code>OR</code> includes both. The Boolean operator that corresponds to the English use of the term "OR" is the <code>XOR</code>.

The above tables next to the digital gates are called ''truth tables''.
=== Burglary in progress ===
Suppose you want to build a security system which only works at night and responds to a window or the front door being opened. If you have light sensors you can use these for giving off a signal indicating the truth of the statement:

{| class="wikitable"
|-
! Front door open !! Window open !! Burglary in progress
|-
| False || False || False
|-
| False || True || True
|-
| True || False || True
|-
| True || True || True
|}

We can make the propositions as complex as we want:

<pre>(Burglary in progress) = (NOT(It is daytime)) AND ((Window open) OR (Front door open))

---------
Front door open -----> | |
| OR |-----
Window open -----> | | | ---------
--------- +-----> | |
| AND |-----> Burglary in progress
--------- +-----> | |
| | | ---------
It is day time -----> | NOT |-----
| |
---------
</pre>

=== Precedence and maintaining order ===

Boolean operations are carried out in a well defined order or "precedence":

{| class="wikitable"
|-
! Operator !! Symbol !! Precedence
|-
| NOT || ' || Highest
|-
| AND || . || Middle
|-
| OR || + || Lowest
|}

Expressions inside brackets are always evaluated first, overriding the default precedence order. The Boolean equation of the ''Burglary in progress'' block diagram in fully bracketed form:
<pre>B = ((T'). ((W) + (F)))</pre>
Using the precedence rules this can be simplified to:

<pre>B = T'. (W + F)</pre>

=== Manipulating boolean expressions ===

==== Unary NOT ====

And there are rules to manipulate Boolean expressions. The most simple are the rules for the unary operator NOT:

<pre>(A')' = A
A' . A = 0
A' + A = 1</pre>

==== General rules for AND and OR ====
For more see Group theory Boolean Algebra http://mathworld.wolfram.com/BooleanAlgebra.html

Associative:
<pre>(A . B) .C = A . (B . C)
(A + B) + C = A + (B + C)</pre>
Commutative:
<pre>A . B = B . A
A + B = B + A</pre>
Distributive:
<pre>A . (B + C) = A . B + A . C
A + (B . C) = (A + B) . (A + C)</pre>

==== Simplification rules ====

The first group of simplification rules uses one variable:

<pre>A . A = A
A + A = A</pre>

The second group uses Boolean constants:

<pre>A . 0 = 0
A . 1 = A
A + 0 = A
A + 1 = 1</pre>

==== Example ====
=== Minimisation with Morgan's laws ===

== K-map groupings ==

== Don't care conditions ==

== Debugging notes ==

* If the errors show patterns of consistency, it's somewhere in the logic.
* If inconsistent, it's probably a loose connection.
* Test each component separately.
* Don't kill friends.
* Check power and GND for all appropriate pins.
* It can be frustrating. Don't kill family members.
* Check pin connections if done as specified in (sketchy) circuit diagram and that they aren't tied together.
* It can be very frustrating. Don't kill any acquaintances either.
* Check minimisations and pinouts.
* On rare occasions it is not me/you/us or what I/you/we did, and it is one of the components (a bent and curled pin, faulty chip, split wire (inside the insulation), faulty test equipment). If that turns out to be the case, don't kill component producer associated people. Or anybody else for that matter.
== Projects ==
Notes on these projects are hosted on the [https://undisconnect.miraheze.org/wiki/Arcane_mixing_techniques | UnDisConnect wiki]:
* A [https://undisconnect.miraheze.org/wiki/Greenhouse_sprinkler_system | greenhouse sprinkler system]? Sure. Definitely not needs an arduino. That would be a tremendous overkill of resources.
* A [https://undisconnect.miraheze.org/wiki/4-bit_processor | 4-bit processor]? Why not? If only to understand processor concepts and applied principles better.

== Resources ==
=== Simulators ===

* SmartSim http://smartsim.org.uk/downloads/manual/smartsim_user_manual.pdf (contains installation and getting started instructions. Because of glibc, I installed the stand-alone version)

=== Breadboards ===
[[File:Cable-bus.png|300px|thumb|right]]

You can buy solderless breadboards in a shop or online. Mind that you pick scalable like the SD-12 so you can hook up breadboards together and they become reuseful for other projects such as prototyping our own embedded water management and short range radio (repeater) systems.

Alternatively, start gathering and collecting IDC ribbon connectors (Floppy drive cables, old HD cables, SCSI cables) to build your own breadboards with. To do this, you will also need a solder iron, hot glue, wire (for example telephone wire will do), a small screw driver, and a clamp. This board will also be scalable. Fine grained scalable. :D

I'll do both. Buy a few, and make a few.

=== Power supply ===

=== Switches for input ===

=== Light emitting diodes (LEDs) ===

=== Transistors ===
Transistors are three pin devices. The amount of current that can flow between the collector and emitter is a function of the current flowing through the base of the transistor. If no current is flowing through the base of the transistor, no current will flow through the collector and emitter.
* Transistors http://hyperphysics.phy-astr.gsu.edu/hbase/solids/trans.html#c1

With the transistor operating in digital mode, it will be configured to carry the maximum (on) or minimum (off) current from the collector to the emitter that the circuit will allow.

=== Resistors ===

=== Capacitors ===

=== Chips ===

==== TTL Logic ====

TTL ('''t'''ransistor-'''t'''ransistor-'''l'''ogic) chips were developed with use of transistor switches for logical operations and defines the binary values as:
* 0 V to 0.8 V = logic 0
* 2 V to 5 V = logic 1

The 7400 series logic chips have been manufactured since the 1960s and were used to design and build computers but they are no longer used for that. They still have many uses in teaching digital logic. They are easy to obtain and fairly inexpensive. They draw a lot of power and must be supplied with +5 volts. Individual gates may draw 3 to 4 mA. The low power Schottky versions of TTL chips draw only 20% of the power, but are more expensive. These chips have LS in the middle of their model number.
* 7400 (Quad 2-Input NAND gate) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nand.html#c3
* 7402 (NOR, haven't found it yet) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nor.html#c3
* 7404 (Triple voltage regulator) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/buffer.html#c5
* 7410 (Triple 3-Input NAND gate) http://pdf1.alldatasheet.com/datasheet-pdf/view/7823/NSC/7410.html
* 7420 (Dual 4-Input NAND Gate) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nand3.html#c1
* 74151 (Data Selectors/Multiplexers) http://pdf1.alldatasheet.com/datasheet-pdf/view/7831/NSC/74151.html
* 74175 (Hex/Quad D Flip-Flops with Clear) http://pdf1.alldatasheet.com/datasheet-pdf/view/27396/TI/SN74175N.html
* Timer (haven't found one yet) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/t555.html#c1

==== CMOS Logic ====

CMOS ('''c'''omplementary '''m'''etal '''o'''xide '''s'''emiconductor) chips are lower in power requirements (drawing about 1 mA) and operate with a wide range of supply voltages (typically 3 to 18 volts). The CMOS model number will have a C in the middle of it. A bigh drawback is extreme sensitivity to static electricity - they must be carefully protected from static discharges.

==== NMOS and PMOS Logic ====

PMOS and NMOS ('''p'''- and '''n'''-channel '''m'''etal '''o'''xide '''s'''emiconductors) offer the advantage of higher component density than TTL chips. They too are sensitive to damage from electrical discharge.

=== Sensors ===

Sensors can be video, sonar, radar, inductive, magnetic, capacitive, PVDF wire, pneumatic treadle. These use use significant electrical power, so each sensor must be connected to the power distribution network. New types of sensors can use magnetic technology. Clusters of such sensors transmit data to a receiver mounted on an electrical pole up to 300 meters away, which
relays the data to a processing station. The nodes can operate from a small internal lithium battery for at least 10 years and are vulnerable to heat and shock.

== Related ==
* [[Encrypting everything]]

== References ==

Arcane mixing techniques

2015-09-29T21:38:10Z

Lilith2:

== Digital techniques basics ==
For conceptual basics from the atomic level up see ''Semiconductors, Diodes and Transistors'' https://www.youtube.com/watch?v=wPHG0DCWcC0 AND ''An Introduction to Logic Gates'' https://www.youtube.com/watch?t=10&v=95kv5BF2Z9E.

=== Digital gates ===
==== AND ====
The <code>AND</code> gate behaves in the same way as the logical ''and'' operator: Output is ''true'' when both inputs are ''true'' and any otherwise ''false'':
[[File:And.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 1
|}

==== OR ====

The <code>OR</code> gate behaves like a logical inclusive ''or'': Output is ''true'' if either or both of the inputs are ''true'' and ''false'' if both inputs are ''false'':

[[File:Or.png|140px|thumb|left]]

{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 1
|}
==== XOR ====

The <code>XOR</code> (e'''x'''clusive-'''OR''') gate acts as a logical ''either/or'': the output is ''true'' if the inputs are different, and ''false'' if the inputs are the same:

[[File:Xor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 0
|}

==== NOT ====

A logical inverter, alias <code>NOT</code> gate, has only one input and reverses logic state:

[[File:Not.png|140px|thumb|left]]

{| class="wikitable"
|-
! Input 1 !! Output
|-
| 0 || 1
|-
| 1 || 0
|}
 

==== NAND ====
A <code>NAND</code> gate works like an <code>AND</code> gate followed by a <code>NOT</code> gate:
[[File:Nand.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 0
|}

==== NOR ====

The <code>NOR</code> gate is a combination <code>OR</code> gate followed by an inverter:
[[File:Nor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 0
|}

==== XNOR ====
<code>XNOR</code> (e'''x'''clusive-'''NOR''') gate is a combination of an <code>XOR</code> gate followed by an inverter:
[[File:Xnor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 1
|}

=== Universal gates ===

The <code>NAND</code> and the <code>NOR</code> gate can be considered to be "universal" because you can make any of the basic operations out of its combinations: an inverter, an <code>OR</code> gate or an <code>AND</code> gate. Non-inverting gates do not have this versatility since they can not make an invert.

=== Labeling diagrams ===

Integrated circuits have many individual circuit components in a single package. For example, the 7400 chip has 4 NAND gates. See resources for datasheets.

In the below circuit an <code>XOR</code> gate is created with NAND, NOR and NOT gates. Three chips are used: the 7400, 7402 and 7404. All inverters are on one chip. Both NAND's are on the same chip too. The chips are IC1, IC2 and IC3 in the diagram, the order in which they will be placed on the breadboard. Using gates located on one chip is also why the pins are numbered as they are (no need to look up datasheets while wiring).

These diagrams were made with SmartSim.

[[File:Labeling.png|800px|thumb|center]]

Simulated:

[[File:Xor00.png|700px|thumb|center]]

[[File:Xor01.png|700px|thumb|center]]

[[File:Xor11.png|700px|thumb|center]]

== Wiring notes ==
* Create (or have at hand) a circuit diagram before starting any wiring. Sketchy is okay.
* Use the longer outer rows for +5V on one side and ground on the other side.
* First wire power to the circuit using a common colour (red) for +5V and another (black) for ground.
* Power off while wiring circuits on the board.
* Maintain a clean household. In general designs tend to get complex and can become difficult to debug. The gate circuits are not but serpent definitely qualifies and good practice can not start soon enough (see Morgan's laws).
* Strip insulation off wires no more than necessary to prevent wires from accidentally touching each other.
* Do not push wires too far into holes to prevent causing open circuits or short circuits.

== Building gate circuits ==

The resistors are sized to insure that the inverter circuit operates in digital mode.

For building the gate circuits, get:
* 2 pn2222 or 2 2n2222 transistors
* 1 ''1 KΩ'' resistor
* 2 ''33 kΩ'' resistor
* 2 LEDs
* 5V power

=== Inverter circuit ===

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
|--------------- LED
|
/
--------- |/
SW 1 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
GND ---------------------------+---------------------------->></pre>

=== NAND circuit ===
If another transistor is added in series with the transistor in the inverter circuit, a <code>NAND</code> gate is created.

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
|--------------- LED
|
/
--------- |/
SW 1 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
/
--------- |/
SW 2 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
GND ---------------------------+---------------------------->></pre>
=== NOR circuit ===

If another transistor is added in parallel with the transistor in the inverter circuit a <code>NOR</code> gate is the result. IOW, a <code>NOR</code> looks like two <code>NOT</code>s sharing a single pull-up transistor of 1 KΩ.

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
| +------------ LED
___|__|_
/ \
--------- |/ pn2222 \| ---------
SW 1 -----| 33 KΩ |-----| |-----| 33 KΩ |----- SW 2
--------- |\ pn2222 /| ---------
\_________/
|
GND ---------------------------+---------------------------->></pre>

== Boolean algebra ==

=== In plain english ===

If "today is Monday'" is ''true'' then "Not(today is Monday)" is ''false''. This logical expression translates into English as "today is Not Monday" and this makes it easier to see that it is ''false'' if today is indeed Monday. Explanations like that rapidly become convoluted and difficult to follow and this is part of the power of Boolean logic. You can write down arguments in symbolic form. To simplify the handling of complex binary connectives, the mathematician George Boole developed Boolean Algebra, using ordinary algebraic notations AND, OR and NOT. Alternatives that you may encounter are ^ for AND, v for OR, and either over-score or prefix for NOT.

And ... The Boolean <code>AND</code> is the same as the English use of the term, but the Boolean <code>OR</code> is a little different: If you are asked "coffee OR tea?", in most western households you are not expected to say "yes" to both, while in the Boolean case <code>OR</code> includes both. The Boolean operator that corresponds to the English use of the term "OR" is the <code>XOR</code>.

The above tables next to the digital gates are called ''truth tables''.
=== Burglary in progress ===
Suppose you want to build a security system which only works at night and responds to a window or the front door being opened. If you have light sensors you can use these for giving off a signal indicating the truth of the statement:

{| class="wikitable"
|-
! Front door open !! Window open !! Burglary in progress
|-
| False || False || False
|-
| False || True || True
|-
| True || False || True
|-
| True || True || True
|}

We can make the propositions as complex as we want:

<pre>(Burglary in progress) = (NOT(It is daytime)) AND ((Window open) OR (Front door open))

---------
Front door open -----> | |
| OR |-----
Window open -----> | | | ---------
--------- +-----> | |
| AND |-----> Burglary in progress
--------- +-----> | |
| | | ---------
It is day time -----> | NOT |-----
| |
---------
</pre>

=== Precedence and maintaining order ===

Boolean operations are carried out in a well defined order or "precedence":

{| class="wikitable"
|-
! Operator !! Symbol !! Precedence
|-
| NOT || ' || Highest
|-
| AND || . || Middle
|-
| OR || + || Lowest
|}

Expressions inside brackets are always evaluated first, overriding the default precedence order. The Boolean equation of the ''Burglary in progress'' block diagram in fully bracketed form:
<pre>B = ((T'). ((W) + (F)))</pre>
Using the precedence rules this can be simplified to:

<pre>B = T'. (W + F)</pre>

=== Manipulating boolean expressions ===

==== Unary NOT ====

And there are rules to manipulate Boolean expressions. The most simple are the rules for the unary operator NOT:

<pre>(A')' = A
A' . A = 0
A' + A = 1</pre>

==== General rules for AND and OR ====
For more see Group theory Boolean Algebra http://mathworld.wolfram.com/BooleanAlgebra.html

Associative:
<pre>(A . B) .C = A . (B . C)
(A + B) + C = A + (B + C)</pre>
Commutative:
<pre>A . B = B . A
A + B = B + A</pre>
Distributive:
<pre>A . (B + C) = A . B + A . C
A + (B . C) = (A + B) . (A + C)</pre>

==== Simplification rules ====

The first group of simplification rules uses one variable:

<pre>A . A = A
A + A = A</pre>

The second group uses Boolean constants:

<pre>A . 0 = 0
A . 1 = A
A + 0 = A
A + 1 = 1</pre>

==== Example ====
=== Minimisation with Morgan's laws ===

== K-map groupings ==

== Don't care conditions ==

== Debugging notes ==

* If the errors show patterns of consistency, it's somewhere in the logic.
* If inconsistent, it's probably a loose connection.
* Test each component separately.
* Don't kill friends.
* Check power and GND for all appropriate pins.
* It can be frustrating. Don't kill family members.
* Check pin connections if done as specified in (sketchy) circuit diagram and that they aren't tied together.
* It can be very frustrating. Don't kill any acquaintances either.
* Check minimisations and pinouts.
* On rare occasions it is not me/you/us or what I/you/we did, and it is one of the components (a bent and curled pin, faulty chip, split wire (inside the insulation), faulty test equipment). If that turns out to be the case, don't kill component producer associated people. Or anybody else for that matter.
== Projects ==
Notes on these projects are hosted on the [https://undisconnect.miraheze.org/wiki/Arcane_mixing_techniques|UnDisConnect wiki]:
* A [https://undisconnect.miraheze.org/wiki/Greenhouse_sprinkler_system| greenhouse sprinkler system]? Sure. Definitely not needs an arduino. That would be a tremendous overkill of resources.
* A [https://undisconnect.miraheze.org/wiki/4-bit_processor| 4-bit processor]? Why not? If only to understand processor concepts and applied principles better.

== Resources ==
=== Simulators ===

* SmartSim http://smartsim.org.uk/downloads/manual/smartsim_user_manual.pdf (contains installation and getting started instructions. Because of glibc, I installed the stand-alone version)

=== Breadboards ===
[[File:Cable-bus.png|300px|thumb|right]]

You can buy solderless breadboards in a shop or online. Mind that you pick scalable like the SD-12 so you can hook up breadboards together and they become reuseful for other projects such as prototyping our own embedded water management and short range radio (repeater) systems.

Alternatively, start gathering and collecting IDC ribbon connectors (Floppy drive cables, old HD cables, SCSI cables) to build your own breadboards with. To do this, you will also need a solder iron, hot glue, wire (for example telephone wire will do), a small screw driver, and a clamp. This board will also be scalable. Fine grained scalable. :D

I'll do both. Buy a few, and make a few.

=== Power supply ===

=== Switches for input ===

=== Light emitting diodes (LEDs) ===

=== Transistors ===
Transistors are three pin devices. The amount of current that can flow between the collector and emitter is a function of the current flowing through the base of the transistor. If no current is flowing through the base of the transistor, no current will flow through the collector and emitter.
* Transistors http://hyperphysics.phy-astr.gsu.edu/hbase/solids/trans.html#c1

With the transistor operating in digital mode, it will be configured to carry the maximum (on) or minimum (off) current from the collector to the emitter that the circuit will allow.

=== Resistors ===

=== Capacitors ===

=== Chips ===

==== TTL Logic ====

TTL ('''t'''ransistor-'''t'''ransistor-'''l'''ogic) chips were developed with use of transistor switches for logical operations and defines the binary values as:
* 0 V to 0.8 V = logic 0
* 2 V to 5 V = logic 1

The 7400 series logic chips have been manufactured since the 1960s and were used to design and build computers but they are no longer used for that. They still have many uses in teaching digital logic. They are easy to obtain and fairly inexpensive. They draw a lot of power and must be supplied with +5 volts. Individual gates may draw 3 to 4 mA. The low power Schottky versions of TTL chips draw only 20% of the power, but are more expensive. These chips have LS in the middle of their model number.
* 7400 (Quad 2-Input NAND gate) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nand.html#c3
* 7402 (NOR, haven't found it yet) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nor.html#c3
* 7404 (Triple voltage regulator) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/buffer.html#c5
* 7410 (Triple 3-Input NAND gate) http://pdf1.alldatasheet.com/datasheet-pdf/view/7823/NSC/7410.html
* 7420 (Dual 4-Input NAND Gate) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nand3.html#c1
* 74151 (Data Selectors/Multiplexers) http://pdf1.alldatasheet.com/datasheet-pdf/view/7831/NSC/74151.html
* 74175 (Hex/Quad D Flip-Flops with Clear) http://pdf1.alldatasheet.com/datasheet-pdf/view/27396/TI/SN74175N.html
* Timer (haven't found one yet) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/t555.html#c1

==== CMOS Logic ====

CMOS ('''c'''omplementary '''m'''etal '''o'''xide '''s'''emiconductor) chips are lower in power requirements (drawing about 1 mA) and operate with a wide range of supply voltages (typically 3 to 18 volts). The CMOS model number will have a C in the middle of it. A bigh drawback is extreme sensitivity to static electricity - they must be carefully protected from static discharges.

==== NMOS and PMOS Logic ====

PMOS and NMOS ('''p'''- and '''n'''-channel '''m'''etal '''o'''xide '''s'''emiconductors) offer the advantage of higher component density than TTL chips. They too are sensitive to damage from electrical discharge.

=== Sensors ===

Sensors can be video, sonar, radar, inductive, magnetic, capacitive, PVDF wire, pneumatic treadle. These use use significant electrical power, so each sensor must be connected to the power distribution network. New types of sensors can use magnetic technology. Clusters of such sensors transmit data to a receiver mounted on an electrical pole up to 300 meters away, which
relays the data to a processing station. The nodes can operate from a small internal lithium battery for at least 10 years and are vulnerable to heat and shock.

== Related ==
* [[Encrypting everything]]

== References ==

Arcane mixing techniques

2015-09-29T20:50:59Z

Lilith2:

== Digital techniques basics ==
For conceptual basics from the atomic level up see ''Semiconductors, Diodes and Transistors'' https://www.youtube.com/watch?v=wPHG0DCWcC0 AND ''An Introduction to Logic Gates'' https://www.youtube.com/watch?t=10&v=95kv5BF2Z9E.

=== Digital gates ===
==== AND ====
The <code>AND</code> gate behaves in the same way as the logical ''and'' operator: Output is ''true'' when both inputs are ''true'' and any otherwise ''false'':
[[File:And.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 1
|}

==== OR ====

The <code>OR</code> gate behaves like a logical inclusive ''or'': Output is ''true'' if either or both of the inputs are ''true'' and ''false'' if both inputs are ''false'':

[[File:Or.png|140px|thumb|left]]

{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 1
|}
==== XOR ====

The <code>XOR</code> (e'''x'''clusive-'''OR''') gate acts as a logical ''either/or'': the output is ''true'' if the inputs are different, and ''false'' if the inputs are the same:

[[File:Xor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 0
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 0
|}

==== NOT ====

A logical inverter, alias <code>NOT</code> gate, has only one input and reverses logic state:

[[File:Not.png|140px|thumb|left]]

{| class="wikitable"
|-
! Input 1 !! Output
|-
| 0 || 1
|-
| 1 || 0
|}
 

==== NAND ====
A <code>NAND</code> gate works like an <code>AND</code> gate followed by a <code>NOT</code> gate:
[[File:Nand.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 1
|-
| 1 || 0 || 1
|-
| 1 || 1 || 0
|}

==== NOR ====

The <code>NOR</code> gate is a combination <code>OR</code> gate followed by an inverter:
[[File:Nor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 0
|}

==== XNOR ====
<code>XNOR</code> (e'''x'''clusive-'''NOR''') gate is a combination of an <code>XOR</code> gate followed by an inverter:
[[File:Xnor.png|140px|thumb|left]]
{| class="wikitable"
|-
! Input 1 !! Input 2 !! Output
|-
| 0 || 0 || 1
|-
| 0 || 1 || 0
|-
| 1 || 0 || 0
|-
| 1 || 1 || 1
|}

=== Universal gates ===

The <code>NAND</code> and the <code>NOR</code> gate can be considered to be "universal" because you can make any of the basic operations out of its combinations: an inverter, an <code>OR</code> gate or an <code>AND</code> gate. Non-inverting gates do not have this versatility since they can not make an invert.

=== Labeling diagrams ===

Integrated circuits have many individual circuit components in a single package. For example, the 7400 chip has 4 NAND gates. See resources for datasheets.

In the below circuit an <code>XOR</code> gate is created with NAND, NOR and NOT gates. Three chips are used: the 7400, 7402 and 7404. All inverters are on one chip. Both NAND's are on the same chip too. The chips are IC1, IC2 and IC3 in the diagram, the order in which they will be placed on the breadboard. Using gates located on one chip is also why the pins are numbered as they are (no need to look up datasheets while wiring).

These diagrams were made with SmartSim.

[[File:Labeling.png|800px|thumb|center]]

Simulated:

[[File:Xor00.png|700px|thumb|center]]

[[File:Xor01.png|700px|thumb|center]]

[[File:Xor11.png|700px|thumb|center]]

== Wiring notes ==
* Create (or have at hand) a circuit diagram before starting any wiring. Sketchy is okay.
* Use the longer outer rows for +5V on one side and ground on the other side.
* First wire power to the circuit using a common colour (red) for +5V and another (black) for ground.
* Power off while wiring circuits on the board.
* Maintain a clean household. In general designs tend to get complex and can become difficult to debug. The gate circuits are not but serpent definitely qualifies and good practice can not start soon enough (see Morgan's laws).
* Strip insulation off wires no more than necessary to prevent wires from accidentally touching each other.
* Do not push wires too far into holes to prevent causing open circuits or short circuits.

== Building gate circuits ==

The resistors are sized to insure that the inverter circuit operates in digital mode.

For building the gate circuits, get:
* 2 pn2222 or 2 2n2222 transistors
* 1 ''1 KΩ'' resistor
* 2 ''33 kΩ'' resistor
* 2 LEDs
* 5V power

=== Inverter circuit ===

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
|--------------- LED
|
/
--------- |/
SW 1 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
GND ---------------------------+---------------------------->></pre>

=== NAND circuit ===
If another transistor is added in series with the transistor in the inverter circuit, a <code>NAND</code> gate is created.

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
|--------------- LED
|
/
--------- |/
SW 1 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
/
--------- |/
SW 2 ----------| 33 KΩ |-----| pn2222
--------- |\
\
|
GND ---------------------------+---------------------------->></pre>
=== NOR circuit ===

If another transistor is added in parallel with the transistor in the inverter circuit a <code>NOR</code> gate is the result. IOW, a <code>NOR</code> looks like two <code>NOT</code>s sharing a single pull-up transistor of 1 KΩ.

<pre>
+ 5V ---------------------------+---------------------------->>
|
--------
| 1 KΩ |
--------
| +------------ LED
___|__|_
/ \
--------- |/ pn2222 \| ---------
SW 1 -----| 33 KΩ |-----| |-----| 33 KΩ |----- SW 2
--------- |\ pn2222 /| ---------
\_________/
|
GND ---------------------------+---------------------------->></pre>

== Boolean algebra ==

=== In plain english ===

If "today is Monday'" is ''true'' then "Not(today is Monday)" is ''false''. This logical expression translates into English as "today is Not Monday" and this makes it easier to see that it is ''false'' if today is indeed Monday. Explanations like that rapidly become convoluted and difficult to follow and this is part of the power of Boolean logic. You can write down arguments in symbolic form. To simplify the handling of complex binary connectives, the mathematician George Boole developed Boolean Algebra, using ordinary algebraic notations AND, OR and NOT. Alternatives that you may encounter are ^ for AND, v for OR, and either over-score or prefix for NOT.

And ... The Boolean <code>AND</code> is the same as the English use of the term, but the Boolean <code>OR</code> is a little different: If you are asked "coffee OR tea?", in most western households you are not expected to say "yes" to both, while in the Boolean case <code>OR</code> includes both. The Boolean operator that corresponds to the English use of the term "OR" is the <code>XOR</code>.

The above tables next to the digital gates are called ''truth tables''.
=== Burglary in progress ===
Suppose you want to build a security system which only works at night and responds to a window or the front door being opened. If you have light sensors you can use these for giving off a signal indicating the truth of the statement:

{| class="wikitable"
|-
! Front door open !! Window open !! Burglary in progress
|-
| False || False || False
|-
| False || True || True
|-
| True || False || True
|-
| True || True || True
|}

We can make the propositions as complex as we want:

<pre>(Burglary in progress) = (NOT(It is daytime)) AND ((Window open) OR (Front door open))

---------
Front door open -----> | |
| OR |-----
Window open -----> | | | ---------
--------- +-----> | |
| AND |-----> Burglary in progress
--------- +-----> | |
| | | ---------
It is day time -----> | NOT |-----
| |
---------
</pre>

=== Precedence and maintaining order ===

Boolean operations are carried out in a well defined order or "precedence":

{| class="wikitable"
|-
! Operator !! Symbol !! Precedence
|-
| NOT || ' || Highest
|-
| AND || . || Middle
|-
| OR || + || Lowest
|}

Expressions inside brackets are always evaluated first, overriding the default precedence order. The Boolean equation of the ''Burglary in progress'' block diagram in fully bracketed form:
<pre>B = ((T'). ((W) + (F)))</pre>
Using the precedence rules this can be simplified to:

<pre>B = T'. (W + F)</pre>

=== Manipulating boolean expressions ===

==== Unary NOT ====

And there are rules to manipulate Boolean expressions. The most simple are the rules for the unary operator NOT:

<pre>(A')' = A
A' . A = 0
A' + A = 1</pre>

==== General rules for AND and OR ====
For more see Group theory Boolean Algebra http://mathworld.wolfram.com/BooleanAlgebra.html

Associative:
<pre>(A . B) .C = A . (B . C)
(A + B) + C = A + (B + C)</pre>
Commutative:
<pre>A . B = B . A
A + B = B + A</pre>
Distributive:
<pre>A . (B + C) = A . B + A . C
A + (B . C) = (A + B) . (A + C)</pre>

==== Simplification rules ====

The first group of simplification rules uses one variable:

<pre>A . A = A
A + A = A</pre>

The second group uses Boolean constants:

<pre>A . 0 = 0
A . 1 = A
A + 0 = A
A + 1 = 1</pre>

==== Example ====
=== Minimisation with Morgan's laws ===

== K-map groupings ==

== Don't care conditions ==

== Debugging notes ==

* If the errors show patterns of consistency, it's somewhere in the logic.
* If inconsistent, it's probably a loose connection.
* Test each component separately.
* Don't kill friends.
* Check power and GND for all appropriate pins.
* It can be frustrating. Don't kill family members.
* Check pin connections if done as specified in (sketchy) circuit diagram and that they aren't tied together.
* It can be very frustrating. Don't kill any acquaintances either.
* Check minimisations and pinouts.
* On rare occasions it is not me/you/us or what I/you/we did, and it is one of the components (a bent and curled pin, faulty chip, split wire (inside the insulation), faulty test equipment). If that turns out to be the case, don't kill component producer associated people. Or anybody else for that matter.
== Projects ==
Notes on these projects are hosted on the [[https://undisconnect.miraheze.org/wiki/Arcane_mixing_techniques|UnDisConnect wiki]]:
* A [[https://undisconnect.miraheze.org/wiki/Greenhouse_sprinkler_system| greenhouse sprinkler system]]? Sure. Definitely not needs an arduino. That would be a tremendous overkill of resources.
* A [[https://undisconnect.miraheze.org/wiki/4-bit_processor| 4-bit processor]]? Why not? If only to understand processor concepts and applied principles better.

== Resources ==
=== Simulators ===

* SmartSim http://smartsim.org.uk/downloads/manual/smartsim_user_manual.pdf (contains installation and getting started instructions. Because of glibc, I installed the stand-alone version)

=== Breadboards ===
[[File:Cable-bus.png|300px|thumb|right]]

You can buy solderless breadboards in a shop or online. Mind that you pick scalable like the SD-12 so you can hook up breadboards together and they become reuseful for other projects such as prototyping our own embedded water management and short range radio (repeater) systems.

Alternatively, start gathering and collecting IDC ribbon connectors (Floppy drive cables, old HD cables, SCSI cables) to build your own breadboards with. To do this, you will also need a solder iron, hot glue, wire (for example telephone wire will do), a small screw driver, and a clamp. This board will also be scalable. Fine grained scalable. :D

I'll do both. Buy a few, and make a few.

=== Power supply ===

=== Switches for input ===

=== Light emitting diodes (LEDs) ===

=== Transistors ===
Transistors are three pin devices. The amount of current that can flow between the collector and emitter is a function of the current flowing through the base of the transistor. If no current is flowing through the base of the transistor, no current will flow through the collector and emitter.
* Transistors http://hyperphysics.phy-astr.gsu.edu/hbase/solids/trans.html#c1

With the transistor operating in digital mode, it will be configured to carry the maximum (on) or minimum (off) current from the collector to the emitter that the circuit will allow.

=== Resistors ===

=== Capacitors ===

=== Chips ===

==== TTL Logic ====

TTL ('''t'''ransistor-'''t'''ransistor-'''l'''ogic) chips were developed with use of transistor switches for logical operations and defines the binary values as:
* 0 V to 0.8 V = logic 0
* 2 V to 5 V = logic 1

The 7400 series logic chips have been manufactured since the 1960s and were used to design and build computers but they are no longer used for that. They still have many uses in teaching digital logic. They are easy to obtain and fairly inexpensive. They draw a lot of power and must be supplied with +5 volts. Individual gates may draw 3 to 4 mA. The low power Schottky versions of TTL chips draw only 20% of the power, but are more expensive. These chips have LS in the middle of their model number.
* 7400 (Quad 2-Input NAND gate) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nand.html#c3
* 7402 (NOR, haven't found it yet) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nor.html#c3
* 7404 (Triple voltage regulator) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/buffer.html#c5
* 7410 (Triple 3-Input NAND gate) http://pdf1.alldatasheet.com/datasheet-pdf/view/7823/NSC/7410.html
* 7420 (Dual 4-Input NAND Gate) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/nand3.html#c1
* 74151 (Data Selectors/Multiplexers) http://pdf1.alldatasheet.com/datasheet-pdf/view/7831/NSC/74151.html
* 74175 (Hex/Quad D Flip-Flops with Clear) http://pdf1.alldatasheet.com/datasheet-pdf/view/27396/TI/SN74175N.html
* Timer (haven't found one yet) http://hyperphysics.phy-astr.gsu.edu/hbase/electronic/t555.html#c1

==== CMOS Logic ====

CMOS ('''c'''omplementary '''m'''etal '''o'''xide '''s'''emiconductor) chips are lower in power requirements (drawing about 1 mA) and operate with a wide range of supply voltages (typically 3 to 18 volts). The CMOS model number will have a C in the middle of it. A bigh drawback is extreme sensitivity to static electricity - they must be carefully protected from static discharges.

==== NMOS and PMOS Logic ====

PMOS and NMOS ('''p'''- and '''n'''-channel '''m'''etal '''o'''xide '''s'''emiconductors) offer the advantage of higher component density than TTL chips. They too are sensitive to damage from electrical discharge.

=== Sensors ===

Sensors can be video, sonar, radar, inductive, magnetic, capacitive, PVDF wire, pneumatic treadle. These use use significant electrical power, so each sensor must be connected to the power distribution network. New types of sensors can use magnetic technology. Clusters of such sensors transmit data to a receiver mounted on an electrical pole up to 300 meters away, which
relays the data to a processing station. The nodes can operate from a small internal lithium battery for at least 10 years and are vulnerable to heat and shock.

== Related ==
* [[Encrypting everything]]

== References ==

Confusing surveillance systems

2015-09-28T21:56:54Z

Lilith2: /* Logical alternatives */

Simple [[surveillance]] systems can be easily attracted, confused and distracted from our brothers and sisters by blogging and spreading nonsensical stories using keywords that trigger these systems on social notworks. And more "sophisticated" surveillance systems are not so easily fooled. Data collection is one thing, being able to get the exact piece of information that answers questions an other thing. For datamining socalled Bayesian systems are used.

== Bayesian vulnerabilities ==

* Bayesian probabilistic explication of inductive inference conflates neutrality of supporting evidence for some hypothesis H (“not supporting H”) with disfavoring evidence (“supporting not-H”). This expressive inadequacy leads to spurious results that are artifacts of a poor choice of inductive logic. <ref>Cosmic Confusions: Not Supporting versus Supporting Not- http://philsci-archive.pitt.edu/9114/ Philosophy of Science. 77 (2010), pp. 501-23., this manuscript is an extensively revised version of "Cosmology and Inductive Inference: A Bayesian Failure," http://philsci-archive.pitt.edu/4866/ Prepared for “Philosophy of Cosmology: Characterising Science and Beyond” St. Anne’s College, Oxford, September 20-22, 2009</ref>
* Any theory of inductive inference depends upon one or more principles or presumptions that distinguish the right inductive inference relations. These principles must be there, whether they are made explicit, or, as is the more usual case, left tacit. The most fundamental of the challenges to Bayesian confirmation theory come from differing views of these principles <ref>Challenges to Bayesian Confirmation Theory http://www.pitt.edu/%7Ejdnorton/papers/Challenges_final.pdf Prepared for Prasanta S. Bandyopadhyay and Malcolm Forster (eds.), Philosophy of Statistics: Vol. 7 Handbook of the Philosophy of Science. Elsevier. Download final.</ref>
* While there is no universal logic of induction, the probability calculus succeeds as a logic of induction in many contexts through its use of several notions concerning inductive inference. They include Addition, through which low probabilities represent disbelief as opposed to ignorance; And Bayes property, which commits the calculus to a 'refute and rescale' dynamics for incorporating new evidence. These notions are independent and it is urged that they be employed selectively according to needs of the problem at hand. It is shown that neither is adapted to inductive inference concerning some in deterministic systems. <ref>Probability Disassembled http://www.pitt.edu/~jdnorton/papers/Prob_diss.pdf British Journal for the Philosophy of Science, 58 (2007), pp. 141-171</ref>
* In a material theory of induction, inductive inferences are warranted by facts that prevail locally. This approach, it is urged, is preferable to formal theories of induction in which the good inductive inferences are delineated as those conforming to universal schemas. An inductive inference problem concerning indeterministic, nonprobabilistic systems in physics is posed, and it is argued that Bayesians cannot responsibly analyze it, thereby demonstrating that the probability calculus is not the universal logic of induction. <ref>There are No Universal Rules for Induction http://www.jstor.org/stable/10.1086/656542?origin=JSTOR-pdf Philosophy of Science, 77 (2010) pp. 765-77</ref>
* In one ideal, a logic of induction would provide us with a belief state representing total ignorance that would evolve towards different belief states as new evidence is learned. That the Bayesian system cannot be such a logic follows from well-known, elementary considerations. In familiar paradoxes to be discussed here, the notion that indifference over outcomes requires equality of probability rapidly leads to contradictions. If our initial ignorance is sufficiently great, there are so many ways to be indifferent that the resulting equalities contradict the additivity of the probability calculus. We can properly assign equal probabilities in a prior probability distribution only if our ignorance is not complete and we know enough to be able to identify which is the right partition of the outcome space over which to exercise indifference. While a zero value can denote ignorance in alternative systems such as that of Shafer-Dempster, representing ignorance by zero probability fails in more than one way. Additivity precludes ignorance on all outcomes, since the sum of probabilities over a partition must be unity; and the dynamics of Bayesian conditionalization makes it impossible to recover from ignorance. <ref>Ignorance and Indifference http://www.pitt.edu/~jdnorton/papers/Ignor_Indiff.pdf Philosophy of Science, 75 (2008), pp. 45-68. </ref>

In short, the greatest vulnerabilities of such systems are 1) its inability to separate ignorance from disbelief and 2) assuming the approach provides a universal logic of induction and *** theirs is IT *** of course. Heh!

== Countermoves ==

Exploiting Bayesian vulnerabilities for confusing surveillance systems can be done by

* Createing fragments of inductive logic that represent ignorance (evidential neutrality)
* Using other, yet competing systems of inductive logic
* Using inductive logic of indeterministic systems for which the probability calculus fails
* Using inductive inferences that are warranted by facts that prevail locally

==Resources==
===Logical alternatives===
* The World of Null A http://conceptualfiction.com/world_of_null_a.html
* Wikipedia Category:Non-classical logic https://en.wikipedia.org/wiki/Category:Non-classical_logic

==Related==
* [[Surveillance]]

== References ==

Confusing surveillance systems

2015-09-28T21:56:25Z

Lilith2: /* Related */

Simple [[surveillance]] systems can be easily attracted, confused and distracted from our brothers and sisters by blogging and spreading nonsensical stories using keywords that trigger these systems on social notworks. And more "sophisticated" surveillance systems are not so easily fooled. Data collection is one thing, being able to get the exact piece of information that answers questions an other thing. For datamining socalled Bayesian systems are used.

== Bayesian vulnerabilities ==

* Bayesian probabilistic explication of inductive inference conflates neutrality of supporting evidence for some hypothesis H (“not supporting H”) with disfavoring evidence (“supporting not-H”). This expressive inadequacy leads to spurious results that are artifacts of a poor choice of inductive logic. <ref>Cosmic Confusions: Not Supporting versus Supporting Not- http://philsci-archive.pitt.edu/9114/ Philosophy of Science. 77 (2010), pp. 501-23., this manuscript is an extensively revised version of "Cosmology and Inductive Inference: A Bayesian Failure," http://philsci-archive.pitt.edu/4866/ Prepared for “Philosophy of Cosmology: Characterising Science and Beyond” St. Anne’s College, Oxford, September 20-22, 2009</ref>
* Any theory of inductive inference depends upon one or more principles or presumptions that distinguish the right inductive inference relations. These principles must be there, whether they are made explicit, or, as is the more usual case, left tacit. The most fundamental of the challenges to Bayesian confirmation theory come from differing views of these principles <ref>Challenges to Bayesian Confirmation Theory http://www.pitt.edu/%7Ejdnorton/papers/Challenges_final.pdf Prepared for Prasanta S. Bandyopadhyay and Malcolm Forster (eds.), Philosophy of Statistics: Vol. 7 Handbook of the Philosophy of Science. Elsevier. Download final.</ref>
* While there is no universal logic of induction, the probability calculus succeeds as a logic of induction in many contexts through its use of several notions concerning inductive inference. They include Addition, through which low probabilities represent disbelief as opposed to ignorance; And Bayes property, which commits the calculus to a 'refute and rescale' dynamics for incorporating new evidence. These notions are independent and it is urged that they be employed selectively according to needs of the problem at hand. It is shown that neither is adapted to inductive inference concerning some in deterministic systems. <ref>Probability Disassembled http://www.pitt.edu/~jdnorton/papers/Prob_diss.pdf British Journal for the Philosophy of Science, 58 (2007), pp. 141-171</ref>
* In a material theory of induction, inductive inferences are warranted by facts that prevail locally. This approach, it is urged, is preferable to formal theories of induction in which the good inductive inferences are delineated as those conforming to universal schemas. An inductive inference problem concerning indeterministic, nonprobabilistic systems in physics is posed, and it is argued that Bayesians cannot responsibly analyze it, thereby demonstrating that the probability calculus is not the universal logic of induction. <ref>There are No Universal Rules for Induction http://www.jstor.org/stable/10.1086/656542?origin=JSTOR-pdf Philosophy of Science, 77 (2010) pp. 765-77</ref>
* In one ideal, a logic of induction would provide us with a belief state representing total ignorance that would evolve towards different belief states as new evidence is learned. That the Bayesian system cannot be such a logic follows from well-known, elementary considerations. In familiar paradoxes to be discussed here, the notion that indifference over outcomes requires equality of probability rapidly leads to contradictions. If our initial ignorance is sufficiently great, there are so many ways to be indifferent that the resulting equalities contradict the additivity of the probability calculus. We can properly assign equal probabilities in a prior probability distribution only if our ignorance is not complete and we know enough to be able to identify which is the right partition of the outcome space over which to exercise indifference. While a zero value can denote ignorance in alternative systems such as that of Shafer-Dempster, representing ignorance by zero probability fails in more than one way. Additivity precludes ignorance on all outcomes, since the sum of probabilities over a partition must be unity; and the dynamics of Bayesian conditionalization makes it impossible to recover from ignorance. <ref>Ignorance and Indifference http://www.pitt.edu/~jdnorton/papers/Ignor_Indiff.pdf Philosophy of Science, 75 (2008), pp. 45-68. </ref>

In short, the greatest vulnerabilities of such systems are 1) its inability to separate ignorance from disbelief and 2) assuming the approach provides a universal logic of induction and *** theirs is IT *** of course. Heh!

== Countermoves ==

Exploiting Bayesian vulnerabilities for confusing surveillance systems can be done by

* Createing fragments of inductive logic that represent ignorance (evidential neutrality)
* Using other, yet competing systems of inductive logic
* Using inductive logic of indeterministic systems for which the probability calculus fails
* Using inductive inferences that are warranted by facts that prevail locally

==Resources==
===Logical alternatives===
* The World of Null A http://conceptualfiction.com/world_of_null_a.html
* Lukasiewicz on the Principle of Contradiction http://web.archive.org/web/20071010110817/http://www.fmag.unict.it/~polphil/PolPhil/Lukas/RaspaLukas.html
* Wikipedia Category:Non-classical logic https://en.wikipedia.org/wiki/Category:Non-classical_logic
==Related==
* [[Surveillance]]

== References ==

Installing linux

2015-09-27T23:58:55Z

Lilith2:

<languages/>
<translate>

M$ wants to keep you locked in to Windows so that it can take your money, your personal data, and your user freedom. They don’t want you to know that you have a choice of better operating systems; operating systems that respect your freedom. There are tons of free "as in freedom" software operating systems that you can download and install at no cost.


And when they’re improved, you can choose whether or not you want to upgrade, without a corporation breathing down your neck. It ''is'' time to upgrade your computer, but not to Windows 8.


[[File:Ohgurl.jpg|470px|thumb|right]]

== Shopping for a linux distro == 
There are many distributions. Ubuntu and mint are considered the easiest for new users who want to get productive in linux as soon as possible without having to master all of its complexities immediately. And more consideration(s) may apply.

=== Myths? === 

==== Linux is hard to install ==== 
Non-sense. New users can get started pretty quickly with USB or CD/DVD systems that allow for trying out a particular linux first. Installing linux is pretty easy these days, but getting everything to work exactly the way you like it can take a little more work.


The install itself is nothing more than a download, making an install medium, some mouse clicks and deciding of basic options like timezone, language, user name, passwords. Even though all are of laughable simplicity in their use, each Linux installation has its own setup utility, different from all the others, or nearly the same like in the case of mint and ubuntu.


This page contains installation guides for [[#Mint|mint]], [[#Ubuntu|ubuntu]] and [[#Debian|debian]] for you to make informed decisions with. This is as good and as bad as it gets.

==== Linux does not recognize my hardware ==== 
This can happen if your machine is younger than 6 months. Reason?


In M$ worlds, firmware is usually a part of the driver you install, not seen by the user. In linux, firmware may be distributed from a number of sources. Some firmware comes from the Linux kernel sources. Others that have redistribution licenses come from upstream. Some firmware unfortunately does not have licenses allowing free redistribution. See [[#Getting a device to work|getting a device to work]].

==== Linux is too complicated to use ==== 
Ehh. Myth. It has a graphical interface with a mouse and windows. It has an auto-updater to keep things current. It has software add/remove capabilities that allow you to install programs with a couple of mouse clicks. It even has many apps found in windows and mac OS X like firefox, thunderbird, pidgin, audacity, …

==== There are no applications available ==== 
Non-sense. If you do not have a "stripped linux", after install you have all you need for usual use (if you pick the right distro for you) and if it isn't there already, thousands of additional applications available for free: Nearly every distro has a centralized location where you can search for, add, or remove software (using a point and click interface or the command line). With [[Linux_applications#Installing_applications|package management systems]], you can open up one tool, search for an application (or group of applications), and install without having to do any web searching (or purchasing).


It is true that windows used to be the better platform for games, but recently Steam has arrived in linux. And some windows games can be run on linux using wine (but not all). The game you wish to play may actually be available for linux now.

=== Reasons for using linux as an activist === 

==== Security ==== 
I don’t think any operating system is ever "secure". A particular configuration of an operating system has a particular degree of resistance to particular types of attacks. Linux and M$ products are not easy to compare but some general statements are regularly made:
* Windows is more difficult to "strip down" than Linux systems.
* Viruses are less of a threat on linux. This also applies to spyware, malware, etc.
And some theories posed:
* Theory 1: Because linux is not owned by a corporation, it can explore the security goal more fully than a corporation can. Businesses must make money; while free software groups simply don’t have this restriction. Facts and Fallacies of Software Engineering (2003) <ref>Facts and Fallacies of Software Engineering (2003) http://books.google.com/books?id=3Ntz-UJzZN0C&pg=PA174#v=onepage&f=false</ref> and When Free Software Isn’t Better (2010) <ref>When Free Software Isn't Better http://mako.cc/writing/hill-when_free_software_isnt_better.html
</ref> kindly dispelled this theory.
* Theory 2: Because the linux kernel code, and many of its drivers and utilities are free, it has likely been reviewed and fixed frequently for coding mistakes that can lead to remote vulnerabilities that a hacker can exploit. This then prevents backdoors.
The shellshock vulnerability will still be found in unpatched systems for the foreseeable future—though the odds of it directly impacting our local machines appear slim if using [[Linux security|standard security precautions]]. If scary or seemingly too complex get a local geek to help you with that or hit the respective forums.

''We are well into the 21st century, but it is astonishing how people can still believe that Linux-based operating systems are completely secure. Indeed, "Linux" and "security" are two words that you rarely see together.''<ref>Don't believe these four myths about linux security https://blogs.sophos.com/2015/03/26/dont-believe-these-four-myths-about-linux-security/</ref>

==== Update ability ==== 
Linux is in a constant state of development and improvement by developers donating their time and skills to the various projects in return for their own enjoyment and learning. The [[Captivating capital and copyfighting|licensing]] allows anyone to add features they need. The usual route for that is: Request feature; not satisfied (takes too long or is rejected?); Fork and go <ref>Don't panic and keep forking Debian™! :^) https://devuan.org/newsletter_22dec.html</ref>.

==== Support ==== 
Support for linux and [[linux applications]] is amazing. You can have an issue with something, check the documentation and the forums, and if nothing found, send out an e-mail to a mailing list or post on a forum, and within 10 minutes be flooded with suggestions. Or it can take hours or days. Or may never come. Still, generally speaking, most problems with Linux have been encountered and are documented. Chances are you’ll find solutions fairly quickly.

==== Self-improvement ====

With Linux you have a room where floor and ceiling can be raised or lowered as high or low as you want them. With Windows, neither move. You can’t go further than M$ has deemed it necessary for you to go. If not satisfied with going through life with blinders on and living the technical "M$ status-quo", linux is an excellent choice. Linux provides an environment for [[Kinky linux command-line|learning new skills and gaining deeper insights into how computers work]]. It also provides a platform to [[Shell_scripting|develop more scripts and tools]]. It’s ways and communities tick most "autonomy" aspects.

==== Low cost ==== 
The cost advantage of linux is huge, especially in resource constrained environments. You get the complete OS, thousands of applications AND support for the grand total price of … <drumroll> … zilch! And we’re not talking about the crumbs that fell off a (big) business plate, not a watered down whiskey, not a feature deprived OS … ye get a full-blown, complete, freakishly configurable and flexible intuitive ready OS, for free. And it runs on old computers.


Linux expands the life of your hardware by reducing overhead and needing less system resources, which is why it runs fine on older machines. And you can do the tests yourself of course. <ref>Phoronix Test Suite http://phoronix-test-suite.com/</ref><ref>Phoronix: Operating Systems http://www.phoronix.com/scan.php?page=category&item=Operating%20Systems</ref>

== You can have both == 
You can have both windows and linux on your machine, each installed on a separate disk or partition, and choose which to start when booting. This configuration is called "dual boot", and the debian installation system can set it up during the hard drive partitioning stage of installation and setting up the bootloader. Follow the standard install, but do partitioning first as described in [[#Dual boot|dual boot]].

== First things first == 
The linux installation (.iso) file(s) can be copied directly onto a USB or "burned" in a way that expands/extracts the image so it contains by the target machine usable files CD/DVD. The new major distro's seem to all use isohybrids.

=== Goodbye microsoft === 
Debian offers a windows program that will download and set up a 32 bit debian installer. You then only need to reboot the computer and choose between normal windows boot or booting the installation program. You can find it here http://ftp.debian.org/debian/tools/win32-loader/stable/ and here http://goodbye-microsoft.com/.


It doesn't have to be a definitive goodbye. You can still leave the M$ door open. The installer will still allow you to choose a so-called [[#Dual boot|dual boot]].

=== For installation via USB you need ... === 
To install via USB you need a USB stick and a bios on the target machine that allows you to boot from that USB stick, and a way to put the iso on the USB stick.

==== A BIOS on the target machine that allows you to boot from USB ==== 
Some older PHOENIX BIOS simply do not have an entry for that. In such a case move over to another type of medium for install like CD/DVD.

==== A USB stick ==== 
Minimal size 8G (or smaller if you are doing a net-install).

==== cp and sync ==== 
For some distributions you can use unetbootin. On linux you can install unetbootin from the repositories, on windows you can download it from http://unetbootin.sourceforge.net/ The current stable debian named jessie is not installable via unetbootin https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775689

And with the iso-hybrid images, it is no longer a necessity to use unetbootin. You can copy and sync isohybrids: https://www.debian.org/releases/stable/amd64/ch04s03.html.en#usb-copy-isohybrid.
=== For installation via a CD/DVD you need ... === 
To install from a CD/DVD, you need a working RW drive, CD/DVD burner software and a blank CD/DVD for your recorder.

==== A working Read/Write drive ==== 
Check the CD/DVD drives on the front or inside the machine and look up the labels on the internet to see if it can record. On linux you can use:
$ dmesg | egrep -i --color 'cdrom|dvd|cd/rw|writer'


[[File:Determine-dvd-drive.png|640px||thumb|center]]


In this output sr0 is my CD/DVD reader and writer. You can see more detailed information with:


$ less /proc/sys/dev/cdrom/info


[[File:Determine-dvd-drive2.png|640px||thumb|center]]

==== DVD burner software ==== 
On linux you can use ''brasero''. If you are using the GNOME desktop, ''brasero'' is installed by default. The menu entry is in Applications > Sound & Video. If not there, you will have to install the application:
* Open up your package manager (synaptic, gnome-packagekit, software center). This example continues with synaptic:
* Search for "brasero"
* Mark the brasero entry for installation
* Click Apply (or Install) to start the installation.
==== A blank CD/DVD ====
DVD+R and DVD-R are different and competing standards for recording on DVDs. The DVD+RW alliance is headed by sony, yamaha, philips, dell, and microsoft, and the DVD Forum by apple, pioneer, mitsubishi, hitachi, and time warner.


The difference between DVD-R and DVD+R is ''not important for DVD players'' because virtually all players play both formats; the difference is ''important in DVD '''recorders'''''.


The main difference between DVD R and DVD RW (whether plussed or dashed) is that the R is a write-once format. Data can be burned on the DVD only once. On the other hand, data on an RW disc can be erased and rewritten several times.

== Choose your kernel and download iso(s) == 
Download mint, ubuntu or debian for your preferred medium from http://www.linuxmint.com/download.php, http://www.ubuntu.com/download or https://www.debian.org/releases/jessie/debian-installer/. Oh dear, which one?


If you are confused about whether your kernel is 32 or 64 bits, in linux do:


$ uname -m


''x86_64'' means a 64-bit kernel and ''i686'' means a 32-bit kernel. In windows try this https://support.microsoft.com/en-us/kb/827218


Note: you can install a 32-bit kernel on a 64-bit CPU. The fundamental difference between 32 and 64 bit systems is the size of memory addresses. In theory, a 32 bit system can not work with more than 4 GB of RAM (232 bytes). In practice, it is possible to work around this by using the ''686-pae'' kernel, so long as the processor handles the PAE (Physical Address Extension). There is a performance price to pay for this, so if you are installing on a server with a huge amount of RAM, use the 64 bit kernel.


To learn more about your CPU in linux do:
$ cat /proc/cpuinfo
For windows you can type dxdiag in the run box.

== Verify downloads == 
Verify all dowloaded iso's. You can verify for example, the md5 or sha256 checksum of the first downloaded debian .iso file in linux with:
$ md5sum debian-8.1.0-amd64-DVD-1.iso
respectively:
$ sha256sum debian-8.1.0-amd64-DVD-1.iso
and compare the output with key listed for ''debian-8.1.0-amd64-DVD-1.iso'' in respectively http://cdimage.debian.org/debian-cd/8.1.0/amd64/iso-dvd/MD5SUMS and http://cdimage.debian.org/debian-cd/8.1.0/amd64/iso-dvd/SHA256SUMS Other checksums are also given.


Windows does not come with ''md5sum''. There are command line utilities (<tt>md5sum.exe</tt>) and the version available from Cygwin is probably easiest to install and update. Once installed, Cygwin's md5sum behaves exactly as the md5sum described for linux above.

== Make a bootable medium == 

=== Burn CD/DVD === 
On windows right click on the iso and choose "burn". Then follow whatever follows on M$ for burning iso's.


On linux, start brasero (or other tool you prefer):
[[File:Brasero.png|640px||thumb|center]]
Select the iso and burn, baby, burn:
[[File:Brasero2.png|640px||thumb|center]]

=== Make an image on USB === 
Isohybrids are making that easy! For example:
$ sudo cp debian-8.1.0-amd64-DVD-1.iso /dev/sdb
$ sudo sync

== Boot box from bootable medium == 


Boot from the USB, CD- or DVD-ROM:
* Put boot medium with installer in slot and restart machine.
* At boot, enter the BIOS by pressing F2. Some machines use other keys http://pcsupport.about.com/od/fixtheproblem/ht/accessbios.htm and if you have a Windows 8.1 tablet or device with a UEFI BIOS, then booting to the BIOS no longer works the way it did on classic desktop PCs http://www.makeuseof.com/tag/how-to-access-the-bios-on-a-windows-8-computer/ and once you are in, you have to shut off Secure Boot and Fast Boot (and Quick Boot) in the UEFI or turn UEFI off completely in favour of "Legacy BIOS". If your EFI based PC is not booting debian, here is an easy way to reinstall grub-efi, the bootloader used by debian on these PCs: https://wiki.debian.org/GrubEFIReinstall#Reinstalling_grub-efi_on_your_hard_drive
* Set the BIOS to boot from your installation medium, save, and exit.
Booting will continue and if all went well, your first installer screen appears. If not, https://www.happyassassin.net/2014/01/25/uefi-boot-how-does-that-actually-work-then/ and/or ask us on IRC.

=== Mint with CINNAMON === 
Mint starts up in Live CD. Have a look around. Going for it? Click the Install Linux Mint CD icon:


[[File:Welcome-mint.png|640px||thumb|center]]


[[File:Preparing-to-install-linux-mint.png|640px||thumb|center]]


[[File:Installation-type-mint.png|640px||thumb|center]]


[[File:Installation-type2-mint.png|640px||thumb|center]]


[[File:Where-are-you-mint.png|640px||thumb|center]]


[[File:Keyboard-layout-mint.png|640px||thumb|center]]


[[File:Keyboard-layout2-mint.png|640px||thumb|center]]


[[File:Who-are-you-mint.png|640px||thumb|center]]


[[File:Welcome2-mint.png|640px||thumb|center]]


[[File:Installation-complete-mint.png|640px||thumb|center]]

==== After install ==== 


Do NOT apt-get update and upgrade on the command-line for the updates. Go to the updater in the menu (4 locations down from the software manager). Use the menu.


[[File:Update-and-software-management-in-mint.png|640px||thumb|center]]


[[File:Update-manager.png|640px||thumb|center]]

=== Ubuntu with GNOME === 


[[File:Welcome.png|640px||thumb|center]]


Click install Ubuntu, then do the preparations. If you play a musician role you will probably want the Fluendo codecs installed, for that tick the "Install this third-party software" box.


[[File:Preparing-to-install.png|640px||thumb|center]]


LVM will allow you to later resize partitions. If it's a laptop you are installing on and you regularly take it into public spaces, encrypt the Ubuntu installation.


[[File:Installation-type.png|640px||thumb|center]]


[[File:Installation-type2.png|640px||thumb|center]]


Pick your keyboard, or if you feel uncertain have the installer detect it for you.


[[File:Keyboard-layout.png|640px||thumb|center]]


Make that minimally six. The "after install" things I do require me to enter my password and I am lazy so I initially make it a small one and make it a better one later.


[[File:Who-are-you.png|640px||thumb|center]]


Installation ...


[[File:Welcome-to-ubuntu.png|640px||thumb|center]]


And yes!


[[File:Installation-complete.png|640px||thumb|center]]Remove boot medium, and set BIOS to start from hard disk.

==== After install ==== 


By default, Ubuntu sends some of your information to third parties without asking you to opt-in. Follow these 6 simple steps to fix Ubuntu and restore your privacy https://fixubuntu.com/


If you are not doing a stealth install, and if you have your network card working, to get the upgrades since the image you installed was made, on the command-line do:


$ sudo apt-get update
$ sudo apt-get dist-upgrade

=== Debian with MATE === 


In the Isolinux bootloader menu choose install:


[[File:Install.png|640px||thumb|center]]


Choose language (this sets language for both the installation process as well as for the to-be-installed linux):


[[File:Select-a-language.png|640px||thumb|center]]


Select location. It doesn't have to be where you actually are. And you can change it later. It sets what repositories you use, but even that can be changed later.


[[File:Select-location.png|640px||thumb|center]]


Locales are used by glibc and other "locale-aware" programs or libraries for rendering text, correctly displaying regional monetary values, time and date formats, alphabetic idiosyncrasies, and other locale-specific standards:


[[File:Configure-locales.png|640px||thumb|center]]


The "American English" keyboard corresponds to the QWERTY layout, otherwise pick yours:[[File:Configure-keyboard.png|640px||thumb|center]]Until now the installation steps were completely contained in the boot image included on the CD/DVD, a file of limited size and loaded into memory by the BIOS when booting from the removable medium. What follows are a few automatic windows showing progress in:
* detecting hardware: detects hardware and tries to identify the drive and loads the modules of the various hardware components detected, and then "mounts" the drive to read it
* loading additional components: loads all the files necessary to continue including additional drivers for remaining hardware (especially the network card), as well as all the components of the installation program
* detecting network hardware: tries to identify the network card and load the associated firmware. If automatic detection fails, you can manually select the firmware to load, and you can do it later. If need be you can use a USB-network-adapter. If doing a netinst installation, this must be made to work now, because the debian packages must be loaded from the network.
Next the installer attempts an automatic network configuration by DHCP (for IPv4) and by IPv6 network discovery. If this fails, the installer provides choices: you can try again, attempt DHCP configuration by declaring the name of the machine (rumpelstiltskin jumps to mind), or set up a static network configuration. This requires having an IP address, a subnet mask, an IP address for a potential gateway, a machine name, and a domain name to enter.


If network is successfully configured, pick whatever name for your host.


[[File:Configure-network.png|640px||thumb|center]]Leave domain name empty and continue. [[File:Configure-network2.png|640px||thumb|center]]


Make the root password long (8 characters or more) and hard to guess. All machines connected to the internet are regularly targeted by automated connection attempts with the most obvious passwords and sometimes even with dictionary attacks. Avoid using names of partners, pets, children, parents, friends, dates of birth, etc. Use password generators.


[[File:Set-up-users-and-passwords.png|640px||thumb|center]]


Debian also imposes the creation of a standard user account on you so that you do not get into the bad habit of working as root. The precautionary principle applies.


[[File:Set-up-user.png|640px||thumb|center]]


If the network is available, the system's internal clock is updated (in a one-shot way) from an NTP server. This way the timestamps on logs will be correct from the first boot.


Partitioning is for dividing the available space on the hard drives. This step also includes choosing the filesystems to be used. All of these decisions will have an influence on performance, data security, and the administration of a server. For more on that, check out [[#Manual partitioning|manual partitioning]].


To install next to a windows system, check out [[#Dual boot|dual boot]] before you continue.


For a standard install of debian on a PC as new user, the very easy way is recommended:


[[File:Partition-disk.png|640px||thumb|center]]


[[File:Partition-disk2.png|640px||thumb|center]]


[[File:Partition-disk3.png|640px||thumb|center]]


[[File:Partition-disk4.png|640px||thumb|center]]


[[File:Partition-disk5.png|640px||thumb|center]]


[[File:Partition-disk6.png|640px||thumb|center]]


The next steps do not require any user interaction. The "base system" packages, [[Linux_applications#dpkg|dpkg]] and [[Linux_applications#apt|apt]] for managing packages, and utilities to boot the system, are installed.


In order to be able to install additional software, apt needs to be configured and told where to find debian packages. You are asked if it must use a network source for packages, or if it should only look for packages on the removable media you are installing from.


[[File:Configuring-apt.png|640px||thumb|center]]


[[File:Configuring-popularity-contest.png|640px||thumb|center]]


MATE:


[[File:Software-selection.png|640px||thumb|center]]


The bootloader is the first program started by the BIOS. This program loads the linux kernel into memory and then executes it. It often offers a menu that allows the user to choose the kernel to load and/or the operating system to boot.
[[File:Install-grub.png|640px||thumb|center]]
If you are installing via USB, best not to automatically install GRUB, but to choose No and set it manually. That list of disks and device files when partitioning.


That's it. Reboot.


[[File:Finish-installation.png|640px||thumb|center]]

==== After install ==== 


Add your user to the "sudoers" list (or else keep su-ing for actions that require administrator permissions). To add a user to the sudoers list you can use visudo (and need to su for that first).


[[File:Visudo-command.png|640px||thumb|center]]


Visudo is opened. Scroll down to this line "root ALL=(ALL:ALL) ALL" and add a line directly below for the user. On a PC you can just copy the entire line (it is still somewhat empty here) for root and change "root" to user. On a server you do not give users ALL permissions. Ya read this http://ubuntuforums.org/showthread.php?t=1132821 and start some happy puzzling.


[[File:Visudo.png|640px||thumb|center]]


After you are done, you are returned to the command-line and can sudo.


[[File:Not-a-sudoer.png|640px||thumb|center]]


If you are not doing a stealth install, and if you have your network card working, to get the upgrades since the image you installed was made, on the command-line do:


$ sudo apt-get update
$ sudo apt-get upgrade

== Dual boot with windows == 
A windows OS should be installed first, because its bootloader is very particular and the installer tends to overwrite the entire hard drive, wiping out any data stored on it. If windows isn't already installed, install it first. If you are able to partition the drive prior to installing windows, leave space for linux during the initial partitioning process. Then you won't have to resize your NTFS partition to make room for your linux later, saving a bit of time.


if you install (or reinstall) windows after linux, the bootloader will be erased. Linux will still be on the hard drive, but will no longer be accessible from the boot menu as the windows bootloader does not recognise other operating systems. If this happens, you can restore the bootloader with the steps in [[#F*&^% bootloader and rescue mode|F*&^% bootloader and rescue mode]].


If you wish to keep using the windows bootloader you can try this:
* Turn off Windows Fast Boot http://www.eightforums.com/tutorials/6320-fast-startup-turn-off-windows-8-a.html
* Shrink the main Windows 8.1 partition. Windows expects a certain partition scheme https://technet.microsoft.com/en-us/library/hh824839.aspx so this can be a bit tricky http://forum.linuxbsdos.com/topic/151/how-to-resize-or-shrink-a-gpt-partitioned-c-drive-in-windows-8 Give Debian between 10G and 100 GB of space.
* Shut off Secure Boot and Fast Boot in the UEFI.
* Download your debian iso and put it on some media as described above.
* Insert your install media and install debian in the space you created. At the partitioning step make notes on what partitions are labeled what (/dev/sdx).
* At the end of installing debian, you can set the installer to install GRUB in the EFI System Partition of Windows 8.1.
Alternatively turn UEFI off (use Legacy) and set the installer to put the GRUB bootloader (that will have an entry for windows) on a linux partition.

== Manual partitioning == 
Manual partitioning allows for choosing the purpose and size of each partition. You need this if you wish to use software RAID or install linux alongside an existing operating system (windows or other).


The first screen of ''manual partitioning'' displays available disks, partitions, and any possible free space that has not yet been partitioned. You can select an item and pressing the Enter key gives a list of possible actions. You can erase a partitions on a disk by selecting it. When selecting free space on a disk, you can manually create a new partition. You can also do this with guided partitioning.


The so-called mount point is the directory tree that will house the contents of the file system on the selected partition. A partition mounted at the directory <code>/home/</code> is meant to contain user data. The directory <code>/</code> is the root of the file tree, and therefore the root of the partition that will host the debian system.


[[File:Partition-disk11.png|640px|thumb|center]]


Virtual memory allows the kernel to free a bit of storage by storing the parts of the RAM that have been inactive for some time on the swap partition of a hard disk. To simulate the additional memory, windows uses a swap file that is directly contained in a file system. Linux uses a partition, hence the term "swap" partition, mounted at <code>/swap/</code>.


When choosing a partition, you can:
* format it and include it in the file tree by choosing a mount point
* use it as a swap partition
* make it into a "physical volume for encryption"
* make it a "physical volume for LVM"
* use it as a RAID device
* leave it unchanged

=== Recommendations === 
* Use LVM
* Depending on your threat model, configure encrypted volumes.
* If you plan to install many programs not part of the your distribution create a separate <code>/usr/local</code> partition.
* You can put <code>/tmp</code> on its own partition with 20–100 MB.
* For multi-user systems or systems with lots of disk space you can put <code>/usr</code>, <code>/var</code>,<code>/tmp</code>, and <code>/home</code> each on their own partitions separate from the <code>/</code>partition. For very complex server systems see the Multi Disk HOWTO http://www.tldp.org/HOWTO/Multi-Disk-HOWTO.html
* If you are setting up a server with lots of user accounts, create a separate large<code>/home</code> partition.
* If your machine will be a mail server, you can make <code>/var/mail</code> a separate partition.

=== Configure the Logical Volume Manager === 
You want this. The size of the partitions are no longer limited by individual disks but by their cumulative volume, and you can resize existing partitions at any time, possibly after adding an additional disk. The partitioning tool can do this automatically or you can configure LVM yourself:
* Create the partitions that will be "physical volumes for LVM".
* To activate LVM, choose "Configure the Logical Volume Manager (LVM)", then on the same configuration screen "Create a volume group"”, to which you associate the existing physical volumes.
* Create logical volumes within this volume group.

=== Configure encrypted volumes === 
For confidentiality of your data you can encrypt the data on some partitions. This feature can be added underneath any filesystem, because linux (dm-crypt driver) uses the Device Mapper to create a virtual partition (with protected content) based on an underlying partition that will store the data in an encrypted form. For more on that see LUKS, the Linux Unified Key Setup, a standard format enabling storage of encrypted data and meta-data on the encryption algorithms used.

[[File:Partition-disk7.png|centre|thumb|640x640px]]


[[File:Partition-disk8.png|centre|thumb|640x640px]]When an encrypted partition is used, the encryption key is stored in memory (RAM). Retrieving it allows the decryption of the data, making it important to not leave a copy of this key anywhere that a thief of the computer or hard drive can access. On laptops the contents of RAM is stored on the swap partition during hibernation. When using encrypted partitions, also encrypt the swap partition.[[File:Partition-disk9.png|centre|thumb|640x640px]]The software will then propose to initialise the physical volume with random data (making the localisation of real data more difficult), and will ask you for an "encryption passphrase" (make it hard but not too hard because you will have to enter it every time you boot your computer or access the content of the encrypted partition.


This partition can be used as a physical volume for LVM so as to protect several partitions (LVM logical volumes) with the same encryption key, including the swap partition.

== F*&^% bootloader and rescue mode == 
Sometimes, things go wrong, and the system you've carefully installed is no longer bootable. Perhaps the boot loader configuration broke while trying out a change, or right after installation of debian the system does not boot, etc.

Boot from your installation medium in rescue mode:


[[File:Rescue-mode.png|640px|thumb|center]]


You'll be shown the first few screens of the installer, with a note in the corner of the display to indicate that this is rescue mode, not a full installation. Your system is not about to be overwritten. Rescue mode simply takes advantage of the hardware detection facilities available in the installer to ensure that your disks, network devices, and so on are available to you while repairing your system.


Then, instead of the partitioning tool, you are shown of the partitions on your system, and are asked to select one of them. You can choose partitions on RAID, LVM devices or partitions directly on disk containing the root file system.


The installer will now give you a shell prompt for doing repairs in the file system you selected. If the installer cannot run a shell in its root file system, the file system may be corrupt or one or many of many other things may have happened. Repairing broken systems can be difficult, and if you have problems, consult another geek, or hit the forums and mailing-lists.


For reinstalling the GRUB boot loader into the master boot record of the first hard disk, do:


# grub-install '(hd0)'


After you exit the shell, the system will reboot. Or not. In which case, hit the mailinglist, IRC channel or forums, or all of them if need be with this excellency challenging question.

== Upgrading distributions == 

=== Recommendations for upgrading === 
* If you are new to linux, use the Software Center.
* Use apt-get for big upgrades, for example a dist-upgrade from one stable to the next, because it will find relatively good solutions quickly, while aptitude will likely find several convoluted solutions (or none at all).
* For regular upgrades in unstable or testing versions of a debian-based distribution, use aptitude: ''aptitude safe-upgrade'' can keep temporarily broken packages on hold while still installing new packages and you can tweak operations dynamically.

== Getting a device to work == 
Hardware like a sound card or ethernet card, etc. have two essential pieces of software that make them function. The first is a working driver, software that lets your system talk to the hardware. The second is firmware, usually a small piece of code uploaded directly to the device and maintained by the company that develops the hardware device. You can think of the firmware as a way of programming the hardware inside the device. It is a black box, and because of that treated like hardware; there’s no accompanying source code that is freely distributed with it.


Firmware can come from one of the following sources:
* The linux-image package (containing the linux kernel and licensed firmware, installed by default)
* The linux-firmware package (containing other licensed firmware, installed by default)
* The linux-firmware-nonfree package in multiverse (containing firmware that are missing redistribution licenses, not installed by default)
* A separate driver package (not installed by default)
* Elsewhere (not installed by default, driver CD, email attachment, website)
If during install you get messages as to hardware that could not be provided for, write down what those messages say. It probably looks something like
e100: eth0: e100_request_firmware: Failed to load firmware "e100/d101m_ucode.bin": -2


When you installed the kernel, the post-installation script went through all loaded modules (listed by 'lsmod') and verified whether modules as provided by the newly installed kernel might require firmware files. This information can be retrieved with modinfo, which needs a path to the modules. You either traverse there, or build the path to the modules using:


$ uname -a
Linux debian 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) x86_64 GNU/Linux
Then with modinfo:
$ modinfo -F firmware /lib/modules/3.16.0-4-amd64/kernel/drivers/net/e100.ko
e100/d102e_ucode.bin
e100/d101s_ucode.bin
e100/d101m_ucode.bin


The firmware packages embed the list of firmware files in their description, so now you can use


$ apt-cache search d101m_ucode.bin
firmware-linux-nonfree - Binary firmware for various drivers in the Linux kernel


You can also use 'apt-file' (provided by the package of the same name) or the web interface at packages.debian.org.


$ apt-file search d101m_ucode.bin
firmware-linux-nonfree: /lib/firmware/e100/d101m_ucode.bin


If the above commands return nothing, you probably need to enable the "non-free" repository in your /etc/apt/sources.list first. You can also enable it within synaptic. And you also want to run 'sudo apt-file update' to have the latest information.


Now you can install the right package, in the example above it was firmware-linux-nonfree.

== Installing debian like an expert == 


At the first screen running the installer choose "Advanced Options" and then "expert install". This makes the installation process "verbose" and details all possible options in the process of installation, and allows for navigation between the various steps without them happening automatically in sequence.


[[File:Debian-installer-main-menu.png|640px|thumb|center]]

=== Select multiple locales === 
The "select locales" choice allows for entering multiple locales:


[[File:Configure-locales-expert.png|640px|thumb|center]]


For some often-used locales see https://wiki.debian.org/Locale

=== Detect and mount CDROM === 
First it finds removable drives other than CDROM:


[[File:Detect-and-mount-CDROM.png|640px|thumb|center]]


Next it detects CDROM:


[[File:Detect-and-mount-CDROM2.png|640px|thumb|center]]

=== Load installer components from CD === 


These are additional components that you can use during install:


[[File:Load-installer-components-from-CD.png|640px|thumb|center]]


Now the main menu contains more items to hopscotch through:


[[File:Debian-installer-main-menu2.png|640px|thumb|center]]

=== Configure network === 


[[File:Configure-network3.png|640px|thumb|center]]


[[File:Configure-network4.png|640px|thumb|center]]


... and forward. This requires filling in an IP address, a subnet mask, an IP address for a potential gateway, a machine name, and a domain name to enter.

=== Configure the clock === 
NTP (Network Time Protocol) allows a machine to synchronize with others fairly accurately, taking into consideration the delays induced by the transfer of information over the network and other possible offsets. There are numerous NTP servers on the Internet, and the more popular ones may be overloaded. The pool.ntp.org NTP server is a group of machines that have agreed to serve as public NTP servers. You can limit use to a sub-group specific to a country, with, for example, us.pool.ntp.org for the united states, or ca.pool.ntp.org for canada, or:


[[File:Configure-the-clock.png|640px|thumb|center]]


[[File:Configure-the-clock2.png|640px|thumb|center]]

=== Partition disks === 
This is the same as during a standard install. You can use guided partitioning (see standard install above) or [[#Manual partitioning|manual partitioning]].

=== Change debconf priority === 


When installing (or upgrading) any single package or a group of packages, debconf asks the packages' configuration questions all at once, and stores the user/admin preferences in a database. Later as the packages are installing, their scripts use the configuration preferences in the database to generate configuration files and otherwise do administrative tasks (e.g. set up servers to start or not start, install other software such as libdvdcss, etc.). This saves the hassle of editing configuration files by hand, and also of waiting for each individual package to install before responding to certain configuration questions. Important during a server install.
[[File:Change-debconf-priority.png|640px|thumb|center]]

=== Install the LILO boot loader on a hard disk === 


GRUB is the default bootloader installed by debian: it works with most filesystems and reads its configuration during boot to find the exact position of the new kernel. That means it doesn't require an update after each installation of a new kernel. Version 1 couldn't handle all combinations of LVM and software RAID and version 2 is more complete, but there may still be situations where LILO can save the day.


[[File:Install-the-LILO-bootloader.png|640px|thumb|center]]If you want to see some of the above expert options applied, see this tutorial from 2011: How to install and multi-boot between Windows, and Debian Testing with full disk encryption http://scarygliders.net/2011/12/06/how-to-install-and-multi-boot-between-windows-and-debian-testing-with-full-disk-encryption/
</translate>

== Stealth install ==
* Download and burn the CD/DVD's via a public machine.
* Official CD images do not include non-free firmware. Download the unofficial CD builds containing non-free firmware from http://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/. If using a USB stick, download from http://cdimage.debian.org/cdimage/unofficial/non-free/firmware/ and unpack to either the root directory or a directory named <code>/firmware</code> of the file system on the medium. If the firmware you need is not included in the tarball, you can also download specific firmware packages from the (non-free section of the) archive http://packages.debian.org/search?keywords=firmware. It is also possible to copy individual firmware files to the medium. Loose firmware could be obtained for example from an already installed system or from a hardware vendor.
* For installing packages, see [[Linux_applications#Stealth_install_of_applications|Stealth install of applications]]

== Related ==

* [[Linux applications]]
* [[Linux security]]
* [[Kinky linux command-line]]
* [[Anonymising your traffic with linux]]
* [[Clean up all the things]]
* [[Shell scripting]]
* [[Encrypting everything]]
* [[Linux virtualisation]]
* [[Making our own linux images]]

== References ==

[[Category:How_To]]

Installing linux

2015-09-27T23:56:31Z

Lilith2:

<languages/>
<translate>

M$ wants to keep you locked in to Windows so that it can take your money, your personal data, and your user freedom. They don’t want you to know that you have a choice of better operating systems; operating systems that respect your freedom. There are tons of free "as in freedom" software operating systems that you can download and install at no cost.


And when they’re improved, you can choose whether or not you want to upgrade, without a corporation breathing down your neck. It ''is'' time to upgrade your computer, but not to Windows 8.


[[File:Ohgurl.jpg|470px|thumb|right]]

== Shopping for a linux distro == 
There are many distributions. Ubuntu and mint are considered the easiest for new users who want to get productive in linux as soon as possible without having to master all of its complexities immediately. And more consideration(s) may apply.

=== Myths? === 

==== Linux is hard to install ==== 
Non-sense. New users can get started pretty quickly with USB or CD/DVD systems that allow for trying out a particular linux first. Installing linux is pretty easy these days, but getting everything to work exactly the way you like it can take a little more work.


The install itself is nothing more than a download, making an install medium, some mouse clicks and deciding of basic options like timezone, language, user name, passwords. Even though all are of laughable simplicity in their use, each Linux installation has its own setup utility, different from all the others, or nearly the same like in the case of mint and ubuntu.


This page contains installation guides for [[#Mint|mint]], [[#Ubuntu|ubuntu]] and [[#Debian|debian]] for you to make informed decisions with. This is as good and as bad as it gets.

==== Linux does not recognize my hardware ==== 
This can happen if your machine is younger than 6 months. Reason?


In M$ worlds, firmware is usually a part of the driver you install, not seen by the user. In linux, firmware may be distributed from a number of sources. Some firmware comes from the Linux kernel sources. Others that have redistribution licenses come from upstream. Some firmware unfortunately does not have licenses allowing free redistribution. See [[#Getting a device to work|getting a device to work]].

==== Linux is too complicated to use ==== 
Ehh. Myth. It has a graphical interface with a mouse and windows. It has an auto-updater to keep things current. It has software add/remove capabilities that allow you to install programs with a couple of mouse clicks. It even has many apps found in windows and mac OS X like firefox, thunderbird, pidgin, audacity, …

==== There are no applications available ==== 
Non-sense. If you do not have a "stripped linux", after install you have all you need for usual use (if you pick the right distro for you) and if it isn't there already, thousands of additional applications available for free: Nearly every distro has a centralized location where you can search for, add, or remove software (using a point and click interface or the command line). With [[Linux_applications#Installing_applications|package management systems]], you can open up one tool, search for an application (or group of applications), and install without having to do any web searching (or purchasing).


It is true that windows used to be the better platform for games, but recently Steam has arrived in linux. And some windows games can be run on linux using wine (but not all). The game you wish to play may actually be available for linux now.

=== Reasons for using linux as an activist === 

==== Security ==== 
I don’t think any operating system is ever "secure". A particular configuration of an operating system has a particular degree of resistance to particular types of attacks. Linux and M$ products are not easy to compare but some general statements are regularly made:
* Windows is more difficult to "strip down" than Linux systems.
* Viruses are less of a threat on linux. This also applies to spyware, malware, etc.
And some theories posed:
* Theory 1: Because linux is not owned by a corporation, it can explore the security goal more fully than a corporation can. Businesses must make money; while free software groups simply don’t have this restriction. Facts and Fallacies of Software Engineering (2003) <ref>Facts and Fallacies of Software Engineering (2003) http://books.google.com/books?id=3Ntz-UJzZN0C&pg=PA174#v=onepage&f=false</ref> and When Free Software Isn’t Better (2010) <ref>When Free Software Isn't Better http://mako.cc/writing/hill-when_free_software_isnt_better.html
</ref> kindly dispelled this theory.
* Theory 2: Because the linux kernel code, and many of its drivers and utilities are free, it has likely been reviewed and fixed frequently for coding mistakes that can lead to remote vulnerabilities that a hacker can exploit. This then prevents backdoors.
The shellshock vulnerability will still be found in unpatched systems for the foreseeable future—though the odds of it directly impacting our local machines appear slim if using [[Linux security|standard security precautions]]. If scary or seemingly too complex get a local geek to help you with that or hit the respective forums.

''We are well into the 21st century, but it is astonishing how people can still believe that Linux-based operating systems are completely secure. Indeed, "Linux" and "security" are two words that you rarely see together.''<ref>Don't believe these four myths about linux security https://blogs.sophos.com/2015/03/26/dont-believe-these-four-myths-about-linux-security/</ref>

==== Update ability ==== 
Linux is in a constant state of development and improvement by developers donating their time and skills to the various projects in return for their own enjoyment and learning. The [[Captivating capital and copyfighting|licensing]] allows anyone to add features they need. The usual route for that is: Request feature; not satisfied (takes too long or is rejected?); Fork and go <ref>Don't panic and keep forking Debian™! :^) https://devuan.org/newsletter_22dec.html</ref>.

==== Support ==== 
Support for linux and [[linux applications]] is amazing. You can have an issue with something, check the documentation and the forums, and if nothing found, send out an e-mail to a mailing list or post on a forum, and within 10 minutes be flooded with suggestions. Or it can take hours or days. Or may never come. Still, generally speaking, most problems with Linux have been encountered and are documented. Chances are you’ll find solutions fairly quickly.

==== Self-improvement ====

With Linux you have a room where floor and ceiling can be raised or lowered as high or low as you want them. With Windows, neither move. You can’t go further than M$ has deemed it necessary for you to go. If not satisfied with going through life with blinders on and living the technical "M$ status-quo", linux is an excellent choice. Linux provides an environment for [[Kinky linux command-line|learning new skills and gaining deeper insights into how computers work]]. It also provides a platform to [[Shell_scripting|develop more scripts and tools]]. It’s ways and communities tick most "autonomy" aspects.

==== Low cost ==== 
The cost advantage of linux is huge, especially in resource constrained environments. You get the complete OS, thousands of applications AND support for the grand total price of … <drumroll> … zilch! And we’re not talking about the crumbs that fell off a (big) business plate, not a watered down whiskey, not a feature deprived OS … ye get a full-blown, complete, freakishly configurable and flexible intuitive ready OS, for free. And it runs on old computers.


Linux expands the life of your hardware by reducing overhead and needing less system resources, which is why it runs fine on older machines. And you can do the tests yourself of course. <ref>Phoronix Test Suite http://phoronix-test-suite.com/</ref><ref>Phoronix: Operating Systems http://www.phoronix.com/scan.php?page=category&item=Operating%20Systems</ref>

== You can have both == 
You can have both windows and linux on your machine, each installed on a separate disk or partition, and choose which to start when booting. This configuration is called "dual boot", and the debian installation system can set it up during the hard drive partitioning stage of installation and setting up the bootloader. Follow the standard install, but do partitioning first as described in [[#Dual boot|dual boot]].

== First things first == 
The linux installation (.iso) file(s) can be copied directly onto a USB or "burned" in a way that expands/extracts the image so it contains by the target machine usable files CD/DVD. The new major distro's seem to all use isohybrids.

=== Goodbye microsoft === 
Debian offers a windows program that will download and set up a 32 bit debian installer. You then only need to reboot the computer and choose between normal windows boot or booting the installation program. You can find it here http://ftp.debian.org/debian/tools/win32-loader/stable/ and here http://goodbye-microsoft.com/.


It doesn't have to be a definitive goodbye. You can still leave the M$ door open. The installer will still allow you to choose a so-called [[#Dual boot|dual boot]].

=== For installation via USB you need ... === 
To install via USB you need a USB stick and a bios on the target machine that allows you to boot from that USB stick, and a way to put the iso on the USB stick.

==== A BIOS on the target machine that allows you to boot from USB ==== 
Some older PHOENIX BIOS simply do not have an entry for that. In such a case move over to another type of medium for install like CD/DVD.

==== A USB stick ==== 
Minimal size 8G (or smaller if you are doing a net-install).

==== cp and sync ==== 
For some distributions you can use unetbootin. On linux you can install unetbootin from the repositories, on windows you can download it from http://unetbootin.sourceforge.net/ The current stable debian named jessie is not installable via unetbootin https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775689

And with the iso-hybrid images, it is no longer a necessity to use unetbootin. You can copy and sync isohybrids: https://www.debian.org/releases/stable/amd64/ch04s03.html.en#usb-copy-isohybrid.
=== For installation via a CD/DVD you need ... === 
To install from a CD/DVD, you need a working RW drive, CD/DVD burner software and a blank CD/DVD for your recorder.

==== A working Read/Write drive ==== 
Check the CD/DVD drives on the front or inside the machine and look up the labels on the internet to see if it can record. On linux you can use:
$ dmesg | egrep -i --color 'cdrom|dvd|cd/rw|writer'


[[File:Determine-dvd-drive.png|640px||thumb|center]]


In this output sr0 is my CD/DVD reader and writer. You can see more detailed information with:


$ less /proc/sys/dev/cdrom/info


[[File:Determine-dvd-drive2.png|640px||thumb|center]]

==== DVD burner software ==== 
On linux you can use ''brasero''. If you are using the GNOME desktop, ''brasero'' is installed by default. The menu entry is in Applications > Sound & Video. If not there, you will have to install the application:
* Open up your package manager (synaptic, gnome-packagekit, software center). This example continues with synaptic:
* Search for "brasero"
* Mark the brasero entry for installation
* Click Apply (or Install) to start the installation.
==== A blank CD/DVD ====
DVD+R and DVD-R are different and competing standards for recording on DVDs. The DVD+RW alliance is headed by sony, yamaha, philips, dell, and microsoft, and the DVD Forum by apple, pioneer, mitsubishi, hitachi, and time warner.


The difference between DVD-R and DVD+R is ''not important for DVD players'' because virtually all players play both formats; the difference is ''important in DVD '''recorders'''''.


The main difference between DVD R and DVD RW (whether plussed or dashed) is that the R is a write-once format. Data can be burned on the DVD only once. On the other hand, data on an RW disc can be erased and rewritten several times.

== Choose your kernel and download iso(s) == 
Download mint, ubuntu or debian for your preferred medium from http://www.linuxmint.com/download.php, http://www.ubuntu.com/download or https://www.debian.org/releases/jessie/debian-installer/. Oh dear, which one?


If you are confused about whether your kernel is 32 or 64 bits, in linux do:


$ uname -m


''x86_64'' means a 64-bit kernel and ''i686'' means a 32-bit kernel. In windows try this https://support.microsoft.com/en-us/kb/827218


Note: you can install a 32-bit kernel on a 64-bit CPU. The fundamental difference between 32 and 64 bit systems is the size of memory addresses. In theory, a 32 bit system can not work with more than 4 GB of RAM (232 bytes). In practice, it is possible to work around this by using the ''686-pae'' kernel, so long as the processor handles the PAE (Physical Address Extension). There is a performance price to pay for this, so if you are installing on a server with a huge amount of RAM, use the 64 bit kernel.


To learn more about your CPU in linux do:
$ cat /proc/cpuinfo
For windows you can type dxdiag in the run box.

== Verify downloads == 
Verify all dowloaded iso's. You can verify for example, the md5 or sha256 checksum of the first downloaded debian .iso file in linux with:
$ md5sum debian-8.1.0-amd64-DVD-1.iso
respectively:
$ sha256sum debian-8.1.0-amd64-DVD-1.iso
and compare the output with key listed for ''debian-8.1.0-amd64-DVD-1.iso'' in respectively http://cdimage.debian.org/debian-cd/8.1.0/amd64/iso-dvd/MD5SUMS and http://cdimage.debian.org/debian-cd/8.1.0/amd64/iso-dvd/SHA256SUMS Other checksums are also given.


Windows does not come with ''md5sum''. There are command line utilities (<tt>md5sum.exe</tt>) and the version available from Cygwin is probably easiest to install and update. Once installed, Cygwin's md5sum behaves exactly as the md5sum described for linux above.

== Make a bootable medium == 

=== Burn CD/DVD === 
On windows right click on the iso and choose "burn". Then follow whatever follows on M$ for burning iso's.


On linux, start brasero (or other tool you prefer):
[[File:Brasero.png|640px||thumb|center]]
Select the iso and burn, baby, burn:
[[File:Brasero2.png|640px||thumb|center]]

=== Make an image on USB === 
Isohybrids are making that easy! For example:
$ sudo cp debian-8.1.0-amd64-DVD-1.iso /dev/sdb
$ sudo sync

== Boot box from bootable medium == 


Boot from the USB, CD- or DVD-ROM:
* Put boot medium with installer in slot and restart machine.
* At boot, enter the BIOS by pressing F2. Some machines use other keys http://pcsupport.about.com/od/fixtheproblem/ht/accessbios.htm and if you have a Windows 8.1 tablet or device with a UEFI BIOS, then booting to the BIOS no longer works the way it did on classic desktop PCs http://www.makeuseof.com/tag/how-to-access-the-bios-on-a-windows-8-computer/ and once you are in, you have to shut off Secure Boot and Fast Boot (and Quick Boot) in the UEFI or turn UEFI off completely in favour of "Legacy BIOS". If your EFI based PC is not booting debian, here is an easy way to reinstall grub-efi, the bootloader used by debian on these PCs: https://wiki.debian.org/GrubEFIReinstall#Reinstalling_grub-efi_on_your_hard_drive
* Set the BIOS to boot from your installation medium, save, and exit.
Booting will continue and if all went well, your first installer screen appears.

=== Mint with CINNAMON === 
Mint starts up in Live CD. Have a look around. Going for it? Click the Install Linux Mint CD icon:


[[File:Welcome-mint.png|640px||thumb|center]]


[[File:Preparing-to-install-linux-mint.png|640px||thumb|center]]


[[File:Installation-type-mint.png|640px||thumb|center]]


[[File:Installation-type2-mint.png|640px||thumb|center]]


[[File:Where-are-you-mint.png|640px||thumb|center]]


[[File:Keyboard-layout-mint.png|640px||thumb|center]]


[[File:Keyboard-layout2-mint.png|640px||thumb|center]]


[[File:Who-are-you-mint.png|640px||thumb|center]]


[[File:Welcome2-mint.png|640px||thumb|center]]


[[File:Installation-complete-mint.png|640px||thumb|center]]

==== After install ==== 


Do NOT apt-get update and upgrade on the command-line for the updates. Go to the updater in the menu (4 locations down from the software manager). Use the menu.


[[File:Update-and-software-management-in-mint.png|640px||thumb|center]]


[[File:Update-manager.png|640px||thumb|center]]

=== Ubuntu with GNOME === 


[[File:Welcome.png|640px||thumb|center]]


Click install Ubuntu, then do the preparations. If you play a musician role you will probably want the Fluendo codecs installed, for that tick the "Install this third-party software" box.


[[File:Preparing-to-install.png|640px||thumb|center]]


LVM will allow you to later resize partitions. If it's a laptop you are installing on and you regularly take it into public spaces, encrypt the Ubuntu installation.


[[File:Installation-type.png|640px||thumb|center]]


[[File:Installation-type2.png|640px||thumb|center]]


Pick your keyboard, or if you feel uncertain have the installer detect it for you.


[[File:Keyboard-layout.png|640px||thumb|center]]


Make that minimally six. The "after install" things I do require me to enter my password and I am lazy so I initially make it a small one and make it a better one later.


[[File:Who-are-you.png|640px||thumb|center]]


Installation ...


[[File:Welcome-to-ubuntu.png|640px||thumb|center]]


And yes!


[[File:Installation-complete.png|640px||thumb|center]]Remove boot medium, and set BIOS to start from hard disk.

==== After install ==== 


By default, Ubuntu sends some of your information to third parties without asking you to opt-in. Follow these 6 simple steps to fix Ubuntu and restore your privacy https://fixubuntu.com/


If you are not doing a stealth install, and if you have your network card working, to get the upgrades since the image you installed was made, on the command-line do:


$ sudo apt-get update
$ sudo apt-get dist-upgrade

=== Debian with MATE === 


In the Isolinux bootloader menu choose install:


[[File:Install.png|640px||thumb|center]]


Choose language (this sets language for both the installation process as well as for the to-be-installed linux):


[[File:Select-a-language.png|640px||thumb|center]]


Select location. It doesn't have to be where you actually are. And you can change it later. It sets what repositories you use, but even that can be changed later.


[[File:Select-location.png|640px||thumb|center]]


Locales are used by glibc and other "locale-aware" programs or libraries for rendering text, correctly displaying regional monetary values, time and date formats, alphabetic idiosyncrasies, and other locale-specific standards:


[[File:Configure-locales.png|640px||thumb|center]]


The "American English" keyboard corresponds to the QWERTY layout, otherwise pick yours:[[File:Configure-keyboard.png|640px||thumb|center]]Until now the installation steps were completely contained in the boot image included on the CD/DVD, a file of limited size and loaded into memory by the BIOS when booting from the removable medium. What follows are a few automatic windows showing progress in:
* detecting hardware: detects hardware and tries to identify the drive and loads the modules of the various hardware components detected, and then "mounts" the drive to read it
* loading additional components: loads all the files necessary to continue including additional drivers for remaining hardware (especially the network card), as well as all the components of the installation program
* detecting network hardware: tries to identify the network card and load the associated firmware. If automatic detection fails, you can manually select the firmware to load, and you can do it later. If need be you can use a USB-network-adapter. If doing a netinst installation, this must be made to work now, because the debian packages must be loaded from the network.
Next the installer attempts an automatic network configuration by DHCP (for IPv4) and by IPv6 network discovery. If this fails, the installer provides choices: you can try again, attempt DHCP configuration by declaring the name of the machine (rumpelstiltskin jumps to mind), or set up a static network configuration. This requires having an IP address, a subnet mask, an IP address for a potential gateway, a machine name, and a domain name to enter.


If network is successfully configured, pick whatever name for your host.


[[File:Configure-network.png|640px||thumb|center]]Leave domain name empty and continue. [[File:Configure-network2.png|640px||thumb|center]]


Make the root password long (8 characters or more) and hard to guess. All machines connected to the internet are regularly targeted by automated connection attempts with the most obvious passwords and sometimes even with dictionary attacks. Avoid using names of partners, pets, children, parents, friends, dates of birth, etc. Use password generators.


[[File:Set-up-users-and-passwords.png|640px||thumb|center]]


Debian also imposes the creation of a standard user account on you so that you do not get into the bad habit of working as root. The precautionary principle applies.


[[File:Set-up-user.png|640px||thumb|center]]


If the network is available, the system's internal clock is updated (in a one-shot way) from an NTP server. This way the timestamps on logs will be correct from the first boot.


Partitioning is for dividing the available space on the hard drives. This step also includes choosing the filesystems to be used. All of these decisions will have an influence on performance, data security, and the administration of a server. For more on that, check out [[#Manual partitioning|manual partitioning]].


To install next to a windows system, check out [[#Dual boot|dual boot]] before you continue.


For a standard install of debian on a PC as new user, the very easy way is recommended:


[[File:Partition-disk.png|640px||thumb|center]]


[[File:Partition-disk2.png|640px||thumb|center]]


[[File:Partition-disk3.png|640px||thumb|center]]


[[File:Partition-disk4.png|640px||thumb|center]]


[[File:Partition-disk5.png|640px||thumb|center]]


[[File:Partition-disk6.png|640px||thumb|center]]


The next steps do not require any user interaction. The "base system" packages, [[Linux_applications#dpkg|dpkg]] and [[Linux_applications#apt|apt]] for managing packages, and utilities to boot the system, are installed.


In order to be able to install additional software, apt needs to be configured and told where to find debian packages. You are asked if it must use a network source for packages, or if it should only look for packages on the removable media you are installing from.


[[File:Configuring-apt.png|640px||thumb|center]]


[[File:Configuring-popularity-contest.png|640px||thumb|center]]


MATE:


[[File:Software-selection.png|640px||thumb|center]]


The bootloader is the first program started by the BIOS. This program loads the linux kernel into memory and then executes it. It often offers a menu that allows the user to choose the kernel to load and/or the operating system to boot.
[[File:Install-grub.png|640px||thumb|center]]
If you are installing via USB, best not to automatically install GRUB, but to choose No and set it manually. That list of disks and device files when partitioning.


That's it. Reboot.


[[File:Finish-installation.png|640px||thumb|center]]

==== After install ==== 


Add your user to the "sudoers" list (or else keep su-ing for actions that require administrator permissions). To add a user to the sudoers list you can use visudo (and need to su for that first).


[[File:Visudo-command.png|640px||thumb|center]]


Visudo is opened. Scroll down to this line "root ALL=(ALL:ALL) ALL" and add a line directly below for the user. On a PC you can just copy the entire line (it is still somewhat empty here) for root and change "root" to user. On a server you do not give users ALL permissions. Ya read this http://ubuntuforums.org/showthread.php?t=1132821 and start some happy puzzling.


[[File:Visudo.png|640px||thumb|center]]


After you are done, you are returned to the command-line and can sudo.


[[File:Not-a-sudoer.png|640px||thumb|center]]


If you are not doing a stealth install, and if you have your network card working, to get the upgrades since the image you installed was made, on the command-line do:


$ sudo apt-get update
$ sudo apt-get upgrade

== Dual boot with windows == 
A windows OS should be installed first, because its bootloader is very particular and the installer tends to overwrite the entire hard drive, wiping out any data stored on it. If windows isn't already installed, install it first. If you are able to partition the drive prior to installing windows, leave space for linux during the initial partitioning process. Then you won't have to resize your NTFS partition to make room for your linux later, saving a bit of time.


if you install (or reinstall) windows after linux, the bootloader will be erased. Linux will still be on the hard drive, but will no longer be accessible from the boot menu as the windows bootloader does not recognise other operating systems. If this happens, you can restore the bootloader with the steps in [[#F*&^% bootloader and rescue mode|F*&^% bootloader and rescue mode]].


If you wish to keep using the windows bootloader you can try this:
* Turn off Windows Fast Boot http://www.eightforums.com/tutorials/6320-fast-startup-turn-off-windows-8-a.html
* Shrink the main Windows 8.1 partition. Windows expects a certain partition scheme https://technet.microsoft.com/en-us/library/hh824839.aspx so this can be a bit tricky http://forum.linuxbsdos.com/topic/151/how-to-resize-or-shrink-a-gpt-partitioned-c-drive-in-windows-8 Give Debian between 10G and 100 GB of space.
* Shut off Secure Boot and Fast Boot in the UEFI.
* Download your debian iso and put it on some media as described above.
* Insert your install media and install debian in the space you created. At the partitioning step make notes on what partitions are labeled what (/dev/sdx).
* At the end of installing debian, you can set the installer to install GRUB in the EFI System Partition of Windows 8.1.
Alternatively turn UEFI off (use Legacy) and set the installer to put the GRUB bootloader (that will have an entry for windows) on a linux partition.

== Manual partitioning == 
Manual partitioning allows for choosing the purpose and size of each partition. You need this if you wish to use software RAID or install linux alongside an existing operating system (windows or other).


The first screen of ''manual partitioning'' displays available disks, partitions, and any possible free space that has not yet been partitioned. You can select an item and pressing the Enter key gives a list of possible actions. You can erase a partitions on a disk by selecting it. When selecting free space on a disk, you can manually create a new partition. You can also do this with guided partitioning.


The so-called mount point is the directory tree that will house the contents of the file system on the selected partition. A partition mounted at the directory <code>/home/</code> is meant to contain user data. The directory <code>/</code> is the root of the file tree, and therefore the root of the partition that will host the debian system.


[[File:Partition-disk11.png|640px|thumb|center]]


Virtual memory allows the kernel to free a bit of storage by storing the parts of the RAM that have been inactive for some time on the swap partition of a hard disk. To simulate the additional memory, windows uses a swap file that is directly contained in a file system. Linux uses a partition, hence the term "swap" partition, mounted at <code>/swap/</code>.


When choosing a partition, you can:
* format it and include it in the file tree by choosing a mount point
* use it as a swap partition
* make it into a "physical volume for encryption"
* make it a "physical volume for LVM"
* use it as a RAID device
* leave it unchanged

=== Recommendations === 
* Use LVM
* Depending on your threat model, configure encrypted volumes.
* If you plan to install many programs not part of the your distribution create a separate <code>/usr/local</code> partition.
* You can put <code>/tmp</code> on its own partition with 20–100 MB.
* For multi-user systems or systems with lots of disk space you can put <code>/usr</code>, <code>/var</code>,<code>/tmp</code>, and <code>/home</code> each on their own partitions separate from the <code>/</code>partition. For very complex server systems see the Multi Disk HOWTO http://www.tldp.org/HOWTO/Multi-Disk-HOWTO.html
* If you are setting up a server with lots of user accounts, create a separate large<code>/home</code> partition.
* If your machine will be a mail server, you can make <code>/var/mail</code> a separate partition.

=== Configure the Logical Volume Manager === 
You want this. The size of the partitions are no longer limited by individual disks but by their cumulative volume, and you can resize existing partitions at any time, possibly after adding an additional disk. The partitioning tool can do this automatically or you can configure LVM yourself:
* Create the partitions that will be "physical volumes for LVM".
* To activate LVM, choose "Configure the Logical Volume Manager (LVM)", then on the same configuration screen "Create a volume group"”, to which you associate the existing physical volumes.
* Create logical volumes within this volume group.

=== Configure encrypted volumes === 
For confidentiality of your data you can encrypt the data on some partitions. This feature can be added underneath any filesystem, because linux (dm-crypt driver) uses the Device Mapper to create a virtual partition (with protected content) based on an underlying partition that will store the data in an encrypted form. For more on that see LUKS, the Linux Unified Key Setup, a standard format enabling storage of encrypted data and meta-data on the encryption algorithms used.

[[File:Partition-disk7.png|centre|thumb|640x640px]]


[[File:Partition-disk8.png|centre|thumb|640x640px]]When an encrypted partition is used, the encryption key is stored in memory (RAM). Retrieving it allows the decryption of the data, making it important to not leave a copy of this key anywhere that a thief of the computer or hard drive can access. On laptops the contents of RAM is stored on the swap partition during hibernation. When using encrypted partitions, also encrypt the swap partition.[[File:Partition-disk9.png|centre|thumb|640x640px]]The software will then propose to initialise the physical volume with random data (making the localisation of real data more difficult), and will ask you for an "encryption passphrase" (make it hard but not too hard because you will have to enter it every time you boot your computer or access the content of the encrypted partition.


This partition can be used as a physical volume for LVM so as to protect several partitions (LVM logical volumes) with the same encryption key, including the swap partition.

== F*&^% bootloader and rescue mode == 
Sometimes, things go wrong, and the system you've carefully installed is no longer bootable. Perhaps the boot loader configuration broke while trying out a change, or right after installation of debian the system does not boot, etc.

Boot from your installation medium in rescue mode:


[[File:Rescue-mode.png|640px|thumb|center]]


You'll be shown the first few screens of the installer, with a note in the corner of the display to indicate that this is rescue mode, not a full installation. Your system is not about to be overwritten. Rescue mode simply takes advantage of the hardware detection facilities available in the installer to ensure that your disks, network devices, and so on are available to you while repairing your system.


Then, instead of the partitioning tool, you are shown of the partitions on your system, and are asked to select one of them. You can choose partitions on RAID, LVM devices or partitions directly on disk containing the root file system.


The installer will now give you a shell prompt for doing repairs in the file system you selected. If the installer cannot run a shell in its root file system, the file system may be corrupt or one or many of many other things may have happened. Repairing broken systems can be difficult, and if you have problems, consult another geek, or hit the forums and mailing-lists.


For reinstalling the GRUB boot loader into the master boot record of the first hard disk, do:


# grub-install '(hd0)'


After you exit the shell, the system will reboot. Or not. In which case, hit the mailinglist, IRC channel or forums, or all of them if need be with this excellency challenging question.

== Upgrading distributions == 

=== Recommendations for upgrading === 
* If you are new to linux, use the Software Center.
* Use apt-get for big upgrades, for example a dist-upgrade from one stable to the next, because it will find relatively good solutions quickly, while aptitude will likely find several convoluted solutions (or none at all).
* For regular upgrades in unstable or testing versions of a debian-based distribution, use aptitude: ''aptitude safe-upgrade'' can keep temporarily broken packages on hold while still installing new packages and you can tweak operations dynamically.

== Getting a device to work == 
Hardware like a sound card or ethernet card, etc. have two essential pieces of software that make them function. The first is a working driver, software that lets your system talk to the hardware. The second is firmware, usually a small piece of code uploaded directly to the device and maintained by the company that develops the hardware device. You can think of the firmware as a way of programming the hardware inside the device. It is a black box, and because of that treated like hardware; there’s no accompanying source code that is freely distributed with it.


Firmware can come from one of the following sources:
* The linux-image package (containing the linux kernel and licensed firmware, installed by default)
* The linux-firmware package (containing other licensed firmware, installed by default)
* The linux-firmware-nonfree package in multiverse (containing firmware that are missing redistribution licenses, not installed by default)
* A separate driver package (not installed by default)
* Elsewhere (not installed by default, driver CD, email attachment, website)
If during install you get messages as to hardware that could not be provided for, write down what those messages say. It probably looks something like
e100: eth0: e100_request_firmware: Failed to load firmware "e100/d101m_ucode.bin": -2


When you installed the kernel, the post-installation script went through all loaded modules (listed by 'lsmod') and verified whether modules as provided by the newly installed kernel might require firmware files. This information can be retrieved with modinfo, which needs a path to the modules. You either traverse there, or build the path to the modules using:


$ uname -a
Linux debian 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) x86_64 GNU/Linux
Then with modinfo:
$ modinfo -F firmware /lib/modules/3.16.0-4-amd64/kernel/drivers/net/e100.ko
e100/d102e_ucode.bin
e100/d101s_ucode.bin
e100/d101m_ucode.bin


The firmware packages embed the list of firmware files in their description, so now you can use


$ apt-cache search d101m_ucode.bin
firmware-linux-nonfree - Binary firmware for various drivers in the Linux kernel


You can also use 'apt-file' (provided by the package of the same name) or the web interface at packages.debian.org.


$ apt-file search d101m_ucode.bin
firmware-linux-nonfree: /lib/firmware/e100/d101m_ucode.bin


If the above commands return nothing, you probably need to enable the "non-free" repository in your /etc/apt/sources.list first. You can also enable it within synaptic. And you also want to run 'sudo apt-file update' to have the latest information.


Now you can install the right package, in the example above it was firmware-linux-nonfree.

== Installing debian like an expert == 


At the first screen running the installer choose "Advanced Options" and then "expert install". This makes the installation process "verbose" and details all possible options in the process of installation, and allows for navigation between the various steps without them happening automatically in sequence.


[[File:Debian-installer-main-menu.png|640px|thumb|center]]

=== Select multiple locales === 
The "select locales" choice allows for entering multiple locales:


[[File:Configure-locales-expert.png|640px|thumb|center]]


For some often-used locales see https://wiki.debian.org/Locale

=== Detect and mount CDROM === 
First it finds removable drives other than CDROM:


[[File:Detect-and-mount-CDROM.png|640px|thumb|center]]


Next it detects CDROM:


[[File:Detect-and-mount-CDROM2.png|640px|thumb|center]]

=== Load installer components from CD === 


These are additional components that you can use during install:


[[File:Load-installer-components-from-CD.png|640px|thumb|center]]


Now the main menu contains more items to hopscotch through:


[[File:Debian-installer-main-menu2.png|640px|thumb|center]]

=== Configure network === 


[[File:Configure-network3.png|640px|thumb|center]]


[[File:Configure-network4.png|640px|thumb|center]]


... and forward. This requires filling in an IP address, a subnet mask, an IP address for a potential gateway, a machine name, and a domain name to enter.

=== Configure the clock === 
NTP (Network Time Protocol) allows a machine to synchronize with others fairly accurately, taking into consideration the delays induced by the transfer of information over the network and other possible offsets. There are numerous NTP servers on the Internet, and the more popular ones may be overloaded. The pool.ntp.org NTP server is a group of machines that have agreed to serve as public NTP servers. You can limit use to a sub-group specific to a country, with, for example, us.pool.ntp.org for the united states, or ca.pool.ntp.org for canada, or:


[[File:Configure-the-clock.png|640px|thumb|center]]


[[File:Configure-the-clock2.png|640px|thumb|center]]

=== Partition disks === 
This is the same as during a standard install. You can use guided partitioning (see standard install above) or [[#Manual partitioning|manual partitioning]].

=== Change debconf priority === 


When installing (or upgrading) any single package or a group of packages, debconf asks the packages' configuration questions all at once, and stores the user/admin preferences in a database. Later as the packages are installing, their scripts use the configuration preferences in the database to generate configuration files and otherwise do administrative tasks (e.g. set up servers to start or not start, install other software such as libdvdcss, etc.). This saves the hassle of editing configuration files by hand, and also of waiting for each individual package to install before responding to certain configuration questions. Important during a server install.
[[File:Change-debconf-priority.png|640px|thumb|center]]

=== Install the LILO boot loader on a hard disk === 


GRUB is the default bootloader installed by debian: it works with most filesystems and reads its configuration during boot to find the exact position of the new kernel. That means it doesn't require an update after each installation of a new kernel. Version 1 couldn't handle all combinations of LVM and software RAID and version 2 is more complete, but there may still be situations where LILO can save the day.


[[File:Install-the-LILO-bootloader.png|640px|thumb|center]]If you want to see some of the above expert options applied, see this tutorial from 2011: How to install and multi-boot between Windows, and Debian Testing with full disk encryption http://scarygliders.net/2011/12/06/how-to-install-and-multi-boot-between-windows-and-debian-testing-with-full-disk-encryption/
</translate>

== Stealth install ==
* Download and burn the CD/DVD's via a public machine.
* Official CD images do not include non-free firmware. Download the unofficial CD builds containing non-free firmware from http://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/. If using a USB stick, download from http://cdimage.debian.org/cdimage/unofficial/non-free/firmware/ and unpack to either the root directory or a directory named <code>/firmware</code> of the file system on the medium. If the firmware you need is not included in the tarball, you can also download specific firmware packages from the (non-free section of the) archive http://packages.debian.org/search?keywords=firmware. It is also possible to copy individual firmware files to the medium. Loose firmware could be obtained for example from an already installed system or from a hardware vendor.
* For installing packages, see [[Linux_applications#Stealth_install_of_applications|Stealth install of applications]]

== Related ==

* [[Linux applications]]
* [[Linux security]]
* [[Kinky linux command-line]]
* [[Anonymising your traffic with linux]]
* [[Clean up all the things]]
* [[Shell scripting]]
* [[Encrypting everything]]
* [[Linux virtualisation]]
* [[Making our own linux images]]

== References ==

[[Category:How_To]]

Digital threats, detection, protection and (counter) moves

2015-09-27T23:53:23Z

Lilith2: /* TLS/SSL */

This page lists theoretical defenses and detection methods for selected groups of leaked surveillance programs and services. This is just a thought experiment covering (theoretical) defenses against these attacks and not intended to spread fear, uncertainty or doubt about surveillance states.

Due to the age and limited scope of the leaked documents and what we are up against <ref>Cryptome: Communications privacy folly http://cryptome.org/2012/06/comms-folly.htm
</ref>, the defenses mentioned in these tables should not be relied upon for protection and I make no guarantees to their accuracy. Do your own research and make informed decisions, knowing it is impossible to be completely safe. Things move incredibly fast in this arena and I will update these tables when more is found and/or theorised.

Ideally we discuss these exploits and counter moves without the NSA being privy to those discussion, but that would make spreading possible counter moves too slow for the surveillance development cycle (security arms race) and effectively exclude non-techie activists from being able to defend themselves to some extent, so the next best solution is to discuss everything in the open, rather than not discussing them at all. May we not have to rely on a new class of technocracy.

== Social engineering ==
These tables were started off from http://www.itsecurity.be/social-engineering-what-is-it-and-how-to-defend-yourself and then added to. I started off from a to gendersec external source to increase chances of finding that which we may be overlooking.

=== Internal threats ===

{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Power of scarcity
|| Scarcity is when people are told something they need or want has limited availability and to get it they must comply with a certain attitude or action. Many times the desired behavior is not even spoken, but the way it is conveyed is by showing people who are acting "properly" getting rewards.

Mental shortcuts are often beneficial because they enable us to more efficiently navigate the complexity of life. But the power of scarcity can also mislead us. If some people get something not everyone else has (energy, food, water), it makes them feel "special", "unique", "high status", and gives a "position" to defend. And fear of loss of that is an attachment.

An example is a government, in this case South Africa, taking something necessary to life, and making it "scarce" and available only to supporters — a malicious, but very effective, manipulation tactic: http://www.social-engineer.org/wiki/archives/Governments/Governments-FoodElectionWeapon.html See [[timeline merchants of death]] for more examples.

|| Scarcity complicates Gut/Head decision processes because, often, there is only a brief window of opportunity in which we can select something that is in scarce supply. When we can see the supply diminishing before our eyes, we feel particularly compelled to take action.

Each context and scarcity is different, but mind not to rule out the head too quickly in [[Scenario planning#Gut.2C head.2C and decision making|"Gut versus Head"]] decisions.
|-
| Fear of authority || Many people are apprehensive in the presence of someone they perceive to be an authority figure, it is not that person they are apprehensive about but most likely the position and power of the person that intimidates them.
<nowiki> </nowiki>The attackers take on roles of authority figures such as law enforcement officers or high-ranking company officials to extract sensitive information from the victims.
|| Protect yourself with learning about [[Self-dox#Elicitation|elicitation]] (in roleplay settings) and then, time for some controlled folly!
|-
| Desire to be helpful
|| People in their desire to be helpful and to solve other peoples queries, give out a lot of information that otherwise should not be. || Do not disclose information to an outsider as it could give an attacker a chance to get unauthorised access.
|-
| Laziness || All of us have come across some job that requires us to do only a specified set of activities and nothing more. This causes boredom to the person who performs the same task repeatedly on a daily basis and over time the "bored" learn "shortcuts" to do the tasks using minimal effort while meeting targets. This leads to a laid back attitude and becoming susceptible to attackers who target such individuals knowing they can get the required information with much ease. || Find ways of keeping yourself alert. Go do something else. You are becoming a sitting duck.
|-
| Ego || Many a times, the attacker makes a person more emotionally sure of himself/herself, thus removing the logical awareness of the security breach that is occurring. The result is that the person being hacked senses no harm in providing whatever it is that the attacker is requesting. The reason that such an attack succeeds is that the attacker is a receptive audience for victims to display how much knowledge they have. || The ego isn’t going to go away and whether it makes you vulnerable depends on the context. A bit of swagger based on achievements can be quite okay now and then! In some other contexts allowing your ego to call the shots can be a problem.

The simplest detector is asking yourself: Do you feel superior (or inferior) to others? If yes, then your ego is in control. Depending on context choose "not now, not here, not with this" or "do some counter-intelligence" or "Arrrrr, and a bottle of rum!"
|-
| Excitement of victory & fear of loss || Yes! An opportunity! Easy money!!! Out of excitement security is switched off, links are clicked and documents downloaded that turn out to be corrupted (and contained malware that allows the email sender to gain remote access to the machine, or someone's funds, or data, or ...).

Or, ''"You have won 1 Million Dollars and to claim the winning amount, deposit $75,000 in Account number: XXXXXX in 10 days from receiving this e-mail, failing to which the winning amount would be declared unclaimed and there would be a nee lucky-draw to decide the next winner"''

An example is the 419 scams many people fell for http://www.social-engineer.org/wiki/archives/ConMen/ConMen-Scam-NigerianFee.html
|| It is another ego thing isn't it?
|-
| Insufficient knowledge || People with insufficient knowledge can easily be exploited by creating a sense of urgency and not allowing much time to think and understanding the fact that they are under attack. || Gather (and spread) knowledge in a relaxed way. Stay with your own timing. Do the best you can.
|}

=== External threats ===
{| class="wikitable"
|-
! !! More information !! (Counter) moves
|-
| Shoulder surfing || Shoulder surfing is a security attack where-in, the attacker uses observational techniques, such as looking over someone's shoulder, to get information while they are performing some action that involves explicit usage of sensitive, visible information. This can be performed at a close range as well as at a long range using binoculars or other vision-enhancing devices. || Don't work on anything important in public.
|-
| Dumpster diving || Going through the trash can yield one of the most lucrative payoffs for information gathering. People often throw away invoices, notices, letters, CDs, computers, USB keys, and a plethora of other devices and reports that can truly give amazing amounts of information. The attacker can use these items to get a huge amount of information about people, organisations they participate in, and network structure.

Some people shred documents but some types of shredding can be thwarted with a little time and patience and some tape.
|| Mind what you throw away where and how.

Using a shredder that shreds both directions into a fine minced mess makes taping documents back together nearly impossible. Or incinerate. Or both.
|-
| Role playing || Role playing is one of the key weapons for a social engineer. It involves persuading or gathering information through the use of online chat sessions, emails, phones or any other method that you use to interact online with others, and in which the social engineer plays the role of a helpdesk or technician, helplessness, or whatever may work in that context to get targets to divulge confidential information. || Don't "burn" the social engineer. Apply controlled folly instead, and then let's see who is the cat here and who the mouse: Feed them incorrect information. Remember what the intent is. Intent is the key element in intelligence and counter-intelligence operations.
|-
| Trojan horses || It is one of the most predominant methods currently used by online criminals and intelligence agencies that involve tricking victims into downloading a malicious file to their machine, which on execution creates a backdoor in the machine that can be used by the attacker any time in the future and thus having complete access of the victim's machine. Compromising a browser is relatively easy and it is cross-platform, hence an often chosen attack vector. <ref>Meet The Hackers Who Sell Spies The Tools To Crack Your PC (And Get Paid Six-Figure Fees) http://www.forbes.com/sites/andygreenberg/2012/03/21/meet-the-hackers-who-sell-spies-the-tools-to-crack-your-pc-and-get-paid-six-figure-fees/</ref>.
<nowiki> </nowiki>Among all of the NSA hacking operations exposed by whistleblower Edward Snowden over the last two years, one in particular has stood out for its sophistication and stealthiness. Known as Quantum Insert, the man-on-the-side hacking technique has been used to great effect since 2005 by the NSA and its partner spy agency, Britain’s GCHQ, to hack into high-value, hard-to-reach systems and implant malware.

|| Detection focuses on identifying anomalies in the data packets that get sent to a victim’s browser client when the browser attempts to access web pages http://blog.fox-it.com/2015/04/20/deep-dive-into-quantum-insert/

For more such tools run by intelligence agency for targeted surveillance see [[#Infected_firmware|infected firmware]] below.
|-
| Phishing || Phishing is the act of creating and using websites and e-mails designed to look like those of well-known legitimate businesses, financial institutions and government agencies to deceive receivers into disclosing their personal information.
|| These are semantic attacks.
* Be suspicious of any email with urgent requests for personal (financial) information.
* Set your email client to receive email in plaintext, never html format.
* Check any url in the email carefully. It can be just one letter different or missing.
* If phishy, report it.
|-
| Information on websites, online forums and social media|| Huge amounts of information regarding organisational structures (formal and informal), email adresses, phone numbers, are all available publicly on websites, forums and social media. This information can be used by the attacker to refine his approach and create a plan on whom to target and methods most likely to succeed. || Several moves are possible (not necessarily exclusive):
* Delete your social media presence
* Use different identities for your various social contexts
|-
| Reverse social engineering || A reverse social engineering attack is an attack in which an attacker convinces a target that he or she has a problem or might have a certain problem in the future and that the attacker, is ready to help solve the problem.

Reverse social engineering involves three parts:
* Sabotage: After the attacker gains a simple access to the system, he corrupts the system or gives it an appearance of being corrupted. When the user sees the system in the corrupted state, he starts looking for help so as to solve the problem.
* Marketing: In order to make sure that the user approaches the attacker with the problem, the attacker advertises himself as the only person who can solve the problem.
* Support: In this step, the attacker gains the trust of the target and obtains access to sensitive information.
|| If your alarm didn't go off in the sabotage step, when your system appearing corrupt coincided with this remarkable good fortune of someone being right there at the right time ready to help, then let your alarm go off at the [[Propaganda#False_Dilemma|false dilemma]] you are presented with in the marketing step.
|}

== Targeted surveillance ==

=== Hardware implants ===
Unless you are targeted by a government intelligence agency, there seems to be no need to worry about installing commodity hardware from reputable vendors.

Detection: Looking for physical devices will always be the easiest solution to detect them. The links to NSA exploits of the day were added for the comments. :)

{| class="wikitable"
|-
! !! More information !! Possible types of attacks !! Detection
|-
| Godsurge || Godsurge is a physical device plugged-in to the Joint Test Action Group or JTAG headers on a system's motherboard. JTAG headers can be found on many systems and are notoriously common in embedded devices. These headers are used during the development process for debugging purposes: they give you a direct interface with the CPU and are extremely helpful. They are commonly left on the production boards, so finding them on a device is normal and not a security concern. However, if there is a chip or board wired in to a device's JTAG headers that you did not wire in yourself, then something fishy may be going on. || The JTAG debugging interface can be used to reflash the BIOS from scratch, for example loading a compromised version of the software.

|| Look for the JTAG connecter on the motherboard. The location of the JTAG headers may differ, but they tend to be near the CPU and may have exposed pins (or not). See the Wikipedia page on JTAG for more information and to see what they look like <ref>Wikipedia JTAG http://en.wikipedia.org/wiki/Joint_Test_Action_Group</ref>
|-
| Ginsu and Bulldozer
|| Ginsu provides software application persistence on target systems with the PCI bus hardware implant, Bulldozer
http://leaksource.files.wordpress.com/2013/12/nsa-ant-ginsu.jpg

https://www.schneier.com/blog/archives/2014/01/ginsu_nsa_explo.html
|| Exploit persistence from a PCI card ROM
|| Open up the computer's case and look for a PCI card that does not belong. For example, if you find a PCI card that appears to serve no purpose (e.g.. not your network card or it has no external ports), then perhaps try removing it and see how things work. If a few black SUVs roll up to your house after unplugging the PCI card, it's probably not because your domicile is the set for a rap superstar's new music video. Or maybe it is.
|-
| Cottonmouth I, Cottonmouth II and Cottonmouth III
|| These bugs are embedded somewhere along the USB bus and function as an air gap bridge to assist in exfiltration of data as well as allow persistent compromise. It can be embedded directly in the USB headers in an existing USB peripheral (USB hub or keyboard). These devices allow for exfiltration of data over unknown radio frequencies to listening devices in the area, even on a system that is not connected to the internet.
http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-i.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-i_n.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonmouth-ii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-ii.html

http://leaksource.files.wordpress.com/2013/12/nsa-ant-cottonomouth-iii.jpg

https://www.schneier.com/blog/archives/2014/03/cottonmouth-iii.html
|| USB host attack
|| Open up the keyboard or USB hub and identify a board that serves no purpose to the device. The malicious USB device would also likely show up on your computer's list of USB devices, so just checking there would be a good place to start.
|}

=== Radio frequency exfiltration ===
The devices used can vary, but they all employ a similar method of communication via an unknown radio/radar protocol. The frequencies are not known and based on the information leaked the devices are passive (only power on when data is being extracted) making identification of (type of) signal (and intelligence) hard.

Detection: First check for evidence of a device which has been wired in to an existing device such as a keyboard or other peripheral. The SpiderBlog describes the following possibility: You can check if there are RF transmitters in a device by monitoring the spectrum (using an amateur RF listening bug detector) while the device is off (to get a baseline for ambient RF background noise) and then monitoring it again after the suspect device is turned on and transmitting data via radio frequency. The detection device would pick up the signal and alert the user. <ref>Detecting A Surveillance State - Part 2 Radio Frequency Exfiltration https://www.trustwave.com/Resources/SpiderLabs-Blog/Detecting-A-Surveillance-State---Part-2-Radio-Frequency-Exfiltration/
</ref> Further analysis of the signal and its intelligence is still hard, as the exact protocols are (still) unknown.
{| class="wikitable"
|-
! !! More information !! Possible types of attack
|-
| Howlermonkey
|| Short to medium range Radio Transceiver

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/howlermonkey_ns.html
|| HOWLERMONKEY is a COTS-based transceiver deigned to be compatible with CONJECTURE/SPECULATION networks and STRIKEZONE devices running a HOWLERMONKEY personality. PCB layouts are tailored to individual implant space requirements and can vary greatly in form factor.
|-
| Ragemaster
|| Hardware implant in a VGA cable that sends video data over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/03/ragemaster_nsa.html
|| The RAGEMASTER taps the red video line between the video card within the desktop unit and the computer monitor, typically an LCD. When the RAGEMASTER is illuminated by a radar unit, the illuminating signal is modulated with the red video information. This information is re-radiated, where it is picked up at the radar, demodulated, and passed onto the processing unit, such as a LFS-2 and an external monitor, NIGHTWATCH, GOTHAM, or (in the future) VIEWPLATE. The processor recreates the horizontal and vertical sync of the targeted monitor, thus allowing TAO personnel to see what is displayed on the targeted monitor.
|-
| Loudauto
|| Hardware device that sends amplified audio over RF

https://leaksource.files.wordpress.com/2013/12/nsa-ant-howlermonkey.jpg

https://www.schneier.com/blog/archives/2014/01/loudauto_nsa_ex.html

|| Room audio is picked up by the microphone and converted into an analog electrical signal. This signal is used to pulse position modulate (PPM) a square wave signal running at a pre-set frequency. This square wave is used to turn a FET (field effect transistor) on and off. When the unit is illuminated with a CW signal from a nearby radar unit, the illuminating signal is amplitude-modulated with the PPM square wave. This signal is re-radiated, where it is picked up by the radar, then processed to recover the room audio. Processing is currently performed by COTS equipment with FM demodulation capability (Rohde & Schwarz FSH-series portable spectrum analyzers, etc.) LOUDAUTO is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|-
| Surleyspawn
|| Hardware implant in a keyboard that emits keystrokes over RF

http://leaksource.files.wordpress.com/2013/12/nsa-ant-surlyspawn.jpg

https://www.schneier.com/blog/archives/2014/02/surlyspawn_nsa.html
|| The board taps into the data line from the keyboard to the processor. The board generates a square wave oscillating at a preset frequency. The data-line signal is used to shift the square wave frequency higher or lower, depending on the level of the data-line signal. The square wave, in essence, becomes frequency shift keyed (FSK). When the unit is illuminated by a CW signal from a nearby radar, the illuminating signal is amplitude-modulated (AM) with this square wave. The signal is re-radiated, where it is received by the radar, demodulated, and the demodulated signal is processed to recover the keystrokes. SURLYSPAWN is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
|}

=== Infected firmware ===
Detection: Dump the BIOS to a bin file and compare the hash with a clean BIOS hash. See flashrom for identifying, reading, writing, erasing, and verifying BIOS/ROM/flash chips <ref>Debian packages: flashrom https://packages.debian.org/jessie/flashrom</ref>.

Removal: For each of these infections, where applicable, pulling the chip and replacing it with a new freshly burned BIOS chip or compact flash card would be sufficient. When dealing with built-in firmware it's a bit more difficult than pulling and replacing. You will need to re-flash the device using an operating system that is not at risk of being attacked by the infected firmware. You could boot the device into a low level OS in hopes that the firmware infection isn't able to protect itself. Or, you could wire in a debugging header to the device (such as JTAG , if available) to read or write the firmware on the device to clean things up for good.

{| class="wikitable"
|-
! !! More information !! Possible types of attack || Protection
|-
| Dietybounce
|| Motherboard BIOS Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-deitybounce.jpg?w=1208&h=1562

https://www.schneier.com/blog/archives/2014/01/nsa_exploit_of.html

|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS on a target machine to implant DEITYBOUNCE and its payload (the implant installer). Implantation via interdiction may be accomplished by nontechnical operator through use of a USB thumb drive. Once implanted, DEITYBOUNCE's frequency of execution (dropping the payload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Swap
|| Hard Drive Firmware Infector

https://leaksource.files.wordpress.com/2013/12/nsa-ant-swap.jpg

https://www.schneier.com/blog/archives/2014/02/swap_nsa_exploi.html
|| Through remote access or interdiction, ARKSTREAM is used to reflash the BIOS and TWISTEDKILT to write the Host Protected Area on the hard drive on a target machine in order to implant SWAP and its payload (the implant installer). Once implanted, SWAP's frequency of execution (dropping the playload) is configurable and will occur when the target machine powers on.
|| Most motherboards can be flashed with coreboot https://www.coreboot.org/Supported_Motherboards tutorials: https://www.coreboot.org/Category:Tutorials
|-
| Headwater, sierramontana, and jetplow
|| Firmware backdoors that target popular networking hardware

https://leaksource.files.wordpress.com/2013/12/nsa-ant-headwater.jpg

https://www.schneier.com/blog/archives/2014/01/headwater_nsa_e.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-sierramontana.jpg

https://www.schneier.com/blog/archives/2014/01/sierramontana_n.html

https://leaksource.files.wordpress.com/2013/12/nsa-ant-jetplow.jpg

https://www.schneier.com/blog/archives/2014/01/jetplow_nsa_exp.html
|| HEADWATER PBD implant will be transferred remotely over the Internet to the selected target router by Remote Operations Center (ROC) personnel. After the transfer, the PBD will be installed in the router's boot ROM via an upgrade command. The PBD will then be activated after a system reboot. Once activated, the ROC operators will be able to use DNT's HAMMERMILL Insertion Tool (HIT) to control the PBD as it captures and examines all IP packets passing through the host router.

Currently, the intended DNT Implant to persist is VALIDATOR, which must be run as a user process on the target operating system. The vector of attack is the modification of the target's BIOS. The modification will add the necessary software to the BIOS and modify its software to execute the SIERRAMONTANA implant at the end of its native System Management Mode (SMM) handler.

JETPLOW persists DNT's BANANAGLEE software implant and modifies the Cisco firewall's operating system (OS) at boot time. If BANANAGLEE support is not available for the booting operating system, it can install a Persistent Backdoor (PDB) designed to work with BANANAGLEE'S communications structure, so that full access can be reacquired at a later time.
|| Most routers can be flashed with openWRT http://wiki.openwrt.org/toh/start
|}

== Dragnet surveillance ==

Best protection: Smash your PC to bits, distribute the pieces randomly among a dozen scrap heaps, and move into the woods.
{| class="wikitable"
|-
! !! More information !! !! Protection
|-
| PRISM || PRISM is part of the overall NSA surveillance effort - a program authorised in the united states under the FISA Amendments Act (FAA), now located in Section 702 of the FISA, that allows the NSA to collect communications of specifically identified foreign targets.

When people in the rest of the world use the net, they are effectively using US-based services, making them a legal target for US intelligence. NSA has no "direct access" to the servers of companies like Microsoft, Facebook and Google - other agencies collect the data. More [[Surveillance#PRISM|here]]
|| Gathers emails, chat--video, voice, photos, stored data, VoIP, file transfers, video conferencing, notifications of target activity--logins, etc, online social networking details, and special requests collected by other programs from nine major Silicon Valley technology companies: Facebook, Google, Yahoo, Microsoft, PalTalk, Skype, YouTube, Apple, and AOL.
||
* Put political pressure on the united states - economical and political dependencies are making that unlikely, see the power of scarcity above.

* Avoid using popular web services.
* FOIA's, court cases on legitimacy
|-
| Upstream collection
|| It appears that through Upstream collection, the NSA gains access to data by forming partnerships with both foreign intelligence agencies and foreign telecommunication companies, by partnering with united states telecommunications companies that then make agreements with international companies for their internet data. More [[Surveillance#Upstream_collection|here]]
|| BLARNEY seems to focus specifically on metadata information about communications traffic and network devices (where and when they were sent).

FAIRVIEW collects just about everything on packet and session levels.
||
* Research which countries and telecommunication companies participate
* Avoid using services associated with these programs
* [[Step 1#Understanding_and_minimising_our_digital_shadows|Understanding and minimising our digital shadows]]
* Court cases on legitimacy
|}

== Darknet surveillance ==

=== Tor ===

=== I2P ===

=== FreeNet ===

== TLS/SSL ==

illusoryTLS http://illusorytls.com/
Law Enforcement Appliance Subverts SSL http://www.wired.com/2010/03/packet-forensics/

== Server attacks ==

== Censorship ==
Table initially filled with data from How to effectively argue against Internet Censorship ideas http://rys.io/en/94.txt.

Proxy servers, especially anonymous ones, located outside the area where a censorship solution is deployed can be used quite easily to circumvent any blocking method; users can modify their operating system or browser settings, or install browser additions that make using this circumvention method trivial. It is possible to block the proxy servers themselves (via IP-blocking, keyword blocking, etc.), however it is infeasible to block them all, as they are easy to set-up.

Virtual Private Networks (including “poor man’s VPNs” like SSH tunnels) require more technical prowess and usually a (usually commercial) VPN service (or SSH server) outside the area with blocking deployed. Blocking all VPN/SSH traffic is possible, but requires deep packet inspection and is a serious problem for many legitimate businesses using VPNs (and SSH) as their daily tools of trade, to allow their employees access to corporate networks from outside physical premises, via a secured link on the Internet.

TOR, or The Onion Router, is a very effective (if a bit slow) circumvention method. It is quite easy to set-up — users can simply download the TOR Browser Bundle and use it to access the Internet. Due to the way it works it is nigh-impossible to block TOR traffic (as it looks just like vanilla HTTPS traffic), to the point that it is known to allow access to the uncensored Internet to those living in areas with most aggressive Internet censorship policies — namely China, North Korea and Iran. See Tor threats.

None of the censorship solutions is able to block content on darknets — virtual networks accessible anonymously only via specialised software (for instance TOR, I2P, FreeNet), and guaranteeing high resilience to censorship through technical composition of the networks themselves. Because darknets are both practically impossible to block entirely and not allowing for any content blocking within them, they are effectively the ultimate circumvention methods. The downside to using darknets is their lower bandwidth. Deploying Internet censorship pushes the to-be-blocked content into darknets, making it ever-harder for law enforcement gather evidence and researchers gather data on the popularity of a given type of censored content.
{| class="wikitable"

! Blocking type
! How it works
! Circumvention
! DPI
|-
|DNS-based blocking
|DNS-based blocking requires ISPs (who usually run their own DNS servers, being default for their clients) to de-list certain domains (so that they are not resolvable when using these DNS servers). This means that the costs of implementing it are small.
|Custom DNS server settings can be used to easily circumvent DNS-based blocking. It does not require almost any technical prowess and can be used by anybody. There is a number of publicly available DNS servers that can be used for this purpose. There is no way to easily block the use of this method without deploying censorship methods other than pure DNS-blocking.
|no
|-
|IP address-based blocking
|IP-based blocking requires the ISPs to either block certain IP addresses
<nowiki> </nowiki>internally or route all the outgoing connections via a central, government-mandated censoring entity. It is only superficially harder to circumvent, while retaining most if not all problems of DNS-based blocking.
|
|no
|-
|URL-based blocking
|Because this method blocks only certain, URL-identified content, not whole websites or servers (as do DNS-based and IP-based methods), it has much lower potential for accidental over-blocking. This also entails it has a higher potential for under-blocking, as the content can be available on the same server under many different URLs, and changing just a small part of the name defeats the filter.
|
|yes
|-
|Dynamic blocking
|This method uses deep packet inspection to read the contents of data being transmitted, and compares it with a list of keywords, or with image samples or video (depending on the content type).
|
|yes
|-
|Hash-based blocking
|Hash-based blocking uses deep packet inspection to inspect the contents of data-streams, hashes them with cryptographic hash functions and compares to a known database of hashes to be blocked.
|
|yes
|-
|Hybrid solutions
|In order to compromise between high-resource, low-over-blocking hash-based blocking and low-resource, high-over-blocking IP- or DNS-based solutions, a hybrid solution might be proposed. Usually it means that there is a list of IP addresses or domain names for which the hash-based blocking is enabled, hence only operating for a small part of content. This method does employ deep packet inspection.
|
|yes
|}

== Related ==
* [[Timeline that is soooo 1984 ...]]
* [[Timeline merchants of death]]
* [[Timeline masters of the internet]]
* [[Threat modeling the quick and dirty way]]
* [[Threats, detection, protection and (counter) moves]]

== References ==

[[Category:How_To]]

Self-dox

2015-09-24T09:47:37Z

Lilith2: /* Profiling and fingerprinting */

''Doxing is a technique of tracing someone or gathering information about an individual using sources on the internet. Its name is derived from “Documents” or “Docx”. Doxing method is based purely on the ability of the hacker to recognize valuable information about his target and use this information to his benefit. It is also based around the idea that, “The more you know about your target, the easier it will be to find his or her flaws”'' ~ [http://www.urbandictionary.com/define.php?term=doxing Urban dictionary].

Such flaws can then be exploited. The exploitation wildly differs depending on the adversary. For example, profiling information can be used for encapsulation attempts by intelligence agencies. If those attempts are tried and fail and you are still enough of an annoyance to your government, expect [[Psychological warfare|aggressive demonisation by government paid trolls]] (and exceptionalism at its current flourishing best).

Doxing is also a part of reconnaissance where it is called 'human intelligence' or 'humint', and its focus is on its shadow side. The techniques can be simple or complex, and are useful for background checks (infiltrant agents will probably remain invisible with these techniques but littler petty tyrant may be detected) <ref>Investigation Online: Gathering Information to Assess Risk https://modelviewculture.com/pieces/investigation-online-gathering-information-to-assess-risk
</ref>.

There are tools particularly created for d0xing. I use as little automated tools as possible. Not only because of the risk of poisoning, also because I prefer to pay more attention to little-easily-overlooked-details. The more you rely on auto-anything, the more you overlook. But if you want to use a tool and it is unknown to you, please do some research (possibly) follow the money, where the makers are located, for assessing likelihood of the code being poisoned.

This page collects doxxing resources and if you have one, please do add!

[[File:Lulz-vs-lulz.jpg|480px|thumb|right|Doxed: how Sabu was outed by former Anons long before his arrest http://arstechnica.com/tech-policy/2012/03/doxed-how-sabu-was-outed-by-former-anons-long-before-his-arrest/]]

== Storing information during research ==
[[Linux_applications#Storing_information|Store gathered data]] on an encrypted external disk.

== Resources ==

=== Spies online ===

You can use the same (or similar) spies online techniques <ref>Spies online http://www.spiesonline.net/</ref> to see yourself through the eyes of the adversaries that might d0x you. That information can then be used to protect yourself from your adversaries as best as you can. And when setting up other, anonymous or pseudonymous identities, these same resources can be used for test-driven-development of the identity.

=== Elicitation ===
*Social engineer: elicitation http://www.social-engineer.org/framework/influencing-others/elicitation/
* Information Elicitation via Social Engineering http://capec.mitre.org/data/definitions/410.html

=== Search engines ===

* Duck Duck Go https://duckduckgo.com/ (anonymous search)
* IxQuick https://ixquick.com/
* StartPage https://startpage.com/
* YaCy http://yacy.net/en/index.html (distributed search)

=== People ===

* Check Usernames http://checkusernames.com/
* Lullar http://com.lullar.com/
* Peek You http://www.peekyou.com/
* Pipl https://pipl.com/
* Recorded Future https://www.recordedfuture.com/
* Social Mention http://socialmention.com/
* WebMii http://webmii.com/
* Whos Talkin http://www.whostalkin.com/

=== Images ===

* TinEye https://tineye.com/

=== Posts and discussions ===

* Omgili http://omgili.com/

=== IP lookup ===

'''Technical information related to whois'''
* RFC 954 – NICNAME/WHOIS </nowiki>http://www.faqs.org/rfcs/rfc954.html
* RFC 1834 – Whois and Network Information Lookup Service, Whois++ </nowiki>http://www.faqs.org/rfcs/rfc1834.html
'''Whois clients'''
* Linux online manpage jwhois http://linux.die.net/man/1/whois
* Ripe whois client http://sourceforge.net/projects/whois/files/ (does not check for supported flags at the client side, except for -h (whois host) and -p (whois port))
'''Webbased whois lookups'''
* Geektools http://www.geektools.com/whois.php (also serves as proxy)
* Whois at the Open Directory Project http://www.dmoz.org/Computers/Internet/Domain_Names/Name_Search/ with (list of whois servers)
'''Useful whois servers'''
* InterNIC http://www.internic.net/whois.html (the main Internic whois)
* American Registry for Internet Numbers http://whois.arin.net/ (gives info on who owns a netblock)
* Advanced query RADb http://www.radb.net/query/?advanced_query=1 (for routing information from an IP address)
* Robtex http://www.robtex.com/ (goodies in a toolbar)
'''RWhois'''
* RWHOIS project on ARIN http://projects.arin.net/rwhois/
* RWhois Web Interface http://projects.arin.net/rwhois/prwhois.html (a sample rwhois web interface on ARIN using the Net::Rwhois perl5 client)
'''Other interesting lookups'''
* The New Ultimates, mostly US data http://www.newultimates.com/

=== D0xing tools ===

* Sleeping Time http://sleepingtime.org/
* Metagoofil http://www.edge-security.com/metagoofil.php
* theHarvester http://www.edge-security.com/theharvester.php

=== Reuseful archives ===

Such as wandering through archives, yellow pages, phone directories and other possibly useful information made publicly available.

'''Europe'''
* Archives portal europe http://www.archivesportaleurope.net/directory
* Archiwum Dokumentow Elektronicznych (POL) http://www.nac.gov.pl/
* Bundesarchiv, Federal Archives (GER) http://www.bundesarchiv.de/index.html.de
* Central State Electronic Archives of Ukraine http://tsdea.archives.gov.ua/
* The National Archives (UK) https://www.nationalarchives.gov.uk/
* White and Yellow Pages Europe http://www.wayp.com/eng/europa3.shtml
* Офіційний веб-портал Державної архівної служби України http://www.archives.gov.ua/

'''Africa'''

* National Archives of South Africa (NASA) http://www.national.archsrch.gov.za/sm300cv/smws/sm300dl
* White and Yellow Pages Africa http://www.wayp.com/eng/afrika3.shtml
* White Pages (ZA) http://www.whitepages.co.za/

'''Asia'''

* White and Yellow Pages Asia http://www.wayp.com/eng/asia3.shtml

'''Oceania'''

* New Zealand Web Archive http://natlib.govt.nz/collections/a-z/new-zealand-web-archive
* PANDORA, Australia's Web Archive http://pandora.nla.gov.au/
* White and Yellow Pages South Pacific http://www.wayp.com/eng/southpas3.shtml

'''South America'''

* White and Yellow Pages America http://www.wayp.com/eng/amerika3.shtml
'''North America'''

* National Archives (US) http://www.archives.gov/
* WayBackMachine https://archive.org/web/
* White and Yellow Pages America http://www.wayp.com/eng/amerika3.shtml
* White Pages (US) http://www.whitepages.com/
== Related ==
* [[Psychological warfare]]
* [[Trolling]]
* [[Roleplay]]
* [[Elicitation]]

== References ==

[[Category:How_To]]

Reconnaissance

2015-09-24T09:45:54Z

Lilith2:

The pentesting (jumpy and creative) "reconnaissance" process can be useful for research. What if, for example, we want to know (more about) the current state of [[Timeline_that_is_soooo_1984_...#Smart_Cities|smart cities]]?

== Querying DNS servers ==
The <code>whois</code> system is used by system administrators to obtain contact information for IP address assignments or domain name administrators. <code>dig</code> is a networking tool that can query DNS servers for information. It can be very helpful for diagnosing problems with domain pointing and is a good way to verify that your server configuration is working. An alternative to <code>dig</code> is a command called <code>host</code>. This command functions in a very similar way to dig, with many of the same options. And if <code>dig</code> and <code>whois</code> do not provide you with enough information, tools like <code>dnsmap</code> and <code>dnsenum</code> can be handy.

When you have an IP address an IP lookup will provide details such as ISP name, country, state, city, longitude and latitude. Domain names can help us to find out important information such as address, email id and phone number.

Using whois from the command line you may or may not get useful results. It runs on port 43, and information returned is in plain ASCII format, but because whois servers all over the internet are managed by a wide variety of organisations, information returned may vary. And the different whois clients have different functionality too.

Whois proxies can be used between a client and a server. Those usually use the http or https protocol. If port 43 is blocked, that is not a problem when a client is using proxies through a browser. Also, likely a proxy will determine which server to contact for different lookups.

Almost all services prevent data mining for preventing data gathering for spamming, and that also limits the service for other purposes such as intelligence gathering. Recently, some ISP’s are discussing limiting their service even further.

RWhois (referral whois) is a directory services protocol which extends the whois protocol in a hierarchical and scalable way. It focuses on the distribution of “network objects” (domain names, IP addresses, email addresses) and uses the hierarchical nature of these network objects to more accurately discover the requested information. It is similar to DNS but apparently, still not in general use.

== Enumerating targets ==
Enumerating targets on a local network can be done with <code>nmap</code>, <code>arping</code>, <code>hping</code> and <code>fping</code>. The last three allow for constructing arbitrary packets for almost any networking protocol, for analysis of replies.

== Resources ==
* Hack Back! A DIY Guide for Those Without the Patience to Wait for Whistleblowers http://leaksource.info/2014/08/09/hack-back-a-diy-guide-for-those-without-the-patience-to-wait-for-whistleblowers/

Self-dox

2015-09-24T09:43:21Z

Lilith2: /* Profiling and fingerprinting */

''Doxing is a technique of tracing someone or gathering information about an individual using sources on the internet. Its name is derived from “Documents” or “Docx”. Doxing method is based purely on the ability of the hacker to recognize valuable information about his target and use this information to his benefit. It is also based around the idea that, “The more you know about your target, the easier it will be to find his or her flaws”'' ~ [http://www.urbandictionary.com/define.php?term=doxing Urban dictionary].

Such flaws can then be exploited. The exploitation wildly differs depending on the adversary. For example, profiling information can be used for encapsulation attempts by intelligence agencies. If those attempts are tried and fail and you are still enough of an annoyance to your government, expect [[Psychological warfare|aggressive demonisation by government paid trolls]] (and exceptionalism at its current flourishing best).

Doxing is also a part of reconnaissance where it is called 'human intelligence' or 'humint', and its focus is on its shadow side. The techniques can be simple or complex, and are useful for background checks (infiltrant agents will probably remain invisible with these techniques but littler petty tyrant may be detected) <ref>Investigation Online: Gathering Information to Assess Risk https://modelviewculture.com/pieces/investigation-online-gathering-information-to-assess-risk
</ref>.

There are tools particularly created for d0xing. I use as little automated tools as possible. Not only because of the risk of poisoning, also because I prefer to pay more attention to little-easily-overlooked-details. The more you rely on auto-anything, the more you overlook. But if you want to use a tool and it is unknown to you, please do some research (possibly) follow the money, where the makers are located, for assessing likelihood of the code being poisoned.

This page collects doxxing resources and if you have one, please do add!

[[File:Lulz-vs-lulz.jpg|480px|thumb|right|Doxed: how Sabu was outed by former Anons long before his arrest http://arstechnica.com/tech-policy/2012/03/doxed-how-sabu-was-outed-by-former-anons-long-before-his-arrest/]]

== Storing information during research ==
[[Linux_applications#Storing_information|Store gathered data]] on an encrypted external disk.

== Profiling and fingerprinting ==
=== IP lookup ===

When you have an IP address an IP lookup will provide details such as ISP name, country, state, city, longitude and latitude. Domain names can help us to find out important information such as address, email id and phone number.

Using whois from the command line you may or may not get useful results. It runs on port 43, and information returned is in plain ASCII format, but because whois servers all over the internet are managed by a wide variety of organisations, information returned may vary. And the different whois clients have different functionality too.

Whois proxies can be used between a client and a server. Those usually use the http or https protocol. If port 43 is blocked, that is not a problem when a client is using proxies through a browser. Also, likely a proxy will determine which server to contact for different lookups.

Almost all services prevent data mining for preventing data gathering for spamming, and that also limits the service for other purposes such as intelligence gathering. Recently, some ISP’s are discussing limiting their service even further.

RWhois (referral whois) is a directory services protocol which extends the whois protocol in a hierarchical and scalable way. It focuses on the distribution of “network objects” (domain names, IP addresses, email addresses) and uses the hierarchical nature of these network objects to more accurately discover the requested information. It is similar to DNS but apparently, still not in general use.

== Resources ==

=== Spies online ===

You can use the same (or similar) spies online techniques <ref>Spies online http://www.spiesonline.net/</ref> to see yourself through the eyes of the adversaries that might d0x you. That information can then be used to protect yourself from your adversaries as best as you can. And when setting up other, anonymous or pseudonymous identities, these same resources can be used for test-driven-development of the identity.

=== Elicitation ===
*Social engineer: elicitation http://www.social-engineer.org/framework/influencing-others/elicitation/
* Information Elicitation via Social Engineering http://capec.mitre.org/data/definitions/410.html

=== Search engines ===

* Duck Duck Go https://duckduckgo.com/ (anonymous search)
* IxQuick https://ixquick.com/
* StartPage https://startpage.com/
* YaCy http://yacy.net/en/index.html (distributed search)

=== People ===

* Check Usernames http://checkusernames.com/
* Lullar http://com.lullar.com/
* Peek You http://www.peekyou.com/
* Pipl https://pipl.com/
* Recorded Future https://www.recordedfuture.com/
* Social Mention http://socialmention.com/
* WebMii http://webmii.com/
* Whos Talkin http://www.whostalkin.com/

=== Images ===

* TinEye https://tineye.com/

=== Posts and discussions ===

* Omgili http://omgili.com/

=== IP lookup ===

'''Technical information related to whois'''
* RFC 954 – NICNAME/WHOIS </nowiki>http://www.faqs.org/rfcs/rfc954.html
* RFC 1834 – Whois and Network Information Lookup Service, Whois++ </nowiki>http://www.faqs.org/rfcs/rfc1834.html
'''Whois clients'''
* Linux online manpage jwhois http://linux.die.net/man/1/whois
* Ripe whois client http://sourceforge.net/projects/whois/files/ (does not check for supported flags at the client side, except for -h (whois host) and -p (whois port))
'''Webbased whois lookups'''
* Geektools http://www.geektools.com/whois.php (also serves as proxy)
* Whois at the Open Directory Project http://www.dmoz.org/Computers/Internet/Domain_Names/Name_Search/ with (list of whois servers)
'''Useful whois servers'''
* InterNIC http://www.internic.net/whois.html (the main Internic whois)
* American Registry for Internet Numbers http://whois.arin.net/ (gives info on who owns a netblock)
* Advanced query RADb http://www.radb.net/query/?advanced_query=1 (for routing information from an IP address)
* Robtex http://www.robtex.com/ (goodies in a toolbar)
'''RWhois'''
* RWHOIS project on ARIN http://projects.arin.net/rwhois/
* RWhois Web Interface http://projects.arin.net/rwhois/prwhois.html (a sample rwhois web interface on ARIN using the Net::Rwhois perl5 client)
'''Other interesting lookups'''
* The New Ultimates, mostly US data http://www.newultimates.com/

=== D0xing tools ===

* Sleeping Time http://sleepingtime.org/
* Metagoofil http://www.edge-security.com/metagoofil.php
* theHarvester http://www.edge-security.com/theharvester.php

=== Reuseful archives ===

Such as wandering through archives, yellow pages, phone directories and other possibly useful information made publicly available.

'''Europe'''
* Archives portal europe http://www.archivesportaleurope.net/directory
* Archiwum Dokumentow Elektronicznych (POL) http://www.nac.gov.pl/
* Bundesarchiv, Federal Archives (GER) http://www.bundesarchiv.de/index.html.de
* Central State Electronic Archives of Ukraine http://tsdea.archives.gov.ua/
* The National Archives (UK) https://www.nationalarchives.gov.uk/
* White and Yellow Pages Europe http://www.wayp.com/eng/europa3.shtml
* Офіційний веб-портал Державної архівної служби України http://www.archives.gov.ua/

'''Africa'''

* National Archives of South Africa (NASA) http://www.national.archsrch.gov.za/sm300cv/smws/sm300dl
* White and Yellow Pages Africa http://www.wayp.com/eng/afrika3.shtml
* White Pages (ZA) http://www.whitepages.co.za/

'''Asia'''

* White and Yellow Pages Asia http://www.wayp.com/eng/asia3.shtml

'''Oceania'''

* New Zealand Web Archive http://natlib.govt.nz/collections/a-z/new-zealand-web-archive
* PANDORA, Australia's Web Archive http://pandora.nla.gov.au/
* White and Yellow Pages South Pacific http://www.wayp.com/eng/southpas3.shtml

'''South America'''

* White and Yellow Pages America http://www.wayp.com/eng/amerika3.shtml
'''North America'''

* National Archives (US) http://www.archives.gov/
* WayBackMachine https://archive.org/web/
* White and Yellow Pages America http://www.wayp.com/eng/amerika3.shtml
* White Pages (US) http://www.whitepages.com/
== Related ==
* [[Psychological warfare]]
* [[Trolling]]
* [[Roleplay]]
* [[Elicitation]]

== References ==

[[Category:How_To]]

Self-dox

2015-09-24T09:39:08Z

Lilith2: /* Storing information during research */

''Doxing is a technique of tracing someone or gathering information about an individual using sources on the internet. Its name is derived from “Documents” or “Docx”. Doxing method is based purely on the ability of the hacker to recognize valuable information about his target and use this information to his benefit. It is also based around the idea that, “The more you know about your target, the easier it will be to find his or her flaws”'' ~ [http://www.urbandictionary.com/define.php?term=doxing Urban dictionary].

Such flaws can then be exploited. The exploitation wildly differs depending on the adversary. For example, profiling information can be used for encapsulation attempts by intelligence agencies. If those attempts are tried and fail and you are still enough of an annoyance to your government, expect [[Psychological warfare|aggressive demonisation by government paid trolls]] (and exceptionalism at its current flourishing best).

Doxing is also a part of reconnaissance where it is called 'human intelligence' or 'humint', and its focus is on its shadow side. The techniques can be simple or complex, and are useful for background checks (infiltrant agents will probably remain invisible with these techniques but littler petty tyrant may be detected) <ref>Investigation Online: Gathering Information to Assess Risk https://modelviewculture.com/pieces/investigation-online-gathering-information-to-assess-risk
</ref>.

There are tools particularly created for d0xing. I use as little automated tools as possible. Not only because of the risk of poisoning, also because I prefer to pay more attention to little-easily-overlooked-details. The more you rely on auto-anything, the more you overlook. But if you want to use a tool and it is unknown to you, please do some research (possibly) follow the money, where the makers are located, for assessing likelihood of the code being poisoned.

This page collects doxxing resources and if you have one, please do add!

[[File:Lulz-vs-lulz.jpg|480px|thumb|right|Doxed: how Sabu was outed by former Anons long before his arrest http://arstechnica.com/tech-policy/2012/03/doxed-how-sabu-was-outed-by-former-anons-long-before-his-arrest/]]

== Storing information during research ==
[[Linux_applications#Storing_information|Store gathered data]] on an encrypted external disk.

== Profiling and fingerprinting ==
=== IP lookup ===

When you have an IP address an IP lookup will provide details such as ISP name, country, state, city, longitude and latitude. Domain names can help us to find out important information such as address, email id and phone number.

Using whois from the command line you may or may not get useful results. It runs on port 43, and information returned is in plain ASCII format, but because whois servers all over the internet are managed by a wide variety of organisations, information returned may vary. And the different whois clients have different functionality too.

Whois proxies can be used between a client and a server. Those usually use the http or https protocol. If port 43 is blocked, that is not a problem when a client is using proxies through a browser. Also, likely a proxy will determine which server to contact for different lookups.

Almost all services prevent data mining for preventing data gathering for spamming, and that also limits the service for other purposes such as intelligence gathering. Recently, some ISP’s are discussing limiting their service even further.

RWhois (referral whois) is a directory services protocol which extends the whois protocol in a hierarchical and scalable way. It focuses on the distribution of “network objects” (domain names, IP addresses, email addresses) and uses the hierarchical nature of these network objects to more accurately discover the requested information. It is similar to DNS but apparently, still not in general use.

=== Crunch ===
=== Wyd ===

== Resources ==

=== Spies online ===

You can use the same (or similar) spies online techniques <ref>Spies online http://www.spiesonline.net/</ref> to see yourself through the eyes of the adversaries that might d0x you. That information can then be used to protect yourself from your adversaries as best as you can. And when setting up other, anonymous or pseudonymous identities, these same resources can be used for test-driven-development of the identity.

=== Elicitation ===
*Social engineer: elicitation http://www.social-engineer.org/framework/influencing-others/elicitation/
* Information Elicitation via Social Engineering http://capec.mitre.org/data/definitions/410.html

=== Search engines ===

* Duck Duck Go https://duckduckgo.com/ (anonymous search)
* IxQuick https://ixquick.com/
* StartPage https://startpage.com/
* YaCy http://yacy.net/en/index.html (distributed search)

=== People ===

* Check Usernames http://checkusernames.com/
* Lullar http://com.lullar.com/
* Peek You http://www.peekyou.com/
* Pipl https://pipl.com/
* Recorded Future https://www.recordedfuture.com/
* Social Mention http://socialmention.com/
* WebMii http://webmii.com/
* Whos Talkin http://www.whostalkin.com/

=== Images ===

* TinEye https://tineye.com/

=== Posts and discussions ===

* Omgili http://omgili.com/

=== IP lookup ===

'''Technical information related to whois'''
* RFC 954 – NICNAME/WHOIS </nowiki>http://www.faqs.org/rfcs/rfc954.html
* RFC 1834 – Whois and Network Information Lookup Service, Whois++ </nowiki>http://www.faqs.org/rfcs/rfc1834.html
'''Whois clients'''
* Linux online manpage jwhois http://linux.die.net/man/1/whois
* Ripe whois client http://sourceforge.net/projects/whois/files/ (does not check for supported flags at the client side, except for -h (whois host) and -p (whois port))
'''Webbased whois lookups'''
* Geektools http://www.geektools.com/whois.php (also serves as proxy)
* Whois at the Open Directory Project http://www.dmoz.org/Computers/Internet/Domain_Names/Name_Search/ with (list of whois servers)
'''Useful whois servers'''
* InterNIC http://www.internic.net/whois.html (the main Internic whois)
* American Registry for Internet Numbers http://whois.arin.net/ (gives info on who owns a netblock)
* Advanced query RADb http://www.radb.net/query/?advanced_query=1 (for routing information from an IP address)
* Robtex http://www.robtex.com/ (goodies in a toolbar)
'''RWhois'''
* RWHOIS project on ARIN http://projects.arin.net/rwhois/
* RWhois Web Interface http://projects.arin.net/rwhois/prwhois.html (a sample rwhois web interface on ARIN using the Net::Rwhois perl5 client)
'''Other interesting lookups'''
* The New Ultimates, mostly US data http://www.newultimates.com/

=== D0xing tools ===

* Sleeping Time http://sleepingtime.org/
* Metagoofil http://www.edge-security.com/metagoofil.php
* theHarvester http://www.edge-security.com/theharvester.php

=== Reuseful archives ===

Such as wandering through archives, yellow pages, phone directories and other possibly useful information made publicly available.

'''Europe'''
* Archives portal europe http://www.archivesportaleurope.net/directory
* Archiwum Dokumentow Elektronicznych (POL) http://www.nac.gov.pl/
* Bundesarchiv, Federal Archives (GER) http://www.bundesarchiv.de/index.html.de
* Central State Electronic Archives of Ukraine http://tsdea.archives.gov.ua/
* The National Archives (UK) https://www.nationalarchives.gov.uk/
* White and Yellow Pages Europe http://www.wayp.com/eng/europa3.shtml
* Офіційний веб-портал Державної архівної служби України http://www.archives.gov.ua/

'''Africa'''

* National Archives of South Africa (NASA) http://www.national.archsrch.gov.za/sm300cv/smws/sm300dl
* White and Yellow Pages Africa http://www.wayp.com/eng/afrika3.shtml
* White Pages (ZA) http://www.whitepages.co.za/

'''Asia'''

* White and Yellow Pages Asia http://www.wayp.com/eng/asia3.shtml

'''Oceania'''

* New Zealand Web Archive http://natlib.govt.nz/collections/a-z/new-zealand-web-archive
* PANDORA, Australia's Web Archive http://pandora.nla.gov.au/
* White and Yellow Pages South Pacific http://www.wayp.com/eng/southpas3.shtml

'''South America'''

* White and Yellow Pages America http://www.wayp.com/eng/amerika3.shtml
'''North America'''

* National Archives (US) http://www.archives.gov/
* WayBackMachine https://archive.org/web/
* White and Yellow Pages America http://www.wayp.com/eng/amerika3.shtml
* White Pages (US) http://www.whitepages.com/
== Related ==
* [[Psychological warfare]]
* [[Trolling]]
* [[Roleplay]]
* [[Elicitation]]

== References ==

[[Category:How_To]]

Linux applications

2015-09-24T09:38:12Z

Lilith2:

[[File:Repository2.jpg|640px|thumb|right|Caek repository]]

Linux distributions include great software by default and their repositories. And with so many flavours of linux, finding the right application for getting things done can be tough. Hence my proposal to share interesting apps. The page was kicked off with a short explanation on installing, and a few categories. Add your 2 cents! If a fitting category is not available, create it.

For each application its license is mentioned if found (others will still have to be added). For more on licensing see [[Captivating capital and copyfighting]].

There are many ways to install applications, via graphical front-ends and via the command-line. You can use all of them. It used to be a problem when apt-get did not track which packages were automatically installed while aptitude did, but now that both packages share this list, there is no reason to avoid switching back and forth between the two or between one and front-ends related to the other.
== Graphical front-ends ==
The Gnome PackageKit is one of two graphical package managers installed out of the box if and when you installed GNOME as desktop manager in Ubuntu. In older GNOME you can access it from the menu at ''Applications > System Tools > Add/Remove Software. ''PackageKit is designed to unify all the software graphical tools used in different distributions. It abstracts the various underlying package management technologies like yum, apt, smart etc. and provides unified graphical and command line frontends.

The Software Center is the graphical package manager developed by and default available on Ubuntu. You can access it from ''Applications > System Tools > Administration > Software Center'' in GNOME. It is a very high-level, new-user-friendly GUI. It will hide most of the packaging details and will only present installed/available applications. Software is nicely categorized so that, if you're not exactly sure what app you want, you can find what you need quickly. The Software Center also allows you to purchase commercial applications, just in case you wish to roleplay "consumer".

[[File:Software-center.png|640px||thumb|center]]

Update-manager is a simple front-end for installing security updates and other daily upgrades (especially useful if you are running testing versions). When the upgrade is too complicated for update-manager, it will suggest to run synaptic.

[[File:Synaptic.png|640px||thumb|center]]

Synaptic used to be the graphical package managers on debian-based distributions. You can start it from the menu: ''Applications > System Tools > Administration > Synaptic Package Manager'' in GNOME or ''System -> Administration -> Synaptic Package Manager'' in MATE.

Aptitude is a console-based GUI APT front-end. Run aptitude from the command-line without parameter, and it will start a powerful console-based GUI. Much like synaptic, you can have multiple views of the installed/available packages and mark packages for installation/upgrade/removal/purge before executing everything at once. It is very similar to apt-get and on occasion seems to deal better with odd and unexpected dependency situations. So if you have a problem with that for a particular application, you can try aptitude.

[[File:Aptitude.png|640px||thumb|center]]

== Command-line based package managers and apt front-ends ==
In debian-based distributions, installing and managing software from the command-line can be done with apt-get and other components of the Advanced Packaging Tool (APT). For example, to install an application like Ubuntu’s Software Center, you can do:

$ sudo apt-get install software-center

You can find out more by reading the man page. Just type ''man apt-get'' on the command-line. It's a handy tool if you know the exact package name of what you want to install and don't want to spend time clicking through a GUI to get it.

Some noteworthy alternatives to apt-get are cupt, wajig and xbps. Cupt wraps several tools including apt-get, and wajig is a (partial) APT reimplementation. Xbps was written from scratch.
=== xbps ===
The X Binary Package System (in short XBPS) is a binary package system designed and implemented from scratch. Its goal is to be fast, easy to use, bug-free, featureful and portable as much as possible. The XBPS code is totally compatible with POSIX/SUSv2/C99 standards, and released with a Simplified BSD license (2 clause). There is a well documented API provided by the XBPS Library that is the basis for its frontends to handle binary packages and repositories. It is used in the voidlinux distro.

https://github.com/voidlinux/xbps/blob/master/README.md
=== apt ===
Apt makes collection of software available to the user and does the dirty work of downloading all the required packages and installing them by calling dpkg in the correct order to respect the dependencies. Apt is a library and several frontends have been developed on top of that library, of which apt-get is one. I not works, check internet connection.

For installing (multiple) packages:

$ sudo apt-get install packagename1 packagename2 packagename3

If a package is already installed, apt will try to upgrade it to the latest version. If it isn't already installed, apt-get will also try to install any software needed to install or run it, the so-called dependencies.

For removing an installed package (except its configuration files):

$ sudo apt-get remove packagename

For cleaning out the configuration files as well as the package:

$ sudo apt-get purge packagename

For resynchronising package index files and upgrading (including security updates):

<pre>$ sudo apt-get update
$ sudo apt-get upgrade</pre>

You can also let apt-get handle changing dependencies with new versions of packages; apt-get has a "smart" conflict resolution system, and it will attempt to upgrade the most important packages at the expense of less important ones if necessary if you use:

<pre>$ sudo apt-get update
$ sudo apt-get dist-upgrade</pre>

For doing a "dry run" of a procedure in order to get an idea of what an action will do, you can pass the <code>-s</code> ('''s'''imulate) flag:

$ sudo apt-get install -s packagename

=== dpkg ===
Package managers like apt-get, aptitude, synaptic, and the Ubuntu Software Center are all just front-ends to either dpkg or apt, which is in itself a front-end to dpkg. This is the low-level tool that takes a .deb file and extracts its content on the disk, or that takes the name of a package to remove the associated files, etc.

For installing a <code>.deb</code> package use the command with <code>-i</code> option:

# dpkg -i packagename.deb

Listing all installed packages:

# dpkg -l

For removing a package (but not the configuration files):

# dpkg -r packagename

For removing both configuration and package files:

# dpkg -p packagename

For showing whether a deb package is installed or not:

# dpkg -s packagename

== Verification with checksums ==
For more on checksum concepts see [[Encrypting_everything#Checksums|encrypting everything: checksums]].

=== Package managers ===
Integrity checks are integrated in debian package managers. Never ever continue with installation if you get a:

WARNING: The following packages cannot be authenticated!

0. It may be that your system is set to use a proxy server for a network you are not connected to. Set your Network Proxy method to None and Apply System Wide, and check if that helped.

1. Check repositories are not corrupted in <code>/etc/apt/sources.list</code>. Try again.

2. Check GPG keys (apt-key)
# apt-key list
3. Update the local keyring with the keyring of archive keys and remove from the keyring the archive keys which are no longer valid:

<pre># apt-key update
# apt-get update</pre>

4. If still not works, reinstall your archive-keyring.

# aptitude reinstall [distro]-archive-keyring

* Kali Keyring – kali-archive-keyring
* Debian Keyring – debian-archive-keyring
* Ubuntu Keyring – ubuntu-archive-keyring
5. Still not works? Contact us on IRC.

=== Download software ===
And when downloading software, .iso's and .deb's and the like, download the xxxsum (choose from what is available) as well as the software and check. For example do:

$ md5sum some-file.iso

or

$ sha256sum some-file.iso

and compare the output from the command with the key listed in the associated md5 or sha256 file.

== Compiling source code ==
Source code usually comes in the form of compressed tar files (.tar.gz or .tar.bz2 extensions). Tools used for packing the source code into these tar balls are <code>tar</code> (used for combining multiple files into one) and <code>gzip</code> or <code>bzip2</code> (used for compression). To fetch the source code tarball for a particular software you need to know the URL to the tarball.

1. Get the url of the tarball

2. Fetch it using <code>wget</code> or <code>curl</code> (both download it in current directory):
$ wget tarballurl

3. Unpack the tarball in order to get access to the source code and other files:
$ tar -zxvf tarballname.tar.gz
or
$ tar -zxvf tarballname.tar.bz2

4. If present, the documentation in the extracted directories provides information about the software, changes since last version, links to more documentation, information regrading the author of the software, steps for compilation and installation of software etc. Name of the files (and case) may differ but usually the information is divided into two files, a "Readme" and an "Install". The last covers all the information required for compilation and installation, and all the other information is covered in the "Readme" file.

5. Most source code packages also come with a configuration script that can be used for configuring the environment. Again, names may differ, but usually it is named <code>configure</code> and accepts parameters used to control features of the software. The script also makes sure all the tools required for compilation are present in the system. Most have a "help". Try:

$ configure --help

For configuring the build environment, execute the <code>configure</code> script:

$ ./configure

If the script does not crash, throw a fit, or runs off into a black hole, it creates a <code>makefile</code>.

6. The <code>makefile</code> is used in the compilation of the software:

$ make

If compilation encounters a problem an error is thrown on the console. If compilation successfully jumped through all of the hoops, the binaries are created.

7. Install the binaries in the standard paths so they can be invoked from anywhere in the filesystem (this may require root privileges):

# make install

8. Remove the directory created while unpacking the software tarball.

== Stealth install of applications ==
* Launch Synaptic on the off-line computer you wish to install software packages on.
* Mark the packages you wish to install.
* Choose File-> Generate package download script.
* Save the script to your USB stick.
* Take the USB stick to an online linux computer and run the script on the USB stick. It will download only the packages required by the offline computer to the USB stick.
* Insert the USB stick into the offline computer.
* Launch Synaptic and click on File-> Add downloaded packages
* Select the directory on your USB stick containing the downloaded *.deb files and click Open. The packages will be installed.

== Transferring application data to another distribution == 
If you set up a separate <code>/home</code> during partitioning for your current system, when you perform the new installation, you can tell the installer to reformat the first partition <code>/</code> (to start from scratch), but leave the second partition alone and just mount it again to <code>/home</code>. Then, all you need to do is make sure that you set up the same username and password as before, and everything should be back to the way it was (later you can change them).

If you did not, go into <code>/home/user</code> and backup all personal and application data on for example, an external disk. Your personal files are probably all visible and easily copied. For the application data, use the View menu in your file manager to set Show hidden files and copy all <code>.application</code> directories you wish to keep. Mind dependencies. Especially keys. Those may need exporting and copy too. Then after installing the other distro or re-installing or upgrading the existing linux, put the <code>.application</code> directories in the new <code>/home/user</code> directory.

The only thing you’d still have to do is reinstall your applications. For many applications, the settings will be picked up on. Import the exported keys.

Word of warning: For some switches, like from Ubuntu to Fedora, this (either way) may not work well, even if both are using Gnome. Incompatabilities may arise. Good, that way we have to "sort it out" and dive into the peculiarities of the new distro immediately. :) Also, configurations in system files are not transferred (unless you thought of those and your external disk had enough space).

== Shell ==
Computers understand the language of zeros and ones known as binary language. If not familiar with number conversions, a decent tutorial can be found in http://www.cstutoringcenter.com/tutorials/general/convert.php In the early days of computing, instructions were provided using binary language, which is difficult for all of us to read and write. In linux, the shell accepts human readable commands and translates them into something the kernel can read and process.

* The shell is a user program or it is an environment provided for user interaction.
* It is a command language interpreter that executes commands read from the standard input device such as keyboard or from a file.
* The shell gets started when you log in or open a console (terminal).
* Quick and dirty way to execute utilities.
* The shell is not part of system kernel, but uses the system kernel to execute programs, create files etc.
* Common shells available for linux are:
** <code>bash</code> ( Bourne-Again SHell ) - Most common shell in Linux. It's Open Source.
** <code>csh</code> (C SHell) - The C shell's syntax and usage are very similar to the C programming language.
** <code>ksh</code> (Korn SHell) - Created by David Korn at AT & T Bell Labs. The Korn Shell also was the base for the POSIX Shell standard specifications.
** <code>tcsh</code> - an enhanced but completely compatible version of the Berkeley UNIX C shell (CSH).

Each shell does the same job, but each understands different command syntax and provides different built-in functions. The Kinky linux command-line page is based on <code>bash</code>.
=== Shells ===
==== zsh ====
Zsh is a shell designed for interactive use, although it is also a powerful scripting language. Many of the useful features of bash, ksh, and tcsh were incorporated into zsh; many original features were added. The introductory document details some of the unique features of zsh. It assumes basic knowledge of the standard UNIX shells; the intent is to show a reader already familiar with one of the other major shells what makes zsh more useful or more powerful. This document is not at all comprehensive; read the manual entry for a description of the shell that is complete, concise and up-to-date, although somewhat overwhelming and devoid of examples. Alternatively, the user guide offers wordy explanations of many of the shell's features.

* Website: http://zsh.sourceforge.net/
* Documentation: http://zsh.sourceforge.net/Doc/
* Tips, tricks and examples: http://grml.org/zsh/zsh-lovers.html
* Available from repository.

== Graphical environment ==
From the bottom up:
* Xorg, XFree86 and X11 are display servers https://en.wikipedia.org/wiki/Display_server. This creates the graphical environment.
* [gkxsw]dm, lxdm, lightdm and nodm are display managers alias login managers https://wiki.debian.org/DisplayManager. This is the first X program run by the system if the system (not the user) is starting X and allows you to log on to the local system, or network systems.
* A window manager controls the placement and decoration of windows http://xwinman.org/intro.php. That is, the window border and controls are the decoration. Some of these are stand alone (WindowMaker, sawfish, fvwm, etc). Some depend on an accompanying desktop environment.
* A desktop environment such as XFCE, CINNAMON, MATE, KDE or GNOME are a suite of applications designed to integrate well with each other to provide a consistent experience http://www.linux.org/resources/categories/linux-desktop-environments.22/.
* A terminal emulator, terminal application, term, or tty for short, is a program that emulates a video terminal within some other display architecture.

In theory (and mostly so in practice) any of those components are interchangeable.

=== Window managers ===
==== openbox ====
Openbox is included in most popular linux distributions. It can be run within GNOME and KDE. And can also be used without a desktop environment (the lightweight approach). You can install additional features yourself, making it very customisable. It does not work well straight out of the box. You will probably need to install tools for power mangement, networking, managing displays, etc. Openbox comes with a default autostart which sets up an environment for both GNOME and KDE applications to run properly in (if you have them installed), as well as providing support for SCIM language input. See the autostart documentation for more details on setting it up. It is excellent for spending some experimentation time on, gaining insights in how window managers work and how else they could work.

* Website: http://openbox.org/wiki/Main_Page
* Autostart documentation http://openbox.org/wiki/Help:Autostart
* FAQ: http://openbox.org/wiki/Help:FAQ

==== awesome ====
Awesome (''GNU GPLv2'') is a configurable framework window manager for X. It is primarly targeted at power users, developers and any people dealing with every day computing tasks and who want to have fine-grained control on their graphical environment using the Lua programming language.

* Website: http://awesome.naquadah.org/
* Documentation: http://awesome.naquadah.org/wiki/Main_Page
* Community: http://awesome.naquadah.org/community/ (IRC and mailinglists)

=== Terminal emulator managers ===
==== terminator ====
Terminator (GPL-2.0+) is a useful tool for arranging terminals, inspired by programs such as gnome-multi-term, quadkonsole, etc. in that the main focus is arranging terminals in grids (tabs is the most common default method, which Terminator also supports).

* Website: https://www.openhub.net/p/gnome-terminator
* Available from repository.

==== tmux ====
tmux is a terminal multiplexer and lets you switch easily between several programs in one terminal, detach them (they keep running in the background) and reattach them to a different terminal.

* Website: http://tmux.github.io/
* Available from repository. Requirements: libevent and ncurses.

=== GUI-based file managers ===
==== xfe ====
X File Explorer (Xfe) is an M$-Explorer or Commander like file manager for X. It is based on the popular, but discontinued, X Win Commander, originally developed by Maxim Baranov. Xfe aims to be the file manager of choice for all light thinking Unix addicts! By default it presents a traditional tree view on the left and file panel on the right, but three other modes are available -- two panels, tree and two panels, or a single panel. ''Open With'' and ''Associate'' options are available from the right-click menu. Although it does not have a lot of the advanced features of Konqueror or Nautilus, it should be more than sufficient for the average user. Its speed and familiarity make it a good choice for older systems, newer users, or those of us who'd run filer.exe under Wine if it could handle file permissions.

* Website: http://roland65.free.fr/xfe/

==== gentoo ====
Gentoo is the GTK+ file manager (the name is not related to the Gentoo linux distribution) using a two-pane layout with icon associations for most file extensions. Gentoo also relies on a button bar, located below the file panes, for most operations, but each button may have a secondary function, accessed from the middle mouse button. A second, smaller button bar to the left provides shortcuts to specific directories. The configuration menu is not very extensive, and you can change options with the mouse. Gentoo also provides click-to-sort functionality in the file panes, mount and unmount capabilities, and display of file permissions by color. You can add and edit buttons and give them a specific color and tool tip.

* Wesite: http://www.obsession.se/gentoo/

==== gcmd ====
GNOME Commander offers a small footprint and a polished GUI. It provides the more typical tool bar and menu system, in addition to mapping buttons to each function key. It also provides a command history, capacity for multiple predefined FTP sessions, filesearch, advanced rename tool - allowing for quick renaming of multiple files using regular expressions, counters, and case matching. A compact, polished file manager, especially suitable for those running the GNOME desktop.

* Website: http://gcmd.github.io/

=== Desktop file managers ===
==== spacefm ====
SpaceFM is a multi-panel tabbed file and desktop manager for Linux with built-in VFS, udev- or HAL-based device manager, customisable menu system, and bash integration. SpaceFM aims to provide a stable, capable file manager with significant customisation capabilities. SpaceFM & udevil '''can be used completely without systemd, consolekit, policykit, dbus, udisks, gvfs & fuse''' (although it can coexist with any of these).

* Website: http://ignorantguru.github.io/spacefm/

== System tools ==
=== System monitoring ===
==== conky ====
Conky is a free, light-weight system monitor for X, that displays any information on your desktop. Conky is licensed under the ''GPL'' and runs on Linux and BSD.

* Website: http://conky.sourceforge.net/
* Documentation: http://conky.sourceforge.net/documentation.html

== Research ==
=== Storing information ===
Only use those fitting your desktop manager, or else you'll end up with a spaghetti distro.
==== Basket (KDE) ====
* Website http://basket.kde.org/

==== Tomboy (GNOME) ====
* Website https://github.com/GNOME/tomboy
==== KeepNote ====
Windows, Linux, and MacOS X

* Website http://keepnote.org/

== Security applications ==
=== Anti-malware ===
==== rkhunter ====
Rootkit Hunter (GNU GPL) is a Unix-based tool that scans for rootkits, backdoors and possible local exploits. Specifically, rkhunter is a shell script which carries out various checks on the local system to try and detect known rootkits and malware. It also performs checks to see if commands have been modified, if the system startup files have been modified, and various checks on the network interfaces, including checks for listening applications. rkhunter has been written to be as generic as possible, and so should run on most Linux and UNIX systems. It is provided with some support scripts should certain commands be missing from the system, and some of these are Perl scripts.

* Website: http://rkhunter.sourceforge.net/
* FAQ: http://rkhunter.cvs.sourceforge.net/viewvc/rkhunter/rkhunter/files/FAQ
* Mailing list: http://sourceforge.net/p/rkhunter/mailman/
* Available from repository. Requirements: Bourne Again Shell (Bash) and exim

==== chrootkit ====
chkrootkit (Free software) is a tool to locally check for signs of a rootkit. It tests the following applications: aliens, asp, bindshell, lkm, rexedcs, sniffer, w55808, wted, scalper, slapper, z2, chkutmp, amd, basename, biff, chfn, chsh, cron, crontab, date, du, dirname, echo, egrep, env, find, fingerd, gpm, grep, hdparm, su, ifconfig, inetd, inetdconf, identd, init, killall, ldsopreload, login, ls, lsof, mail, mingetty, netstat, named, passwd, pidof, pop2, pop3, ps, pstree, rpcinfo, rlogind, rshd, slogin, sendmail, sshd, syslogd, tar, tcpd, tcpdump, top, telnetd, timed, traceroute, vdir, w, and write.

* Website: http://www.chkrootkit.org/
* README: http://chkrootkit.org/README
* FAQ: http://chkrootkit.org/faq/
* Available from repository.

=== Encryption ===
==== gnupg ====
GnuPG (GNU GPL v3+) stands for GNU Privacy Guard and is a tool for secure communication and data storage. The software has two main uses. The first is to encrypt data to ensure its privacy. The second is to "sign" data so that others can determine it is authentic and unmodified. It includes an advanced key management facility and is compliant with the proposed OpenPGP Internet standard as described in RFC 2440. GnuPG is a command line tool with features for easy integration with other applications. The default public key algorithms are DSA and Elgamal, but RSA is also supported. Symmetric algorithms available are AES (with 128, 192, and 256 bit keys), 3DES, Blowfish, CAST5 and Twofish. Digest algorithms available are MD5, RIPEMD/160, SHA-1, SHA-256, SHA-384, and SHA-512. Compression algorithms available are ZIP, ZLIB, and BZIP2 (with libbz2 installed).

* Website: https://www.gnupg.org/
* Documentation: https://www.gnupg.org/documentation/index.html (includes guides, FAQ's and mailinglists)
* Available from repository.

==== mcrypt ====
mcrypt (GPLv2) is a replacement for the old crypt package and crypt command, with extensions. It allows developers to use a wide range of encryption functions, without making drastic changes to their code. It allows users to encrypt files or data streams without having to be cryptographers. The companion to mCrypt is libmcrypt, which contains the actual encryption functions themselves, and provides a standardized mechanism for accessing them.

* Website http://mcrypt.sourceforge.net/
* Available from repository.
==== steghide ====
steghide (GNU GPL) is a steganography program that is able to hide data in various kinds of image- and audio-files. Steghide employs an algorithm which is undetectable by color-frequency based statistical tests. Steghide uses a graph-theoretic approach to steganography. The default encryption algorithm is Rijndael with a key size of 128 bits (which is AES - the advanced encryption standard) in the cipher block chaining mode.

* Website: http://steghide.sourceforge.net/
* Documentation: http://steghide.sourceforge.net/documentation.php (english and spanish)
* Manual: http://steghide.sourceforge.net/documentation/manpage.php (english and spanish)
* Mailing lists: http://sourceforge.net/p/steghide/mailman/
* Available from repository. Requirements: libmhash, libmcrypt, libjpeg, zlib

==== stunnel ====
Stunnel (GNU GPL v2) is a program that allows you to encrypt arbitrary TCP connections inside SSL (Secure Sockets Layer). It allows for securing non-SSL aware daemons and protocols (like POP, IMAP, LDAP, etc) by having Stunnel provide the encryption, requiring no changes to the daemon's code. Stunnel does not contain any crytographic code itself -- instead it relies on external SSL libraries.

* Website: https://www.stunnel.org/index.html
* Documentation: http://www.stunnel.org/docs.html (english and polish)
* HowTo: http://www.stunnel.org/howto.html (Installation and Authentication)
* FAQ: http://www.stunnel.org/faq.html (Troubleshooting and Applications)
* Available from repository as stunnel4.

==== veracrypt ====
VeraCrypt is disk encryption software forked from the discontinued TrueCrypt software. VeraCrypt adds enhanced security to the algorithms used for system and partitions encryption, and solves many vulnerabilities and security issues found in TrueCrypt.

* Website: https://veracrypt.codeplex.com/
* Documentation: https://veracrypt.codeplex.com/documentation
==== ciphershed ====
CipherShed (will carry an OSI approved license (probably either Apache or BSD)) is free (as in free-of-charge and free-speech) encryption software and was started in June 2014 as a response to the end of life announcement for TrueCrypt. As of October 2014 CipherShed source code is hosted at GitHub. CipherShed is a program that can be used to create encrypted files or encrypt entire drives (including USB flash drives and external HDDs). There’s no complicated commands or knowledge required; a simple wizard guides you step-by-step through every process. After creating an encrypted file or disk drive, the encrypted volume is mounted through CipherShed. The mounted volume shows up as a regular disk that can be read and written to on-the-fly. The encryption is transparent to the operating system and any programs. When finished, the volume can be unmounted, and stored or transported elsewhere, fully secured. Encryption volumes can be moved from OS-to-OS (eg, Windows to Mac) with full compatibility.

* Website: https://ciphershed.org/
* Forum: https://forum.ciphershed.org/

=== Firewalls ===
==== gufw and ufw ====
Gufw (''GNU GPL v3'') is for users bamboozled by firewalls. It has an easy to use interface for setting up inbound and outbound traffic rules for apps/services and ports. It is designed for beginners. Gufw is a GUI front-end to <code>ufw</code>, which is in itself a front-end to <code>netfilter</code> and <code>iptables</code>

* Website: http://gufw.org/

==== netfilter and iptables ====
Netfilter (''GNU GPL v2'') is a framework that provides a set of hooks inside the Linux kernel that allows kernel modules to register callback functions with the network stack. A registered callback function is then called back for every packet that traverses the respective hook within the network stack. Software inside this framework enables packet filtering, network address [and port] translation (NA[P]T) and other packet mangling. iptables is a generic table structure for the definition of rulesets. Each rule within an IP table consists of a number of classifiers (iptables matches) and one connected action (iptables target).

iptables (''GNU GPL v2'') is a user space application program that allows a system administrator to configure the Netfilter tables, chains, and rules. The iptables package also includes ip6tables. ip6tables is used for configuring the IPv6 packet filter.

* Website: http://www.netfilter.org/
* HOWTOs http://www.netfilter.org/documentation/index.html#documentation-howto
* FAQ http://www.netfilter.org/documentation/index.html#documentation-faq
* Mailing lists http://www.netfilter.org/mailinglists.html

=== Intrusion detection ===
One of the key differences between Tripwire and Aide is their commercial status. Tripwire was originally a free, open source product and is now a commercial product but a free version (branched in 2000) is still being developed at http://sourceforge.net/projects/tripwire/. Aide is entirely open source and licensed via the GPL and much, much easier in its configuration. Tripwire offers extended features that make it more secure than Aide, but storing aide databases on an encrypted filesystem on another machine (or an external drive) can be considered good enough to not need tripwire's extended feature(s) and opt for aide instead. Not that you can't sign the databases AND put them on another machine or external drive.

==== aide ====
The Advanced Intrusion Detection Environment (''GPL''), for short ''aide'', is a file integrity checker (FIA). It creates a database from the regular expression rules that it finds from the config file(s). Once this database is initialized it can be used to verify the integrity of the files. It has several message digest algorithms (md5, sha1, rmd160, tiger, crc32, sha256, sha512, whirlpool (additionally with libmhash: gost, haval, crc32b)) that are used to check the integrity of the file. All of the usual file attributes (File type, Permissions, Inode, Uid, Gid, Link name, Size, Block count, Number of links, Mtime, Ctime and Atime) can also be checked for inconsistencies. It can read databases from older or newer versions.

* Website: http://aide.sourceforge.net
* Manual: http://aide.sourceforge.net/stable/manual.html
* Available from repository.
* Example install and configuration of aide on debian: [[Linux security#Installing intrusion detection|Linux security: Installing intrusion detection]]

==== tripwire ====
Open Source Tripwire is a free software security and data integrity tool useful for monitoring and alerting on specific file change(s) on a range of systems. Open Source Tripwire functions as a host-based intrusion detection system. It is suitable for monitoring a small number of servers where centralized control and reporting is not needed and professional support is not a requirement.

* Website: http://www.tripwire.com
* Mailinglists: http://sourceforge.net/p/tripwire/mailman/
* Available from repository.
* Example install and configuration of tripwire on debian: [[Linux security#Installing intrusion detection|Linux security: Installing intrusion detection]] and [[Linux server security#Installing intrusion detection|Linux server security: Installing intrusion detection]]

=== Traffic capture ===
=== Traffic analysis ===
==== kismet ====
Kismet (''GNU GPL'') is an 802.11 layer2 wireless network detector, sniffer, and intrusion detection system. Kismet will work with any wireless card which supports raw monitoring (rfmon) mode, and can sniff 802.11b, 802.11a, and 802.11g traffic. Kismet identifies networks by passively collecting packets and detecting standard named networks, detecting (and given time, decloaking) hidden networks, and inferring the presence of nonbeaconing networks via data traffic.

* Website: http://www.kismetwireless.net/
* Documentation: http://www.kismetwireless.net/documentation.shtml
* Forums: http://www.kismetwireless.net/Forum/General/
* Available from repository. Requirements: Libpcap, GPSD, Imagemagick, Expat, GMP. '''Optional: DBUS''' (but not required)
==== wireshark ====
Wireshark (''GNU GPL v2'') is a network packet analyzer. A network packet analyzer captures network packets and tries to display that packet data as detailed as possible. A network packet analyzer can be regarded as a measuring device to examine what's going on inside a network cable, just like a voltmeter is used by an electrician to examine what's going on inside an electric cable (but at a higher level, of course). It is developed and maintained by a global team of protocol experts. It used to be known as Ethereal, and was renamed to Wireshark in May 2006.

* Website: http://www.wireshark.org/
* Documentation: https://www.wireshark.org/docs/
* Available from repository.

=== Packet crafting ===
==== hping ====
Hping3 (''GNU GPL v2'') is a command-line oriented TCP/IP packet assembler/analyser. The interface is inspired from the ping unix command, but hping isn't only able to send ICMP echo requests. It supports TCP, UDP, ICMP and RAW-IP protocols, has a traceroute mode, the ability to send files between a covered channel, and many other features. hping3 can handle fragmentation, and almost arbitrary packet size and content, using the command line interface.

* Website: http://www.hping.org/
* Documentation: http://www.hping.org/documentation.php
* Available from repository. Requirements: libpcap, Tcl/Tk (Optional)

==== scapy ====
Scapy (''GNU GPL v2'') is a powerful interactive packet manipulation program. It is able to forge or decode packets of a wide number of protocols, send them on the wire, capture them, match requests and replies, etc. It can handle most classical tasks like scanning, tracerouting, probing, unit tests, attacks or network discovery (it can replace hping, 85% of nmap, arpspoof, arp-sk, arping, tcpdump, tethereal, p0f, etc.).

* Website: http://www.secdev.org/projects/scapy/
* Mailinglist: http://news.gmane.org/gmane.comp.security.scapy.general
* Available from repository. Requirements: python.

=== Port scanning ===
==== ipscan ====
Angry IP Scanner (''GNU GPL v2'') is an open-source and cross-platform network scanner designed to be fast and simple to use. It scans IP addresses and ports.

* Website: http://angryip.org/
* Documentation: http://angryip.org/documentation/
* FAQ: http://angryip.org/faq/

=== Vulnerability scanning ===
==== tiger ====
Tiger (''GNU GPL'') is a security tool that can be use both as a security audit and intrusion detection system. It is a set of scripts that scan a Un*x system looking for security problems, in the same fashion as Dan Farmer's COPS. It was originally developed to provide a check of UNIX systems on the A&M campus that want to be accessed from off campus (clearance through the packet filter). Unlike other tools, Tiger needs only of POSIX tools and is written entirely in shell language.

* Website: http://www.nongnu.org/tiger/
* README: http://cvs.savannah.gnu.org/viewvc/*checkout*/tiger/tiger/README?content-type=text%2Fplain&revision=HEAD
* HOWTO: http://cvs.savannah.gnu.org/viewvc/*checkout*/tiger/tiger/USING?content-type=text%2Fplain&revision=HEAD
* Project page: http://savannah.nongnu.org/projects/tiger
* Available from repository.

=== Data removal ===
==== bleachbit ====
BleachBit deletes unnecessary files to free valuable disk space, maintain privacy, and remove junk. It removes cache, Internet history, temporary files, cookies, and broken shortcuts. It handles cleaning of Adobe Reader, Bash, Beagle, Epiphany, Firefox, Flash, GIMP, Google Earth, Java, KDE, OpenOffice.org, Opera, RealPlayer, rpmbuild, Second Life Viewer, VIM, XChat, and more. Beyond simply erasing junk files, BleachBit wipes free disk space (to hide previously deleted files for privacy and to improve compression of images), vacuums Firefox databases (to improve performance without deleting data), and securely shreds arbitrary files.

* Website: http://bleachbit.sourceforge.net/
* Documentation: http://bleachbit.sourceforge.net/documentation
* Forums: http://bleachbit.sourceforge.net/forum
* Available from repository.

=== Password management ===
==== keepassx ====
KeePassX (''GNU GPL v2'') is a multi-platform port of KeePass, an open source and cross-platform password manager. This utility helps you to manage your passwords in a secure way. You can put all your passwords in one database, which is locked with one master key or a key-disk. This lets users only need to remember one single master password or insert the key-disk to unlock the whole database. The databases are encrypted using the algorithms AES (alias Rijndael) or Twofish using a 256 bit key. KeePassX currently uses same the database file format as the KeePass 1.x (Classic) password manager software for Windows.

* Website: http://www.keepassx.org/
* FAQ: http://www.keepassx.org/faq
* Available from repository. Requirements: Qt library 4.3 or higher, libXtst, X11 with XRandR, Xinerama, and XTest extensions, glibc 2.4

=== VPN ===
==== openvpn ====
OpenVPN (GNU GPL v2) is a full-featured virtual private network (VPN) solution accomodating a wide range of configurations, including remote access, site-to-site VPNs, WiFi security, and enterprise-scale remote access solutions with load balancing, failover, and fine-grained access-controls. OpenVPN implements OSI layer 2 or 3 secure network extension using the industry standard SSL/TLS protocol, supports flexible client authentication methods based on certificates, smart cards, and/or 2-factor authentication, and allows user or group-specific access control policies using firewall rules applied to the VPN virtual interface.

* Website: https://openvpn.net/
* FAQ: https://community.openvpn.net/openvpn/wiki/FAQ
* HOWTO: https://openvpn.net/index.php/open-source/documentation/howto.html
* Available via repository. Requirements: TUN and/or TAP driver to allow user-space programs to control a virtual point-to-point IP or Ethernet device

== Desktop applications ==
=== Graphics editors ===
==== GIMP ====
The GNU Image Manipulation Program (''GNU GPL v2'') is a freely distributed raster graphic editor for photo retouching, image composition and image authoring. It can be used as a simple paint program, an expert quality photo retouching program, an online batch processing system, a mass production image renderer, an image format converter, etc. It also makes the task of screenshots easy. It is designed to be augmented with plug-ins and extensions. The advanced scripting interface allows simple tasks as well as complex image manipulation procedures to scripted.

* Website: http://www.gimp.org/
* Wiki: http://wiki.gimp.org/index.php/Main_Page
* Available from repositories.
* Requirements: gegl and GTK+ (dbus and gvfs are optional)
* GIMP configuration http://www.linuxfromscratch.org/blfs/view/svn/xsoft/gimp.html

==== Inkscape ====
Inkscape (''GNU GPL v2'') is a vector graphics editor with capabilities similar to Illustrator, Freehand, CorelDraw, and Xara X. The interface is designed to conform to GNOME standards. It uses the W3C standard Scalable Vector Graphics (SVG) file format, which is an open, industry-standard XML-based format for vector graphics developed by the W3C organisation. The popularity of this format is growing fast.

Supported SVG features include shapes, paths, text, markers, clones, alpha blending, transforms, gradients, patterns, and grouping. Inkscape also supports Creative Commons meta-data, node editing, layers, complex path operations, bitmap tracing, text-on-path, flowed text, direct XML editing, and more. It imports formats such as JPEG, PNG, TIFF, and others and exports PNG as well as multiple vector-based formats.

* Website: http://www.inkscape.org/
* Community: https://inkscape.org/en/community/
* Requirements: GTK+ version 2.8.x, gtkmm2 => 2.4, boehm-gc Garbage Collection Libraries => 6.4

==== Xara Xtreme ====
Xara (''GPL'') is a general purpose graphics program for Unix platforms including Linux, FreeBSD and (in development) OS X. It is a port of Xara's flagship software package, renowed for being the fastest vector package available, combined with an efficient working environment. May need a bit of tweaking to install.

* Website: http://www.xaraxtreme.org/
* Community: http://www.xaraxtreme.org/community.html

=== Image processing ===
==== gm ====
GraphicsMagick (''MIT X11 style license'') provides a collection of tools and libraries which support reading, writing, and manipulating an image in over 88 major formats including important formats like DPX, GIF, JPEG, JPEG-2000, PNG, PDF, PNM, and TIFF. Similar as imagemagick.

* Website: http://www.graphicsmagick.org/index.html
* FAQ: http://www.graphicsmagick.org/FAQ.html

== Minimalist applications ==
=== Text editors ===
==== vi and vim ====
Yes, even if you can't believe it, there are a lot fans of the 30-years-old vi editor (or its more recent, just-15-years-old, best clone & great improvement, vim). No, they are not dinosaurs who don't want to catch up with the times - the community of vi users just keeps growing. Yes, there are definite reasons why the vi/vim editing model is just superior to any other out there. And you don't need to be a Unix whiz to use it, either: vim is available for free for almost any platform out there, and there are plug-ins to get the functionality inside all major IDEs. <ref>Why, oh WHY, do those #?@! nutheads use vi? http://www.viemu.com/a-why-vi-vim.html</ref>

* Cheatsheets: vi http://www.lagmonster.org/docs/vi.html, vim http://zzapper.co.uk/vimtips.html
* Editing MediaWiki articles with VIM http://blog.bigsmoke.us/2009/07/10/vim-mediawiki-editing
* Available from repository.

=== Console-based file managers ===
Servers often boot to run level 3, which provides no X display, but a good file manager can be crucial to administration.

==== mc ====
Midnight Commander is one of the best-known console file managers. Midnight Commander allows you to view two directories, a directory with file attributes, or a directory and file preview at the same time. On curses-based systems file types and permissions are shown by color, but you can specify "slow" and "ASCII-only" display modes from the command line. These are especially valuable for slow connections and on very old systems. Midnight Commander's help system is well-written, and invaluable, as Ctrl and Alt key combinations make up the bulk of functions. Pull-down menus are also available, and some functions are mapped to function keys, listed at the bottom of the display.

Midnight Commander supports FTP, console mouse, and file undeletion (on ext2 filesystems only). Many versions are available for both the console and X Windows. Midnight Commander's consistency between console and X file managers is a strong asset if you'd rather not learn two file managers, or run a console application in an xterm window.

* Website: http://www.midnight-commander.org/
* Documentation: http://www.midnight-commander.org/wiki/doc

==== vifm ====
Vifm lets you cut with dd, paste with p, and rename a file with cw. Movement keys are the standard <code>vi</code> hjkl, with h and l ascending and descending the directory tree. The right panel toggles between a display of a second directory and file properties. You can call shell commands from the command line with :!, with %f and %F corresponding to highlighted files in the left and right pane. Vifm also mirrors its treatment of bookmarks, visual file selection, and user-defined macros from vi. Most importantly, :help opens an extensive help file with more options. Like Midnight Commander vifm displays file properties by color, and it can set permissions and ownership; unlike Midnight Commander it does not offer file preview, although pressing Enter or l on a file will open it in vi. :apropos brings up a menu of all matching man pages, from which you can invoke man for the selected entry. If you like vi, try vifm.

* Website: http://vifm.info/
* Documentation: http://vifm.info/vimdoc.shtml
* Manual: http://vifm.info/manual.shtml

==== ranger ====
Ranger is a terminal based file manager with vi style keybindings and many features including 'rifle' file opener, previewing, tabs, bookmarks, tagging. ranger is not a two pane file manager like mc or vifm, but uses 'miller columns' showing the directory hierarchy (similar to mac os 'finder'). The centre column is active, the left column the higher level directory, and the right column shows either a deeper directory or file information or preview.

* Website: http://ranger.nongnu.org/

=== Websearch ===
==== surfraw ====
Shell Users' Revolutionary Front Rage Against the Web, surfraw for short, provides a fast unix command line interface to a variety of popular WWW search engines and other artifacts of power. It reclaims google, altavista, babelfish, dejanews, freshmeat, research index, slashdot and many others from the false-prophet, pox-infested heathen lands of html-forms, placing these wonders where they belong, deep in unix heartland, as god loving extensions to the shell. Surfraw abstracts the browser away from input. Doing so lets it get on with what it's good at. Browsing. Interpretation of linguistic forms is handed back to the shell, which is what it, and human beings are good at. Combined with netscape-remote or incremental text browsers, such as lynx, links or w3m, along with screen a surfraw liberateur is capable of navigating speeds that leave GUI tainted idolaters agape with fear and wonder.

* Website: http://surfraw.alioth.debian.org/
* Usage: http://surfraw.alioth.debian.org/#usage
* List of elvi: http://surfraw.alioth.debian.org/#elvilist
* Available from repository.

==== vimperator firefox extension ====
Vimperator is a Firefox browser extension with strong inspiration from the Vim text editor, with a mind towards faster and more efficient browsing. It has similar key bindings and you could call it a modal web browser, as key bindings differ according to which mode you are in. For example, it has a special Hint mode, where you can follow links easily with the keyboard only. Also most functionality is available as commands, typing :back will go back within the current page history, just like hitting the back button in the toolbar. In case you wish to control the web browser without using the mouse.

* Website: http://www.vimperator.org/vimperator (download link to .xpi on page)

=== Email ===
==== mutt ====
Mutt is a small but very powerful text-based mail client for Unix operating systems.

* Website: http://www.mutt.org/
* FAQ: http://www.fefe.de/muttfaq/faq.html
* Wiki: http://dev.mutt.org/trac/wiki
* Mutt sidebar patch for mutt with folders listed in a sidebar http://www.lunar-linux.org/mutt-sidebar/
* Despising quiche mutt guide http://realprogrammers.com/jump_start/mutt/
* Getting Up and Running With a Sane Mutt Setup http://blog.developwithpassion.com/2013/05/02/getting-up-and-running-with-a-sane-mutt-setup/

==== mu ====
With the enormous amounts of e-mail many people gather and the importance of e-mail messages in our daily work-flow, it is very important to be able to quickly deal with all that - in particular, to instantly find that one important e-mail you need right now. For that, mu was created. mu is a tool for dealing with e-mail messages stored in the Maildir-format, on Unix-like systems. mu's main purpose is to help you to find the messages you need, quickly; in addition, it allows you to view messages, extract attachments, create new maildirs,...

* Website: http://www.djcbsoftware.nl/code/mu/

==== notmuch ====
Notmuch is a program for searching email. If you've been looking for a fast, global-search and tag-based email system to use within your text editor or in a terminal...If you're the kind of person that gets excited about being able to write shell scripts for exploring and manipulating your email...If you're a developer of an existing email program and would love a good library interface for fast, global search with support for arbitrary tags...If you want the convenience of fast, reliable search of all your email, but don't want to give a 3rd-party access to your email...Then notmuch may be exactly what you've been looking for.

* Website: http://notmuchmail.org/

== Library ==
== Development ==
=== Development environments ===
==== vagrant ====

Vagrant is for creating and configuring lightweight, reproducible, and portable development environments.

* Website: https://www.vagrantup.com/
* Documentation: http://docs.vagrantup.com/v2/getting-started/index.html

== Server administration ==
=== File managers ===
==== sshfs ====

SSH Filesystem is a filesystem client based on the SSH File Transfer Protocol. Since most SSH servers already support this protocol it is very easy to set up: i.e. on the server side there's nothing to do. On the client side mounting the filesystem is as easy as logging into the server with ssh.

* Website: http://fuse.sourceforge.net/sshfs.html
* FAQ: http://sourceforge.net/p/fuse/wiki/SshfsFaq/
* Documentation: http://sourceforge.net/p/fuse/wiki/Main_Page/

=== Intrusion detection ===
There are two major types of Intrusion Detection Systems: network-based and host-based. Tripwire is host-based and Snort is a network-based IDS/IPS. Two side of a coin working in tandem.

==== tripwire ====
Open Source Tripwire is a free software security and data integrity tool useful for monitoring and alerting on specific file change(s) on a range of systems. Open Source Tripwire functions as a host-based intrusion detection system. It is suitable for monitoring a small number of servers where centralized control and reporting is not needed and professional support is not a requirement.

* Website: http://www.tripwire.com
* Mailinglists: http://sourceforge.net/p/tripwire/mailman/
* Available from repository.

==== snort ====
Snort (''GNU GPL v2'') is an open source network intrusion prevention and detection system sing a rule-driven language, which combines the benefits of signature, protocol and anomaly based inspection methods. It can perform real-time traffic analysis, alerting, blocking and packet logging on IP networks. It utilizes a combination of protocol analysis and pattern matching in order to detect anomalies, misuse and attacks. It detects a variety of attacks and probes, such as buffer overflows, stealth port scans, CGI attacks, SMB probes, OS fingerprinting attempts, and much more. Snort uses a flexible rules language to describe activity that can be considered malicious or anomalous as well as an analysis engine that incorporates a modular plugin architecture. Snort is capable of detecting and responding in real-time, sending alerts, performing session sniping, logging packets, or dropping sessions/packets when deployed in-line.

* Website: http://www.snort.org/
* User's manual: http://manual.snort.org/
* Community: https://www.snort.org/community
* FAQ: https://github.com/vrtadmin/snort-faq/blob/master/README.md
* Available from repository. Requirements: libpcap

=== Network monitoring ===
==== nagios ====
Nagios (GNU GPL v2) is a host and service monitor designed to inform you of network problems. The monitoring daemon runs periodic checks on hosts and services specified using external "plugins" which return status information to Nagios. When problems are encountered, the daemon can send notifications out to administrative contacts in a variety of different ways (email, instant message, SMS, etc.). Current status information, historical logs, and reports can all be accessed via a web browser. Although Nagios is powerful and flexible, it does require some time for it to be installed and configured correctly.

* Website: http://www.nagios.org/
* Documentation: https://www.nagios.org/documentation/
* Debian tutorials: http://www.debianhelp.co.uk/nagios.htm
* Mailinglists: http://sourceforge.net/p/nagios/mailman/
* Available from repository. Requirements: C compiler

==== tcpdump ====
Tcpdump is a common computer network debugging tool that runs under the command line. It allows the user to intercept and display TCP/IP and other packets being transmitted or received over a network to which the computer is attached. This program allows you to dump the traffic on a network. It can be used to print out the headers of packets on a network interface, filter packets that match a certain expression. You can use this tool to track down network problems, to detect "ping attacks" or to monitor network activities.

* Website: http://www.tcpdump.org/
* Available from repository. Requirements: libpcap

=== Traffic analysis ===
==== tcptrace ====

Tcptrace (''GNU GPL'') is a tool designed for analysis of TCP dump files. It can tell you detailed information about TCP connections by sifting through dump files. Specifically, tcptrace can produce several different types of output containing information on each connection seen, such as elapsed time, bytes and segments sent and recieved, retransmissions, round trip times, window advertisements, throughput, and more. It can also produce a number of graphs for further analysis.

* Website: http://www.tcptrace.org/
* Manual: http://www.tcptrace.org/manual.html
* FAQ: http://www.tcptrace.org/faq.html
* Mailinglists: http://www.tcptrace.org/maillist.html

=== System tools ===
==== ncdu ====

NCurses Disk Usage (''MIT License'') is a disk usage analyzer with an ncurses interface. It is designed to find space hogs on a remote server where you don't have an entire gaphical setup available, but it is a useful tool even on regular desktop systems. Ncdu aims to be fast, simple and easy to use, and should be able to run in any minimal POSIX-like environment with ncurses installed.

* Website: http://dev.yorhel.nl/ncdu
* Manual: http://dev.yorhel.nl/ncdu/man

=== Backups ===
==== rsync ====

Rsync (''GPL v3'') is an open source utility that provides fast incremental file transfer. Rsync is a file-copying tool which can copy locally and to/from a remote host. It offers many options to control its behavior, and its remote-update protocol can minimize network traffic to make transferring updates between machines fast and efficient. It is widely used for backups and mirroring and as an improved copy command for everyday use. This package provides both the rsync command line tool and optional daemon functionality.

* Website: http://rsync.samba.org/
* Documentation: http://rsync.samba.org/documentation.html

==== unison ====
Unison (''GPL v3'') is a file-synchronization tool for OSX, Unix, and Windows. Unison is written in OCaml. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other. Unison can run on and synchronize between Windows and many UNIX platforms. Unison requires no root privileges, system access or kernel changes to function. Unison can synchronize changes to files and directories in both directions, on the same machine, or across a network using ssh or a direct socket connection. Transfers are optimised using a version of the rsync protocol, making it ideal for slower links. Unison has a clear and precise specification, and is resilient to failure due to its careful handling of the replicas and its private structures.

* Website: http://www.cis.upenn.edu/~bcpierce/unison/
* User manual: http://www.cis.upenn.edu/~bcpierce/unison/docs.html
* Mailinglists: http://www.cis.upenn.edu/~bcpierce/unison/lists.html

== Services ==
=== Mailing lists ===
==== mailman ====
Mailman is free software for managing electronic mail discussion and e-newsletter lists. Mailman is integrated with the web, making it easy for users to manage their accounts and for list owners to administer their lists.

* Website: http://www.gnu.org/software/mailman/index.html

== Related ==
* [[Installing linux]]
* [[Linux security]]
* [[Kinky linux command-line]]
* [[Anonymising your traffic with linux]]
* [[Clean up all teh things]]
* [[Shell scripting]]
* [[Encrypting everything]]

== References ==

[[Category:How_To]]

Self-dox

2015-09-24T09:24:50Z

Lilith2: /* Storing information during research */

''Doxing is a technique of tracing someone or gathering information about an individual using sources on the internet. Its name is derived from “Documents” or “Docx”. Doxing method is based purely on the ability of the hacker to recognize valuable information about his target and use this information to his benefit. It is also based around the idea that, “The more you know about your target, the easier it will be to find his or her flaws”'' ~ [http://www.urbandictionary.com/define.php?term=doxing Urban dictionary].

Such flaws can then be exploited. The exploitation wildly differs depending on the adversary. For example, profiling information can be used for encapsulation attempts by intelligence agencies. If those attempts are tried and fail and you are still enough of an annoyance to your government, expect [[Psychological warfare|aggressive demonisation by government paid trolls]] (and exceptionalism at its current flourishing best).

Doxing is also a part of reconnaissance where it is called 'human intelligence' or 'humint', and its focus is on its shadow side. The techniques can be simple or complex, and are useful for background checks (infiltrant agents will probably remain invisible with these techniques but littler petty tyrant may be detected) <ref>Investigation Online: Gathering Information to Assess Risk https://modelviewculture.com/pieces/investigation-online-gathering-information-to-assess-risk
</ref>.

There are tools particularly created for d0xing. I use as little automated tools as possible. Not only because of the risk of poisoning, also because I prefer to pay more attention to little-easily-overlooked-details. The more you rely on auto-anything, the more you overlook. But if you want to use a tool and it is unknown to you, please do some research (possibly) follow the money, where the makers are located, for assessing likelihood of the code being poisoned.

This page collects doxxing resources and if you have one, please do add!

[[File:Lulz-vs-lulz.jpg|480px|thumb|right|Doxed: how Sabu was outed by former Anons long before his arrest http://arstechnica.com/tech-policy/2012/03/doxed-how-sabu-was-outed-by-former-anons-long-before-his-arrest/]]

== Storing information during research ==
Store gathered data on an encrypted external disk.

=== Basket (KDE) ===
http://basket.kde.org/

=== Tomboy (GNOME) ===

=== KeepNote ===

== Profiling and fingerprinting ==
=== IP lookup ===

When you have an IP address an IP lookup will provide details such as ISP name, country, state, city, longitude and latitude. Domain names can help us to find out important information such as address, email id and phone number.

Using whois from the command line you may or may not get useful results. It runs on port 43, and information returned is in plain ASCII format, but because whois servers all over the internet are managed by a wide variety of organisations, information returned may vary. And the different whois clients have different functionality too.

Whois proxies can be used between a client and a server. Those usually use the http or https protocol. If port 43 is blocked, that is not a problem when a client is using proxies through a browser. Also, likely a proxy will determine which server to contact for different lookups.

Almost all services prevent data mining for preventing data gathering for spamming, and that also limits the service for other purposes such as intelligence gathering. Recently, some ISP’s are discussing limiting their service even further.

RWhois (referral whois) is a directory services protocol which extends the whois protocol in a hierarchical and scalable way. It focuses on the distribution of “network objects” (domain names, IP addresses, email addresses) and uses the hierarchical nature of these network objects to more accurately discover the requested information. It is similar to DNS but apparently, still not in general use.

=== Crunch ===
=== Wyd ===

== Resources ==

=== Spies online ===

You can use the same (or similar) spies online techniques <ref>Spies online http://www.spiesonline.net/</ref> to see yourself through the eyes of the adversaries that might d0x you. That information can then be used to protect yourself from your adversaries as best as you can. And when setting up other, anonymous or pseudonymous identities, these same resources can be used for test-driven-development of the identity.

=== Elicitation ===
*Social engineer: elicitation http://www.social-engineer.org/framework/influencing-others/elicitation/
* Information Elicitation via Social Engineering http://capec.mitre.org/data/definitions/410.html

=== Search engines ===

* Duck Duck Go https://duckduckgo.com/ (anonymous search)
* IxQuick https://ixquick.com/
* StartPage https://startpage.com/
* YaCy http://yacy.net/en/index.html (distributed search)

=== People ===

* Check Usernames http://checkusernames.com/
* Lullar http://com.lullar.com/
* Peek You http://www.peekyou.com/
* Pipl https://pipl.com/
* Recorded Future https://www.recordedfuture.com/
* Social Mention http://socialmention.com/
* WebMii http://webmii.com/
* Whos Talkin http://www.whostalkin.com/

=== Images ===

* TinEye https://tineye.com/

=== Posts and discussions ===

* Omgili http://omgili.com/

=== IP lookup ===

'''Technical information related to whois'''
* RFC 954 – NICNAME/WHOIS </nowiki>http://www.faqs.org/rfcs/rfc954.html
* RFC 1834 – Whois and Network Information Lookup Service, Whois++ </nowiki>http://www.faqs.org/rfcs/rfc1834.html
'''Whois clients'''
* Linux online manpage jwhois http://linux.die.net/man/1/whois
* Ripe whois client http://sourceforge.net/projects/whois/files/ (does not check for supported flags at the client side, except for -h (whois host) and -p (whois port))
'''Webbased whois lookups'''
* Geektools http://www.geektools.com/whois.php (also serves as proxy)
* Whois at the Open Directory Project http://www.dmoz.org/Computers/Internet/Domain_Names/Name_Search/ with (list of whois servers)
'''Useful whois servers'''
* InterNIC http://www.internic.net/whois.html (the main Internic whois)
* American Registry for Internet Numbers http://whois.arin.net/ (gives info on who owns a netblock)
* Advanced query RADb http://www.radb.net/query/?advanced_query=1 (for routing information from an IP address)
* Robtex http://www.robtex.com/ (goodies in a toolbar)
'''RWhois'''
* RWHOIS project on ARIN http://projects.arin.net/rwhois/
* RWhois Web Interface http://projects.arin.net/rwhois/prwhois.html (a sample rwhois web interface on ARIN using the Net::Rwhois perl5 client)
'''Other interesting lookups'''
* The New Ultimates, mostly US data http://www.newultimates.com/

=== D0xing tools ===

* Sleeping Time http://sleepingtime.org/
* Metagoofil http://www.edge-security.com/metagoofil.php
* theHarvester http://www.edge-security.com/theharvester.php

=== Reuseful archives ===

Such as wandering through archives, yellow pages, phone directories and other possibly useful information made publicly available.

'''Europe'''
* Archives portal europe http://www.archivesportaleurope.net/directory
* Archiwum Dokumentow Elektronicznych (POL) http://www.nac.gov.pl/
* Bundesarchiv, Federal Archives (GER) http://www.bundesarchiv.de/index.html.de
* Central State Electronic Archives of Ukraine http://tsdea.archives.gov.ua/
* The National Archives (UK) https://www.nationalarchives.gov.uk/
* White and Yellow Pages Europe http://www.wayp.com/eng/europa3.shtml
* Офіційний веб-портал Державної архівної служби України http://www.archives.gov.ua/

'''Africa'''

* National Archives of South Africa (NASA) http://www.national.archsrch.gov.za/sm300cv/smws/sm300dl
* White and Yellow Pages Africa http://www.wayp.com/eng/afrika3.shtml
* White Pages (ZA) http://www.whitepages.co.za/

'''Asia'''

* White and Yellow Pages Asia http://www.wayp.com/eng/asia3.shtml

'''Oceania'''

* New Zealand Web Archive http://natlib.govt.nz/collections/a-z/new-zealand-web-archive
* PANDORA, Australia's Web Archive http://pandora.nla.gov.au/
* White and Yellow Pages South Pacific http://www.wayp.com/eng/southpas3.shtml

'''South America'''

* White and Yellow Pages America http://www.wayp.com/eng/amerika3.shtml
'''North America'''

* National Archives (US) http://www.archives.gov/
* WayBackMachine https://archive.org/web/
* White and Yellow Pages America http://www.wayp.com/eng/amerika3.shtml
* White Pages (US) http://www.whitepages.com/
== Related ==
* [[Psychological warfare]]
* [[Trolling]]
* [[Roleplay]]
* [[Elicitation]]

== References ==

[[Category:How_To]]

Self-dox

2015-09-24T09:24:13Z

Lilith2: /* Dradis */

''Doxing is a technique of tracing someone or gathering information about an individual using sources on the internet. Its name is derived from “Documents” or “Docx”. Doxing method is based purely on the ability of the hacker to recognize valuable information about his target and use this information to his benefit. It is also based around the idea that, “The more you know about your target, the easier it will be to find his or her flaws”'' ~ [http://www.urbandictionary.com/define.php?term=doxing Urban dictionary].

Such flaws can then be exploited. The exploitation wildly differs depending on the adversary. For example, profiling information can be used for encapsulation attempts by intelligence agencies. If those attempts are tried and fail and you are still enough of an annoyance to your government, expect [[Psychological warfare|aggressive demonisation by government paid trolls]] (and exceptionalism at its current flourishing best).

Doxing is also a part of reconnaissance where it is called 'human intelligence' or 'humint', and its focus is on its shadow side. The techniques can be simple or complex, and are useful for background checks (infiltrant agents will probably remain invisible with these techniques but littler petty tyrant may be detected) <ref>Investigation Online: Gathering Information to Assess Risk https://modelviewculture.com/pieces/investigation-online-gathering-information-to-assess-risk
</ref>.

There are tools particularly created for d0xing. I use as little automated tools as possible. Not only because of the risk of poisoning, also because I prefer to pay more attention to little-easily-overlooked-details. The more you rely on auto-anything, the more you overlook. But if you want to use a tool and it is unknown to you, please do some research (possibly) follow the money, where the makers are located, for assessing likelihood of the code being poisoned.

This page collects doxxing resources and if you have one, please do add!

[[File:Lulz-vs-lulz.jpg|480px|thumb|right|Doxed: how Sabu was outed by former Anons long before his arrest http://arstechnica.com/tech-policy/2012/03/doxed-how-sabu-was-outed-by-former-anons-long-before-his-arrest/]]

== Storing information during research ==
Store gathered data on an encrypted external disk.

=== Basket (KDE) ===
http://basket.kde.org/

=== Tomboy (GNOME) ===

== Profiling and fingerprinting ==
=== IP lookup ===

When you have an IP address an IP lookup will provide details such as ISP name, country, state, city, longitude and latitude. Domain names can help us to find out important information such as address, email id and phone number.

Using whois from the command line you may or may not get useful results. It runs on port 43, and information returned is in plain ASCII format, but because whois servers all over the internet are managed by a wide variety of organisations, information returned may vary. And the different whois clients have different functionality too.

Whois proxies can be used between a client and a server. Those usually use the http or https protocol. If port 43 is blocked, that is not a problem when a client is using proxies through a browser. Also, likely a proxy will determine which server to contact for different lookups.

Almost all services prevent data mining for preventing data gathering for spamming, and that also limits the service for other purposes such as intelligence gathering. Recently, some ISP’s are discussing limiting their service even further.

RWhois (referral whois) is a directory services protocol which extends the whois protocol in a hierarchical and scalable way. It focuses on the distribution of “network objects” (domain names, IP addresses, email addresses) and uses the hierarchical nature of these network objects to more accurately discover the requested information. It is similar to DNS but apparently, still not in general use.

=== Crunch ===
=== Wyd ===

== Resources ==

=== Spies online ===

You can use the same (or similar) spies online techniques <ref>Spies online http://www.spiesonline.net/</ref> to see yourself through the eyes of the adversaries that might d0x you. That information can then be used to protect yourself from your adversaries as best as you can. And when setting up other, anonymous or pseudonymous identities, these same resources can be used for test-driven-development of the identity.

=== Elicitation ===
*Social engineer: elicitation http://www.social-engineer.org/framework/influencing-others/elicitation/
* Information Elicitation via Social Engineering http://capec.mitre.org/data/definitions/410.html

=== Search engines ===

* Duck Duck Go https://duckduckgo.com/ (anonymous search)
* IxQuick https://ixquick.com/
* StartPage https://startpage.com/
* YaCy http://yacy.net/en/index.html (distributed search)

=== People ===

* Check Usernames http://checkusernames.com/
* Lullar http://com.lullar.com/
* Peek You http://www.peekyou.com/
* Pipl https://pipl.com/
* Recorded Future https://www.recordedfuture.com/
* Social Mention http://socialmention.com/
* WebMii http://webmii.com/
* Whos Talkin http://www.whostalkin.com/

=== Images ===

* TinEye https://tineye.com/

=== Posts and discussions ===

* Omgili http://omgili.com/

=== IP lookup ===

'''Technical information related to whois'''
* RFC 954 – NICNAME/WHOIS </nowiki>http://www.faqs.org/rfcs/rfc954.html
* RFC 1834 – Whois and Network Information Lookup Service, Whois++ </nowiki>http://www.faqs.org/rfcs/rfc1834.html
'''Whois clients'''
* Linux online manpage jwhois http://linux.die.net/man/1/whois
* Ripe whois client http://sourceforge.net/projects/whois/files/ (does not check for supported flags at the client side, except for -h (whois host) and -p (whois port))
'''Webbased whois lookups'''
* Geektools http://www.geektools.com/whois.php (also serves as proxy)
* Whois at the Open Directory Project http://www.dmoz.org/Computers/Internet/Domain_Names/Name_Search/ with (list of whois servers)
'''Useful whois servers'''
* InterNIC http://www.internic.net/whois.html (the main Internic whois)
* American Registry for Internet Numbers http://whois.arin.net/ (gives info on who owns a netblock)
* Advanced query RADb http://www.radb.net/query/?advanced_query=1 (for routing information from an IP address)
* Robtex http://www.robtex.com/ (goodies in a toolbar)
'''RWhois'''
* RWHOIS project on ARIN http://projects.arin.net/rwhois/
* RWhois Web Interface http://projects.arin.net/rwhois/prwhois.html (a sample rwhois web interface on ARIN using the Net::Rwhois perl5 client)
'''Other interesting lookups'''
* The New Ultimates, mostly US data http://www.newultimates.com/

=== D0xing tools ===

* Sleeping Time http://sleepingtime.org/
* Metagoofil http://www.edge-security.com/metagoofil.php
* theHarvester http://www.edge-security.com/theharvester.php

=== Reuseful archives ===

Such as wandering through archives, yellow pages, phone directories and other possibly useful information made publicly available.

'''Europe'''
* Archives portal europe http://www.archivesportaleurope.net/directory
* Archiwum Dokumentow Elektronicznych (POL) http://www.nac.gov.pl/
* Bundesarchiv, Federal Archives (GER) http://www.bundesarchiv.de/index.html.de
* Central State Electronic Archives of Ukraine http://tsdea.archives.gov.ua/
* The National Archives (UK) https://www.nationalarchives.gov.uk/
* White and Yellow Pages Europe http://www.wayp.com/eng/europa3.shtml
* Офіційний веб-портал Державної архівної служби України http://www.archives.gov.ua/

'''Africa'''

* National Archives of South Africa (NASA) http://www.national.archsrch.gov.za/sm300cv/smws/sm300dl
* White and Yellow Pages Africa http://www.wayp.com/eng/afrika3.shtml
* White Pages (ZA) http://www.whitepages.co.za/

'''Asia'''

* White and Yellow Pages Asia http://www.wayp.com/eng/asia3.shtml

'''Oceania'''

* New Zealand Web Archive http://natlib.govt.nz/collections/a-z/new-zealand-web-archive
* PANDORA, Australia's Web Archive http://pandora.nla.gov.au/
* White and Yellow Pages South Pacific http://www.wayp.com/eng/southpas3.shtml

'''South America'''

* White and Yellow Pages America http://www.wayp.com/eng/amerika3.shtml
'''North America'''

* National Archives (US) http://www.archives.gov/
* WayBackMachine https://archive.org/web/
* White and Yellow Pages America http://www.wayp.com/eng/amerika3.shtml
* White Pages (US) http://www.whitepages.com/
== Related ==
* [[Psychological warfare]]
* [[Trolling]]
* [[Roleplay]]
* [[Elicitation]]

== References ==

[[Category:How_To]]