Difference between revisions of "Linux development process"
From Gender and Tech Resources
m (→Threading carefully: init scripts) |
m (→Threading carefully: init scripts) |
||
Line 55: | Line 55: | ||
== Threading carefully: init scripts == | == Threading carefully: init scripts == | ||
− | + | In Unix-based computer operating systems, init (short for initialization) is the first process started during booting of the computer system. Init is a daemon process that continues running until the system is shut down. It is the direct or indirect ancestor of all other processes and automatically adopts all orphaned processes. Init is started by the kernel using a hard-coded filename; a kernel panic will occur if the kernel is unable to start it. Init is typically assigned process identifier 1 (PID 1). | |
+ | |||
+ | Several replacement init implementations have been created, attempting to address design limitations in the standard versions. These include launchd, the Service Management Facility, systemd and Upstart; as of March 2015, systemd has been adopted by major Linux distributions although it remains controversial http://ewontfix.com/14/: | ||
''So how should init be done right?'' | ''So how should init be done right?'' | ||
Line 61: | Line 63: | ||
* ''The Unix way: with simple self-contained programs that do one thing and do it well. Get everything out of PID 1.'' | * ''The Unix way: with simple self-contained programs that do one thing and do it well. Get everything out of PID 1.'' | ||
* ''The systemd way https://people.debian.org/~stapelberg/docs/systemd-dependencies.html: Take advantage of special properties of pid 1 to the maximum extent possible. This leads to ever-expanding scope creep and exacerbates all of the problems described in http://ewontfix.com/14/ (and probably many more yet to be discovered).'' | * ''The systemd way https://people.debian.org/~stapelberg/docs/systemd-dependencies.html: Take advantage of special properties of pid 1 to the maximum extent possible. This leads to ever-expanding scope creep and exacerbates all of the problems described in http://ewontfix.com/14/ (and probably many more yet to be discovered).'' | ||
+ | * ''The right way: Do away with everything special about pid 1 by making pid 1 do nothing but start the real init script and then just reap zombies (see script by Rich Felker below)'' | ||
− | + | === PID 1 === | |
− | + | ||
− | === | + | |
''There has been some interest in having a proper free software license on the trivial init code included below. I originally considered it too trivial to even care about copyright or need a license on it, but I don't want this to keep anyone from using or reusing it, so I'm explicitly licensing it under the standard MIT license'' ~ Rich Felker | ''There has been some interest in having a proper free software license on the trivial init code included below. I originally considered it too trivial to even care about copyright or need a license on it, but I don't want this to keep anyone from using or reusing it, so I'm explicitly licensing it under the standard MIT license'' ~ Rich Felker |
Revision as of 19:18, 21 July 2015
Contents
A typical linux distribution
Linux distributions aren’t just the Linux kernel. They all contain other critical software, like the Grub bootloader, Bash shell, GNU shell utilities, daemons, X.org graphical server, a desktop environment, and more.
Bootloader
The BIOS or UEFI firmware loads the software from a boot device. The first program that loads with any operating system is the boot loader. In linux this is generally the Grub boot loader handling the process of actually booting linux, issuing command-line options and allowing for booting in other ways for troubleshooting purposes.
The linux kernel
The software Grub boots is the kernel, the core of the system. It manages CPU, memory, and input/output devices like keyboard, mice, and displays. The kernel speaks directly to the hardware and it includes many hardware drivers.
Start up daemons
Daemons are background processes (system-level processes you generally don’t notice). They often start as part of the boot process, so they’re one of the next things that loads after the kernel and before you see your graphical login screen.
Shell
Most Linux systems use the Bash shell by default. A shell provides a command processor interface, allowing you to control your computer by typing commands at a text interface, see Kinky linux command-line. Shells can also run shell scripts, which are a collection of commands and operations run in the order specified in the script, see Shell scripting.
Shell utilities
Commands as critical as the cp
command for copying a file, ls
command for listing files in a directory, and rm
command for deleting files are part of the GNU Core Utilities package. The Bash shell is also part of the GNU project. Not all shell utilities and command-line programs are developed by the GNU project. Some commands and terminal programs have their own projects.
X server
The graphical desktop is provided by a type of package known as an "X server" implementing the "X window system". The most used graphical server is X.org. It interfaces with video card, monitor, mouse, and other devices but doesn’t provide a full desktop environment, just a graphical system that desktop environments and toolkits can be developed on.
Desktop environment
A Linux desktop is a desktop environment. Ubuntu includes the Unity desktop environment, Fedora includes GNOME, Kubuntu includes KDE, Mint generally includes Cinnamon or MATE, and Debian gives a variety of choices during install. These desktop environments provide everything you see — the desktop background, panels, window title bars and borders and have their own utilities developed to fit in with the desktop environment.
Desktop programs
Not every desktop program is a part of a particular desktop environment. For example, Firefox and Icedove are not tied to a particular desktop environment. You can run any Linux desktop program in any desktop environment, but it can (and most likely will drag in other processes). For example, if you tried to install the Nautilus file manager on KDE, it will attempt to install a variety of GNOME libraries and probably start GNOME desktop processes in the background when you open it. It would work, you can use it, at a performance price. For a list of growing wishes see Linux applications.
Upstream and downstream
From the perspective of a linux distro, upstream usually refers to the original author(s) of the source code for a particular package. Downstream may refer to a distribution that has forked/branched from the perspective of the distro you are looking at.
So if Debian receives a bug relating to Apache, then a package maintainer or anyone reading the bugs may do some investigation. If they determine the bug to be in the original source code, and not caused by something unique to the distro they will work with the upstream authors to work on fixing the bug in program.
If the upstream developer of the troublesome package is uncooperative or went awol, then the investigating developer may add a patch to the package. This can be seen as a (hopefully temporary) fork of upstream development. Once bug and maintainer are found and bug is solved, Debian may work with downstream distros like Ubuntu to make sure the bug is fixed everywhere. Downstream maintainers can also submit issues related to things having to do with a packaging or any patches added.
If we are working on something like mint, which is based on ubuntu, which in turn is based on debian, the term upstream could mean any one of the maintainers of mint, ubuntu and debian, or it could refer to the original author(s) of a package. When maintainers and author(s) of packages are working well together usually all of them will be subscribed to similar lists, and bugtrackers and anyone with some responsibility for a given package will be notified in some form about bugs and new patches made at any level.
Kernel development
On occasion code is merged into the upstream kernel, the kernel tree managed by Linus Torvalds and available from kernel.org.
Merging source code upstream takes effort, dedication and time. The threshold for getting source code accepted into the Linux kernel is relatively high. If bad quality code got accepted, maintaining the linux kernel would become much more difficult.
Threading carefully: init scripts
In Unix-based computer operating systems, init (short for initialization) is the first process started during booting of the computer system. Init is a daemon process that continues running until the system is shut down. It is the direct or indirect ancestor of all other processes and automatically adopts all orphaned processes. Init is started by the kernel using a hard-coded filename; a kernel panic will occur if the kernel is unable to start it. Init is typically assigned process identifier 1 (PID 1).
Several replacement init implementations have been created, attempting to address design limitations in the standard versions. These include launchd, the Service Management Facility, systemd and Upstart; as of March 2015, systemd has been adopted by major Linux distributions although it remains controversial http://ewontfix.com/14/:
So how should init be done right?
- The Unix way: with simple self-contained programs that do one thing and do it well. Get everything out of PID 1.
- The systemd way https://people.debian.org/~stapelberg/docs/systemd-dependencies.html: Take advantage of special properties of pid 1 to the maximum extent possible. This leads to ever-expanding scope creep and exacerbates all of the problems described in http://ewontfix.com/14/ (and probably many more yet to be discovered).
- The right way: Do away with everything special about pid 1 by making pid 1 do nothing but start the real init script and then just reap zombies (see script by Rich Felker below)
PID 1
There has been some interest in having a proper free software license on the trivial init code included below. I originally considered it too trivial to even care about copyright or need a license on it, but I don't want this to keep anyone from using or reusing it, so I'm explicitly licensing it under the standard MIT license ~ Rich Felker
#define _XOPEN_SOURCE 700 #include <signal.h> #include <unistd.h> int main() { sigset_t set; int status; if (getpid() != 1) return 1; sigfillset(&set); sigprocmask(SIG_BLOCK, &set, 0); if (fork()) for (;;) wait(&status); sigprocmask(SIG_UNBLOCK, &set, 0); setsid(); setpgid(0, 0); return execve("/etc/rc", (char *[]){ "rc", 0 }, (char *[]){ 0 }); }
Yes, that's really all that belongs in PID 1. Then there's no way it can fail at runtime, and no need to upgrade it once it's successfully running.
Exception handling
Initialisation
Mounting virtual file systems
Setting hostname
Kill
Sending SIGTERM to processes
Sending SIGKILL to processes
Mounting real file systems
Mounting local file systems
Unmounting real file systems
Managing temporary files
Daemons
Auto grouping
Socket activation
Which init system?
Package development
Resources
Kernel
- A Guide to the Kernel Development Process http://www.linuxfoundation.org/content/1-guide-kernel-development-process
- The Linux Kernel Archives https://www.kernel.org/
- The kernel guide http://www.linux.org/threads/linux-kernel-reading-guide.5384/
Init systems
- The Epoch Init System: A New Beginning http://universe2.us/epoch.html
- runit - a UNIX init scheme with service supervision http://smarden.org/runit/
- Supervisor: A Process Control System http://supervisord.org/
Daemon tools
- Daemontools: http://cr.yp.to/daemontools.html
- Eudev https://wiki.gentoo.org/wiki/Project:Eudev