Chapter 1: Welcome to Linux

1.2 Basic survival skills

1.2.1 The Linux file system

• UNIX Filesystem Hierarchy Standard (FHS)
• Figure 1.2 Common top-level directories as defined by the UNIX FHS
  

1.2.3 Getting things done: Linux file management tools

• “Touching” an existing file with touch updates its time stamp without making any changes. This can be useful if, for some reason, you want to change how various commands like ls list or display a file. (It can also be helpful if you’d like your boss to think that you’ve been hard at work on a data file that, in fact, you haven’t opened for weeks.)
• Every object within a Linux file system is represented by a unique collection of metadata called an inode. I suppose you could say that the file system index discussed earlier is built from the metadata associated with all the many inodes on a drive. To display more information about the file you just created using touch, including inode information, you can use the stat command:
  
• As you can see, the output includes data describing the file’s name, attributes, and time stamps. But it also tells you its inode ID number. It’s important to be aware that when you move, copy, or delete a file or directory, all you’re really doing is editing its inode attributes, not its ID. An inode, by the way, is an object used by UNIX systems to identify the disk location and attributes of files within a file system (as illustrated in figure 1.2). Usually there’ll be exactly one inode for each file or directory.

1.2.5 Pseudo file systems

• A normal file is a collection of data that can be reliably accessed over and over again, even after a system reboot. By contrast, the contents of a Linux pseudo (or virtual) file, like those that might exist in the /sys/ and /proc/ directories, don’t really exist in the normal sense. A pseudo file’s contents are dynamically generated by the OS itself to represent specific values.

  $ cat /sys/block/sda/size
  1937389568

• Originally, sda probably stood for SCSI Device A, but I find that thinking of it as Storage Device A makes it more meaningful. You might also run into device designations like /dev/hda (hard drive), /dev/sr0 (DVD drive), /dev/cdrom (that’s right, a CD-ROM drive), or even /dev/fd0 (floppy drive).
  

1.3 Getting help

1.3.2 Info

$ info

• Figure 1.3 The first screen of Info’s main menu. Info links may appear different on your system depending on what software you’ve installed.
  

1.3.3 The internet

• On nearly all modern Linux distributions (with the notable exception of Ubuntu 14.04), you can access all system logs through journalctl:

  $ journalctl

• In case you’d prefer to see only those lines that don’t contain the word error, you’d add -v (for inverted results):

  $ journalctl | grep filename.php | grep -v error

Summary

• Linux uses pseudo file systems to expose data on the hardware environment to processes and users.

Key terms

• File globbing involves using wildcard characters to refer to multiple files with a single command.
• Pseudo file systems are directories containing files with dynamic data automati-cally generated at or after system boot.

Chapter 3: Remote connectivity: Safely accessing networked machines

3.2 Getting started with OpenSSH

• You can force a process (like SSH) to automatically load on system startup using systemctl enable ssh, or to not load on startup with systemctl disable ssh.

3.4 Password-free SSH access

3.4.2 Copying the public key over a network

• … the backslash character (), which tells Bash to read the next line as part of the current line. Make sure there are no characters (including a space) after the backslash.

3.7 Linux process management

3.7.1 Viewing processes with the ps command

• If you want to visualize parent and child shells/processes, you can use the pstree command (adding the -p argument to display the PIDs for each process). Note how the first process (assigned PID 1) is systemd. On older versions of Linux (Ubuntu 14.04 and earlier, for instance), this would have been called init instead:

Summary

• On most modern Linux distributions, processes are managed by systemd through the systemctl tool.

Key terms

• A password is a string of regular characters, while a passphrase can include spaces and punctuation.

Command-line review

• $ cat.ssh/id_rsa.pub | ssh ubuntu@10.0.3.142 "cat >> .ssh/authorized_keys" copies a local key and pastes it on a remote machine.
• ssh -X ubuntu@10.0.3.142 allows you to log in to a remote host for a graphics-enabled session.

Chapter 4: Archive management: Backing up or copying entire file systems

4.2 What to archive

• It’s pretty simple to tell which partitions are used for pseudo files: if the file designation is tmpfs and the number of bytes reported in the Used column is 0, then the odds are you’re looking at a temporary rather than a normal file system.
• … keep your /boot/ directory in the largest partition.

4.4 Archiving files and file systems using tar

4.4.2 Streaming file system archives

• … create an archive image of a working Linux installation and stream it to a remote storage location …

$ tar czvf - --one-file-system / /usr /var \ 
--exclude=/home/andy/ | ssh username@10.0.3.141 \
"cat > /home/username/workstation-backup-Apr-10.tar.gz"

• … I used a dash (czvf -). The dash outputs data to standard output.
• The --one-file-system argument excludes all data from any file system besides the current one. This means that pseudo partitions like /sys/ and /dev/ won’t be added to the archive.

4.4.3 Aggregating files with find

• Here’s a single command that will search the /var/www/html/ hierarchy for files with names that include the file extension .mp4. When a file is found, tar will be executed with the argument -r to append (as opposed to overwrite) the video file to a file called videos.tar:
  
• By default, locate searches the entire system for files matching the string that you specify.
• locate isn’t actually searching the file system itself, but simply running your search string against entries in a preexisting index.
• Normally the index is updated every time the system boots, but you can also manually do the job by running updatedb

4.4.4 Preserving permissions and ownership…and extracting archives

• The first dash (① in the figure) means that the object being listed is a file. It would be replaced with a d if it were a directory. The next three characters ② are a representation of the file’s permissions as they apply to its owner, the next three ③ are the permissions as they apply to its group, and the final three ④ represent the permissions all other users have over this file.
• Figure 4.5 A breakdown of the data displayed by the ls -l command
  
• If you want to change a file’s permissions, use the change mode (chmod) tool:

  $ chmod o-r /bin/zcat
  $ chmod g+w /bin/zcat

• This example removes the ability of others (o) to read the file and adds write permissions for the group (g). A file’s owner would be represented by the letter u (for user).

Permission	Character	Number
Read	r	4
Write	w	2
Execute	x	1

• The && characters will execute a second command only if the first command was successful.
• Extracting an archive overwrites any files with the same names in the current directory without warning.
• Generally, only users with administrator powers can work with resources in other users’ accounts.
• The solution is to perform these operations as an administrator, using sudo.

4.5 Archiving partitions with dd

• Using dd, … can make perfect byte-for-byte images of, well, just about anything digital.
• NOTE As always with dd, pause and think very carefully before pressing that Enter key!

4.6 Synchronizing archives with rsync

# Creates 10 files named file1 to file10
$ touch file{1..10}

Command-line review

• find /var/www/ -iname "*.mp4" -exec tar -rvf videos.tar {} \; finds files meeting a set criteria and streams their names to tar to include in an archive.
• chmod o-r /bin/zcat removes read permissions for others.

Chapter 5: Automated administration: Configuring automated offsite backups

5.1 Scripting with Bash

5.1.1 A sample script for backing up system files

• Cron, by default, will always run as root.

5.1.2 A sample script for changing filenames

• … convert spaces within filenames to … underscore characters

  #!/bin/bash
  echo "which directory would you like to check?"
  read directory
    find $directory -type f | while read file; do 
        if [[ "$file" = *[[:space:]]* ]]; then
        mv "$file" `echo $file | tr ' ' '_'`
        fi;
    done

5.3 Scheduling regular backups with cron

• If you’ve got, say, a file system backup script in an executable file you want run at set intervals, you copy it to the appropriate directory: cron.hourly/ for execution each hour, cron.daily/ to be run daily, and so on.
• NOTE By default, on some systems a user won’t be able to create crontab jobs unless a file called /etc/cron.allow containing their username is created. Debian/Ubuntu, however, allow individual crontab jobs right out of the box.

5.4 Scheduling irregular backups with anacron

• … anacrontab … is where you schedule operations to run at a set time after each system boot.

5.5 Scheduling regular backups with systemd timers

• Figure 5.4 The command systemctl list-timers –all provides a rich range of historical data for all existing systemd timer jobs.
  

  $ systemctl start site-backup.timer
  $ systemctl enable site-backup.timer

  $ systemctl is-enabled backup.timer
  enabled
  $ systemctl is-active backup.timer
  active

Summary

• Linux keeps user account and authentication information in plain text files (named passwd, group, shadow, and gshadow) in the /etc/ directory.
• Adding a directive to the anacrontab file executes commands relative to system boots, rather than at absolute times.
• systemd timers can be set to run based on both absolute time and in reaction to system events, like changes to hardware states.

Key terms

• All Linux commands output exit codes upon completion: 0 represents a successful execution, but all positive integers can be set by a program to represent various failed states.
• A bucket is an AWS object that works much the same way as a directory on an operating system.

Command-line review

• #!/bin/bash (the so-called “shebang line”) tells Linux which shell interpreter you’re going to be using for a script.
• || inserts an or condition into a script. Think of this as either “the command to the left is successul” or “execute the command to the right.”
• && - inserts an and condition into a script. Think of this as “if the command to the left is successful” and “execute the command to the right.”
• test -f /etc/filename tests for the existence of the specified file or directory name.

Chapter 6: Emergency tools: Building a system recovery device

6.1 Working in recovery/rescue mode

6.1.1 The GRUB bootloader

• What’s GRUB? It’s the GNU GRand Unified Bootloader.
• Master Boot Record (MBR)
• Figure 6.3 The key steps in the boot process of a Linux computer
  

6.2 Building a live-boot recovery drive

6.2.2 Writing live-boot images to USB drives

• But if the image will be written to a USB drive and if you happen to be working on an Ubuntu host, you’ll first need to modify the image by adding an MBR to the .ISO archive so that BIOS and UEFI firmware will know what to do with it.
• On Ubuntu hosts, you use isohybrid for that image modification. The apt package containing isohybrid is called syslinux-utils.

  $ apt update
  $ apt install syslinux-utils
  $ cd ~/Downloads
  $ isohybrid systemrescuecd-x86-5.0.2.iso

• … lsblk, which stands for list block devices.
  
• First unmount the drive itself so dd can get full access.
  

Summary

• File systems can be mounted and administered using a virtual process called chroot.

Key terms

• The tool chroot opens virtual root shells within mounted file systems.

Command-line review

• isohybrid systemrescuecd-x86-5.0.2.iso adds a USB-friendly MBR to a live-boot image.
• mount /dev/sdc1 /run/temp-directory mounts a partition to a directory on the live file system.
• ddrescue -d /dev/sdc1 /run/usb-mount/sdc1-backup.img /run/usb-mount/ sdc1-backup.logfile saves files on a damaged partition to an image named sdc1-backup.img and writes events to a log file.
• chroot /run/mountdir/ opens a root shell on a file system.

Chapter 8: Networked file sharing: Building a Nextcloud file-sharing server

8.3 Installing Nextcloud manually

8.3.4 Downloading and unpacking Nextcloud

• Unpacking a .tar.bz2 archive requires the xjf arguments, rather than the xzf you’d use for a .gz:
  

Chapter 9: Securing your web server

9.2 Controlling network access

9.2.2 Using nonstandard ports

• Network ports: The 65,535 available network ports are divided into three categories:

1. Ports between 1 and 1023 are designated as well-known and have been set aside for recognized services like SSH (22) and HTTP (80). You should never use a well-known port number for your own applications, as you’re likely to cause a conflict.
2. Ports between 1024 and 49151 are registered, meaning companies and organizations have requested that specific ports in this range be set aside for their applications even if they haven’t become universally adopted. Examples of this are port 1812, which is used for the RADIUS authentication protocol, and 3306, MySQL’s dedicated port.
3. Ports between 49152 and 65535 are unregistered and are considered dynamic (or private). These ports are available for any temporary or ad hoc use, particularly on private networks. You can be confident that they won’t clash with known applications or services.

9.5 Auditing system resources

9.5.1 Scanning for open ports

• The netstat command displays open ports along with a wealth of information about how those ports are being used.
• … -n tells netstat to include the numeric ports and addresses, -l includes only listening sockets, and -p adds the process ID of the listening program.
  
  

9.5.2 Scanning for active services

Key terms

• Discretionary access control systems (DACs) allow users control over file system resources.
• Control over resources on mandatory access control systems (MACs) is ultimately managed by system-wide policies.

Chapter 10: Securing network connections: Creating a VPN or DMZ

10.2 Building intrusion-resistant networks

10.2.1 Demilitarized zones (DMZs)

• One popular isolation architecture is known as a DMZ (a contraction of the phrase used to describe a geographic buffer between two distrustful nations: demilitarized zone).

Summary

• iptables and nftables are deep and flexible command-line implementations of the netfilters Linux kernel firewall tool.

Chapter 11: System monitoring: Working with log files

11.1 Working with system logs

• For decades, Linux logging has been managed by the syslogd daemon. Syslogd would collect the log messages that system processes and applications sent to the /dev/log pseudo device. Then it would direct the messages to appropriate plain text log files in the /var/log/ directory.
• Figure 11.1 The flow of log data from sample sources through the syslogd daemon
  
• Linux logging is now also handled by journald.
• But the fact that journald stores log data in a binary file rather than plain text is definitely an issue.
• Figure 11.2 The journald logs (including those generated by syslogd) are consumed using the journalctl command-line tool.
  

11.3 Consuming large files

11.3.2 Using awk

• By default, awk divides a line of text into individual fields, each separated by one or more spaces.

Key terms

• A network-based intrusion detection system (NIDS) monitors a private network for evidence of infiltration.
• A host-based intrusion detection system (HIDS) monitors servers for evidence system files have been maliciously altered.

Chapter 13: Troubleshooting system performance issues

13.1 CPU load problems

13.1.1 Measuring CPU load

• A load average of 1.27 on a system with one CPU would mean that, on average, the CPU is working to capacity and another 27% of processes are waiting for the their turn with the CPU. By contrast, a load average of 0.27 on a system with one CPU would mean that, on average, the CPU was unused for 73% of the time. On a four-core system, you might see load averages in the range of 2.1, which would be just over 50% of capacity (or unused for around 52% of the time).

13.1.2 Managing CPU load

• SETTING PRIORITIES WITH NICE：Sometimes you won’t be able to kill a process because it’s a necessary part of a mission-critical service. But you can limit the CPU resources it gets using nice. By default, a new process is given a nice value of 0, but you can change that to any number between -20 and 19. The higher the number, the nicer the process will be when it comes to giving up resources in favor of other processes. And, by contrast, the lower the number, the less nice the process will be as it grabs as many resources as it can get, regardless of the pain and suffering that might cause others.
• … the dash (-) followed by 15 tells Linux that the script will run with a very nice attitude. This means that when there’s a conflict over resource access, your script will back off, but otherwise it’ll take whatever’s available:

  $ nice -15 /var/scripts/mybackup.sh

• If running your script is an urgent priority that has to complete as soon as possible, you could add a second dash to give the process a negative value (-15), as in this example:

  $ nice --15 /var/scripts/mybackup.sh

• You can also use renice to change the way a process behaves even after it’s started. This example will, when necessary, limit the resources available to the process currently assigned PID 2145:

  $ renice 15 -p 2145

• Symbols for CPU-related metrics displayed by top

Metric	Meaning
us	Time running high-priority (un-niced) processes
sy	Time running kernel processes
ni	Time running low-priority (nice) processes
id	Time spent idling
wa	Time waiting for I/O events to complete
hi	Time spent managing hardware interrupts
si	Time spent managing software interrupts
st	Time stolen from this VM by its hypervisor (host)

13.2 Memory problems

13.2.1 Assessing memory status

• … shared is memory that’s used by tmpfs to maintain the various pseudo file systems we’ve come to know and love, like /dev/ and /sys/. Buffers and cache are associated with memory used by the kernel for block level I/O operations

13.2.2 Assessing swap status

• You can get a glimpse of the way swap is being used on your system from vmstat.
• The two columns you should watch most closely are si, which measures data transfers from swap into system memory, and so, which reports transfers out of system memory into swap.
• If you’re getting consistent movement into and out of swap, you should consider adding physical RAM, unless the slower performance isn’t a problem for your workload.

13.3 Storage availability problems

13.3.1 Inode limits

$ find . -xdev -type f | cut -d "/" -f 2 | sort | uniq -c | sort -n

• Table 13.2 explains what it all means.
• Table13.2 find command syntax

Syntax	Function
.	Start searching within and below the current directory.
-xdev	Remain within a single file system.
-type f	Search for objects of type file.
cut -d “/“	Remove text identified by the delimiter (/ character, in this case).
-f 2	Select the second field found.
sort	Sort lines of output, and send to standard out (stout).
uniq -c	Count the number of lines sent by sort.
sort -n	Display the output in numeric order.

13.5 Monitoring tools

13.5.2 Visualizing your data

• … you can use a tool called nmonchart to convert the data files to the much more user-friendly .html format.

Summary

• iftop and NetHogs are two among dozens of Linux tools for accessing network load data, and tc can be used to control usage.

Key terms

• Swap memory is space on a hard drive designated as virtual RAM, in case you run out of the real thing.

Security best practices

• Running iftop can show you remote network hosts with live connections to your system.

Command-line review

• find . -xdev -type f | cut -d "/" -f 2 | sort | uniq -c | sort-n counts and displays the numbers of files by parent directory.
• nethogs eth0 displays processes and transfers data related to network connections using the eth0 interface.
• tc qdisc add dev eth0 root netem delay 100ms slows down all network transfers through the eth0 interface by 100 milliseconds.

Chapter 14: Troubleshooting network issues

14.1 Understanding TCP/IP addressing

Working with NAT addressing

• The NAT protocol sets aside three IPv4 address ranges that can only be used for private addressing:

1. 10.0.0.0 to 10.255.255.255
2. 172.16.0.0 to 172.31.255.255
3. 192.168.0.0 to 192.168.255.255

• NOTE: Following networking conventions, DHCP servers generally don’t assign the numbers 0, 1, and 255 to network devices.

14.3 Troubleshooting outbound connectivity

14.3.1 Tracking down the status of your network

• You can list all the PCI-based hardware currently installed using lspci.
• NOTE: The Peripheral Component Interconnect (PCI) is a hardware standard used to allow peripheral devices to connect to the microprocessors on computer motherboards through the PCI bus. Various newer standards, like PCI Express (PCIe), also exist, each using its own unique form factor to physically connect to a motherboard.
• Besides lspci, you can also use the lshw tool to display the networking hardware your system knows about. By itself, lshw returns a complete hardware profile, but lshw -class network will show you only the subset of that profile that relates to networking. Try it.

Security best practices

• It’s good to periodically use a tool like nmap to audit your system for inappropriately open ports.

Command-line review

• dmesg | grep -A 2 Ethernet searches the dmesg logs for references to the string Ethernet and displays references along with the subsequent two lines of output.
• ip route add default via 192.168.1.1 dev eth0 manually sets a new network route for a computer.
• dhclient enp0s3 requests a dynamic (DHCP) IP address for the enp0s3 interface.
• ip addr add 192.168.1.10/24 dev eth0 assigns a static IP address to the eth0 interface, which won’t persist past the next system restart.
• ip link set dev enp0s3 up starts the enp0s3 interface (useful after editing the configuration).
• netstat -l | grep http scans a local machine for a web service listening on port 80.
• nc -z -v bootstrap-it.com 443 80 scans a remote web site for services listening on the ports 443 or 80.

Chapter 15: Troubleshooting peripheral devices

15.1 Identifying attached devices

• … lsusb lists any USB devices Linux is aware of, …
• When run with root permissions, lshw prints a complete hardware profile of your system.

Command-line review

• lshw -c memory (or lshw -class memory) displays the memory section of a sys- tem’s hardware profile.
• ls /lib/modules/`uname -r` lists the contents of the directory under /lib/modules/ containing modules for your current, active kernel.
• lsmod lists all active modules.
• modprobe -c lists all available modules.
• find /lib/modules/$(uname -r) -type f -name ath9k* searches for a file among the available kernel modules with a name starting with ath9k.
• modprobe ath9k loads the specified module into the kernel.
• GRUB_CMDLINE_LINUX_DEFAULT="systemd.unit=runlevel3.target" (in the /etc/default/grub file) loads Linux as a multiuser, nongraphic session.