![]() |
# The Debian Linux Magic Spells File Cheat Sheet
# by Elf Qrin - https://labs.geody.com/
v1.93 r2022-06-20 fr2005-09-xx
# "One man cast a lingering spell of awe and wonder, of magical innocence overcoming evil, of simple courage conquering fear" -- J.R.R. Tolkien's The Lord of the Rings (1978 Ralph Bakshi animated film)
# 'someone who thinks of code as magical incantations and asks only "what do I need to type to make this happen?"' -- from the definition of Script Kiddies in the Jargon File
----- Notes: - In this file, jed is used as the default editor for it's more intuitive than others. You'll have to install it ( apt-get install jed ). Alternatively, you might want to use vi or vim ( apt-get install vim ) instead. - Most commands work also on Ubuntu / Kubuntu / Xubuntu / Lubuntu / Edubuntu, Mint, Xandros, Raspbian, and other Debian-based distributions (Debian derivatives). Some commands work on all Linux distributions. - You may send contributions and suggestions to (see: https://www.geody.com/contacts.php ) ----- Terminal: - You can access the terminal in most Debian based distributions (including Debian itself and Ubuntu) by pressing CTRL+ALT+F1 (actually you can switch between terminals with CTRL+ALT+function keys. With CTRL+ALT+F7 you normally return to the graphic interface in Ubuntu). In Xandros, you have to press CTRL+ALT+T. - To access a remote terminal via SSH from Windows you may use the free and open source software PuTTY. - To access a remote terminal via SSH from Android you may use the free and open source software ConnectBot. Keyboard shortcuts for the terminal: TAB # Autocomplete the command, directory or file name you are typing. If it can't autocomplete your text from the characters you've typed, hit it again to show all matching possibilities CTRL+A # Move the cursor to the beginning of the line CTRL+E # Move the cursor to the end of the line CTRL+L # Clear the Screen (scrolls all the all the text above the visible area, same as printf "\033c" ) CTRL+C # Break (Halt) the current running program (signal 2) CTRL+D # Exit / Logout CTRL+U # Delete all characters to the left of the cursor up to the beginning of the line CTRL+K # Delete all characters to the right of the cursor up to the end of the line CTRL+W # Delete all characters to the left up to the beginning of the current word CTRL+Y # Paste the text erased using CTRL+U, CTRL+K, CTRL+W. Note that this uses a separate clipboard. CTRL+P or CURSOR UP # Show the previous command in History CTRL+N or CURSOR DOWN # Show the next command in History ----- Linux Filesystem Hierarchy Meaning of Linux directories: / # Root directory. The operating system kernel must be located in either / or /boot. /bin # Essential user command binaries (for use by all users). It contains commands that may be used by both the system administrator and by users, but which are required when no other filesystems are mounted (e.g. in single user mode). It may also contain commands which are used indirectly by scripts. /boot # Static files of the boot loader. Files required for booting the system. This directory contains everything required for the boot process except configuration files not needed at boot time and the map installer. Thus /boot stores data that is used before the kernel begins executing user-mode programs. This may include saved master boot sectors and sector map files. /cdrom # Non standard directory for CD Roms. It should be inside /media instead. /dev # Device files. /etc # Et cetera. Host-specific system configuration. The /etc hierarchy contains configuration files. A "configuration file" is a local file used to control the operation of a program; it must be static and cannot be an executable binary. It is recommended that files be stored in subdirectories of /etc rather than directly in /etc. /etc/opt # Configuration files for /opt . /etc/X11 # Configuration for the X Window System (optional). /etc/sgml # Configuration files for SGML (optional). /etc/xml # Configuration files for XML (optional). /home # User home directories. No program should assume any specific location for a home directory, rather it should query for it. (optional). /lib # Essential shared libraries and kernel modules needed to boot the system and run the commands in the root filesystem, ie. by binaries in /bin and /sbin. /lost+found # Files recovered after a system crash. /media # Mount point for removable media. This directory contains subdirectories which are used as mount points for removable media such as floppy disks, cdroms, zip disks. /mnt # Mount point for a temporarily mounted filesystem. This directory is provided so that the system administrator may temporarily mount a filesystem as needed. The content of this directory is a local issue and should not affect the manner in which any program is run. This directory must not be used by installation programs: a suitable temporary directory not in use by the system must be used instead. /opt # Add-on application software packages. A package to be installed in /opt must locate its static files in a separate /opt/[package] or /opt/[provider] directory tree, where [package] is a name that describes the software package and [provider] is the provider's LANANA (Linux Assigned Names And Numbers Authority) registered name. /proc # Kernel and process information virtual filesystem. /root # Home directory for the root user (optional). The root account's home directory may be determined by developer or local preference, but this is the recommended default location. /run # Run-time variable data. This directory contains system information data describing the system since it was booted. Files under this directory must be cleared (removed or truncated as appropriate) at the beginning of the boot process. The purposes of this directory were once served by /var/run. In general, programs may continue to use /var/run to fulfill the requirements set out for /run for the purposes of backwards compatibility. Programs which have migrated to use /run should cease their usage of /var/run, except as noted in the section on /var/run. Programs may have a subdirectory of /run; this is encouraged for programs that use more than one run-time file. Users may also have a subdirectory of /run, although care must be taken to appropriately limit access rights to prevent unauthorized use of /run itself and other subdirectories. /sbin # Essential system binaries. Utilities used for system administration (and other root-only commands) are stored in /sbin, /usr/sbin, and /usr/local/sbin. /sbin contains binaries essential for booting, restoring, recovering, and/or repairing the system in addition to the binaries in /bin. Programs executed after /usr is known to be mounted (when there are no problems) are generally placed into /usr/sbin. Locally-installed system administration programs should be placed into /usr/local/sbin. /srv # Data for services provided by this system. /sys # Kernel and system information virtual filesystem. /tmp # Temporary files. Programs must not assume that any files or directories in /tmp are preserved between invocations of the program. It is recommended that files and directories located in /tmp be deleted whenever the system is booted. Temporary files that can survive a reboot should be stored in /var/tmp . /usr # "usr" stands for "UNIX System Resources", not "user". It contains shareable, read-only data. /usr/bin # Most user commands. This is the primary directory of executable commands on the system. /usr/include # Directory for standard include files. This is where all of the system's general-use include files for the C programming language should be placed. /usr/lib # Libraries for programming and packages. /usr/libexec # Binaries run by other programs (optional). It includes internal binaries that are not intended to be executed directly by users or shell scripts. Applications may use a single subdirectory under /usr/libexec. Applications which use /usr/libexec in this way must not also use /usr/lib to store internal binaries, though they may use /usr/lib for the other purposes documented here. /usr/lib can be also used if /usr/libexec is missing. /usr/local # Local hierarchy, for use by the system administrator when installing software locally. It needs to be safe from being overwritten when the system software is updated. It may be used for programs and data that are shareable amongst a group of hosts, but not found in /usr. Locally installed software must be placed within /usr/local rather than /usr unless it is being installed to replace or upgrade software in /usr. /usr/sbin # Non-essential standard system binaries. This directory contains any non-essential binaries used exclusively by the system administrator. System administration programs that are required for system repair, system recovery, mounting /usr, or other essential functions must be placed in /sbin instead. There must be no subdirectories in /usr/sbin. /usr/share # Architecture-independent data. This hierarchy is intended to be shareable among all architecture platforms of a given OS; thus, for example, a site with i386, Alpha, and PPC platforms might maintain a single /usr/share directory that is centrally-mounted. Note, however, that /usr/share is generally not intended to be shared by different OSes or by different releases of the same OS. Any program or package which contains or requires data that doesn't need to be modified should store that data in /usr/share (or /usr/local/share, if installed locally). It is recommended that a subdirectory be used in /usr/share for this purpose. Applications using a single file may use /usr/share/misc. Game data stored in /usr/share/games must be purely static data. Any modifiable files, such as score files, game play logs, and so forth, should be placed in /var/games. /usr/share/color # Color management information (optional). /usr/share/dict # Word lists (optional). /usr/share/man # Manual pages. /usr/share/misc # Miscellaneous architecture-independent data. /usr/share/ppd # Printer definitions (optional). /usr/share/sgml # SGML data (optional). /usr/share/xml # XML data (optional). /usr/src # Source code (optional). The only source code that should be placed in a specific location is the Linux kernel source code. It is located in /usr/src/linux. /var # variable data files. This includes spool directories and files, administrative and logging data, and transient and temporary files. Some portions of /var are not shareable between different systems. For instance, /var/log, /var/lock, and /var/run. Other portions may be shared, notably /var/mail, /var/cache/man, /var/cache/fonts, and /var/spool/news. /var exists in order to make it possible to mount /usr read-only. Everything that once went into /usr that is written to during system operation (as opposed to installation and software maintenance) must be in /var. /var/account # Process accounting logs (optional). This directory holds the current active process accounting log and the composite process usage data (as used in some UNIX-like systems by lastcomm and sa). /var/cache # Application cache data. /var/cache is intended for cached data from applications. Such data is locally generated as a result of time-consuming I/O or calculation. The application must be able to regenerate or restore the data. Unlike /var/spool, the cached files can be deleted without data loss. The data must remain valid between invocations of the application and rebooting the system. Files located under /var/cache may be expired in an application specific manner, by the system administrator, or both. The application must always be able to recover from manual deletion of these files (generally because of a disk space shortage). No other requirements are made on the data format of the cache directories. /var/cache/fonts # Locally-generated fonts (optional). It should be used to store any dynamically-created fonts. In particular, all of the fonts which are automatically generated by mktexpk must be located in appropriately-named subdirectories of /var/cache/fonts. An application (or a group of inter-related applications) must use a subdirectory of /var/lib for its data. There is one required subdirectory, /var/lib/misc, which is intended for state files that don't need a subdirectory; the other subdirectories should only be present if the application in question is included in the distribution. /var/cache/man # Locally-formatted manual pages (optional). /var/crash # System crash dumps (optional). /var/games # Variable game data (optional). /var/lib # Variable state information. State information is data that programs modify while they run, and that pertains to one specific host. Users must never need to modify files in /var/lib to configure a package's operation, and the specific file hierarchy used to store the data must not be exposed to regular users. /var/lib/[editor] # Editor backup files and state (optional). /var/lib/color # Color management information (optional). /var/lib/hwclock # State directory for hwclock (optional). /var/lib/misc # Miscellaneous variable data. /var/lock # Lock files. Lock files for devices and other resources shared by multiple applications, such as the serial device lock files that were originally found in either /usr/spool/locks or /usr/spool/uucp, must now be stored in /var/lock. The naming convention which must be used is "LCK.." followed by the base name of the device. For example, to lock /dev/ttyS0 the file "LCK..ttyS0" would be created. The format used for the contents of such lock files must be the HDB UUCP lock file format. The HDB format is to store the process identifier (PID) as a ten byte ASCII decimal number, with a trailing newline. For example, if process 1230 holds a lock file, it would contain the eleven characters: space, space, space, space, space, space, one, two, three, zero, and newline. /var/log # Log files and directories. This directory contains miscellaneous log files. Most logs must be written to this directory or an appropriate subdirectory. The following files, or symbolic links to files, must be in /var/log, if the corresponding subsystem is installed: lastlo: record of last login of each user; messages: system messages from syslogd; wtmp: record of all logins and logouts. /var/mail # User mailbox files (optional). User mailbox files in this location must be stored in the standard UNIX mailbox format. /var/opt # Variable data for /opt . This is where the static data from an add-on software package is stored, except where superseded by another file in /etc . /var/run # Run-time variable data. This directory was once intended for system information data describing the system since it was booted. These functions have been moved to /run; this directory exists to ensure compatibility with systems and software using an older version of this specification. /var/spool # Application spool data. It contains data which is awaiting some kind of later processing. Data in /var/spool represents work to be done in the future (by a program, user, or administrator); often data is deleted after it has been processed. /var/spool/cron # cron and at jobs. This directory contains the variable data for the cron and at programs. /var/spool/lpd # Line-printer daemon print queues (optional). The lock file for lpd, lpd.lock, must be placed in /var/spool/lpd. It is suggested that the lock file for each printer be placed in the spool directory for that specific printer and named lock. /var/spool/rwho # Rwhod files (optional). /var/tmp # Temporary files preserved between system reboots. /var/yp # Network Information Service (NIS) database files (optional). Variable data for the Network Information Service (NIS), formerly known as the Sun Yellow Pages (YP), must be placed in this directory. ----- System and Software installation: Install a software package using APT (Advanced Package Tool): apt-get install base-config apt-setup Set APT sources (repositories) manually: jed /etc/apt/sources.list # use vi /etc/apt/sources.list if jed (or other editor) hasn't been installed yet. vi should always be present on Debian Linux You can also use edit the sources file with apt edit-sources # Edit APT sources file using the default editor or the one set in export EDITOR=jed ; export VISUAL=jed ; # (or another editor. Remember to install it before to set it as a system default) deb http://http.us.debian.org/debian stable main contrib non-free deb-src http://http.us.debian.org/debian stable main contrib non-free deb http://security.debian.org stable/updates main contrib non-free deb-src http://security.debian.org stable/updates main contrib non-free You can use a local repository instead of the US Debian one, for example debian.fastweb.it instead of http.us.debian.org Use unstable instead of stable if you want bleeding edge technology. Not very recommended in production environments. To keep an old version up to date: deb http://archive.debian.org/debian/ DEBIAN_VERSION main non-free contrib deb-src http://archive.debian.org/debian/ DEBIAN_VERSION main non-free contrib deb http://archive.debian.org/debian-security/ DEBIAN_VERSION/updates main non-free contrib deb-src http://archive.debian.org/debian-security/ DEBIAN_VERSION/updates main non-free contrib Update the list of packages: # After adding a new apt-get source, or before installing new packages, enter this command apt-get update apt update apt update ; apt list --upgradable ; # check for upgradable packages and list them if found apt update 2>/dev/null | grep packages # check for upgradable packages and show the count of them if found (output is redirected to /dev/null to remove the message: WARNING: apt does not have a stable CLI interface. Use with caution in scripts.) apt update 2>/dev/null | tail --lines=1 # check for upgradable packages and show the count of them if found (output is redirected to /dev/null to remove the message: WARNING: apt does not have a stable CLI interface. Use with caution in scripts.) Should a "Segmentation fault" error occur when updating the list of packages, you can fix it with these commands: rm -r /var/lib/apt/lists/* rm /var/cache/apt/*.bin # will delete pkgcache.bin and srcpkgcache.bin Show upgradable packages without upgrading them (after an update) apt-get -u upgrade --assume-no apt list --upgradable Upgrade installed packages: apt-get upgrade # upgrade already installed packages but don't remove anything or install anything new apt upgrade Upgrade a distribution: # This is just a basic procedure to upgrade a Debian release. Check https://www.debian.org/releases/stable/amd64/release-notes/ch-upgrading.html for mode detailed information. apt-get update apt-get upgrade aptitude search '~i(!~ODebian)' # List third-party packages that may be lost after a dist-upgrade dpkg --audit # List Half-Installed or Failed-Config packages apt-get autoremove # remove packages installed as dependency of already uninstalled packages jed /etc/apt/sources.list # Set APT sources (repositories) for the new Debian release apt-get update apt-get upgrade apt-get dist-upgrade # upgrade everything removing/adding packages as required to resolve dependencies apt-get autoremove reboot Make sure the system is up to date after a fresh installation: dpkg --audit apt-get update apt-get upgrade apt-get dist-upgrade dpkg --audit If the system is installed on a VMWare machine, you'd better install VMware Tools or Open VM Tools. The package for VMware Tools can be downloaded from VMWare website, while its open counterpart (Open VM Tools) can be installed via apt-get: apt-get install open-vm-tools Prevent a package to be upgraded: apt-get install wajig wajig hold PACKAGE # the specified package will no longer be upgraded (for example with an apt-get upgrade) wajig unhold PACKAGE # the package will be upgraded again wajig hold mysql-server-5.0 # a common problem with Kernel 2.4 is that latest releases of MySQL server 5 are not compatible with such kernel version, interfering with the upgrade process Search for a package: apt-cache search TEXT # search for a package containing the specified text or keyword in its name or description List installed packages: dpkg-query -l # List all installed packages dpkg --list # List all installed packages dpkg --get-selections # List all installed packages dpkg -l # List all installed packages with version and description dpkg --get-selections | wc --lines # Count installed packages grep install /var/log/dpkg.log | tail -10 # Last 10 installed packages dpkg --listfiles PACKAGE # List all files "owned" by PACKAGE cat /var/log/dpkg.log | grep "status installed " # Show last installed packages cat /var/log/apt/history.log # Show last apt commands (installed and upgraded packages) aptitude search '~i(!~ODebian)' # List third-party packages only Check if a package is installed: dpkg -s PACKAGE dpkg-query -l PACKAGE dpkg-query -W -f='${Status} ${Version}\n' PACKAGE # If the package is installed, show status and version dpkg-query -l "TEXT*" # Show all installed packages starting with TEXT dpkg-query -l "*TEXT" # Show all installed packages ending with TEXT dpkg-query -l "*TEXT*" # Show all installed packages containing TEXT cat /var/log/dpkg.log | grep "status installed PACKAGE" # Search for a specific package among the last installed ones Install a package: apt-get install PACKAGE # get and install a PACKAGE from a repository specified in /etc/apt/sources.list dpkg -i PACKAGE.deb # manually install a package available locally (already downloaded) apt-get install --reinstall PACKAGE # reinstall a PACKAGE Show information about a package and all its dependencies apt-cache show PACKAGE aptitude show PACKAGE # similar to apt-cache show PACKAGE but shows extra information apt-cache showpkg PACKAGE # show dependencies and reverse dependencies Verify if all packages are correctly installed and configured: dpkg --audit # List Half-Installed or Failed-Config packages. If everything is fine, it doesn't return anything. Check for broken dependencies: apt-get check Uninstall a package: apt-get remove PACKAGE aptitude purge PACKAGE # remove PACKAGE and its configuration files Show a random non installed package: apt-cache search ~g | while read ; do echo "$RANDOM $REPLY" ; done | sort -n | head -1 | cut -d' ' -f2- Store/Restore Packages: dpkg --get-selections > installedpackages.txt # Store the list of installed packages into the file installedpackages.txt dpkg --set-selections < installedpackages.txt # Restore the list of installed packages from the file installedpackages.txt apt-get install `cat installedpackages.txt` # Restore the list of installed packages from the file installedpackages.txt Clean up packages: apt-get autoremove # remove packages installed as dependency of already uninstalled packages apt-get clean # delete all downloaded packages (that is, all files stored in /var/cache/apt/archives/ and /var/cache/apt/archives/partial/ , except for lock files). If you want to reinstall one of such packages, apt-get will have to download them again. apt-get autoclean # only delete packages which is no longer possible to download (obsolete). If you get one of the following error messages: E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable) E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it? dpkg: status database area is locked by another process or anyway if something goes wrong when upgrading packages, you may fix it this way: rm /var/lib/apt/lists/lock rm /var/lib/dpkg/lock rm /var/cache/apt/archives/lock dpkg --configure -a # to fix interrupted configure (may take a while) apt-get -f install # to fix broken dependencies If you still get error messages there are probably running processes working with installation files, you either have to find and kill them all or reboot before than attempting to recover the upgrade process again. After upgrading the Linux kernel, it's better to perform a update-grub and then reboot the system. Debian Linux (Sarge, Kernel 2.6): workaround for E: This installation run will require temporarily removing the essential package e2fsprogs due to a Conflicts/Pre-Depends loop. This is often bad, but if you really want to do it, activate the APT::Force-LoopBreak option. apt-get remove e2fsprogs apt-get install e2fsprogs sysvinit initscripts upgrade glibc when requested. # note that this is a workaround, it should be fixed with apt-get dist-upgrade Note: when the Debian stable changes, if you don't upgrade your system accordingly, you'll have to modify the /etc/apt/sources.list file to specify you are using an old version of Debian (that is, you are no longer using the "current stable" version of Debian). For example, if you are using Sarge, and Debian Etch is released as stable, and you are not upgrading to Etch, you have to change all references to stable as sarge in your source.list file. Which means, you have to change "deb http://debian.example.com/debian stable main contrib non-free" as "deb http://debian.example.com/debian sarge main contrib non-free". If you don't make this change, when you'll try to install new packages, you'll receive the following message: The following packages will be REMOVED: base-config initrd-tools kernel-image-2.4.27-2-386 This is an example, the kernel image can be different in your case, however you'll not be able to install the package and it may damage your system. In fact, if you procede, you will be warned by this message: You are running a kernel (version 2.4.27-2-386) and attempting to remove the same version. This is a potentially disastrous action. Not only will /boot/vmlinuz-2.4.27-2-386 be removed, making it impossible to boot it, (you will have to take action to change your boot loader to boot a new kernel), it will also remove all modules under the directory /lib/modules/2.4.27-2-386. Just having a copy of the kernel image is not enough, you will have to replace the modules too. I repeat, this is very dangerous. If at all in doubt, answer no. If you know exactly what you are doing, and are prepared to hose your system, then answer Yes. If you get an error like 'Template parse error near `Description-sr@latin.UTF-8', in stanza #X of /PATH/FILE.templates then edit the specified file removing the lines containing the given text (and, if present, the following line with the description) from /PATH/FILE.templates If you get an error with locales, like locale: Cannot set LC_CTYPE to default locale you might be able to fix it with apt-get install locales If you get a message ending with the following request after a power outage: Give root password for maintenance (or type Control-D to continue): enter the root password then check the file system of the faulty device (read the whole message to identify the faulty driver, generally is /dev/hda0 on a single hard drive system or /dev/md0 on a RAID system): fsck -f FAULTY_DEVICE reply yes (y) to all questions. Note that if it asks to connect to lost+found more than once some of your files might be seriously damaged. However you can also use fsck -f -y FAULTY_DEVICE to reply automatically "yes" to all questions. then reboot If you can't perform a simple operation that always used to work (for example, copying a file) make sure the disk is not full (check with df). Set keyboard layout # Enter this command to localize the keyboard for your country apt-get install console-common Set system default editor used for apt edit-sources and crontab -e (parameters in the following commands are case sensitive). If not set the system will use its default editor, tipically "vi". Remember to install the editor first, before to set it as a system default. export EDITOR=jed export VISUAL=jed You can also use the apt command instead of apt-get for most common tasks: apt install # apt-get install apt remove # apt-get remove apt purge # apt-get purge apt update # apt-get update apt upgrade # apt-get upgrade apt autoremove # apt-get autoremove apt full-upgrade # apt-get dist-upgrade apt search # apt-cache search apt show # apt-cache show ----- Install a software package from the source (compile a package source): To compile packages you need to install a C compiler and other tools: apt-get install build-essential If a script uses OpenSSL to handle encryption, you may also need to install OpenSSL headers: apt-get install libssl-dev Packages in form of source code are generally stored into .tar.gz (.tgz) or .tar.bz2 (.tbz) files. You may need to download them from the Internet: wget http://www.example.com/source/package.tgz Once you have the package (from the Internet, a CD, or otherwise) in your work directory, you'll have to decompress it: tar -xjvf package.tbz # extract all files from a TAR+BZIP2 compressed archive or tar -xzvf package.tgz # extract all files from a TAR+GZIP compressed archive it should decompress inside a new directory. Move inside this new directory: cd PACKAGE_SOURCE_DIRECTORY_NAME and execute these commands: ./configure make su ROOT_PASSWORD # (if you're not logged as root) make install The executable file should be created inside /usr/local/bin , and you should be able to invoke it typing its name in the console and then hitting ENTER. To uninstall a package, you have to move inside the directory containing the source (if you kept it): cd PACKAGE_SOURCE_DIRECTORY_NAME and uninstall it: make uninstall If you haven't kept the source, you'll have to locate all files (try locate PACKAGE_NAME) and delete them manually. ----- Change root password without knowing the existing one You need physical access to the system. Add init=/bin/bash to boot parameters mount -o remount,rw # mount file system passwd # set new password. WARNING: sudo passwd will change root password, regardless of the current account. Recover a deleted password file: # Debian makes regular backups of the password file in /var/backups/ cp /var/backups/passwd.bak /etc/passwd chmod 644 /etc/passwd ----- Writing text files for Linux using an editor running on another Operating System: When you edit a system file, a shell script, a fortune cookie text file, or another file meant to be processed by a Linux shell command or application, make sure that the lines are divided UNIX style (LF only, x0A; not CR/LF, x0D0A as in Windows), otherwise the line breaks will not be recognized as such and the whole file will be treated as a single line. ----- Install SSH: apt-get install ssh Get SSH version: ssh -V Start a SSH connection (port 22): ssh HOST Configure SSH: jed /etc/ssh/sshd_config # If your client can't authenticate to your server, it might be sending the password as a cleartext and your server is not accepting it. In this case you have to enable tunnelled clear text passwords setting PasswordAuthentication yes # Make sure SSH server is not using the outdated Protocol 1. This line should be present in the configuration file instead: Protocol 2 # Set SSH timeout (in seconds). A logged user will be logged out after idling past specified time. ClientAliveInterval 600 ClientAliveCountMax 0 /etc/init.d/ssh restart # Restart SSH to make changes take effect. However note that they will apply to next SSH session. Start a Telnet (unencrypted) connection: telnet HOST # start a telnet connection on HOST at default port 23 telnet HOST:PORT # start a telnet connection on HOST at the specified PORT ----- Bash shells: Show full name of the terminal type: tput longname Show BASH version: echo $BASH_VERSION Command Line Navigation: CURSOR LEFT / CURSOR RIGHT # move cursor one character left or right on command line LEFT ALT+[B/F] # move cursor to previous / next word on command line CURSOR UP / CURSOR DOWN # move to previous / next command in history CTRL+R # search for a command in History: hit CTRL+R, type the string you want to search, hit CTRL+R again to find further matches Switch bash shells: LEFT ALT+[F1-F6] Cycle between bash shells: LEFT ALT+[CURSOR LEFT / CURSOR RIGHT] Show current terminal: tty Terminal line settings: stty -a # Print all terminal line settings in human readable format stty -g # Print all terminal line settings in stty readable format Reset a "corrupted" shell: reset # It will also reset the terminal terminal window size to 80 x 36 characters Clear shell terminal window: clear # Clear screen (if your terminal has a buffer, you can still scroll back to see the text previously on screen) printf "\033c" # clear the screen by scrolling all the text above the visible area (same as CTRL+L) Set Terminal Window Size: echo -e "\e[8;LINES;COLUMNSt" # resize the terminal window to LINES x COLUMNS . For example, tlins=40; tcols=160; echo -e "\e[8;$tlins;${tcols}t"; (or echo -e "\e[8;40;160t" ) resizes the window to 160 x 40 characters. Get Terminal Window Size: tput lines # return the height in characters (number of lines) of the terminal window tput cols # return the width in characters (number of columns) of the terminal window echo "$(tput cols) x $(tput lines)"; # show terminal window size # The environmental variables $LINES and $COLUMNS contain the values of lines and colums of the terminal window, but their value is not preserved within scripts, so that you'll have to add this line to your script assign them the expected values: COLUMNS="$(tput cols)"; LINES="$(tput lines)"; Open a new Bash shell: bash Duplicate a terminal: apt-get install conspy conspy N # N can range from 1 to 6, then execute another conspy N with the same number on another bash shell Show current command line: cat /proc/self/cmdline # typical output: cat/proc/self/cmdline Exit from a non login shell: exit Show last exit code (exit status): echo $? # Return the exit status of last command (0: Success (no errors, if the last performed command was executed properly with a clean exit), 1 (or different than 0): Failure). Note that if you execute the command twice, the second time you'll get the exit status of the first execution of echo #? Exit Codes with special meanings (Reserved Exit Codes): Exit Code Number: Meaning / Example / Comments: 0: No errors / echo "Hello World" / Command executed successfully. Note that quitting a command with CTRL+D will also return 0. 1: Catchall for general errors / let "var1 = 1/0" / Miscellaneous errors, such as "divide by zero" 2: Misuse of shell builtins / empty_function() {} / Missing keyword or command, or permission problem (and diff return code on a failed binary file comparison). 126: Command invoked cannot execute / /dev/null / Permission problem or command is not an executable 127: "command not found" / illegal_command / Possible problem with $PATH or a typo 128: Invalid argument to exit / exit 3.14159 / exit takes only integer args in the range 0 - 255 (see first footnote) 128+n: Fatal error signal "n" / kill -9 $PPID of script / kill -9 returns 137 (128 + 9) ; CTRL+C returns 130 (128 + 2), CTRL+Z returns 148 (128 + 20). 255*: Exit status out of range / exit -1 / exit takes only integer args in the range 0 - 255 Show environment variables: printenv # List all environment variables export -p # List all exported (global scope) variables Change prompt: export PS1="[\t] \u@\h:\w\\\$ " # show time in 24 hours format between square brackets, the logged user name, the host name, the current path, and the root status ("#" if root, "$" otherwise) export PS1="\[\e[1;32m\][\t] \u@\h:\w\\\$ \[\e[0m\] " # same prompt, but colored in light green export PS1="\[\e[1;32m\][\$(date +%a\ %d%b%Y\ %H:%M:%S)] \u@\h:\w\\\$\[\e[0m\] " # full date and time between square brackets, the logged user name, the host name, the current path, and the root status ("#" if root, "$" otherwise), colored in light green. Note that the shell command date is used, invoked with \$() . If $() (without the leading backslash) were used, the date would be generated only the first time and then stored into the variable and wouldn't be updated every time the prompt is shown. Note: if you want to set Windows Command Prompt to match your Linux Shell Prompt you have to set the %prompt% environment variable. For example: set prompt=[$d$s$t]$s%username%@%computername%:$p$+$g$s export PS1="\[\e[1;32m\][\$(date +%Y-%m-%d\ %H:%M:%S\ %a)] \u@\h:\w\\\$\[\e[0m\] " # full date and time (in a sortable format) between square brackets, the logged user name, the host name, the current path, and the root status ("#" if root, "$" otherwise), colored in light green. echo $PS1 # See current prompt Edit the script executed at the shell start-up (user login shell): # Custom Prompt and Aliases can be set here jed ~/.bash_profile Sample bash profile script: (see https://labs.geody.com/systatus/ ) --- #!/bin/bash # ~/.bash_profile # Systatus BASH Shell Start-up # r2020-12-27 fr2016-10-18 # by Valerio Capello - https://labs.geody.com/ - License: GPL v3.0 # Config tshilon="\e[0;33m"; tshilof="\e[0m"; tsalerton="\e[0;31m"; tsalertof="\e[0m"; # Set Terminal Window Size # tlins=40; tcols=160; echo -e "\e[8;$tlins;${tcols}t"; # Get Terminal Window Size COLUMNS="$(tput cols)"; LINES="$(tput lines)"; # Prompt export PS1="\[\e[1;32m\][\$(date +%Y-%m-%d\ %H:%M:%S\ %a)] \u@\h:\w\\\$\[\e[0m\] " # History Date Format export HISTTIMEFORMAT="%F %T " # Aliases alias l="ls -laF --group-directories-first --color=auto" alias d="ls -aF --group-directories-first --color=auto" # Message echo; clear date "+%a %d %b %Y %H:%M:%S %Z (UTC%:z)" echo -n "Hello "; echo -ne "$tshilon"; echo -n "$(whoami)"; echo -ne "$tshilof"; if [[ "$SSH_CONNECTION" ]]; then echo -n " ("; echo -ne "$tshilon"; echo -n "`echo $SSH_CLIENT | awk '{print $1}'`"; echo -ne "$tshilof)"; fi echo -n ", "; echo -n "welcome to "; echo -ne "$tshilon"; echo -n "$(hostname)"; echo -ne "$tshilof"; echo -n " ("; echo -ne "$tshilon"; echo -n "$(hostname -i)"; echo -ne "$tshilof)"; echo "."; echo -n "Machine ID: "; echo -n "$(cat /etc/machine-id) "; echo -n "Boot ID: "; echo -n "$(cat /proc/sys/kernel/random/boot_id) "; echo -n "Session ID: "; echo "$(cat /proc/self/sessionid)"; echo -n "You are "; if [[ -n "$SSH_CONNECTION" ]]; then echo -n "connected remotely via SSH"; elif [[ "${DISPLAY%%:0*}" != "" ]]; then echo -n "connected remotely "; echo -ne "$tsalerton"; echo -n "NOT"; echo -ne "$tsalertof"; echo " via SSH (which is Bad)"; else echo -n "connected locally"; fi utty="$(tty)"; if [[ $utty == "/dev/"* ]]; then utty=$(cut -c 6- <<< $utty); fi echo -n " on $utty"; echo ". Your Terminal Window Size is $COLUMNS x $LINES" if [[ $EUID -eq 0 ]]; then echo -ne "$tsalerton"; echo -n "You have ROOT superpowers!"; echo -e "$tsalertof"; fi echo # Machine # echo -n "Vendor: "; echo "$(cat /sys/class/dmi/id/sys_vendor)"; echo -n "Machine: "; echo "$(cat /sys/class/dmi/id/product_name)"; # echo -n "Machine Type: "; echo "$MACHTYPE"; # echo -n "Board: "; echo "$(cat /sys/class/dmi/id/board_vendor) $(cat /sys/class/dmi/id/board_name)"; # echo -n "BIOS: "; echo "$(cat /sys/class/dmi/id/bios_vendor) $(cat /sys/class/dmi/id/bios_version) $(cat /sys/class/dmi/id/bios_date)"; echo -n "CPU: "; echo -n "$(grep 'model name' /proc/cpuinfo | head -1). "; echo -n "Cores: "; grep -c 'processor' /proc/cpuinfo echo -n "CPU average load: "; uptime | awk -F'[a-z]:' '{print $2}' | xargs | awk '{print "1 m: "$1" 5 m: "$2" 15 m: "$3}'; free -h | xargs | awk '{print "Memory: Size: "$8" Used: "$9" Free: "$10" Avail: "$13}'; df / -Th | xargs | awk '{print "FS: "$9" Type: "$10" Size: "$11" Used: "$12" ("$14") Avail: "$13" ("(100-$14)"%)"}'; echo "Last Boot / Uptime: `uptime -s` (`uptime -p`)"; echo # Software version lsb_release -ds uname -a echo "Bash version: $BASH_VERSION" # Webserver version echo -n "$(/usr/sbin/apache2 -v|head --lines=1) "; echo "$(/usr/sbin/apache2 -v|tail --lines=1)"; openssl version -v php -v|head --lines=1 mysql -V # echo echo -n "Installed Packages: "; dpkg --get-selections | wc -l; # echo "Last installed packages:"; grep install /var/log/dpkg.log | tail --lines=5; echo # Users echo "Last logged users:"; last -n 5 | sed '/^$/d' echo; echo "Currently logged users:"; who echo; echo -n "Current user: "; id echo # Security # Shellshock vulnerability check (reports to root only) if [[ $EUID -eq 0 ]]; then env x='() { :;}; echo Bash vulnerable to Shellshock' bash -c 'echo -n' fi --- You may want to replicate your bash profile script as an executable script (called for example status.sh ) containing information provided in the sections starting from Message and below (you'd better change echo -n "welcome to "; to echo -n "this is ";). Customize the MOTD (Message Of The Day), message shown after logging in, before than starting the shell: jed /etc/motd # edit the MOTD Customize the message shown before than the log in request: jed /etc/issue # edit the welcome message Customize the command line: jed ~/.inputrc Sample .inputrc script: --- # ~/.inputrc # .inputrc # r2020-11-12 fr2020-11-12 # by Valerio Capello - https://labs.geody.com/ - License: GPL v3.0 # Enable autocomplete set disable-completion off # Show autocomplete results in pages set page-completions on # Show file types for completed results (like ls -F ) set visible-stats on # If there are too many completion results, ask to show them all set completion-query-items 100 # Bell # set bell-style audible # Go to next autocomplete result with Shift-Tab "\e[Z": menu-complete # History search (Arrows Up and Down) "\e[A": history-search-backward "\e[B": history-search-forward --- Description of a command: whatis COMMAND Manual of a command: man COMMAND apropos KEYWORD # list all manual pages related to the KEYWORD apropos . | shuf -n 1 | awk '{print$1}' | xargs man # show a random manual page Return the path of a command: which COMMAND Return the type of a command: # Bash builtins (internal commands): alias, bg, bind, break, builtin, case, cd, command, compgen, complete, continue, declare, dirs, disown, echo, enable, eval, exec, exit, export, fc, fg, getopts, hash, help, history, if, jobs, kill, let, local, logout, popd, printf, pushd, pwd, read, readonly, return, set, shift, shopt, source, suspend, test, times, trap, type, typeset, ulimit, umask, unalias, unset, until, wait, while. type COMMAND # Get the command type. If it's a file or an alias, it also returns the path to COMMAND. type -t COMMAND # Get the command type in a single keyword: alias, keyword, function, builtin, file. List internal commands: help help COMMAND # show information about an internal command Identify the type of a file: file PATH/FILE file -z PATH/FILE.zip # Attempts to check the file type of files contained inside a compressed file Alias alias l="ls -laF --color=auto" # create an alias: makes "l" an alias for "ls -laF --color=auto" alias {helloworld,hellow}="echo 'Hello World'" # define multiple aliases alias # list all existing aliases type ALIAS # show the commands associated to the ALIAS unalias ALIAS # remove an alias (if the alias was defined together with other multiple aliases, the other aliases will be unaffected and remain active) Symlinks # Symlinks / Sym links / Symbolic links / Soft links ln -s /PATH/TARGET_FILE /PATH/SYMLINK # Create a SYMLINK to TARGET_FILE . Permissions for the file will be the ones for TARGET_FILE, regardless of the ones for SYMLINK. Use rm to remove the SYMLINK (TARGET_FILE will not be affected) find . -type l ! -exec test -e {} \; -print # Find broken symlinks Repeat last command: !! Start a comment (ignore following text): # Do nothing, successfully: true Do nothing, unsuccessfully: false Execute a command multiple times at a given time interval watch -n SECONDS COMMAND watch -n 1 date # Execute the command date every second Conditional execution: if [[ 1 == 1 ]]; then echo "true"; fi if [[ 1 == 1 ]]; then echo "true"; else echo "false"; fi Execute a random command between two: ((RANDOM%2 == 0)) && echo "Yes" || echo "No" if [[ $((RANDOM % 2)) == 0 ]]; then echo "Yes"; else echo "No"; fi Find the time required to perform a command line: # It may useful to compare commands, or parameters of the same command, and see which one is faster. # The "real" value is the actual elapsed time which is the data you most likely need, while The "user" value shows the time spent by the code in user mode, and the "sys" value shows the time spent by the code in kernel mode. time (ls) # Shows the time needed to list the current directory Process priority: nice -n0 COMMAND # execute a command with default priority (0) nice -n20 COMMAND # execute a command with the lowest priority (20) nice -n-20 COMMAND # execute a command with the highest priority (-20). Only root can assign negative (high) priority renice 10 PID # reassign priority to an existing process List running processes and their PIDs: echo $$ # show the PID of the current shell (or script, if invoked from a script) ps # show all processes with a tty in current shell ps -a # show all processes with a tty in all shells ps -A # show all processes ps -A|grep "STRING" # show all processes that matches STRING ps -A|awk '$2~/pts\// { print $0 }' # show all processes for remote terminal devices ps -A|grep " pts/." # show all processes for remote terminal devices pgrep -f "STRING" # show the PID for all processes that matches STRING pgrep -fo "STRING" # show the PID for the least recent (oldest) process that matches STRING pgrep -fn "STRING" # show the PID for the most recent (newest) process that matches STRING ps auxf | sort -nr -k 4 # show processes sorted by memory usage ps auxf | sort -nr -k 4 | head --lines=10 # show processes sorted by memory usage (first 10 only) ps auxf | sort -nr -k 3 # show processes sorted by CPU usage ps auxf | sort -nr -k 3 | head --lines=10 # show processes sorted by CPU usage (first 10 only) Show running processes and system load: top # Show running processes and system load. Press "q" to quit top -o %CPU # Show running processes and system load, sorted by CPU usage. Press "q" to quit top -o %MEM # Show running processes and system load, sorted by Memory usage. Press "q" to quit top -b -n1 | grep COMMAND | awk '{ SUM += $10} END { print SUM }' # Show the sum of CPU usage grouped by COMMAND (for example: apache2 , mysqld , fail2ban-server ) top -b -n1 | grep COMMAND | awk '{ SUM += $9} END { print SUM }' # Show the sum of memory usage grouped by COMMAND (for example: apache2 , mysqld , fail2ban-server ) Show running processes accessing the disk or doing other Input/Output activity: apt-get install iotop iotop --only # press "q" to quit Services: service --status-all # Show the status of all services (+ for active, - for inactive) service SERVICE status | grep -m 1 'Active:' | cut -d ":" -f2- | sed 's/^ *//g' # Show the status of a SERVICE, if it's active (running) or it's inactive. Some popular services are: ssh, apache2, tor, mysql, sendmail, fail2ban, ufw Jobs: CTRL+Z # Stop (Pause) current program and send it to the background. Returns Signal 20 (Exit Code 148) jobs # List all jpbs (programs) in the background with their job id fg JOB_ID # Restart the specified job (program) and restore it to the foreground # If you get the message "There are stopped jobs." when logging out ('logout' from the console), run 'jobs' to see the list of jobs in the background. Then you have to options: the cleanest one is to bring the jobs to the foreground ('fg JOB_ID') and then close them as required by that specific job, or list all the running processes ('ps'), look for the one corresponding to the job stuck in the background and kill it ('kill -9 PID'). Processes: # LSBInitScripts require to support the following actions (as per Debian Policy, chapter 9.3.2 Writing the scripts): start, stop, restart, try-restart (optional), reload (optional), force-reload, status (optional but encouraged, see Debian Bug #291148 2005-01-19), version (not required, but useful). /etc/init.d/COMMAND start # Start the service. Note: if the process is already running it should return success, not start another copy. /etc/init.d/COMMAND stop # Stop the service. /etc/init.d/COMMAND restart # Restart the service if it's already running, otherwise just report success (optional). /etc/init.d/COMMAND try-restart # Restart the service if it's already running, otherwise just report success (optional). /etc/init.d/COMMAND reload # Cause the configuration of the service to be reloaded without actually stopping and restarting the service (optional). /etc/init.d/COMMAND force-reload # Cause the configuration to be reloaded if the service supports this, otherwise restart the service. /etc/init.d/COMMAND status # Report the current status of the service (optional but encouraged, see Debian Bug #291148 2005-01-19) /etc/init.d/COMMAND version # Return version number and / or release date (currently not required for LSB compliance) Show which user launched a process: fuser PID Terminate a process: kill -3 PID # quit a process kill -15 PID # term a process kill -9 PID # kill a process (most effective to quit an unresponsive process) Terminate all processes with a matching name: pgrep NAME | xargs kill -9 killall NAME killall -I NAME # ignore cases killall -i NAME # ask for confirmation before to kill Note: If a service won't stop, kill all its PIDs. For example with Apache 2: pgrep apache2 | xargs kill -9 Show information about all loaded module: lsmod Show Boot parameters for last (current) boot: cat /proc/cmdline Show open files: apt-get install lsof lsof # Show all open files lsof PATH # Show all open files within PATH lsof / | awk '{ if($7 > 1048576) print $7/1048576 "MB" " " $9 " " $1 }' | sort -n -u | tail --lines=10 # show the 10 largest open files with their size in MB and the processes that keep them open Test to check if your Bash shell is vulnerable to Shellshock (you should only see a dot if your system is not vulnerable): # Shellshock vulnerability: CVE-2014-6271, discovered on 12 September 2014, initially patched and disclosed on 24 September 2014; related vulnerabilities CVE-2014-6277, CVE-2014-6278, CVE-2014-7169, CVE-2014-7186, CVE-2014-7187. env x='() { :;}; echo Bash vulnerable to Shellshock' bash -c 'echo .' ----- Information about the vendor: cat /sys/class/dmi/id/sys_vendor # System Vendor Information about the machine (virtual or physical): cat /sys/class/dmi/id/product_name # Product Name cat /sys/class/dmi/id/product_uuid # Product UUID cat /sys/class/dmi/id/product_serial # Product Serial cat /sys/class/dmi/id/product_version # Product Version (if any) cat /sys/class/dmi/id/chassis_vendor # Chassis Vendor (if there is an enclosure) cat /sys/class/dmi/id/chassis_type # Chassis Type cat /sys/class/dmi/id/chassis_version # Chassis Version (if any) cat /sys/class/dmi/id/chassis_serial # Chassis Serial (if any) cat /sys/class/dmi/id/chassis_asset_tag # Chassis Asset Tag (if any) cat /sys/class/dmi/id/board_vendor # Board Vendor cat /sys/class/dmi/id/board_name # Board Name cat /sys/class/dmi/id/board_version # Board Version (if any) cat /sys/class/dmi/id/board_serial # Board Serial (if any) cat /sys/class/dmi/id/board_asset_tag # Board Asset Tag (if any) cat /sys/class/dmi/id/bios_vendor # BIOS Vendor cat /sys/class/dmi/id/bios_version # BIOS Version cat /sys/class/dmi/id/bios_date # BIOS Date Show Machine's Unique ID: cat /etc/machine-id # show the unique machine ID of the local system that is set during installation. The machine ID is a single newline-terminated, hexadecimal, 32-character, lowercase machine ID string. When decoded from hexadecimal, this corresponds with a 16-byte/128-bit string (from the manual). Also in /var/lib/dbus/machine-id Boot ID: cat /proc/sys/kernel/random/boot_id # show the random ID regenerated at each boot Session ID cat /proc/self/sessionid # show the session ID, which is unique for each login Random UUID: cat /proc/sys/kernel/random/uuid # generate a random UUID, different every time is invoked Show system name: hostname cat /proc/sys/kernel/hostname Show system's IP address: hostname -i # Local IP ifconfig | grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' | grep -Eo '([0-9]*\.){3}[0-9]*' | grep -v '127.0.0.1' # Global IPv4 ifconfig | sed -En 's/127.0.0.1//;s/.*inet (addr:)?(([0-9]*\.){3}[0-9]*).*/\2/p' # Global IPv4 Show client's IP address in a SSH connection: echo $SSH_CLIENT | awk '{print $1}' echo $SSH_CONNECTION | awk '{print $1}' # Note that $SSH_CONNECTION (not processed with awk) also contains the server's IP address echo "$(whoami) (`echo $SSH_CLIENT | awk '{print $1}'`) @ $(hostname) ($(hostname -i))"; # client (client IP) @ host (host IP) Change system name as NEW_NAME: hostname NEW_NAME jed /etc/hostname # change it as NEW_NAME jed /etc/hosts # you'd better also add NEW_NAME as an alias hostname for 127.0.0.1 (localhost) Show system information: uname -a uname -r # show only kernel version cat /proc/sys/kernel/osrelease # kernel version cat /etc/issue # Welcome message: contains Linux version (unless otherwise edited by the user) cat /etc/os-release # Get Distribution Name and Version awk -F'=' '/^ID=/ {print $2}' /etc/os-release # Get Distribution ID with quotes awk -F'=' '/^ID=/ {print $2}' /etc/os-release | sed -e 's/"//g' # Get Distribution ID without quotes awk -F'=' '/^ID=/ {print $2}' /etc/os-release | tr -d '"' # Get Distribution ID without quotes awk -F'=' '/^NAME=/ {print $2}' /etc/os-release # Get Distribution Name with quotes awk -F'=' '/^NAME=/ {print $2}' /etc/os-release | sed -e 's/"//g' # Get Distribution Name without quotes awk -F'=' '/^NAME=/ {print $2}' /etc/os-release | tr -d '"' # Get Distribution Name without quotes awk -F'=' '/^VERSION=/ {print $2}' /etc/os-release # Get Distribution Version with quotes awk -F'=' '/^VERSION=/ {print $2}' /etc/os-release | sed -e 's/"//g' # Get Distribution Version without quotes awk -F'=' '/^VERSION=/ {print $2}' /etc/os-release | tr -d '"' # Get Distribution Version without quotes awk -F'=' '/^VERSION_ID=/ {print $2}' /etc/os-release # Get Distribution Version ID with quotes awk -F'=' '/^VERSION_ID=/ {print $2}' /etc/os-release | sed -e 's/"//g' # Get Distribution Version ID without quotes awk -F'=' '/^VERSION_ID=/ {print $2}' /etc/os-release | tr -d '"' # Get Distribution Version ID without quotes awk -F'=' '/^PRETTY_NAME=/ {print $2}' /etc/os-release # Get Distribution Pretty Name (Name and Version) with quotes awk -F'=' '/^PRETTY_NAME=/ {print $2}' /etc/os-release | sed -e 's/"//g' # Get Distribution Pretty Name (Name and Version) without quotes awk -F'=' '/^PRETTY_NAME=/ {print $2}' /etc/os-release | tr -d '"' # Get Distribution Pretty Name (Name and Version) without quotes lsb_release -a # Get Distribution Name and Version (more detailed than /etc/os-release ) lsb_release --description|awk '{$1 = ""; print $0}'|sed 's/^ //g' # Get Distribution Pretty Name (more detailed than /etc/os-release ) lsb_release -ds # Get Distribution Pretty Name (more detailed than /etc/os-release ) cat /etc/debian_version # Debian version Show the kernel ring buffer (to print out the bootup messages): dmesg Show last 20 lines of the system messages file: tail --lines=20 /var/log/messages System Up Marker: By default, on most Debian based distributions up to Debian Etch, every 20 minutes syslog marks the message file with "-- MARK --" to log that the system is up and running. You can modify this behavior by editing the syslog startup file: jed /etc/init.d/sysklogd Locate the line that defines the variable SYSLOGD (generally, SYSLOGD="" ). If it's missing you can add it at the beginning of the configuration file, where other variables are defined. and use the switch -m to set the delay between marks in minutes, or set it to 0 to disable the notification. Example: SYSLOGD="-m 60" marks the message file every hour, SYSLOGD="-m 0" disable the notification. Restart syslog to make the change effective: /etc/init.d/sysklogd restart After Etch, the time marker is disabled by default. You can enable it editing the rsyslog configuration file and enabling the immark module (which is commented out): jed /etc/rsyslog.conf module(load="immark") # provides --MARK-- message capability You should also be able to set the delay between marks setting the RSYSLOGD_OPTIONS variable in rsyslog, unfortunately RSYSLOGD_OPTIONS is ignored: jed /etc/init.d/rsyslog RSYSLOGD_OPTIONS="-m 60" marks the message file every hour Restart syslog to make the change effective: /etc/init.d/rsyslog restart Check last marks in syslog and messages: # Time marks should coincide in both syslog and messages grep "MARK" /var/log/syslog | tail --lines=10 ; echo ; grep "MARK" /var/log/messages | tail --lines=10 # Check last times rsyslog was stopped grep "] exiting" /var/log/messages | tail --lines=10 # You could also grep "] exiting" /var/log/syslog | tail --lines=10 # Check last times rsyslog was started grep "] start" /var/log/messages | tail --lines=10 # You could also grep "] start" /var/log/syslog | tail --lines=10 Start Debian base system configuration tool: base-config Show Hardware info: dmidecode -t TYPE_OR_SUPERTYPE # Valid types are: 0: BIOS, 1: System, 2: Base Board, 3: Chassis, 4: Processor, 5: Memory Controller, 6: Memory Module, 7: Cache, 8: Port Connector, 9: System Slots, 10: On Board Devices, 11: OEM Strings, 12: System Configuration Options, 13: BIOS Language, 14: Group Associations, 15: System Event Log, 16: Physical Memory Array, 17: Memory Device, 18: 32-bit Memory Error, 19: Memory Array Mapped Address, 20: Memory Device Mapped Address, 21: Built-in Pointing Device, 22: Portable Battery, 23: System Reset, 24: Hardware Security, 25: System Power Controls, 26: Voltage Probe, 27: Cooling Device, 28: Temperature Probe, 29: Electrical Current Probe, 30: Out-of-band Remote Access, 31: Boot Integrity Services, 32: System Boot, 33: 64-bit Memory Error, 34: Management Device, 35: Management Device Component, 36: Management Device Threshold Data, 37: Memory Channel, 38: IPMI Device, 39: Power Supply; Valid supertypes are: bios (0, 13), system (1, 12, 15, 23, 32), baseboard (2, 10), chassis (3), processor (4), memory (5, 6, 16, 17), cache (7), connector (8), slot (9). dmidecode -t 6|grep -i "Size" # Return installed memory banks dmidecode -t 1|grep -i "Manufacturer\|Product\|Serial" # Return system's Manufacturer, Product Name, and Serial Number Show RAM info: cat /proc/meminfo grep MemTotal /proc/meminfo # return total RAM seen by the system egrep 'Mem|Cache|Swap' /proc/meminfo # return information about RAM, Cache, Swap vmstat -s # show information about memory activity Show memory usage: free # show total, used, free, available memory in Bytes free -w # show total, used, free, available memory in Bytes, in wide format (separating values for buffers and cache) free -k # show total, used, free, available memory in KB free -m # show total, used, free, available memory in MB free -h # show total, used, free, available memory in human readable format free -wh -c 3 -s 1 # show memory usage 3 times with 1 second interval free -h | xargs | awk '{print "Memory: Size: "$8" Used: "$9" Free: "$10" Avail: "$13}' # Return Total, Used, Free, Available Memory Swap memory: cat /proc/sys/vm/swappiness # Show system Swappiness (Range: 0 - 100 ; 0: Never use Swap (disabled) , 100: Always use Swap ; Default: 60 ) sysctl vm.swappiness=10 # Set Swappiness to 10 swapoff -a # Disable Swap memory swapon -a # Enable Swap memory (wait about 30 seconds after disabling Swap before to enable it again, to let your system clear Swap memory proprerly swapon --show # Show summary of swap devices and their swap memory usage in human readable format swapon --show --noheadings --raw --bytes # Show summary of swap devices and their swap memory usage in bytes without headers swapon --show --noheadings --raw --bytes | xargs | awk '{print $1","$3","$4}' # Show swap memory device name and usage in bytes, comma separated swapon --show --noheadings --raw | xargs | awk '{print "Swap File: Dev: "$1" Total: "$3" Used: "$4}' # Show swap memory device name and usage in verbose human readable format for file in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done # List Swap space usage per process for file in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done | sort -k 2 -n -r # List Swap space usage per process and sort results Virtual memory statistics: vmstat # report information about processes, memory, paging, block IO, traps, and cpu activity. vmstat -s # event counters and memory statistics vmstat -d # disk statistics Show CPU info: lscpu lscpu | grep "mode(s)" # return CPU op-modes (32-bit, 64-bit) lscpu | grep "Byte Order" # return CPU Byte Order (Little Endian, Big Endian) cat /proc/cpuinfo grep 'model name' /proc/cpuinfo # return CPU model grep -c 'processor' /proc/cpuinfo # count CPU cores grep 'flags' /proc/cpuinfo # return available features of the CPU cat /proc/loadavg # show load average of the system: the first three fields in this file are load average figures giving the number of jobs in the run queue (state R) or waiting for disk I/O (state D) averaged over 1, 5, and 15 minutes. They are the same as the load average numbers given by uptime(1) and other programs. The fourth field consists of two numbers separated by a slash (/). The first of these is the number of currently executing kernel scheduling entities (processes, threads); this will be less than or equal to the number of CPUs. The value after the slash is the number of kernel scheduling entities that currently exist on the system. The fifth field is the PID of the process that was most recently created on the system. grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage "%"}' # show CPU load uptime | awk -F'[a-z]:' '{print $2}' # Show load average of the system over the last 1, 5, and 15 minutes. echo -n "CPU average load (Cores: "; grep 'cpu cores' /proc/cpuinfo | xargs | awk '{printf $4}'; echo -n "): "; uptime | awk -F'[a-z]:' '{print $2}' | xargs | awk '{print "1 m: "$1" 5 m: "$2" 15 m: "$3}'; # Show load average of the system over the last 1, 5, and 15 minutes (verbose). Show Display info: cat /sys/class/graphics/*/name # Show Display Adapter Name cat /sys/class/drm/*/modes # Show available Video modes cat /sys/class/graphics/*/modes # Show current Video mode cat /sys/class/graphics/*/virtual_size # Show Virtual Display resolution cat /sys/class/graphics/*/bits_per_pixel # Show color depth List all PCI buses and devices: lspci Show system temperature (if system supports ACPI): apt-get install acpi acpi -t ----- Localization: dpkg-reconfigure locales # edit locale / add new locales locale # show locale settings locale -a # show available locales locale|cut -d= -f1|xargs locale -kc|less # list fields in locale database locale territory # show the locale territory # echo "This server is configured for `locale territory`" printf "%'d\n" 1234567 # print an integer number grouping thousands with the locale configuration l /usr/share/i18n/locales/ # List all available locales l /usr/share/i18n/locales/en_* # List all available English locales localectl # Show system locales cat /etc/default/locale # Show system locale variables jed /etc/default/locale # Set system locale variables Set the following variables to the given values if you want to get the date in 12 or 24 hours: # 24 hours ---------- LANG=en_IE.UTF-8 LANGUAGE=en_IE:en ---------- # 12 hours (AM/PM) ---------- LANG=en_US.UTF-8 LANGUAGE=en_US:en ---------- jed /etc/locale.gen # Locales to be generated. Uncomment (remove # ) the locales you want to generate locale-gen # generate locales (the ones anabled in /etc/locale.gen ) ----- List of all commands entered in the shell (history): jed ~/.bash_history # edit history file manually cat ~/.bash_history # show all commands in history (up to last session) less ~/.bash_history # navigate in history history | grep "apt-get install \|apt install " # Search for last packages installed manually cat ~/.bash_history | tr "\|\;" "\n" | sed -e "s/^ //g" | cut -f 1 -d " " | sort -n | uniq -c | sort -nr | head -n 10 # show the 10 most used commands in history (including piped commands) history | awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a; }' | grep -v "./" | column -c3 -s " " -t | sort -nr | nl | head -n 10 # show the 10 most used commands in history (first command only). NOTE: if dates are enabled in history, it will show days when more commands were issued history | awk '{CMD[$4]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a; }' | grep -v "./" | column -c3 -s " " -t | sort -nr | nl | head -n 10 # show the 10 most used commands in history (first command only) if dates are enabled history # show all commands in history together with their offset history -d OFFSET # delete the specified history entry history -c # clear the whole history file !!:p # show last executed command !-1:p # show last executed command !-2:p # show second last executed command !TEXT:p # show last command starting with TEXT (for example, ls) !! # repeat (execute again) last executed command !-1 # repeat (execute again) last executed command !-2 # repeat (execute again) second last executed command !TEXT:p # repeat (execute again) last command starting with TEXT (for example, ls) export HISTTIMEFORMAT="%F %T " # Set history date format (normally by default date and time of executed commands is not preserved in history) echo $HISTTIMEFORMAT # See history date format Edit user creation preferences: jed /etc/adduser.conf Create a new user: # useradd is similar to adduser adduser USER # add a new USER with a home directory in the default path ( /home/USER ) adduser --home DIR USER # add a new USER specifying a non standard path for the home directory adduser --no-create-home USER # add a new USER without a home directory Modify a user information: usermod [OPTIONS] USER Edit user deletion preferences: jed /etc/deluser.conf Remove a user: # userdel is similar to deluser deluser USER # remove the specified user deluser --remove-home USER # remove the specified user and his home directory and mail spool deluser --remove-all-files USER # remove the specified user and all files owned by the user (be careful) Change user contact information: chfn [OPTIONS] USER Change a user's default shell: chsh [OPTIONS] USER Get a user hashed password: getent shadow|grep "USER"|cut -f 2- -d ":"|cut -f 1 -d ":" # Get the hash type, salt and hashed password for given USER getent shadow|grep "$(whoami)"|cut -f 2- -d ":"|cut -f 1 -d ":" # Get the hash type, salt and hashed password for current user getent shadow|grep "$(whoami)"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 4- -d "$" # Get the hashed password for given USER getent shadow|grep "$(whoami)"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 4- -d "$" # Get the hashed password for current user getent shadow|grep "USER"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 2- -d "$"|cut -f 2- -d "$"|cut -f 1 -d "$" # Get the salt for given USER getent shadow|grep "$(whoami)"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 2- -d "$"|cut -f 2- -d "$"|cut -f 1 -d "$" # Get the salt for current user getent shadow|grep "USER"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 2- -d "$"|cut -f 1 -d "$" # Get the hash type for given USER (1: md5, 5: sha-256, 6: sha-512) getent shadow|grep "$(whoami)"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 2- -d "$"|cut -f 1 -d "$" # Get the hash type for current user (1: md5, 5: sha-256, 6: sha-512) Change a user password: passwd USER Set a user password to expire: # note that having frequently changing passwords is usually not a good policy, as it becomes harder for users to remember them, and they generally end up with weak passwords chage -m 3 -M 30 -w 2 USER # USER's password can last up to 30 days, can't be changed earlier than 3 days after last change, and will be warned 2 days before password expiration Create a new user group: # addgroup is a link to adduser ( adduser --group ). There's also a groupadd command. addgroup GROUP Modify a group information: groupmod [OPTIONS] GROUP Remove a group: # There's also a delgroup link to deluser ( deluser --group ) groupdel GROUP Add a user to a group: adduser USER GROUP Execute a command with root privileges (root password will be asked): sudo COMMAND Change current user's password: passwd Show all existing users: getent passwd # format: username:password (usually shadowed, a 'x' is shown instead):user id:group id:real name:home path:console. If you are root you can access directly the file with cat /etc/passwd getent shadow # show users and their hashed password. If you are root you can access directly the file with cat /etc/shadow getent passwd | grep -c . # count all existing users Show all logged users: users # list all logged users who # show information about all logged users w # show more information than "who" who | grep -c . # count all logged users Show last logged users: last # Show last logins last USER # show last times when USER logged in last -n 10 # show last 10 logged users last -n 5 USER # show last 5 times when USER logged in touch /var/log/lastlog # Create the login log file to enable lastlog lastlog # Show last time each user logged in lastlog -u USER # Show last time the USER logged in lastlog -t 30 # Show only users who logged in during the last 30 days lastlog -b 365 # Show only users who last logged more than 365 days ago lastb -n 10 # show last 10 bad login attempts (which are likely break-in attempts) lastb -n 10 root # show last 10 bad login attempts to root (which are likely break-in attempts) grep "Accepted" /var/log/auth.log | tail --lines=10 # See last accepted password events for succeful logins grep "Failed" /var/log/auth.log | tail --lines=10 # See last rejected password events for failed login attemps (which include possible break-in attempts) grep "Failed password for invalid user" /var/log/auth.log | tail --lines=10 # See last rejected password events for failed login attemps with non existing users (which are likely break-in attempts) Show failed login attempts: To enable faillog: ----- Create the faillog log file: touch /var/log/faillog jed /etc/pam.d/common-auth Add these lines at the top of the file: # auth required pam_tally.so per_user magic_root even_deny_root deny=5 unlock_time=86400 onerr=fail # Note that pam_tally and pam_tally2 modules have been removed from PAM starting from Debian 11 (Bullseye), so this is valid up to Debian 10 (Buster) auth required pam_faillock.so per_user magic_root even_deny_root deny=5 unlock_time=86400 onerr=fail # Failed logins are logged to /var/log/faillog by default jed /etc/pam.d/sshd Add the following lines immediately before @include common-auth (generally at the beginning of the file): # Log failed login attempts to /var/log/faillog # auth required pam_tally.so per_user magic_root even_deny_root deny=5 unlock_time=86400 onerr=fail # Note that pam_tally and pam_tally2 modules have been removed from PAM starting from Debian 11 (Bullseye), so this is valid up to Debian 10 (Buster) auth required pam_faillock.so per_user magic_root even_deny_root deny=5 unlock_time=86400 onerr=fail # Failed logins are logged to /var/log/faillog by default jed /etc/ssh/sshd_config Enable PAM, search for UsePAM and set it to yes if not enabled, or add the whole line if it's missing: UsePAM yes Restart SSH: service ssh restart ----- faillog # Show all users who attempted to log in without success faillog -a # Show all failed login attempts including the ones of users who eventually logged in failed -u USER # Show all failed login attempts for the specified USER, even if he eventually logged in faillog -t 30 # Show only failed logins occurred during the last 30 days faillog -l 5 # Lock the account for 5 seconds after each failed attempt faillog -m 5 # Disable the account after 5 failed attempts (0 means that infinite attempts are allowed. You'd better leave the value for root to 0 to prevent a DoS attack) faillog -u USER -r # Reset the counter of failed logins for the given user, enabling his account again if it was locked because of too many failed attempts (as specified in faillog -m N) faillog -r # Reset counters of failed logins for all users cat /var/log/faillog # Show the actual failed attempts log file grep "authentication failure" /var/log/messages # extract failed login attempts from the messages file Show current user's name: whoami Show information about a user: finger USER Show groups to which current user belongs: groups Show user and group IDs: id # show user and group IDs for the current user id USER # show user and group IDs for the specified USER Send a message to a logged user (to the output console of his terminal): # Check the user device with a w or a who command first then redirect the output of an echo command to such device echo -ne "Hello\n">/dev/pts/1 # Send "Hello" to the user logged with pts/1 # Send a message to a logged user (to the output console of his terminal) who's using a specific process (identified by its PID): echo -ne "Hello\n" > /proc/PID/fd/0 Send keystrokes to a terminal: # (use ps -A|grep "PROCESS_NAME" to find in which terminal is running a specific process, in case you need to send keystroke to a process waiting for an input) perl -e '$TIOCSTI = 0x5412; $tty = "/dev/pts/TERMINAL"; $char = "\n"; open($fh, ">", $tty); ioctl($fh, $TIOCSTI, $char);'; # Send a single keystroke (ENTER in this example) to the given TERMINAL perl -e '$TIOCSTI = 0x5412; $tty = "/dev/pts/TERMINAL"; $str = "TEXT_STRING\n"; for my $i (1..length($str)) { open($fh, ">", $tty); $char=substr($str, $i-1, 1); ioctl($fh, $TIOCSTI, $char); }'; # Send a TEXT_STRING to the given TERMINAL Send a message to all logged users: wall PATH/FILE # show the content of FILE to all logged users (max 20 lines) wall # use standard input (normally the keyboard) to show a message to all logged users. Message must be terminated with an EOF (End Of File) character (usually CTRL+D) Execute a command as another user (impersonated user password will be requested): su USER COMMAND Start a console as another user (impersonated user password will be requested): su USER su # if no user is specified, then root is assumed by default ----- File status and information: file PATH/FILE # identify file type stat PATH/FILE # Show file properties (file status) stat --printf="%U" PATH/FILE # Get user name of the owner of the file stat --printf="%u" PATH/FILE # Get user ID of the owner of the file stat --printf="%s" PATH/FILE # Get file size wc -c PATH/FILE | awk '{print $1}' | tr -d '\n' # Get file size basename PATH/FILE # return the file name (FILE) of the specified FILE s1='PATH/FILE'; echo ${s1#$(dirname "$s1")/}; # return the file name (FILE) of the specified FILE dirname PATH/FILE # return the path (from root) of the specified FILE realpath PATH/FILE # Get the full path of a file s1='DIR1/DIRN/FILE'; echo ${s1#$(dirname "$(dirname "$s1")")/}; # return the last directory (DIRN) and the file name (FILE) of the specified FILE Search for a file: find PATH -type f # find all files in PATH and its subdirectories find PATH -type d # find all subdirectories in PATH and its subdirectories find PATH -name FILE_MASK # find all files and directories with given mask inside PATH and its subdirectories. Example: find / -name a* # find all files and directories starting with a in the server find PATH -type f -name FILE_MASK # find all files with given mask inside PATH and its subdirectories. Example: find / -type f -name *.php # find any PHP file in the server find PATH -type d -name FILE_MASK # find all directories with given mask inside PATH and its subdirectories. Example: find / -type d -name test* # find all test directories in the server find PATH -type f -regex ".*\.EXT$" # Find all files in PATH and its subdirectories with given EXTension find PATH -type f -regex ".*\.[0-9]*$" # Find all files in PATH and its subdirectories with a numeric extension whereis FILENAME # return the paths of files with matching FILENAME locate NAME # return all paths and files with a matching NAME string grep -r "STRING" PATH # search for all files containing the given STRING, within the PATH and its subdirectories (-r) grep -r -l "STRING" PATH # search for all files containing the given STRING, within the PATH and its subdirectories (-r) and return just matching files name and not each occurrence of the string within every file (-l) find PATH | xargs grep "STRING" -sl # search for all files containing the given STRING, within the PATH and its subdirectories, returns only filenames with path find PATH -type f -newermt "2019-01-01" \! -newermt "2020-09-01" -exec ls -ld {} \; # Find all files in PATH and its subdirectories created between the two given dates find PATH -type f -newermt "2019-01-01" \! -newermt "2020-09-01" -exec ls -ld {} \; # Find all subdirectories in PATH and its subdirectories created between the two given dates Copy a file or a directory: cp -rp SOURCE DESTINATION # copy a file or a directory tree preserving owners and properties mkdir -p PATH && cp -rp SOURCE $_ # copy a file or a directory tree (preserving owners and properties) and create destination path if it doesn't exist Copy a file over a network: apt-get install netcat # On the Destination computer (listener). Make sure the PORT you are listening isn't blocked. Timeout (-w) is set to 3 seconds: nc -vvn -l -p PORT -w 3 > DEST_PATH/FILE # On the Source computer (sender). Make sure the PORT you are sending to isn't blocked. It will quit 2 seconds after receiving an EOF. If -q is not specified, then netcat must be killed manually: nc -vvn DESTINATION_IP PORT -q 2 < SOURCE_PATH/FILE Copy a file securely (through SSH) over a network (it will ask for passwords if needed for authentication on remote hosts): # Use blowfish encryption (-c blowfish) instead of the default TripleDES (3DES) because it's more secure and faster. scp -c blowfish SOURCEPATH/NAME DESTUSER@DESTHOST:DESTPATH/NAME # copies a file from the local system to a remote host scp -c blowfish SOURCEUSER@SOURCEHOST:DSOURCEPATH/NAME DESTPATH/NAME # copies a file from a remote host to the local system scp -c blowfish SOURCEUSER@SOURCEHOST:DSOURCEPATH/NAME DESTUSER@DESTHOST:DESTPATH/NAME # copies a file from a remote host to another remote host scp -c blowfish -r SOURCEUSER@SOURCEHOST:DSOURCEPATH DESTUSER@DESTHOST:DESTPATH/ # copies a directory tree from a remote host to another remote host Move a file or a directory tree (or rename): mv SOURCEPATH/NAME DESTPATH/NAME for a in *; do mv -v "$a" "${a//STRING_SEARCH/STRING_REPLACE}"; done # batch rename all files in the current directory for a in *; do mv -v "$a" "${a}_`date --iso-8601`"; done # append date as YYYY-MM-DD to all file names in the current directory (note that it will be appended after any extension) find PATH -type f -name "*" -exec mv "{}" "{}".txt \; # add an extension (.txt in this example) to all files in given PATH find PATH -type f -not -name "*.*" -exec mv "{}" "{}".txt \; # Add an extension (.txt in this example) to all files without an extension in given PATH for i in `ls PATH -1`; do mv $i "${i,,}" ; done # rename all files in the given PATH to lowercase for i in `ls PATH -1`; do mv $i "${i^^}" ; done # rename all files in the given PATH to uppercase for f in * ; do mv "$f" "$f.txt"; done # add .txt extension (suffix) to all files (in the current directory) for f in *.txt ; do mv "$f" "${f%txt}asc"; done # change extension (suffix) from .txt to .asc for f in *.txt ; do mv "$f" "${f%.txt}"; done # remove .txt extension (suffix) for f in * ; do mv "$f" "prefix_$f"; done # add _prefix prefix to all files (in the current directory) for f in prefix_* ; do mv "$f" "${f#prefix_}"; done; # remove prefix_ for f in prefix_* ; do mv "$f" "x_${f#prefix_}"; done; # replace prefix_ with x_ Synchronize two directories: apt-get install rsync rsync -a -v /SOURCE_PATH/* /DEST_PATH/ # copy all files from /SOURCE_PATH that doesn't exist in /DEST_PATH rsync -a --delete -v /SOURCE_PATH/* /DEST_PATH/ # copy all files from /SOURCE_PATH that doesn't exist in /DEST_PATH, and delete all files in /DEST_PATH that doesn't (no longer) exist in /SOURCE_PATH rsync -a --delete -v rsync://www.example.com/dir/ /DEST_PATH # synchronize from an external rsync directory Create a link to a file: ln -s PATH/ACTUAL_FILE PATH/LINK_TO_CREATE Create an empty file: touch PATH/NAME # Create an empty file. If the given file already exists, it updates its access date Create a dummy file: fallocate -l 1K PATH/NAME # generate a dummy file sized 1Kbyte (doesn't work on ext3 and ZFS, and could contain sensible information from previously unallocated data of deleted files) dd if=/dev/zero of=PATH/NAME bs=1K count=2 # generate a dummy file sized 2Kbyte and filled with zeroes (slower than fallocate, but it does work on ext3 and ZFS and the output file doesn't contain unwanted data) tr '\000' $'\xff' < /dev/zero | dd of=PATH/NAME bs=1K count=1 # generate a dummy file sized 1Kbyte and filled with hex FF (slower than fallocate, but it does work on ext3 and ZFS and the output file doesn't contain unwanted data) dd if=/dev/random of=PATH/NAME bs=1K count=2 # generate a dummy file sized 2Kbyte and filled with random data (slower than fallocate, but it does work on ext3 and ZFS and the output file doesn't contain unwanted data) Create a unique temporary file or directory with a pseudo random name: mktemp "tmp_XXXXXX" # Create an empty file with a random name generated from the given mask, for example tmp_92pIKB or tmp_Z1Pdz1 mktemp -d "tmp_XXXXXX" # Create an empty directory with a random name generated from the given mask, for example tmp_olD9rA or tmp_5Fw0JI Sample usage of mktemp (from mktemp man): TMPFILE=`mktemp /tmp/example.XXXXXX` || exit 1 echo "program output" >> $TMPFILE Split a file in chunks: split -b=BYTES PATH/FILE PREFIX # split FILE in chunks of BYTES bytes, using PREFIX to name generated chunks split -l LINES PATH/FILE.txt PREFIX # split PATH/FILE.txt in chunks of LINES lines, using PREFIX to name generated chunks Rebuild a split file (concatenate files): cat PREFIX* > PATH/FILE sed -e '$a\' test?.txt > testx1b.txt # concatenate text files appending an empty line at the end of every line if there isn't already one (for r in PREFIX*;do sed s/\$/\ $r/ < "$r";done) > PATH/FILE # concatenate text files appending the file name after each line cat $(ls PREFIX* -t) > PATH/FILE # concatenate files sorted by date and time (newer first, older last) cat $(ls PREFIX* -tr) > PATH/FILE # concatenate files sorted by date and time (older first, newer last) Type a file: cat PATH/FILE cat -n PATH/FILE # show a text file with line numbers cat PATH/FILE | tr [:upper:] [:lower:] # show a text all in lower case cat PATH/FILE | tr [:lower:] [:upper:] # show a text all in Upper case tac PATH/FILE # show a text from the last line to the first (thus listing lines backwards), useful to ordering elements in a last-in, first-out (LIFO) way fold -w COLOUMN PATH/FILE # wrap a text file, cutting lines at the given coloumn. Useful to format data as input for a program fmt -u -w80 file.txt # format a text file output: 1 space between words, 2 spaces between sentences, 80 coloumns nl PATH/FILE # show line numbers for every line (empty lines are not counted by default) nl -ba PATH/FILE # show line numbers for every line and count all lines, including empty ones nl -s": " PATH/FILE # show line numbers for every line and separate line numbers from line text using the specified separator (": " in this case) pr -l 50 test.txt # show a text as pages, each page is 50 lines long (minimum 11, default is 66) pr -l 50 --column 2 test.txt # show a text as pages, each page is 50 lines long and divided into 2 columns more PATH/FILE # type a file page by page less PATH/FILE # show a file page by page, and let scroll through pages. It shows escaped characters as plain text (get it with apt-get install less) less -F PATH/FILE # show a file page by page, and let scroll through pages (get it with apt-get install less). If the file output fits a single screen it quits without having to press "q" grep "TEXT" PATH/FILE # show only lines containing "TEXT" grep -v "TEXT" PATH/FILE # show only lines NOT containing "TEXT" grep -i "TEXT" PATH/FILE # show only lines containing "TEXT" (case insensitive) grep -vi "TEXT" PATH/FILE # show only lines NOT containing "TEXT" (case insensitive) grep -m 1 "TEXT" PATH/FILE # show only the first matching line containing "TEXT" grep -v -m 1"TEXT" PATH/FILE # show only the first matching line NOT containing "TEXT" grep -i -m 1"TEXT" PATH/FILE # show only the first matching line containing "TEXT" (case insensitive) grep -vi -m 1"TEXT" PATH/FILE # show only the first matching line NOT containing "TEXT" (case insensitive) grep -B 1 -A 1 "TEXT" PATH/FILE # show lines containing "TEXT" and show 1 line Before and 1 line After each match grep -B 2 -A 2 -n "TEXT" PATH/FILE # show lines containing "TEXT" and show 2 lines Before and 2 lines After each match, together with line numbers head PATH/FILE # show the head (first 10 lines) of FILE head --lines=20 PATH/FILE # show the first 20 lines of FILE head --bytes=1024 PATH/FILE # show the first 1024 bytes of FILE head --lines=-50 PATH/FILE # show all the FILE but the last 20 lines head --bytes=-2048 PATH/FILE # show all the FILE but the first 2048 bytes tail PATH/FILE # show the tail (last 10 lines) of FILE tail --lines=20 -F PATH/FILE # show the last 20 lines of FILE, and new lines as the file grows. If the files doesn't exist yet, it waits for it (useful for log files) tail --lines=+20 -F PATH/FILE # show the last lines of FILE starting from line 20 tail --bytes=1024 PATH/FILE # show the last 1024 bytes of FILE tail -f PATH/FILE # keep showing the last line of FILE whenever there are added new ones (useful for log files) until stopped tail -f PATH/FILE | sed '/^ENDSTRING$/ q' # keep showing the last line of FILE whenever there are added new ones (useful for log files) until a given string appears sed -ne "3p" PATH/FILE # show line number 3 of FILE sed -ne "2,4p" PATH/FILE # show lines 2 to 4 of FILE sed -ne "1,5p" PATH/FILE # show lines 1 (beginning) to 5 of FILE sed -ne "5p,$" PATH/FILE # show lines 5 to the end of FILE sed -n -e '3,5p' -e '9,15p' PATH/FILE # show lines 3 to 5 and 9 to 15 of FILE sed -e "5d" PATH/FILE # show all lines of FILE except for the ones from 2 to 30 included, and except for line 5 sed -e "2,30d" PATH/FILE # show all lines of FILE except for the ones from 2 to 30 included sed -e "5d;2,30d" PATH/FILE # show all lines of FILE except for line 5, and except for the ones from 2 to 30 included awk 'NR==2 {print;exit}' SOURCE # Print line number 2 from SOURCE file awk 'NR==2,NR==4' SOURCE # Print line numbers 2 to 4 (inclusive) from SOURCE file awk '{print $1}' PATH/FILE # print the first word of every line in FILE awk '{print substr ($1,0,1)}' PATH/FILE # print the first character of every line in FILE awk 'NF>=1{print $NF}' PATH/FILE # print the last word of every line in FILE grep -o '.$' PATH/FILE # print the last character of every line in FILE cat PATH/FILE | awk 'length($0) < 5' # show lines longer less than 5 characters in FILE cat PATH/FILE | awk 'length($0) > 5' # show lines longer more than 5 characters in FILE cat PATH/FILE | awk 'length($0) == 5' # show lines longer with exactly 5 characters in FILE cat PATH/FILE | awk 'length($0) >= 5' # show lines with at least 5 characters in FILE cat PATH/FILE | awk 'length($0) >= 5 { print NR ":" $0 }' # show lines with at least 5 characters in FILE, preceded by line numbers of matching lines grep '.\{5\}' PATH/FILE # show lines with at least 5 characters in FILE Binary dump of a file: xxd -b PATH/FILE # binary dump of FILE xxd -b PATH/FILE | less # navigate the binary dump of FILE Hexadecimal dump of a file: xxd PATH/FILE # hex dump of FILE xxd PATH/FILE | less # navigate the hex dump of FILE xxd -u PATH/FILE # hex dump of FILE. Show hex numbers in Upper case xxd -a PATH/FILE # hex dump of FILE. Replace multiple NUL (00) lines with an asterisk ( * ) od -Ax -tx1z -v PATH/FILE # hex dump of FILE hexdump -C PATH/FILE # hex dump of FILE hexdump PATH/FILE # hex dump of FILE without the ASCII equivalent Extrapolate strings of text from binary files: strings -a -n 4 PATH/FILE # extrapolate strings of text from binary files (a string of text is considered at least 4 bytes long, you can change this value with the option -n) (get it with apt-get install binutils ) strings -a -t x PATH/FILE # extrapolate strings of text from binary files and return the offset of the string (-t x specifies the offset as hex, -t d decimal, -t o octal) Search for data in a CSV (Comma Separated Values) / XSV (Character Separated Values) / SSV (Separator Separated Values) file: awk -F',' '$1 == "TEXT"' /PATH/FILE.csv # If the first item (field) matches with the given TEXT in a comma (',') separated file, then return the whole line (row / record) awk -F',' '{ if ($1 == "TEXT") { print $2 } }' /PATH/FILE.csv # If the first item (field) matches with the given TEXT in a comma (',') separated file, then return the second record in the line (row / record) Count lines in text files: wc PATH/FILE # line/word/byte count (handy options: --lines or -l print only the newline counts, --words print only the word counts, --bytes print only the byte counts, --chars print the character counts) wc -L PATH/FILE # print the lenght of the longest line of FILE grep "STRING" PATH/FILE|wc --lines # count how many lines in FILE contains STRING wc -l PATH/*.txt # count the lines of all files with .txt extension in a directory find PATH -name "*.txt" | xargs wc -l # count the lines of all files with .txt extension in a directory and its subdirectories Sort a text file: sort PATH/FILE.txt # sort FILE.txt alphabetically, case sensitive sort -i PATH/FILE.txt # sort FILE.txt alphabetically, cosindering only printable characters sort -d PATH/FILE.txt # sort FILE.txt alphabetically, cosindering only dictionary characters (alphanumeric characters and blanks) sort -f PATH/FILE.txt # sort FILE.txt alphabetically, case insensitive sort -fs PATH/FILE.txt # sort FILE.txt alphabetically, case insensitive and stable (sort according which case comes first for each letter) sort -b PATH/FILE.txt # sort FILE.txt ignoring leading blank spaces sort -r PATH/FILE.txt # sort FILE.txt in reverse alphabetical order sort -n PATH/FILE.txt # sort numbers in FILE.txt sort -h PATH/FILE.txt # sort human readable numbers (like 5K, 2G) in FILE.txt sort -M PATH/FILE.txt # sort months (like Jan, Feb, Mar) in FILE.txt sort -R PATH/FILE.txt # sort lines randomly in FILE.txt sort -u PATH/FILE.txt # sort lines in FILE.txt and keep only unique lines sort -k 2 PATH/FILE.txt # sort lines in FILE.txt using characters from column 2 (any blank character separates columns) sort -t : -k 2 PATH/FILE.txt # sort lines in FILE.txt using characters from column 2 using ":" as a separator for columns sort PATH/FILE.txt -o "PATH/DEST_FILE.txt" # sort FILE.txt and save the output in DEST_FILE.txt Remove lines from a text file: sed -i '1d' PATH/TEXT_FILE # Remove the first line from a file cp PATH/TEXT_FILE PATH/TEXT_FILE.tmp ; sed '1d' PATH/TEXT_FILE.tmp > PATH/TEXT_FILE ; rm -f PATH/TEXT_FILE.tmp # sed older than v3.95 doesn't have the option -i and requires a temporary file sed -i '$ d' PATH/TEXT_FILE # Remove the last line from a file cp PATH/TEXT_FILE PATH/TEXT_FILE.tmp ; sed '$ d' PATH/TEXT_FILE.tmp > PATH/TEXT_FILE ; rm -f PATH/TEXT_FILE.tmp # sed older than v3.95 doesn't have the option -i and requires a temporary file More file operations: # All these commands should work also with "gawk" (GNU awk) and "nawk" (new awk by AT&T) awk '1; { print "" }' SOURCE>DESTINATION # Add an empty line after each line awk 'NF { print $0 "\n" }' SOURCE>DESTINATION # Add an empty line after each line, except when an empty line is already present awk '{ print FNR ".\t" $0 }' SOURCE>DESTINATION # Add line numbers followed by dot and tab before than each line awk '{sub(/^[ \t]+/, "")};1' SOURCE>DESTINATION # Remove leading spaces from each line awk '{sub(/[ \t]+$/, "")};1' SOURCE>DESTINATION # Remove trailing spaces from each line sed "s/[ \t]*$//" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Remove trailing spaces and tabs from a text file awk '{gsub(/^[ \t]+|[ \t]+$/,"")};1' SOURCE>DESTINATION # Remove leading and trailing spaces from each line awk '{sub(/A/,"B")}; 1' SOURCE>DESTINATION # Replace the first occurrence of A with B in each line awk '{gsub(/A/,"B")}; 1' SOURCE>DESTINATION # Replace every occurrence of A with B in each line awk '/X/{gsub(/A/, "B")}; 1' SOURCE>DESTINATION # Replace every occurrence of A with B in each line that contains X awk '!/X/{gsub(/A/, "B")}; 1' SOURCE>DESTINATION # Replace every occurrence of A with B in each line that does not contain X awk 'BEGIN{while (a++<100) s=s "."; print s}' >DESTINATION # Create a file containing 100 dots (.) awk 'END{print NR}' SOURCE # count lines (emulates "wc -l") # Strip trailing line break (new line) from output echo "Test" | tr -d '\n' # In this example, this would be the same as echo -n "Test" Return the average lenght in lines for all text files in the tree: find PATH -name "*.txt" -exec wc -l {} \; | awk 'BEGIN {x=0;y=0} {x+=1; y+=$1} END {print y/x}' Count the occurrences of all the words in a text file: tr -c a-zA-Z '\n' < PATH/FILE.txt | sed '/^$/d' | sort -n | uniq -ci | sort -nr # case insensitive tr -c a-zA-Z '\n' < PATH/FILE.txt | sed '/^$/d' | sort -n | uniq -c | sort -nr # case sensitive Count the occurrences of a string within another string: needle=","; haystack="one, two, three, four, five"; occurrences=$(grep -o "$needle" <<< "$haystack" | wc -l); echo $occurrences Insert characters into a string: echo '74657374' | sed 's/.\{2\}/&\./g' # Insert a dot every 2 characters and (it may leave a final dot) echo '74657374' | sed 's/.\{2\}/&\./g' | awk '{sub(/\.$/, "")};1' # Insert a dot every 2 characters and remove the last dot if present (warning: it may remove a pre existing final dot) echo '74657374' | sed 's/.\{4\}/& /g' # Insert a space every 4 characters and (it may leave a final space) echo '74657374' | sed 's/.\{4\}/& /g' | awk '{sub(/ $/, "")};1' # Insert a space every 4 characters and remove the last space if present (warning: it may remove a pre existing final space) Truncate a string: echo $(cut -c 3 <<< "this_is_an_example") # Get the third character of a string. Example: before: "this_is_an_example", after: "h". echo "this_is_an_example" | cut -c 3 # Get the third character of a string. Example: before: "this_is_an_example", after: "h". echo "this_is_an_example" | cut -c 1 # Get the first character from a string. Example: before: "this_is_an_example", after: "t". echo "this_is_an_example" | cut -c -3 # Get the first 3 characters from a string. Example: before: "this_is_an_example", after: "thi". echo "this_is_an_example" | cut -c 4-9 # Get the characters ranging in position 4 up to 9 from a string. Example: before: "this_is_an_example", after: "s_is_a". echo "this_is_an_example" | cut -c 3- # Cut the first 2 characters from a string (returns the string starting from the 3rd character). Example: before: "this_is_an_example", after: "is_is_an_example". echo "this_is_an_example" | rev | cut -c 3 # Get the third character from the end of a string. Example: before: "this_is_an_example", after: "p". echo "this_is_an_example" | rev | cut -c 1 # Get the last character of a string. Example: before: "this_is_an_example", after: "e". echo "this_is_an_example" | rev | cut -c 1-3 | rev # Get the last 3 characters from the end of a string. Example: before: "this_is_an_example", after: "ple". echo "this_is_an_example" | rev | cut -c 4-9 | rev # Get the characters ranging in position 4 up to 9 from the end of a string. Example: before: "this_is_an_example", after: "n_exam". echo "this_is_an_example" | rev | cut -c 3- | rev # Cut the last 2 characters from a string (returns the string up to the 3rd character from the end). Example: before: "this_is_an_example", after: "this_is_an_examp". Truncate a string using a marker: # The marker (delimiter) to set the point where to truncate a string is ":" in the following examples cut -d ":" -f1 <<< "this:is:an:example" # Truncate a string at the first occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "this". cut -d ":" -f2- <<< "this:is:an:example" # Truncate a string after the first occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "is:an:example". echo "this:is:an:example" | cut -d ":" -f1 # Truncate a string at the first occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "this". echo "this:is:an:example" | cut -d ":" -f2- # Truncate a string after the first occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "is:an:example". echo "this:is:an:example" | rev | cut -d ":" -f2- | rev # Truncate a string up to the last occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "this:is:an". echo "this:is:an:example" | rev | cut -d ":" -f1 | rev # Truncate a string after the last occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "example". Return a text string: echo "STRING" # print STRING echo -n "STRING" # print STRING without new line echo {A,B,C}{A,B,C} # return all possible combinations of given characters (this example returns AA AB AC BA BB BC CA CB CC ) Text attributes: tput bold; echo "Bold Text"; tput sgr0; tput smul; echo "Underlined Text"; tput rmul; tput bold; tput smul; echo "Bold and Underlined Text"; tput rmul; tput sgr0; tput dim; echo "Dimmed (Half Bright) Text"; tput sgr0; tput blink; echo "Blinking Text"; tput sgr0; # Note: most terminals don't support blinking text and will show a reverse color text instead. tput rev; echo "Reverse Color Text"; tput sgr0; tput smso; echo "Standout Text"; tput rmso; # The visual output of a standout text depends on the terminal: generally is rendered as a reverse color text or a bold text. tput sgr0 # Turn off all attributes. txtbold=$(tput bold); txtnorm=$(tput sgr0); echo "Normal Text, ${txtbold}Bold Text${txtnorm}, Normal Text again." Colored text: # Note: Colors are not a POSIX (Portable Operating System Interface) feature in the Bash shell, so it might not be shown in some systems tput colors # Show the number of available colors in the terminal (generally 8) Typical ANSI colors in 8 color terminals: 0: Black 1: Red 2: Green 3: Yellow 4: Blue 5: Magenta 6: Cyan 7: White 8: Not used 9: Reset to default color tput setaf COLOR # Set foreground color (ANSI). tput setab COLOR # Set background color (ANSI). tput setab 1; tput setaf 3; echo -n "TEXT"; tput setaf 9; tput setab 9; echo; # Type TEXT in yellow over red # Escaped text colors echo -e "\e[1;37m WHITE \e[0;37m LIGHT GRAY \e[1;30m GRAY \e[0;30m BLACK \e[0;31m RED \e[1;31m LIGHT RED \e[0;32m GREEN \e[1;32m LIGHT GREEN \e[0;33m YELLOW \e[1;33m LIGHT YELLOW \e[0;34m BLUE \e[1;34m LIGHT BLUE \e[0;35m PURPLE \e[1;35m PINK \e[0;36m CYAN \e[1;36m LIGHT CYAN \e[0m DEFAULT"; Remove Escaped colors and other control characters from a text file: cat -v PATH/FILE | sed "s/\^\[\[.\;..m//g" | sed "s/\^\[\[.m//g" | sed "s/\^M$//g" Type a text string (y by default) until interrupted: yes # Type y until interrupted yes n # Type n until interrupted yes hello world | head --lines=5 # Type hello world 5 times yes | COMMAND # automatically reply with "y" to all questions prompted by the COMMAND yes n | COMMAND # automatically reply with "n" to all questions prompted by the COMMAND Convert character encoding: apt-get install recode recode iso-8859-15..utf8 < FILE.TXT > file.utf8 # recode FILE.TXT from Latin Western Europe to UTF8 recode utf8..iso-8859-15 < FILE.UTF8 > file.txt # recode FILE.UTF8 from UTF8 to Latin Western Europe recode ../b64 < FILE.TXT > file.b64 # recode FILE.TXT to Base 64 recode /qp.. < FILE.TXT > file.qp # recode FILE.TXT to quoted printable Convert New Line Format: sed -i "s/\r//" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # From \r\n (Windows) to \n (Unix/Linux/Mac) sed -i "s/\n/\r\n/" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # From \n (Unix/Linux/Mac) to \r\n (Windows) Convert tabs to spaces in a text file: expand PATH/FILE # convert only tabs at the beginning of the line expand -a PATH/FILE # convert all tabs, not just the ones at the beginning of the lines Convert spaces to tabs in a text file: unexpand PATH/FILE # convert all tabs, not just the ones at the beginning of the lines unexpand -i PATH/FILE # convert only spaces at the beginning of the lines Search and Replace: sed "s/SOURCE_STRING/REPLACEMENT/g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace all occurrences of a string inside a text file, case sensitive sed "s/SOURCE_STRING/REPLACEMENT/gi" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace all occurrences of a string inside a text file, case insensitive sed "s/[Bb]ig/small/g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace the word Big or big to small sed "10,20 s/SOURCE_STRING/REPLACEMENT/gi" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace all occurrences of a string inside a text file, case insensitive, and process only lines 10 to 20 included sed "/MATCH_STRING/ s/SOURCE_STRING/REPLACEMENT/g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace SOURCE_STRING with REPLACEMENT only in lines where MATCH_STRING is present sed 's/TAG1.*TAG2/TAG1 REPLACEMENT TAG2/g' PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace anything between TAG1 and TAG2 keeping the tags, case sensitive sed 's/TAG1.*TAG2/TAG1 REPLACEMENT TAG2/gi' PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace anything between TAG1 and TAG2 keeping the tags, case insensitive sed 's/TAG1.*TAG2/REPLACEMENT/g' PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace anything between TAG1 and TAG2 removing the tags, case sensitive sed 's/TAG1.*TAG2/REPLACEMENT/gi' PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace anything between TAG1 and TAG2 removing the tags, case insensitive find . \( -type d -name .git -prune \) -o -type f -name "*.txt" -print0 | xargs -0 sed -e 's/SOURCE_STRING/REPLACEMENT/g' # Replace all occurrences of SOURCE_STRING with REPLACEMENT (case sensitive) in all files which names ends with .txt inside the current directory and all its subdirectories except for the subdirectories called .git . This is a dry run ( -e flag for sed will print the result on standard output, without actually writing to the file ) find . \( -type d -name .git -prune \) -o -type f -name "*.txt" -print0 | xargs -0 sed -i 's/SOURCE_STRING/REPLACEMENT/g' # Replace all occurrences of SOURCE_STRING with REPLACEMENT (case sensitive) in all files which names ends with .txt inside the current directory and all its subdirectories except for the subdirectories called .git . This will actually replace the text in the files Replace multiple spaces with a single space in a text file: sed "s/ */ /g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE Replace multiple spaces with a single space, and add a space before of every character in a text file: sed "s/ */ /g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE Remove comments from a text file: sed "/ *#/d;" PATH/SOURCE_FILE>PATH/DESTINATION_FILE Remove blank lines from a text file: sed "/^ *$/d" PATH/SOURCE_FILE>PATH/DESTINATION_FILE Remove comments and blank lines from a text file: sed "/ *#/d; /^ *$/d" PATH/SOURCE_FILE>PATH/DESTINATION_FILE Add a text at the end of each line: sed "s/$/\;/g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Append a semicolon (";") at the end of each line Add a text at the beginning of each line: sed "s/^/LINE : /g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Add "LINE : " at the beginning of each line Remove blank lines (empty lines) in a text file: sed '/^$/d' PATH/SOURCE_FILE>PATH/DESTINATION_FILE sed '/^[[:space:]]*$/d' PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Remove all blank lines, including the ones that are not actually empty but contains spaces and/or tabs Add blank lines after every line in a text file: sed "G" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Add 1 blank line after every line in a text file sed "G;G" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Add 2 blank lines after every line in a text file Append a line of text to a file: echo "line 1" >> PATH/FILE.txt Append multiple lines of text to a file (in scripts): echo "line 1 line 2" >> PATH/FILE.txt Append multiple lines of text to a file (in scripts): cat <> PATH/FILE.txt line 1 line 2 EOT Case conversion: echo "Test"|tr '[:lower:]' '[:upper:]' # Return the text in UPPER case echo "Test"|tr '[:upper:]' '[:lower:]' # Return the text in lower case cat PATH/SOURCE_FILE|tr '[:lower:]' '[:upper:]'>PATH/DESTINATION_FILE # Convert a text file to Upper case cat PATH/SOURCE_FILE|tr '[:upper:]' '[:lower:]'>PATH/DESTINATION_FILE # Convert a text file to lower case Extract items from a delimited text file: cut -d: -f1 PATH/SOURCE_FILE>PATH/DESTINATION_FILE # choose the first item of a series delimited by ":" Remove consecutive duplicate lines from a text file: uniq PATH/FILE uniq -u PATH/FILE # remove only successive duplicated lines (all occurrences) Reverse lines of a file rev PATH/FILE.txt > PATH/REVFILE.txt # Reverse the lines of a given file(s) Get the checksum of a file: sum PATH/FILE # return the checksum and block count for FILE sha1sum PATH/FILE # return the SHA1 (160-bit) checksum for FILE cksum PATH/FILE # return the CRC checksum and byte count for FILE Compare and show differences between two binary files: cmp -b -l PATH/FILE1 PATH/FILE2 # compare two files and returns the position of the different bytes followed by the value and the corresponding ASCII character for each file cmp -b -l --bytes=10 PATH/FILE1 PATH/FILE2 # limit comparison to the first 10 bytes Compare and show differences between two text files: diff -abB PATH/FILE1 PATH/FILE2 # Report differences between two text files diff -abBi PATH/FILE1 PATH/FILE2 # Report differences between two text files ignoring case differences in their content Compare and show differences between three text files: diff3 -e FILE1 FILE2 FILE3 # output unmerged changes from FILE2 to FILE3 into FILE1 Truncate a file: cp /dev/null PATH/FILE # empty FILE. If FILE doesn't exist, it will be created. truncate -s 0 PATH/FILE # truncate FILE to 0 bytes (empty FILE). If FILE doesn't exist, it will be created. truncate -s 0 -c PATH/FILE # truncate FILE to 0 bytes (empty FILE). If FILE doesn't exist, it will not be created. truncate -s 100 PATH/FILE # truncate FILE to 100 bytes. If FILE doesn't exist, or it's shorter than the given size, it will be filled by zeroes (0). truncate -r PATH/REF_FILE PATH/FILE # truncate FILE to the size of REF_FILE. Split or limit lines in a text file: cut -c 80 PATH/FILE.txt # cut every line of FILE.txt to 80 characters (extra characters will be lost) fold -w 80 PATH/FILE.txt # split every line of FILE.txt after 80 columns (extra characters will be wrapped to the next line) fold -w 80 -s PATH/FILE.txt # split every line of FILE.txt after 80 columns, breaking at spaces (full words will be preserved) Delete lines from a text file: sed -e '10d' PATH/FILE.txt > PATH/DESTFILE.txt # remove line 10 sed -e '5,15d' PATH/FILE.txt > PATH/DESTFILE.txt # remove lines 5 to 15 Empty files: # Delete the content of a file without removing the file. It can be useful to don't break programs which expect to find the file, or before to delete a very large file. cp /dev/null > PATH/NAME # Empty a file. It will also work as "> PATH/NAME" (without quotes) echo -n '' > PATH/NAME # Empty a file find PATH -type f -exec cp /dev/null {} \; # Empty all files in PATH and its subdirectories find PATH -type f -regex ".*\.EXT$" -exec cp /dev/null {} \; # Empty all files in PATH and its subdirectories with given EXTension find PATH -type f -regex ".*\.[0-9]*$" -exec cp /dev/null {} \; # Empty all files in PATH and its subdirectories with a numeric extension Delete files: rm PATH/NAME rm -i PATH/NAME # ask for confirmation before to delete find PATH -maxdepth 1 -type f -empty -delete # Delete all empty files in the given PATH (also: find PATH -maxdepth 1 -type f -size 0 -delete ) find PATH -maxdepth 1 -type f ! -empty -delete # Delete all non empty files in the given PATH (also: find PATH -maxdepth 1 -type f -size +0 -delete ) find PATH -type f -empty -delete # Delete all empty files in the given PATH and its subdirectories (also: find PATH -type f -size 0 -delete ) find PATH -type f ! -empty -delete # Delete all non empty files in the given PATH and its subdirectories (also: find PATH -type f -size +0 -delete ) find /var/www/test/t5 -type f -delete # Delete all files in /var/www/test/t5 and its subdirectories find /var/www/test/t5 -type f -regex ".*\.EXT$" -delete # Delete all files in /var/www/test/t5 and its subdirectories with given EXTension find /var/www/test/t5 -type f -regex ".*\.[0-9]*$" -delete # Delete all files in /var/www/test/t5 and its subdirectories with a numeric extension Wipe a file (secure file deletion): # Important note: wiping overwrites a file one or more time before to delete it, to make it unrecoverable. This technique does not work with journaled file systems, like ReiserFS, Reiser4, or ext3 in journaled mode. It also doesn't work with solid state drives (such as SD cards). apt-get install wipe # if it doesn't work, try apt-get install secure-delete wipe PATH/FILE # wipe a file (asks for confirmation, write protected files will be spared) wipe -c -f PATH/FILE # wipe a file wipe -c -f -s PATH/FILE # wipe a file, quiet mode (no output returned except for fatal errors like "No such file or directory") wipe -c -f -k PATH/FILE # wipes the content of a file, but keeps the file in place (with a lenght of 0 bytes) wipe -c -f -e PATH/FILE # wipes the file up to its exact size, without wiping extra data in the last used block Undelete a file: apt-get install testdisk # PhotoRec photorec MEDIA # attempt to recover deleted files from given MEDIA (for example /dev/sda2 ). You'll have then to confirm the media, choose partition, select recovery options, specify the file type of the files you want to recover, specify the filesystem type where the files were stored, choose if you want to search for deleted files in the empty space or the whole partition, choose the destination directory (in a separate disk) where you want to save the recovered files. Create a directory: mkdir PATH mkdir -p PATH # Create the whole path (not just the final directory in the path) if it doesn't exist Delete a directory: rmdir PATH # Delete an empty directory with no subdirectories rmdir -r PATH # Delete an empty directory with empty subdirectories (recursive) find PATH -depth -type d -empty -exec rmdir {} \; # Delete an empty directory with empty subdirectories (recursive) Prune a directory tree (remove a directory with all its subdirectories, even if not empty, recursive): rm -rf PATH ; rm -rf PATH/.* # Prune a directory tree mkdir /tmp/empty/ ; rsync -a --delete /tmp/empty/ PATH/DIRECTORY_TO_REMOVE/ ; rmdir PATH/DIRECTORY_TO_REMOVE # Delete a directory or prune a directory tree with a large number (thousands and more) of files. Note: you can use any empty directory, just make sure it's actually empty. Wipe a directory tree (recursive secure file deletion): apt-get install wipe wipe -c -r PATH # wipe a directory tree (asks for confirmation) wipe -c -f -r PATH # wipe a directory tree wipe -c -f -r -s PATH # wipe a directory tree, quiet mode (no output returned except for fatal errors like "No such file or directory") Show current directory: pwd Change directory: cd PATH cd .. # go to the parent directory (that is, one directory up) cd - # go to the previous directory (which is stored in $OLDPWD) echo $OLDPWD # see previous directory pushd PATH # change directory and remember the current directory in the dirs stack popd # return to the previous directory in the dirs stack dirs # (or dirs -l ) list directories in the dirs stack keeping the output in a single line dirs -p # list directories in the dirs stack showing the output in multiple lines dirs -v # list directories in the dirs stack showing the output in multiple lines and line numbers dirs -c # clear the dirs stack Show the content of a directory: ls PATH ls -l -d PATH # show only the properties of a directory, not its content. ls -laF --color=auto PATH # extended information and colored output ls -laFXB PATH # sort by extension ls -laFSr PATH # sort by size, small to large ls -laFS PATH # sort by size, large to small ls -laFtr PATH # sort by size, old to recent ls -laFt PATH # sort by date, recent to old ls -laFtur PATH # sort by access date, old to recent ls -laFtu PATH # sort by access date, recent to old ls -A # show hidden files ls --group-directories-first # show directories first ls -d */. # list only directories within current directory ls -d PATH/*/. # list only directories within given PATH ls -R PATH | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/' # show directory tree (add | less at the end of the line to navigate the tree up and down) find PATH -ls # The -ls parameters make find return results as the output of ls . You may also add the -print0 parameter and pipe to xargs -0 ls -lt (that is: -print0 | xargs -0 ls -lt ) find PATH -type f -mtime -30 # list files modified within last 30 days find PATH -depth -type d -empty -exec echo {} \; # list empty subdirectories within given PATH (recursive) find PATH -maxdepth 1 -type f -empty # Find all empty files in the given PATH (also: find PATH -maxdepth 1 -type f -size 0 ) find PATH -maxdepth 1 -type f ! -empty # Find all non empty files in the given PATH (also: find PATH -maxdepth 1 -type f -size +0 ) find PATH -type f -empty # Find all empty files in the given PATH and its subdirectories (also: find PATH -type f -size 0 ) find PATH -type f ! -empty # Find all non empty files in the given PATH and its subdirectories (also: find PATH -type f -size +0 ) find PATH/DIRECTORY -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort -u # return all different file extensions in given DIRECTORY Compare and show differences between two directories: diff PATH/DIR1 PATH/DIR2 # Report different files between two directories diff -q PATH/DIR1 PATH/DIR2 # Report different files between two directories without showing the different content of files with identical file names diff -s PATH/DIR1 PATH/DIR2 # Report different files and identical files between two directories diff --ignore-file-name-case PATH/DIR1 PATH/DIR2 # Report different files between two directories ignoring case differences in file names diff -i PATH/DIR1 PATH/DIR2 # Report different files between two directories ignoring case differences in the content of text files diff -r PATH/DIR1 PATH/DIR2 # Report different files between two directories and all their subdirectories dir1="PATH/DIR1"; dir2="PATH/DIR2"; diff -q "$dir1" "$dir2" | grep "$dir2" | sed "s/$dir2\///g" | awk '{print $4}' # Report different files and files that are only present in DIR2 Pick a random file from a directory: ls DIR | shuf -n 1 # Pick a random file from the given DIR directory ls DIR -pR | grep -v / | sed '/^$/d' | shuf -n 1 # Pick a random file from the given DIR directory and its subdirectories find DIR -maxdepth 1 -type f | shuf -n 1 # Pick a random file (including path) from the given DIR directory find DIR -type f | shuf -n 1 # Pick a random file (including path) from the given DIR directory and its subdirectories Change access permissions: # Users: uuugggooo (ugo = u: owner user, g: all the users of the group, o: all other users) # Modes: 0: ---, 1: --x, 2: -w-, 3: -wx, 4: r--, 5: r-x, 6: rw- , 7: rwx chmod MOD FILE # Change permissions for a file chmod -v MOD FILE # Change permissions for a file, with verbose output chmod -R MOD DIRECTORY # Change permissions for a directory tree (recursive) chmod -vR MOD DIRECTORY # Change permissions for a directory tree (recursive), with verbose output find PATH -type f -exec chmod MOD {} + # Change permissions for all files (only files, not directories) in a directory tree (recursive) find PATH -type d -exec chmod MOD {} + # Change permissions for all directories (only directories, not files) in a directory tree (recursive) Change owner: chown OWNER FILE # Change the owner of a file chown OWNER:GROUP FILE # Change the owner and the group of a file chown -R OWNER DIRECTORY # Change the owner of a directory tree (recursive) chown -R OWNER:GROUP DIRECTORY # Change the owner and a group of a directory tree (recursive) Change group: chgrp OWNER FILE # Change the group of a file chgrp -R OWNER DIRECTORY # Change the group of a directory tree (recursive) File Permissions Backup and Restore getfacl -R / > PATH/fsperms.txt # Back up file system permissions to a text file setfacl --restore=PATH/fsperms.txt # Restore file system permissions from a text file Raw copy: dd if=/dev/fd0 of=/dev/fd1 # copy a floppy disk (using two floppy disk drives) dd if=/dev/hda0 of=/dev/hda1 # copy a hard disk (or partition) into another one dd if=/dev/hda | gzip -9v | dd of=/mnt/hdb/hda.img # back up the whole hard disk as a gzipped file (you may need to mount a partition of another hard disk as hdb: mount /dev/hdb2 /mnt/hdb ) dd of=/mnt/hdb/hda.img | gzip | if=/dev/hda # restore the gzipped image of a hard disk dd if=/dev/fd0 of=/tmp/disk.bak # copy the whole floppy disk as a file dd if=/dev/hda of=/tmp/mbr.bin count=1 bs=512 # back up MBR (boot loader and partition table) Create and edit disk partitions: apt-get install parted parted # starts the partition utility parted commands: print # show current partitions mkpart primary START END # create a new primary partition mkpart extended START END # create a new extended partition mkpart logical START END # create a new logical partition (within the range of an extended one) resize PARTITON_N NEW_START NEW_END # resize an existing partition rm PARTITION_N # remove a partition quit # quit parted Show available devices: fdisk -l fdisk -l --bytes # show size in bytes rather than human readable format lsblk # list information about all available devices lsblk -f # lsblk -a # list information about all available devices including empty devices and RAM disk devices lsblk -b # list information about all available devices, show size in bytes rather than human readable format blkid # show information about available block devices Show device partitions: fdisk -l /dev/DEVICE Make a file system (format) a disk (partition): mkfs.ext3 /dev/hda1 # Format hda1 as an ext3 partition mkfs -V -t ext3 /dev/hda1 # Format hda1 as an ext3 partition (Verbose output) mkfs -t msdos /dev/hda2 # Format hda1 as a MS-DOS partition mkfs -t msdos /dev/fd0 # Format a floppy disk using the MS-DOS file system mkswap /dev/hda2 # Format hda2 as the swap partition dd if=/dev/zero of=/dev/foo1 bs=512 count=1 # If you created or changed a DOS partition, you can use dd to zero the first 512 bytes SMART Devices monitoring: apt install smartmontools smartctl -V # Show smartctl version and configure arguments service smartd start ; chkconfig smartd on # Enable Service smartctl -s on /dev/sda # Enable SMART on /dev/sda (use lsblk and fdisk -l to see all devices in your system). Some devices such as Virtual Disks do not support SMART smartctl -i /dev/sda # Print SMART information about /dev/sda smartctl --all /dev/sda # Print SMART information and data about /dev/sda smartctl -t short /dev/sda # Short Test (it usually teakes minutes to complete) smartctl -t long /dev/sda # Long Test (it usually teakes hours to complete) smartctl -H /dev/sda # Check if the hard drive passed or failed last test Create a RAMDisk: mkdir -p /media/ramdisk mount -t tmpfs -o size=2048M tmpfs /media/ramdisk # 2GB RAMDisk, it can use swap partition if needed, and doesn't preallocate disk space. mount -t ramfs -o size=1024M ramfs /media/ramdisk # 1GB RAMDisk, it only uses physical RAM, not the swap partition, and preallocates disk space. Make sure you have enough spare RAM. umount /media/ramdisk # unmount RAMDisk Access a XFS partition: # You typically need this to read disks from NAS devices, via USB. mkdir /ramdisk/xfs1 # If you are using a Linux boot disk, like Knoppix, you'll probably need to mount the XFS partition in the RAMDisk fdisk -l # Use fdisk to locate the XFS partition mount -t xfs /dev/XFS_PARTITION /ramdisk/xfs1 # use the path to XFS partition you've found using fdisk instead of /dev/XFS_PARTITION cp -rp /ramdisk/xfs1/* /DESTINATION # copy all files from the XFS formatted partition to another device ( /DESTINATION ) Check and repair a file system: fsck fsck -f -y FILE_SYSTEM # Note: Generally the device containing the file system to check can be /dev/hda0 on a single hard drive system or /dev/md0 on a RAID system); -y automatically reply yes (y) to all questions. Note that if it asks to connect to lost+found more than once, you may have some damaged files. List RAID arrays: mdadm --examine --scan Check if a RAID array is up and working properly: mdadm --detail /dev/md0 # assuming your RAID volume is /dev/md0 Check if two devices are in a RAID array: mdadm --examine --scan /dev/sda1 mdadm --examine --scan /dev/sdb1 # Must return same MD array and UUID. Path to actual devices in your system may differ from /dev/sda1 and /dev/sdb1 of this example Copy partitions from a device to another one: sfdisk -d /dev/sda | sfdisk /dev/sdb # copy partitions from sda to sdb Choose what partitions should be automatically mounted at system startup: jed /etc/fstab # edit static file system information Show free disk space: df -T # Show disk usage for all filesystems df -Th # Show disk usage for all filesystems in human readable format(in Megabytes, Gigabytes...) df -Th / # Show disk usage for the filesystem mounted on root in human readable format (Megabytes, Gigabytes...) df -h / | xargs | awk '{print "Disk ("$8"): Size: " $9 " Used: " $11 " Free: " $10}'; # Show disk usage for the filesystem mounted on root in a easily human readable format df -P -h | awk '0+$5 >= 90 {print "FS: "$1" ("$6") Size: "$2" Used: "$3" (\033[1;31m"$5"\033[0m) Free: "$4" (\033[1;31m"(100-$5)"%\033[0m)";}' # Only show near full file systems Summarize disk usage: # du shows size in KBytes by default (switch: -k), if you want to see the size in bytes, use the switch -b du PATH/FILE # disk usage for specified file du -bs PATH/DIRECTORY | awk '{print $1}' | tr -d '\n' # Total disk usage in bytes of the DIRECTORY in the given PATH du -s PATH # disk usage for the whole directory (recursively), total size du -s --exclude=*.mp3 PATH # disk usage for the whole directory (recursively), total size, excluding all files ending with .mp3 du PATH # disk usage for the whole directory (recursively) du -a PATH # disk usage for the whole directory (recursively), showing disk usage for all files and directories, not just directories du -S PATH # disk usage for the whole directory (recursively, but size is separated for each directory) du -a PATH | sort -nr | less # show directories and files sorted by size for the given PATH and its subdirectories du -a PATH | sort -nr | head -n 25 # show the 25 largest directories and files within the given PATH . Use root ( / ) as PATH to see the largest directories and files in the whole disk find PATH -type f -exec file -b '{}' \; -printf '%s\n' | awk -F , 'NR%2 {i=$1} NR%2==0 {a[i]+=$1} END {for (i in a) printf("%12u %s\n",a[i],i)}' | sort -nr # show the size occupied by files within a given PATH sorted by file type (slow) du -cs PATH/FILE* # show size of files matching a pattern du -cs PATH/FILE* | tail --lines=1 | awk '{print $1}' # show total size of files matching a pattern du -cs /var/log/apache2/*.gz | grep "$(printf '\t')total" # show total size of apache 2 compressed log files du -csh PATH/FILE* # show size of files matching a pattern in human readable format du -csh PATH/FILE* | tail --lines=1 | awk '{print $1}' # show total size of files matching a pattern in human readable format du -csh /var/log/apache2/*.gz | grep "$(printf '\t')total" # show total size of apache 2 compressed log files in human readable format Show deleted files locked by a running process: # If you can't see more free space in your disk after deleting a large file it's probably locked by a running process so that it has been unlinked but not actually deleted yet. In this case you'll have to kill, stop, or restart the process to unlock the file and free its disk space lsof +L1 Count files and subdirectories within directories: ls -1 /PATH/ | wc -l # count files and subdirectories within a directory ls -1R /PATH/ | wc -l # count files and subdirectories within a directory including subdirectories (recursively) ls -1 /PATH/ | grep -i .*.EXT | wc -l # count files and subdirectories with a given EXTension within a directory ls -1R /PATH/ | grep -i .*.EXT | wc -l # count files and subdirectories with a given EXTension within a directory including subdirectories (recursively) Count files within directories: find PATH -maxdepth 1 -type f | wc -l # count files within a directory find PATH -type f | wc -l # count files within a directory including subdirectories (recursively) find PATH -maxdepth 1 -type f | grep -i .*.EXT | wc -l # count files with a given EXTension within a directory find PATH -type f | grep -i .*.EXT | wc -l # count files with a given EXTension within a directory including subdirectories (recursively) Show mounted devices: mount Mount a device: mount DEVICE # example: mount /dev/cdrom mount -t FILESYSTEM DEVICE MOUNTPOINT # Mount a DEVICE at MOUNTPOINT as a given FILESYSTEM; example: mount -t ext3 /dev/sda1 /test Mount an ISO image of a CD: mount -o loop cd.iso /mnt/DIR_NAME/ Unmount a device: umount DEVICE # example: umount /dev/cdrom Unmount and eject a device (like a CD-ROM): eject # eject default device eject -d # show default device eject -r # eject CD-ROM Force changed blocks cached in RAM to disk: sync ----- Mount a remote directory tree: apt-get install sshfs fuse-utils To let other users to handle the remote directory tree: # create a group called fuse containing users allowed to mount the remote directory tree chgrp fuse /usr/bin/fusermount chmod u+s /usr/bin/fusermount adduser USER fuse Create the local directory where the remote directory tree will be mounted: mkdir /mnt/sshdir chown USER /mnt/sshdir Mount a remote directory tree: sshfs USER@HOST:/PATH /mnt/sshdir ----- Configure boot devices and partitions: # see GNU GRUB (GRand Unified Bootloader) configuration: less /boot/grub/grub.cfg # /boot/grub/grub.cfg replaces /boot/grub/menu.lst and it's not meant to be edited # in distros other than Debian the path could be /boot/grub/grub.conf or /etc/grub.conf # edit LILO (LInux LOader) configuration: jed /etc/lilo.conf ----- Print a file on a printer: lpr -Pprinter PATH/FILE lpr -Pprinter -#N PATH/FILE # print N copies of FILE on printer List print queue: lpq Remove a job from the print queue: lprm JOB_ID lprm # remove all jobs from the print queue ----- Indentation: Indent C code (or similarly formatted code): apt-get install indent indent PATH/SOURCE -o PATH/DEST indent -nprs PATH/SOURCE -o PATH/DEST # Doesn't add a space after an open bracket and before a closed one sed -i 's/^[[:space:]]*//g' PATH/FILE # Remove indentation from any text file Indent HTML code: apt-get install hindent hindent PATH/SOURCE.HTML>PATH/DEST.HTML hindent -i 0 PATH/SOURCE.HTML>PATH/DEST.HTML # Remove indentation hindent -c PATH/SOURCE.HTML>PATH/DEST.HTML # Force all tags to lowercase (otherwise all tags are forced to UPPERCASE by default) hindent -c -i 2 PATH/SOURCE.HTML>PATH/DEST.HTML # Set indentation level to 2 and force all tags to lowercase hindent -c PATH/SOURCE.HTML>PATH/DEST.HTML # Force all tags to lowercase hindent -f PATH/SOURCE.HTML>PATH/DEST.HTML # Flow: dump only HTML tags discarding any data between hindent -l PATH/SOURCE.HTML>PATH/DEST.HTML # List all HTML tags contained in the source document ----- Scheduled jobs with CronTab: # Edit CronTab file crontab -e # Edit CronTab file using the default editor or the one set in export EDITOR=jed ; export VISUAL=jed ; # (or another editor) Format: # Line of comments start with the dash character (#) minute (0-59) hour (0-23) day of month (1-31) month (1-12) day of week (0-6, Sunday=0) COMMAND # Use asterisks to specify any occurrence of the time parameter * * * * * echo "test" # execute the command every minute (not recommended) 30 * * * 1 echo "test" # execute the command every 30 minutes, every monday # separate multiple istances of the same time parameter with a comma (,) 15,30,45 * * * 1,3,5 echo "test" # execute the command every 15 minutes, every monday, wednesday, friday # separate two istances with a dash (-) to create a time range 0 0,12 * 6-9 1-5 echo "test" # execute the command at midnight amd at twelve, every day from monday to friday, during every month from June to September # Percent character (%) must be escaped (\) 30 * * * 1 touch /PATH/file_$(date +\%Y\%m\%d).txt # Normally, crontab sends a mail to the user with the output of the command executed, including errors. Append ">/dev/null 2>&1" to the command line to prevent this behaviour * * * * * echo "test">/dev/null # list all scheduled jobs crontab -l # erase the crontab file (not recommended because remove all jobs, including ones that might be necessary for the system, you'd better edit the crontab files and delete entries manually) crontab -r ----- Networking: arp -a # show DNS, IP, and MAC address for all network interfaces ifconfig -a # show DNS, IP, and MAC address for all network interfaces ifconfig eth0 # show DNS, IP, and MAC address for all network interfaces # show DNS, IP, and MAC address for the given interface (eth0) Change MAC Address of an interface (eth0): ifconfig eth0 down ifconfig eth0 hw ether xx:xx:xx:xx:xx:xx ifconfig eth0 up ifconfig # show local network interfaces ifconfig INTERFACE IP_ADDRESS # change the IP address for a network interface ifconfig INTERFACE down # disable a network interface (WARNING: if you are connected from a remote system and disable the network interface to which you are connected you'll be disconnected, and if there isn't another interface available or a script that sets the interface up again automatically, you'll need physical access to the computer to configure it) ifconfig INTERFACE up # enable a network interface iwconfig # show local wireless network interfaces iwconfig INTERFACE IP_ADDRESS # change the IP address for a wireless network interface iwconfig INTERFACE down # disable a wireless network interface (WARNING: if you are connected from a remote system and disable the network interface to which you are connected you'll be disconnected, and if there isn't another interface available or a script that sets the interface up again automatically, you'll need physical access to the computer to configure it) iwconfig INTERFACE up # enable a wireless network interface Wake On LAN (WOL): apt-get install ethtool ethtool -s eth0 wol g # Set Wake On LAN on the specified interface (eth0). The system must support Wake On LAN and it has to be enabled from the BIOS ethtool -s eth0 wol d # Unset Wake On LAN on the specified interface (eth0) apt-get install etherwake etherwake xx:xx:xx:xx:xx:xx # Wake On LAN the remote system with the specified MAC address Default Gateway: route add default gw GATEWAY_IP # Set Default Gateway. GATEWAY_IP in most cases is 192.168.0.1 or 192.168.0.254 or 192.168.1.0 or 192.168.1.254 (should you make an educated guess) ip r # Show Default Gateway route -n | grep 'UG[ \t]' | awk '{print $2}' # Get the IP of the Default Gateway # Specify DNS Servers to resolve domain names jed /etc/resolv.conf # OpenDNS servers nameserver 208.67.222.222 nameserver 208.67.220.200 jed /etc/hosts # assign hosts to specific IPs jed /etc/network/options jed /etc/network/interfaces /etc/init.d/networking restart Show active Internet connections: netstat -anp netstat -an | awk '{print $5}' | grep -o "[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}" | egrep -v "(`for i in \`ip addr | grep inet |grep eth0 | cut -d/ -f1 | awk '{print $2}'\`;do echo -n "$i|"| sed 's/\./\\\./g;';done`127\.|0\.0\.0)" | sort -n | uniq -c | sort -rn # show open connections sorted by IP address netstat -an | grep 'ESTABLISHED' | awk '{print $5}' | grep -o "[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}" | egrep -v "(`for i in \`ip addr | grep inet |grep eth0 | cut -d/ -f1 | awk '{print $2}'\`;do echo -n "$i|"| sed 's/\./\\\./g;';done`127\.|0\.0\.0)" | sort -n | uniq -c | sort -rn # show established connections sorted by IP address netstat -anp | grep 'IP' # Show current connections for the given IP netstat -anp | grep 'ESTABLISHED' | grep 'IP' # Show current estabilished connections for the given IP netstat -plntu # show applications listening TCP/UDP sockets netstat -plntu | grep /apache | awk '{print $4}' | rev | cut -d ":" -f1 | rev | uniq # Show ports on which Apache webserver is listening, if enabled netstat -plntu | grep /apache | awk '{print $4}' | rev | cut -d ":" -f1 | rev | uniq | tr '\n' ',' | awk '{sub(/,$/, "")};1' # Show ports on which Apache webserver is listening, if enabled (comma separated) netstat -plntu | grep /tor | awk '{print $4}' | rev | cut -d ":" -f1 | rev | uniq # Show ports on which TOR server is listening, if enabled netstat -plntu | grep /mysqld | awk '{print $4}' | rev | cut -d ":" -f1 | rev | uniq # Show ports on which MySQL server is listening, if enabled netstat -plntu | grep /sshd | awk '{print $4}' | rev | cut -d ":" -f1 | rev | uniq # Show ports on which SSH is listening, if enabled netstat -plntu | grep /sendmail | awk '{print $4}' | rev | cut -d ":" -f1 | rev | uniq # Show ports on which sendmail is listening, if enabled netstat -A inet -lnp # show only listening servers netstat -ant | awk '{print $4 " " $6}' | grep ':22 ' | grep 'ESTABLISHED' | wc -l # count estabilished SSH connections netstat -ant | awk '{print $4 " " $6}' | grep ':80 ' | grep 'ESTABLISHED' | wc -l # count estabilished HTTP connections netstat -ant | awk '{print $4 " " $6}' | grep ':443 ' | grep 'ESTABLISHED' | wc -l # count estabilished HTTPS connections netstat -anp | grep -v ":443 " | grep -v ":80 " | grep 'ESTABLISHED' # show NOT HTTPS/HTTP estabilished connections (excluding ports 443, 80) Show network sockets in use: lsof -i tcp:80 # show all processes that are using port 80 TCP Scan an IP for open ports: echo "Open ports:"; for i in {1..65535}; do (echo < /dev/tcp/127.0.0.1/$i) &>/dev/null && echo $i; done # Scan localhost (127.0.0.1) for open ports Show bandwidth usage: apt-get install nload nload # show traffic in real time apt-get install iftop iftop # show connections and their bandwidth usage Network usage stats: apt install vnstat jed /etc/vnstat.conf # Configuration vnstat -u -i eth0 # Create the database file. If the interface is not eth0 change it accordingly chown vnstat:vnstat /var/lib/vnstat/eth0 # Set vnstat as the owner of the database file. If the interface is not eth0 type the file name accordingly service vnstat start # Start vnstat service vnstat stop # Stop vnstat vnstat -h # Hourly Stats vnstat -d # Daily Stats vnstat -w # Weekly Stats vnstat -m # Monthly Stats vnstat -t # Top Transfers Ping host: ping HOST # ping a host until CTRL-C is pressed ping -c N HOST # ping a host N times Trace route to host: traceroute HOST Network Mapping: apt-get install nmap nmap -sS 192.0.2.0 # search for open ports in the given IP nmap -sS 192.0.2.0/24 # search for connected IPs (and related open ports) in the given IP mask nmap -sS -O -v 192.0.2.0 # return Operating System and uptime for the given IP Port scanning: apt-get install netcat # scan TCP ports: nc -vvn -z 192.0.2.2 1-80 # scan TCP ports from 1 upto 80 in the given IP address # scan UDP ports: nc -u -vvn -z 192.0.2.8 1-255 # scan UDP ports from 1 upto 255 in the given IP address Calculate broadcast, network, Cisco wildcard mask, and host range apt-get install ipcalc ipcalc 192.0.2.0/24 ipcalc 192.0.2.0/255.255.255.0 ipcalc 192.0.2.0-192.0.2.100 # deaggregate address range DNS lookup: apt-get install host # If you want to use the host command # Resolve Hostname to IP address dig +short www.example.com | awk '{ print ; }' getent hosts www.example.com | awk '{ print $1 ; }' host www.example.com | awk '/has address/ { print $4 ; }' nslookup www.example.com | awk '/^Address: / { print $2 ; }' # Resolve Hostname to IP address (only get the first result if there are more then one IP) dig +short www.example.com | awk '{ print ; exit }' getent hosts www.example.com | awk '{ print $1 ; exit }' host www.example.com | awk '/has address/ { print $4 ; exit }' nslookup www.example.com | awk '/^Address: / { print $2 ; exit }' # Resolve Hostname to IPv4 or IPv6 address host www.example.com # return both IPv4 and IPv6 addresses host -t A www.example.com # return the IPv4 addresses host -t AAAA www.example.com # return the IPv6 addresses # Reverse DNS lookup host 192.0.2.0 nslookup 192.0.2.0 DNS queries: dig www.example.com A # get the IP address of the given domain dig www.example.com TXT # get text annotations about the given domain dig www.example.com MX # get the mail servers of the given domain dig www.example.com NS # get the authoritative DNS servers for the given domain dig www.example.com ANY # get all DNS information for the given domain whois example.com # Information about a domain name # IP Tables: # iptables work with IPv4 addresses. Use ip6tables to work with IPv6 addresses. iptables -S # Show rules (doesn't resolve IPs) iptables -L # List rules (resolve IPs) iptables -L -n # List rules (doesn't resolve IPs) iptables -L INPUT --line-numbers # List only INPUT rules together with their line numbers (resolve IPs) iptables -L INPUT -n --line-numbers # List only INPUT rules together with their line numbers (doesn't resolve IPs) iptables -I INPUT -s 192.0.2.100 -j DROP # Block a specific (single) IP iptables -I INPUT -s 192.0.2.100/32 -j DROP # Block a specific (single) IP (CIDR 32 only includes a single IP) iptables -I INPUT -s 192.0.2.100/255.255.255.255 -j DROP # Block a specific (single) IP (subnet mask 255.255.255.255 only includes a single IP) iptables -I INPUT -s 192.0.2.0/255.255.255.0 -j DROP # Block a range of IPs using a subnet mask iptables -I INPUT -s 192.0.2.0/24 -j DROP # Block a range of IPs using CIDR iptables -I INPUT -s 192.0.2.100 -j DROP -m time --timestart 22:00 --timestop 23:00 # Block the given IP only for one hour (22:00 to 23:00) every day iptables -I INPUT -s 192.0.2.100 -j REJECT --reject-with icmp-port-unreachable # Reject a specific (single) IP. The difference between DROP and REJECT is that REJECT returns an error packet, while DROP returns no reply. Generally, REJECT is preferred on local networks and DROP on the Internet. iptables -D INPUT LINE_NUMBER # Delete the INPUT rule with specified LINE_NUMBER iptables -D INPUT 1 # Delete the last INPUT rule iptables -D INPUT -s 192.0.2.100 -j DROP; # Unblock the matching IP Address iptables -D INPUT -s 192.0.2.0/255.255.255.0 -j DROP # Unblock the matching range of IPs using a subnet mask iptables -D INPUT -s 192.0.2.0/24 -j DROP # Unblock the matching range of IPs using CIDR iptables -S|grep " DROP"|awk '{sub(/-A /,"iptables -I ")}; 1' > iptablesdrop.sh ; chmod 777 iptablesdrop.sh ; # Back up Drop instances in IP Tables into a shell script iptables -S|grep " REJECT"|awk '{sub(/-A /,"iptables -I ")}; 1' > iptablesdrop.sh ; chmod 777 iptablesdrop.sh ; # Back up Reject instances in IP Tables into a shell script iptables -S|grep -E "( DROP| REJECT)"|awk '{sub(/-A /,"iptables -I ")}; 1' > iptablesdrop.sh ; chmod 777 iptablesdrop.sh # Back up Drop and Reject instances in IP Tables into a shell script iptables -S|grep -E "( DROP| REJECT)"|grep -Ev "^-A f2b-"|awk '{sub(/-A /,"iptables -I ")}; 1' > iptablesdrop.sh ; chmod 777 iptablesdrop.sh # Back up Drop and Reject instances in IP Tables, excluding the ones created by fail2ban, into a shell script iptables-save > /etc/iptables/rules.v4 # Back up all IP Tables for IPv4 iptables-restore < /etc/iptables/rules.v4 # Restore all IP Tables for IPv4 ip6tables-save > /etc/iptables/rules.v6 # Back up all IP Tables for IPv6 ip6tables-restore < /etc/iptables/rules.v6 # Restore all IP Tables for IPv6 Script to clear IPTables: --- #!/bin/bash # Clear iptables iptables -F iptables -X iptables -t nat -F iptables -t nat -X iptables -t mangle -F iptables -t mangle -X iptables -t raw -F iptables -t raw -X iptables -P INPUT ACCEPT iptables -P FORWARD ACCEPT iptables -P OUTPUT ACCEPT --- Download files from the Internet: # note: default ports: HTTP: 80, FTP: 21 wget "http://www.example.com/file.gz" # download the document at the given URL to the local directory wget "http://www.example.com/file.gz" -O PATH/downloaded_file.gz # download the document at the given URL to the specified directory and with the specified file name wget -S "http://www.example.com/file.gz" # download the document at the given URL in the local directory, and save the HTTP header at the beginning of the file wget -c --wait=2 --limit-rate=20k "http://www.example.com/file.gz" # continue an interrupted download, and pause 2 secods after getting every file and limit download speed to 20KBps wget -t 50 "http://www.example.com/file.gz" # retries up to 50 times if the download fails or it's incomplete wget -t 0 "http://www.example.com/file.gz" # retries forever if the download fails or it's incomplete, until it can get the requested file echo 'wget "http://www.example.com/file.gz"'|at 02:00 # start a download at 2:00am wget -O- "http://www.example.com/path/page.html" # dump the output of the page on the console (normally on the screen) wget -q -O- "http://www.example.com/path/page.html"|grep 'a href' # show all links within given document wget -r -l1 "http://www.example.com/path/page.html" # download the first layer of links from given document wget -r -l1 --no-parent "http://www.example.com/path/page.html" # download the first layer of links from given document, ignoring links that are not within the given URL path wget -r -l1 --no-parent -nc "http://www.example.com/path/page.html" # download the first layer of links from given document, ignoring links that are not within the given URL path, and avoid downloading files already present in the destination directory (useful when resuming an interrupted download) wget "http://www.example.com/doc?id="{1..50} // Download all files called doc?id= ranging from 1 to 50 from the given URL wget "http://www.example.com/doc?id="{01..50} // Download all files called doc?id= ranging from 01 to 50 from the given URL wget -r -l0 "http://www.example.com/path/page.html" # download a whole website to the local directory wget --mirror "http://www.example.com/" # download a whole website to the local directory wget -r -l1 --no-parent -A "*.gif" "http://www.example.com/dir/" # download all files matching a certain pattern (all GIF files in this case) from the specified URL wget -U USER_AGENT_STRING "http://www.example.com/file.gz" # you may set the User Agent to pretend to get the file from the server with a normal browser Dump the content of a webpage on the screen: lynx -dump http://www.example.com/ lynx --source http://www.example.com/ # Show document source lynx -head -dump http://www.elfqrin.com/ # Show document info (headers) wget -qO- http://www.example.com/ | cat # Show document source wget --server-response --spider -q http://www.example.com/ # Show document info (headers) curl http://www.example.com/ # Show document source curl -I http://www.example.com/ # Show document info (headers) Run a script on the web server: wget -O- "http://localhost/script.php" # run a script on the local server wget -q -O- "http://localhost/script.php" # run a script on the local server hiding the output wget -q -T 0 -O- "http://localhost/script.php" # By default wget drops connection after 900 seconds (15 minutes), in this case you'll get a "Read error (Connection timed out) in headers. Retrying." and if the script always takes more than the given time, it will retry uselessly until the maximum allowed retries (by default 20, but it can be changed with -t). To avoid time outs you can set a longer time using -T followed by the number of seconds (or 0 for infinite time, in this case make sure the script is working or it will run forever and if it uses resources incrementally it may crash the server). Also, if you experienced a time out with wget while running a script on localhost you should better restart the webserver as the script may continue running despite of wget timing out and dropping the connection. lynx -dump http://localhost/script.php >/dev/null # run a script on the local server (less reliable than wget for this purpose) hiding the output Run a script: php PATH/FILE.php # run a PHP script php -r "echo 'Hello World';"; # Execute a PHP line in the shell php -r 'echo "Hello World\n";'; # Execute a PHP line in the shell. Use quotes to print character sequences. perl -e "print 'Hello World';"; # Execute a Perl line in the shell python -c "print('Hello World')"; # Execute a Python line in the shell python3 -c "print('Hello World')"; # Execute a Python3 line in the shell ruby -e "puts 'Hello World'"; # Execute a Ruby line in the shell # If you receive this error message from the PPP Daemon (PPPD) # pppd: The remote system is required to authenticate itself but I couldn't find any secret (password) which would let it use an IP address. # You can fix it adding the string "noauth" (or changing the existing string "auth" to "noauth") in /etc/ppp/options ----- Mirror a disk through the network: If you have two computers with identical hardware you can mirror a system into the other machine through the network This is especially useful with clusters / virtual servers Start both machines from a live CD (like Debian Live - https://www.debian.org/CD/live/ ) Connect both machine with a cross cable through their eth1 interfaces and set them this way: root@source_pc# ifconfig eth1 192.168.0.1 root@destin_pc# ifconfig eth1 192.168.0.2 Then execute these commands on the machines to start transfers: root@source_pc# dd if=/dev/sda bs=128M | nc _w2 192.168.0.2 9000 root@destin_pc# nc _l _p 9000 | dd of=/dev/sda Depending from the size of the disk it may need many hours ----- Maths: Calculate the result of an expression: # operators: + - * / % < > = >= <= != echo $(( 10+5 )) # Sum echo $(( 10-5 )) # Subtraction echo $(( 10*5 )) # Multiplication echo $(( 10/5 )) # Division echo $(( 10**2 )) # Power echo $(( 15%10 )) # Module echo $(( (1+2)*3 )) # Result = 20 echo $(( 5>3 )) # Returns 1 (true) echo $(( 3>5 )) # Returns 0 (false) # Using expr, operators must be escaped and separated by a space expr 10 \+ 5 # Sum expr 10 \- 5 # Subtraction expr 10 \* 5 # Multiplication expr 10 \/ 5 # Division expr 15 \% 10 # Module expr \( 1 \+ 2 \) \* 3 # Result = 20 expr 5 \> 3 # Returns 1 (true) expr 3 \> 5 # Returns 0 (false) Return the absolute value of a number echo $(( 2-3 )) | tr -d - Show Multiplication Table: for i in {1..9}; do for j in $(seq 1 $i); do echo -ne $i\x$j=$((i*j))\\t; done; echo; done; Return the prime factors of a number: factor NUMBER factor NUMBER1 NUMBER2 NUMBER3... # return factors for the given numbers Return a sequence of numbers: echo {FIRST..LAST} # return a sequence of integer numbers from FIRST to LAST, separated by a space. Numbers can be negative, and LAST can be less than FIRST printf '%s\n' {1..10} # return a sequence of integer numbers from FIRST to LAST, each in its own line of text (separated by new lines). printf '%s,' {1..10} | rev | cut -c 2- | rev # return a sequence of integer numbers from FIRST to LAST, comma separated (rev and cut are used to remove the comma at the end of the string) printf '%s, ' {1..10} | rev | cut -c 3- | rev # return a sequence of integer numbers from FIRST to LAST, comma and space separated (rev and cut are used to remove the comma and space at the end of the string) for ((i=FIRST; i<=LAST; i+=INCREMENT)); do echo $i; done # create a sequence from FIRST up to LAST number with given INCREMENT seq LAST # create a sequence up to LAST number (default FIRST is 1, INCREMENT is 1) seq FIRST LAST # create a sequence from FIRST up to LAST number (default INCREMENT is 1) seq FIRST INCREMENT LAST # create a sequence from FIRST up to LAST number with given INCREMENT. Numbers can be negative, and END can be less than START, if the INCREMENT is negative seq -s "," FIRST INCREMENT LAST # create a sequence from FIRST up to LAST number with given INCREMENT. Numbers are separated by a comma (",") instead of a line break seq -s "," -w FIRST INCREMENT LAST # create a sequence from FIRST up to LAST number with given INCREMENT. Numbers are separated by a comma (",") instead of a line break. Numbers with less digit than the greater number are padded with 0s (or the sign for negative numbers) seq FIRST INCREMENT LAST | shuf # shuffle a sequence of numbers PI: apt-get install pi pi N # Print the mathematical constant PI to the given number of decimal figures. Note that pi 0 returns an empty string, pi 1 returns "3.", pi 2 returns "3.1", pi 3 returns "3.14", and so on. Default parameter for pi is 100 (that is, if you enter "pi" you get the same result as "pi 100"). Random numbers: echo $RANDOM # return a random number between 0 and 32767 (16-bit integer) echo $((RANDOM%(MAX+1-MIN)+MIN)) # return a random number between MIN and MAX expr \( $RANDOM \% \( MAX \+ 1 \- MIN \) \+ MIN \) # return a random number between MIN and MAX Base conversion: apt-get install bc echo 'ibase=10; obase=16; 155'|bc # base conversion (155 from decimal to hexadecimal) echo 'ibase=16; obase=2; F'|bc # base conversion (F from hexadecimal to binary) echo '2*(3+4)'|bc -l # perform math operations Measure conversion: apt-get install units units # interactive units "1 in" "cm" # convert 1 inch to centimeters Human Readable Values: numfmt --to=iec-i --suffix=B BYTES # Convert the number of BYTES into a human readable value numfmt --to=iec-i --suffix=B --padding=7 BYTES # Convert the number of BYTES into a human readable value, more instances will be padded numfmt --to=iec-i --suffix=B --format="%.3f" BYTES # Convert the number of BYTES into a human readable value with 3 decimal digits precision ----- Time: Set date and time: date MMDDhhmmYYYY.ss Set date from NTP (SNTP) server: apt-get install ntpdate ntpdate SERVER # set the clock ntpdate -q SERVER # query only for the time, don't set the clock SNTP servers: pool.ntp.org # NTP Pool Project ntp.ubuntu.com # Ubuntu ntp1.inrim.it # INRiM, Torino, Italy ntp.univ-lyon1.fr # University of Lyon, Lyon, France vega.cbk.poznan.pl # Astrogeodynamic Observatory, Borowiec, Poland bitsy.mit.edu # MIT, Cambridge, Massachusetts, USA bonehed.lcs.mit.edu # MIT, Cambridge, Massachusetts, USA clock.isc.org # Internet Systems Consortium, Palo Alto, California, USA Start a NTP server: apt-get install ntp-refclock (or another ntp* package) jed /etc/ntp.conf /etc/init.d/ntp-server restart Date tokens: %% : % %a : Weekday's abbreviated name (for example, Mon) %A : Weekday's full name (for example, Monday) %b : Month's abbreviated name (for example, Jan; same as %h) %B : Month's full name (for example, January) %C : Century (for example, 21) %d : Day of month (for example, 01) %e : day of month, space padded; same as %_d %F : Full date formatted as full year - month - day (same as %Y-%m-%d ) %g : last two digits of year of ISO week number (see %G) %G : year of ISO week number (see %V); normally useful only with %V %H : Hour in 24 hours format with padded 0 (00-23) %I : Hour in 12 hours format with padded 0 (01-12) %j : Day of year with padded 0 (001-366) %k : Hour in 24 hours format with padded space ( 0-23) %l : Hour in 12 hours format with padded space ( 1-12) %m : Month with padded 0 (01-12) %M : Minute with padded 0 (00-59) %n : Newline %N : Nanoseconds (000000000-999999999) %p : AM or PM, uppercase %P : AM or PM, lowercase %R : 24-hour hour and minute (same as %H:%M ) %s : Seconds since Linux Epoch (1970-01-01 00:00:00 UTC) %S : Second with padded 0 (00-60) %t : Tab %T : Time (same as %H:%M:%S ) %u : ISO Day of week (1-7: 1 is Monday, 7 is Sunday) %U : Week number of year, with Sunday as first day of week and padded 0 (00-53) %V : ISO Week number of the year, with Monday as first day of week and padded 0 (01-53) %w : Day of week (0-6: 0 is Sunday, 6 is Saturday) %W : ISO Week number of year, with Monday as first day of week and padded 0 (00-53) %y : Last two digits of year (00-99) %Y : Full Year %z : Timezone as hhmm (e.g., -0200) %:z : Timezone as hh:mm (e.g., -02:00) %Z : Timezone's abbreviated name (e.g., CET) Show current date and time: date date '+%a %d %b %Y %H:%M:%S %Z (UTC%:z)' # show day of the week, day of the month, month (short format), year (full digits), hour, minutes, seconds, timezone. Example: Wed 16 Sep 2020 21:18:15 CEST (UTC+02:00) echo "$(date '+%a %d %b %Y %H:%M:%S %Z (UTC%:z)')"; # echo day of the week, day of the month, month (short format), year (full digits), hour, minutes, seconds, timezone Show information about your local time: timedatectl status Show System Time Zone cat /etc/timezone Linux Epoch (1 January 1970 00:00:00) date +"%s" # Timestamp: seconds from Epoch to current time date -d "2000-01-01" +"%s" # Seconds from Epoch to the specified date (from 1 January 1970 to 1 January 2000 in this example) date -d "2000-01-01 12:00:00" +"%s" # Seconds from Epoch to the specified date and time (from 1 January 1970 00:00:00 to 1 January 2000 12:00:00 in this example) date -d @1474303375 # Converts seconds from Epoch to date Dates: date +"%F" -d now # Return current day in the specified format date +"%F" -d yesterday # Return yesterday's day in the specified format date +"%F" -d tomorrow # Return tomorrow's day in the specified format date +"%F" -d last-week # Return last week's date on current day in the specified format date +"%F" -d next-week # Return next week's date on current day in the specified format date +"%F" -d last-month # Return last month's date on current day in the specified format date +"%F" -d next-month # Return next month's date on current day in the specified format date +"%F" -d last-year # Return last year's date on current day in the specified format date +"%F" -d next-year # Return next year's date on current day in the specified format date +"%F" -d last-WEEKDAY # Return the date of the last given weekday (for example last-monday) in the specified format date +"%F" -d next-WEEKDAY # Return the date of the next given weekday (for example next-monday) in the specified format Calculate dates: date +"%F" -d "now +15 days" # Show the resulting date adding the specified time to the current date in the specified format (date only in this example) date +"%F" -d "2000-01-01 +15 days" # Show the resulting date adding the specified time to the given date in the specified format (date only in this example) date +"%F %H:%M:%S" -d "now +10 seconds" # Show the resulting date adding the specified time to the current date in the specified format (date and time in this example) date +"%F %H:%M:%S" -d "2000-01-01 10:10:00 UTC +30 seconds" # Show the resulting date adding the specified time to the given date in the specified format (date and time in this example) Difference between dates: date1='20050901'; date2='20200912'; echo $(( ( $( date +%s -d "$date2" ) - $( date +%s -d "$date1" ) )/86400 )); # Difference between two dates in days time1='10:08'; time2='11:10'; IFS=: read hour1 min1 <<< "$time1"; IFS=: read hour2 min2 <<< "$time2"; min1=$((10#$hour1*60 + 10#$min1)); min2=$((10#$hour2*60 + 10#$min2)); echo "$((min2 - min1))"; # Difference between two times in minutes date1='2020-11-10 12:00:10'; date2='2020-11-30 17:56'; echo $((($(date -ud "$date2" +'%s') - $(date -ud "$date1" +'%s'))/60)) # Difference between two dates and times in minutes Get the day of the week for a specified date: date -d "2000-01-01" +"%A" Extract Date and Time from Apache Logs: head /var/log/apache2/access.log --lines=1 | sed 's#[^[]*[[]\([^]][^]]*\).*#\1#' # Get Date and Time (including Time Zone) of the first entry in the Apache Log access.log head /var/log/apache2/access.log --lines=1 | sed 's#[^[]*[[]\([^]][^]]*\).*#\1#' | awk '{print $1}' # Get Date and Time (excluding Time Zone) of the first entry in the Apache Log access.log tail /var/log/apache2/access.log --lines=1 | sed 's#[^[]*[[]\([^]][^]]*\).*#\1#' # Get Date and Time (including Time Zone) of the last entry in the Apache Log access.log tail /var/log/apache2/access.log --lines=1 | sed 's#[^[]*[[]\([^]][^]]*\).*#\1#' | awk '{print $1}' # Get Date and Time (excluding Time Zone) of the last entry in the Apache Log access.log date -d "$(echo 'dd/MMM/yyyy:hh:mm:ss' | sed -e 's,/,-,g' -e 's,:, ,')" "+%a %d %b %Y %H:%M:%S" # Convert Apache log date/time to a better human readable date date -d "$(echo 'dd/MMM/yyyy:hh:mm:ss' | sed -e 's,/,-,g' -e 's,:, ,')" +"%s" # Convert Apache log date/time to Linux timestamp Calculate how long takes a command to be executed: time COMMAND COMMAND_PARAMETERS time cat # works like a stopwatch: counts time until you send an EOF to stop it (which generally means until you press CTRL+D that closes the cat command) Create filenames containing current date and time: touch data_`date +%F`.bak # create a file with the name data_YYYY-MM-DD.bak touch log_`date +%F_%H-%M-%S`.txt # create a file with the name log_YYYY-MM-DD_HH-MM-SS.txt User and system times: times Pause: sleep S # pause for S seconds Wait for a key press: read -rs -n 1 -p "Press any key to continue..." wkey # Wait for any key read -rs -p "Hit ENTER to continue..." wkey # Wait for ENTER Show a calendar: # cal # show dayweeks in a row, Sunday first (use -m to have Monday first) # ncal # show dayweeks in a coloumn, Monday first cal # show a calendar for the current month cal -y # show a calendar for the current year cal -y YEAR # show a calendar for the given year cal MONTH YEAR # show a calendar for the given MONTH and YEAR ncal -e YEAR # show date of Easter (for Western Churches) ncal -o YEAR # show date of Orthodox Easter (Greek and Russian Orthodox Churches) Memorable dates: The directory /usr/share/calendar/ contains some collections of remarkable dates. For example: cat /usr/share/calendar/calendar.history # Important historical dates cat /usr/share/calendar/calendar.music # Important dates in music history cat /usr/share/calendar/calendar.lotr # Lord of the Rings dates ----- Archiving and Compression: Install archivers: apt-get install bzip2 apt-get install zip unzip apt-get install zoo apt-get install lha apt-get install rar apt-get install unp Archiving with ISO: dd if=/dev/dvd of=dvd.iso # Create an ISO file from a DVD dd if=/dev/cdrom of=cd.iso # Create an ISO file from a CD-ROM dd if=/dev/scd0 of=cd.iso # Create an ISO file from a SCSI CD-ROM mkisofs -o /PATH_DEST/cd.iso /PATH_SOURCE/ # Create an ISO file from a directory tree Archiving with TAR: tar -cvf file.tar text.txt # archive text.txt as file.tar tar -cvf file.tar *.txt # archive all files ending with ".txt" in the current directory as file.tar tar -xvf file.tar # extract all files from file.tar in the current directory Compressed TAR archives (tarball): # Use tar to compress a directory tree into a single file, and gzip or bzip2 to compress each file into a directory tree tar -cjvf file.tbz text.txt # TAR+BZIP2 compression (canonical extension .tbz or .tar.bz2) tar -xjvf file.tbz # extract all files from a TAR+BZIP2 compressed archive tar -czvf file.tgz text.txt # TAR+GZIP compression (canonical extension .tgz or .tar.gz) tar -xzvf file.tgz # extract all files from a TAR+GZIP compressed archive Compression with GZIP: gzip -9vk text.txt # compress text.txt as text.txt.gz using highest compression and verbose output, keeping the original uncompressed file gzip -9v text.txt > file.gz # compress text.txt as file.gz using highest compression and verbose output, keeping the original uncompressed file gzip -9v text.txt # compress text.txt as text.txt.gz using highest compression and verbose output, deleting the original uncompressed file gzip -r9v file.txt file2 file3 /home/docs # compress file.txt file2 file3 and every file in /home/docs (if such directory exists), deleting the original uncompressed files gzip -l file.gz # show the content of the compressed file gzip -dk file.gz # extract the content of file.gz in the current directory and keep the compressed file (file.gz) gzip -d file.gz # extract the content of file.gz in the current directory and delete the compressed file (file.gz) Compression with BZIP2: bzip2 text.txt # compress text.txt as text.txt.bz2 bunzip2 file.bz2 # extract the content of file.bz2 into the current directory Compression with ZIP: zip file.zip file.txt # add file.txt to the compressed file file.zip (if it doesn't exist, it will created) zip -r file.zip /home/docs # compress the whole content of the directory /home/docs into file.zip zip -r filename.zip file.txt file2 file3 /home/docs # compress file.txt file2 file3 and the content of /home/docs (if such directory exists) into file.zip zip -F brokenfile.zip # try to fix a broken zipped file (use -FF to try even harder) unzip file.zip # extract the content of file.zip into the current directory unzip file.zip -d PATH_DEST # extract the content of file.zip into the specified directory Encrypted Compression with ZIP: # Note that Zip encryption is not very strong zip -e file.zip file.txt # encrypt file.txt into file.zip prompting for a password zip -P PASSWORD FILE.zip file.txt # encrypt file.txt into file.zip with the given PASSWORD (beware of shoulder surfing) unzip file.zip # extract the content of file.zip and prompt for password if encrypted unzip -P PASSWORD file.zip # unencrypt and extract the content of file.zip with given PASSWORD (beware of shoulder surfing) Compression with LHa (LHarc) / LZH: lha a file.lha text.txt # compress text.txt as file.lha (or add text.txt to an already existing file.lha compressed archive) lha e file.lha # extract the content of file.lha into the current directory (will prompt for existing files) Compression with ZOO: zoo a file.zoo text.txt # compress text.txt as file.zoo (or add text.txt to an already existing file.zoo compressed archive) zoo e file.zoo # extract the content of file.zoo into the current directory (will ignore existing files) Compression with RAR (RAR is shareware): rar a file.rar text.txt # compress text.txt as file.rar (or add text.txt to an already existing file.rar compressed archive) rar e file.rar # extract the content of file.rar into the current directory (will prompt for existing files) Unpack most common formats: unp FILE # Supported extensions and formats: tar[.gz,.bz2], gz, bz2, Z, ar/deb, rpm, shar, rar, arj, zip, LHa, cab, ace, tnef, uu (mail, news), mime, hqx, sea, zoo, pmd, cpio, afio, lzop ----- Encoding: Install encoders/decoders: apt-get install mpack apt-get install uudeview apt-get install yencode MIME/Base64: mpack -s SUBJECT -o PATH/DEST_FILE PATH/SOURCE_FILE # encode (B64encode) within a MIME message with header munpack PATH/FILE # decode (B64decode) to the original file name uudeview -i PATH/FILE # decode (B64decode) to the original file name Base64: uuenview -b PATH/SOURCE_FILE>PATH/DEST_FILE # encode (B64encode). will not be decoded without MIME header. base64 PATH/SOURCE_FILE>PATH/DEST_FILE # encode (B64encode) base64 -d PATH/SOURCE_FILE>PATH/DEST_FILE # decode (B64decode) echo "TEXT"|base64 # encode a TEXT (B64encode) echo "VEVYVAo="|base64 -d # decode an encoded TEXT (B64decode) UU (Unix-to-Unix): uuenview -u PATH/SOURCE_FILE>PATH/DEST_FILE # encode (uuencode) uudeview -i PATH/FILE # decode (uudecode) to the original file name XX: uuenview -x PATH/SOURCE_FILE>PATH/DEST_FILE # encode (xxencode) uudeview -i PATH/FILE # decode (xxdecode) to the original file name Yenc: uuenview -y PATH/SOURCE_FILE>PATH/DEST_FILE # encode (yencode) uudeview -i PATH/FILE # decode (ydecode) to the original file name # uudeview handy options: +o don't overwrite existing files, -c autoclear (delete) successfully decoded files, -d desperate mode (process incomplete files) yencode PATH/SOURCE_FILE # encode (yencode) SOURCE_FILE as SOURCE_FILE.ync ydecode PATH/FILE # decode (ydecode) to the original file name ydecode -f PATH/FILE # decode (ydecode) to the original file name (force overwriting already existing files, without prompting) ----- Hashing: md5sum PATH/FILE # Return the MD5 hash for the content of the given FILE md5sum PATH/FILE* # Return the MD5 hash for the content of the files matching the given mask md5sum PATH/FILE1 PATH/FILE2 PATH/FILE3 # Return the MD5 hash for the given FILEs echo -n "STRING" | md5sum | sed "s/ -//g" # Return the MD5 hash of the given STRING VARIABLE=`echo -n "STRING" | md5sum | sed "s/ -//g"`; # Store the MD5 hash of the given STRING into a VARIABLE VARIABLE=`echo -n "STRING" | md5sum | sed "s/ -//g"`; echo "[$VARIABLE]"; # Store the MD5 hash of the given STRING into a VARIABLE and output it between square brackets ----- Obfuscation: Text scrambling (anagrams): apt-get install wordplay wordplay "text string" # list all the possible anagrams of the given text string Alphabet scrambling: apt-get install rotix rotix # accept a text from standard input and return its ROT-13 to the standard output rotix -f PATH/SOURCE_FILE -o PATH/DEST_FILE # read SOURCE_FILE and return its ROT-13 to DEST_FILE (note: a ROT-13 can be deobfuscated performing the same operation again) rotix -r 15 -L -f PATH/SOURCE_FILE -o PATH/DEST_FILE # read SOURCE_FILE and return its ROT-15 to the Left to DEST_FILE (note: this can be deobfuscated performing a ROT-15 to the Right) rotix -r 12 -R -f PATH/SOURCE_FILE -o PATH/DEST_FILE # read SOURCE_FILE and return its ROT-12 to the Right to DEST_FILE (note: this can be deobfuscated performing a ROT-12 to the Left) Steganography: apt-get install steghide steghide --embed -p PASSWORD -ef SOURCE_FILE_TO_HIDE -cf DEST_OBFUSCATION_FILE # hide a file into another (AES encryption and compression are applied by default) steghide --embed -p PASSWORD -ef SOURCE_FILE_TO_HIDE -cf OBFUSCATION_FILE -sf DEST_FILE # hide a file into another (AES encryption and compression are applied by default), but save the result into a new file steghide --info STEGOFILE -p PASSWORD # return information about the file obfuscated into STEGOFILE steghide --extract -p PASSWORD -sf STEGOFILE -xf DESTFILE # extract the file obfuscated into STEGOFILE as DESTFILE # steghide applies AES encryption and compression by default. It can hide information into JPEG, BMP, WAV, and AU files. apt-get install stegdetect stegdetect FILE.jpg # try to find steganographed information into a JPEG file ----- Encryption with openssl: # Note that AES-256-CBC method in unsecure as it does not provide authenticated encryption and is vulnerable to padding oracle attacks. apt-get install openssl openssl aes-256-cbc -pbkdf2 -salt -in PATH/CLEARTEXT -out PATH/ENCRYPTED # Encrypt CLEARTEXT file to ENCRYPTED file. It will prompt for password twice. openssl aes-256-cbc -d -pbkdf2 -in PATH/ENCRYPTED -out PATH/CLEARTEXT # Decrypt ENCRYPTED file to CLEARTEXT file. It will prompt for password. openssl aes-256-cbc -pbkdf2 -salt -pass pass:PASSWORD -in PATH/CLEARTEXT -out PATH/ENCRYPTED # Encrypt CLEARTEXT file to ENCRYPTED file using PASSWORD. openssl aes-256-cbc -d -pbkdf2 -pass pass:PASSWORD -in PATH/ENCRYPTED -out PATH/CLEARTEXT # Decrypt ENCRYPTED file to CLEARTEXT file using PASSWORD. echo -n "CLEARTEXT"|openssl aes-256-cbc -pbkdf2 -salt -pass pass:PASSWORD|base64 # Encrypt CLEARTEXT to ENCRYPTED using PASSWORD and encode it as Base64. echo -n "U2FsdGVkX19YTYqwG/Dxp5t0k7mhGPYyvv70hiJfvOQ="|base64 -d|openssl aes-256-cbc -d -pbkdf2 -pass pass:PASSWORD # Decrypt Base64 encoded ENCRYPTED text to CLEARTEXT using PASSWORD. Encryption with GnuPG (GPG) and Pretty Good Privacy (PGP) wrapper: apt-get install pgpgpg # PGP pgp -h # Help pgp -k # Key Manager functions help pgp -kg # Generate key pgp -kxa USER_ID PATH/DEST_FILE.ASC # Export User's public key to file pgp -ka PATH/PUBLIC.KEY # Add public key to keyring pgp -ea PATH/SOURCE_CLEARTEXT RECIPIENT_ID # Encrypt file for Recipient pgp PATH/ENCRYPTED_FILE # Decrypt file (need password) # Handy options: # (Usage example: PGP -e = binary object file, PGP -ea = ASCII object file) # -a ASCII armour # -w Wipe (overwrite&delete) original file (when crypting) # -m No output file (on screen display, "eyes only") (when decrypting) # GPG gpg -h # Help gpg --version # Show GPG version and supported algorithms gpg --gen-key # generate key gpg --list-keys # list public keys in the public keyring gpg --list-secret-keys # list secret (private) keys in the secret keyring gpg --export -a > PATH/DEST_FILE.ASC # Export all public keys to a file gpg --export -a "USER_ID" > PATH/DEST_FILE.ASC # Export User's public key to file gpg --import PATH/PUBLIC.KEY # Add public key to keyring gpg -s -r "USER_ID" PATH/SOURCE_CLEARTEXT # Sign file for Recipient, file name is automatically generated gpg -s -a -r "USER_ID" PATH/SOURCE_CLEARTEXT # Create a signature for the given file for Recipient and output with an ASCII armour, file name is automatically generated gpg -se -r "USER_ID" PATH/SOURCE_CLEARTEXT # Sign and encrypt file for Recipient, file name is automatically generated gpg -se -a -r "USER_ID" PATH/SOURCE_CLEARTEXT # Sign and encrypt file for Recipient and output with an ASCII armour gpg --verify PATH/SIGNATURE.SIG # Verify a signature (without showing the signed content) gpg PATH/ENCRYPTED_FILE # Decrypt file (need password), file name is automatically generated gpg -d PATH/ENCRYPTED_FILE > PATH/DEST_FILE # Decrypt file specifying the output file name (need password) gpg -d PATH/ENCRYPTED_FILE # Decrypt file, on screen display, "eyes only" (need password) ----- Passwords: Password generation: tr -dc 'a-z0-9' < /dev/urandom | head -c 10 # generate a random password of 10 characters with only lower case letters and numbers apt-get install apg apg -n 1 -M NL -a 1 -m 8 -x 12 # Generate a random password from 8 to 12 characters long made with only lower case letters and numbers apg -n 1 -M NL -a 0 -m 10 -x 15 # Generate a random but human readable password from 10 to 15 characters long made with only lower case letters and numbers apt-get install pwgen pwgen -1 -A -n -B # Generate a password of 8 characters made with only lower case letters (-A) and at least one number (-n, default behavior) without ambiguous characters (-B) ----- Audio: Play a sound: printf '\7' # Ring the Bell Extract audio tracks from a CD: apt-get install cdda2wav cdda2wav -B # extract tracks as WAVs from the default CD player cdda2wav -D /dev/cdrom -B # extract tracks as WAVs from the specified CD player Create an Audio CD writing WAVs as audio tracks: apt-get install cdrecord cdrecord dev=0,0,0 speed=4 -eject -pad -audio -raw -swab *.wav # write all WAVs from the current directory Create an Audio CD writing MP3s/OGGs/flacs as audio tracks: apt-get install mp3burn mp3burn *.mp3 # create an audio CD from all MP3s in the current directory mp3burn -p "PLAYLIST.m3u" # create an audio CD fetching files listed in a M3U playlist Play MP3 files: apt-get install mpg321 mpg321 FILE.mp3 Edit ID3v1 tags: apt-get install id3ed id3ed -i FILE.mp3 # return id3v1 tag id3ed -r FILE.mp3 # remove id3v1 tag id3ed -s SONGNAME -n ARTIST -a ALBUM -y YEAR -c COMMENT -k TRACKNUM -g GENRE_NUMBER FILE.mp3 # set id3v1 for FILE.mp3 id3ed -s SONGNAME -n ARTIST -a ALBUM -y YEAR -c COMMENT -k TRACKNUM -g GENRE_NUMBER *.mp3 # set id3v1 for all mp3 files in the current directory Edit ID3v1 and ID3v2 tags: apt-get install id3v2 id3v2 -l FILE.mp3 # return id3 tags id3v2 -1 -l FILE.mp3 # return id3v1 tag id3v2 -2 -l FILE.mp3 # return id3v2 tag id3v2 --delete-v1 FILE.mp3 # remove id3v1 tag id3v2 --delete-v2 FILE.mp3 # remove id3v2 tag id3v2 --delete-all FILE.mp3 # remove all tags (id3 v1 and v2) id3v2 -C FILE.mp3 # convert id3v1 to id3v2 tag id3v2 -a "ARTIST" -A "ALBUM" -t "SONG" -c "COMMENT" -g GENRE_NUMBER -y YEAR -T TRACK FILE.mp3 # set id3 for FILE.mp3 id3v2 -a "ARTIST" -A "ALBUM" -t "SONG" -c "COMMENT" -g GENRE_NUMBER -y YEAR -T TRACK *.mp3 # set id3 for all mp3 files in the current directory id3v2 -1 -a "ARTIST" -A "ALBUM" -t "SONG" -c "COMMENT" -g GENRE_NUMBER -y YEAR -T TRACK FILE.mp3 # set id3v1 for FILE.mp3 id3v2 -2 -a "ARTIST" -A "ALBUM" -t "SONG" -c "COMMENT" -g GENRE_NUMBER -y YEAR -T TRACK FILE.mp3 # set id3v2 for FILE.mp3 Check MP3 files: apt-get install mp3check mp3check FILE.mp3 # check crc and headers of mp3 files for consistency mp3check -a FILE.mp3 # check mp3 file for differences from layer 3, 44.1kHz, 128kB, joint stereo, no emphasis, has crc Lossless MP3 normalization (peak and loudness normalization): apt-get install mp3gain mp3gain FILE1.mp3 FILE2.mp3 FILE3.mp3 # normalize specified mp3 files (warning: overwrites source files) mp3gain -r FILE1.mp3 FILE2.mp3 FILE3.mp3 # normalize specified mp3 files all to equal loudness (warning: overwrites source files) mp3gain -a FILE1.mp3 FILE2.mp3 FILE3.mp3 # normalize specified mp3 files to the average loudness, intended for all audio files from a same album (warning: overwrites source files) mp3gain -u FILE.mp3 # undo changes made by mp3gain (based on info stored on tag) Convert WAV to MP3: apt-get install lame lame FILE.wav FILE.mp3 Convert MP3 to WAV: apt-get install mpg321 mpg321 -w FILE.wav FILE.mp3 # convert FILE.mp3 to FILE.wav Convert MP3 to OGG: apt-get install mp32ogg mp32ogg FILE.mp3 # convert FILE.mp3 to FILE.ogg mp32ogg --rename=DEST.ogg SOURCE.mp3 # convert SOURCE.mp3 to DEST.ogg mp32ogg --rename=%a_%l_%t.ogg *.mp3 # convert all mp3 files in the current directory renaming generated OGG files as ARTIST_ALBUM_TRACKTITLE.ogg (make sure that id3 tag contains all such information, otherwise files will be called as SOURCEFILE.ogg) Convert an audio format into another audio format apt-get install sox sox SOURCE_FILE.SOURCE_EXT DEST_FILE.DEST_EXT # convert SOURCE_FILE.SOURCE_EXT to DEST_FILE.DEST_EXT. File format conversion is based on standard extensions sox FILE.ogg FILE.mp3 # convert FILE.ogg into FILE.mp3 (note that MP3 support is optional. check sox --help for supported formats) Reverse audio: sox -V SOURCE_FILE.wav DEST_FILE.wav reverse ----- Manipulate video files apt-get install ffmpeg Convert a video format into another video format: ffmpeg -i PATH/SOURCE_VIDEO.avi PATH/DEST_VIDEO.mpg # convert from AVI to MPEG ffmpeg -i PATH/SOURCE_VIDEO.avi -sameq FILE.avi PATH/DEST_VIDEO.mpg # convert from AVI to MPEG, keeping same quality of the source file ffmpeg -i PATH/SOURCE_VIDEO.avi -ss 00:00:00 -t 00:45:00 PATH/DEST_VIDEO.mpg # convert from AVI to MPEG, taking only the first 45 minutes of the source video (from 00:00:00 to 00:45:00) ffmpeg -i PATH/SOURCE_VIDEO.avi -i PATH/SOURCE_AUDIO.mp3 PATH/DEST_FILE.avi # create a video with audio taking video and audio from two different sources ffmpeg -loop_input -i PATH/SOURCE_IMAGE.jpg -i PATH/SOURCE_AUDIO.mp3 -shortest -acodec copy PATH/DEST_FILE.mp4 # create a video with a static image and audio Dump all frames from a video: ffmpeg -i PATH/SOURCE_VIDEO -an -r 1/1 PATH/FILE_%06d.jpg # Dump all frames as JPG, you can dump them in .bmp or .png by changing the extension of the destination file. In this example the generated file name will have 6 digits Speed up or slow down a video: ffmpeg -i PATH/SOURCE_VIDEO -filter:v "setpts=SPEED*PTS" PATH/DEST_VIDEO # SPEED must be set to 1/(wanted speed x), for example set SPEED: 0.5 : double speed (2x), 0.25 : 4x speed, 2.0 = half speed (1/2x). If you want the destination video to have a specific duration, you can calculate the value of SPEED with this formula: 1 / (source video duration in seconds / destination video wanted duration in seconds) Extract the audio track from a video: ffmpeg -i PATH/SOURCE_VIDEO.mpg -vn -f mp3 AUDIO_TRACK.mp3 # extract the audio track as MP3 from a video ffmpeg -i PATH/SOURCE_VIDEO.mpg -vn -ac 1 -f mp3 AUDIO_TRACK.mp3 # extract the audio track as MP3 from a video as mono. Use -ac 2 to force stereo even if the input file is mono. ----- Play a video: apt-get install vlc vlc MOVIE.AVI # play a video (even in the text console) vlc http://www.example.com:8081/file.mpg # play a video from the web Streaming video server: vlc -vvv file.mpg --sout '#standard{access=http,mux=mpeg1,url=www.example.com:8081/file.mpg}' --ttl 12 --loop # Stream a MPEG file (loop) from the url http://www.example.com:8081/file.mpg # Note that you must have a streaming format like ASF to create a real stream of data vlc -vvv file.asf --sout '#standard{access=mmsh,mux=asf,url=www.example.com:8082}' --ttl 12 --loop # Stream an ASF file (loop) from the url mms://www.example.com:8082/file.asf ----- Manipulate and convert images: apt-get install imagemagick convert SOURCE.jpg DEST.png # Convert a JPEG image into PNG convert -resize 50% SOURCE.jpg DEST.jpg # resize SOURCE.jpg at 50% of the original size, and save it as DEST.jpg convert -resize 600x300 SOURCE.jpg DEST.jpg # resize SOURCE.jpg to 600x300 (width x height) pixels, and save it as DEST.jpg convert -resize 600 SOURCE.jpg DEST.jpg # resize SOURCE.jpg to a width of 600 pixels and relative height, and save it as DEST.jpg convert -resize x300 SOURCE.jpg DEST.jpg # resize SOURCE.jpg to a height of 300 pixels and relative width, and save it as DEST.jpg convert -thumbnail 150 SOURCE.jpg DEST.jpg # create a thumbnail of SOURCE.jpg with a width of 150 pixels and relative height, and save it as DEST.jpg # the difference between -resize and -thumbnail is that -thumbnail strips all meta data (like EXIF information) from the original image convert -size 1x1 xc:none IMAGE.png # create a transparent .png image sized 1 x 1 pixels convert -thumbnail x100 PATH/FILE.pdf[0] THUMBNAIL.png # create a thumbnail with a height of 100 pixels for the first page of a PDF document convert -thumbnail x100 PATH/FILE.pdf THUMBNAIL.png # create a thumbnail with a height of 100 pixels for every page of a PDF document ----- EXIF information: apt-get install exiftool exiftool FILE.jpg # show EXIF tags inside a JPEG image exiftool -geotag= FILE.jpg # remove geographical coordinates from the given image exiftool -all= FILE.jpg # remove all EXIF information from the given image for i in PATH/*.jpg; do echo "Processing: $i"; exiftool -all= "$i"; done # remove all EXIF information from all the JPEG images in the given PATH apt-get install exif exif FILE.jpg # show EXIF tags inside a JPEG image exif -im FILE.jpg # return EXIF tags inside a JPEG image in a machine readable format (-i ID instead of tags, -m tabs delimited output) exif --ifd=0 -t TAG --set-value=VALUE SOURCE.jpg -o=DEST.jpg # set TAG as VALUE in SOURCE.jpg and save the resulting manipulated image as DEST.jpg exif --remove --ifd=0 SOURCE.jpg -o=DEST.jpg # remove all EXIF information from SOURCE.jpg and save the resulting cleaned image as DEST.jpg exif -r SOURCE.jpg -o=DEST.jpg # remove the thumbnail from SOURCE.jpg and save the resulting cleaned image as DEST.jpg exif -e SOURCE.jpg -o=THUMBNAIL.jpg # extract the thumbnail from SOURCE.jpg and save it as THUMBNAIL.jpg exif FILE.jpg -n=THUMBNAIL.jpg # add THUMBNAIL.jpg to FILE.jpg and save the resulting manipulated image as FILE.jpg.modified.jpeg ----- GPS data: apt-get install gpsbabel gpsbabel -i gdb -f PATH/SOURCE_FILE.gdb -o gpx -F PATH/DEST_FILE.gpx # convert a Garmin .gdb file into .gpx (GPS eXchange Format, XML) ----- Disk quotas: apt-get install quota quotatool quota # display disk usage and limits quotacheck # scan a file system for disk usage quotactl # set disk quotas ----- Quitting a session, turning off (halting) or rebooting system: # Note that when a process receives a signal may behave differently from the default behavior CTRL+C # terminate an application (sends a SIGINT (2) signal to the process, exits with Exit Code 130) CTRL+\ # terminate the process and dump to a core file (sends a SIGQUIT signal to the process) CTRL+D # send an EOT (End Of Tranmission) signal to the process (exits with Exit Code 0) CTRL+Z # suspend an application (sends a SIGSUSP (20) signal to the process, exits with Exit Code 148) return # return from a function or a sourced script exit # exit from a non login shell logout # logout from current session (you can also press CTRL+D ) shutdown -h now # halt the system shutdown -h -P now # halt the system and turn off the machine (power off) shutdown -r now # reboot the system shutdown -r 22:00 # reboot the system at 22:00 (10:00 pm) shutdown -r +5 # reboot the system after 5 minutes reboot # reboot the system (like shutdown -r), might not work on older systems reboot -f # force reboot without using shutdown Show system reboot history: last reboot last -x|grep shutdown for wtmp in `ls -t /var/log/wtmp*`; do last reboot -f $wtmp; done|grep reboot last reboot|head -1 # Show only last reboot Show uptime: # A sysadmin will be judged by his uptime and by how many reboots he performed uptime # Show uptime, total users, and load average of the system over the last 1, 5, and 15 minutes. uptime -p # Show only uptime in pretty form uptime -s # Show system up since (last boot) # Edit system startup script # Commands that will be executed when the system boots. Different runlevels are allowed. jed /etc/inittab Choose what command must be performed when CTRl+ALT+DEL is pressed: jed /etc/inittab # What to do when CTRL-ALT-DEL is pressed. ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now ----- Install a LAMP webserver: Install servers and related software: apt-get install apache2 apt-get install lynx apt-get install openssl # apt-get install php4 libapache2-mod-php4 php4-pear php4-gd php4-mcrypt # PHP 4 # apt-get install php4-mysql # MySQL support for PHP 4 # apt-get install php5 libapache2-mod-php5 php-pear php5-gd php5-mcrypt php5-sqlite # PHP 5 # apt-get install php5-mysql # MySQL support for PHP 5 # apt install php7.0 libapache2-mod-php7.0 php-pear php7.0-gd php7.0-mcrypt php7.0-mbstring php7.0-sqlite3 # PHP 7 # apt install php7.0-mysql # MySQL support for PHP 7 apt install php7.3 libapache2-mod-php7.3 php-pear php7.3-gd php7.3-mbstring php7.3-sqlite3 # PHP 7.3 apt install php-imagick # ImageMagick for PHP apt install php7.3-mysql # MySQL support for PHP 7.3 # apt-get install mysql-server # Obsolete since Debian GNU/Linux 10 (buster) apt install mariadb-server-core-10.3 mariadb-server-10.3 mariadb-plugin-tokudb mariadb-plugin-spider mariadb-plugin-oqgraph mariadb-plugin-mroonga mariadb-plugin-connect apt install phpmyadmin Apache 2 configuration: /usr/sbin/apache2 -v # Return Apache 2 version openssl version -v # Get OpenSSL version apache2 -l # list compiled in Apache 2 modules apache2ctl -l # list compiled in Apache 2 modules apache2ctl -M # list loaded Apache 2 modules a2query -M # Return Apache Multi-Processing Module (MPM): event, prefork, worker a2enmod # Add a module choosing it from a list. You'll have to restart Apache 2 then. a2enmod rewrite # Add rewrite module. Restart Apache to enable it. jed /etc/apache2/apache2.conf If your webserver is too slow, you may have to adjust the value for MaxClients: if it's too low it will allow too few simultaneous accesses, if it's too high it will require too much resources to handle them. Try with MaxClients 50 or MaxClients 100 . MaxClients 100 Also, you'd better set MaxKeepAliveRequests to a value higher than the default: MaxKeepAliveRequests 2000 You can also add the expires module and set expiry periods for the cache of file types you don't change often on the server (like images):ExpiresActive On ExpiresDefault "access plus 2 days" ExpiresByType text/php "access plus 1 second" ExpiresByType text/x-php "access plus 1 second" ExpiresByType application/php "access plus 1 second" ExpiresByType application/x-php "access plus 1 second" ExpiresByType application/x-httpd-php "access plus 1 second" ExpiresByType text/html "access plus 2 days" ExpiresByType text/plain "access plus 2 days" ExpiresByType image/jpg "access plus 1 month" ExpiresByType image/jpeg "access plus 1 month" ExpiresByType image/gif "access plus 1 month" ExpiresByType image/png "access plus 1 month" ExpiresByType text/css "access plus 1 month" ExpiresByType application/pdf "access plus 1 month" ExpiresByType text/x-javascript "access plus 1 week" # ExpiresByType application/x-shockwave-flash "access plus 1 month" ExpiresByType image/x-icon "access plus 1 year" If you use .htaccess files in your web directories to have them to be executed by the web server, you must be sure that AllowOverride is set to All for such directories:Options Indexes FollowSymLinks AllowOverride All Require all granted Specify available charsets and default charset: AddCharset ISO-8859-1 .iso8859-1 .latin1 AddCharset UTF-8 .utf8 # AddDefaultCharset UTF-8 AddDefaultCharset ISO-8859-1 jed /etc/apache2/ports.conf # Port listened by the webserver (normally 80 for HTTP connections and 443 for HTTPS secure connections) jed /etc/apache2/sites-enabled/default.conf # set DocumentRoot and create Virtual Hosts (vhosts). Older versions of Apache2 stores sites by default in /etc/apache2/sites-available/default This is an example of a VirtualHost: ServerName www.example.com ServerAdmin webmaster@example.com DocumentRoot "/var/www/example.com/www" ServerSignature On LogLevel Warn ErrorLog /var/log/apache2/example-com_error.log CustomLog /var/log/apache2/example-com_access.log combined Check Apache configuration for errors: apache2ctl configtest Restart Apache after modifying its configuration: /etc/init.d/apache2 restart See if Apache is running: netstat -plntu | grep apache # if you get a line containing the socket listened by the Apache webserver, then it's running See Apache error log: cat /var/log/apache2/error.log Delete Apache access and error logs rm /var/log/apache2/*.gz # Delete archived logs rm /var/log/apache2/* # In case you want to delete access and error logs for all websites. IMPORTANT: Note that you'll have to restart Apache to set up logs and have them running again Make sure the PHP package is enabled on Apache: a2query -m php7.3 If it replies with "No module matches php7.3", then enable it: a2enmod php7.3 Then restart Apache PHP configuration on Apache 2: jed /etc/php4/apache2/php.ini # PHP 4 configuration jed /etc/php5/apache2/php.ini # PHP 5 configuration jed /etc/php/7.x/apache2/php.ini # PHP 7.x configuration (make sure to replace x with your current subversion of PHP 7 ) Make sure engine = On in the PHP configuration file or PHP will not work at all. You may want to set short_open_tag = On if your PHP code uses short tags, which means < ? ... ? > instead of < ?php ... ? > You may specify the charset (character encoding) used by PHP generated pages. Note that it overrides charsets specified in the Apache configuration or in the HTML document (such as in HTML 4 or in HTML 5), so you'd probably better leave it empty default_charset = "" however in case you need to set it, most popular settings are: default_charset = "UTF-8" or default_charset = "iso-8859-1" Show PHP version: php -v Show PHP info: php -i List compiled-in PHP modules: php -m To send e-mails from PHP scripts, you need to install sendmail apt-get install sendmail sendmail-bin sendmail-doc rmail Then edit the hosts file ( jed /etc/hosts ) and set the alias for the IP 127.0.0.1 as 127.0.0.1 DOMAIN localhost localhost.localdomain HOSTNAME where DOMAIN is a domain name (for example: example.com ) addressed to your server's IP and HOSTNAME is the hostname of your server (that you can get with the command hostname ). The domain name is not necessary but strongly recommended because some antispam services blacklist servers which send e-mails identifying themselves (EHLO) as "localhost.localdomain". After setting the hosts file you have to reconfigure and restart sendmail: sendmailconfig # Normally you have to just reply with Y to all questions service sendmail restart To send a test e-mail from the console: echo "Subject: test"; echo "Body Test"; |/usr/lib/sendmail -v -F SENDER@MAILBOX.EXAMPLE.COM -t RECEIVER@MAILBOX.EXAMPLE.COM You should check sendmail_path in PHP configuration (php.ini) and make sure it's either disabled or set to the default value of sendmail -t -i To send an e-mail from a PHP page you have to use the mail command, like in this example: mail($emailto, $emailsubject, $emailbody, "From: ".$emailfrom."\nReply-To: ".$emailreplyto."\nX-Mailer: ".$xmailer); Of course all sample variables used must be set properly. This is a test without using variables, you need to replace email addresses in this example with actual working ones: mail('user2@mailbox.example.com', 'Test', "This is a test\nfrom PHP", "From: ".'user1@mailbox.example.com'."\nReply-To: ".'user1@mailbox.example.com'."\nX-Mailer: ".'PHP'); Check the log of last sent e-mails: tail --lines=20 /var/log/mail.log Show MySQL version: mysql -V See if MySQL is running: netstat -plntu | grep mysql # if you get a line containing the socket listened by the MySQL server, then it's running Set MySQL root password: $ mysql mysql> SET PASSWORD FOR root@localhost = PASSWORD('newpassword'); Create an alternate user with root-like access: $ mysql -u root -pROOT_PASSWORD mysql mysql> CREATE USER 'USER_NAME'@'localhost' IDENTIFIED BY 'SET_PASSWORD_HERE'; GRANT ALL PRIVILEGES ON *.* TO 'USER_NAME'@'localhost'; FLUSH PRIVILEGES; Create a new mysql user and assign a database to him $ mysql -u root -pROOT_PASSWORD mysql mysql> insert into user (Host, User, Password, Select_priv) values ('localhost', 'USERNAME', password('USER_PASSWORD'), 'N'); Query OK, 1 row affected (0.00 sec) mysql> insert into db (Host, Db, User, Select_priv, Insert_priv, Update_priv, Delete_priv, Create_priv, Drop_priv, Grant_priv, References_priv, Index_priv, Alter_priv, Create_tmp_table_priv, Lock_tables_priv) values ('localhost', 'USER_DATABASE', 'USER_NAME', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y'); Query OK, 1 row affected (0.00 sec) mysql> quit $ mysqladmin -u root -pROOT_PASSWORD reload The user can access the assigned database: $ mysql -u USERNAME -pUSER_PASSWORD USER_DATABASE Remove an user: mysql> drop user USERNAME@HOSTNAME; Or mysql> delete from mysql.user where User='USERNAME' and Host='HOSTNAME'; mysql> revoke all privileges on *.* from USERNAME@HOSTNAME; mysql> revoke grant option on *.* from USERNAME@HOSTNAME; mysql> flush privileges; Show all MySQL users: mysql> select user, host from mysql.user; # show only user name and related host name for every user mysql> select * from mysql.user; # show detailed information for every user Show all databases: mysql> show databases; Show all tables in the current database: mysql> show tables; mysql> show table status; # provide information about every table Show all fields (coloumns) of a table: mysql> show fields from TABLE; # same as describe TABLE; mysql> show fields from TABLE from DB; # show fields of a table from another database mysql> show full fields from TABLE; # also lists privileges available to the current user Show all indexes (keys) of a table: mysql> show index from TABLE; mysql> show index from TABLE from DB; # show indexes of a table from another database Create an index (key) for a field (coloumn): mysql> alter table TABLE add index (`FIELD`) ; Count all records within a table: mysql> select count(*) from TABLE; Backup a database to a MySQL dump file (structure only): mysqldump -no-data -u MYSQLUSER -pPASSWORD DATABASE > MYSQLDUMP.sql Backup a database to a MySQL dump file (data only): mysqldump -c --no-create-db --no-create-info -u MYSQLUSER -pPASSWORD DATABASE > MYSQLDUMP.sql Backup a database to a MySQL dump file (structure and data): mysqldump -c --add-drop-database --add-drop-table --add-locks -u MYSQLUSER -pPASSWORD DATABASE > MYSQLDUMP.sql Back up a MySQL database into a GZIP file: mysqldump -c --add-drop-database --add-drop-table --add-locks -u MYSQLUSER -pPASSWORD DATABASE | gzip -9v > MYSQLDUMP_`date +%Y%m%d`.sql.gz Restore a database from a MySQL dump file: mysql -h localhost -u MYSQLUSER -pPASSWORD DATABASE < MYSQLDUMP.sql Delete a database: mysql> drop database if exists DATABASE; Delete a table: mysql> drop table if exists TABLE; Delete a field: mysql> alter table TABLE drop FIELD; Delete an index: mysql> drop index INDEX on TABLE; Empty a table without deleting it: mysql> truncate table TABLE; Change engine type: mysql> alter table TABLE type = MYISAM; # change the engine tipe of TABLE to MyISAM Analyzes indexes of a table: # for MyISAM and BDB tables only mysql> analyze table TABLE; Check a table for errors: mysql> check table TABLE; Attempt to repair a corrupted table mysql> repair table TABLE; Optimize a table (defragment and rebuild indexes): mysql> optimize table TABLE; Show MySQL server status: mysql> show status; Check running MySQL processes (queries): $ mysqladmin -h localhost -u root -pROOT_PASSWORD processlist Or, from the MySQL console: mysql> show processlist; Kill a MySQL process: $ mysqladmin -u root -pROOT_PASSWORD kill ID Or, from the MySQL console: mysql> kill ID; Quit MySQL console: mysql> quit Show MySQL version: mysql -V To restart MySQL: /etc/init.d/mysql restart # If MySQL fails to start make sure the directories /var/log/mysql and /var/log/mysqld are owned by mysql:adm Run a MySQL file and store output into another file: mysql -u USER --password=PASSWORD DATABASE_NAME < SOURCE.sql > DESTINATION.txt This may fix these errors that may happen after an upgrade: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' ERROR 1577 (HY000) at line 1: Cannot proceed because system tables used by Event Scheduler were found damaged at server start apt-get install php5-mysql mysql-server mysql-server-5.1 mv /etc/mysql/my.cnf /etc/mysql/my.cnf.bak1 mv /etc/mysql/my.cnf.dpkg-dist /etc/mysql/my.cnf /etc/init.d/mysql start apt-get install php5-mysql mysql-server mysql-server-5.1 Upgrading from PHP 4 to PHP 5: # Note: this will also upgrade from MySQL 4 to MySQL 5, if MySQL 4 is installed # Warning: newer versions of MySQL often use different formats for tables than earlier versions. In some cases you can attempt to make restore such tables compatible using repair table TABLE; and optimize table TABLE; You'd better dump all your tables before to upgrade, in case you'll need to restore them after. In fact you may need to DROP tables which are no longer working as expected and rebuild them. A common issue are tables containing floating point fields: queries containing comparisons based on those fields will no longer work as they used to. # stop the servers /etc/init.d/apache2 stop /etc/init.d/mysql stop # remove PHP 4 apt-get remove libapache2-mod-php4 php4 php4-gd php4-pear php4-mcrypt php4-mysql # You have to remove MySQL as well apt-get remove mysql-server # Install PHP 5 apt-get install php5 libapache2-mod-php5 php5-sqlite php-pear php5-gd php5-mcrypt # PHP 5 apt-get install php5-mysql # MySQL support for PHP 5 # Install MySQL again apt-get install mysql-server # Reinstall phpmyadmin (if you need it) apt-get install phpmyadmin # Removing PHP 4 will remove phpmyadmin as well, you'll have to reinstall it if you need it # Make sure that there are no references to PHP 4 left in the Apache configuration jed /etc/apache2/apache2.conf # Changeas # Add .php5 to AddType application/x-httpd-php # Comment out ... and everything between # restart the servers /etc/init.d/mysql restart /etc/init.d/apache2 restart Switch from PHP 5 to PHP 7: If you have two different versions of PHP, for example PHP 5.6 and PHP 7.0, as it can happen after upgrading Debian from jessie to stretch, you can switch between them with these commands: update-alternatives --config php # Set the default PHP version for the command php (this will not influence the version used on the Apache webserver) a2dismod php5 # Dismount the Apache2 Module for PHP5 a2enmod php7.0 # Mount the Apache2 Module for PHP7 systemctl restart apache2 # Restart Apache phpMyAdmin: # If you can't see the phpmyadmin directory into /var/www/ then the package likely installed it into /usr/share/ , in this case you have to link it manually to access it from the web ln -s /usr/share/phpmyadmin /var/www/ Install phpMyAdmin manually: # You may want to install phpMyAdmin manually, especially if your Debian is set to Stable and there's no 'phpmyadmin' package available. # Create a directory for phpMyAdmin mkdir /usr/share/phpmyadmin # Create a directory to save phpMyAdmin source mkdir /usr/share/phpmyadmin/src # Go to phpmyadmin source directory and use it as a work directory cd /usr/share/phpmyadmin/src # Download latest version of phpMyAdmin wget -P /usr/share/phpmyadmin/src https://www.phpmyadmin.net/downloads/phpMyAdmin-latest-all-languages.tar.gz # Download phpMyAdmin public keyring to verify the source wget -P /usr/share/phpmyadmin/src https://files.phpmyadmin.net/phpmyadmin.keyring # Import phpMyAdmin public keyring into GPG gpg --import phpmyadmin.keyring # Get phpMyAdmin public key wget -P /usr/share/phpmyadmin/src https://www.phpmyadmin.net/downloads/phpMyAdmin-latest-all-languages.tar.gz.asc # Verify phpMyAdmin source gpg --verify phpMyAdmin-latest-all-languages.tar.gz.asc # You should get something like: ---------- gpg: assuming signed data in 'phpMyAdmin-latest-all-languages.tar.gz' gpg: Signature made Thu 15 Oct 2020 20:10:40 CEST gpg: using RSA key 3D06A59ECE730EB71B511C17CE752F178259BD92 gpg: Good signature from "Isaac Bennetch" [unknown] gpg: aka "Isaac Bennetch " [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: 3D06 A59E CE73 0EB7 1B51 1C17 CE75 2F17 8259 BD92 ---------- # Verify that the key returned by GPG is the same as shown in phpMyAdmin's website for the given release manager person: https://docs.phpmyadmin.net/en/latest/setup.html#verifying-phpmyadmin-releases # In this case, Isaac Bennetch's fingerprint is 3D06 A59E CE73 0EB7 1B51 1C17 CE75 2F17 8259 BD92 # If everything is OK, unpack the compressed file into phpMyAdmin's directory tar xvf phpMyAdmin-latest-all-languages.tar.gz --strip-components=1 -C /usr/share/phpmyadmin/ # You can now remove the download directory (note that if left in the position given in this tutorial, it will also be available from the Web) cd /usr/share/phpmyadmin/ rm /usr/share/phpmyadmin/src/* rmdir /usr/share/phpmyadmin/src # Create a configuration file for phpMyAdmin cp /usr/share/phpmyadmin/config.sample.inc.php /usr/share/phpmyadmin/config.inc.php # Edit phpMyAdmin configuration file jed /usr/share/phpmyadmin/config.inc.php # And set a password for the cookies $cfg['blowfish_secret'] = 'PUT_YOUR_PASSWORD_HERE'; # Set modes for the configuration file chmod 660 /usr/share/phpmyadmin/config.inc.php # Set user and group for phpMyAdmin's directory chown -R www-data:www-data /usr/share/phpmyadmin # Create a symlink to make it accessible from the Webserver ln -s /usr/share/phpmyadmin /var/www/ # Make phpMyAdmin available from the Web, for example: jed /etc/apache2/sites-enabled/default.conf --------- ServerName mysql.EXAMPLE.COM ServerAdmin webmaster@EXAMPLE.COM DocumentRoot "/var/www/phpmyadmin" ServerSignature On LogLevel Warn ErrorLog /var/log/apache2/phpmyadmin_error.log CustomLog /var/log/apache2/phpmyadmin_access.log combined --------- # If you want to upgrade manually phpMyAdmin: # Make a backup of the current phpMyAdmin directory cp -rp /usr/share/phpmyadmin /usr/share/phpmyadmin_bak # Empty phpMyAdmin directory without removing it rm -r /usr/share/phpmyadmin/* # Follow the steps to install phpMyAdmin manually except for: Create a directory for phpMyAdmin (if you've emptied the existing one without removing it), Create a symlink to make it accessible from the Webserver, Make phpMyAdmin available from the Web. # If the new version works properly, you can remove the backup you've created rm -r /usr/share/phpmyadmin /usr/share/phpmyadmin_bak # Restart Apache Webserver /etc/init.d/apache2 restart HTTPS: If you want to have https websites (so that the client can access the server securely through an encrypted connection to the server) hosted in your webserver, follow this procedure: Certificates and keys should be stored in these directories: /etc/ssl/certs # for certificates /etc/ssl/private # for keys Generate a Key: openssl genrsa -out /etc/ssl/private/DOMAIN.key 2048 Enter a valid domain name Generate a request certificate: openssl req -new -sha256 -key /etc/ssl/private/DOMAIN.key -out www.example.com.csr The only fields you must fill are Organization Name (eg, company) and Common Name (e.g. server FQDN or YOUR name), you can leave empty all other fields. Note that fields with a given default value must be filled with a dot (.) to make them empty, if you simply press RETURN the default value will be used instead: ---------- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]:. State or Province Name (full name) [Some-State]:. Locality Name (eg, city) []: Organization Name (eg, company) [Internet Widgits Pty Ltd]:COMPANY Organizational Unit Name (eg, section) []: Common Name (e.g. server FQDN or YOUR name) []:DOMAIN Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: ---------- If you want to validate your domain (to claim its possession) via e-mail, enable the e-mail address admin@DOMAIN (that is, make sure you can receive e-mails to admin) for validation Buy a certificate from a Certificate Authority (CA) https://www.digicert.com/ https://www.ssl.com/ (via comodo) https://www.comodo.com/ Validate the domain (claim it as yours) according to the procedure requested by the Certificate Authority Download the certificates ( ca-bundle-client.crt , DOMAIN.crt ) Upload the certificates into /etc/ssl/certs (the private key you've generated should already be into /etc/ssl/private/ ) Edit the Apache configuration file: jed /etc/apache2/sites-enabled/default.conf Create a VirtualHost for the DOMAIN for port 443 with the following directives:SSLEngine on SSLCertificateFile /etc/ssl/certs/DOMAIN.crt SSLCertificateKeyFile /etc/ssl/private/DOMAIN.key SSLCertificateChainFile /etc/ssl/certs/ca-bundle-client.crt # HSTS (mod_headers is required) (15768000 seconds = 6 months) # Header always set Strict-Transport-Security "max-age=15768000" # ... For example:SSLEngine on SSLCertificateFile /etc/ssl/certs/example_com.crt SSLCertificateKeyFile /etc/ssl/private/example_com.key SSLCertificateChainFile /etc/ssl/certs/ca-bundle-client.crt # HSTS (mod_headers is required) (15768000 seconds = 6 months) # Header always set Strict-Transport-Security "max-age=15768000" ServerName www.example.com ServerAdmin webmaster@example.com DocumentRoot "/var/www/example.com/www" ServerSignature On LogLevel Warn ErrorLog /var/log/apache2/example-com_error.log CustomLog /var/log/apache2/example-com_access.log combined If you still want the website to be available via http (without encryption), you also have to configure it without SSL (normally for port 80). Note that it can be configured to have a different document directory () and thus provide different content from the https (encrypted) website.ServerName www.example.com ServerAdmin webmaster@example.com DocumentRoot "/var/www/example.com/www" ServerSignature On LogLevel Warn ErrorLog /var/log/apache2/example-com_error.log CustomLog /var/log/apache2/example-com_access.log combined If you want to redirect all http traffic to https for a portion of the website, add this line inside the VirtualHost for http (port :80) Redirect /ALWAYS_HTTPS_DIRECTORY_NAME https://www.example.com/ALWAYS_HTTPS_DIRECTORY_NAME If you want to redirect all http traffic to https for the whole website, add this line inside the VirtualHost for http (port :80) Redirect / https://www.example.com/ Restart the webserver when you're done with editing the configuration file: /etc/init.d/apache2 restart When you renew a certificate, you'll have to upload the newly reissued certificates ( ca-bundle-client.crt , DOMAIN.crt ) replacing the old ones into /etc/ssl/certs and restart the webserver. Password protected directories in Apache: # Create a password file, it should be placed in a directory not served by the webserver (for example, /var/ , which normally is out of the htdocs directory) touch PATH/.htpasswd chmod 644 PATH/.htpasswd # Create users and assign passwords to them htpasswd -b PATH/.htpasswd USER PASSWORD # Create a password for every user. The same statement can be used to change a password of an already existing user htpasswd -D PATH/.htpasswd USER # Delete a user from the password file # Create an access control file in the directory containing the files with restricted access # Use Require valid-user to allow access to any user in the password file, or list specific users allowed to access the restricted files jed PATH_TO_PROTECT/.htaccess --- AuthUserFile PATH/.htpasswd AuthName "Enter Password" AuthType Basic# Require valid-user Require user USER1 Require user USER2 --- Web site statistics: grep " 404 " /var/log/apache2/ACCESS.log | awk -F\" '$9=404{print $9" "$2}' | sort | uniq -c | sort -rg | head --lines=25 # Show top 404 error ("Not Found") requested pages grep "GET /" /var/log/apache2/ACCESS.log | awk {'print $7'} | sort | uniq -c | sort -g | tail --lines=15 | tac # Show top accessed pages awk {'print $7'} /var/log/apache2/ACCESS.log | grep ".php" | sort | uniq -c | sort -g | tail --lines=15 | tac # Show top accessed PHP pages grep -i "GET /PATH/DOCUMENT" /var/log/apache2/ACCESS.log | awk '{print $1}' | sort -n | uniq -c | sort -rn | head --lines=25 # Show top IPs accessing the given DOCUMENT (web page) Webalizer: apt-get install webalizer jed /etc/webalizer.conf # or jed /etc/webalizer/webalizer.conf # or another custom .conf file webalizer /var/log/apache2/access.log.1 # create a webalizer report for access.log.1 webalizer /var/log/apache2/example-com_access.log # create a webalizer report for example-com_access.log If you get the following error message: Error Opening file /usr/share/GeoIP/GeoIP.dat you can fix it installing the GeoIP database: apt-get install geoip-database To have statistics available on the web for many websites, create a different access log, webalizer configuration file and output directory, for every website and a configuration file, and call webalizer from a script. Example: # 1. Set Apache log files jed /etc/apache2/sites-enabled/default.conf[...] CustomLog /var/log/apache2/example-com_www_access.log combined [...] [...] CustomLog /var/log/apache2/example-net_my_access.log combined [...] /etc/init.d/apache2 restart # 2. Create output directories mkdir /var/www/webalizer mkdir /var/www/webalizer/www.example.com mkdir /var/www/webalizer/my.example.net # 3. Assign ownership of the directories to www-data chown www-data:www-data /var/www/webalizer/www.example.com chown www-data:www-data /var/www/webalizer/my.example.net # 4. Create custom Webalizer configuration files jed /etc/webalizer/webalizer_www.example.com.conf LogFile /var/log/apache2/example-com_www_access.log OutputDir /var/www/webalizer/www.example.com Incremental yes HostName www.example.com HideSite *example.com HideReferrer example.com/ [...] jed /etc/webalizer/webalizer_my.example.net.conf LogFile /var/log/apache2/example-net_my_access.log OutputDir /var/www/webalizer/my.example.net Incremental yes HostName my.example.net HideSite *example.com HideReferrer example.com/ [...] # 5. Create a script that invokes webalizer for every website mkdir /etc/scripts jed /etc/scripts/webalize.sh --- #!/bin/bash # Webalize webalizer -c /etc/webalizer/webalizer_www.example.com.conf>/dev/null webalizer -c /etc/webalizer/webalizer_my.example.net.conf>/dev/null --- Sample lines to add to Crontab: --- 1 0,12 * * * /var/www/scripts/webalize.sh>/dev/null 21 6 * * * /var/www/scripts/webalize.sh>/dev/null --- The first line tells Crontab to invoke Webalizer every day at 00:01 and 12:01, the second line makes Crontab invoke Webalizer every day at 6:21 . This is because logs start and end at a certain hour of the day (for example, 06:26), and you'd miss all the hours between last call of Webalizer, every day. To know the exact time when your logs starts and stops check the head and tail of the previous log, for example: head --lines=10 /var/log/apache2/www.example.net_access.log.1 , tail --lines=10 /var/log/apache2/www.example.net_access.log.1 # If you want to reset information cached from previous runs of Webalizer (especially if you've set Incremental yes in the webalizer configuration file), delete the files webalizer.current and webalizer.hist in the Webalizer OutputDir. # Example: # rm /var/www/webalizer/webalizer.current # rm /var/www/webalizer/webalizer.hist ----- SQLite: apt-get install sqlite sqlite3 libsqlite3-dev # To fix these errors: sqlite_exec(): attempt to write a readonly database; sqlite_exec(): unable to open database file chmod 666 *.sdb chmod 777 the directory containing the .sdb file ----- FTP server: # Note that FTP protocol is deprecated in favor of SSH connections, because it's unencrypted and unsecure. PROFTP: apt-get install proftpd (run as standalone) Create an user called "www" to access the www directory: adduser --home /var/www www Allow FTP access to the user "www": jed /etc/proftpd.conf Add the following lines in the user section of the proftpd configuration file:User www Group www DirFakeUser on ftp DirFakeGroup on ftp RequireValidShell off MaxClients 2 DisplayLogin welcome.msg DisplayFirstChdir .message /etc/init.d/proftpd restart VSFTP: # VSFTP allows virtual users, besides real shell users and anonymous users (you can simulate a chroot'ed FTP) apt-get install vsftpd apt-get install libpam-pwdfile Create a password file for virtual users: # Note: htpasswd only allows passwords up to 8 characters long htpasswd -bc /etc/vsftpd_passwd USERNAME PASSWORD Additional users can be created without the -c parameter. The same statement can be used to change the password for an existing user: htpasswd -b /etc/vsftpd_passwd USERNAME PASSWORD If you want to enter the password manually, omit the -b parameter: htpasswd /etc/vsftpd_passwd USERNAME Use this command to remove a virtual user: htpasswd -D /etc/vsftpd_passwd USERNAME You'll need to create a directory for each virtual user (unless you configure vsftp otherwise): First, create the main directory for all virtual users: mkdir /var/www/ftp Then create a directory for each virtual user with proper read/write permissions: mkdir /var/www/ftp/USERNAME chmod 755 /var/www/ftp/USERNAME chown ftp /var/www/ftp/USERNAME Configure PAM (Pluggable Authentication Modules aka X/Open Single Sign-on) for vsftp: jed /etc/pam.d/vsftpd # Log in using htpasswd password file auth required pam_pwdfile.so pwdfile /etc/vsftpd_passwd account required pam_permit.so jed /etc/vsftpd.conf # Config file /etc/vsftpd.conf listen=YES #listen_ipv6=YES anonymous_enable=NO local_enable=YES write_enable=YES local_umask=022 anon_upload_enable=NO anon_mkdir_write_enable=NO dirmessage_enable=YES xferlog_enable=YES connect_from_port_20=YES #chown_uploads=YES #chown_username=ftp xferlog_file=/var/log/vsftpd.log #xferlog_std_format=YES idle_session_timeout=600 data_connection_timeout=120 #nopriv_user=ftp #async_abor_enable=YES #ascii_upload_enable=YES #ascii_download_enable=YES ftpd_banner=Welcome #deny_email_enable=YES #banned_email_file=/etc/vsftpd.banned_emails chroot_local_user=YES #chroot_list_file=/etc/vsftpd.chroot_list #ls_recurse_enable=YES secure_chroot_dir=/var/run/vsftpd pam_service_name=vsftpd rsa_cert_file=/etc/ssl/certs/vsftpd.pem guest_enable=YES virtual_use_local_privs=YES #user_config_dir=/etc/vsftpd_user_conf #guest_username=ftp user_sub_token=$USER local_root=/var/www/ftp/$USER hide_ids=YES #cmds_allowed=PASV,BYE,LIST,ABOR,CWD,NLST,PORT,PWD,QUIT,RETR,SIZE,TYPE # Note that the option user_config_dir lets you create custom vsftp configuration files for each virtual users # You can create a config file with the name of the virtual user within the specified path jed /etc/vsftpd_user_conf/USERNAME # VSFTP extra config for user USERNAME local_root=/var/www/example.com/www/USERNAME Restart VSFTP to make it work with the new users or configuration: /etc/init.d/vsftpd restart ----- Samba: apt-get install samba smbfs smbclient samba-doc mkdir /public mkdir /public/shared chmod -v 0777 /public chmod -v 0777 /public/shared jed /etc/samba/smb.conf --- # samba configuration file [global] allow hosts = 192.168.0.0/255.255.255.0 workgroup = network server string = %h server (Samba %v) log file = /var/log/samba/log.%m max log size = 1024 ; security = user encrypt passwords = true passdb backend = tdbsam guest guest account = nobody invaluid users = root preserve case = yes short preserve case = yes ; include /home/samba/etc/smb.conf.%m socket options = TCP_NODELAY [homes] comment = Home directories browseable = no writable = no create mask = 0700 directory mask = 0700 [shared] comment = Public shared directory path = /public/shared public = yes browseable = yes ; read only = no writable = yes ; printable = no create mask = 0766 directory mask = 0766 guest ok = yes ; valid users = user_name --- testparm /etc/samba/smb.conf /etc/init.d/samba restart smbpasswd -a nobody New SMB password: Retype new SMB password: log file (default): /var/log/samba/log.%m smbtree # find Windows machines ----- UFW (Uncomplicated Firewall): apt-get install ufw ufw version # Show UFW version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # Enable support for IPv6 jed /etc/default/ufw IPV6=yes # Restart (disable and enable) ufw to make changes to the configuration into effect ufw app list # List of available apps on ufw ufw default deny incoming ufw default allow incoming ufw default deny incoming ufw default allow outgoing ufw allow ssh ufw allow 22/tcp ufw allow http ufw allow 80/tcp ufw allow https ufw allow 443/tcp ufw allow 5000:6000/tcp ufw allow 5000:6000/udp ufw allow from 192.0.2.100 ufw deny from 192.0.2.100 ufw allow from 192.0.2.110 to any port 22 ufw allow from 192.0.2.100/24 ufw allow from 192.0.2.100/24 to any port 22 ufw delete deny from 192.0.2.100 ufw delete allow from 192.0.2.100 ufw status numbered ufw delete NUMBER # delete the rule with the given number ufw reset # reset all rules ufw disable # stop ufw Firewall stopped and disabled on system startup ufw enable # start ufw Command may disrupt existing ssh connections. Proceed with operation (y|n)? y Firewall is active and enabled on system startup echo 'y' | ufw enable # start ufw with automatic confirmation Command may disrupt existing ssh connections. Proceed with operation (y|n)? Firewall is active and enabled on system startup ufw --force enable # start ufw without confirmation prompt Firewall is active and enabled on system startup ufw disable; ufw enable; # restart ufw ufw disable; ufw --force enable; # force restart ufw ufw status Status: inactive ufw status Status: active To Action From -- ------ ---- Anywhere DENY 192.0.2.100 22 ALLOW 192.0.2.110 22/tcp ALLOW Anywhere 80/tcp ALLOW Anywhere 443/tcp ALLOW Anywhere 22/tcp (v6) ALLOW Anywhere (v6) 80/tcp (v6) ALLOW Anywhere (v6) 443/tcp (v6) ALLOW Anywhere (v6) ufw status verbose Status: active Logging: on (low) Default: allow (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- Anywhere DENY IN 192.0.2.100 22 ALLOW IN 192.0.2.110 22/tcp ALLOW IN Anywhere 80/tcp ALLOW IN Anywhere 443/tcp ALLOW IN Anywhere 22/tcp (v6) ALLOW IN Anywhere (v6) 80/tcp (v6) ALLOW IN Anywhere (v6) 443/tcp (v6) ALLOW IN Anywhere (v6) ufw version ; ufw status verbose | head --lines=4 ; ufw status numbered | grep -v 'Status' ; Status: active Logging: on (low) Default: allow (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- [ 1] Anywhere DENY IN 192.0.2.100 [ 2] 22 ALLOW IN 192.0.2.110 [ 3] 22/tcp ALLOW IN Anywhere [ 4] 80/tcp ALLOW IN Anywhere [ 5] 443/tcp ALLOW IN Anywhere [ 6] 22/tcp (v6) ALLOW IN Anywhere (v6) [ 7] 80/tcp (v6) ALLOW IN Anywhere (v6) [ 8] 443/tcp (v6) ALLOW IN Anywhere (v6) ----- Scripting / Linux Shell scripts: # First line headers Bourne Shell (sh) #!/bin/sh Bourne Again Shell (bash) #!/bin/bash C-Shell (csh) #!/bin/csh Korn Shell (ksh) #!/bin/ksh Perl #!/usr/bin/perl PHP #!/usr/bin/php Python #!/usr/bin/python Python3 #!/usr/bin/env python3 Turbo Shell (tcsh) #!/usr/bin/tcsh TCL/Tk #!/usr/bin/wish -f True #!/bin/true # Do not execute the script and exit with a Success (0) exit status False #!/bin/false # Do not execute the script and exit with a Failure (1) exit status Unless you expressely need to code for a specific shell, your choice of header should be #!/bin/bash (that is, Bourne Again Shell (bash)) Set the executable flag: chmod +x FILE # declares the file as executable # Parameters if [ "$#" -eq 0 ] || [ -z "$1" ]; then echo "Missing parameters"; fi # Variables varn=1 # number (floating or integer) vars="hi" # string of characters echo $varn echo $vars # Arrays arr=(10 4 5 20 8) echo "Array Length: ${#arr[@]}"; echo -n "Highest number in array: "; printf "%d\n" "${arr[@]}" | sort -nr | head --lines=1 echo -n "Lowest number in array: "; printf "%d\n" "${arr[@]}" | sort -nr | tail --lines=1 msga[0]="Hello"; msga[1]="World"; msgb=("Hello" "World") msgc=([3]="Hello" [5]="World") echo "${msga[0]} ${msga[1]}" echo "${msgb[0]} ${msgb[1]}" echo "${msgc[3]} ${msgc[5]}" printf '%s\n' "${arr[@]}" # print the whole content of the array arr, separated by new lines printf '%s, ' "${arr[@]}" # print the whole content of the array arr, separated by commas and spaces (note that there will be a comma and a space after the last element in the array too) printf '%s, ' "${arr[@]}" | rev | cut -c 3- | rev # print the whole content of the array arr, separated by commas and spaces and removing the trailing comma and space # Functions # Declare a function with function function helloworld { echo "Hello World"; } # Declare a function with round brackets hellow () { echo "Hello World"; } # Declare a function that accepts parameters helloany () { echo "Hello $1"; } # Return a value from a function function sum() { local r=0; if [ $# -ge 1 ]; then for i in "$@"; do local r=$(( $r+$i )); done echo -n $r; return 0; else echo -n $r; return 1; fi } # Call functions helloworld hellow helloany "everybody" echo $( sum 3 4 2 ); echo $( sum 5 5 ); echo $( sum 2 0 ); echo $( sum 6 ); echo $( sum ); # Branching # if ... then ... elif ... else ... fi # Check if a variable is empty if [ -z "$v1" ]; then echo "The variable is empty"; fi if [ -n "$v1" ]; then echo "The variable is NOT empty"; fi if [ ! -z "$v1" ]; then echo "The variable is NOT empty"; fi if [ ! -n "$v1" ]; then echo "The variable is empty"; fi # Compare numbers $v1=5; $v2=8; if [ $v1 -eq $v2 ]; then echo "$v1 is equal to $v2"; fi if [ $v1 -ne $v2 ]; then echo "$v1 is not equal to $v2"; fi if [ $v1 -gt $v2 ]; then echo "$v1 is greater than $v2"; fi if [ $v1 -ge $v2 ]; then echo "$v1 is greater than or equal to $v2"; fi if [ $v1 -lt $v2 ]; then echo "$v1 is less than $v2"; fi if [ $v1 -le $v2 ]; then echo "$v1 is less than or equal to $v2"; fi if (( "$v1" == "$v2" )); then echo "$v1 is equal to $v2"; fi if (( "$v1" != "$v2" )); then echo "$v1 is not equal to $v2"; fi if (( "$v1" > "$v2" )); then echo "$v1 is greater than $v2"; fi if (( "$v1" >= "$v2" )); then echo "$v1 is greater than or equal to $v2"; fi # >= and <= don't work with [[ ]] if (( "$v1" < "$v2" )); then echo "$v1 is less than $v2"; fi if (( "$v1" <= "$v2" )); then echo "$v1 is less than or equal to $v2"; fi # >= and <= don't work with [[ ]] if (( "$v1" == "$v2" )); then echo "$v1 is equal to $v2"; elif (( "$v1" < "$v2" )); then echo "$v1 is less than $v2"; else echo "$v1 is greater than $v2"; fi # Compare strings $v1="hello"; $v2="world"; if [[ "$v1" == "$v2" ]]; then echo "$v1 is equal to $v2"; fi if [[ "$v1" != "$v2" ]]; then echo "$v1 is not equal to $v2"; fi if [[ "$v1" == "$v2"* ]]; then echo "$v1 begins with $v2"; fi # Wildcards don't work with (( )) if [[ "$v1" == *"$v2" ]]; then echo "$v1 ends with $v2"; fi # Wildcards don't work with (( )) if [[ "$v1" == *"$v2"* ]]; then echo "$v1 contains $v2"; fi # Wildcards don't work with (( )) if [[ "$v1" == *"$v2"* ]] && [[ "$v1" != *"$v2" ]] && [[ "$v1" != "$v2"* ]]; then echo "$v1 contains with $v2, but not at the beginning or at the end."; fi # Wildcards don't work with (( )) if (echo "$v1" | grep -Eq $regex); then echo "$v1 matches $regex"; fi # This is the POSIX way to match Regular Expressions. if [[ "$v1" =~ $regex ]]; then echo "$v1 matches $regex"; fi # This is Bash specific to match Regular Expressions. If you get a "[[: not found" error in your script, remember to add #!/bin/bash at the top of your script. if [[ "$v1" > "$v2" ]]; then echo "$v1 is greater than $v2"; fi if [[ "$v1" < "$v2" ]]; then echo "$v1 is less than $v2"; fi if [[ "$v1" == "$v2" ]]; then echo "$v1 is equal to $v2"; else echo "$v1 is not equal to $v2"; fi # Compare versions v1="3.50.2"; v2="4.2.3"; if [[ "$v1" == "$v2" ]]; then echo "$v1 is equal to $v2"; fi if [[ "$(sort --version-sort <<< "$(printf '%s\n' "$v1" "$v2")" | head --lines=1)" = "$v1" ]]; then echo "$v1 is lower than or equal to $v2"; fi if [[ "$(sort --version-sort <<< "$(printf '%s\n' "$v1" "$v2")" | head --lines=1)" != "$v1" ]]; then echo "$v1 is higher than $v2"; fi if [[ "$v1" == "$v2" ]]; then echo "$v1 is equal to $v2"; elif [[ "$(sort --version-sort <<< "$(printf '%s\n' "$v1" "$v2")" | head --lines=1)" = "$v1" ]]; then echo "$v1 is lower than $v2"; else echo "$v1 is higher than $v2"; fi # File System if [ -e $v1 ]; then echo "$v1 exists as a File or a Directory"; fi if [ -f $v1 ]; then echo "$v1 is a File"; fi if [ -d $v1 ]; then echo "$v1 is a Directory"; fi if [ -L $v1 ]; then echo "$v1 is a SymLink"; fi if [ -s $v1 ]; then echo "$v1 file size is greater than 0"; fi if [ -N $v1 ]; then echo "$v1 was modified after its last read"; fi if [ -O $v1 ]; then echo "$v1 is owned by current user"; fi if [ -G $v1 ]; then echo "$v1 is owned by current user's group"; fi if [ -r $v1 ]; then echo "$v1 has Read permission"; fi if [ -w $v1 ]; then echo "$v1 has Write permission"; fi if [ -x $v1 ]; then echo "$v1 has Execute permission"; fi if [ $v1 -ef $v2 ]; then echo "$v1 and $v2 are hard links to the same file"; fi if [ $v1 -nt $v2 ]; then echo "$v1 is newer than $v2"; fi if [ $v1 -ot $v2 ]; then echo "$v1 is older than $v2"; fi # Check for File or Directory if [[ -d $v ]]; then echo "Directory" elif [[ -f $v ]]; then echo "File" else echo "Not found" fi # case # Config onekeyc=1; # Single Keypress for menu choice, rather than having to hit Enter (1: True, 0: False) echo -n "Enter choice "; if [ $onekeyc -eq 1 ]; then read -r -n 1 -p "" choice # Key press echo ; else read -r -p "" choice # Type choice and hit Enter fi case $choice in "1") echo "1" ;; "2") echo "2" ;; "q"|"Q"|"x"|"X") echo "Q or X, upper or lower case" ;; *) echo "Default" ;; esac # Looping # for for i in 1 2 3 4 5 do echo "N. $i" done echo for i in {1..5} do echo "N. $i" done echo for (( i=1; i<=5; ++i )) do echo "N. $i" done echo for (( i=1; i<=10; i=i+2 )) do echo "N. $i" done echo # Bash v4.0 and greater support step for i in {1..10..2} do echo "N. $i" done echo # Before than Bash v4.0 step could be achieved with the additional seq command for i in $(seq 1 2 10) do echo "N. $i" done echo # List all elements in given list i=0; for el in Hello World !; do i=$(( i + 1 )) echo "Element N. $i: $el" done echo # List all elements in given list in a variable str="Hello World !" i=0; for el in $str; do i=$(( i + 1 )) echo "Element N. $i: $el" done echo # List all elements in given array arr=("Hello" "World" "!") i=0; for el in "${arr[@]}"; do i=$(( i + 1 )) echo "Element N. $i: $el" done # List all files in PATH for file in PATH/* ; do echo $file done for (( ; ; )) do echo "infinite loop (kill the process, or terminate it with CTRL+C)" done # while i=1 while (( $i <= 5 )) do echo "N. $i" (( ++i )) done echo i=0 while (( $i <= 10 )) do i=$(( i + 2 )) if (( i == 4 )); then continue; fi echo "N. $i" if (( i >=8 )); then break; fi done echo while : do echo "infinite loop (kill the process, or terminate it with CTRL+C)" done # until i=1 until (( $i > 5 )) do echo "N. $i" (( ++i )) done echo i=0 until (( $i > 10 )) do i=$(( i + 2 )) if (( i == 4 )); then continue; fi echo "N. $i" if (( i >=8 )); then break; fi done echo until false : do echo "infinite loop (kill the process, or terminate it with CTRL+C)" done ----- Invoke Linux shell commands from PHP: PHP can perform calls to the Linux shell using the shell_exec() function: $output = shell_exec('ls -laF'); echo 'AllowAll '.$output.''; or enclosing the Linux shell command between backticks: $output = `ls -laF`; echo ''.$output.''; PHP also has a system() function that returns the value returned by the Linux shell command and the last line of output of such command: $last_line = system('ls -laF', $returned_value); ----- GUI commands # Commands for the Graphical User Interface xwinifo # Enter this command and then click on any window to get information about it such as size in pixels and color depth. You may also click on the desktop. sudo rm -rf ~/.local/share/Trash/files/* # Empty trashcan ----- Fortune cookies: apt-get install fortunes /usr/games/fortune # return a fortune cookie (fetches a random sentence (% separated lines) from a random file in /usr/share/games/fortunes/ ) /usr/games/fortune PATH/FILE # return a fortune cookie fetching a random sentence (% separated lines) from the given FILE To create your own fortune files: - Create a text file separating each fortune cookie with lines containing only a percent (%) sign. - Create a database file for that text file, with the same file name followed by a .dat extension, and place it in the same directory: strfile PATH/FILENAME PATH/FILENAME.dat ----- ASCII Art: Labyrinth: c=(\/ \\); n=${#c[@]}; clear; printf "%s" "${c[RANDOM%n]"{1..1920}"}" c=(\/ \\); n=${#c[@]}; clear; while :; do printf -- "${c[RANDOM%n]}"; done # Endless labyrinth (break with CTRL+C) apt-get install figlet figlet "Hello" # Type Hello using ASCII Art apt-get install cowsay /usr/games/cowsay "Hello" # Show a cow saying Hello /usr/games/cowthink "Hello" # Show a cow thinking Hello /usr/games/cowsay -f CHARACTER "Hello" # Use alternate CHARACTERs, for example: tux, moose, sheep, milk /usr/games/cowthink -f CHARACTER "Hello" # Use alternate CHARACTERs, for example: tux, moose, sheep, milk ls /usr/share/cowsay/cows # Show available cows (characters) /usr/games/fortune|/usr/games/cowsay # Fortune teller cow (requires fortune cookies to be installed) /usr/games/cowsay -f "$(ls /usr/share/cowsay/cows | sort -R | head -1)" "$(/usr/games/fortune -s)" # Random character tells fortune apt-get install sl sl # show a running steam locomotive in ASCII Art sl -c # show a small steam locomotive sl -F # show a flying steam locomotive sl -a # show people crying for help into a steam locomotive sl -a # allow the animation to be interrupted with CTRL+C apt-get install lolcat echo 'Somewhere over the rainbow' | /usr/games/lolcat # Return the given text as a raibow echo 'Somewhere over the rainbow' | /usr/games/lolcat -a # Return the given text as an animated raibow /usr/games/fortune|/usr/games/cowsay|/usr/games/lolcat # Fortune teller cow in a rainbow (requires fortune cookies and cowsay to be installed) /usr/games/fortune|/usr/games/cowsay|/usr/games/lolcat -a # Fortune teller cow in an animated rainbow (requires fortune cookies and cowsay to be installed) ----- Text editors: Jed apt-get install jed jed FILE.txt # Create or edit FILE.txt jed FILE.txt -g LINE # Open FILE.txt for editing, set cursor position at the given LINE jed FILE.txt -s "TEXT" # Open FILE.txt for editing, set cursor position at the first occurrence of the given TEXT Vim apt-get install vim vim FILE.txt # Create or edit FILE.txt (cursor on the first line) vim + FILE.txt # Edit FILE.txt and place cursor on the last line vim +N FILE.txt # Edit FILE.txt and place cursor on line N # While Jed is an easy and intuitive text editor, it might happen that it's not available in the system on which you are working. # In this case you'll likely have to use vim instead which is powerful in trained hands but much less intuitive. # This is a list of some basic and useful commands Cursor keys / h, l, j, k : Cursor movement (left, right, down, up) b : Jump to the word on the left w : Jump to the word on the right ( : Jump to the sentence on the left ) : Jump to the sentence on the right ^ / 0 : Jump to the beginning of the line $ / gm : Jump to the end of the line - : Jump to the first non blank character of the previous line + : Jump to the first non blank character of the next line { : Jump to the previous paragraph } : Jump to the next paragraph [[ : Jump to the beginning of the file ]] : Jump to the end of the file [LINE_NUMBER]G / [LINE_NUMBER]gg : Jump to the given LINE NUMBER [VALUE]% : Jump to the specified position of the document [COLOUMN]| : Jump to the specified COLOUMN in the current line [LINE]H : Jump to the given LINE visible in the current window, from the top [LINE]L : Jump to the given LINE visible in the current window, from the bottom M : Jump to the line in the middle of the current window zt : Place current line at the top of the window zz : Place current line at the middle of the window zb : Place current line at the bottom of the window i : Enter Insert mode I : Jump at the beginning of the line and enter Insert mode s : Delete the character under the cursor and enter Insert mode R : Enter Replace mode ESC : Exit Edit mode (either Insert or Replace (Overwrite)) Commands in Edit mode (^ means CONTROL): ^w : Delete characters from cursor position to the beginning of the current word ^u : Delete characters from cursor position to the beginning of the current line ^t : Add a Tab at the beginning of the current line (Add indentation) ^r : Remove a Tab from the beginning of the current line (Remove indentation) r : Replace a single character at the cursor position o : Create a new empty line below the current one and enter Insert mode O : Create a new empty line above the current one and enter Insert mode gu[CURSOR_KEY] : Change a single character to lowercase in the direction specified with the cursor key gU[CURSOR_KEY] : Change a single character to uppercase in the direction specified with the cursor key x : Remove the character at the cursor (Delete) X : Remove the character before than the cursor (Backspace) D : Delete until the end of the current line dd : Delete the current line J : Join the current line with the next one adding a space inbetween Y / yy : Copy the current line P / p : Paste the line in the clipboard ga : Show the ASCII value of the character at the cursor position /[TEXT] : Search for the given TEXT n : Repeat last search, forward N : Repeat last search, backwards :s/[SEARCH]/[REPLACE]/ : Search and Replace the first occurrence in the current line only, case sensitive :s/[SEARCH]/[REPLACE]/g : Search and Replace every occurrence in the current line only, case sensitive :%s/[SEARCH]/[REPLACE]/g : Search and Replace every occurrence in the whole file, case sensitive :%s/[SEARCH]/[REPLACE]/gI : Search and Replace every occurrence in the whole file, case sensitive :%s/[SEARCH]/[REPLACE]/gi : Search and Replace every occurrence in the whole file, case insensitive :%s/[SEARCH]/[REPLACE]/gic : Search and Replace every occurrence in the whole file, case insensitive, and ask for confirmation :%s/\s\+$// : Remove trailing spaces in the whole file :%s/^\s\+// : Remove leading spaces in the whole file :set number / :set nu : Display line numbers :set nonumber / :set nonu : Turn off line numbers :set number! / :set nu! : Toggle line numbers :set relativenumber : Display line numbers relative to the cursor line :set norelativenumber : Remove line numbers relative to the cursor line :set relativenumber! : Toggle line numbers relative to the cursor line # If number and relativenumber are both enabled the absolute line number will be shown in the cursor line (instead of 0) :set list : Display special characters :set nolist : Hide special characters :set list! : Toggle special characters :set listchars=eol:$,tab:>-,trail:~,nbsp=! : Configure how to show special characters :help listchars : Display help for listchars (quit with :q) :syntax on : Enable syntax :syntax off : Disable syntax :set syntax=[LANGUAGE] : Display syntax for the specified language (example: html, php, whitespace) :w : Save current file :wq / :x / ZZ : Save current file and quit :q : Quit (it only quits if the current file is saved, otherwise returns an error message) :q! : Quit without saving (doesn't ask for confirmation) ----- Easter eggs: $ apt-get moo (__) (oo) /------\/ / | || * /\---/\ ~~ ~~ ...."Have you mooed today?"... $ apt-get -h [...] This APT has Super Cow Powers. $ aptitude -h [...] This aptitude does not have Super Cow Powers. $ aptitude moo There are no Easter Eggs in this program. $ aptitude -v moo There really are no Easter Eggs in this program. $ aptitude -vv moo Didn't I already tell you that there are no Easter Eggs in this program? $ aptitude -vvv moo Stop it! $ aptitude -vvvv moo Okay, okay, if I give you an Easter Egg, will you go away? $ aptitude -vvvvv moo All right, you win. /----\ -------/ \ / \ / | -----------------/ --------\ ---------------------------------------------- Happy? $ aptitude -vvvvvv moo What is it? It's an elephant being eaten by a snake, of course. # Note: aptitude's easter egg is a reference to "The Little Prince" written in 1943 by Antoine de Saint-Exupéry. -----