Geody Labs

# The Debian Linux Magic Spells File Cheat Sheet
# by Elf Qrin - http://labs.geody.com/
v1.41 r08feb2017 fr--sep2005


# "One man cast a lingering spell of awe and wonder, of magical innocence overcoming evil, of simple courage conquering fear" -- J.R.R. Tolkien's The Lord of the Rings (1978 Ralph Bakshi animated film)
# "A script kiddie is someone who thinks of code as magical incantations and asks only 'what do I need to type to make this happen?'" -- anonymous

----- Notes: - In this file, jed is used as the default editor for it's more intuitive than others. You'll have to install it ( apt-get install jed ). Alternatively, you might want to use vi or vim instead. - Most commands work also on Ubuntu / Kubuntu / Xubuntu / Edubuntu, Mint and Xandros. Some commands work on all Linux distributions. - You may send contributions and suggestions to (see: http://www.geody.com/contacts.php ) ----- You can access the terminal in most Debian based distributions (including Debian itself and Ubuntu) by pressing CTRL+ALT+F1 (actually you can switch between terminals with CTRL+ALT+function keys. With CTRL+ALT+F7 you normally return to the graphic interface in Ubuntu). In Xandros, you have to press CTRL+ALT+T. To access a remote terminal via SSH from Windows use the free and open source software PuTTY. ----- Install a software package using APT (Advanced Package Tool): apt-get install base-config apt-setup Set an APT source (repository) manually: jed /etc/apt/sources.list The normal way of accessing the mirror is by putting these lines in /etc/apt/sources.list : deb http://http.us.debian.org/debian stable main contrib non-free deb-src http://http.us.debian.org/debian stable main contrib non-free deb http://security.debian.org stable/updates main contrib non-free deb-src http://security.debian.org stable/updates main contrib non-free You can use a local repository instead of the US Debian one, for example debian.fastweb.it instead of http.us.debian.org Use unstable instead of stable if you want bleeding edge technology. Not very recommended in production environments. To keep an old version up to date: deb http://archive.debian.org/debian/ DEBIAN_VERSION main non-free contrib deb-src http://archive.debian.org/debian/ DEBIAN_VERSION main non-free contrib deb http://archive.debian.org/debian-security/ DEBIAN_VERSION/updates main non-free contrib deb-src http://archive.debian.org/debian-security/ DEBIAN_VERSION/updates main non-free contrib Update the list of packages: # After adding a new apt-get source, or before installing new packages, enter this command apt-get update Should a "Segmentation fault" error occur when updating the list of packages, you can fix it with these commands: rm -r /var/lib/apt/lists/* rm /var/cache/apt/*.bin # will delete pkgcache.bin and srcpkgcache.bin Upgrade installed packages: apt-get upgrade Upgrade a distribution: # It's recommended after a fresh install; always preceded by an apt-get update and an apt-get upgrade # apt-get upgrade # upgrade already installed packages but don't remove anything or install anything new # apt-get dist-upgrade # upgrade everything removing/adding packages as required to resolve dependencies apt-get dist-upgrade Prevent a package to be upgraded: apt-get install wajig wajig hold PACKAGE # the specified package will no longer be upgraded (for example with an apt-get upgrade) wajig unhold PACKAGE # the package will be upgraded again wajig hold mysql-server-5.0 # a common problem with Kernel 2.4 is that latest releases of MySQL server 5 are not compatible with such kernel version, interfering with the upgrade process Search for a package: apt-cache search TEXT # search for a package containing the specified text or keyword in its name or description List installed packages: dpkg-query -l # List all installed packages dpkg --list # List all installed packages dpkg --get-selections # List all installed packages dpkg -l # List all installed packages with version and description dpkg --get-selections | wc --lines # Count installed packages grep install /var/log/dpkg.log | tail -10 # Last 10 installed packages dpkg --listfiles PACKAGE # List all files "owned" by PACKAGE cat /var/log/dpkg.log | grep "status installed " # Show last installed packages cat /var/log/apt/history.log # Show last apt commands (installed and upgraded packages) Check if a package is installed: dpkg -s PACKAGE dpkg-query -l PACKAGE dpkg-query -W -f='${Status} ${Version}\n' PACKAGE # If the package is installed, show status and version dpkg-query -l "TEXT*" # Show all installed packages starting with TEXT dpkg-query -l "*TEXT" # Show all installed packages ending with TEXT dpkg-query -l "*TEXT*" # Show all installed packages containing TEXT cat /var/log/dpkg.log | grep "status installed PACKAGE" # Search for a specific package among the last installed ones Install a package: apt-get install PACKAGE # get and install a PACKAGE from a repository specified in /etc/apt/sources.list dpkg -i PACKAGE.deb # manually install a package available locally (already downloaded) apt-get install --reinstall PACKAGE # reinstall a PACKAGE Show information about a package and all its dependencies apt-cache show PACKAGE aptitude show PACKAGE # similar to apt-cache show PACKAGE but shows extra information apt-cache showpkg PACKAGE # show dependencies and reverse dependencies Verify if all packages are correctly installed and configured: dpkg --audit # It doesn't return anything, if everything is fine Check for broken dependencies: apt-get check Uninstall a package: apt-get remove PACKAGE aptitude purge PACKAGE # remove PACKAGE and its configuration files Show a random non installed package: aptitude search ~g | while read ; do echo "$RANDOM $REPLY" ; done | sort -n | head -1 | cut -d' ' -f2- Store/Restore Packages: dpkg --get-selections > installedpackages.txt # Store the list of installed packages into the file installedpackages.txt dpkg --set-selections < installedpackages.txt # Restore the list of installed packages from the file installedpackages.txt apt-get install `cat installedpackages.txt` # Restore the list of installed packages from the file installedpackages.txt Clean up packages: apt-get autoremove # remove packages installed as dependency of already uninstalled packages apt-get clean # delete all downloaded packages (that is, all files stored in /var/cache/apt/archives/ and /var/cache/apt/archives/partial/ , except for lock files). If you want to reinstall one of such packages, apt-get will have to download them again. apt-get autoclean # only delete packages which is no longer possible to download (obsolete). If you get one of the following error messages: E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable) E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it? dpkg: status database area is locked by another process or anyway if something goes wrong when upgrading packages, you may fix it this way: rm /var/lib/apt/lists/lock rm /var/lib/dpkg/lock rm /var/cache/apt/archives/lock dpkg --configure -a # to fix interrupted configure (may take a while) apt-get -f install # to fix broken dependencies If you still get error messages there are probably running processes working with installation files, you either have to find and kill them all or reboot before than attempting to recover the upgrade process again. After upgrading the Linux kernel, it's better to perform a update-grub and then reboot the system. Debian Linux (Sarge, Kernel 2.6): workaround for E: This installation run will require temporarily removing the essential package e2fsprogs due to a Conflicts/Pre-Depends loop. This is often bad, but if you really want to do it, activate the APT::Force-LoopBreak option. apt-get remove e2fsprogs apt-get install e2fsprogs sysvinit initscripts upgrade glibc when requested. # note that this is a workaround, it should be fixed with apt-get dist-upgrade Note: when the Debian stable changes, if you don't upgrade your system accordingly, you'll have to modify the /etc/apt/sources.list file to specify you are using an old version of Debian (that is, you are no longer using the "current stable" version of Debian). For example, if you are using Sarge, and Debian Etch is released as stable, and you are not upgrading to Etch, you have to change all references to stable as sarge in your source.list file. Which means, you have to change "deb http://debian.example.com/debian stable main contrib non-free" as "deb http://debian.example.com/debian sarge main contrib non-free". If you don't make this change, when you'll try to install new packages, you'll receive the following message: The following packages will be REMOVED: base-config initrd-tools kernel-image-2.4.27-2-386 This is an example, the kernel image can be different in your case, however you'll not be able to install the package and it may damage your system. In fact, if you procede, you will be warned by this message: You are running a kernel (version 2.4.27-2-386) and attempting to remove the same version. This is a potentially disastrous action. Not only will /boot/vmlinuz-2.4.27-2-386 be removed, making it impossible to boot it, (you will have to take action to change your boot loader to boot a new kernel), it will also remove all modules under the directory /lib/modules/2.4.27-2-386. Just having a copy of the kernel image is not enough, you will have to replace the modules too. I repeat, this is very dangerous. If at all in doubt, answer no. If you know exactly what you are doing, and are prepared to hose your system, then answer Yes. If you get an error like 'Template parse error near `Description-sr@latin.UTF-8', in stanza #X of /PATH/FILE.templates then edit the specified file removing the lines containing the given text (and, if present, the following line with the description) from /PATH/FILE.templates If you get an error with locales, like locale: Cannot set LC_CTYPE to default locale you might be able to fix it with apt-get install locales If you get a message ending with the following request after a power outage: Give root password for maintenance (or type Control-D to continue): enter the root password then check the file system of the faulty device (read the whole message to identify the faulty driver, generally is /dev/hda0 on a single hard drive system or /dev/md0 on a RAID system): fsck -f FAULTY_DEVICE reply yes (y) to all questions. Note that if it asks to connect to lost+found more than once some of your files might be seriously damaged. However you can also use fsck -f -y FAULTY_DEVICE to reply automatically "yes" to all questions. then reboot If you can't perform a simple operation that always used to work (for example, copying a file) make sure the disk is not full (check with df). Set keyboard layout # Enter this command to localize the keyboard for your country apt-get install console-common ----- Install a software package from the source (compile a package source): Packages in form of source code are generally stored into .tar.gz (.tgz) or .tar.bz2 (.tbz) files. You may need to download them from the Internet: wget http://www.example.com/source/package.tgz Once you have the package (from the Internet, a CD, or otherwise) in your work directory, you'll have to decompress it: tar -xjvf package.tbz # extract all files from a TAR+BZIP2 compressed archive or tar -xzvf package.tgz # extract all files from a TAR+GZIP compressed archive it should decompress inside a new directory. Move inside this new directory: cd PACKAGE_SOURCE_DIRECTORY_NAME and execute these commands: ./configure make su ROOT_PASSWORD # (if you're not logged as root) make install The executable file should be created inside /usr/local/bin , and you should be able to invoke it typing its name in the console and then hitting ENTER. To uninstall a package, you have to move inside the directory containing the source (if you kept it): cd PACKAGE_SOURCE_DIRECTORY_NAME and uninstall it: make uninstall If you haven't kept the source, you'll have to locate all files (try locate PACKAGE_NAME) and delete them manually. ----- Change root password without knowing the existing one You need physical access to the system. Add init=/bin/bash to boot parameters mount -o remount,rw # mount file system passwd # set new password. WARNING: sudo passwd will change root password, regardless of the current account. Recover a deleted password file: # Debian makes regular backups of the password file in /var/backups/ cp /var/backups/passwd.bak /etc/passwd chmod 644 /etc/passwd ----- Writing text files for Linux using an editor running on another Operating System: When you edit a system file, a shell script, a fortune cookie text file, or another file meant to be processed by a Linux shell command or application, make sure that the lines are divided UNIX style (LF only, x0A; not CR/LF, x0D0A as in Windows), otherwise the line breaks will not be recognized as such and the whole file will be treated as a single line. ----- Install SSH: apt-get install ssh Get SSH version: ssh -V Start a SSH connection (port 22): ssh HOST Configure SSH: jed /etc/ssh/sshd_config # If your client can't authenticate to your server, it might be sending the password as a cleartext and your server is not accepting it. In this case you have to enable tunnelled clear text passwords setting PasswordAuthentication yes # Make sure SSH server is not using the outdated Protocol 1. This line should be present in the configuration file instead: Protocol 2 # Set SSH timeout (in seconds). A logged user will be logged out after idling past specified time. ClientAliveInterval 600 ClientAliveCountMax 0 /etc/init.d/ssh restart # Restart SSH to make changes take effect. However note that they will apply to next SSH session. Start a Telnet (unencrypted) connection: telnet HOST # start a telnet connection on HOST at default port 23 telnet HOST:PORT # start a telnet connection on HOST at the specified PORT ----- Bash shells: Show BASH version: echo $BASH_VERSION Command Line Navigation: CURSOR LEFT / CURSOR RIGHT # move cursor one character left or right on command line LEFT ALT+[B/F] # move cursor to previous / next word on command line CURSOR UP / CURSOR DOWN # move to previous / next command in history CTRL+R # search for a command in History: hit CTRL+R, type the string you want to search, hit CTRL+R again to find further matches Switch bash shells: LEFT ALT+[F1-F6] Cycle between bash shells: LEFT ALT+[CURSOR LEFT / CURSOR RIGHT] Show current terminal: tty Reset a "corrupted" shell: reset Clear shell terminal window: clear Set Terminal Window Size: echo -e "\e[8;LINES;COLUMNSt" # resize the terminal window to LINES x COLUMNS . For example, tlins=40; tcols=160; echo -e "\e[8;$tlins;${tcols}t"; (or echo -e "\e[8;40;160t"; ) resizes the window to 40 x 160 characters. Get Terminal Window Size: tput lines # return the height in characters (number of lines) of the terminal window tput cols # return the width in characters (number of columns) of the terminal window echo "$(tput cols) x $(tput lines)"; # show terminal window size # The environmental variables $LINES and $COLUMNS contain the values of lines and colums of the terminal window, but their value is not preserved within scripts, so that you'll have to add this line to your script assign them the expected values: COLUMNS="$(tput cols)"; LINES="$(tput lines)"; Open a new Bash shell: bash Exit from a non login shell: exit Duplicate a terminal: apt-get install conspy conspy N # N can range from 1 to 6, then execute another conspy N with the same number on another bash shell Show current command line: cat /proc/self/cmdline # typical output: cat/proc/self/cmdline Change prompt: export PS1="[\t] \u@\h:\w\\\$ " # show time in 24 hours format between square brackets, the logged user name, the host name, the current path, and the root status ("#" if root, "$" otherwise) export PS1="\[\e[1;32m\][\t] \u@\h:\w\\\$ \[\e[0m\] " # same prompt, but colored in light green export PS1="\[\e[1;32m\][\$(date +%a\ %d%b%Y\ %H:%M:%S)] \u@\h:\w\\\$\[\e[0m\] " # full date and time between square brackets, the logged user name, the host name, the current path, and the root status ("#" if root, "$" otherwise), colored in light green. Note that the shell command date is used, invoked with \$() . If $() (without the leading backslash) were used, the date would be generated only the first time and then stored into the variable and wouldn't be updated every time the prompt is shown. Note: if you want to set Windows Command Prompt to match your Linux Shell Prompt you have to set the %prompt% environment variable. For example: set prompt=[$d$s$t]$s%username%@%computername%:$p$+$g$s export PS1="\[\e[1;32m\][\$(date +%Y-%m-%d\ %H:%M:%S\ %a)] \u@\h:\w\\\$\[\e[0m\] " # full date and time (in a sortable format) between square brackets, the logged user name, the host name, the current path, and the root status ("#" if root, "$" otherwise), colored in light green. echo $PS1 # See current prompt Edit the script executed at the shell start-up (user login shell): # Custom Prompt and Aliases can be set here jed ~/.bash_profile Sample bash profile script: (see http://labs.geody.com/systatus/ ) --- # ~/.bash_profile # BASH Shell Start-up # r2017-02-08 fr2016-10-18 # by Valerio Capello - http://labs.geody.com/ - License: GPL v3.0 # Config tshilon="\e[0;33m"; tshilof="\e[0m"; tsalerton="\e[0;31m"; tsalertof="\e[0m"; # Set Terminal Window Size # tlins=40; tcols=160; echo -e "\e[8;$tlins;${tcols}t"; # Get Terminal Window Size COLUMNS="$(tput cols)"; LINES="$(tput lines)"; # Prompt export PS1="\[\e[1;32m\][\$(date +%Y-%m-%d\ %H:%M:%S\ %a)] \u@\h:\w\\\$\[\e[0m\] " # History Date Format export HISTTIMEFORMAT="%F %T " # Aliases alias l="ls -laF --group-directories-first --color=auto" alias d="ls -aF --group-directories-first --color=auto" # Message echo; clear date "+%a %d %b %Y %H:%M:%S %Z (UTC%:z)" echo -n "Hello "; echo -ne "$tshilon"; echo -n "$(whoami)"; echo -ne "$tshilof"; if [ "$SSH_CONNECTION" ]; then echo -n " ("; echo -ne "$tshilon"; echo -n "`echo $SSH_CLIENT | awk '{print $1}'`"; echo -ne "$tshilof)"; fi echo -n ", "; echo -n "welcome to "; echo -ne "$tshilon"; echo -n "$(hostname)"; echo -ne "$tshilof"; echo -n " ("; echo -ne "$tshilon"; echo -n "$(hostname -i)"; echo -ne "$tshilof)"; echo "."; echo -n "Machine ID: "; echo -n "$(cat /etc/machine-id) "; echo -n "Boot ID: "; echo -n "$(cat /proc/sys/kernel/random/boot_id) "; echo -n "Session ID: "; echo "$(cat /proc/self/sessionid)"; echo -n "You are "; if [ -n "$SSH_CONNECTION" ]; then echo -n "connected remotely via SSH"; elif [[ "${DISPLAY%%:0*}" != "" ]]; then echo -n "connected remotely "; echo -ne "$tsalerton"; echo -n "NOT"; echo -ne "$tsalertof"; echo " via SSH (which is Bad)"; else echo -n "connected locally"; fi echo ". Your Terminal Window Size is $COLUMNS x $LINES" if [ $EUID -eq 0 ]; then echo -ne "$tsalerton"; echo -n "You have ROOT superpowers!"; echo -e "$tsalertof"; fi echo # Software version uname -a echo "Bash version: $BASH_VERSION" # Webserver version echo -n "$(/usr/sbin/apache2 -v|head --lines=1) "; echo "$(/usr/sbin/apache2 -v|tail --lines=1)"; php -v|head --lines=1 mysql -V # echo # echo -n "Installed Packages: "; dpkg --get-selections | wc -l; # echo "Last installed packages:"; grep install /var/log/dpkg.log | tail --lines=5; echo # System status # echo -n "Vendor: "; echo "$(cat /sys/class/dmi/id/sys_vendor)"; echo -n "Machine: "; echo "$(cat /sys/class/dmi/id/product_name)"; # echo -n "Machine Type: "; echo "$MACHTYPE"; # echo -n "Board: "; echo "$(cat /sys/class/dmi/id/board_vendor) $(cat /sys/class/dmi/id/board_name)"; # echo -n "BIOS: "; echo "$(cat /sys/class/dmi/id/bios_vendor) $(cat /sys/class/dmi/id/bios_vendor) $(cat /sys/class/dmi/id/bios_version) $(cat /sys/class/dmi/id/bios_date)"; echo -n "CPU: "; echo -n "$(grep 'model name' /proc/cpuinfo | head -1). "; echo -n "Cores: "; grep -c 'processor' /proc/cpuinfo free -mh | xargs | awk '{print "Memory: Size: " $8 " Used: " $9 " Free: " $10}'; df -h | xargs | awk '{print "Disk ("$8"): Size: " $9 " Used: " $11 " Free: " $10}'; # df -P -h | awk '(0+$5 < 90 && 0+$5 > 0) {print "FS: "$1" ("$6") Size: "$2" Used: "$3" ("$5") Free: "$4" ("(100-$5)"%)";}'; echo -ne "$tsalerton"; df -P -h | awk '0+$5 >= 90 {print "FS: "$1" ("$6") Size: "$2" Used: "$3" ("$5") Free: "$4" ("(100-$5)"%)";}'; echo -ne "$tsalertof"; echo; echo -n "Uptime: "; uptime echo # Users echo "Last logged users:"; last -n 5 echo; echo "Currently logged users:"; who echo; echo -n "Current user: "; id echo # Security # Shellshock vulnerability check (reports to root only) if [[ $EUID -eq 0 ]]; then env x='() { :;}; echo Bash vulnerable to Shellshock' bash -c 'echo -n' fi --- You may want to replicate your bash profile script as an executable script (called for example status.sh ) containing information provided in the sections starting from Message and below (you'd better change echo -n "welcome to "; to echo -n "this is ";). Customize the MOTD (Message Of The Day), message shown after logging in, before than starting the shell: jed /etc/motd # edit the MOTD Customize the message shown before than the log in request: jed /etc/issue # edit the welcome message Description of a command: whatis COMMAND Manual of a command: man COMMAND apropos KEYWORD # list all manual pages related to the KEYWORD Return the path of a command: which COMMAND type COMMAND # specifies if COMMAND is a shell builtin (internal) List internal commands: help help COMMAND # show information about an internal command Identify the type of a file: file PATH/FILE file -z PATH/FILE.zip # Attempts to check the file type of files contained inside a compressed file Create an alias: alias l="ls -laF --color=auto" # makes "l" an alias for "ls -laF --color=auto" List all existing aliases: alias Remove an alias: unalias ALIAS Repeat last command: !! Start a comment (ignore following text): # Do nothing, successfully: true Do nothing, unsuccessfully: false Conditional execution: if [ 1 == 1 ]; then echo "true"; fi if [ 1 == 1 ]; then echo "true"; else echo "false"; fi Execute a random command between two: ((RANDOM%2 == 0)) && echo "Yes" || echo "No" if [ $((RANDOM % 2)) == 0 ]; then echo "Yes"; else echo "No"; fi Find the time required to perform a command line: # It may useful to compare commands, or parameters of the same command, and see which one is faster. # The "real" value is the actual elapsed time which is the data you most likely need, while The "user" value shows the time spent by the code in user mode, and the "sys" value shows the time spent by the code in kernel mode. time (ls) # Shows the time needed to list the current directory Process priority: nice -n0 COMMAND # execute a command with default priority (0) nice -n20 COMMAND # execute a command with the lowest priority (20) nice -n-20 COMMAND # execute a command with the highest priority (-20). Only root can assign negative (high) priority renice 10 PID # reassign priority to an existing process List running processes and their PIDs: echo $$ # show the PID of the current shell (or script, if invoked from a script) ps # show all processes with a tty in current shell ps -a # show all processes with a tty in all shells ps -A # show all processes ps -A|grep "STRING" # show all processes that matches STRING ps auxf | sort -nr -k 4 # show processes sorted by memory usage ps auxf | sort -nr -k 4 | head --lines=10 # show processes sorted by memory usage (first 10 only) ps auxf | sort -nr -k 3 # show processes sorted by CPU usage ps auxf | sort -nr -k 3 | head --lines=10 # show processes sorted by CPU usage (first 10 only) Show running processes and CPU load: top # press "q" to quit Show running processes accessing the disk or doing other Input/Output activity: apt-get install iotop iotop --only # press "q" to quit Show jobs running in the background: jobs Show which user launched a process: fuser PID Terminate a process: kill -3 PID # quit a process kill -15 PID # term a process kill -9 PID # kill a process (most effective to quit an unresponsive process) Terminate all processes with a matching name: killall NAME killall -I NAME # ignore cases killall -i NAME # ask for confirmation before to kill /etc/init.d/COMMAND start # start a service, system daemon /etc/init.d/COMMAND stop # stop a service, system daemon /etc/init.d/COMMAND restart # restart a service, system daemon Show information about all loaded module: lsmod Show open files: apt-get install lsof lsof # Show all open files lsof PATH # Show all open files within PATH lsof / | awk '{ if($7 > 1048576) print $7/1048576 "MB" " " $9 " " $1 }' | sort -n -u | tail --lines=10 # show the 10 largest open files with their size in MB and the processes that keep them open Test to check if your Bash shell is vulnerable to Shellshock (you should only see a dot if your system is not vulnerable) env x='() { :;}; echo Bash vulnerable to Shellshock' bash -c 'echo .' ----- Information about the vendor: cat /sys/class/dmi/id/sys_vendor # System Vendor Information about the machine (virtual or physical): cat /sys/class/dmi/id/product_name # Product Name cat /sys/class/dmi/id/product_uuid # Product UUID cat /sys/class/dmi/id/product_serial # Product Serial cat /sys/class/dmi/id/product_version # Product Version (if any) cat /sys/class/dmi/id/chassis_vendor # Chassis Vendor (if there is an enclosure) cat /sys/class/dmi/id/chassis_type # Chassis Type cat /sys/class/dmi/id/chassis_version # Chassis Version (if any) cat /sys/class/dmi/id/chassis_serial # Chassis Serial (if any) cat /sys/class/dmi/id/chassis_asset_tag # Chassis Asset Tag (if any) cat /sys/class/dmi/id/board_vendor # Board Vendor cat /sys/class/dmi/id/board_name # Board Name cat /sys/class/dmi/id/board_version # Board Version (if any) cat /sys/class/dmi/id/board_serial # Board Serial (if any) cat /sys/class/dmi/id/board_asset_tag # Board Asset Tag (if any) cat /sys/class/dmi/id/bios_vendor # BIOS Vendor cat /sys/class/dmi/id/bios_version # BIOS Version cat /sys/class/dmi/id/bios_date # BIOS Date Show Machine's Unique ID: cat /etc/machine-id # show the unique machine ID of the local system that is set during installation. The machine ID is a single newline-terminated, hexadecimal, 32-character, lowercase machine ID string. When decoded from hexadecimal, this corresponds with a 16-byte/128-bit string (from the manual). Also in /var/lib/dbus/machine-id Boot ID: cat /proc/sys/kernel/random/boot_id # show the random ID regenerated at each boot Session ID cat /proc/self/sessionid # show the session ID, which is unique for each login Random UUID: cat /proc/sys/kernel/random/uuid # generate a random UUID, different every time is invoked Show system name: hostname cat /proc/sys/kernel/hostname Show system's IP address: hostname -i Show client's IP address in a SSH connection: echo $SSH_CLIENT | awk '{print $1}' echo $SSH_CONNECTION | awk '{print $1}' # Note that $SSH_CONNECTION (not processed with awk) also contains the server's IP address echo "$(whoami) (`echo $SSH_CLIENT | awk '{print $1}'`) @ $(hostname) ($(hostname -i))"; # client (client IP) @ host (host IP) Change system name as NEW_NAME: hostname NEW_NAME jed /etc/hostname # change it as NEW_NAME jed /etc/hosts # you'd better also add NEW_NAME as an alias hostname for 127.0.0.1 (localhost) Show system information: uname -a uname -r # show only kernel version cat /proc/sys/kernel/osrelease # kernel version cat /etc/issue # Welcome message: contains Linux version (unless otherwise edited by the user) cat /etc/debian_version # Debian version Show the kernel ring buffer (to print out the bootup messages): dmesg Show last 20 lines of the system messages file: tail --lines=20 /var/log/messages By default, on most Debian based distributions, every 20 minutes syslog marks the message file with "localhost -- MARK --" to log that the system is up and running. You can modify this behavior by editing the syslog startup file: jed /etc/init.d/sysklogd Locate the line that defines the variable SYSLOGD (generally, SYSLOGD="" ). If it's missing you can add it at the beginning of the configuration file, where other variables are defined. and use the switch -m to set the delay between marks in minutes, or set it to 0 to disable the notification. Example: SYSLOGD="-m 60" marks the message file every hour, SYSLOGD="-m 0" disable the notification. Restart syslog to make the change effective: /etc/init.d/sysklogd restart Start Debian base system configuration tool: base-config Show CPU info: lscpu cat /proc/cpuinfo grep 'model name' /proc/cpuinfo # return CPU model grep 'cpu cores' /proc/cpuinfo # return number of CPU cores grep -c 'processor' /proc/cpuinfo # count CPU cores grep 'flags' /proc/cpuinfo # return available features of the CPU cat /proc/loadavg # show load average: the first three fields in this file are load average figures giving the number of jobs in the run queue (state R) or waiting for disk I/O (state D) averaged over 1, 5, and 15 minutes. They are the same as the load average numbers given by uptime(1) and other programs. The fourth field consists of two numbers separated by a slash (/). The first of these is the number of currently executing kernel scheduling entities (processes, threads); this will be less than or equal to the number of CPUs. The value after the slash is the number of kernel scheduling entities that currently exist on the system. The fifth field is the PID of the process that was most recently created on the system. Show RAM info: cat /proc/meminfo grep MemTotal /proc/meminfo # return total RAM seen by the system egrep 'Mem|Cache|Swap' /proc/meminfo # return information about RAM, Cache, Swap free -mh | xargs | awk '{print "Used/Free/Total Memory: " $9 " / " $10 " / " $8}' # Return Used, Free, Total Memory free -k # show total, used, free memory in KB free -h # show total, used, free memory in human readable format free -m | xargs | awk '{print "Free/Total Memory: " $17 " / " $8 " MB"}' vmstat -s # show information about memory activity List all PCI buses and devices: lspci Show system temperature (if system supports ACPI): apt-get install acpi acpi -t Localization: dpkg-reconfigure locales # edit locale / add new locales locale # show locale settings locale -a # show available locales locale|cut -d= -f1|xargs locale -kc|less # list fields in locale database locale territory # show the locale territory # echo "This server is configured for `locale territory`" printf "%'d\n" 1234567 # print an integer number grouping thousands with the locale configuration List of all commands entered in the shell (history): jed ~/.bash_history # edit history file manually cat ~/.bash_history # show all commands in history less ~/.bash_history # navigate in history cat ~/.bash_history | tr "\|\;" "\n" | sed -e "s/^ //g" | cut -f 1 -d " " | sort -n | uniq -c | sort -n | tail -n 15 | sort -nr # show the 15 most used commands in history history # show all commands in history together with their offset history -d OFFSET # delete the specified history entry history -c # clear the whole history file !!:p # show last executed command !-1:p # show last executed command !-2:p # show second last executed command !! # repeat (execute again) last executed command !-1 # repeat (execute again) last executed command !-2 # repeat (execute again) second last executed command export HISTTIMEFORMAT="%F %T " # Set history date format (normally by default date and time of executed commands is not preserved in history) echo $HISTTIMEFORMAT # See history date format Edit user creation preferences: jed /etc/adduser.conf Create a new user: # useradd is similar to adduser adduser USER # add a new USER with a home directory in the default path ( /home/USER ) adduser --home DIR USER # add a new USER specifying a non standard path for the home directory adduser --no-create-home USER # add a new USER without a home directory Modify a user information: usermod [OPTIONS] USER Edit user deletion preferences: jed /etc/deluser.conf Remove a user: # userdel is similar to deluser deluser USER # remove the specified user deluser --remove-home USER # remove the specified user and his home directory and mail spool deluser --remove-all-files USER # remove the specified user and all files owned by the user (be careful) Change user contact information: chfn [OPTIONS] USER Change a user's default shell: chsh [OPTIONS] USER Get a user hashed password: getent shadow|grep "USER"|cut -f 2- -d ":"|cut -f 1 -d ":" # Get the hash type, salt and hashed password for given USER getent shadow|grep "$(whoami)"|cut -f 2- -d ":"|cut -f 1 -d ":" # Get the hash type, salt and hashed password for current user getent shadow|grep "$(whoami)"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 4- -d "$" # Get the hashed password for given USER getent shadow|grep "$(whoami)"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 4- -d "$" # Get the hashed password for current user getent shadow|grep "USER"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 2- -d "$"|cut -f 2- -d "$"|cut -f 1 -d "$" # Get the salt for given USER getent shadow|grep "$(whoami)"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 2- -d "$"|cut -f 2- -d "$"|cut -f 1 -d "$" # Get the salt for current user getent shadow|grep "USER"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 2- -d "$"|cut -f 1 -d "$" # Get the hash type for given USER (1: md5, 5: sha-256, 6: sha-512) getent shadow|grep "$(whoami)"|cut -f 2- -d ":"|cut -f 1 -d ":"|cut -f 2- -d "$"|cut -f 1 -d "$" # Get the hash type for current user (1: md5, 5: sha-256, 6: sha-512) Change a user password: passwd USER Set a user password to expire: # note that having frequently changing passwords is usually not a good policy, as it becomes harder for users to remember them, and they generally end up with weak passwords chage -m 3 -M 30 -w 2 USER # USER's password can last up to 30 days, can't be changed earlier than 3 days after last change, and will be warned 2 days before password expiration Create a new user group: # addgroup is a link to adduser ( adduser --group ). There's also a groupadd command. addgroup GROUP Modify a group information: groupmod [OPTIONS] GROUP Remove a group: # There's also a delgroup link to deluser ( deluser --group ) groupdel GROUP Add a user to a group: adduser USER GROUP Execute a command with root privileges (root password will be asked): sudo COMMAND Change current user's password: passwd Show all existing users: getent passwd # format: username:password (usually shadowed, a 'x' is shown instead):user id:group id:real name:home path:console. If you are root you can access directly the file with cat /etc/passwd getent shadow # Show users and their hashed password. If you are root you can access directly the file with cat /etc/shadow Show all logged users: users # list all logged users who # show information about all logged users w # show more information than "who" Show last logged users: last # Show last logins last USER # show last times when USER logged in last -n 10 # show last 10 logged users last -n 5 USER # show last 5 times when USER logged in touch /var/log/lastlog # Create the login log file to enable lastlog lastlog # Show last time each user logged in lastlog -u USER # Show last time the USER logged in lastlog -t 30 # Show only users who logged in during the last 30 days lastlog -b 365 # Show only users who last logged more than 365 days ago Show failed login attempts: To enable faillog: ----- Create the faillog log file: touch /var/log/faillog jed /etc/pam.d/common-auth Add these lines at the top of the file: # Log failed logins to /var/log/faillog auth required pam_tally.so per_user magic_root onerr=fail jed /etc/pam.d/sshd Add the following lines immediately before @include common-auth (generally at the beginning of the file): # Log failed login attempts to /var/log/faillog auth required pam_tally.so per_user onerr=fail jed /etc/ssh/sshd_config Enable PAM, search for UsePAM and set it to yes if not enabled, or add the whole line if it's missing: UsePAM yes Restart SSH: service ssh restart ----- faillog # Show all users who attempted to log in without success faillog -a # Show all failed login attempts including the ones of users who eventually logged in failed -u USER # Show all failed login attempts for the specified USER, even if he eventually logged in faillog -t 30 # Show only failed logins occurred during the last 30 days faillog -l 5 # Lock the account for 5 seconds after each failed attempt faillog -m 5 # Disable the account after 5 failed attempts (0 means that infinite attempts are allowed. You'd better leave the value for root to 0 to prevent a DoS attack) faillog -u USER -r # Reset the counter of failed logins for the given user, enabling his account again if it was locked because of too many failed attempts (as specified in faillog -m N) faillog -r # Reset counters of failed logins for all users cat /var/log/faillog # Show the actual failed attempts log file grep "authentication failure" /var/log/messages # extract failed login attempts from the messages file Show current user's name: whoami Show information about a user: finger USER Show groups to which current user belongs: groups Show user and group IDs: id # show user and group IDs for the current user id USER # show user and group IDs for the specified USER Send a message to a logged user (to the output console of his terminal): # Check the user device with a w or a who command first then redirect the output of an echo command to such device echo -ne "Hello\n">/dev/pts/1 # Send "Hello" to the user logged with pts/1 # Send a message to a logged user (to the output console of his terminal) who's using a specific process (identified by its PID): echo -ne "Hello\n" > /proc/PID/fd/0 Send keystrokes to a terminal: # (use ps -A|grep "PROCESS_NAME" to find in which terminal is running a specific process, in case you need to send keystroke to a process waiting for an input) perl -e '$TIOCSTI = 0x5412; $tty = "/dev/pts/TERMINAL"; $char = "\n"; open($fh, ">", $tty); ioctl($fh, $TIOCSTI, $char);'; # Send a single keystroke (ENTER in this example) to the given TERMINAL perl -e '$TIOCSTI = 0x5412; $tty = "/dev/pts/TERMINAL"; $str = "TEXT_STRING\n"; for my $i (1..length($str)) { open($fh, ">", $tty); $char=substr($str, $i-1, 1); ioctl($fh, $TIOCSTI, $char); }'; # Send a TEXT_STRING to the given TERMINAL Send a message to all logged users: wall PATH/FILE # show the content of FILE to all logged users (max 20 lines) wall # use standard input (normally the keyboard) to show a message to all logged users. Message must be terminated with an EOF (End Of File) character (usually CTRL+D) Execute a command as another user (impersonated user password will be requested): su USER COMMAND Start a console as another user (impersonated user password will be requested): su USER su # if no user is specified, then root is assumed by default ----- Search for a file: find PATH -name FILE_MASK # find all files with given mask inside PATH and all subdirectories. Example: find / -name *.php # find any PHP file in the server whereis FILENAME # return the paths of files with matching FILENAME locate NAME # return all paths and files with a matching NAME string grep -r "STRING" PATH # Search for all files containing the given STRING, within the PATH and its subdirectories (-r) grep -r -l "STRING" PATH # Search for all files containing the given STRING, within the PATH and its subdirectories (-r) and return just matching files name and not each occurrence of the string within every file (-l) find PATH | xargs grep "STRING" -sl # Search for all files containing the given STRING, within the PATH and its subdirectories, returns only filenames with path Get information about a file: stat PATH/FILE basename PATH/FILE # return the file name of the specified file dirname PATH/FILE # return the path (from root) of the specified file s1='/path/to/file'; echo ${s1#$(dirname "$(dirname "$s1")")/}; # return the last directory and the file name of a given path/file s1='/path/to/file'; echo ${s1#$(dirname "$s1")/}; # return the file name of a given path/file find PATH/DIRECTORY -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort -u # return all different file extensions in given DIRECTORY Copy a file or a directory: cp -rp SOURCE DESTINATION # copy a file or a directory tree preserving owners and properties mkdir -p PATH && cp -rp SOURCE $_ # copy a file or a directory tree (preserving owners and properties) and create destination path if it doesn't exist Copy a file over a network: apt-get install netcat # On the Destination computer (listener). Make sure the PORT you are listening isn't blocked. Timeout (-w) is set to 3 seconds: nc -vvn -l -p PORT -w 3 > DEST_PATH/FILE # On the Source computer (sender). Make sure the PORT you are sending to isn't blocked. It will quit 2 seconds after receiving an EOF. If -q is not specified, then netcat must be killed manually: nc -vvn DESTINATION_IP PORT -q 2 < SOURCE_PATH/FILE Copy a file securely (through SSH) over a network (it will ask for passwords if needed for authentication on remote hosts): # Use blowfish encryption (-c blowfish) instead of the default TripleDES (3DES) because it's more secure and faster. scp -c blowfish SOURCEPATH/NAME DESTUSER@DESTHOST:DESTPATH/NAME # copies a file from the local system to a remote host scp -c blowfish SOURCEUSER@SOURCEHOST:DSOURCEPATH/NAME DESTPATH/NAME # copies a file from a remote host to the local system scp -c blowfish SOURCEUSER@SOURCEHOST:DSOURCEPATH/NAME DESTUSER@DESTHOST:DESTPATH/NAME # copies a file from a remote host to another remote host scp -c blowfish -r SOURCEUSER@SOURCEHOST:DSOURCEPATH DESTUSER@DESTHOST:DESTPATH/ # copies a directory tree from a remote host to another remote host Move a file or a directory tree (or rename): mv SOURCEPATH/NAME DESTPATH/NAME for a in *; do mv -v "$a" "${a//STRING_SEARCH/STRING_REPLACE}"; done # Batch rename all files in the current directory for a in *; do mv -v "$a" "${a}_`date --iso-8601`"; done # Append date as YYYY-MM-DD to all file names in the current directory (note that it will be appended after any extension) find PATH -type f -name "*" -exec mv "{}" "{}".txt \; # Add an extension (.txt in this example) to all files in given PATH find PATH -type f -not -name "*.*" -exec mv "{}" "{}".txt \; # Add an extension (.txt in this example) to all files without an extension in given PATH for i in `ls [PATH] -1`; do mv $i "${i,,}" ; done # rename all files in the given PATH to lowercase for i in `ls [PATH] -1`; do mv $i "${i^^}" ; done # rename all files in the given PATH to uppercase Synchronize two directories: apt-get install rsync rsync -a -v /SOURCE_PATH/* /DEST_PATH/ # copy all files from /SOURCE_PATH that doesn't exist in /DEST_PATH rsync -a --delete -v /SOURCE_PATH/* /DEST_PATH/ # copy all files from /SOURCE_PATH that doesn't exist in /DEST_PATH, and delete all files in /DEST_PATH that doesn't (no longer) exist in /SOURCE_PATH rsync -a --delete -v rsync://www.example.com/dir/ /DEST_PATH # synchronize from an external rsync directory Create a link to a file: ln -s PATH/ACTUAL_FILE PATH/LINK_TO_CREATE Create an empty file: touch PATH/NAME # Create an empty file. If the given file already exists, it updates its access date Create a unique temporary file or directory with a pseudo random name: mktemp "tmp_XXXXXX" # Create an empty file with a random name generated from the given mask, for example tmp_92pIKB or tmp_Z1Pdz1 mktemp -d "tmp_XXXXXX" # Create an empty directory with a random name generated from the given mask, for example tmp_olD9rA or tmp_5Fw0JI Sample usage of mktemp (from mktemp man): TMPFILE=`mktemp /tmp/example.XXXXXX` || exit 1 echo "program output" >> $TMPFILE Split a file in chunks: split -b=BYTES PATH/FILE PREFIX # split FILE in chunks of BYTES bytes, using PREFIX to name generated chunks split -l LINES TEXT_FILE PREFIX # split TEXT_FILE in chunks of LINES lines, using PREFIX to name generated chunks Rebuild a splitted file (concatenate files): cat PREFIX* > PATH/FILE sed -e '$a\' test?.txt > testx1b.txt # concatenate text files appending an empty line at the end of every line if there isn't already one (for r in PREFIX*;do sed s/\$/\ $r/ < "$r";done) > PATH/FILE # concatenate text files appending the file name after each line cat $(ls PREFIX* -t) > PATH/FILE # concatenate files sorted by date and time (newer first, older last) cat $(ls PREFIX* -tr) > PATH/FILE # concatenate files sorted by date and time (older first, newer last) Type a file: cat PATH/FILE cat -n PATH/FILE # show a text file with line numbers cat PATH/FILE | tr [:upper:] [:lower:] # show a text all in lower case cat PATH/FILE | tr [:lower:] [:upper:] # show a text all in Upper case tac PATH/FILE # show a text from the last line to the first (thus listing lines backwards), useful to ordering elements in a last-in, first-out (LIFO) way fold -w COLOUMN PATH/FILE # wrap a text file, cutting lines at the given coloumn. Useful to format data as input for a program fmt -u -w80 file.txt # format a text file output: 1 space between words, 2 spaces between sentences, 80 coloumns nl PATH/FILE # show line numbers for every line (empty lines are not counted by default) nl -ba PATH/FILE # show line numbers for every line and count all lines, including empty ones nl -s: PATH/FILE # show line numbers for every line and separate line numbers from line text using the specified separator (":" in this case) pr -l 50 test.txt # show a text as pages, each page is 50 lines long (minimum 11, default is 66) pr -l 50 --column 2 test.txt # show a text as pages, each page is 50 lines long and divided into 2 columns more PATH/FILE # type a file page by page less PATH/FILE # show a file page by page, and let scroll through pages. It shows escaped characters as plain text (get it with apt-get install less) less -F PATH/FILE # show a file page by page, and let scroll through pages (get it with apt-get install less). If the file output fits a single screen it quits without having to press "q" grep "TEXT" PATH/FILE # show only lines containing "TEXT" grep -v "TEXT" PATH/FILE # show only lines NOT containing "TEXT" grep -i "TEXT" PATH/FILE # show only lines containing "TEXT" (case insensitive) grep -vi "TEXT" PATH/FILE # show only lines NOT containing "TEXT" (case insensitive) grep -B 1 -A 1 "TEXT" PATH/FILE # show lines containing "TEXT" and show 1 line Before and 1 line After each match grep -B 2 -A 2 -n "TEXT" PATH/FILE # show lines containing "TEXT" and show 2 lines Before and 2 lines After each match, together with line numbers head PATH/FILE # show the head (first 10 lines) of FILE head --lines=20 PATH/FILE # show the first 20 lines of FILE head --bytes=1024 PATH/FILE # show the first 1024 bytes of FILE head --lines=-50 PATH/FILE # show all the FILE but the last 20 lines head --bytes=-2048 PATH/FILE # show all the FILE but the first 2048 bytes tail PATH/FILE # show the tail (last 10 lines) of FILE tail --lines=20 -F PATH/FILE # show the last 20 lines of FILE, and new lines as the file grows. If the files doesn't exist yet, it waits for it (useful for log files) tail --lines=+20 -F PATH/FILE # show the last lines of FILE starting from line 20 tail --bytes=1024 PATH/FILE # show the last 1024 bytes of FILE tail -f PATH/FILE # keep showing the last line of FILE whenever there are added new ones (useful for log files) until stopped tail -f PATH/FILE | sed '/^ENDSTRING$/ q' # keep showing the last line of FILE whenever there are added new ones (useful for log files) until a given string appears sed -ne "3p" PATH/FILE # show line number 3 of FILE sed -ne "2,4p" PATH/FILE # show lines 2 to 4 of FILE sed -ne "1,5p" PATH/FILE # show lines 1 (beginning) to 5 of FILE sed -ne "5p,$" PATH/FILE # show lines 5 to the end of FILE sed -n -e '3,5p' -e '9,15p' PATH/FILE # show lines 3 to 5 and 9 to 15 of FILE sed -e "2,30d" PATH/FILE # show all lines of FILE excepts for the ones from 2 to 30 included Hexadecimal dump of a file: od -Ax -tx1z -v PATH/FILE # hex dump of FILE od -Ax -tx1z -v PATH/FILE | less # navigate the hex dump of FILE Extrapolate strings of text from binary files: strings -a -n 4 PATH/FILE # extrapolate strings of text from binary files (a string of text is considered at least 4 bytes long, you can change this value with the option -n) (get it with apt-get install binutils ) strings -a -t x PATH/FILE # extrapolate strings of text from binary files and return the offset of the string (-t x specifies the offset as hex, -t d decimal, -t o octal) Count lines in text files: wc PATH/FILE # line/word/byte count (handy options: --lines or -l print only the newline counts, --words print only the word counts, --bytes print only the byte counts, --chars print the character counts) wc -L PATH/FILE # print the lenght of the longest line of FILE grep "STRING" PATH/FILE|wc --lines # count how many lines in FILE contains STRING wc -l PATH/*.txt # count the lines of all files with .txt extension in a directory find . -name 'PATH/*.txt' | xargs wc -l # count the lines of all files with .txt extension in a directory and its subdirectories Sort a text file: sort PATH/FILE.txt # sort FILE.txt alphabetically, case sensitive sort -i PATH/FILE.txt # sort FILE.txt alphabetically, cosindering only printable characters sort -d PATH/FILE.txt # sort FILE.txt alphabetically, cosindering only dictionary characters (alphanumeric characters and blanks) sort -f PATH/FILE.txt # sort FILE.txt alphabetically, case insensitive sort -fs PATH/FILE.txt # sort FILE.txt alphabetically, case insensitive and stable (sort according which case comes first for each letter) sort -b PATH/FILE.txt # sort FILE.txt ignoring leading blank spaces sort -r PATH/FILE.txt # sort FILE.txt in reverse alphabetical order sort -n PATH/FILE.txt # sort numbers in FILE.txt sort -h PATH/FILE.txt # sort human readable numbers (like 5K, 2G) in FILE.txt sort -M PATH/FILE.txt # sort months (like Jan, Feb, Mar) in FILE.txt sort -R PATH/FILE.txt # sort lines randomly in FILE.txt sort -u PATH/FILE.txt # sort lines in FILE.txt and keep only unique lines sort -k 2 PATH/FILE.txt # sort lines in FILE.txt using characters from column 2 (any blank character separates columns) sort -t : -k 2 PATH/FILE.txt # sort lines in FILE.txt using characters from column 2 using ":" as a separator for columns sort PATH/FILE.txt -o "PATH/DEST_FILE.txt" # sort FILE.txt and save the output in DEST_FILE.txt More file operations: # All these commands should work also with "awk", the older version of "nawk" nawk '1; { print "" }' SOURCE>DESTINATION # Add an empty line after each line nawk 'NF { print $0 "\n" }' SOURCE>DESTINATION # Add an empty line after each line, except when an empty line is already present nawk '{ print FNR ".\t" $0 }' SOURCE>DESTINATION # Add line numbers followed by dot and tab before than each line nawk '{sub(/^[ \t]+/, "")};1' SOURCE>DESTINATION # Remove leading spaces from each line nawk '{sub(/[ \t]+$/, "")};1' f1.txt SOURCE>DESTINATION # Remove trailing spaces from each line nawk '{gsub(/^[ \t]+|[ \t]+$/,"")};1' SOURCE>DESTINATION # Remove leading and trailing spaces from each line nawk '{sub(/A/,"B")}; 1' SOURCE>DESTINATION # Replace the first occurrence of A with B in each line nawk '{gsub(/A/,"B")}; 1' SOURCE>DESTINATION # Replace every occurrence of A with B in each line nawk '/X/{gsub(/A/, "B")}; 1' SOURCE>DESTINATION # Replace every occurrence of A with B in each line that contains X nawk '!/X/{gsub(/A/, "B")}; 1' SOURCE>DESTINATION # Replace every occurrence of A with B in each line that does not contain X nawk 'BEGIN{while (a++<100) s=s "."; print s}' >DESTINATION # Create a file containing 100 dots (.) nawk 'NR==2 {print;exit}' SOURCE # Print line number 2 from SOURCE file nawk 'NR==2,NR==4' SOURCE # Print line numbers 2 to 4 (inclusive) from SOURCE file nawk 'END{print NR}' SOURCE # count lines (emulates "wc -l") Return the average lenght in lines for all text files in the tree: find . -name "*.txt" -exec wc -l {} \; | nawk 'BEGIN {x=0;y=0} {x+=1; y+=$1} END {print y/x}' Count the occurrences of all the words in a text file: tr -c a-zA-Z '\n' < PATH/FILE.txt | sed '/^$/d' | sort -n | uniq -ci | sort -nr # case insensitive tr -c a-zA-Z '\n' < PATH/FILE.txt | sed '/^$/d' | sort -n | uniq -c | sort -nr # case sensitive Count the occurrences of a string within another string: needle=","; haystack="one, two, three, four, five"; occurrences=$(grep -o "$needle" <<< "$haystack" | wc -l); echo $occurrences Truncate a string: # The marker (delimiter) to set the point where to truncate a string is ":" in the following examples cut -d ":" -f1 <<< "this:is:an:example" # Truncate a string at the first occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "this". cut -d ":" -f2- <<< "this:is:an:example" # Truncate a string after the first occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "is:an:example". echo "this:is:an:example" | cut -d ":" -f1 # Truncate a string at the first occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "this". echo "this:is:an:example" | cut -d ":" -f2- # Truncate a string after the first occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "is:an:example". echo "this:is:an:example" | rev | cut -d ":" -f2- | rev # Truncate a string up to the last occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "this:is:an". echo "this:is:an:example" | rev | cut -d ":" -f1 | rev # Truncate a string after the last occurrence of a marker (removing the marker). Example: before: "this:is:an:example", after: "example". Clear screen: clear # You can still scroll back to see the text previously on screen Return a text string: echo "STRING" # print STRING echo -n "STRING" # print STRING without new line echo {A,B,C}{A,B,C} # return all possible combinations of given characters (this example returns AA AB AC BA BB BC CA CB CC ) Colored text: # Note: Colors are not a POSIX (Portable Operating System Interface) feature in the Bash shell, so it might not be shown in some systems echo -e "\e[1;37m WHITE \e[0;37m LIGHT GRAY \e[1;30m GRAY \e[0;30m BLACK \e[0;31m RED \e[1;31m LIGHT RED \e[0;32m GREEN \e[1;32m LIGHT GREEN \e[0;33m YELLOW \e[1;33m LIGHT YELLOW \e[0;34m BLUE \e[1;34m LIGHT BLUE \e[0;35m PURPLE \e[1;35m PINK \e[0;36m CYAN \e[1;36m LIGHT CYAN \e[0m DEFAULT"; Remove colors and other control characters from a text file: cat -v PATH/FILE | sed "s/\^\[\[.\;..m//g" | sed "s/\^\[\[.m//g" | sed "s/\^M$//g" Type a text string (y by default) until interrupted: yes STRING Convert character encoding: apt-get install recode recode iso-8859-15..utf8 < FILE.TXT > file.utf8 # recode FILE.TXT from Latin Western Europe to UTF8 recode utf8..iso-8859-15 < FILE.UTF8 > file.txt # recode FILE.UTF8 from UTF8 to Latin Western Europe recode ../b64 < FILE.TXT > file.b64 # recode FILE.TXT to Base 64 recode /qp.. < FILE.TXT > file.qp # recode FILE.TXT to quoted printable Convert New Line Format: sed -i "s/\r//" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # From \r\n (Windows) to \n (Unix/Linux/Mac) sed -i "s/\n/\r\n/" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # From \n (Unix/Linux/Mac) to \r\n (Windows) Convert tabs to spaces in a text file: expand PATH/FILE # convert only tabs at the beginning of the line expand -a PATH/FILE # convert all tabs, not just the ones at the beginning of the lines Convert spaces to tabs in a text file: unexpand PATH/FILE # convert all tabs, not just the ones at the beginning of the lines unexpand -i PATH/FILE # convert only spaces at the beginning of the lines Search and Replace: sed "s/SOURCE_STRING/REPLACEMENT/g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace all occurrences of a string inside a text file, case sensitive sed "s/SOURCE_STRING/REPLACEMENT/gi" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace all occurrences of a string inside a text file, case insensitive sed "s/[Bb]ig/small/g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace the word Big or big to small sed "10,20 s/SOURCE_STRING/REPLACEMENT/gi" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace all occurrences of a string inside a text file, case insensitive, and process only lines 10 to 20 included sed "/MATCH_STRING/ s/SOURCE_STRING/REPLACEMENT/g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Replace SOURCE_STRING with REPLACEMENT only in lines where MATCH_STRING is present Replace multiple spaces with a single space in a text file: sed "s/ */ /g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE Remove comments from a text file: sed "/ *#/d;" PATH/SOURCE_FILE>PATH/DESTINATION_FILE Remove blank lines from a text file: sed "/^ *$/d" PATH/SOURCE_FILE>PATH/DESTINATION_FILE Remove comments and blank lines from a text file: sed "/ *#/d; /^ *$/d" PATH/SOURCE_FILE>PATH/DESTINATION_FILE Remove trailing spaces and tabs from a text file: sed "s/[ \t]*$//" PATH/SOURCE_FILE>PATH/DESTINATION_FILE Add a text at the end of each line: sed "s/$/\;/g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Append a semicolon (";") at the end of each line Add a text at the beginning of each line: sed "s/^/LINE : /g" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Add "LINE : " at the beginning of each line Add blank lines after every line in a text file: sed "G" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Add 1 blank line after every line in a text file sed "G;G" PATH/SOURCE_FILE>PATH/DESTINATION_FILE # Add 2 blank lines after every line in a text file Append a line of text to a file: echo "line 1" >> PATH/FILE.txt Append multiple lines of text to a file (in scripts): echo "line 1 line 2" >> PATH/FILE.txt Append multiple lines of text to a file (in scripts): cat <<EOT >> PATH/FILE.txt line 1 line 2 EOT Case conversion: echo "Test"|tr '[:lower:]' '[:upper:]' # Return the text in Upper case echo "Test"|tr '[:upper:]' '[:lower:]' # Return the text in lower case cat PATH/SOURCE_FILE|tr '[:lower:]' '[:upper:]'>PATH/DESTINATION_FILE # Convert a text file to Upper case cat PATH/SOURCE_FILE|tr '[:upper:]' '[:lower:]'>PATH/DESTINATION_FILE # Convert a text file to lower case Extract items from a delimited text file: cut -d: -f1 PATH/SOURCE_FILE>PATH/DESTINATION_FILE # choose the first item of a series delimited by ":" Remove consecutive duplicate lines from a text file: uniq PATH/FILE uniq -u PATH/FILE # return only successive duplicated lines Reverse lines of a file rev PATH/TEXTFILE > PATH/REVTEXTFILE # Reverse the lines of a given file(s) Get the checksum of a file: sum PATH/FILE # return the checksum and block count for FILE sha1sum PATH/FILE # return the SHA1 (160-bit) checksum for FILE cksum PATH/FILE # return the CRC checksum and byte count for FILE Compare and show differences between two binary files: cmp -b -l PATH/FILE1 PATH/FILE2 # compare two files and returns the position of the different bytes followed by the value and the corresponding ASCII character for each file cmp -b -l --bytes=10 PATH/FILE1 PATH/FILE2 # limit comparison to the first 10 bytes Compare and show differences between two text files: diff -abB PATH/FILE1 PATH/FILE2 # Report differences between two text files diff -abBi PATH/FILE1 PATH/FILE2 # Report differences between two text files ignoring case differences in their content Compare and show differences between three text files: diff3 -e MYFILE OLDFILE YOURFILE # output unmerged changes from OLDFILE to YOURFILE into MYFILE Truncate a file: cp /dev/null PATH/FILE # empty FILE. If FILE doesn't exist, it will be created. truncate -s 0 PATH/FILE # truncate FILE to 0 bytes (empty FILE). If FILE doesn't exist, it will be created. truncate -s 0 -c PATH/FILE # truncate FILE to 0 bytes (empty FILE). If FILE doesn't exist, it will not be created. truncate -s 100 PATH/FILE # truncate FILE to 100 bytes. If FILE doesn't exist, or it's shorter than the given size, it will be filled by zeroes (0). truncate -r 100 PATH/REFFILE PATH/FILE # truncate FILE to the size of REFFILE. Split or limit lines in a text file: cut -c 80 PATH/FILE.txt # cut every line of FILE.txt to 80 characters (extra characters will be lost) fold -w 80 PATH/FILE.txt # split every line of FILE.txt after 80 columns (extra characters will be wrapped to the next line) fold -w 80 -s PATH/FILE.txt # split every line of FILE.txt after 80 columns, breaking at spaces (full words will be preserved) Delete a file: rm PATH/NAME rm -i PATH/NAME # ask for confirmation before to delete Wipe a file (secure file deletion): # Important note: wiping overwrites a file one or more time before to delete it, to make it unrecoverable. This technique does not work with journaled file systems, like ReiserFS, Reiser4, or ext3 in journaled mode apt-get install wipe # if it doesn't work, try apt-get install secure-delete wipe PATH/FILE # wipe a file (asks for confirmation, write protected files will be spared) wipe -c -f PATH/FILE # wipe a file wipe -c -f -s PATH/FILE # wipe a file, quiet mode (no output returned except for fatal errors like "No such file or directory") wipe -c -f -k PATH/FILE # wipes the content of a file, but keeps the file in place (with a lenght of 0 bytes) wipe -c -f -e PATH/FILE # wipes the file up to its exact size, without wiping extra data in the last used block Undelete a file: apt-get install testdisk # PhotoRec photorec MEDIA # attempt to recover deleted files from given MEDIA (for example /dev/sda2 ). You'll have then to confirm the media, choose partition, select recovery options, specify the file type of the files you want to recover, specify the filesystem type where the files were stored, choose if you want to search for deleted files in the empty space or the whole partition, choose the destination directory (in a separate disk) where you want to save the recovered files. Create a directory: mkdir PATH mkdir -p PATH # Create the whole path (not just the final directory in the path) if it doesn't exist Delete a directory (must be empty): rmdir PATH find PATH -depth -type d -empty -exec rmdir {} \; # remove empty subdirectories within given PATH (recursive) Delete a directory tree (recursive): rm -rf PATH Delete a directory with a large number (thousands and more) of files: mkdir /empty/ rsync -a --delete /empty/ /PATH/dir/ # Note: if you haven't just created it, make sure the /empty/ directory is actually empty rmdir /PATH/dir/ Wipe a directory tree (recursive secure file deletion): apt-get install wipe wipe -c -r PATH # wipe a directory tree (asks for confirmation) wipe -c -f -r PATH # wipe a directory tree wipe -c -f -r -s PATH # wipe a directory tree, quiet mode (no output returned except for fatal errors like "No such file or directory") Show current directory: pwd Change directory: cd PATH pushd PATH # change directory and remember the current directory in the dirs stack popd # return to the previous directory in the dirs stack dirs # (or dirs -l ) list directories in the dirs stack keeping the output in a single line dirs -p # list directories in the dirs stack showing the output in multiple lines dirs -v # list directories in the dirs stack showing the output in multiple lines and line numbers dirs -c # clear the dirs stack Show the content of a directory: ls [PATH] ls -laF --color=auto [PATH] # extended information and colored output ls -laFXB [PATH] # sort by extension ls -laFSr [PATH] # sort by size, small to large ls -laFS [PATH] # sort by size, large to small ls -laFtr [PATH] # sort by size, old to recent ls -laFt [PATH] # sort by date, recent to old ls -laFtur [PATH] # sort by access date, old to recent ls -laFtu [PATH] # sort by access date, recent to old ls -A # show hiddem files ls --group-directories-first # show directories first ls -d */. # list only directories within current directory ls -d PATH/*/. # list only directories within given PATH ls -R [PATH] | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/' # show directory tree (add | less at the end of the line to navigate the tree up and down) find PATH -type f -mtime -30 -print0 | xargs -0 ls -lt # list files modified within last 30 days find PATH -depth -type d -empty -exec echo {} \; # list empty subdirectories within given PATH (recursive) Compare and show differences between two directories: diff PATH/DIR1 PATH/DIR2 # Report different files between two directories diff -q PATH/DIR1 PATH/DIR2 # Report different files between two directories without showing the different content of files with identical file names diff -s PATH/DIR1 PATH/DIR2 # Report different files and identical files between two directories diff --ignore-file-name-case PATH/DIR1 PATH/DIR2 # Report different files between two directories ignoring case differences in file names diff -i PATH/DIR1 PATH/DIR2 # Report different files between two directories ignoring case differences in the content of text files diff -r PATH/DIR1 PATH/DIR2 # Report different files between two directories and all their subdirectories dir1="PATH/DIR1"; dir2="PATH/DIR2"; diff -q "$dir1" "$dir2" | grep "$dir2" | sed "s/$dir2\///g" | awk '{print $4}' # Report different files and files that are only present in DIR2 Return the full path of a file realpath PATH/FILE Change access permissions: # Users: uuugggooo (ugo = u: owner user, g: all the users of the group, o: all other users) # Modes: 0: ---, 1: --x, 2: -w-, 3: -wx, 4: r--, 5: r-x, 6: rw- , 7: rwx chmod MOD FILE # Change permissions for a file chmod -v MOD FILE # Change permissions for a file, with verbose output chmod -R MOD DIRECTORY # Change permissions for a directory tree (recursive) chmod -vR MOD DIRECTORY # Change permissions for a directory tree (recursive), with verbose output find . -type f -exec chmod MOD {} + # Change permissions for all files (only files, not directories) in a directory tree (recursive) find . -type d -exec chmod MOD {} + # Change permissions for all directories (only directories, not files) in a directory tree (recursive) Change owner: chown OWNER FILE # Change the owner of a file chown OWNER:GROUP FILE # Change the owner and the group of a file chown -R OWNER DIRECTORY # Change the owner of a directory tree (recursive) chown -R OWNER:GROUP DIRECTORY # Change the owner and a group of a directory tree (recursive) Change group: chgrp OWNER FILE # Change the group of a file chgrp -R OWNER DIRECTORY # Change the group of a directory tree (recursive) Raw copy: dd if=/dev/fd0 of=/dev/fd1 # copy a floppy disk (using two floppy disk drives) dd if=/dev/hda0 of=/dev/hda1 # copy a hard disk (or partition) into another one dd if=/dev/hda | gzip -9v | dd of=/mnt/hdb/hda.img # back up the whole hard disk as a gzipped file (you may need to mount a partition of another hard disk as hdb: mount /dev/hdb2 /mnt/hdb ) dd of=/mnt/hdb/hda.img | gzip | if=/dev/hda # restore the gzipped image of a hard disk dd if=/dev/fd0 of=/tmp/disk.bak # copy the whole floppy disk as a file dd if=/dev/hda of=/tmp/mbr.bin count=1 bs=512 # back up MBR (boot loader and partition table) Create and edit disk partitions: apt-get install parted parted # starts the partition utility parted commands: print # show current partitions mkpart primary START END # create a new primary partition mkpart extended START END # create a new extended partition mkpart logical START END # create a new logical partition (within the range of an extended one) resize PARTITON_N NEW_START NEW_END # resize an existing partition rm PARTITION_N # remove a partition quit # quit parted Show available devices: fdisk -l Show device partitions: fdisk -l /dev/DEVICE Make a file system (format) a disk (partition): mkfs.ext3 /dev/hda1 # Format hda1 as an ext3 partition mkfs -V -t ext3 /dev/hda1 # Format hda1 as an ext3 partition (Verbose output) mkfs -t msdos /dev/hda2 # Format hda1 as a MS-DOS partition mkfs -t msdos /dev/fd0 # Format a floppy disk using the MS-DOS file system mkswap /dev/hda2 # Format hda2 as the swap partition dd if=/dev/zero of=/dev/foo1 bs=512 count=1 # If you created or changed a DOS partition, you can use dd to zero the first 512 bytes Create a RAMDisk: mkdir -p /media/ramdisk mount -t tmpfs -o size=2048M tmpfs /media/ramdisk # 2GB RAMDisk, it can use swap partition if needed, and doesn't preallocate disk space. mount -t ramfs -o size=1024M ramfs /media/ramdisk # 1GB RAMDisk, it only uses physical RAM, not the swap partition, and preallocates disk space. Make sure you have enough spare RAM. umount /media/ramdisk # unmount RAMDisk Access a XFS partition: # You typically need this to read disks from NAS devices, via USB. mkdir /ramdisk/xfs1 # If you are using a Linux boot disk, like Knoppix, you'll probably need to mount the XFS partition in the RAMDisk fdisk -l # Use fdisk to locate the XFS partition mount -t xfs /dev/XFS_PARTITION /ramdisk/xfs1 # use the path to XFS partition you've found using fdisk instead of /dev/XFS_PARTITION cp -rp /ramdisk/xfs1/* /DESTINATION # copy all files from the XFS formatted partition to another device ( /DESTINATION ) Check and repair a file system: fsck fsck -f -y FILE_SYSTEM # Note: Generally the device containing the file system to check can be /dev/hda0 on a single hard drive system or /dev/md0 on a RAID system); -y automatically reply yes (y) to all questions. Note that if it asks to connect to lost+found more than once, you may have some damaged files. List RAID arrays: mdadm --examine --scan Check if a RAID array is up and working properly: mdadm --detail /dev/md0 # assuming your RAID volume is /dev/md0 Check if two devices are in a RAID array: mdadm --examine --scan /dev/sda1 mdadm --examine --scan /dev/sdb1 # Must return same MD array and UUID. Path to actual devices in your system may differ from /dev/sda1 and /dev/sdb1 of this example Copy partitions from a device to another one: sfdisk -d /dev/sda | sfdisk /dev/sdb # copy partitions from sda to sdb Choose what partitions should be automatically mounted at system startup: jed /etc/fstab # edit static file system information Show memory usage: free free -m # show values in Megabytes free -mh | xargs | awk '{print "Memory: Size: " $8 " Used: " $9 " Free: " $10}'; Virtual memory statistics: vmstat # report information about processes, memory, paging, block IO, traps, and cpu activity. vmstat -s # event counters and memory statistics vmstat -d # disk statistics Show free disk space: df -T # show free disk space, returns also file system type df -Th # show human readable values (in Megabytes, Gigabytes...) df -h | xargs | awk '{print "Disk ("$8"): Size: " $9 " Used: " $11 " Free: " $10}'; df -P -h | nawk '0+$5 >= 90 {print "FS: "$1" ("$6") Size: "$2" Used: "$3" (\033[1;31m"$5"\033[0m) Free: "$4" (\033[1;31m"(100-$5)"%\033[0m)";}' # Only show near full file systems Summarize disk usage: # du shows size in KBytes by default (switch: -k), if you want to see the size in bytes, use the switch -b du PATH/FILE # disk usage for specified file du -s PATH # disk usage for the whole directory (recursively), total size du -s --exclude=*.mp3 PATH # disk usage for the whole directory (recursively), total size, excluding all files ending with .mp3 du PATH # disk usage for the whole directory (recursively) du -a PATH # disk usage for the whole directory (recursively), showing disk usage for all files and directories, not just directories du -S PATH # disk usage for the whole directory (recursively, but size is separated for each directory) du -a PATH | sort -nr | less # show directories and files sorted by size for the given PATH and its subdirectories du -a PATH | sort -nr | head -n 25 # show the 25 largest directories and files within the given PATH . Use root ( / ) as PATH to see the largest directories and files in the whole disk find PATH -type f -exec file -b '{}' \; -printf '%s\n' | awk -F , 'NR%2 {i=$1} NR%2==0 {a[i]+=$1} END {for (i in a) printf("%12u %s\n",a[i],i)}' | sort -nr # show the size occupied by files within a given PATH sorted by file type (slow) Show deleted files locked by a running process: # If you can't see more free space in your disk after deleting a large file it's probably locked by a running process so that it has been unlinked but not actually deleted yet. In this case you'll have to kill, stop, or restart the process to unlock the file and free its disk space lsof +L1 Count files and subdirectories within directories: ls -1 /PATH/ | wc -l # count files and subdirectories within a directory ls -1R /PATH/ | wc -l # count files and subdirectories within a directory including subdirectories (recursively) ls -1 /PATH/ | grep -i .*.EXT | wc -l # count files and subdirectories with a given EXTension within a directory ls -1R /PATH/ | grep -i .*.EXT | wc -l # count files and subdirectories with a given EXTension within a directory including subdirectories (recursively) Count files within directories: find . -maxdepth 1 -type f | wc -l # count files within a directory find . -type f | wc -l # count files within a directory including subdirectories (recursively) find . -maxdepth 1 -type f | grep -i .*.EXT | wc -l # count files with a given EXTension within a directory find . -type f | grep -i .*.EXT | wc -l # count files with a given EXTension within a directory including subdirectories (recursively) Show mounted devices: mount Mount a device: mount DEVICE # example: mount /dev/cdrom mount -t FILESYSTEM DEVICE MOUNTPOINT # Mount a DEVICE at MOUNTPOINT as a given FILESYSTEM; example: mount -t ext3 /dev/sda1 /test Mount an ISO image of a CD: mount -o loop cd.iso /mnt/DIR_NAME/ Unmount a device: umount DEVICE # example: umount /dev/cdrom Unmount and eject a device (like a CD-ROM): eject # eject default device eject -d # show default device eject -r # eject CD-ROM Force changed blocks cached in RAM to disk: sync ----- Mount a remote directory tree: apt-get install sshfs fuse-utils To let other users to handle the remote directory tree: # create a group called fuse containing users allowed to mount the remote directory tree chgrp fuse /usr/bin/fusermount chmod u+s /usr/bin/fusermount adduser USER fuse Create the local directory where the remote directory tree will be mounted: mkdir /mnt/sshdir chown USER /mnt/sshdir Mount a remote directory tree: sshfs USER@HOST:/PATH /mnt/sshdir ----- Install VIM editor: atp-get install vim Install Jed editor: apt-get install jed ----- Configure boot devices and partitions: # see GNU GRUB (GRand Unified Bootloader) configuration: less /boot/grub/grub.cfg # /boot/grub/grub.cfg replaces /boot/grub/menu.lst and it's not meant to be edited # in distros other than Debian the path could be /boot/grub/grub.conf or /etc/grub.conf # edit LILO (LInux LOader) configuration: jed /etc/lilo.conf ----- Print a file on a printer: lpr -Pprinter PATH/FILE lpr -Pprinter -#N PATH/FILE # print N copies of FILE on printer List print queue: lpq Remove a job from the print queue: lprm JOB_ID lprm # remove all jobs from the print queue ----- Indentation: Indent C code (or similarly formatted code): apt-get install indent indent PATH/SOURCE -o PATH/DEST indent -nprs PATH/SOURCE -o PATH/DEST # Doesn't add a space after an open bracket and before a closed one sed -i 's/^[[:space:]]*//g' PATH/FILE # Remove indentation from any text file Indent HTML code: apt-get install hindent hindent PATH/SOURCE.HTML>PATH/DEST.HTML hindent -i 0 PATH/SOURCE.HTML>PATH/DEST.HTML # Remove indentation hindent -c PATH/SOURCE.HTML>PATH/DEST.HTML # Force all tags to lowercase (otherwise all tags are forced to UPPERCASE by default) hindent -c -i 2 PATH/SOURCE.HTML>PATH/DEST.HTML # Set indentation level to 2 and force all tags to lowercase hindent -c PATH/SOURCE.HTML>PATH/DEST.HTML # Force all tags to lowercase hindent -f PATH/SOURCE.HTML>PATH/DEST.HTML # Flow: dump only HTML tags discarding any data between hindent -l PATH/SOURCE.HTML>PATH/DEST.HTML # List all HTML tags contained in the source document ----- Scheduled jobs with CronTab: # You can change the editor for Crontab (parameters in the following commands are case sensitive): export EDITOR=jed export VISUAL=jed # Edit CronTab file (using the specified editor, or the default editor, tipically "vi", otherwise) crontab -e Format: # Line of comments start with the dash character (#) minute (0-59) hour (0-23) day of month (1-31) month (1-12) day of week (0-6, Sunday=0) COMMAND # Use asterisks to specify any occurrence of the time parameter * * * * * echo "test" # execute the command every minute (not recommended) 30 * * * 1 echo "test" # execute the command every 30 minutes, every monday # separate multiple istances of the same time parameter with a comma (,) 15,30,45 * * * 1,3,5 echo "test" # execute the command every 15 minutes, every monday, wednesday, friday # separate two istances with a dash (-) to create a time range 0 0,12 * 6-9 1-5 echo "test" # execute the command at midnight amd at twelve, every day from monday to friday, during every month from June to September # Percent character (%) must be escaped (\) 30 * * * 1 touch /PATH/file_$(date +\%Y\%m\%d).txt # Normally, crontab sends a mail to the user with the output of the command executed, including errors. Append ">/dev/null 2>&1" to the command line to prevent this behaviour * * * * * echo "test">/dev/null # list all scheduled jobs crontab -l # erase the crontab file (not recommended because remove all jobs, including ones that might be necessary for the system, you'd better edit the crontab files and delete entries manually) crontab -r ----- Networking: arp -a # show DNS, IP, and MAC address for all network interfaces ifconfig -a # show DNS, IP, and MAC address for all network interfaces ifconfig eth0 # show DNS, IP, and MAC address for all network interfaces # show DNS, IP, and MAC address for the given interface (eth0) Change MAC Address of an interface (eth0): ifconfig eth0 down ifconfig eth0 hw ether xx:xx:xx:xx:xx:xx ifconfig eth0 up Wake On LAN (WOL): apt-get install ethtool ethtool -s eth0 wol g # Set Wake On LAN on the specified interface (eth0). The system must support Wake On LAN and it has to be enabled from the BIOS ethtool -s eth0 wol d # Unset Wake On LAN on the specified interface (eth0) apt-get install etherwake etherwake xx:xx:xx:xx:xx:xx # Wake On LAN the remote system with the specified MAC address ifconfig # show local network interfaces ifconfig INTERFACE IP_ADDRESS # change the IP address for a network interface ifconfig INTERFACE down # disable a network interface (WARNING: if you are connected from a remote system and disable the network interface to which you are connected you'll be disconnected, and if there isn't another interface available or a script that sets the interface up again automatically, you'll need physical access to the computer to configure it) ifconfig INTERFACE up # enable a network interface iwconfig # show local wireless network interfaces iwconfig INTERFACE IP_ADDRESS # change the IP address for a wireless network interface iwconfig INTERFACE down # disable a wireless network interface (WARNING: if you are connected from a remote system and disable the network interface to which you are connected you'll be disconnected, and if there isn't another interface available or a script that sets the interface up again automatically, you'll need physical access to the computer to configure it) iwconfig INTERFACE up # enable a wireless network interface Set default Gateway: route add default gw GATEWAY_IP # GATEWAY_IP in most cases is 192.168.0.254 or 192.168.1.254 # Specify DNS Servers to resolve domain names jed /etc/resolv.conf # OpenDNS servers nameserver 208.67.222.222 nameserver 208.67.220.200 jed /etc/hosts # assign hosts to specific IPs jed /etc/network/options jed /etc/network/interfaces /etc/init.d/networking restart Show active Internet connections: netstat -atunlp netstat -atun | awk "{print $5}" | cut -d: -f1 | sed -e "/^$/d" | sort | uniq -c | sort -n # show open connections sorted by IP address netstat -plntu # show applications listening TCP/UDP sockets netstat -A inet -lnp # only servers netstat -A inet # without servers Show network sockets in use: lsof -i tcp:80 # show all processes that are using port 80 TCP Ping host: ping HOST # ping a host until CTRL-C is pressed ping -c N HOST # ping a host N times Trace route to host: traceroute HOST Network Mapping: apt-get install nmap nmap -sS 192.0.2.0 # search for open ports in the given IP nmap -sS 192.0.2.0/24 # search for connected IPs (and related open ports) in the given IP mask nmap -sS -O -v 192.0.2.0 # return Operating System and uptime for the given IP Port scanning: apt-get install netcat # scan TCP ports: nc -vvn -z 192.0.2.2 1-80 # scan TCP ports from 1 upto 80 in the given IP address # scan UDP ports: nc -u -vvn -z 192.0.2.8 1-255 # scan UDP ports from 1 upto 255 in the given IP address Calculate broadcast, network, Cisco wildcard mask, and host range apt-get install ipcalc ipcalc 192.0.2.0/24 ipcalc 192.0.2.0/255.255.255.0 ipcalc 192.0.2.0-192.0.2.100 # deaggregate address range DNS lookup: apt-get install host host www.example.com # return both IPv4 and IPv6 addresses host -t A www.example.com # return the IPv4 addresses host -t AAAA www.example.com # return the IPv6 addresses host 192.0.2.0 # Reverse DNS lookup. Note that nslookup is now deprecated DNS queries: dig www.example.com A # get the IP address of the given domain dig www.example.com TXT # get text annotations about the given domain dig www.example.com MX # get the mail servers of the given domain dig www.example.com NS # get the authoritative DNS servers for the given domain dig www.example.com ANY # get all DNS information for the given domain Information about a domain name: whois example.com # IP Tables: iptables -I INPUT -s 192.0.2.100 -j DROP # Block a specific IP iptables -I INPUT -s 192.0.2.0/255.255.255.0 -j DROP # Block a range of IPs using a netmask iptables -I INPUT -s 192.0.2.0/24 -j DROP # Block a range of IPs using CIDR iptables -S # Show rules iptables -L # List rules iptables -L INPUT -n --line-numbers # List only INPUT rules together with their line numbers iptables -D INPUT LINE_NUMBER # Delete the INPUT rule with specified LINE_NUMBER Script to clear IPTables: --- #!/bin/bash # Clear iptables iptables -F iptables -X iptables -t nat -F iptables -t nat -X iptables -t mangle -F iptables -t mangle -X iptables -t raw -F iptables -t raw -X iptables -P INPUT ACCEPT iptables -P FORWARD ACCEPT iptables -P OUTPUT ACCEPT --- Download files from the Internet: # note: default ports: HTTP: 80, FTP: 21 wget "http://www.example.com/file.gz" # download the document at the given URL to the local directory wget "http://www.example.com/file.gz" -O PATH/downloaded_file.gz # download the document at the given URL to the specified directory and with the specified file name wget -S "http://www.example.com/file.gz" # download the document at the given URL in the local directory, and save the HTTP header at the beginning of the file wget -c --wait=2 --limit-rate=20k "http://www.example.com/file.gz" # continue an interrupted download, and pause 2 secods after getting every file and limit download speed to 20KBps wget -t 50 "http://www.example.com/file.gz" # retries up to 50 times if the download fails or it's incomplete wget -t 0 "http://www.example.com/file.gz" # retries forever if the download fails or it's incomplete, until it can get the requested file echo 'wget "http://www.example.com/file.gz"'|at 02:00 # start a download at 2:00am wget -O- "http://www.example.com/path/page.html" # dump the output of the page on the console (normally on the screen) wget -q -O- "http://www.example.com/path/page.html"|grep 'a href' # show all links within given document wget -r -l1 "http://www.example.com/path/page.html" # download the first layer of links from given document wget -r -l1 --no-parent "http://www.example.com/path/page.html" # download the first layer of links from given document, ignoring links that are not within the given URL path wget -r -l0 "http://www.example.com/path/page.html" # download a whole website to the local directory wget --mirror "http://www.example.com/" # download a whole website to the local directory wget -r -l1 --no-parent -A "*.gif" "http://www.example.com/dir/" # download all files matching a certain pattern (all GIF files in this case) from the specified URL wget -U USER_AGENT_STRING "http://www.example.com/file.gz" # you may set the User Agent to pretend to get the file from the server with a normal browser Dump the content of a webpage on the screen: lynx -dump http://www.example.com/ Run a script on the web server: wget -O- "http://localhost/script.php" # run a script on the local server wget -q -O- "http://localhost/script.php" # run a script on the local server hiding the output wget -q -T 0 -O- "http://localhost/script.php" # By default wget drops connection after 900 seconds (15 minutes), in this case you'll get a "Read error (Connection timed out) in headers. Retrying." and if the script always takes more than the given time, it will retry uselessly until the maximum allowed retries (by default 20, but it can be changed with -t). To avoid time outs you can set a longer time using -T followed by the number of seconds (or 0 for infinite time, in this case make sure the script is working or it will run forever and if it uses resources incrementally it may crash the server). Also, if you experienced a time out with wget while running a script on localhost you should better restart the webserver as the script may continue running despite of wget timing out and dropping the connection. lynx -dump http://localhost/script.php >/dev/null # run a script on the local server (less reliable than wget for this purpose) hiding the output php PATH/FILE.php # run a PHP script php -r "echo 'Hello World';"; # Execute a PHP line in the shell php -r 'echo "Hello World\n";'; # Execute a PHP line in the shell. Use quotes to print character sequences. perl -e "print 'Hello World';"; # Execute a Perl line in the shell python -c "print 'Hello World'"; # Execute a Python line in the shell ruby -e "puts 'Hello World'"; # Execute a Ruby line in the shell # If you receive this error message from the PPP Daemon (PPPD) # pppd: The remote system is required to authenticate itself but I couldn't find any secret (password) which would let it use an IP address. # You can fix it adding the string "noauth" (or changing the existing string "auth" to "noauth") in /etc/ppp/options ----- Mirror a disk through the network: If you have two computers with identical hardware you can mirror a system into the other machine through the network This is especially useful with clusters / virtual servers Start both machines from a live CD (like Knoppix - http://www.knoppix.org/ ) Connect both machine with a cross cable through their eth1 interfaces and set them this way: root@source_pc# ifconfig eth1 192.168.0.1 root@destin_pc# ifconfig eth1 192.168.0.2 Then execute these commands on the machines to start transfers: root@source_pc# dd if=/dev/sda bs=128M | nc _w2 192.168.0.2 9000 root@destin_pc# nc _l _p 9000 | dd of=/dev/sda Depending from the size of the disk it may need many hours ----- Maths: Return the result of an expression: # operators must be escaped and separated by a space # operators: + - * / % < > = >= <= != expr \( 1 \+ 2 \) \* 3 # Result = 20 expr 5 \> 3 # Returns 1 (true) expr 3 \> 5 # Returns 0 (false) Show Multiplication Table: for i in {1..9}; do for j in $(seq 1 $i); do echo -ne $i\x$j=$((i*j))\\t;done; echo; done; Return the prime factors of a number: factor NUMBER factor NUMBER1 NUMBER2 NUMBER3... # return factors for the given numbers Return a sequence of numbers: echo {FIRST..LAST} # return a sequence of integer numbers from FIRST to LAST, numbers can be negative, and LAST can be less than FIRST for ((i=FIRST; i<=LAST; i+=INCREMENT)); do echo $i; done # create a sequence from FIRST up to LAST number with given INCREMENT seq LAST # create a sequence up to LAST number (default FIRST is 1, INCREMENT is 1) seq FIRST LAST # create a sequence from FIRST up to LAST number (default INCREMENT is 1) seq FIRST INCREMENT LAST # create a sequence from FIRST up to LAST number with given INCREMENT. Numbers can be negative, and END can be less than START, if the INCREMENT is negative seq -s "," FIRST INCREMENT LAST # create a sequence from FIRST up to LAST number with given INCREMENT. Numbers are separated by a comma (",") instead of a line break seq -s "," -w FIRST INCREMENT LAST # create a sequence from FIRST up to LAST number with given INCREMENT. Numbers are separated by a comma (",") instead of a line break. Numbers with less digit than the greater number are padded with 0s (or the sign for negative numbers) seq FIRST INCREMENT LAST | shuf # shuffle a sequence of numbers Random numbers: echo $RANDOM # return a random number between 0 and 32767 (16-bit integer) echo $((RANDOM%(MAX+1-MIN)+MIN)) # return a random number between MIN and MAX expr \( $RANDOM \% \( MAX \+ 1 \- MIN \) \+ MIN \) # return a random number between MIN and MAX Base conversion: apt-get install bc echo 'ibase=10; obase=16; 155'|bc # base conversion (155 from decimal to hexadecimal) echo 'ibase=16; obase=2; F'|bc # base conversion (F from hexadecimal to binary) echo '2*(3+4)'|bc -l # perform math operations Measure conversion: apt-get install units units # interactive units "1 in" "cm" # convert 1 inch to centimeters ----- Time: Set date and time: date MMDDhhmmYYYY.ss Set date from NTP (SNTP) server: apt-get install ntpdate ntpdate SERVER # set the clock ntpdate -q SERVER # query only for the time, don't set the clock SNTP servers: ntp.ubuntu.com # Ubuntu ntp1.inrim.it # INRiM, Torino, Italy ntp.univ-lyon1.fr # University of Lyon, Lyon, France vega.cbk.poznan.pl # Astrogeodynamic Observatory, Borowiec, Poland bitsy.mit.edu # MIT, Cambridge, Massachusetts, USA bonehed.lcs.mit.edu # MIT, Cambridge, Massachusetts, USA clock.isc.org # Internet Systems Consortium, Palo Alto, California, USA Start a NTP server: apt-get install ntp-refclock (or another ntp* package) jed /etc/ntp.conf /etc/init.d/ntp-server restart Date tokens: %% : % %a : Weekday's abbreviated name (for example, Mon) %A : Weekday's full name (for example, Monday) %b : Month's abbreviated name (for example, Jan; same as %h) %B : Month's full name (for example, January) %C : Century (for example, 21) %d : Day of month (for example, 01) %e : day of month, space padded; same as %_d %F : Full date formatted as full year - month - day (same as %Y-%m-%d ) %g : last two digits of year of ISO week number (see %G) %G : year of ISO week number (see %V); normally useful only with %V %H : Hour in 24 hours format with padded 0 (00-23) %I : Hour in 12 hours format with padded 0 (01-12) %j : Day of year with padded 0 (001-366) %k : Hour in 24 hours format with padded space ( 0-23) %l : Hour in 12 hours format with padded space ( 1-12) %m : Month with padded 0 (01-12) %M : Minute with padded 0 (00-59) %n : Newline %N : Nanoseconds (000000000-999999999) %p : AM or PM, uppercase %P : AM or PM, lowercase %R : 24-hour hour and minute (same as %H:%M ) %s : Seconds since Linux Epoch (1970-01-01 00:00:00 UTC) %S : Second with padded 0 (00-60) %t : Tab %T : Time (same as %H:%M:%S ) %u : ISO Day of week (1-7: 1 is Monday, 7 is Sunday) %U : Week number of year, with Sunday as first day of week and padded 0 (00-53) %V : ISO Week number of the year, with Monday as first day of week and padded 0 (01-53) %w : Day of week (0-6: 0 is Sunday, 6 is Saturday) %W : ISO Week number of year, with Monday as first day of week and padded 0 (00-53) %y : Last two digits of year (00-99) %Y : Full Year %z : Timezone as hhmm (e.g., -0200) %:z : Timezone as hh:mm (e.g., -02:00) %Z : Timezone's abbreviated name (e.g., CET) Show current date and time: date date '+%a %d %b %Y %H:%M:%S %Z (UTC%:z)' # show day of the week, day of the month, month (short format), year (full digits), hour, minutes, seconds, timezone echo "`date '+%a %d %b %Y %H:%M:%S %Z (UTC%:z)'`"; # echo day of the week, day of the month, month (short format), year (full digits), hour, minutes, seconds, timezone Show information about your local time: timedatectl status Show System Time Zone cat /etc/timezone Linux Epoch (1 January 1970 00:00:00) date +"%s" # Seconds from Epoch to current time date -d "2000-01-01" +"%s" # Seconds from Epoch to the specified date (from 1 January 1970 to 1 January 2000 in this example) date -d @1474303375 # Converts seconds from Epoch to date Dates: date +"%F" -d now # Return current day in the specified format date +"%F" -d yesterday # Return yesterday's day in the specified format date +"%F" -d tomorrow # Return tomorrow's day in the specified format date +"%F" -d last-week # Return last week's date on current day in the specified format date +"%F" -d next-week # Return next week's date on current day in the specified format date +"%F" -d last-month # Return last month's date on current day in the specified format date +"%F" -d next-month # Return next month's date on current day in the specified format date +"%F" -d last-year # Return last year's date on current day in the specified format date +"%F" -d next-year # Return next year's date on current day in the specified format date +"%F" -d last-WEEKDAY # Return the date of the last given weekday (for example last-monday) in the specified format date +"%F" -d next-WEEKDAY # Return the date of the next given weekday (for example next-monday) in the specified format Calculate dates: date +"%F" -d "now +15 days" # Show the resulting date adding the specified time to the current date in the specified format (date only in this example) date +"%F" -d "2000-01-01 +15 days" # Show the resulting date adding the specified time to the given date in the specified format (date only in this example) date +"%F %H:%M:%S" -d "now +10 seconds" # Show the resulting date adding the specified time to the current date in the specified format (date and time in this example) date +"%F %H:%M:%S" -d "2000-01-01 10:10:00 UTC +30 seconds" # Show the resulting date adding the specified time to the given date in the specified format (date and time in this example) Get the day of the week for a specified date: date -d "2000-01-01" +"%A" Calculate how long takes a command to be executed: time COMMAND COMMAND_PARAMETERS time cat # works like a stopwatch: counts time until you send an EOF to stop it (which generally means until you press CTRL+D that closes the cat command) Create filenames containing current date and time: touch data_`date +%F`.bak # create a file with the name data_YYYY-MM-DD.bak touch log_`date +%F_%H-%M-%S`.txt # create a file with the name log_YYYY-MM-DD_HH-MM-SS.txt User and system times: times Show a calendar: # cal # show dayweeks in a row, Sunday first (use -m to have Monday first) # ncal # show dayweeks in a coloumn, Monday first cal # show a calendar for the current month cal -y # show a calendar for the current year cal -y YEAR # show a calendar for the given year cal MONTH YEAR # show a calendar for the given MONTH and YEAR ncal -e YEAR # show date of Easter (for Western Churches) ncal -o YEAR # show date of Orthodox Easter (Greek and Russian Orthodox Churches) Pause: sleep S # pause for S seconds Memorable dates: The directory /usr/share/calendar/ contains some collections of remarkable dates. For example: cat /usr/share/calendar/calendar.history # Important historical dates cat /usr/share/calendar/calendar.music # Important dates in music history cat /usr/share/calendar/calendar.lotr # Lord of the Rings dates ----- Archiving and Compression: Install archivers: apt-get install bzip2 apt-get install zip unzip apt-get install zoo apt-get install lha apt-get install rar apt-get install unp Archiving with ISO: dd if=/dev/dvd of=dvd.iso # Create an ISO file from a DVD dd if=/dev/cdrom of=cd.iso # Create an ISO file from a CD-ROM dd if=/dev/scd0 of=cd.iso # Create an ISO file from a SCSI CD-ROM mkisofs -o /PATH_DEST/cd.iso /PATH_SOURCE/ # Create an ISO file from a directory tree Archiving with TAR: tar -cvf file.tar text.txt # archive text.txt as file.tar tar -cvf file.tar *.txt # archive all files ending with ".txt" in the current directory as file.tar tar -xvf file.tar # extract all files from file.tar in the current directory Compressed TAR archives (tarball): # Use tar to compress a directory tree into a single file, and gzip or bzip2 to compress each file into a directory tree tar -cjvf file.tbz text.txt # TAR+BZIP2 compression (canonical extension .tbz or .tar.bz2) tar -xjvf file.tbz # extract all files from a TAR+BZIP2 compressed archive tar -czvf file.tgz text.txt # TAR+GZIP compression (canonical extension .tgz or .tar.gz) tar -xzvf file.tgz # extract all files from a TAR+GZIP compressed archive Compression with GZIP: gzip -9vk text.txt # compress text.txt as text.txt.gz using highest compression and verbose output, keeping the original uncompressed file gzip -9v text.txt > file.gz # compress text.txt as file.gz using highest compression and verbose output, keeping the original uncompressed file gzip -9v text.txt # compress text.txt as text.txt.gz using highest compression and verbose output, deleting the original uncompressed file gzip -r9v file.txt file2 file3 /home/docs # compress file.txt file2 file3 and every file in /home/docs (if such directory exists), deleting the original uncompressed files gzip -l file.gz # show the content of the compressed file gzip -dk file.gz # extract the content of file.gz in the current directory and keep the compressed file (file.gz) gzip -d file.gz # extract the content of file.gz in the current directory and delete the compressed file (file.gz) Compression with BZIP2: bzip2 text.txt # compress text.txt as text.txt.bz2 bunzip2 file.bz2 # extract the content of file.bz2 into the current directory Compression with ZIP: zip file.zip file.txt # add file.txt to the compressed file file.zip (if it doesn't exist, it will created) zip -r file.zip /home/docs # compress the whole content of the directory /home/docs into file.zip zip -r filename.zip file.txt file2 file3 /home/docs # compress file.txt file2 file3 and the content of /home/docs (if such directory exists) into file.zip zip -F brokenfile.zip # try to fix a broken zipped file (use -FF to try even harder) unzip file.zip # extract the content of file.zip into the current directory unzip file.zip -d PATH_DEST # extract the content of file.zip into the specified directory Compression with LHa (LHarc) / LZH: lha a file.lha text.txt # compress text.txt as file.lha (or add text.txt to an already existing file.lha compressed archive) lha e file.lha # extract the content of file.lha into the current directory (will prompt for existing files) Compression with ZOO: zoo a file.zoo text.txt # compress text.txt as file.zoo (or add text.txt to an already existing file.zoo compressed archive) zoo e file.zoo # extract the content of file.zoo into the current directory (will ignore existing files) Compression with RAR (RAR is shareware): rar a file.rar text.txt # compress text.txt as file.rar (or add text.txt to an already existing file.rar compressed archive) rar e file.rar # extract the content of file.rar into the current directory (will prompt for existing files) Unpack most common formats: unp FILE # Supported extensions and formats: tar[.gz,.bz2], gz, bz2, Z, ar/deb, rpm, shar, rar, arj, zip, LHa, cab, ace, tnef, uu (mail, news), mime, hqx, sea, zoo, pmd, cpio, afio, lzop ----- Encoding: Install encoders/decoders: apt-get install mpack apt-get install uudeview apt-get install yencode MIME/Base64: mpack -s SUBJECT -o PATH/DEST_FILE PATH/SOURCE_FILE # encode (B64encode) within a MIME message with header munpack PATH/FILE # decode (B64decode) to the original file name uudeview -i PATH/FILE # decode (B64decode) to the original file name Base64: uuenview -b PATH/SOURCE_FILE>PATH/DEST_FILE # encode (B64encode). will not be decoded without MIME header. UU (Unix-to-Unix): uuenview -u PATH/SOURCE_FILE>PATH/DEST_FILE # encode (uuencode) uudeview -i PATH/FILE # decode (uudecode) to the original file name XX: uuenview -x PATH/SOURCE_FILE>PATH/DEST_FILE # encode (xxencode) uudeview -i PATH/FILE # decode (xxdecode) to the original file name Yenc: uuenview -y PATH/SOURCE_FILE>PATH/DEST_FILE # encode (yencode) uudeview -i PATH/FILE # decode (ydecode) to the original file name # uudeview handy options: +o don't overwrite existing files, -c autoclear (delete) successfully decoded files, -d desperate mode (process incomplete files) yencode PATH/SOURCE_FILE # encode (yencode) SOURCE_FILE as SOURCE_FILE.ync ydecode PATH/FILE # decode (ydecode) to the original file name ydecode -f PATH/FILE # decode (ydecode) to the original file name (force overwriting already existing files, without prompting) ----- Hashing: md5sum PATH/FILE # Return the MD5 hash for the content of the given FILE md5sum PATH/FILE* # Return the MD5 hash for the content of the files matching the given mask md5sum PATH/FILE1 PATH/FILE2 PATH/FILE3 # Return the MD5 hash for the given FILEs echo -n "STRING" | md5sum | sed "s/ -//g" # Return the MD5 hash of the given STRING VARIABLE=`echo -n "STRING" | md5sum | sed "s/ -//g"`; # Store the MD5 hash of the given STRING into a VARIABLE VARIABLE=`echo -n "STRING" | md5sum | sed "s/ -//g"`; echo "[$VARIABLE]"; # Store the MD5 hash of the given STRING into a VARIABLE and output it between square brackets ----- Obfuscation: Text scrambling (anagrams): apt-get install wordplay wordplay "text string" # list all the possible anagrams of the given text string Alphabet scrambling: apt-get install rotix rotix # accept a text from standard input and return its ROT-13 to the standard output rotix -f PATH/SOURCE_FILE -o PATH/DEST_FILE # read SOURCE_FILE and return its ROT-13 to DEST_FILE (note: a ROT-13 can be deobfuscated performing the same operation again) rotix -r 15 -L -f PATH/SOURCE_FILE -o PATH/DEST_FILE # read SOURCE_FILE and return its ROT-15 to the Left to DEST_FILE (note: this can be deobfuscated performing a ROT-15 to the Right) rotix -r 12 -R -f PATH/SOURCE_FILE -o PATH/DEST_FILE # read SOURCE_FILE and return its ROT-12 to the Right to DEST_FILE (note: this can be deobfuscated performing a ROT-12 to the Left) Steganography: apt-get install steghide steghide --embed -p PASSWORD -ef SOURCE_FILE_TO_HIDE -cf DEST_OBFUSCATION_FILE # hide a file into another (AES encryption and compression are applied by default) steghide --embed -p PASSWORD -ef SOURCE_FILE_TO_HIDE -cf OBFUSCATION_FILE -sf DEST_FILE # hide a file into another (AES encryption and compression are applied by default), but save the result into a new file steghide --info STEGOFILE -p PASSWORD # return information about the file obfuscated into STEGOFILE steghide --extract -p PASSWORD -sf STEGOFILE -xf DESTFILE # extract the file obfuscated into STEGOFILE as DESTFILE # steghide applies AES encryption and compression by default. It can hide information into JPEG, BMP, WAV, and AU files. apt-get install stegdetect stegdetect FILE.jpg # try to find steganographed information into a JPEG file ----- Encryption with GnuPG (GPG) and Pretty Good Privacy (PGP) wrapper: apt-get install pgpgpg # PGP pgp -h # Help pgp -k # Key Manager functions help pgp -kg # Generate key pgp -kxa USER_ID PATH/DEST_FILE.ASC # Export User's public key to file pgp -ka PATH/PUBLIC.KEY # Add public key to keyring pgp -ea PATH/SOURCE_CLEARTEXT RECIPIENT_ID # Encrypt file for Recipient pgp PATH/ENCRYPTED_FILE # Decrypt file (need password) # Handy options: # (Usage example: PGP -e = binary object file, PGP -ea = ASCII object file) # -a ASCII armour # -w Wipe (overwrite&delete) original file (when crypting) # -m No output file (on screen display, "eyes only") (when decrypting) # GPG gpg -h # Help gpg --version # Show GPG version and supported algorithms gpg --gen-key # generate key gpg --list-keys # list public keys in the public keyring gpg --list-secret-keys # list secret (private) keys in the secret keyring gpg --export -a > PATH/DEST_FILE.ASC # Export all public keys to a file gpg --export -a "USER_ID" > PATH/DEST_FILE.ASC # Export User's public key to file gpg --import PATH/PUBLIC.KEY # Add public key to keyring gpg -s -r "USER_ID" PATH/SOURCE_CLEARTEXT # Sign file for Recipient, file name is automatically generated gpg -s -a -r "USER_ID" PATH/SOURCE_CLEARTEXT # Create a signature for the given file for Recipient and output with an ASCII armour, file name is automatically generated gpg -se -r "USER_ID" PATH/SOURCE_CLEARTEXT # Sign and encrypt file for Recipient, file name is automatically generated gpg -se -a -r "USER_ID" PATH/SOURCE_CLEARTEXT # Sign and encrypt file for Recipient and output with an ASCII armour gpg --verify PATH/SIGNATURE.SIG # Verify a signature (without showing the signed content) gpg PATH/ENCRYPTED_FILE # Decrypt file (need password), file name is automatically generated gpg -d PATH/ENCRYPTED_FILE > PATH/DEST_FILE # Decrypt file specifying the output file name (need password) gpg -d PATH/ENCRYPTED_FILE # Decrypt file, on screen display, "eyes only" (need password) ----- Passwords: Password generation: tr -dc 'a-z0-9' < /dev/urandom | head -c 10 # generate a random password of 10 characters with only lower case letters and numbers apt-get install apg apg -n 1 -M NL -a 1 -m 8 -x 12 # Generate a random password from 8 to 12 characters long made with only lower case letters and numbers apg -n 1 -M NL -a 0 -m 10 -x 15 # Generate a random but human readable password from 10 to 15 characters long made with only lower case letters and numbers apt-get install pwgen pwgen -1 -A -n -B # Generate a password of 8 characters made with only lower case letters (-A) and at least one number (-n, default behavior) without ambiguous characters (-B) ----- Check for brute force attack: jed /var/log/auth.log # See attempted and successful log ins (which include possible break-in attempts) ----- Audio: Play a sound: printf '\7' # Ring the Bell Extract audio tracks from a CD: apt-get install cdda2wav cdda2wav -B # extract tracks as WAVs from the default CD player cdda2wav -D /dev/cdrom -B # extract tracks as WAVs from the specified CD player Create an Audio CD writing WAVs as audio tracks: apt-get install cdrecord cdrecord dev=0,0,0 speed=4 -eject -pad -audio -raw -swab *.wav # write all WAVs from the current directory Create an Audio CD writing MP3s/OGGs/flacs as audio tracks: apt-get install mp3burn mp3burn *.mp3 # create an audio CD from all MP3s in the current directory mp3burn -p "PLAYLIST.m3u" # create an audio CD fetching files listed in a M3U playlist Play MP3 files: apt-get install mpg321 mpg321 FILE.mp3 Edit ID3v1 tags: apt-get install id3ed id3ed -i FILE.mp3 # return id3v1 tag id3ed -r FILE.mp3 # remove id3v1 tag id3ed -s SONGNAME -n ARTIST -a ALBUM -y YEAR -c COMMENT -k TRACKNUM -g GENRE_NUMBER FILE.mp3 # set id3v1 for FILE.mp3 id3ed -s SONGNAME -n ARTIST -a ALBUM -y YEAR -c COMMENT -k TRACKNUM -g GENRE_NUMBER *.mp3 # set id3v1 for all mp3 files in the current directory Edit ID3v1 and ID3v2 tags: apt-get install id3v2 id3v2 -l FILE.mp3 # return id3 tags id3v2 -1 -l FILE.mp3 # return id3v1 tag id3v2 -2 -l FILE.mp3 # return id3v2 tag id3v2 --delete-v1 FILE.mp3 # remove id3v1 tag id3v2 --delete-v2 FILE.mp3 # remove id3v2 tag id3v2 --delete-all FILE.mp3 # remove all tags (id3 v1 and v2) id3v2 -C FILE.mp3 # convert id3v1 to id3v2 tag id3v2 -a "ARTIST" -A "ALBUM" -t "SONG" -c "COMMENT" -g GENRE_NUMBER -y YEAR -T TRACK FILE.mp3 # set id3 for FILE.mp3 id3v2 -a "ARTIST" -A "ALBUM" -t "SONG" -c "COMMENT" -g GENRE_NUMBER -y YEAR -T TRACK *.mp3 # set id3 for all mp3 files in the current directory id3v2 -1 -a "ARTIST" -A "ALBUM" -t "SONG" -c "COMMENT" -g GENRE_NUMBER -y YEAR -T TRACK FILE.mp3 # set id3v1 for FILE.mp3 id3v2 -2 -a "ARTIST" -A "ALBUM" -t "SONG" -c "COMMENT" -g GENRE_NUMBER -y YEAR -T TRACK FILE.mp3 # set id3v2 for FILE.mp3 Check MP3 files: apt-get install mp3check mp3check FILE.mp3 # check crc and headers of mp3 files for consistency mp3check -a FILE.mp3 # check mp3 file for differences from layer 3, 44.1kHz, 128kB, joint stereo, no emphasis, has crc Lossless MP3 normalization (peak and loudness normalization): apt-get install mp3gain mp3gain FILE1.mp3 FILE2.mp3 FILE3.mp3 # normalize specified mp3 files (warning: overwrites source files) mp3gain -r FILE1.mp3 FILE2.mp3 FILE3.mp3 # normalize specified mp3 files all to equal loudness (warning: overwrites source files) mp3gain -a FILE1.mp3 FILE2.mp3 FILE3.mp3 # normalize specified mp3 files to the average loudness, intended for all audio files from a same album (warning: overwrites source files) mp3gain -u FILE.mp3 # undo changes made by mp3gain (based on info stored on tag) Convert WAV to MP3: apt-get install lame lame FILE.wav FILE.mp3 Convert MP3 to WAV: apt-get install mpg321 mpg321 -w FILE.wav FILE.mp3 # convert FILE.mp3 to FILE.wav Convert MP3 to OGG: apt-get install mp32ogg mp32ogg FILE.mp3 # convert FILE.mp3 to FILE.ogg mp32ogg --rename=DEST.ogg SOURCE.mp3 # convert SOURCE.mp3 to DEST.ogg mp32ogg --rename=%a_%l_%t.ogg *.mp3 # convert all mp3 files in the current directory renaming generated OGG files as ARTIST_ALBUM_TRACKTITLE.ogg (make sure that id3 tag contains all such information, otherwise files will be called as SOURCEFILE.ogg) Convert an audio format into another audio format apt-get install sox sox SOURCE_FILE.SOURCE_EXT DEST_FILE.DEST_EXT # convert SOURCE_FILE.SOURCE_EXT to DEST_FILE.DEST_EXT. File format conversion is based on standard extensions sox FILE.ogg FILE.mp3 # convert FILE.ogg into FILE.mp3 (note that MP3 support is optional. check sox --help for supported formats) Reverse audio: sox -V SOURCE_FILE.wav DEST_FILE.wav reverse ----- Manipulate video files apt-get install ffmpeg Convert a video format into another video format: ffmpeg -i PATH/SOURCE_VIDEO.avi PATH/DEST_VIDEO.mpg # convert from AVI to MPEG ffmpeg -i PATH/SOURCE_VIDEO.avi -sameq FILE.avi PATH/DEST_VIDEO.mpg # convert from AVI to MPEG, keeping same quality of the source file ffmpeg -i PATH/SOURCE_VIDEO.avi -ss 00:00:00 -t 00:45:00 PATH/DEST_VIDEO.mpg # convert from AVI to MPEG, taking only the first 45 minutes of the source video (from 00:00:00 to 00:45:00) ffmpeg -i PATH/SOURCE_VIDEO.avi -i PATH/SOURCE_AUDIO.mp3 PATH/DEST_FILE.avi # create a video with audio taking video and audio from two different sources ffmpeg -loop_input -i PATH/SOURCE_IMAGE.jpg -i PATH/SOURCE_AUDIO.mp3 -shortest -acodec copy PATH/DEST_FILE.mp4 # create a video with a static image and audio Dump all frames from a video: ffmpeg -i PATH/SOURCE_VIDEO -an -r 1/1 PATH/FILE_%06d.jpg # Dump all frames as JPG, you can dump them in .bmp or .png by changing the extension of the destination file. In this example the generated file name will have 6 digits Speed up or slow down a video: ffmpeg -i PATH/SOURCE_VIDEO -filter:v "setpts=SPEED*PTS" PATH/DEST_VIDEO # SPEED must be set to 1/(wanted speed x), for example set SPEED: 0.5 : double speed (2x), 0.25 : 4x speed, 2.0 = half speed (1/2x). If you want the destination video to have a specific duration, you can calculate the value of SPEED with this formula: 1 / (source video duration in seconds / destination video wanted duration in seconds) Extract the audio track from a video: ffmpeg -i PATH/SOURCE_VIDEO.mpg -vn -f mp3 AUDIO_TRACK.mp3 # extract the audio track as MP3 from a video ffmpeg -i PATH/SOURCE_VIDEO.mpg -vn -ac 1 -f mp3 AUDIO_TRACK.mp3 # extract the audio track as MP3 from a video as mono. Use -ac 2 to force stereo even if the input file is mono. ----- Play a video: apt-get install vlc vlc MOVIE.AVI # play a video (even in the text console) vlc http://www.example.com:8081/file.mpg # play a video from the web Streaming video server: vlc -vvv file.mpg --sout '#standard{access=http,mux=mpeg1,url=www.example.com:8081/file.mpg}' --ttl 12 --loop # Stream a MPEG file (loop) from the url http://www.example.com:8081/file.mpg # Note that you must have a streaming format like ASF to create a real stream of data vlc -vvv file.asf --sout '#standard{access=mmsh,mux=asf,url=www.example.com:8082}' --ttl 12 --loop # Stream an ASF file (loop) from the url mms://www.example.com:8082/file.asf ----- Manipulate and convert images: apt-get install imagemagick convert SOURCE.jpg DEST.png # Convert a JPEG image into PNG convert -resize 50% SOURCE.jpg DEST.jpg # resize SOURCE.jpg at 50% of the original size, and save it as DEST.jpg convert -resize 600x300 SOURCE.jpg DEST.jpg # resize SOURCE.jpg to 600x300 (width x height) pixels, and save it as DEST.jpg convert -resize 600 SOURCE.jpg DEST.jpg # resize SOURCE.jpg to a width of 600 pixels and relative height, and save it as DEST.jpg convert -resize x300 SOURCE.jpg DEST.jpg # resize SOURCE.jpg to a height of 300 pixels and relative width, and save it as DEST.jpg convert -thumbnail 150 SOURCE.jpg DEST.jpg # create a thumbnail of SOURCE.jpg with a width of 150 pixels and relative height, and save it as DEST.jpg # the difference between -resize and -thumbnail is that -thumbnail strips all meta data (like EXIF information) from the original image convert -size 1x1 xc:none IMAGE.png # create a transparent .png image sized 1 x 1 pixels convert -thumbnail x100 PATH/FILE.pdf[0] THUMBNAIL.png # create a thumbnail with a height of 100 pixels for the first page of a PDF document convert -thumbnail x100 PATH/FILE.pdf THUMBNAIL.png # create a thumbnail with a height of 100 pixels for every page of a PDF document ----- EXIF information: apt-get install exiftool exiftool FILE.jpg # show EXIF tags inside a JPEG image exiftool -geotag= FILE.jpg # remove geographical coordinates from the given image exiftool -all= FILE.jpg # remove all EXIF information from the given image for i in PATH/*.jpg; do echo "Processing: $i"; exiftool -all= "$i"; done # remove all EXIF information from all the images in the given PATH apt-get install exif exif FILE.jpg # show EXIF tags inside a JPEG image exif -im FILE.jpg # return EXIF tags inside a JPEG image in a machine readable format (-i ID instead of tags, -m tabs delimited output) exif --ifd=0 -t TAG --set-value=VALUE SOURCE.jpg -o=DEST.jpg # set TAG as VALUE in SOURCE.jpg and save the resulting manipulated image as DEST.jpg exif --remove --ifd=0 SOURCE.jpg -o=DEST.jpg # remove all EXIF information from SOURCE.jpg and save the resulting cleaned image as DEST.jpg exif -r SOURCE.jpg -o=DEST.jpg # remove the thumbnail from SOURCE.jpg and save the resulting cleaned image as DEST.jpg exif -e SOURCE.jpg -o=THUMBNAIL.jpg # extract the thumbnail from SOURCE.jpg and save it as THUMBNAIL.jpg exif FILE.jpg -n=THUMBNAIL.jpg # add THUMBNAIL.jpg to FILE.jpg and save the resulting manipulated image as FILE.jpg.modified.jpeg ----- GPS data: apt-get install gpsbabel gpsbabel -i gdb -f PATH/SOURCE_FILE.gdb -o gpx -F PATH/DEST_FILE.gpx # convert a Garmin .gdb file into .gpx (GPS eXchange Format, XML) ----- Disk quotas: apt-get install quota quotatool quota # display disk usage and limits quotacheck # scan a file system for disk usage quotactl # set disk quotas ----- Quitting a session, turning off (halting) or rebooting system: # Note that when a process receives a signal may behave differently from the default behavior CTRL+C # terminate an application (sends a SIGINT signal to the process) CTRL+\ # terminate the process and dump to a core file (sends a SIGQUIT signal to the process) CTRL+D # send an EOT (End Of Tranmission) signal to the process CTRL+Z # suspend an application (sends a SIGSUSP signal to the process) exit # exit from a non login shell logout # logout from current session shutdown -h now # halt the system shutdown -h -P now # halt the system and turn off the machine (power off) shutdown -r now # reboot the system reboot # reboot the system (like shutdown -r), might not work on older systems reboot -f # force reboot without using shutdown Show system reboot history: last reboot last -x|grep shutdown for wtmp in `ls -t /var/log/wtmp*`; do last reboot -f $wtmp; done|grep reboot last reboot|head -1 # Show only last reboot Show uptime: uptime # a sysadmin will be judged by his uptime and by how many reboots he performed # Edit system startup script # Commands that will be executed when the system boots. Different runlevels are allowed. jed /etc/inittab Choose what command must be performed when CTRl+ALT+DEL is pressed: jed /etc/inittab # What to do when CTRL-ALT-DEL is pressed. ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now ----- Install a LAMP webserver: Install servers and related software: apt-get install apache2 apt-get install lynx apt-get install openssl apt-get install php4 libapache2-mod-php4 php4-pear php4-gd php4-mcrypt # PHP 4 apt-get install php4-mysql # MySQL support for PHP 4 apt-get install php5 libapache2-mod-php5 php-pear php5-gd php5-mcrypt php5-sqlite # PHP 5 apt-get install php5-mysql # MySQL support for PHP 5 apt-get install mysql-server apt-get install phpmyadmin Apache 2 configuration: /usr/sbin/apache2 -v # return Apache 2 version apache2 -l # list compiled in Apache 2 modules apache2ctl -M # list loaded Apache 2 modules a2enmod # Add a module choosing it from a list. You'll have to restart Apache 2 then. jed /etc/apache2/apache2.conf If your webserver is too slow, you may have to adjust the value for MaxClients: if it's too low it will allow too few simultaneous accesses, if it's too high it will require too much resources to handle them. Try with MaxClients 50 . You can also add the expires module and set expiry periods for the cache of file types you don't change often on the server (like images): <IfModule expires_module> ExpiresActive On ExpiresDefault "access plus 2 days" ExpiresByType image/jpg "access plus 1 month" ExpiresByType image/jpeg "access plus 1 month" ExpiresByType image/gif "access plus 1 month" ExpiresByType image/png "access plus 1 month" ExpiresByType text/css "access plus 1 month" ExpiresByType application/pdf "access plus 1 month" ExpiresByType text/x-javascript "access plus 1 week" ExpiresByType application/x-shockwave-flash "access plus 1 month" ExpiresByType image/x-icon "access plus 1 year" </IfModule> If you use .htaccess files in your web directories to have them to be executed by the web server, you must be sure that AllowOverride is set to All for such directories: <Directory /var/www/> Options Indexes FollowSymLinks AllowOverride All Require all granted </Directory> jed /etc/apache2/ports.conf # Port listened by the webserver (normally 80) jed /etc/apache2/sites-enabled/default.conf # set DocumentRoot and create Virtual Hosts (vhosts). Older versions of Apache2 stores sites by default in /etc/apache2/sites-available/default Check Apache configuration for errors: apache2ctl configtest Restart Apache after modifying its configuration: /etc/init.d/apache2 restart See if Apache is running: netstat -plntu | grep apache # if you get a line containing the socket listened by the Apache webserver, then it's running See Apache error log: cat /var/log/apache2/error.log # In case you want to delete access and error logs for all websites. IMPORTANT: Note that you'll have to restart Apache to set up logs and have them running again # rm /var/log/apache2/* # If you can't see the phpmyadmin directory into /var/www/ then the package likely installed it into /usr/share/ , in this case you have to link it manually to access it from the web ln -s /usr/share/phpmyadmin /var/www/ PHP configuration on Apache 2: jed /etc/php4/apache2/php.ini # PHP 4 configuration jed /etc/php5/apache2/php.ini # PHP 5 configuration Make sure engine = On or PHP will not work at all. You may want to set short_open_tag = On if your PHP code uses short tags, which means < ? ... ? > instead of < ?php ... ? > You may specify the charset (character encoding) used by PHP generated pages. Note that it overrides charsets specified in the Apache configuration or in the HTML document (such as <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1" /> in HTML 4 or <meta charset="ISO-8859-1" /> in HTML 5), so you'd probably better leave it empty default_charset = "" however in case you need to set it, most popular settings are: default_charset = "UTF-8" or default_charset = "iso-8859-1" Show PHP version: php -v List compiled-in PHP modules: php -m To send e-mails from PHP scripts, you need to install sendmail apt-get install make apt-get install sendmail-bin sensible-mda sendmail-doc rmail sendmail echo "Subject: test"|/usr/lib/sendmail -v -F SENDER@MAILBOX.EXAMPLE.COM -t RECEIVER@MAILBOX.EXAMPLE.COM # send a test e-mail # Note that some antispam services blacklist servers who send e-mails identifying themselves (EHLO) as "localhost.localdomain". To change this information on your server, you have to edit /etc/hosts and set a proper domain name for your server as the first alias for the IP 127.0.0.1 You should check sendmail_path in PHP configuration (php.ini) and make sure it's either disabled or set to the default value of sendmail -t -i To send an e-mail from a PHP page you have to use the mail command, like in this example: mail($emailto, $emailsubject, $emailbody, "From: ".$emailfrom."\nReply-To: ".$emailreplyto."\nX-Mailer: ".$xmailer); Of course all sample variables used must be set properly. Show MySQL version: mysql -V See if Apache is running: netstat -plntu | grep mysql # if you get a line containing the socket listened by the MySQL server, then it's running Set MySQL root password: $ mysql mysql> SET PASSWORD FOR root@localhost = PASSWORD('newpassword'); Create an alternate user with root-like access: $ mysql -u root -pROOT_PASSWORD mysql mysql> CREATE USER 'USER_NAME'@'localhost' IDENTIFIED BY 'SET_PASSWORD_HERE'; GRANT ALL PRIVILEGES ON *.* TO 'USER_NAME'@'localhost'; FLUSH PRIVILEGES; Create a new mysql user and assign a database to him $ mysql -u root -pROOT_PASSWORD mysql mysql> insert into user (Host, User, Password, Select_priv) values ('localhost', 'USERNAME', password('USER_PASSWORD'), 'N'); Query OK, 1 row affected (0.00 sec) mysql> insert into db (Host, Db, User, Select_priv, Insert_priv, Update_priv, Delete_priv, Create_priv, Drop_priv, Grant_priv, References_priv, Index_priv, Alter_priv, Create_tmp_table_priv, Lock_tables_priv) values ('localhost', 'USER_DATABASE', 'USER_NAME', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y'); Query OK, 1 row affected (0.00 sec) mysql> quit $ mysqladmin -u root -pROOT_PASSWORD reload The user can access the assigned database: $ mysql -u USERNAME -pUSER_PASSWORD USER_DATABASE Remove an user: mysql> drop user USERNAME@HOSTNAME; Or mysql> delete from mysql.user where User='USERNAME' and Host='HOSTNAME'; mysql> revoke all privileges on *.* from USERNAME@HOSTNAME; mysql> revoke grant option on *.* from USERNAME@HOSTNAME; mysql> flush privileges; Show all MySQL users: mysql> select user, host from mysql.user; # show only user name and related host name for every user mysql> select * from mysql.user; # show detailed information for every user Show all databases: mysql> show databases; Show all tables in the current database: mysql> show tables; mysql> show table status; # provide information about every table Show all fields (coloumns) of a table: mysql> show fields from TABLE; # same as describe TABLE; mysql> show fields from TABLE from DB; # show fields of a table from another database mysql> show full fields from TABLE; # also lists privileges available to the current user Show all indexes (keys) of a table: mysql> show index from TABLE; mysql> show index from TABLE from DB; # show indexes of a table from another database Create an index (key) for a field (coloumn): mysql> alter table TABLE add index (`FIELD`) ; Count all records within a table: mysql> select count(*) from TABLE; Backup a database to a MySQL dump file (structure only): mysqldump -d -h localhost -u MYSQLUSER -pPASSWORD DATABASE > MYSQLDUMP.sql Backup a database to a MySQL dump file (structure and data): mysqldump -h localhost -u MYSQLUSER -pPASSWORD -c --add-drop-table --add-locks --all --quick --lock-tables DATABASE > MYSQLDUMP.sql Back up a MySQL database into a GZIP file: mysqldump -h localhost -u MYSQLUSER -pPASSWORD -c --add-drop-table --add-locks --all --quick --lock-tables DATABASE | gzip -9v > MYSQLDUMP_`date +%Y%m%d`.sql.gz Restore a database from a MySQL dump file: mysql -h localhost -u MYSQLUSER -pPASSWORD DATABASE < MYSQLDUMP.sql Delete a database: mysql> drop database if exists DATABASE; Delete a table: mysql> drop table if exists TABLE; Delete a field: mysql> alter table TABLE drop FIELD; Delete an index: mysql> drop index INDEX on TABLE; Empty a table without deleting it: mysql> truncate table TABLE; Change engine type: mysql> alter table TABLE type = MYISAM; # change the engine tipe of TABLE to MyISAM Analyzes indexes of a table: # for MyISAM and BDB tables only mysql> analyze table TABLE; Check a table for errors: mysql> check table TABLE; Attempt to repair a corrupted table mysql> repair table TABLE; Optimize a table (defragment and rebuild indexes): mysql> optimize table TABLE; Show MySQL server status: mysql> show status; Check running MySQL processes (queries): $ mysqladmin -h localhost -u root -pROOT_PASSWORD processlist Or, from the MySQL console: mysql> show processlist; Kill a MySQL process: $ mysqladmin -u root -pROOT_PASSWORD kill ID Or, from the MySQL console: mysql> kill ID; Quit MySQL console: mysql> quit Show MySQL version: mysql -V To restart MySQL: /etc/init.d/mysql restart # If MySQL fails to start make sure the directories /var/log/mysql and /var/log/mysqld are owned by mysql:adm Run a MySQL file and store output into another file: mysql -u USER --password=PASSWORD DATABASE_NAME < SOURCE.sql > DESTINATION.txt This may fix these errors that may happen after an upgrade: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' ERROR 1577 (HY000) at line 1: Cannot proceed because system tables used by Event Scheduler were found damaged at server start apt-get install php5-mysql mysql-server mysql-server-5.1 mv /etc/mysql/my.cnf /etc/mysql/my.cnf.bak1 mv /etc/mysql/my.cnf.dpkg-dist /etc/mysql/my.cnf /etc/init.d/mysql start apt-get install php5-mysql mysql-server mysql-server-5.1 Upgrading from PHP 4 to PHP 5: # Note: this will also upgrade from MySQL 4 to MySQL 5, if MySQL 4 is installed # Warning: newer versions of MySQL often use different formats for tables than earlier versions. In some cases you can attempt to make restore such tables compatible using repair table TABLE; and optimize table TABLE; You'd better dump all your tables before to upgrade, in case you'll need to restore them after. In fact you may need to DROP tables which are no longer working as expected and rebuild them. A common issue are tables containing floating point fields: queries containing comparisons based on those fields will no longer work as they used to. # stop the servers /etc/init.d/apache2 stop /etc/init.d/mysql stop # remove PHP 4 apt-get remove libapache2-mod-php4 php4 php4-gd php4-pear php4-mcrypt php4-mysql # You have to remove MySQL as well apt-get remove mysql-server # Install PHP 5 apt-get install php5 libapache2-mod-php5 php5-sqlite php-pear php5-gd php5-mcrypt # PHP 5 apt-get install php5-mysql # MySQL support for PHP 5 # Install MySQL again apt-get install mysql-server # Reinstall phpmyadmin (if you need it) apt-get install phpmyadmin # Removing PHP 4 will remove phpmyadmin as well, you'll have to reinstall it if you need it # Make sure that there are no references to PHP 4 left in the Apache configuration jed /etc/apache2/apache2.conf # Change <IfModule mod_php4.c> as <IfModule mod_php5.c> # Add .php5 to AddType application/x-httpd-php # Comment out <IfDefine HAVE_PHP4> ... </IfDefine> and everything between # restart the servers /etc/init.d/mysql restart /etc/init.d/apache2 restart Password protected directories in Apache: # Create a password file, it should be placed in a directory not served by the webserver (for example, /var/ , which normally is out of the htdocs directory) touch PATH/.htpasswd chmod 644 PATH/.htpasswd # Create users and assign passwords to them htpasswd -b PATH/.htpasswd USER PASSWORD # Create a password for every user. The same statement can be used to change a password of an already existing user htpasswd -D PATH/.htpasswd USER # Delete a user from the password file # Create an access control file in the directory containing the files with restricted access # Use Require valid-user to allow access to any user in the password file, or list specific users allowed to access the restricted files jed PATH_TO_PROTECT/.htaccess --- AuthUserFile PATH/.htpasswd AuthName "Enter Password" AuthType Basic <Limit GET POST> # Require valid-user Require user USER1 Require user USER2 </Limit> --- Web site statistics: apt-get install webalizer jed /etc/webalizer.conf # or jed /etc/webalizer/webalizer.conf # or another custom .conf file webalizer /var/log/apache2/access.log.1 # create a webalizer report for access.log.1 webalizer /var/log/apache2/example-com_access.log # create a webalizer report for example-com_access.log If you get the following error message: Error Opening file /usr/share/GeoIP/GeoIP.dat you can fix it installing the GeoIP database: apt-get install geoip-database To have statistics available on the web for many websites, create a different access log, webalizer configuration file and output directory, for every website and a configuration file, and call webalizer from a script. Example: # 1. Set Apache log files jed /etc/apache2/sites-available/default <VirtualHost www.example.com> [...] CustomLog /var/log/apache2/example-com_www_access.log combined [...] </VirtualHost> <VirtualHost my.example.net> [...] CustomLog /var/log/apache2/example-net_my_access.log combined [...] </VirtualHost> /etc/init.d/apache2 restart # 2. Create output directories mkdir /var/www/webalizer mkdir /var/www/webalizer/www.example.com mkdir /var/www/webalizer/my.example.net # 3. Assign ownership of the directories to www-data chown www-data:www-data /var/www/webalizer/www.example.com chown www-data:www-data /var/www/webalizer/my.example.net # 4. Create custom Webalizer configuration files jed /etc/webalizer/webalizer_www.example.com.conf LogFile /var/log/apache2/example-com_www_access.log OutputDir /var/www/webalizer/www.example.com Incremental yes HostName www.example.com HideSite *example.com HideReferrer example.com/ [...] jed /etc/webalizer/webalizer_my.example.net.conf LogFile /var/log/apache2/example-net_my_access.log OutputDir /var/www/webalizer/my.example.net Incremental yes HostName my.example.net HideSite *example.com HideReferrer example.com/ [...] # 5. Create a script that invokes webalizer for every website mkdir /etc/scripts # You can have a verbose version to be invoked from a shell window, and a quiet version to be launched from CronTab: jed /etc/scripts/webalizev.sh #!/bin/bash # Webalize (verbose) echo -e "\nWebalizer for www.example.com" webalizer -c /etc/webalizer/webalizer_www.example.com.conf echo -e "\nWebalizer for my.example.net" webalizer -c /etc/webalizer/webalizer_my.example.net.conf jed /etc/scripts/webalize.sh #!/bin/bash # Webalize (quiet) webalizer -c /etc/webalizer/webalizer_www.example.com.conf>/dev/null webalizer -c /etc/webalizer/webalizer_my.example.net.conf>/dev/null Sample lines to add to Crontab: 1 0,12 * * * /var/www/scripts/webalize.sh>/dev/null 21 6 * * * /var/www/scripts/webalize.sh>/dev/null The first line tells Crontab to invoke Webalizer every day at 00:01 and 12:01, the second line makes Crontab invoke Webalizer every day at 6:21 . This is because logs start and end at a certain hour of the day (for example, 06:26), and you'd miss all the hours between last call of Webalizer, every day. To know the exact time when your logs starts and stops check the head and tail of the previous log, for example: head --lines=10 /var/log/apache2/www.example.net_access.log.1 , tail --lines=10 /var/log/apache2/www.example.net_access.log.1 # If you want to reset information cached from previous runs of Webalizer (especially if you've set Incremental yes in the webalizer configuration file), delete the files webalizer.current and webalizer.hist in the Webalizer OutputDir. # Example: # rm /var/www/webalizer/webalizer.current # rm /var/www/webalizer/webalizer.hist ----- SQLite: apt-get install sqlite sqlite3 libsqlite3-dev # To fix these errors: sqlite_exec(): attempt to write a readonly database; sqlite_exec(): unable to open database file chmod 666 *.sdb chmod 777 the directory containing the .sdb file ----- FTP server: PROFTP: apt-get install proftpd (run as standalone) Create an user called "www" to access the www directory: adduser --home /var/www www Allow FTP access to the user "www": jed /etc/proftpd.conf Add the following lines in the user section of the proftpd configuration file: <Anonymous ~www> User www Group www DirFakeUser on ftp DirFakeGroup on ftp RequireValidShell off MaxClients 2 DisplayLogin welcome.msg DisplayFirstChdir .message <Directory *> <Limit READ WRITE STOR> AllowAll </Limit> </Directory> </Anonymous> /etc/init.d/proftpd restart VSFTP: # VSFTP allows virtual users, besides real shell users and anonymous users (you can simulate a chroot'ed FTP) apt-get install vsftpd apt-get install libpam-pwdfile Create a password file for virtual users: # Note: htpasswd only allows passwords up to 8 characters long htpasswd -bc /etc/vsftpd_passwd USERNAME PASSWORD Additional users can be created without the -c parameter. The same statement can be used to change the password for an existing user: htpasswd -b /etc/vsftpd_passwd USERNAME PASSWORD If you want to enter the password manually, omit the -b parameter: htpasswd /etc/vsftpd_passwd USERNAME Use this command to remove a virtual user: htpasswd -D /etc/vsftpd_passwd USERNAME You'll need to create a directory for each virtual user (unless you configure vsftp otherwise): First, create the main directory for all virtual users: mkdir /var/www/ftp Then create a directory for each virtual user with proper read/write permissions: mkdir /var/www/ftp/USERNAME chmod 755 /var/www/ftp/USERNAME chown ftp /var/www/ftp/USERNAME Configure PAM (Pluggable Authentication Modules aka X/Open Single Sign-on) for vsftp: jed /etc/pam.d/vsftpd # Log in using htpasswd password file auth required pam_pwdfile.so pwdfile /etc/vsftpd_passwd account required pam_permit.so jed /etc/vsftpd.conf # Config file /etc/vsftpd.conf listen=YES #listen_ipv6=YES anonymous_enable=NO local_enable=YES write_enable=YES local_umask=022 anon_upload_enable=NO anon_mkdir_write_enable=NO dirmessage_enable=YES xferlog_enable=YES connect_from_port_20=YES #chown_uploads=YES #chown_username=ftp xferlog_file=/var/log/vsftpd.log #xferlog_std_format=YES idle_session_timeout=600 data_connection_timeout=120 #nopriv_user=ftp #async_abor_enable=YES #ascii_upload_enable=YES #ascii_download_enable=YES ftpd_banner=Welcome #deny_email_enable=YES #banned_email_file=/etc/vsftpd.banned_emails chroot_local_user=YES #chroot_list_file=/etc/vsftpd.chroot_list #ls_recurse_enable=YES secure_chroot_dir=/var/run/vsftpd pam_service_name=vsftpd rsa_cert_file=/etc/ssl/certs/vsftpd.pem guest_enable=YES virtual_use_local_privs=YES #user_config_dir=/etc/vsftpd_user_conf #guest_username=ftp user_sub_token=$USER local_root=/var/www/ftp/$USER hide_ids=YES #cmds_allowed=PASV,BYE,LIST,ABOR,CWD,NLST,PORT,PWD,QUIT,RETR,SIZE,TYPE # Note that the option user_config_dir lets you create custom vsftp configuration files for each virtual users # You can create a config file with the name of the virtual user within the specified path jed /etc/vsftpd_user_conf/USERNAME # VSFTP extra config for user USERNAME local_root=/var/www/example.com/www/USERNAME Restart VSFTP to make it work with the new users or configuration: /etc/init.d/vsftpd restart ----- Samba: apt-get install samba smbfs smbclient samba-doc mkdir /public mkdir /public/shared chmod -v 0777 /public chmod -v 0777 /public/shared jed /etc/samba/smb.conf --- # samba configuration file [global] allow hosts = 192.168.0.0/255.255.255.0 workgroup = network server string = %h server (Samba %v) log file = /var/log/samba/log.%m max log size = 1024 ; security = user encrypt passwords = true passdb backend = tdbsam guest guest account = nobody invaluid users = root preserve case = yes short preserve case = yes ; include /home/samba/etc/smb.conf.%m socket options = TCP_NODELAY [homes] comment = Home directories browseable = no writable = no create mask = 0700 directory mask = 0700 [shared] comment = Public shared directory path = /public/shared public = yes browseable = yes ; read only = no writable = yes ; printable = no create mask = 0766 directory mask = 0766 guest ok = yes ; valid users = user_name --- testparm /etc/samba/smb.conf /etc/init.d/samba restart smbpasswd -a nobody New SMB password: Retype new SMB password: log file (default): /var/log/samba/log.%m smbtree # find Windows machines ----- Linux Shell scripts: First line headers: Bourne Shell (sh) #!/bin/sh Bourne Again Shell (bash) #!/bin/bash C-Shell (csh) #!/bin/csh Korn Shell (ksh) #!/bin/ksh Perl #!/usr/bin/perl PHP #!/usr/bin/php Python #!/usr/bin/python Turbo Shell (tcsh) #!/usr/bin/tcsh TCL/Tk #!/usr/bin/wish -f Set the executable flag: chmod +x FILE # declares the file as executable ----- Invoke Linux shell commands from PHP: PHP can perform calls to the Linux shell using the shell_exec() function: $output = shell_exec('ls -laF'); echo '<pre>'.$output.'</pre>'; or enclosing the Linux shell command between backticks: $output = `ls -laF`; echo '<pre>'.$output.'</pre>'; PHP also has a system() function that returns the value returned by the Linux shell command and the last line of output of such command: $last_line = system('ls -laF', $returned_value); ----- GUI commands # Commands for the Graphical User Interface xwinifo # Enter this command and then click on any window to get information about it such as size in pixels and color depth. You may also click on the desktop. sudo rm -rf  ~/.local/share/Trash/files/* # Empty trashcan ----- Fortune cookies: apt-get install fortunes /usr/games/fortune # return a fortune cookie (fetches a random sentence (% separated lines) from a random file in /usr/share/games/fortunes/ ) /usr/games/fortune PATH/FILE # return a fortune cookie fetching a random sentence (% separated lines) from the given FILE To create your own fortune files: - Create a text file separating each fortune cookie with lines containing only a percent (%) sign. - Create a database file for that text file, with the same file name followed by a .dat extension, and place it in the same directory: strfile PATH/FILENAME PATH/FILENAME.dat ----- ASCII Art: Labyrinth: c=(\/ \\); n=${#c[@]}; clear; printf "%s" "${c[RANDOM%n]"{1..1920}"}" c=(\/ \\); n=${#c[@]}; clear; while :; do printf -- "${c[RANDOM%n]}"; done # Endless labyrinth (break with CTRL+C) apt-get install figlet figlet "Hello" # Type Hello using ASCII Art apt-get install cowsay /usr/games/cowsay "Hello" # Show a cow saying Hello /usr/games/cowthink "Hello" # Show a cow thinking Hello /usr/games/cowsay -f CHARACTER "Hello" # Use alternate CHARACTERs, for example: tux, moose, sheep, milk /usr/games/cowthink -f CHARACTER "Hello" # Use alternate CHARACTERs, for example: tux, moose, sheep, milk ls /usr/share/cowsay/cows # Show available cows (characters) /usr/games/fortune|/usr/games/cowsay # Fortune teller cow (requires fortune cookies to be installed) /usr/games/cowsay -f "$(ls /usr/share/cowsay/cows | sort -R | head -1)" "$(/usr/games/fortune -s)" # Random character tells fortune apt-get install sl sl # show a running steam locomotive in ASCII Art sl -c # show a small steam locomotive sl -F # show a flying steam locomotive sl -a # show people crying for help into a steam locomotive sl -a # allow the animation to be interrupted with CTRL+C ----- Text editors: Jed apt-get install jed jed FILE.txt # Create or edit FILE.txt jed FILE.txt -g LINE # Open FILE.txt for editing, set cursor position at the given LINE jed FILE.txt -s "TEXT" # Open FILE.txt for editing, set cursor position at the firt occurrence of the given TEXT Vim # While Jed is an easy and intuitive text editor, it might happen that it's not available in the system on which you are working # In this case you'll likely have to use vim instead which is powerful in trained hands but much less intuitive # This is a list of some basic and useful commands Cursor keys / h, l, j, k : Cursor movement (left, right, down, up) b : Jump to the word on the left w : Jump to the word on the right ( : Jump to the sentence on the left ) : Jump to the sentence on the right ^ / 0 : Jump to the beginning of the line $ / gm : Jump to the end of the line - : Jump to the first non blank character of the previous line + : Jump to the first non blank character of the next line { : Jump to the previous paragraph } : Jump to the next paragraph [[ : Jump to the beginning of the file ]] : Jump to the end of the file [LINE_NUMBER]G / [LINE_NUMBER]gg : Jump to the given LINE NUMBER [VALUE]% : Jump to the specified position of the document [COLOUMN]| : Jump to the specified COLOUMN in the current line [LINE]H : Jump to the given LINE visible in the current window, from the top [LINE]L : Jump to the given LINE visible in the current window, from the bottom M : Jump to the line in the middle of the current window zt : Place current line at the top of the window zz : Place current line at the middle of the window zb : Place current line at the bottom of the window i : Enter Insert mode I : Jump at the beginning of the line and enter Insert mode s : Delete the character under the cursor and enter Insert mode R : Enter Replace mode ESC : Exit Edit mode (either Insert or Replace (Overwrite)) Commands in Edit mode (^ means CONTROL): ^w : Delete characters from cursor position to the beginning of the current word ^u : Delete characters from cursor position to the beginning of the current line ^t : Add a Tab at the beginning of the current line (Add indentation) ^r : Remove a Tab from the beginning of the current line (Remove indentation) r : Replace a single character at the cursor position o : Create a new empty line below the current one and enter Insert mode O : Create a new empty line above the current one and enter Insert mode gu[CURSOR_KEY] : Change a single character to lowercase in the direction specified with the cursor key gU[CURSOR_KEY] : Change a single character to uppercase in the direction specified with the cursor key x : Remove the character at the cursor (Delete) X : Remove the character before than the cursor (Backspace) D : Delete until the end of the current line dd : Delete the current line J : Join the current line with the next one adding a space inbetween Y / yy : Copy the current line P / p : Paste the line in the clipboard ga : Show the ASCII value of the character at the cursor position /[TEXT] : Search for the given TEXT n : Repeat last search, forward N : Repeat last search, backwards :s/[SEARCH]/[REPLACE]/ : Search and Replace the first occurrence in the current line only, case sensitive :s/[SEARCH]/[REPLACE]/g : Search and Replace every occurrence in the current line only, case sensitive :%s/[SEARCH]/[REPLACE]/g : Search and Replace every occurrence in the whole file, case sensitive :%s/[SEARCH]/[REPLACE]/gI : Search and Replace every occurrence in the whole file, case sensitive :%s/[SEARCH]/[REPLACE]/gi : Search and Replace every occurrence in the whole file, case insensitive :%s/[SEARCH]/[REPLACE]/gic : Search and Replace every occurrence in the whole file, case insensitive, and ask for confirmation :%s/\s\+$// : Remove trailing spaces in the whole file :%s/^\s\+// : Remove leading spaces in the whole file :set number / :set nu : Display line numbers :set nonumber / :set nonu : Turn off line numbers :set number! / :set nu! : Toggle line numbers :set relativenumber : Display line numbers relative to the cursor line :set norelativenumber : Remove line numbers relative to the cursor line :set relativenumber! : Toggle line numbers relative to the cursor line # If number and relativenumber are both enabled the absolute line number will be shown in the cursor line (instead of 0) :set list : Display special characters :set nolist : Hide special characters :set list! : Toggle special characters :set listchars=eol:$,tab:>-,trail:~,nbsp=! : Configure how to show special characters :help listchars : Display help for listchars (quit with :q) :syntax on : Enable syntax :syntax off : Disable syntax :set syntax=[LANGUAGE] : Display syntax for the specified language (example: html, php, whitespace) :w : Save current file :wq / :x / ZZ : Save current file and quit :q : Quit (it only quits if the current file is saved, otherwise returns an error message) :q! : Quit without saving (doesn't ask for confirmation) ----- Easter eggs: $ apt-get moo (__) (oo) /------\/ / | || * /\---/\ ~~ ~~ ...."Have you mooed today?"... $ apt-get -h [...] This APT has Super Cow Powers. $ aptitude -h [...] This aptitude does not have Super Cow Powers. $ aptitude moo There are no Easter Eggs in this program. $ aptitude -v moo There really are no Easter Eggs in this program. $ aptitude -vv moo Didn't I already tell you that there are no Easter Eggs in this program? $ aptitude -vvv moo Stop it! $ aptitude -vvvv moo Okay, okay, if I give you an Easter Egg, will you go away? $ aptitude -vvvvv moo All right, you win. /----\ -------/ \ / \ / | -----------------/ --------\ ---------------------------------------------- Happy? $ aptitude -vvvvvv moo What is it? It's an elephant being eaten by a snake, of course. # Note: aptitude's easter egg is a reference to "The Little Prince" written in 1943 by Antoine de Saint-Exupéry. -----






Page issued on 27-Feb-2017 11:40 GMT
Copyright (c) 2017 Geody - Legal notices: copyright, privacy policy, disclaimer