Terminal – Find Duplicate Files (based on size first, then MD5 hash)
find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separateFind Duplicate Files (based on size first, then MD5 hash)
This dup finder saves time by comparing size first, then md5sum, it doesn’t delete anything, just lists them.
Terminal – Alternatives
find -type f -exec md5sum '{}' ';' | sort | uniq --all-repeated=separate -w 33 | cut -c 35-Calculates md5 sum of files. sort (required for uniq to work). uniq based on only the hash. use cut ro remove the hash from the result.
find -type d -name ".svn" -prune -o -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type d -name ".svn" -prune -o -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separateImprovement of the command « Find Duplicate Files (based on size first, then MD5 hash) » when searching for duplicate files in a directory containing a subversion working copy. This way the (multiple dupicates) in the meta-information directories are ignored.
Can easily be adopted for other VCS as well. For CVS i.e. change « .svn » into « .csv »:
find -type d -name ".csv" -prune -o -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type d -name ".csv" -prune -o -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate
find -not -empty -type f -printf "%-30s'\t\"%h/%f\"\n" | sort -rn -t$'\t' | uniq -w30 -D | cut -f 2 -d $'\t' | xargs md5sum | sort | uniq -w32 --all-repeated=separate
Finds duplicates based on MD5 sum. Compares only files with the same size. Performance improvements on:
find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separateThe new version takes around 3 seconds where the old version took around 17 minutes. The bottle neck in the old command was the second find. It searches for the files with the specified file size. The new version keeps the file path and size from the beginning.
find . -type f -exec md5 '{}' ';' | sort | uniq -f 3 -d | sed -e "s/.*(\(.*\)).*/\1/"This works on Mac OS X using the `md5` command instead of `md5sum`, which works similarly, but has a different output format. Note that this only prints the name of the duplicates, not the original file. This is handy because you can add `| xargs rm` to the end of the command to delete all the duplicates while leaving the original.
Source: npm
Splits OpenVPN (.ovpn) files into separate files for private key, user+ca certificates and tls-auth key, for use with network-manager in debian/ubuntu.
openvpn-config-splitter can be installed using npm:
# NPM:
npm install -g openvpn-config-splitter
# Install globally
$ npm install -g openvpn-config-splitter
# Run it, specifying your unsplit OpenVPN configuration file
$ ovpnsplit path/to/some/config.ovpn
# Config is now split into separate files, new configuration
# linking to the split files has been generated
$ ls path/to/some
ca.crt client.key client.ovpn client.split.ovpn ta.key user.crt
var fs = require('fs'),
configPath = '/some/path/to',
splitter = require('openvpn-config-splitter');
var paths = {
'caCert': configPath + '/openvpn-ca.crt',
'userCert': configPath + '/openvpn-user.crt',
'privateKey': configPath + '/openvpn-private.key',
'tlsAuth': configPath + '/openvpn-tls.key'
};
fs.readFile(configPath + '/config.ovpn', function(err, originalConfig) {
if (err) {
console.error('Could not read file (' + err.path + ')');
process.exit(1);
}
splitter.split(originalConfig, paths, function(err, parts, missing) {
if (err) {
console.error(err);
process.exit(1);
}
/**
* `parts` now contain the matched parts of the config + new config
* (caCert, userCert, privateKey, tlsAuth, config)
*
* `missing` is an array containing the parts that were NOT found -
* use this if you want to warn the user or fall back if you require
* a specific part to be present
*/
// Want to write the split files?
splitter.writeToFiles(parts, paths, function(err) {
if (err) {
console.log(err);
process.exit(1);
}
console.log('Hooray, we split the files and wrote them to disk!');
});
});
});# Modify this file accordingly for your specific requirement.
# http://www.thegeekstuff.com
# 1. Delete all existing rules
iptables -F
# 2. Set default chain policies
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT DROP
# 3. Block a specific ip-address
#BLOCK_THIS_IP="x.x.x.x"
#iptables -A INPUT -s "$BLOCK_THIS_IP" -j DROP
Lire la suite...
Introduction
Servers, by definition, are implemented as a means of providing services and making applications and resources accessible to users. However, any computer connected to the internet is inevitably targeted by malicious users and scripts hoping to take advantage of security vulnerabilities.
Firewalls exist and should be used to block access on ports not being utilized by a service, but there is still the question of what to do about services that you want access to, but do not want to expose to everybody. You want access when you need it, but want it blocked off otherwise.
Port knocking is one method of obscuring the services that you have running on your machine. It allows your firewall to protect your services until you ask for a port to be opened through a specific sequence of network traffic.
In this guide, we will discuss how to implement port knocking as a method of obscuring your SSH daemon on an Ubuntu 12.04 VPS using the knockd package.
Note: This tutorial covers IPv4 security. In Linux, IPv6 security is maintained separately from IPv4. For example, « iptables » only maintains firewall rules for IPv4 addresses but it has an IPv6 counterpart called « ip6tables », which can be used to maintain firewall rules for IPv6 network addresses.
If your VPS is configured for IPv6, please remember to secure both your IPv4 and IPv6 network interfaces with the appropriate tools. For more information about IPv6 tools, refer to this guide: How To Configure Tools to Use IPv6 on a Linux VPS
Lire la suite…