Databases, Systems & Networks

Source: microhowto.info

Content

• 1 Objective
• 2 Background
• 3 Symptoms
• 4 Scenario
• 5 Investigation
  • 5.1 Strategy
  • 5.2 Inspecting the tables
  • 5.3 Monitoring traffic flow using counters
  • 5.4 Monitoring traffic flow using the LOG target
    

Objective

To ensure that iptables has been correctly configured.

Background

iptables is a component of the Linux kernel that allows IPv4 traffic to be manipulated as it traverses the network stack. Its two main uses are:

• packet filtering (firewalling) and
• network address translation (NAT).

The behaviour of iptables is controlled by rules, each of which specifies the action to be taken if a packet meets a particular set of conditions. The rules are organised into chains, and the chains into tables. Chains may be either built-in or user-defined.

For more information about the architecture and configuration of iptables see:

• the netfilter/iptables project homepage,
• the iptables tutorial and
• the Packet Filtering and NAT HOWTOs

Symptoms

The most likely symptoms of an iptables configuration error are:

• traffic being dropped or rejected,
• traffic not being NATted when it should have been,
• traffic being NATted when it shouldn’t have been, or
• traffic being NATted to the wrong address.

A wide variety of other effects are possible, but unlikely unless the configuration is an unusual one.

Scenario

Suppose that a machine has been configured to act as a boundary router between a local area network (connected to interface eth0 with the address 192.168.0.1/24) and the public Internet (connected to interface ppp0 with the address 203.0.113.144/32). The default gateway is 203.0.113.1. Because the local area network uses a private address range, iptables on the boundary router has been configured to SNAT them to its public IP address.

In order to test this configuration you have attempted to ping a machine on the public Internet (198.51.100.1) from a machine on the local area network (192.168.0.2), but this has failed.

Lire la suite…

Source: defendagainstddos.wordpress.com

On the 25th of January one of my sites was struggling to stay up, my “Dos Deflate” emails were popping into my inbox at a great frequency.

A quick run of the following command:

netstat -tn 2>/dev/null | grep :80 | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -nr | awk ‘{print $2}’

Gave me a list of addresses connected to the site. The number of addresses was way way above the number normally connected to the site.  A site which although modestly doing 100,000s of page views per day will normally have only 300-500 port 80 connections at any one instance.  This time however we had a 2 or 3 thousand.

A quick copy and paste into the following tool http://software77.net/geo-ip/multi-lookup/ (max 2000 ips per lookup) which is very usefully for bulk ip address to location lookups, yielded the following

1.26.199.253 # CN China
 1.81.157.62 # CN China
 1.83.168.34 # CN China
 1.180.57.96 # CN China
 1.26.201.167 # CN China
 1.62.140.66 # CN China
 1.189.82.78 # CN China
 1.193.76.146 # CN China
 1.26.241.209 # CN China
 1.190.209.36 # CN China
 1.189.162.213 # CN China
 1.31.57.105 # CN China
 1.62.17.196 # CN China
 1.58.137.111 # CN China
 1.83.229.4 # CN China
 1.180.10.45 # CN China
 1.180.5.200 # CN China
 1.26.169.201 # CN China
 1.180.187.10 # CN China
 1.190.2.102 # CN China
 1.25.134.125 # CN China
 1.62.161.233 # CN China
 1.182.28.181 # CN China
 (And on and on)

We clearly had an issue with China, but a quick log in to Google Analytics indicated almost no real-time traffic visiting the site.

Due to the sheer volume of connections for the next 45 mins, I was regularly running this command pasting into excel, then bulk iptabling the the ip address as well as the 65,000 neighbouring addresses.

sudo iptables -A INPUT -s 59.56.0.0/16 -m comment –comment “China IP” -j DROP
sudo iptables -A INPUT -s 1.26.0.0/16 -m comment –comment “China IP” -j DROP

etc etc.

Lire la suite…

IPTABLES for WordPress

Thanks to:
http://bencane.com/2012/09/17/iptables-linux-firewall-rules-for-a-basic-web-server/
http://www.linux.org/threads/base-iptables-rules-that-will-apply-to-virtually-any-web-server.10/ (used this script with modifications)

NOTES:

Here is a simple script that allows all outbound connections and the inbound connections coming back from those outbound connections (conntrack). Also port 80 and port 22 and port 443 are allowed in. A few more rules as well. Some rules are commented out for your use. Make sure you have an alternate connection because if you block yourself out of ssh, you might be out of luck (restarting the pc/server will clear the rules, unless you have a setting that says on boot read these iptables)

CLEAR ALL:

Clear all rules (this is good to keep handy, maybe save it as a script called iptables-clear-all.sh):

iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
 
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT

Lire la suite…

Stopping a DDOS (distributed denial of service attack) or DOS (denial of service attack) is no simple task.  Frequently, these attacks become more than just a nuisance, they completely immobilize your server’s services and keep your users from using your website.

We’ve found a few common sense ways to help ease the pain of DDOS and/or DOS attacks.  While no method is fool proof, we certainly can minimize the profound effect these attacks have on your users and subsystems.

Identify the Source

Good luck with that one.  Many DDOS and DOS attacks are from roaming IP addresses.  A distributed denial of service attack can come from many different IP addresses and it quickly becomes impossible for the Linux system administrator to isolate and confine each IP with a firewall rule.

Wikipedia does a great job of describing the various types of attacks here: http://en.wikipedia.org/wiki/Denial-of-service_attack.  For the purpose of this tutorial, I’ll leave the research on the types of attacks up to you, and address the most common form that we’ve encountered over the years, the Apache directed DDOS or DOS attack.

Apache Based Attacks

Symptoms of the Apache DDOS or DOS attack:

• Website(s) serve slow
• You notice hanging processes
• Apache Top tells you that the same IP address is requesting a system resource
• The system resource continues to multiplex, causing more processes to spawn
• The Command:

  netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

• Says that you have a few too many connections to feel comfortable with.

The end result:

• Apache goes down
• System load goes sky high
• Server stops responding
• You cant ssh to the server node
• You’ve lost connectivity completely and a reboot is mandatory in order to restore access to the system

Preventative Measures and Counter Measures:

• Enable SYN COOKIES at the kernel level

  echo 1 > /proc/sys/net/ipv4/tcp_syncookies

• Enable and Configure iptables to prevent the attack or at least work to identify the attack

  /sbin/iptables -N syn-flood
  /sbin/iptables -A syn-flood -m limit --limit 100/second --limit-burst 150 -j RETURN
  /sbin/iptables -A syn-flood -j LOG --log-prefix "SYN flood: "
  /sbin/iptables -A syn-flood -j DROP

• Install the APF firewall to work to identify risky behavior
  • APF stands for Advanced Policy Firewall.  Its a rock solid firewall that normally plays nice with iptables.  You can grab a the most recent copy here: http://www.rfxn.com/projects/
• Install (D)DosDeflate
  • Great software, rock solid, and plays nice with either APF or iptables.  Install and configure the service in seconds using the commands below.  Edit the .conf file to utilize whichever flavor of firewall you’d like to integrate it with.  Set a few configuration settings and you’re done.
  • To Install (D)DosDeflate:

    wget http://www.inetbase.com/scripts/ddos/install.sh
    chmod 0700 install.sh
    ./install.sh

• If it doesnt workout, its simple to uninstall too.  To uninstall:

  wget http://www.inetbase.com/scripts/ddos/uninstall.ddos
  chmod 0700 uninstall.ddos
  ./uninstall.ddos

So a few tools are outlined above.  We’ve found that this will stop 90% of the attacks that are out there.  Some nice firewall rules above your server (at the router or switch level) also help.  Most of the time we can identify suspicious traffic before it even hits your servers, so a shameless plug here is probably in order.

Source: Liquid Communications

Graphs on Munin

Enable and disable plugins on each node

rm /etc/munin/plugins/diskstats
service munin-node restart

ln -s /usr/share/munin/plugins/diskstats /etc/munin/plugins/diskstats
service munin-node restart

Process Backgrounded
2014/03/10-15:59:47 Munin::Node::Server (type Net::Server::Fork) starting! pid(32231)
Resolved [*]:4949 to [::]:4949, IPv6
Not including resolved host [0.0.0.0] IPv4 because it will be handled by [::] IPv6
Binding to TCP port 4949 on host :: with IPv6
2014/03/10-16:00:04 CONNECT TCP Peer: "[::ffff:203.28.51.227]:45965" Local: "[::ffff:50.23.111.122]:4949"
2014/03/10-16:05:04 CONNECT TCP Peer: "[::ffff:203.28.51.227]:46109" Local: "[::ffff:50.23.111.122]:4949"
2014/03/10-16:10:04 CONNECT TCP Peer: "[::ffff:203.28.51.227]:46109" Local: "[::ffff:50.23.111.122]:4949"