Simulating bad network conditions on Linux

Sometimes, your network is just too good.

Today I ran into this issue as I was testing an application running off a VM in the local network. Latency and bandwidth were excellent, as you’d expect, but nowhere near the conditions you’d encounter over the internet. Testing in these conditions is unrealistic and can lead to underestimating issues your users will experience with your app once it’s deployed.

So let’s change that and add artificial latency, bandwidth limitations, and even drop a few packets, using tc.

Just put the following script in /etc/init.d, modify the values to fit your needs, make it executable, and run /etc/init.d/traffic_shaping.sh start to degrade performance accordingly.

I originally found the script on top web hosts’ website, and added a few things. Props!

A gentle introduction to GNU screen

You probably heard of GNU screen. It’s handy, ubiquitous, and dead simple. Here’s how to use it!

Open a terminal and type:

screen

You’re welcomed by an introduction message, press enter, and… You’re in a shell. Uh?

Screen is simple

screen is a terminal manager, so it’s logical that the first thing you see when you start it is a terminal.

This terminal is as vanilla as the terminal we started from. Just try it:

See? No black magic here, simply a terminal.

Screen is simple

The only difference is that Ctrl+a is now a special key combination that you can use to invoke screen‘s commands.

So let’s take a break and quit screen. Type Ctrl+a to let screen know you want its attention, then d, as in detach. There! You’re back in your first terminal.

Let’s go back in screen and learn some more! Just type:

screen -r

The -r stands for reattach: screen will re-open the last session, the one we detached from. You can see the results of the commands we entered earlier are still here.

We just saw a great feature of screen: the ability to log out and log back in without losing anything. Do you have something long to on a server? SSH into the server, launch screen, launch the task, detach from screen, log out from SSH, go back home, enjoy a good dinner and a well-deserved night of sleep, come back to work, SSH into the server, launch screen -r, and it’s just as if you never left.

Screen is simple

You can already use screen just like that, but let’s just see another nifty feature: multiple terminals!

In screen, type Ctrl-a, then ‘c’, short for create. You’re in a shell. Uh?

Screen is simple

You just created another terminal. screen can manage plenty of simultaneous terminals, not just one. To see a list of them, type Ctrl+a, then the quote symbol “, and you will see your two terminals. Just use the arrows to select which one you want to open.

There you go, you know screen! See, I told you it was simple.

Misc useful commands

Do you want to change the name of a terminal in screen‘s list? In that terminal, Ctrl+a and A.
Do you want to go directly to a specific terminal? Ctrl+a and its number.
Do you want to go to the previous/next terminal? Ctrl+a and p or n.
Do you want to switch to the previous terminal quickly? Ctrl+a and Ctrl+a.
Do you want to remap Ctrl+a to another key, say Ctrl+b? Just put escape ^b in your .screenrc.

Set up a mail server on Amazon EC2

This post will explain how to set up a Postfix mail server on an EC2 instance.

First, a word of warning: Amazon IPs generally aren’t highly considered, spam-wise. Meaning that even if you take all the precautionary steps, your emails might end up in spam folders. If email is business-critical for you, you might want to consider other options: host your mail server somewhere else? Use something like SendGrid?

This said, let’s dive in!

Prerequisites

I assume you have the following:

  • A domain name, with control over the DNS records
  • An EC2 account

Pick an AMI

Let’s start by creating an EC2 instance. I began with the vanilla AWS Linux micro instance, which seem to be somehow Fedora-based, and it was a pain. Now do yourself a favor and pick a Debian-based OS. It will make things much more easier.

Eric Hammond and Canonical themselves provides Debian and Ubuntu AMIs, which are a great first step. You can even bypass the whole Postfix config by using one of these AMIs.

Assign an elastic IP to the instance you launched. We will need it for the DNS setup.

Configure Postfix

Now’s the time to be very lazy and just redirect you to Ivar Abrahamsen‘s excellent howto on setting up Postfix. Actually, most of what I’m writing right now can be found on his howto, but let’s not stop at technicalities.

Configure your DNS

The most important step in having your email properly delivered is in your DNS configuration.

The first step is to define an A record for your Amazon Elastic IP, for example mail.mydomain.com. This will be used to set up a reverse DNS on your web server, so that other SMTP servers know that you’re not a spam relay.

Then add an MX record to the address you just defined, for example mail.mydomain.com. Now each SMTP server sending mail to mydomain.com will contact mail.domain.com, which in turn points to your EC2 instance. Awesome!

The next step is to modify your SPF record. I’ll let you work out the details with the spec and Ivar’s howto, and as an example here is the SPF record for remaildr:

remaildr.com.        1800    IN    TXT    "v=spf1 mx ip4:50.16.218.96 include:mx.ovh.com ~all"

This SPF allows MX servers and the IP address 50.16.218.96 (i.e. the EC2 instance) to send mail for remaildr.com. Only “MX” should be enough, no need for the IP in particular ? Well, I thought so, but it didn’t work so I added the IP address. Now it works. If anyone has an idea why, I’m all ears.

The include:mx.ovh.com is automatically added by OVH themselves and is not a problem in our case.

You can use the dig command to check if your DNS settings are properly set. For example, the SPF field was retrieved with a:

$ dig remaildr.com in txt

As a bonus, you might be interested in setting up DKIM (cryptographic email signing), a half of which takes place in your DNS. I’ll once again refer you to Ivan’s howto because it’s that good.

Tell Amazon you’ll be sending emails

By default, Amazon limits the amount of email you can send from an instance. You can ask them to remove that limitation very easily though, through that page.

This form also allows you to set up the reverse DNS I was telling you about. Go on, do it! Amazon usually answers to this form within 1-2 days.

Done!

That’s it!

Do you end up in spam folders? Try the test at AllAboutSpam, and check if everything’s alright. It covers about any issue your server might have.

A bit of Terminal-fu

Yesterday, my little brother was typing something on a linux bash and suddenly went back to the beginning of the line with a simple Ctrl-a. It blew my mind.

I don’t know why, but it never occurred to me to look for shortcuts in Terminal (and bash in general), other than Ctrl-c for badly behaving processes. I just cursed myself and frantically typed on the left arrow key each time I typed cd instead of cp.

Turns out bash has plenty of them, and here are a few useful ones I’m really glad to know now:
Beginning of the line: Ctrl+a
End of the line: Ctrl+e
Delete the word under the cursor or before: Ctrl+w
Delete all chars before the cursor: Ctrl+u
Delete all chars after the cursor: Ctrl+k

Apparently these shortcuts work in a lot of text fields, everywhere.

You can also move the cursor word-by-word by typing Esc, then f or b. It’s not really practical though, so you’d be better of remapping the following keys in Terminal Settings > Settings > Keyboard:
control cursor left: \033b (\033 is actually Esc)
control cursor right: \033f

Now you can swiftly move word-by-word with Alt+left arrow and Alt+right arrow.

I guess the lesson here is there’s always to learn from your siblings — especially the geeky ones.

Get database size in phpMyAdmin

It looks like phpMyAdmin doesn’t include a way to see how much space takes the MySQL database. I found that a bit weird, but hey, if phpMyAdmin doesn’t do it, let’s cut to the chase and go SQL!

The most straightforward way to get your database size from phpMyAdmin is the following SQL query:

SELECT table_schema "Data Base Name",
sum( data_length + index_length ) / 1024 / 1024 "Data Base Size in MB"
FROM information_schema.TABLES GROUP BY table_schema ;

Originally from the MySQL forums.