Downloading Files With Python's Requests Module

19 Jan 2015 http://docs.python-requests.org/en/latest/user/quickstart/#binary-response-content

From the example:

>>> from PIL import Image
>>> from StringIO import StringIO
>>> i = Image.open(StringIO(r.content))

Nmap Not Finding Hostnames

16 Jan 2015 http://serverfault.com/a/153779

The secret ended up being running the command as sudo. It's odd how "socket.gethostbyaddr()" worked just fine without sudo, but was slower than nmap.

Slow Disk Performance On Dell r620

07 Jan 2015

Update:

I installed openmanage with the instructions on this page and used it to check/update bios settings: http://linux.dell.com/repo/community/deb/OMSA_7.1/

The fix ended up being BIOS power profile settings. It was set to Performance Per Watt (DAPC) and needed to be set to "Performance".

It's very interesting that the BIOS power profile settings didn't seem to affect Ubuntu 14.10's disk read speed.

Troubleshooting:

I have three of the same server. All the servers are Dell r620's with a PERC H710 Mini RAID controller (21.2.0-0007_A04 firmware) and have the exact same hard drives (Seagate ST300MM0006) in RAID 1 configurations. I'm seeing about 50% worse disk read performance on some versions of Ubuntu.

They only differences I can find in the hardware:

The BIOS version on the poorly performing machines is 1.4.8, the machine that's performing well is on 1.6.0. The BIOS change log doesn't appear to have any changes between the two versions that would affect anything.
The chipset version on the RAID controller of the poorly performing machines is rev 01 (ChipRevision: B0). The machine that's performing well has the rev 05 chipset (ChipRevision: D1). The motherboard chipset revision is also different.

Here are the benchmark results for the different Ubuntu versions and RAID controller chipsets:

(mysql read speed test - using the same version of mysql across all 3 machines)
sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=test --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run:

(13.04 - rev 05): read/write requests: 4996362 (83271.11 per sec.)
(14.04 - rev 01): read/write requests: 1906520 (31773.58 per sec.)
(14.10 - rev 01): read/write requests: 5166798 (86111.50 per sec.)

dd if=/dev/zero of=/tmp/output bs=8k count=10k; rm -f /tmp/output:

(13.04 - rev 05): 83886080 bytes (84 MB) copied, 0.0437543 s, 1.9 GB/s
(13.04 - rev 01) 83886080 bytes (84 MB) copied, 0.116811 s, 718 MB/s
(13.10 - rev 01): 83886080 bytes (84 MB) copied, 0.129371 s, 648 MB/s

(14.04 - rev 01): 83886080 bytes (84 MB) copied, 0.157808 s, 532 MB/s

(14.10 - rev 01): 83886080 bytes (84 MB) copied, 0.102355 s, 820 MB/s

hdparm -tT /dev/sda1:
(13.04 - rev 05): Timing cached reads:23154 MB in 2.00 seconds = 11589.16 MB/sec
(13.04 - rev 01): Timing cached reads:17934 MB in 2.00 seconds = 8973.68 MB/sec
(13.10 - rev 01): Timing cached reads:18102 MB in 2.00 seconds = 9058.08 MB/sec
(14.04 - rev 01): Timing cached reads:17846 MB in 1.99 seconds = 8956.28 MB/sec
(14.10 - rev 01): Timing cached reads:21538 MB in 2.00 seconds = 10777.93 MB/sec

At one point, 2 of the rev 01 servers were using Ubuntu 14.04 and were getting similar bad benchmark results. After the upgrading them to 14.10, I immediately saw the disk read speed increase.

The performance on rev 01 machine with 13.04 was bad, but somehow the rev 05 machine with the same OS version was performing well. This told me it's not a problem with the megaraid_sas driver, because the rev 01 and rev 05 machines with 13.04 had the same version.

Here's the closest I can find to a similar issue (it's different hardware though): http://en.community.dell.com/support-forums/servers/f/906/t/19596533

Update 3/1/2015:
Saw an interesting checklist on Hacker News today that also mentioned disabling BIOS power saving settings: http://odetodata.com/2015/02/installation-and-configuration-checklist-for-microsoft-sql-server/

MegaCli64 / MegaCli Only showing "Exit Code: 0x00"

06 Jan 2015 Problem: You see MegaCli64 or MegaCli only showing "Exit Code: 0x00" when you try to run commands.

Solution: Try running as sudo.

Changing Putty Default Settings

02 Jan 2015

If you need to change Putty’s defaults, press “Load” on the saved session for “Default Settings”, make your changes, and “Save”.

putty default settings

Best Django Boilerplate/Skeleton

27 Dec 2014 This is my favorite boilerplate for new projects so far: https://github.com/kirpit/django-sample-app

Because:

It's up-to-date (as of 12/27/2014). Django-skel is currently at only up to 1.5.9.
It's minimal. It doesn't make many design decisions for you or include a bunch of junk. Django-base-template isn't as minimal.
Great structure, it follows this fairly closely: http://stackoverflow.com/a/23469321/1364191

If you're not a fan of bootstrap, django-html5-boilerplate also looks good.

Setting A WTForm Default Date To Today/Now

11 Dec 2014 http://stackoverflow.com/a/27424739/1364191

It's definitely a common mistake for people using WTForms. When you create a form and use something like datetime.date.today() as a default value, that value will always stay the same. The fix is to use "datetime.date.today" instead of "datetime.date.today()".

Dokku Flask Port and Host

05 Dec 2014 In your flask application, you need to:

Set default port=5000 (this seems to be Dokku's default too)
Set default host=0.0.0.0 (this allows your host server to connect to the container)

In your Procfile:

web: python <your-app>.py

If you're using gunicorn, it shouldn't matter because it's not using app.run().

If you still can't connect to your application after it deploys, see this: http://www.paulsprogrammingnotes.com/2014/12/troubleshooting-dokku-not-accessible.html

Troubleshooting Dokku Not Accessible

05 Dec 2014 First, make sure your application isn't giving any errors when it attempts to run:
dokku run <your-app> "<whatever is in your proc file>"

Get the :port number from the line with "upstream" in the following file: /home/dokku/<your-app>/nginx.conf

Try to "wget 127.0.0.1:<your-port>" from the host. Will it connect?

"Requested runtime not available for this stack" - Dokku

04 Dec 2014 While using Dokku, I was getting the following error message: Requested runtime (python-2.7.8) is not available for this stack (cedar-14).

This was due to a connection error between my docker container and the internet. To determine this, I ran "docker images" and got the image id of "programium/buildstep", then got into a container with the following command: sudo docker run -t -i <your image id> /bin/bash

Once in the container, I ran:
curl http://lang-python.s3.amazonaws.com/cedar/runtimes/python-2.7.8.tar.gz -s

If that is unsuccessful, that should confirm it's a connection issue.

This was the solution to the problem: http://www.paulsprogrammingnotes.com/2014/12/rebuilding-buildstep-image-for-dokku.html