Paul's Programming Notes     Archive     Feed     Github

Adding Virtualhost In Apache For Port

Say we wanted to route the <your-domain>.com:8072/misc to /var/www/misc...

Add the following to /etc/apache2/ports.conf:
NameVirtualHost *:8072
Listen 8072

Add a file that describes your site to /etc/apache2/sites-available with this:
<VirtualHost *:8072>
   ServerName <your-domain>.com
   DocumentRoot /var/www/misc/
</VirtualHost>

Run this command: sudo a2ensite misc

And restart apache: sudo service apache2 restart

Deploying Nginx + Gunicorn + Flask

Why use both Nginx and Gunicorn?: http://serverfault.com/a/220047

Simple explanation: http://prakhar.me/articles/flask-on-nginx-and-gunicorn/

Nginx SSL configuration: https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-with-ssl-as-a-reverse-proxy-for-jenkins

The thing I spent the longest on was getting supervisor to work. Gunicorn kept giving me "gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>", because I was missing the "directory" parameter. I needed to tell supervisor which directory to start from.

[program:my_app]
command=/<path>/venv/prod/bin/gunicorn -w 8 main:app -b 127.0.0.1:8080
user=me
autostart=true
directory=/home/me/my_dir/
stdout_logfile=/home/me/my_dir/gunicorn_supervisor.log
stdout_logfile_maxbytes=20MB
redirect_stderr=true

You will need to create an upstart script to get supervisor to run when the system starts: http://serverfault.com/a/96500

Python Excel Library Benchmark Comparison

https://github.com/swistakm/python-excel-benchmarks

benchmark_csv                  0.026495
benchmark_excellent            1.784107
benchmark_openpyxl             2.897072
benchmark_openpyxl_rows        7.025895
benchmark_pyexcelerate         0.550225
benchmark_xlsxcessive          1.430242
benchmark_xlsxwriter           1.611668
benchmark_xlwt                 1.275144

http://xlsxwriter.readthedocs.org/en/latest/working_with_memory.html

pyexcelerate          :  10.11
xlwt                  :  15.67
xlsxwriter (optimised):  19.70
xlsxwriter            :  23.50
openpyxl   (optimised):  95.82
openpyxl              :  95.90

Large Dictionaries Not Released From Memory - Python + Ubuntu

I have a flask application running on Ubuntu which reads from a file and creates several large dictionaries. After the dictionaries are created and flask returns the request, the memory used for the dictionaries is not released back to the OS. This example shows the problem: https://gist.github.com/pawl/c3ed7663b94abec01d75

It's not really a Flask issue, but the Flask guys were super helpful when I asked why my large dictionary wasn't released from memory: https://github.com/mitsuhiko/flask/issues/1202

I dug even deeper and learned this is a Linux thing:
"Python returns memory to the OS on the heap (that allocates other objects than small objects) only on Windows, if you run on Linux, you can only see the total memory used by your program increase."
http://deeplearning.net/software/theano/tutorial/python-memory-management.html

My solution:
Turning it into a function and running it as a separate processhttps://gist.github.com/pawl/95769724848269cff890

If you want to go even deeper down the rabbit hole, read these:

Virtualenvwrapper Basics

This video does a good job of describing the basics of Virtualenvwrapper: https://www.youtube.com/watch?v=thHNYVrY0lU

You'll need to have used virtualenv to appreciate this.

Basically, you create virtual environments with "mkvirtualenv" and they're stored in a central location. When you want to switch to your virtual environment, you use the "workon" command.

You can also create a postactivate script that loads environmental variables when you switch to your virtual environment.

Note: There is also a module called Pew that seems to have a simpler installation process: https://github.com/berdario/pew

Meetup API address_1_error

I was getting this error while using the POST /venues method of the meetup API.

The error seems to happen when you duplicate the address of another venue. I was able to add the venue, but when I tried to add it a second time - I would get this error.

I ended up getting the list of venues and trying to find a match first.