Paul's Programming Notes     Archive     Feed     Github

SQLAlchemy Performance Tip

http://www.mobify.com/blog/sqlalchemy-memory-magic/

"If you can process the results of database queries iteratively (and very often you can), stream the results"

"Since the data needs to travel over the network from the database whether it's streamed or not, this doesn't add a huge overhead, but we'll see that it reduces memory requirements."

Example (using Flask-SQLAlchemy):
db.engine.execution_options(stream_results=True).execute(query)

However, that's not the full story when it comes to MySQLdb (MySQL-python). http://stackoverflow.com/a/3699677

If you're going to do this with mysql, I recommend using oursql: https://pypi.python.org/pypi/oursql

Things I Learned From DEFCON

Pentoo - Linux distro for pen testing.
VMware fusion - Most of the presenters were using a Macbook with VMware fusion managing their VMs.
Chinavasion - One of the presentations mentioned this in the context of dealextreme, I'm assuming this site is similar.
Kali linux - Another distro for pen testing.
Open bts
Sqlmap (SQLi injection) - Python tool used to tell if php pages were vulnerable to SQLi injection.
C99 shell - PHP interface for shell level system functions.
b374k shell - Another shell which allows an user to run file system, database, and shell commands from a web interface.
Accunetix (xss)
Business logic flaws in mobile operators
Doskey /history - Will show you all the previous commands typed into command prompt.
Maria DB - When oracle took over MySQL, it forked and maria db is still run by the previous developers.
Firefox imacro - A presenter said he uses this for all of his bots.
Burp repeater - Looks similar to fiddler, allows repeating http packets. Maybe only for Mac.
Fritzing
Groupie and Geocouch (couch dB) - This was used in combination with the unity engine to display clusters of map points in a video game.
Steganography - The word for hiding data inside of other files. A presenter showed how histograms could be used.
Femtocells - A personal cell tower. One of the presentations talked about how to use a femtocell to intercept text messages to a Verizon (CDMA) phone.
Shark - Has something to do with a hadoop, good for storing large amounts of data.
SuperTimeline

Alembic Cheat Sheet - Python/SQLAlchemy

alembic revision --autogenerate -m "<your message>"
alembic upgrade head

Problem: FAILED: Target database is not up to date.
Solution: alembic upgrade head

Problem: No such revision '5000106def16'
Solution: 
  1. sqlite3 db.sqlite3
  2. drop table alembic_version; (and exit, ctrl+d)
  3. alembic upgrade head

Failed to fetch http://security.ubuntu.com

W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/quantal-security/main/binary-amd64/Packages  404  Not Found [IP: 91.189.91.14 80]

This is happening because you're on an unsupported/old version of Ubuntu.

One way to fix it:
  1. nano /etc/apt/sources.list
  2. Change all entries referring to http://archive.ubuntu.com/ubuntu to http://old-releases.ubuntu.com/ubuntu

Change Percent To Numbers and Keep Color - Flot Pie Chart

You can access the color of the section of the pie chart by using “series.color”:

Use List In Bind Parameters - SQLAlchemy