Thursday, February 18, 2016

git commit --amend -C HEAD

Ever committed something and needed to make changes later? Just rebase and squash it, right? Or maybe "git reset --soft HEAD~1" and commit again?

There's an even better solution: "git commit --amend -C HEAD"

It will add the combine your commit with your last commit. If you already pushed, you will need to force push your change.

pip install -e .

I've wasted a lot of time running "python setup.py install" before testing my changes to flask-admin. It turns out you can pip install a project as "editable", which points the install toward your local directory instead of copying where the rest of your python modules are.

To install a project as editable, navigate to the repo and run "pip install -e .".

Saturday, February 13, 2016

Python - Dropping Into pdb From Nose

This document describes how to drop into pdb when your Python nose tests fail: http://nose.readthedocs.org/en/latest/plugins/debug.html

It just requires running "nosetests --pdb".

Very helpful if you're debugging why a test failed.

Great Hadoop + Spark Tutorial

This great tutorial takes you through installing hadoop and spark to analyze Reddit comment data: http://blog.insightdatalabs.com/spark-cluster-step-by-step/

Hadoop Namenode Not Starting

My namenode was not starting because I had the wrong host configured in yarn-site.xml, mapred-site.xml, and core-site.xml.

When you're running start-dfs.sh on your namenode, ensure the line that says "starting namenode" shows "/usr/local/hadoop/logs/hadoop-ubuntu-namenode-<your namenode's hostname>.out" in the output. This is how you know your configuration is correct.

You can check your server's hostname on Ubuntu by running "echo $(hostname)".