Paul's Programming Notes2022-08-26T06:32:38+00:00https://www.paulsprogrammingnotes.comPaul BrownLinux - Forgetting Exec2022-08-25T18:11:00+00:00https://www.paulsprogrammingnotes.com/2022/08/forgetting-exec<p>Today, I came across a bug where a <a href="https://github.com/celery/celery">Celery</a> worker wasn’t gracefully shutting down, and it was causing some <a href="https://github.com/kubernetes/kubernetes/issues/105703">odd “Connection Refused”</a> errors from requests within the task being run by the worker. It was also shutting down before it could send errors to Rollbar/Sentry for the team to know they need to address it.</p>
<p>This was happening because the entrypoint had a script that effectively did this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>echo "starting celery worker"
celery -A tasks worker
</code></pre></div></div>
<p>The problem with that entrypoint: it’s not using <code class="language-plaintext highlighter-rouge">exec</code> to run the child process. Without <code class="language-plaintext highlighter-rouge">exec</code>, it will <a href="https://unix.stackexchange.com/a/196053">not forward signals like SIGTERM to the process the entrypoint is waiting on</a>. The Celery worker child process won’t know it needs to shut down gracefully if the SIGTERM is not forwarded to it.</p>
<p>The fixed example would look like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>echo "starting celery worker"
exec celery -A tasks worker
</code></pre></div></div>
Linux - Dig's +short Option2022-07-19T18:11:00+00:00https://www.paulsprogrammingnotes.com/2022/07/dig-short-option<p>I learned Linux’s <code class="language-plaintext highlighter-rouge">dig</code> has a “+short” option that returns <strong>only</strong> the IP address for a hostname:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ dig +short google.com
142.251.32.174
</code></pre></div></div>
Linux - Ping Not Showing Lost Packets2022-06-02T18:11:00+00:00https://www.paulsprogrammingnotes.com/2022/06/ping-not-showing-loss<p>I was recently troubleshooting some packet loss with ping on linux, and I noticed by default ping won’t explicitly show lost packets:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ ping x.x.x.x
64 bytes from x.x.x.x: icmp_seq=8 ttl=52 time=18.1 ms
64 bytes from x.x.x.x: icmp_seq=11 ttl=52 time=21.2 ms
</code></pre></div></div>
<p>(notice the skipped 8-10, those were lost packets)</p>
<p>To fix this, you can add run ping with <code class="language-plaintext highlighter-rouge">-O</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ ping -O x.x.x.x
64 bytes from x.x.x.x: icmp_seq=11 ttl=52 time=19.0 ms
no answer yet for icmp_seq=12
no answer yet for icmp_seq=13
64 bytes from x.x.x.x: icmp_seq=14 ttl=52 time=18.9 ms
</code></pre></div></div>
<p><a href="https://askubuntu.com/a/838793">Related askubuntu.com Thread</a></p>
Linux - Improving Traceroute Output2022-06-02T18:11:00+00:00https://www.paulsprogrammingnotes.com/2022/06/improving-traceroute<p>Usually when I use <code class="language-plaintext highlighter-rouge">traceroute</code> without any options, it gets stuck showing output like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> 14 * * *
15 * * *
16 * * *
</code></pre></div></div>
<p>For more info about why this is happening: <a href="https://webmasters.stackexchange.com/a/30957">Link</a></p>
<p>To improve this output, try adding <code class="language-plaintext highlighter-rouge">-I</code> (<code class="language-plaintext highlighter-rouge">traceroute -I</code>) to make it use ICMP ECHO instead of UDP for probes.</p>
<p>If you want better statistics about packet loss, you can also use a tool called <code class="language-plaintext highlighter-rouge">mtr</code> which combines traceroute and ping. If you’re on Ubuntu, you can install this with <code class="language-plaintext highlighter-rouge">sudo apt-get install mtr</code>.</p>
Docker - WORKDIR Creates Directories2022-04-01T18:11:00+00:00https://www.paulsprogrammingnotes.com/2022/04/docker-workdir<p>Which user do you think owns the <code class="language-plaintext highlighter-rouge">src/</code> directory in this Dockerfile example?:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>FROM alpine:3.13.2
RUN adduser -D wendy
USER wendy
WORKDIR src/
COPY --chown=wendy . src/
</code></pre></div></div>
<p>The answer?: <code class="language-plaintext highlighter-rouge">root</code></p>
<p>This happens because <code class="language-plaintext highlighter-rouge">WORKDIR</code> will create directories that don’t exist with the <code class="language-plaintext highlighter-rouge">root</code> user.</p>
<p>What if you wanted <code class="language-plaintext highlighter-rouge">wendy</code> to be the owner of that <code class="language-plaintext highlighter-rouge">src/</code> directory?</p>
<p>You would need to <code class="language-plaintext highlighter-rouge">COPY</code> the <code class="language-plaintext highlighter-rouge">src/</code> directory into place before setting it as the <code class="language-plaintext highlighter-rouge">WORKDIR</code>. For example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>FROM alpine:3.13.2
RUN adduser -D wendy
USER wendy
COPY --chown=wendy . src/
WORKDIR src/
</code></pre></div></div>
<p>More info: <a href="https://github.com/moby/moby/issues/20295">Closed GitHub Issue</a> & <a href="https://github.com/moby/moby/blob/16009830c2e2f3840ee73459918289f4bf54bd1d/container/container.go#L277">Docker Code</a></p>
DIY NAS2022-03-13T18:11:00+00:00https://www.paulsprogrammingnotes.com/2022/03/DIY-NAS<p>If you have an old computer with spare hard drives, it might be useful to use it to share files with computers on your network by turning it into a NAS (network attached storage). There are several popular DIY NAS options:</p>
<ul>
<li><a href="https://www.unraid.net/">Unraid</a> - Paid, Slackware linux based</li>
<li><a href="https://www.truenas.com/freenas/">FreeNAS (now TrueNAS Core)</a> - FreeBSD based</li>
<li><a href="https://github.com/openmediavault/openmediavault">OpenMediaVault</a> - Debian linux based</li>
</ul>
<p>I ended up going with <a href="https://github.com/openmediavault/openmediavault">OpenMediaVault</a> due to it being Debian based and super popular. To set it up, I followed this incredibly thorough guide: <a href="https://michaelxander.com/diy-nas/">link</a></p>
<p>I had a few issues that caused it to stop working after restarting.</p>
<p>The first issue was related to the disk being encrypted. OpenMediaVault started in emergency mode due to being unable to access the encrypted disk. Emergency mode unfortunately doesn’t allow SSH access, which means you need to plug a monitor and keyboard into it. You have to make sure you have <code class="language-plaintext highlighter-rouge">nofail</code> in the options section of the lines of <code class="language-plaintext highlighter-rouge">/etc/fstab</code> starting with <code class="language-plaintext highlighter-rouge">/dev/disk</code>. You also need to add <code class="language-plaintext highlighter-rouge">nonempty</code> and remove <code class="language-plaintext highlighter-rouge">x-systemd.requires=<disk></code> from the options section of the lines starting with <code class="language-plaintext highlighter-rouge">/srv/dev-disk-by-uuid</code> (for mergerfs). I’ve had to redo the removing <code class="language-plaintext highlighter-rouge">x-systemd.requires</code> part every time I apply settings from the web UI. More information about this: <a href="https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-unionfilesystems/issues/36">1</a> <a href="https://www.bananatronics.org/openmediavault-issues/">2</a> <a href="https://forum.openmediavault.org/index.php?thread/6851-unionfilesystem-plugin/&postID=98406#post98406">3</a> <a href="https://github.com/openmediavault/openmediavault/issues/850#issuecomment-752193645">4</a> <a href="https://forum.openmediavault.org/index.php?thread/37268-omv5-not-surviving-first-reboot/&postID=277434#post277434">5</a> <a href="https://forum.openmediavault.org/index.php?thread/32870-oops-can-t-boot-after-unionfs-changes/&postID=241306#post241306">6</a> <a href="https://forum.openmediavault.org/index.php?thread/30129-problem-booting-up-with-luks-and-unionfilesystem/&pageNo=2">7</a></p>
<p>The second issue involved losing network adapter configuration after restarting. I had to run <code class="language-plaintext highlighter-rouge">omv-firstaid</code> and configure the network adapter with a static IP to resolve this. This may have something to do with an empty configuration from the web UI ovewriting the existing working configuration.</p>
Docker - ufw rules ignored2022-01-01T18:11:00+00:00https://www.paulsprogrammingnotes.com/2022/01/docker-ignores-ufw<p>I recently learned that docker will ignore ufw (uncomplicated firewall) rules by default. This means that it will still expose ports that are blocked by ufw.</p>
<p>The fix involved adding this to <code class="language-plaintext highlighter-rouge">/etc/docker/daemon.json</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{
"iptables": false,
"ip6tables": false
}
</code></pre></div></div>
<p>Then I restarted the docker daemon with <code class="language-plaintext highlighter-rouge">sudo systemctl restart docker</code>.</p>
<p><a href="https://github.com/moby/moby/issues/9889#issuecomment-849165540">More details</a></p>
Raspberry Pi Zero Not Connecting to Unifi AP2021-12-27T18:11:00+00:00https://www.paulsprogrammingnotes.com/2021/12/raspberry-pi-zero-not-connecting-unifi-wifi<p>I recently had a lot of trouble getting my Raspberry Pi Zero W 2 to connect to WiFi on my Ubiquiti Unifi AP AC Lite after a firmware upgrade to 5.43.46. It’s a 2.4Ghz-only device, and actually all of my 2.4Ghz-only devices wouldn’t connect.</p>
<p>Apparently something I did set a setting called PMF (Protected Management Frames) to “Required”, and the Pi Zero W 2 <a href="https://forums.raspberrypi.com/viewtopic.php?t=293617">doesn’t support that</a>. In theory, this setting should help prevent clients from getting disconnected through de-auth packets from an attacker. However, not all devices support this.</p>
<p>To fix the issue, you need to find the PMF setting under Settings -> Wifi -> <code class="language-plaintext highlighter-rouge"><your network></code> -> Advanced -> Security and change it to “Optional”.</p>
Fixing Memory Leaks In Popular Python Libraries2021-12-20T02:11:00+00:00https://www.paulsprogrammingnotes.com/2021/12/python-memory-leaks<p>I was recently able to make a <a href="https://github.com/celery/celery/issues/4843#issuecomment-988492732">minimal example</a> that reproduced a Celery memory leak. The memory leak would happen on the main Celery worker process that’s forked to make child processes, which makes the leak especially bad. Issue <a href="https://github.com/celery/celery/issues/4843">#4843</a> has been around for 3+ years and has 140+ comments, so this one has been causing a lot of problems for Celery users for a while.</p>
<p>The memory leak has been causing a lot of issues at my work too, and I was able to get some help resolving the issue during a work hackathon. My coworker Michael Lazar was able to find <a href="https://github.com/celery/celery/issues/4843#issuecomment-991394349">the root cause</a> of the issue and make a <a href="https://github.com/celery/py-amqp/pull/374">pull request</a> to fix it in py-amqp (a celery dependency when using RabbitMQ as a broker). The code with the issue was <a href="https://github.com/celery/py-amqp/blame/v5.0.5/amqp/connection.py#L469">10 years old</a>!</p>
<p>Here’s what the bug looks like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>try:
sock.shutdown(socket.SHUT_RDWR)
sock.close()
except OSError:
pass
</code></pre></div></div>
<p>The problem occurs when <code class="language-plaintext highlighter-rouge">socket.shutdown</code> fails on an <code class="language-plaintext highlighter-rouge">OSError</code> and doesn’t proceed to <code class="language-plaintext highlighter-rouge">socket.close</code> to clean up the socket and allow garbage collection to release the memory used for it. The <code class="language-plaintext highlighter-rouge">OSError</code> on <code class="language-plaintext highlighter-rouge">shutdown</code> can occur when the remote side of the connection closes the connection first.</p>
<p>The fixed example (with separate <code class="language-plaintext highlighter-rouge">try</code>/<code class="language-plaintext highlighter-rouge">except</code> blocks):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>try:
sock.shutdown(socket.SHUT_RDWR)
except OSError:
pass
try:
sock.close()
except OSError:
pass
</code></pre></div></div>
<p>I was able to make the same fix to a few other popular Python libraries too:</p>
<ul>
<li><a href="https://github.com/redis/redis-py/pull/1797">redis-py</a> (<a href="https://github.com/redis/redis-py/blame/v4.0.2/redis/connection.py#L682">11 year old code!</a>)</li>
<li><a href="https://github.com/Thriftpy/thriftpy2/pull/183">thriftpy2</a></li>
<li><a href="https://github.com/kovidgoyal/calibre/pull/1552">calibre</a></li>
</ul>
<p>I also found another way to reduce memory usage of <code class="language-plaintext highlighter-rouge">Connection</code>s in <a href="https://github.com/celery/py-amqp/pull/377">py-amqp</a> and <a href="https://github.com/celery/librabbitmq/pull/160">librabbitmq</a> by changing how active channel IDs are stored.</p>
<p>Update 12/20: Hacker News user <a href="https://news.ycombinator.com/item?id=29626897">js2 pointed out</a> that Python will <a href="https://docs.python.org/3/howto/sockets.html#disconnecting">automatically</a> close the socket when all the references to the socket are gone.</p>
<p>Update 12/23: I got a <a href="https://github.com/celery/kombu/pull/1470">pull request</a> merged into Kombu with the same memory usage reduction fix I made to <a href="https://github.com/celery/py-amqp/pull/377">py-amqp</a> and <a href="https://github.com/celery/librabbitmq/pull/160">librabbitmq</a>. I also opened another <a href="https://github.com/celery/kombu/pull/1476">pull request</a> to Kombu that should fix a <a href="https://github.com/celery/celery/issues/4843#issuecomment-999168967">memory leak issue</a> when using Celery with Redis.</p>
<p>Update 12/25: I wrote a section for the Celery docs about <a href="https://github.com/celery/celery/pull/7186">optimizing memory usage</a>. I also fixed <a href="https://github.com/celery/celery/pull/7187">another leak</a> in Celery that happens when connection errors occur on a prefork worker.</p>
Fn Key At The Software Level2021-12-04T02:11:00+00:00https://www.paulsprogrammingnotes.com/2021/12/fn-key<p>I was trying to remap the “Fn” key on my keyboard and learned something interesting: The “Fn” key is not a real key at the software level.</p>
<p>However, you can remap combinations of “Fn” + other keys.</p>
<p>More details: <a href="https://askubuntu.com/a/827953">https://askubuntu.com/a/827953</a></p>
Firefox - arkenfox user.js2021-12-04T02:11:00+00:00https://www.paulsprogrammingnotes.com/2021/12/arkenfox<p>I recently learned about the <a href="https://github.com/arkenfox/user.js">arkenfox user.js project</a>, which is an easy way to update Firefox’s settings to improve security and privacy.</p>
<p>It suggests you add your own overrides to the bottom of the file. I added these:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/* re-enable location bar using search ***/
user_pref("keyword.enabled", true);
/* override recipe: enable session restore ***/
user_pref("browser.startup.page", 3); // 0102
user_pref("browser.privatebrowsing.autostart", false); // 0110 required if you had it set as true
user_pref("places.history.enabled", true); // 0862 required if you had it set as false
user_pref("browser.sessionstore.privacy_level", 0); // 1003 optional [to restore cookies/formdata]
user_pref("network.cookie.lifetimePolicy", 0); // 2801 optional [so cookies persist]
user_pref("privacy.clearOnShutdown.history", false); // 2811
user_pref("privacy.cpd.history", false); // 2812 to match when you use Ctrl-Shift-Del
</code></pre></div></div>
Comparing Tags On Different Branches With GitHub2021-11-23T02:12:00+00:00https://www.paulsprogrammingnotes.com/2021/11/comparing-github-tags<p>I recently compared two tags on github using the “compare” dropdown, and it displayed unexpectedly few changes between the tags. However, one of the tags was from a different branch and by default GitHub doesn’t show the other differences between the branches. To see other differences besides the tagged commit compared to the main/master branch, you need to change the “…” in the compare url to “..”.</p>
<p>Now, I’m thinking it’s probably a good idea to avoid making tags from non-master/main branches.</p>
<p>More info: <a href="https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-comparing-branches-in-pull-requests#three-dot-and-two-dot-git-diff-comparisons">GitHub Docs</a></p>
Air Quality Monitor2021-11-23T02:11:00+00:00https://www.paulsprogrammingnotes.com/2021/11/air-quality-monitor<p>I build the air quality monitor from <a href="https://www.jeffgeerling.com/blog/2021/airgradient-diy-air-quality-monitor-co2-pm25">Jeff Geerling’s post</a>:
<img src="/generated/assets/images/air_quality_monitor_1-800-12084f20d.jpg" srcset="/generated/assets/images/air_quality_monitor_1-400-12084f20d.jpg 400w, /generated/assets/images/air_quality_monitor_1-600-12084f20d.jpg 600w, /generated/assets/images/air_quality_monitor_1-800-12084f20d.jpg 800w, /generated/assets/images/air_quality_monitor_1-960-12084f20d.jpg 960w" /></p>
Python - Singleton2021-09-21T06:00:00+00:00https://www.paulsprogrammingnotes.com/2021/09/singleton<p>I saw some interesting Python singleton code in the wild today. It was used for a Redis connection pool.</p>
<p>It looked like <a href="https://stackoverflow.com/q/53848389">this</a>.</p>
Starting To Learn About GraphQL2021-08-19T06:00:00+00:00https://www.paulsprogrammingnotes.com/2021/08/graphql<p>I’ve started learning about GraphQL.</p>
<p>The best summary I’ve seen so far is the <a href="https://docs.graphene-python.org/en/latest/quickstart/">graphene-python “Getting Started” guide</a>.</p>
<p>This <a href="https://medium.com/joor-engineering/graphql-on-django-at-joor-f31dc3251482">post</a> also talks about the <a href="https://docs.graphene-python.org/en/latest/execution/dataloader/">dataloader</a> pattern to helps solve n+1 query issues.</p>
<p><a href="https://www.apollographql.com/blog/tooling/apollo-codegen/typescript-graphql-code-generator-generate-graphql-types/">Apollo Codegen</a> also looks like a good way to share GraphQL types with the front-end.</p>
Celery - Tuning worker_prefetch_multiplier2021-08-19T06:00:00+00:00https://www.paulsprogrammingnotes.com/2021/08/celery-tasks<p>I came across an interesting article that talks about speeding up the processing of fast running celery tasks by tuning worker_prefetch_multiplier: <a href="https://www.lorenzogil.com/blog/2020/03/01/celery-tasks/">https://www.lorenzogil.com/blog/2020/03/01/celery-tasks/</a></p>
<p>The default scheduling strategy in Celery is <a href="https://docs.celeryproject.org/en/3.0/whatsnew-4.0.html#ofair-is-now-the-default-scheduling-strategy">now “fair”</a>, so part of the article doesn’t apply anymore.</p>
Raspberry Pi Imager - Secret Menu2021-07-23T06:00:00+00:00https://www.paulsprogrammingnotes.com/2021/07/raspberry-pi-imager<p>The v1.6 update for the Raspberry Pi Imager added a new shortcut (CTRL + Shift + X) that allows enabling SSH, configuring WiFi, setting hostname, and more.</p>
<p><a href="https://www.raspberrypi.org/blog/raspberry-pi-imager-update-to-v1-6/">More Info</a></p>
I'm on Hackaday!2021-07-23T06:00:00+00:00https://www.paulsprogrammingnotes.com/2021/07/on-hackaday<p>Hackaday posted an <a href="https://hackaday.com/2021/07/21/raspberry-pi-server-cluster-in-1u-rack-mount-case/">article</a> about my <a href="https://github.com/pawl/raspberry-pi-1u-server">Raspberry Pi 1U Server</a> project.</p>
ssh-keygen -R hostname2021-07-12T00:11:00+00:00https://www.paulsprogrammingnotes.com/2021/07/ssh-keygen-r<p>I’ve been manually removing the line from .known_hosts when I saw this message:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:<key>.
Please contact your system administrator.
Add correct host key in /Users/<user>/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /Users/<user>/.ssh/known_hosts:85
ECDSA host key for <ip> has changed and you have requested strict checking.
Host key verification failed.
</code></pre></div></div>
<p>Today I found out about this command to remove entries from known_hosts: <code class="language-plaintext highlighter-rouge">ssh-keygen -R hostname</code></p>
Raspberry Pi 1U Server2021-06-24T02:11:00+00:00https://www.paulsprogrammingnotes.com/2021/06/raspberry-pi-1u-server<p>I’ve been working on a power efficient 1U server made with Raspberry Pi’s to take advantage of cheap server colocation options:
<img src="/generated/assets/images/pi_finished_1-800-882761c27.jpg" srcset="/generated/assets/images/pi_finished_1-400-882761c27.jpg 400w, /generated/assets/images/pi_finished_1-600-882761c27.jpg 600w, /generated/assets/images/pi_finished_1-800-882761c27.jpg 800w, /generated/assets/images/pi_finished_1-1000-882761c27.jpg 1000w" /></p>
<p><img src="/generated/assets/images/pi_finished_2-800-82bcd0572.jpg" srcset="/generated/assets/images/pi_finished_2-400-82bcd0572.jpg 400w, /generated/assets/images/pi_finished_2-600-82bcd0572.jpg 600w, /generated/assets/images/pi_finished_2-800-82bcd0572.jpg 800w, /generated/assets/images/pi_finished_2-1000-82bcd0572.jpg 1000w" /></p>
<p><img src="/generated/assets/images/pi_finished_3-800-a6bb194ad.jpg" srcset="/generated/assets/images/pi_finished_3-400-a6bb194ad.jpg 400w, /generated/assets/images/pi_finished_3-600-a6bb194ad.jpg 600w, /generated/assets/images/pi_finished_3-800-a6bb194ad.jpg 800w, /generated/assets/images/pi_finished_3-1000-a6bb194ad.jpg 1000w" /></p>
<p>More details: <a href="https://github.com/pawl/raspberry-pi-1u-server">Github</a></p>
Python - Automatic Type Hinting With Monkeytype2021-05-12T02:11:00+00:00https://www.paulsprogrammingnotes.com/2021/05/python-monkeytype<p>I’ve been poking around with adding type hints to Python code automatically. Here’s my process so far:</p>
<noscript><pre>1. `pip install MonkeyType`
1. run tests with `monkeytype run` instead of `python`
1. pytest: `monkeytype run -m pytest`
1. `mkdir stubs`
1. `monkeytype list-modules | xargs -n1 -I{} sh -c 'monkeytype stub {} > stubs/{}.pyi'`
1. `pip install pytype`
1. `python move_files.py` (file in this gist)
1. `mypy <code folder name>`
1. remove duplicate imports added by merge-pyi, fix import sorting issues
1. clean up super long type hints by inlining with params</pre></noscript>
<script src="https://gist.github.com/15ea14d194b48a4000dd6446b9b3f35d.js"> </script>
Python - setup.py project_urls2021-05-08T02:11:00+00:00https://www.paulsprogrammingnotes.com/2021/05/python-setup-py-project-urls<p>I saw the <code class="language-plaintext highlighter-rouge">attrs</code> project has a link to their changelog on pypi:
<img src="/generated/assets/images/attrs_screenshot_2-549-3fb8b95d6.png" srcset="/generated/assets/images/attrs_screenshot_2-400-3fb8b95d6.png 400w, /generated/assets/images/attrs_screenshot_2-549-3fb8b95d6.png 549w" /></p>
<p>It turns out that’s controlled by the <code class="language-plaintext highlighter-rouge">project_urls</code> section of the setup.py.</p>
<p>More details: <a href="https://packaging.python.org/guides/distributing-packages-using-setuptools/#project-urls">Python Docs</a></p>
Python - virtualenv & Dockerfiles2021-04-23T23:43:00+00:00https://www.paulsprogrammingnotes.com/2021/04/python-virtualenv-dockerfile<p>I recently saw a Dockerfile based on the official docker Debian image that was installing dependencies into a virtualenv.</p>
<p>I’m pretty sure using a virtualenv in an official Debian-based Dockerfile is unnecessary, because there’s no system Python to isolate from:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ docker run -it debian /bin/bash
root@21ca17310079:/# python
bash: python: command not found
</code></pre></div></div>
git pickaxe2021-04-23T23:43:00+00:00https://www.paulsprogrammingnotes.com/2021/04/git-pickaxe<p>I learned about the “git pickaxe” this week, and I used it to find the first commit with a line of code that had been moved between a few different files.</p>
<p>More details: <a href="https://remireuvekamp.nl/blog/the-git-pickaxe.html">https://remireuvekamp.nl/blog/the-git-pickaxe.html</a></p>
apt-get install --no-install-recommends2021-04-19T23:43:00+00:00https://www.paulsprogrammingnotes.com/2021/04/no-install-recommends<p>I learned about apt-get install’s “–no-install-recommends” flag, and I used it to prevent unnecessary “recommended” packages from getting installed. This helped reduce some unnecesary bloat in a Docker image.</p>
<p>More details: <a href="https://ubuntu.com/blog/we-reduced-our-docker-images-by-60-with-no-install-recommends">https://ubuntu.com/blog/we-reduced-our-docker-images-by-60-with-no-install-recommends</a></p>
WLED2021-03-14T01:43:00+00:00https://www.paulsprogrammingnotes.com/2021/03/WLED<p>Today I learned about the amazing WLED project: <a href="https://github.com/Aircoookie/WLED">https://github.com/Aircoookie/WLED</a></p>
<p>All you need to do is flash their binary to an ESP32 and it will bring up a WIFI access point that allows you to control the LEDs through an app or web page.</p>
<p><img src="/generated/assets/images/2021-03-13-20.38.23-800-16757995f.jpg" srcset="/generated/assets/images/2021-03-13-20.38.23-400-16757995f.jpg 400w, /generated/assets/images/2021-03-13-20.38.23-600-16757995f.jpg 600w, /generated/assets/images/2021-03-13-20.38.23-800-16757995f.jpg 800w, /generated/assets/images/2021-03-13-20.38.23-960-16757995f.jpg 960w" /></p>
Event Driven Architecture Talk2021-01-22T05:21:00+00:00https://www.paulsprogrammingnotes.com/2021/01/event-driven-architecture<p>I really liked this talk by Martin Fowler about Event Driven Architecture: <a href="https://youtu.be/STKCRSUsyP0">https://youtu.be/STKCRSUsyP0</a></p>
<p>It’s interesting to think about Git as an example of event sourcing.</p>
Microservice Communication2021-01-20T18:21:00+00:00https://www.paulsprogrammingnotes.com/2021/01/microservice-communication<p>I’ve been learning more about async communication between microservices and came across this article that talks about some of the trade-offs between sync and async communication: <a href="https://dzone.com/articles/patterns-for-microservices-sync-vs-async">https://dzone.com/articles/patterns-for-microservices-sync-vs-async</a></p>
<h1 id="summary">Summary</h1>
<h2 id="sync">Sync</h2>
<h3 id="pros">Pros</h3>
<ul>
<li>“synchronous calls are simpler to grasp, debug and implement”</li>
</ul>
<h3 id="cons">Cons</h3>
<ul>
<li>“A temporary burst at one component can flood other services with requests.”</li>
<li>Risk of Cascading Failures (domino effect of failures if one service experiences an otuage)</li>
<li>tighter coupling (requires endpoint versioning in the owning service)</li>
</ul>
<h2 id="async">Async</h2>
<h3 id="pros-1">Pros</h3>
<ul>
<li>“removes the need to wait for a response thereby decoupling the execution of two or more services”</li>
<li>“deals better with sporadic bursts of traffic”</li>
</ul>
<h3 id="cons-1">Cons</h3>
<ul>
<li>“Asynchronous systems tend to be significantly more complex than synchronous ones.”</li>
<li>“consumers need to adapt to work with an asynchronous system”</li>
<li>“the message bus the Achilles heel of the system as it remains a central point of failure”</li>
<li>Eventual Consistency (potentially out of date data)</li>
</ul>
Python - Mypy2021-01-08T18:21:00+00:00https://www.paulsprogrammingnotes.com/2021/01/learning-mypy<p>My new job has me working on a larger codebase than previous jobs, and it’s also my first time using <a href="https://mypy.readthedocs.io/en/stable/">mypy</a>.</p>
<p>I’m starting to understand why Guido’s work on mypy had a lot to do with Dropbox’s Python 3 migration. <a href="https://dropbox.tech/application/our-journey-to-type-checking-4-million-lines-of-python">Dropbox wrote an article</a> with details on how they used mypy on their “4 million lines of Python”. With that much code, I can understand why they needed to treat code like “cattle, not pets”.</p>
<p>Python 3 disallows some comparisons with <code class="language-plaintext highlighter-rouge">None</code> types, makes huge changes to strings/bytes, and has many other changes involving types. When your codebase is massive, you have to reach for automated tooling to consistently find and prevent those bugs. With the same tooling you can also prevent entire categories of other bugs from reaching production. The type hints can also be helpful documentation. For those reasons and more, I think implementing type hints on a large codebase like Dropbox’s will definitely be worth it in the long run.</p>
<p>It has me wondering if I should have been taking the extra time to use type hints and checking (or a statically typed language) this whole time. Was going without type hints one less distraction? Or will be the price be paid in maintenance difficulties and bug fixes later?</p>
<p>At this point, not much of the Python ecosystem has type hints (not even Python 2 compatible comment-style type hints). I’m starting to think it would be a good use of time to work on changing that.</p>
Ruby - Jekyll2020-12-25T08:21:00+00:00https://www.paulsprogrammingnotes.com/2020/12/blog-updates<p>I migrated this blog from blogger/blogspot to a static site generated with Jekyll and hosted with Netlify.</p>
<p>This blog post describes the process: <a href="http://joshualande.com/jekyll-github-pages-poole">http://joshualande.com/jekyll-github-pages-poole</a></p>
<p>Initially, I started using Python’s Pelican, but pelican-import (pelican’s tool for migrating from blogger) doesn’t work as well as jekyll-import. It turned comments into posts and threw exceptions while processing draft posts without content. Also, the first docs that show up on google for Pelican aren’t the latest docs. This causing issues when I followed the old docs that said to use Python 2.7, but Pelican only supports Python 3.6+ now.</p>
"A (multi-) monorepo setup with Git Submodules"2020-12-25T00:21:00+00:00https://www.paulsprogrammingnotes.com/2020/12/a-multi-monorepo-setup-with-git<p><a href="https://www.jannikbuschke.de/blog/git-submodules/">https://www.jannikbuschke.de/blog/git-submodules/</a></p>Arduino - Oven Thermometer2020-12-20T20:38:00+00:00https://www.paulsprogrammingnotes.com/2020/12/arduino-oven-thermometer<p>Parts:</p><ul style="text-align: left;"><li><span class="pl-c">HiLetgo DC 3-5V MAX6675 Module + K Type Thermocouple Temperature Sensor</span></li><li><span class="pl-c"><span class="pl-c">UCTRONICS 0.96 Inch OLED Module 12864 128x64 Yellow Blue SSD1306 Driver I2C</span></span></li><li><span class="pl-c"><span class="pl-c"><span class="pl-c">ESP32</span> </span> </span> </li></ul><p>Code: <a href="https://gist.github.com/pawl/73a1ccfbf2b2be6f934e651b39c51082">https://gist.github.com/pawl/73a1ccfbf2b2be6f934e651b39c51082</a> <br /></p>Linux - Sparse Files2020-12-12T23:54:00+00:00https://www.paulsprogrammingnotes.com/2020/12/linux-sparse-files<p>Found a massive faillog/lastlog file recently and thought it was responsible for using up all my disk space. Turns out it's actually a sparse file and doesn't take up that much physical space: <a href="https://unix.stackexchange.com/a/530157">https://unix.stackexchange.com/a/530157</a><br /></p>Arduino - Coffee Roasting Monitor2020-12-12T23:42:00+00:00https://www.paulsprogrammingnotes.com/2020/12/arduino-coffee-roasting-monitor<p>Parts:</p><ul style="text-align: left;"><li>Arduino Uno</li><li>2x MAX6675 w/ thermocouple<br /></li></ul><p>Code: <a href="https://gist.github.com/pawl/5ed1f61e0b0b27aa839fc2b4499a92c2">https://gist.github.com/pawl/5ed1f61e0b0b27aa839fc2b4499a92c2</a></p><p> <br /></p>Django - Duplicate Unformatted Logs2020-09-04T20:27:00+00:00https://www.paulsprogrammingnotes.com/2020/09/django-duplicate-unformatted-logs<p>Recently I worked on a bug that was causing duplicate unformatted log messages to appear in a Django app's logs. I made a repository that demonstrates the issue: <a href="https://github.com/pawl/django_duplicate_unformatted_logs_example">https://github.com/pawl/django_duplicate_unformatted_logs_example</a></p><p>The problem was caused by an accidental call to <code>logging.info</code> (without using <code>logging.getLogger</code> to get a specific logger) while the root logger isn't already configured.</p><p>The solution ended up being to get rid of the accidental calls to <code>logging.info</code> and configuring the root logger to prevent it from accidentally happening again. I go into more details in that repo.<br /></p>Vue - vue-custom-element2020-05-12T01:46:00+00:00https://www.paulsprogrammingnotes.com/2020/05/vue-vue-custom-element<a href="https://github.com/karol-f/vue-custom-element">Vue-custom-element</a> seems like the best way to integrate vue into an existing application that uses server side templates.<br /><br />This approach allows you to use vue components (<a href="https://medium.com/@rodrigosmaniotto/integrating-django-and-vuejs-with-vue-cli-3-and-webpack-loader-145c3b98501a">built with the build pipeline</a>) in your existing html without making your entire application a SPA.<br /><br />Initially I started following this guide: <a href="https://robinverton.de/blog/2018/06/22/django-vue.js-integration-as-a-widget/">https://robinverton.de/blog/2018/06/22/django-vue.js-integration-as-a-widget/</a><br /><br />Vue-custom-element turned that code into 3 lines and allowed using components like normal (without the data attributes).<br /><br />This approach seems much better than: <a href="https://vsupalov.com/vue-js-in-django-template/">https://vsupalov.com/vue-js-in-django-template/</a> Which doesn't allow for using a build pipeline for babel, linting, testing, single page components, live reload, modules with import/require, etc.<br /><br />Targeting your builds for web components might do the same thing: <a href="https://cli.vuejs.org/guide/build-targets.html#web-component">https://cli.vuejs.org/guide/build-targets.html#web-component</a> It says it's not compatible with IE11, but that might not be a big deal in most situations.Django - Requiring Prefetching2020-05-06T03:21:00+00:00https://www.paulsprogrammingnotes.com/2020/05/django-requiring-prefetching<a href="https://github.com/pawl/django_require_prefetch_select_related_example">https://github.com/pawl/django_require_prefetch_select_related_example</a> <br /><br />I made a repository to try to figure out the best way to require prefech_related/select_related when fetching a model's related objects. I also want to figure out a good pattern for avoiding duplicated prefetch_related/select_related for multiple places in a codebase that need the related objects eager loaded in the same way.<br /><br /><br />Nginx - WebDAV2020-04-29T23:44:00+00:00https://www.paulsprogrammingnotes.com/2020/04/nginx-webdavJust learned Nginx has WebDAV support: <a href="https://nginx.org/en/docs/http/ngx_http_dav_module.html">https://nginx.org/en/docs/http/ngx_http_dav_module.html</a><br /><br />It's a lot faster than the Python package I was using: <a href="https://github.com/mar10/wsgidav">https://github.com/mar10/wsgidav</a><br /><br />Wsgidav is great though. It's super customizable and the documentation is great too.Django - ModelChoiceField queryset caching2020-04-23T03:58:00+00:00https://www.paulsprogrammingnotes.com/2020/04/django-modelchoicefield-queryset-caching<p>It would be nice to know a better way to cache the ModelChoiceField’s queryset when it’s used in a form that runs “is_valid()” in a loop (like Formsets do). The best way I know how at the moment is by not using a ModelChoiceField at all. The solution requires using a ChoiceField, running the query for choices outside of the loop, then passing the choices into the form to override the ChoiceField choices.</p>
<p>Here’s an example:</p>
<noscript><pre># See full example: https://github.com/pawl/django_modelchoicefield_caching/blob/master/myapp/views.py#L31-L51
for song in playlist:
form_data = {'title': song["title"], 'artist': song["artist"]}
song_form = forms.SongFormWithModelChoiceField(data=form_data)
song_form.is_valid() # runs a query to get the ModelChoiceField queryset each time
print('ModelChoiceField - query count AFTER validating all songs:',
len(connection.queries)) # 5 queries
# query for choices outside of the loop to prevent unnecessary queries
artist_choices = [(artist.pk, artist.name)
for artist in models.Artist.objects.all()]
for song in playlist:
form_data = {'title': song["title"], 'artist': song["artist"]}
# pass choices into the Form
song_form = forms.SongFormWithChoiceField(
artist_choices=artist_choices,
data=form_data)
song_form.is_valid()
print('ChoiceField w/ choices passed in - query count AFTER validating all songs:',
len(connection.queries)) # 6 queries (only 1 more query!)</pre></noscript>
<script src="https://gist.github.com/fdaa8c916bf914cbe28ef60acd8ff78f.js"> </script>
Django API Forms2020-04-23T03:06:00+00:00https://www.paulsprogrammingnotes.com/2020/04/django-api-formsJust found a really cool library for using Django's built-in forms to validate JSON: <a href="https://github.com/Sibyx/django_api_forms">https://github.com/Sibyx/django_api_forms</a><br /><br />I made a small example with it to show how it works with ModelChoiceFields: <a href="https://github.com/pawl/django_api_forms_modelchoicefield_example">https://github.com/pawl/django_api_forms_modelchoicefield_example</a><br /><br />Seems like a more lightweight version of DRF's serializers (but only for deserializing and validating) and with a similar API to Django's built-in forms.Learning Vue Through Terrible Pull Requests2020-04-18T18:58:00+00:00https://www.paulsprogrammingnotes.com/2020/04/learning-vue-through-terrible-pullI made a pull request asking to "remove the unused public/index.html" file from django-vue-template: <a href="https://github.com/gtalarico/django-vue-template/pull/53">https://github.com/gtalarico/django-vue-template/pull/53</a><br /><br />Well, the author responded and it turns out it's definitely used (by vue magic): <a href="https://cli.vuejs.org/guide/html-and-static-assets.html">https://cli.vuejs.org/guide/html-and-static-assets.html</a><br /><br />Oops, definitely should have googled "vue public/index.html" before making that pull request. I made another pull request to add a link about it to the django-vue-template docs: <a href="https://github.com/gtalarico/django-vue-template/pull/54">https://github.com/gtalarico/django-vue-template/pull/54</a>Server Side Templates Vs Front End Framework2020-04-12T04:57:00+00:00https://www.paulsprogrammingnotes.com/2020/04/server-side-templates-vs-front-endThese Reddit comments have some of the best descriptions I've seen for the type of scenario where you'd use a front end framework over server side templates:<br /><br />"Server side templating can work really well when the website is static. The problem is that as you add more and more dynamic elements, you either have to go to the server and re-render everything with every change (which can be slow), or you can make the changes on the client side with something like jQuery. But the latter option presents a challenge because you'll eventually find yourself duplicating functionality.<br /><br />For example, imagine a to-do list website that you render on the server with a templating language. It fetches the user's to-do items from the database and then renders the list. Now let's say you want to let a user add a new item without re-rendering the entire page. You could use AJAX to tell the server to add the new item, but now you need to update your UI list accordingly. You could use jQuery to construct a new DOM element and append it, but now that list element exists in two different places: in your jQuery code and in your template on the server. If your website becomes very dynamic, this situation can get really unwieldy from a programming perspective.<br /><br />Both Angular (from what I've read) and React (from personal experience) are highly suitable for creating single page apps, with the goal of creating a dynamic, fast, and maintainable website. There are other benefits, as well as downsides, to using Angular and React, but I hope this clears it up a little bit. Both of them make server side templating unnecessary by moving UI logic/rendering from your server to the client's browser. So a user who visits your React powered to-list website would get the React code as a JS bundle, it would make an AJAX call to your server to get just the data, and then it would render the UI based on the data." -/u/VertiGuo (<a href="https://www.reddit.com/r/node/comments/6t22cr/back_end_templating_engine_or_front_end_framework/dlhukkf/">https://www.reddit.com/r/node/comments/6t22cr/back_end_templating_engine_or_front_end_framework/dlhukkf/)</a><br /><br />"for example putting together a form, and handle the response is easy, but if you have to manage a list of items where the user can add/remove items, or change the schema of items based on a selector (e.g. phone type or address) you have to split the logic, and put some on client side scripts, which can become very messy. not to mention live updates like chat, comments etc.<br /><br />creating a separate backend and frontend solves this problem, as the backend's responsibility will be only manipulating the database, retrieve data and run jobs providing an api and the frontend's will be the only the representation of these data, providing the user interface. this separation lets developers use different technologies and languages on the two side, as well as specializing in these. for the beginning this approach is a bit hard to learn, but it will worth it on the long run." -/u/bdvx (<a href="https://www.reddit.com/r/webdev/comments/cbrdte/html_templating_vs_spa/etifiar/">https://www.reddit.com/r/webdev/comments/cbrdte/html_templating_vs_spa/etifiar/</a>)Integrating Django and Vue.js2020-04-08T01:03:00+00:00https://www.paulsprogrammingnotes.com/2020/04/integrating-django-and-vuejsThis is a great guide for including vue bundles in server side templates: <a href="https://medium.com/@rodrigosmaniotto/integrating-django-and-vuejs-with-vue-cli-3-and-webpack-loader-145c3b98501a">https://medium.com/@rodrigosmaniotto/integrating-django-and-vuejs-with-vue-cli-3-and-webpack-loader-145c3b98501a</a><br /><br />Otherwise, if you're starting a fresh project with a separate front-end, this is a good template: <a href="https://github.com/gtalarico/django-vue-template/">https://github.com/gtalarico/django-vue-template/</a>The deepest reason why modern JavaScript frameworks exist2020-04-04T02:16:00+00:00https://www.paulsprogrammingnotes.com/2020/04/the-deepest-reason-why-modernAnother really good article about why using a front-end framework is a good idea:<br /><a href="https://medium.com/dailyjs/the-deepest-reason-why-modern-javascript-frameworks-exist-933b86ebc445">https://medium.com/dailyjs/the-deepest-reason-why-modern-javascript-frameworks-exist-933b86ebc445</a><br /><br /> Make your life easier with Vue.js2020-04-02T16:37:00+00:00https://www.paulsprogrammingnotes.com/2020/04/make-your-life-easier-with-vuejsThis is a really great article that shows how vue can make dynamic forms so much better: <a href="https://dev.to/tsanak/make-your-life-easier-with-vuejs-4mj5">https://dev.to/tsanak/make-your-life-easier-with-vuejs-4mj5</a>Django - RelatedManager.set not removing models2020-02-14T03:36:00+00:00https://www.paulsprogrammingnotes.com/2020/02/django-relatedmanagerset-not-removing<noscript><pre>In my opinion, one of django's biggest gotchas is using `RelatedManager.set` with models that have non-nullable ForeignKey fields.
### Example
Models.py (with a one-to-many relationship and a non-nullable ForeignKey):
```
class Reporter(models.Model):
name = models.CharField(max_length=255)
class Article(models.Model):
title = models.CharField(max_length=255)
reporter = models.ForeignKey(Reporter, on_delete=models.CASCADE)
```
Guess how many articles are on the reporter after you do this?:
```
reporter = Reporter.objects.create(name="paul")
# add 2 articles to the reporter
article_1 = reporter.article_set.create(title="blah")
reporter.article_set.create(title="blah2")
# set only 1 article on the reporter
reporter.article_set.set([article_1])
```
I expected just one article to be on the reporter. Nope:
```
reporter.article_set.count()
2
```
How do the docs say it should work?
>This method accepts a clear argument to control how to perform the operation. If False (the default), the elements missing from the new set are removed using remove() and only the new ones are added.
Well, crap, what's going on?
It turns out you need to read the docs for `RelatedManager.remove` for a hint:
>For ForeignKey objects, this method only exists if null=True.
So, it turns out that `set()` is using `remove()` and `remove()` is only available on models with ForeignKeys that are `null=True`. What does `set()` do with things it needs to `remove()` when `remove()` isn't available? Nothing...</pre></noscript>
<script src="https://gist.github.com/74c734a1da1758b021b9f1c69b84397e.js"> </script>
Django - ChoiceField Display Text2020-01-14T01:57:00+00:00https://www.paulsprogrammingnotes.com/2020/01/django-choicefield-display-textI made a repo to demonstrate how to get a django ChoiceField's current selection's display text: <a href="https://github.com/pawl/django_choicefield_display_example/">https://github.com/pawl/django_choicefield_display_example/</a><br /><br />I came across a few other examples, but they didn't do a good job of displaying the initial value or handling choices with integer values. Python - Pipenv to pip-tools2019-12-20T17:31:00+00:00https://www.paulsprogrammingnotes.com/2019/12/python-pipenv-to-pip-toolsI've been using Pipenv for the last few months and my biggest issue is that "--keep-outdated" has been broken in the latest release (2018.11.26) for a while. I've needed to install Pipenv from the master branch to make it functional. However, the last time I used "--keep-outdated" from the master branch, it wouldn't automatically update the hash of the dependency being updated.<br /><br />Updating specific requirements is something I need to do pretty often, and it's not fun to explain all the Pipenv quirks to the team.<br /><br />Pip-tools looks like it does everything I need and has fewer quirks, so I ended up making the switch.<br /><br />Pipenv uses pip-tools under the hood, so the migration to pip-tools was very smooth. The migration process was:<br /><ol><li>Copy the dev-packages and packages sections of the Pipfile to their own requirements.in files.</li><li>Run pip-compile</li><li>Copy over the specific versions and hashes from the Pipfile.lock to the generated requirements.txt. </li></ol>I did have a small issue where updating a specific package with pip-tools removed a bunch of dependencies from the requirements.txt unexpectedly, but running pip-compile with "--rebuild" fixed it.HTML5 - video playsinline2019-07-25T04:05:00+00:00https://www.paulsprogrammingnotes.com/2019/07/html5-video-playsinlineI added "autoplay muted" to my <video> tag to make a video autoplay in a carousel. It worked on desktop Chrome or Firefox, but didn't work on iOS Chrome or Safari. <br /><br />I learned about the "playsinline" fix from here: <a href="https://webkit.org/blog/6784/new-video-policies-for-ios/">https://webkit.org/blog/6784/new-video-policies-for-ios/</a><br /><br />Adding "playsinline" to my video tag fixed it.<br /><br />ESP32 Plant Sensor2019-07-12T01:43:00+00:00https://www.paulsprogrammingnotes.com/2019/07/esp32-plant-sensor<p>I set up a ESP32 houseplant soil water + temperature + humidity + light sensor that sends me a daily status update message.</p>
<p>Here’s the code for it: <a href="https://gist.github.com/pawl/6a408afde1411c6b1f58980e3d1dff83">https://gist.github.com/pawl/6a408afde1411c6b1f58980e3d1dff83</a></p>
<p><img src="/assets/images/06_2.png" alt="esp32 plant sensor 1" /></p>
<p><img src="/assets/images/36_2.png" alt="esp32 plant sensor 2" /></p>
<p><img src="/assets/images/2019-07-11+20.38.41.png" alt="esp32 plant sensor 3" /></p>
AWS - redirecting domain to url using a 302 redirect (without running a server)2019-07-11T00:26:00+00:00https://www.paulsprogrammingnotes.com/2019/07/aws-redirecting-domain-to-url-using-302I wanted to make a domain name (heckingoodboys.com) redirect to a multisubreddit for dog pictures, but I didn't want to run a web server for it.<br /><br />Here's what I did:<br /><ol><li>Purchase the domain using Route53.</li><li>Create two public s3 buckets (www.heckingoodboys.com and heckingoodboys.com)</li><li>Enable "Static website hosting" on www.heckingoodboys.com and redirect to heckingoodboys.com.</li><li>Enable "Static website hosting" on heckingoodboys.com, select "use this bucket to host this website", and use routing rules similar to this:<br /><blockquote class="tr_bq"><RoutingRules><br /> <RoutingRule><br /> <Redirect><br /> <Protocol>https</Protocol><br /> <HostName>www.reddit.com</HostName><br /> <HttpRedirectCode>302</HttpRedirectCode><br /> <ReplaceKeyPrefixWith>user/heckingoodboys/m/heckingoodboys/</ReplaceKeyPrefixWith><br /> </Redirect><br /> </RoutingRule><br /></RoutingRules></blockquote></li><li>Back to Route53 - Create an A record for both www.heckingoodboys.com and heckingoodboys.com using the alias to their respective buckets. (this will be the first option in autocomplete)</li></ol>For more details: <a href="https://medium.com/@P_Lessing/single-page-apps-on-aws-part-1-hosting-a-website-on-s3-3c9871f126">https://medium.com/@P_Lessing/single-page-apps-on-aws-part-1-hosting-a-website-on-s3-3c9871f126</a><br /><br />Why not just use a CNAME from www.heckingoodboys.com to heckingoodboys.com? AWS says they don't charge for aliases, but they do charge for CNAMEs. So, I used an alias to a bucket instead. Django - When To Use Signals?2018-02-08T04:18:00+00:00https://www.paulsprogrammingnotes.com/2018/02/django-when-to-use-signalsAccording to this blog post, almost never: <a href="https://lincolnloop.com/blog/django-anti-patterns-signals/">https://lincolnloop.com/blog/django-anti-patterns-signals/</a>Django - Custom Storage Backends2018-02-08T04:17:00+00:00https://www.paulsprogrammingnotes.com/2018/02/django-storage-backendsI just learned about the custom storage systems in Django.<br /><br />What can you do with custom storage systems?:<br /><blockquote class="tr_bq">"Django abstracts file storage using storage backends, from simple filesystem storage to things like S3. This can be used for processing file uploads, storing static assets, and more." -<a href="https://tartarus.org/james/diary/2013/07/18/fun-with-django-storage-backends">https://tartarus.org/james/diary/2013/07/18/fun-with-django-storage-backends</a></blockquote>Sphinx Search - Lessons Learned2018-01-20T02:26:00+00:00https://www.paulsprogrammingnotes.com/2018/01/sphinx-search-lessons-learnedHere are a few things I've learned while working on a project that uses Sphinx search:<br /><div><ul><li>It's important to know the difference between fields and attributes. <a href="https://www.percona.com/blog/2013/01/15/sphinx-search-performance-optimization-attribute-based-filtering/">Attributes are basically unindexed columns</a> and you should try to avoid filtering only on these columns. Fields support full text search. </li><li>It supports its own custom binary protocol and the MySQL protocol (recently they also added a HTTP API). When you see "listen = localhost:9306:mysql41" in the config, that means it's listening for MySQL protocol traffic on port 9306.</li><li><a href="https://github.com/a1tus/sphinxapi-py3">https://github.com/a1tus/sphinxapi-py3</a> appears to be the best Python client for the binary api at the moment. This doesn't support INSERTing things into the index (you'll need to use the MySQL protocol for that).</li><li>The version of <a href="https://pypi.python.org/pypi/sphinxapi-py3/2.1.11">sphinxapi-py3 on pypi</a> is a <a href="https://github.com/atuchak/sphinxapi-py3">fork</a> with just a few minor fixes and appears to be safe.</li><li>It does not match partial words by default. Turning on partial matching can also <a href="http://sphinxsearch.com/docs/current/conf-max-substring-len.html">increase the size of your index dramatically</a>. You can also limit the fields that support partial matching with the "infix_fields" and "prefix_fields" setting.</li><li>Stemmers aren't turned on by default. So, searching for "dog" will not match "dogs".</li><li>Most special characters ($, @, &, etc) are ignored by default. You will need to add them to <a href="http://sphinxsearch.com/docs/current/conf-charset-table.html">charset_table</a> if you want them to be searchable.</li><li>Ruby's thinking-sphinx looks much more battle tested than all of the Python binary api clients: <a href="https://github.com/pat/thinking-sphinx">https://github.com/pat/thinking-sphinx</a></li><li>You will need to use a real-time index if you want to INSERT/DELETE records immediately.</li><li>If you're using a real-time index, you will probably need to increase the "rt_mem_limit" from its default of 128mb. If this limit is too low, you'll see a high number of "disk chunks" when you run the "SHOW INDEX rtindex STATUS" query. More info: <a href="http://sphinxsearch.com/blog/2014/02/12/rt_performance_basics/">http://sphinxsearch.com/blog/2014/02/12/rt_performance_basics/</a></li><li>You have to use a special dialect if you want to use SQLAlchemy with sphinx: <a href="https://github.com/conversant/sqlalchemy-sphinx">https://github.com/conversant/sqlalchemy-sphinx</a></li><li>This appears to be the best Dockerfile for sphinx: <a href="https://github.com/leodido/dockerfiles">https://github.com/leodido/dockerfiles</a></li></ul><div>I probably won't be using Sphinx search for any new projects. Elasticsearch seems preferable these days.</div></div>MySQL - Duplicate Errors & Trailing Whitespace2017-12-12T01:36:00+00:00https://www.paulsprogrammingnotes.com/2017/12/mysql-duplicate-errors-trailingI had a unique constraint on a VARCHAR column and I inserted two rows with the following values:<br /><br /><ol><li>"name" (without trailing whitespace)</li><li>"name " (with trailing whitespace)</li></ol><br />To my surprise, I got a duplicate error on that 2nd insert. It turns out that MySQL ignores that trailing whitespace when it makes comparisons.<br /><br />The MySQL docs say this: "All MySQL collations are of type PAD SPACE. This means that all CHAR, VARCHAR, and TEXT values are compared without regard to any trailing spaces. “Comparison” in this context does not include the LIKE pattern-matching operator, for which trailing spaces are significant." (<a href="https://dev.mysql.com/doc/refman/5.7/en/char.html">https://dev.mysql.com/doc/refman/5.7/en/char.html</a>)<br /><br />The solution? You should probably be trimming trailing whitespace in your API endpoints and on your front-end.Gevent + Requests Performance With verify=True/False2017-08-11T00:48:00+00:00https://www.paulsprogrammingnotes.com/2017/08/gevent-requests-performance-withIf you use gevent with requests.get on a HTTPS URL with the default verify=True enabled, you'll see almost 2x longer execution times than with verify=False.<div><br /></div><div><div>I made a script to test: <a href="https://gist.github.com/pawl/56100a4ef958374a433840be8037b11b">https://gist.github.com/pawl/56100a4ef958374a433840be8037b11b</a></div></div><div><br /></div><div>Here are the results:</div><div><blockquote class="tr_bq">verify=True took: 40.3454630375 secs<br />verify=False took: 39.3803040981 secs<br />gevent verify=True took: 2.23735189438 secs<br />gevent verify=False took: 1.58263015747 secs</blockquote></div><div>I suspect that gevent is having trouble using pyopenssl concurrently because it's a C library.</div>Backing up or dumping a memcached server2017-06-28T03:35:00+00:00https://www.paulsprogrammingnotes.com/2017/06/backing-up-or-dumping-memcached-serverI was needing to move from an old cache server to a larger one, but I wanted to do it without flushing cache.<br /><br />The first thing I came across was this "memcached-tool" which has a dump command: <a href="https://github.com/memcached/memcached/blob/master/scripts/memcached-tool">https://github.com/memcached/memcached/blob/master/scripts/memcached-tool</a><br /><br />There's another article that mentions using memdump and memcat: <a href="http://www.dctrwatson.com/2010/12/how-to-dump-memcache-keyvalue-pairs-fast/">http://www.dctrwatson.com/2010/12/how-to-dump-memcache-keyvalue-pairs-fast/</a><br /><br />Unfortunately, those methods only dumped a few mb of data. This post explains why: <a href="https://stackoverflow.com/a/13941700">https://stackoverflow.com/a/13941700</a><br /><span style="color: #242729; font-family: inherit; font-size: inherit; font-style: inherit; font-variant-caps: inherit; font-variant-ligatures: inherit; font-weight: inherit;"><br /></span><blockquote class="tr_bq"><span style="color: #242729; font-family: inherit; font-size: inherit; font-style: inherit; font-variant-caps: inherit; font-variant-ligatures: inherit; font-weight: inherit;">You can only dump one page per slab class (1MB of data)</span></blockquote>So, I ended up writing a script that loops through the expected cache keys, gets the data in cache, then sets the data in the new cache server. Microservice Best Practices2017-06-27T05:59:00+00:00https://www.paulsprogrammingnotes.com/2017/06/microservice-best-practices<a href="http://www.vinaysahni.com/best-practices-for-building-a-microservice-architecture">http://www.vinaysahni.com/best-practices-for-building-a-microservice-architecture</a>Gunicorn - "Resource temporarily unavailable"2017-06-13T05:14:00+00:00https://www.paulsprogrammingnotes.com/2017/06/gunicorn-resource-temporarilyAre you seeing this error in your logs while your server is under high load?:<br /><blockquote class="tr_bq">[error] 10#0: *14843 connect() to unix:/tmp/gunicorn.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 96.44.145.186, server: , request: "GET / HTTP/1.0", upstream: "http://unix:/tmp/gunicorn.sock:/", host: "45.55.46.84"</blockquote>I ended up making an example dockerfile with nginx + gunicorn + flask to reproduce this problem: <a href="https://github.com/pawl/somaxconn_test">https://github.com/pawl/somaxconn_test</a><br /><br />Bumping the "net.core.somaxconn" setting ended up fixing it.SQLAlchemy - empty lists + in_() causing crazy queries (before 1.2.0)2017-03-18T02:31:00+00:00https://www.paulsprogrammingnotes.com/2017/03/sqlalchemy-empty-lists-in-causing-crazy<p>Before SQLAlchemy 1.2.0, if you use an empty list with in_(), it will emit some crazy SQL that will query your entire table. The best solution is probably to upgrade to SQLAlchemy 1.2.0.</p>
<noscript><pre># demonstrates the issue fixed in: https://bitbucket.org/zzzeek/sqlalchemy/issues/3907
from sqlalchemy import create_engine, Column, Integer
from sqlalchemy.orm import scoped_session, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
engine = create_engine('mysql://root@localhost/test?charset=utf8mb4',
convert_unicode=True,
echo=True)
session = scoped_session(sessionmaker(autocommit=False,
autoflush=False,
bind=engine))
Base = declarative_base()
Base.query = session.query_property()
class Post(Base):
__tablename__ = 'posts'
id = Column(Integer, primary_key=True)
#Base.metadata.drop_all(engine)
Base.metadata.create_all(engine)
# create new rows if database is empty
for x in range(10):
session.add(Post())
session.commit()
# empty lists trigger insane queries before 1.2.0
Post.query.filter(Post.id.in_([])).all()
"""
SELECT posts.id AS posts_id
FROM posts
WHERE posts.id != posts.id
"""</pre></noscript>
<script src="https://gist.github.com/893c185519823b76150faa6a4bf74be5.js"> </script>
The Robustness Principle2017-03-14T06:05:00+00:00https://www.paulsprogrammingnotes.com/2017/03/the-robustness-principleI learned about this at a talk called "Implementing Evolvable APIs" at SXSW: <a href="https://en.wikipedia.org/wiki/Robustness_principle">https://en.wikipedia.org/wiki/Robustness_principle</a><br /><br />For example, making an API that throws errors when an unexpected parameter is provided is a bad idea. What if you need to make changes to the client to add the new parameter? You will need to make sure you deploy the code on the server side first, otherwise it will cause errors.SQLAlchemy - DISTINCT, LIMIT, or OFFSET Causing Subqueries2017-02-15T02:26:00+00:00https://www.paulsprogrammingnotes.com/2017/02/sqlalchemy-distinct-limit-or-offsetThis section at the bottom of <a href="http://docs.sqlalchemy.org/en/latest/orm/loading_relationships.html#the-zen-of-eager-loading">The Zen of Eager loading</a> is really important:<br /><blockquote class="tr_bq">When using joined eager loading, <b>if the query contains a modifier that impacts the rows returned externally to the joins, such as when using DISTINCT, LIMIT, OFFSET or equivalent, the completed statement is first wrapped inside a subquery, and the joins used specifically for joined eager loading are applied to the subquery</b>. SQLAlchemy’s joined eager loading goes the extra mile, and then ten miles further, to absolutely ensure that it does not affect the end result of the query, only the way collections and related objects are loaded, no matter what the format of the query is.</blockquote>I made an example to illustrate this here: <a href="https://gist.github.com/pawl/cb57e0ddbdd0b2e64b75e94116873367">https://gist.github.com/pawl/cb57e0ddbdd0b2e64b75e94116873367</a><br /><br />On MySQL this can be responsible for some really poor query performance, because it can cause it to use temporary tables and filesort.<br /><br />The best way I've found to prevent the subqueries is by first querying for the ids only, then running another query that includes all relations. For example:<br /><blockquote class="tr_bq">ids = session.query(Product.id).limit(20)<br />Product.query.filter(Product.id.in_(ids))</blockquote>SQLAlchemy - "Base.query = db_session.query_property()"2017-02-13T00:23:00+00:00https://www.paulsprogrammingnotes.com/2017/02/sqlalchemy-basequeryI just saw this line in the <a href="http://flask.pocoo.org/docs/0.12/patterns/sqlalchemy/">SQLAlchemy flask patterns example</a>:<br /><blockquote class="tr_bq">Base.query = db_session.query_property()</blockquote><div>I went searching for what it does and found this: <a href="https://eoyilmaz.blogspot.com/2014/01/i-feel-so-much-hungry-about-posting.html">https://eoyilmaz.blogspot.com/2014/01/i-feel-so-much-hungry-about-posting.html</a></div><div><br /></div><div>Now I know how Flask-SQLAlchemy made queries work with Model.query instead of session.query(Model).</div>SQLAlchemy - Lost connection to MySQL server during query2017-02-11T18:55:00+00:00https://www.paulsprogrammingnotes.com/2017/02/sqlalchemy-lost-connection-to-mysql<div>Here's was my situation:</div><ul><li>The database was set up behind behind an AWS ELB and HAProxy. </li><li>The idle connection timeout on the ELB was set to 60 mins.</li><li>All the relevant timeouts on HAProxy seemed to be set to 60 mins too. </li><li>The pool_recycle in SQLAlchemy was set to 30 mins.</li><li>I was still seeing the occasional "Lost connection to MySQL server during query" when small queries were running after the connection had some time to sit around.</li></ul><div>The solution ended up being setting my pool_recycle down to 5 mins, but I'm still not sure what was causing connections to time out after 5 mins.</div><div><br /></div><div><div>There are definitely other things that can cause this problem too. For example, it can happen if your data exceeds max_allowed_packet. See this page for more details: <a href="https://dev.mysql.com/doc/refman/5.7/en/error-lost-connection.html">https://dev.mysql.com/doc/refman/5.7/en/error-lost-connection.html</a></div></div><div><br /></div><div><div>Most of this also applies for "MySQL server has gone away".</div></div><div><br /></div><div><div>You should also make sure your pool_recycle is set lower than your 'interactive_timeout' and 'wait_timeout' properties in the mysql config file to the values you need.</div><div><br /></div></div>SQLAlchemy - Unexpected Lazy Loading 2017-01-27T01:32:00+00:00https://www.paulsprogrammingnotes.com/2017/01/sqlalchemy-unexpected-lazy-loadingIf you're seeing unexpected lazy loading on a lazy="joined" relationship in SQLAlchemy, it might be because you're accessing those relationships after you've already run session.commit(). By default, session.commit() will expire the data on your relationships, meaning it will try to fetch it again next time you try to access those attributes.<br /><br />The relevant section of the docs for <a href="http://docs.sqlalchemy.org/en/latest/orm/session_api.html#sqlalchemy.orm.session.Session.commit">session.commit()</a>:<br /><blockquote class="tr_bq">By default, the Session also expires all database loaded state on all ORM-managed attributes after transaction commit. This so that subsequent operations load the most recent data from the database. This behavior can be disabled using the expire_on_commit=False option to sessionmaker or the Session constructor.</blockquote><br /><br />Python - David Beazley's "Understanding The Python GIL"2017-01-20T01:05:00+00:00https://www.paulsprogrammingnotes.com/2017/01/python-david-beazleys-understandingThis talk explains a lot about Python's multithreading: <a href="http://www.dabeaz.com/python/UnderstandingGIL.pdf">http://www.dabeaz.com/python/UnderstandingGIL.pdf</a><br /><br />In Python 2.7, it knows to switch to different thread whenever it starts waiting on IO. And, it will check if it needs to switch threads every 100 bytecode instructions.SQLAlchemy - Is is necessary to commit after session.execute?2017-01-17T04:50:00+00:00https://www.paulsprogrammingnotes.com/2017/01/sqlalchemy-is-is-necessary-to-commit<p>The docs for <a href="http://docs.sqlalchemy.org/en/latest/orm/session_api.html#sqlalchemy.orm.session.Session.execute">session.execute</a> say:</p>
<blockquote>
<p>Execute a SQL expression construct or string statement <strong>within the current transaction</strong>.</p>
</blockquote>
<p>So, the answer is yes, you need to issue a commit after running session.execute().</p>
<p>Here’s a code example along with the output from “echo=True” showing it begins the session but doesn’t end it unless you run session.commit():</p>
<noscript><pre>from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
engine = create_engine('mysql://root@localhost/test')
db_session = scoped_session(sessionmaker(bind=engine))
db_session.execute('''
INSERT INTO user (first_name, last_name, username, email)
VALUES ("blah", "blah", "blah", "blah@blah.com");
''')</pre></noscript>
<script src="https://gist.github.com/f3d66fb501065a589ce5ba2985de0ba2.js"> </script>
Global .gitignore2016-11-27T02:37:00+00:00https://www.paulsprogrammingnotes.com/2016/11/global-gitignore<a href="https://help.github.com/articles/ignoring-files/#create-a-global-gitignore">https://help.github.com/articles/ignoring-files/#create-a-global-gitignore</a>SQLAlchemy 1.1.4's New "server_side_cursors" Option2016-11-18T05:43:00+00:00https://www.paulsprogrammingnotes.com/2016/11/sqlalchemy-114s-new-serversidecursorsBefore SQLAlchemy 1.1.4, if you wanted to stream your MySQL query using server side cursors, you would need to run your query using "execution_options(stream_results=True)" and pass SSCursor into create_engine's connect_args. Now, all you need to do is pass "server_side_cursors=True" into create_engine and it will automatically stream the results for your select queries.<br /><br /><a href="http://docs.sqlalchemy.org/en/latest/dialects/mysql.html#server-side-cursors">http://docs.sqlalchemy.org/en/latest/dialects/mysql.html#server-side-cursors</a>Is it necessary to run SQLAlchemy's session.remove()?2016-10-23T22:21:00+00:00https://www.paulsprogrammingnotes.com/2016/10/is-it-necessary-to-run-sqlalchemysIf you're using flask-sqlalchemy, then the answer is no - it's already doing it for you when the request finishes: <a href="https://github.com/mitsuhiko/flask-sqlalchemy/blob/7d65e2dc1f4e798f3234aaef01e615e3db319b05/flask_sqlalchemy/__init__.py#L836">https://github.com/mitsuhiko/flask-sqlalchemy/blob/2.1/flask_sqlalchemy/__init__.py#L823</a><br /><br />If you're using scoped_session and not explicitly running session.remove(), then your connections will only be returned to the pool after your thread finishes and garbage collection occurs. It's not a good idea to leave this to garbage collection, because you can't guarantee your connections will be returned to the pool when your application is busy. This will often lead to errors like this: "TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30"<br /><br />The solution is to make sure session.remove() runs when the work finishes (like flask-sqlalchemy does), as described here: <a href="http://docs.sqlalchemy.org/en/latest/orm/contextual.html#using-thread-local-scope-with-web-applications">http://docs.sqlalchemy.org/en/latest/orm/contextual.html#using-thread-local-scope-with-web-applications</a><br /><br />Another alternative is not making the session global and using a context manager, as described at the bottom of this section: <a href="http://docs.sqlalchemy.org/en/latest/orm/session_basics.html#when-do-i-construct-a-session-when-do-i-commit-it-and-when-do-i-close-it">http://docs.sqlalchemy.org/en/latest/orm/session_basics.html#when-do-i-construct-a-session-when-do-i-commit-it-and-when-do-i-close-it</a> However, this can make things difficult when you need to access the query results outside of the context manager. If you access the query results outside of the context manager without running session.expunge_all(), you'll see all kinds of errors like this: "DetachedInstanceError: Instance <ReportingJob at 0xa41cd8c> is not bound to a Session; attribute refresh operation cannot proceed"<br /><br />Another option is setting <a href="http://docs.sqlalchemy.org/en/latest/orm/session_transaction.html#autocommit-mode">autocommit</a> to True, but that has quite a few gotchas. Turning on autocommit will acquire connections from the engine on an as-needed basis and return them immediately after their use. You will have to explicitly start transactions with session.begin() if you enable it. It seems like this setting should be called "autotransaction" instead. It's also not very efficient, because it needs to get and return a connection each time you run a query.PHP - max_execution_time doesn't work for socket operations2016-09-02T23:07:00+00:00https://www.paulsprogrammingnotes.com/2016/09/php-maxexecutiontime-doesnt-work-forIn new relic transaction logs, I was seeing some PHP requests last as long as an hour. It turns out that time spent waiting on sockets doesn't apply toward max_execution_time, and loops that involved waiting on sockets could end up taking a really long time without timing out.<br /><br />PHP docs have this to say about max_execution_time:<br />"The set_time_limit() function and the configuration directive max_execution_time only affect the execution time of the script itself. Any time spent on activity that happens outside the execution of the script such as system calls using system(), stream operations, database queries, etc. is not included when determining the maximum time that the script has been running. This is not true on Windows where the measured time is real."<br /><br />The solution ended up being setting <b>request_terminate_timeout</b> in php-fpm.Python - Cachetools LRUCache KeyError2016-08-27T22:34:00+00:00https://www.paulsprogrammingnotes.com/2016/08/python-cachetools-lrucache-keyerrorIf your application is threaded and you're getting a "KeyError" while using the non-decorator version of cachetool's LRUCache, then you need to put whatever is manipulating the cache object inside of a lock. Also, since LRUCache is modified when values are gotten from it, you will also need to make sure you're locking when you get values from cache too. If you can use the decorator version of LRUCache, that's preferred since it has built-in locking.<br /><br />Here's an example of the error:<br /><br /><script src="https://gist.github.com/pawl-rs/5d99ece9eaea723c901dfdb27bd4d9c1.js"></script> And an example of the fix: <a href="https://bitbucket.org/zzzeek/dogpile.cache/pull-requests/32/add-a-cachetools-lru-lfu-in-memory-backend/diff#comment-22242704">https://bitbucket.org/zzzeek/dogpile.cache/pull-requests/32/add-a-cachetools-lru-lfu-in-memory-backend/diff#comment-22242704</a>Javascript's Round vs PHP's Round2016-06-22T06:33:00+00:00https://www.paulsprogrammingnotes.com/2016/06/javascripts-round-vs-phps-roundToday I learned that by default PHP rounds differently than javascript.<br /><h3>PHP (using the default PHP_ROUND_HALF_UP)</h3><blockquote class="tr_bq">php > echo round(-1.5);<br />-2</blockquote>"Round val up to precision decimal places away from zero, when it is half way there. Making 1.5 into 2 and -1.5 into -2."<br /><div><a href="https://secure.php.net/manual/en/function.round.php">https://secure.php.net/manual/en/function.round.php</a></div><br /><h3>Javascript</h3><blockquote class="tr_bq">Math.round(-1.5);<br />-1</blockquote>"For negative numbers, if the decimal portion is exactly -0.5, the return value is the smallest integer that is greater than the number."<br /><a href="https://msdn.microsoft.com/en-us/library/5cza0web(v=vs.94).aspx">https://msdn.microsoft.com/en-us/library/5cza0web(v=vs.94).aspx</a>Django-Moderation2016-05-21T22:41:00+00:00https://www.paulsprogrammingnotes.com/2016/05/django-moderationIt would be nice if there were a sqlalchemy/flask equivalent to this: <a href="https://github.com/dominno/django-moderation">https://github.com/dominno/django-moderation</a><br /><br />The equivalent might use sqlalchemy_utils for its generic relationships: <a href="https://sqlalchemy-utils.readthedocs.io/en/latest/generic_relationship.html">https://sqlalchemy-utils.readthedocs.io/en/latest/generic_relationship.html</a>Modified Preorder Tree Traversal2016-05-20T03:25:00+00:00https://www.paulsprogrammingnotes.com/2016/05/modified-preorder-tree-traversalThis was a great explanation of Modified Preorder Tree Traversal (or MPTT): <a href="https://www.sitepoint.com/hierarchical-data-database-2/">https://www.sitepoint.com/hierarchical-data-database-2/</a><br /><br />A common use-case for MPTT is categorization data, like with the way django-oscar uses django-mptt: <a href="https://github.com/django-mptt/django-mptt">https://github.com/django-mptt/django-mptt</a>Datatables - Uncaught TypeError: Cannot read property 'style' of undefined2016-05-09T00:52:00+00:00https://www.paulsprogrammingnotes.com/2016/05/datatables-uncaught-typeerror-cannotIf you see this error in datatables: "Uncaught TypeError: Cannot read property 'style' of undefined"<br /><br />It's likely that you're missing a header column in your table. Make sure you have the same number of <th> elements as items in the "columns" section of your datatables initialization.Select2 Replacement2016-05-08T23:52:00+00:00https://www.paulsprogrammingnotes.com/2016/05/select2-replacementThis selectize.js library looks like a good replacement for Select2: <a href="https://selectize.github.io/selectize.js/">https://selectize.github.io/selectize.js/</a>Javascript Data Tables2016-05-05T04:04:00+00:00https://www.paulsprogrammingnotes.com/2016/05/javascript-data-tables<br /><ul><li><a href="https://fooplugins.github.io/FooTable/index.html">https://fooplugins.github.io/FooTable/index.html</a> - good, no inline editing though</li><li><a href="https://datatables.net/">https://datatables.net/</a> - editable plugin requires purchase</li><li><a href="https://github.com/mleibman/SlickGrid">https://github.com/mleibman/SlickGrid</a> - unmaintained</li><li><a href="https://handsontable.com/">https://handsontable.com/</a> - responsiveness doesn't work very well</li><li><a href="https://github.com/mindmup/editable-table">https://github.com/mindmup/editable-table</a> - only supports editing</li><li><a href="https://github.com/David-Mulder/paper-datatable">https://github.com/David-Mulder/paper-datatable</a> - pretty, but doesn't seem very battle-tested</li><li><a href="https://www.ag-grid.com/">https://www.ag-grid.com/</a> - no tests</li><li><a href="https://github.com/angular-ui/ui-grid">https://github.com/angular-ui/ui-grid</a> - requires angular</li><li><a href="http://ng-table.com/">http://ng-table.com/</a> - requires angular</li><li><a href="http://backgridjs.com/">http://backgridjs.com/</a> - requires backbone</li><li><a href="https://github.com/daniel-nagy/md-data-table">https://github.com/daniel-nagy/md-data-table</a> - pretty, but requires angular</li><li><a href="https://www.dynatable.com/">https://www.dynatable.com/</a> - not editable</li><li><a href="http://www.jtable.org/">http://www.jtable.org/</a> - last updated in 2014, styling looks dated</li><li><a href="https://datazenit.com/static/sensei-grid/examples/">https://datazenit.com/static/sensei-grid/examples/</a> - simple, editable</li></ul>Falsehoods programmers believe about time and time zones2016-04-19T05:31:00+00:00https://www.paulsprogrammingnotes.com/2016/04/falsehoods-programmers-believe-aboutThis article has a lot of great lessons learned about timezones: <a href="http://www.creativedeletion.com/2015/01/28/falsehoods-programmers-date-time-zones.html">http://www.creativedeletion.com/2015/01/28/falsehoods-programmers-date-time-zones.html</a><br /><br /><br />SQLAlchemy - JSON not changing on commit2016-04-10T23:36:00+00:00https://www.paulsprogrammingnotes.com/2016/04/sqlalchemy-json-not-changing-on-commitIf your changes to a JSON, JSONB, or HSTORE field aren't saving after you commit, then you need to wrap your field in the MutableDict shown here: <a href="http://docs.sqlalchemy.org/en/latest/dialects/postgresql.html#sqlalchemy.dialects.postgresql.HSTORE">http://docs.sqlalchemy.org/en/latest/dialects/postgresql.html#sqlalchemy.dialects.postgresql.HSTORE</a>
SQLAlchemy - AttributeError: type object 'JSONB' has no attribute 'lower'2016-04-10T05:59:00+00:00https://www.paulsprogrammingnotes.com/2016/04/sqlalchemy-attributeerror-type-objectThis error happened because I mistyped "db.column" instead of "db.Column" when I was creating my database model using SQLAlchemy.<br /><br />This might be your problem if db.create_all() isn't creating all the columns you expected.PHP - Problems With Long Running Processes2016-03-19T07:18:00+00:00https://www.paulsprogrammingnotes.com/2016/03/great-article-about-problems-with-usingGreat article about the problems with using PHP for long running processes: <a href="https://software-gunslinger.tumblr.com/post/48215406921/php-is-meant-to-die-continued">https://software-gunslinger.tumblr.com/post/48215406921/php-is-meant-to-die-continued</a><br /><br />Describes the main reason why I started using Python for long-running jobs.Ansible Vault - storing secrets in repos2016-03-04T03:52:00+00:00https://www.paulsprogrammingnotes.com/2016/03/ansible-vault-storing-secrets-in-reposWhen others are deploying a project for you, it's easy for mistakes to be made when secrets must be updated in environmental variables. Ansible-vault takes a different approach and encrypts the secrets - allowing you to store the secrets in your repo.<br /><br />To encrypt a file: <b>ansible-vault encrypt secrets.py</b><br />To decrypt a file: <b>ansible-vault decrypt secrets.py</b><br /><br />More documentation is available here: <a href="http://docs.ansible.com/ansible/playbooks_vault.html">http://docs.ansible.com/ansible/playbooks_vault.html</a>Python - isort, useful tool for sorting python imports2016-03-04T03:44:00+00:00https://www.paulsprogrammingnotes.com/2016/03/python-isort-useful-tool-for-sorting<a href="http://timothycrosley.github.io/isort/">http://timothycrosley.github.io/isort/</a><br /><br />I saw it in the django coding style documentation: <a href="https://docs.djangoproject.com/en/1.9/internals/contributing/writing-code/coding-style/">https://docs.djangoproject.com/en/1.9/internals/contributing/writing-code/coding-style/</a><br /><br />To make an entire project's imports easier to read, all you need to do is:<br /><br /><ul><li>pip install isort</li><li>isort -rc .</li></ul><div><br /></div><div>There's also a sublime plugin for it here: <a href="https://github.com/thijsdezoete/sublime-text-isort-plugin#install">https://github.com/thijsdezoete/sublime-text-isort-plugin#install</a></div>git commit --amend -C HEAD2016-02-19T04:29:00+00:00https://www.paulsprogrammingnotes.com/2016/02/git-commit-amend-c-headEver committed something and needed to make changes later? Just rebase and squash it, right? Or maybe "git reset --soft HEAD~1" and commit again?<br /><br />There's an even better solution: "git commit --amend -C HEAD"<br /><br />It will add the combine your commit with your last commit. If you already pushed, you will need to force push your change.pip install -e .2016-02-19T04:19:00+00:00https://www.paulsprogrammingnotes.com/2016/02/pip-install-eI've wasted a lot of time running "python setup.py install" before testing my changes to flask-admin. It turns out you can pip install a project as "editable", which points the install toward your local directory instead of copying where the rest of your python modules are.<div><br /></div><div>To install a project as editable, navigate to the repo and run "<b>pip install -e .</b>".</div><div><br /></div><div><a href="https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs">https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs</a></div>Python - Dropping Into pdb From Nose2016-02-14T01:09:00+00:00https://www.paulsprogrammingnotes.com/2016/02/dropping-into-pdb-from-noseThis document describes how to drop into pdb when your Python nose tests fail: <a href="http://nose.readthedocs.org/en/latest/plugins/debug.html">http://nose.readthedocs.org/en/latest/plugins/debug.html</a><br /><br />It just requires running "nosetests --pdb".<br /><br />Very helpful if you're debugging why a test failed.Great Hadoop + Spark Tutorial2016-02-14T01:07:00+00:00https://www.paulsprogrammingnotes.com/2016/02/great-hadoop-spark-tutorialThis great tutorial takes you through installing hadoop and spark to analyze Reddit comment data: <a href="http://blog.insightdatalabs.com/spark-cluster-step-by-step/">http://blog.insightdatalabs.com/spark-cluster-step-by-step/</a>Hadoop Namenode Not Starting2016-02-14T01:05:00+00:00https://www.paulsprogrammingnotes.com/2016/02/hadoop-namenode-not-startingMy namenode was not starting because I had the wrong host configured in yarn-site.xml, mapred-site.xml, and core-site.xml.<br /><br />When you're running start-dfs.sh on your namenode, ensure the line that says "starting namenode" shows "/usr/local/hadoop/logs/hadoop-ubuntu-namenode-<b><your namenode's hostname></b>.out" in the output. This is how you know your configuration is correct.<br /><br />You can check your server's hostname on Ubuntu by running "echo $(hostname)".Python - "The Darker Side of type"2015-12-26T08:31:00+00:00https://www.paulsprogrammingnotes.com/2015/12/python-darker-side-of-type<a href="https://www.jeffknupp.com/blog/2013/12/28/improve-your-python-metaclasses-and-dynamic-classes-with-type/">https://www.jeffknupp.com/blog/2013/12/28/improve-your-python-metaclasses-and-dynamic-classes-with-type/</a><br /><br />That blog post has a great description of how the type keyword can be used for another purpose, creating new types.<br /><br />Here's an example of the syntax:<b> Foo = type(str('Foo'), (object, ), {})</b><br /><br />This page has more details: <a href="http://python-3-patterns-idioms-test.readthedocs.org/en/latest/Metaprogramming.html">http://python-3-patterns-idioms-test.readthedocs.org/en/latest/Metaprogramming.html</a>SQLAlchemy's "scale" Argument & WTForms2015-12-23T01:24:00+00:00https://www.paulsprogrammingnotes.com/2015/12/sqlalchemys-scale-argument-wtforms<br /><div>Note: Unless otherwise stated, this applies to SQLAlchemy 1.0.</div><div><br /></div><div><div>Here are a few things I learned while researching the fix for this <a href="https://github.com/flask-admin/flask-admin/issues/1141">Flask-Admin Issue #1141</a>:</div></div><ul><li>With SQLAlchemy's Generic Float type, the "scale" argument is ignored. "scale" is not listed as an argument and the docs say "Additional arguments here are ignored by the default Float type.". However, the object does have a "scale", but it's always "None".</li><li>WTForms' "places" default for DecimalField's is 2.</li><li>The MySQL float does have a "scale" argument, it defaults to "None". MySQL and Postgres have default limits on the length of the precision of floats, MySQL only shows 6 digits after the decimal and Postgres has a default column length of 17.</li><li>Numeric columns do have a "scale" argument, and the default is "None" in SQLAlchemy 0.7 and 1.0.</li><li>"decimal_return_scale" was added to both Float and Numeric in SQLAlchemy 0.9.</li><li>If you raise the number of "places" in WTForms' DecimalField greater than "decimal_return_scale" in the SQLAlchemy field, the digits after the decimal place set in "decimal_return_scale" will show 0's.</li><li>In SQLAlchemy, "_default_decimal_return_scale = 10" is only used in a property/method called "_effective_decimal_return_scale". The default for "decimal_return_scale" is "None" and not 10. Since "_default_decimal_return_scale" and "_effective_decimal_return_scale" start with an underscore, so it's only intended for internal use.</li><li>MySQL's "FLOAT" in SQLAlchemy doesn't have "decimal_return_scale". It probably should? REAL and DOUBLE have this.</li><li>WTForms' FloatField does not have "places".</li><li>With SQLAlchemy 1.0, "db.DECIMAL()" will create a numeric(28, 6) in Postgres, but it creates a DECIMAL(10,0) in MySQL. This seemed pretty odd, since the default "scale" is "None". Maybe it's just using whatever the default is in each backend.</li></ul><br /><br />Sources:<br /><br /><ul><li>SQLAlchemy 0.7 docs for Float type: <a href="http://docs.sqlalchemy.org/en/rel_0_7/core/types.html?highlight=float#sqlalchemy.types.Float">http://docs.sqlalchemy.org/en/rel_0_7/core/types.html?highlight=float#sqlalchemy.types.Float</a></li><li>Latest SQLAlchemy docs for Float type: <a href="http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.Float">http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.Float</a></li><li>Latest docs for MySQL Float type: <a href="http://docs.sqlalchemy.org/en/latest/dialects/mysql.html?highlight=float#sqlalchemy.dialects.mysql.FLOAT">http://docs.sqlalchemy.org/en/latest/dialects/mysql.html?highlight=float#sqlalchemy.dialects.mysql.FLOAT</a></li></ul>Python Gotcha - Copying Nested Dicts2015-10-27T03:18:00+00:00https://www.paulsprogrammingnotes.com/2015/10/python-gotcha-preventing-argument-changeHere's the best blog post I've found that talks about the need for deep or shallow copy when using nested dicts in Python:<br /><span style="color: #0000ee;"><u><a href="http://www.peterbe.com/plog/must__deepcopy__">http://www.peterbe.com/plog/must__deepcopy__</a></u></span>Image Duplicate Detection In Python2015-10-25T06:02:00+00:00https://www.paulsprogrammingnotes.com/2015/10/image-duplicate-detection-in-pythonGreat article on detecting duplicate images in Python: <a href="http://7webpages.com/blog/image-duplicates-detection-python/">http://7webpages.com/blog/image-duplicates-detection-python/</a>GoDaddy To Amazon Route 53 - Lessons Learned2015-10-11T05:52:00+00:00https://www.paulsprogrammingnotes.com/2015/10/godaddy-to-amazon-route-53-lessonsThe first thing you'll want to do is created a "Hosted Zone" with the same A and CNAME records (and any other relevant records that aren't specific to GoDaddy).<br /><br />When you transfer the domain and it asks you if you want to use the old name servers - answer no. I messed this one up and kept using GoDaddy's "domaincontrol.com" name servers and DNS stopped resolving eventually. If you mess this up too, you'll need to go to the "Hosted Zone" for your site and copy the records in the "NS" list to domain's name servers on the registered domains page.Awesome-ETL List2015-10-07T08:00:00+00:00https://www.paulsprogrammingnotes.com/2015/10/awesome-etl-listFinally decided to put all the interesting ETL software I find into a list: <a href="https://github.com/pawl/awesome-etl">https://github.com/pawl/awesome-etl</a>Python Idioms2015-09-24T09:13:00+00:00https://www.paulsprogrammingnotes.com/2015/09/python-idiomsHere's a great page describing Python idioms: <a href="https://david.goodger.org/projects/pycon/2007/idiomatic/handout.html">https://david.goodger.org/projects/pycon/2007/idiomatic/handout.html</a><br /><br />This gist of a Raymond Hettinger talk is also great: <a href="https://gist.github.com/JeffPaine/6213790">https://gist.github.com/JeffPaine/6213790</a>
Best "Command Line" Beginners Guide2015-09-19T21:06:00+00:00https://www.paulsprogrammingnotes.com/2015/09/best-command-line-beginners-guide<a href="http://www.learnenough.com/command-line">http://www.learnenough.com/command-line</a><br /><br />It's a short guide, but I think it covers everything a beginner needs to know.Current State Of Workflow Engines / Frameworks2015-09-15T06:49:00+00:00https://www.paulsprogrammingnotes.com/2015/09/current-state-of-workflow-frameworksThis is a great article about the current state of workflow engines (and frameworks?): <a href="http://stackstorm.com/2015/04/10/the-return-of-workflows/">http://stackstorm.com/2015/04/10/the-return-of-workflows/</a><br /><br />Taskflow looks especially interesting. The article is unfortunately missing Airbnb's Airflow: <a href="https://github.com/airbnb/airflow">https://github.com/airbnb/airflow</a>Installing Postgres 9.5 On OSX2015-08-02T04:51:00+00:00https://www.paulsprogrammingnotes.com/2015/08/installing-postgres-95-on-osx<p>Edit 12/25/2020: I’d recommend using: <a href="https://postgresapp.com/">https://postgresapp.com/</a></p>
<p>Ended up being surprisingly easy:</p>
<noscript><pre>brew uninstall postgresql
brew install postgresql-9.5
brew link -f postgresql-9.5
mv /usr/local/var/postgres /usr/local/var/postgres.old
initdb -D /usr/local/var/postgres
# restore old database?
# pg_upgrade -b /usr/local/Cellar/postgresql/9.0.4/bin -B /usr/local/Cellar/postgresql/9.1.2/bin -d /usr/local/var/postgres.old -D /usr/local/var/postgres
# start postgres
pg_ctl -D /usr/local/var/postgres -l logfile start</pre></noscript>
<script src="https://gist.github.com/4d0213c5aef2bbba66f1.js"> </script>
New Macbook Pro Setup2015-07-26T22:21:00+00:00https://www.paulsprogrammingnotes.com/2015/07/new-macbook-pro-setup<p>I’m finally switching to a Macbook Pro as my personal computer after a few months of using one at work. Local development on OSX is a lot easier and bug-free than on a Windows computer.</p>
<p>I started a gist with what I did for setup:</p>
<noscript><pre>Important tips:
* to open applications quickly, command + space and type the application name
* mac doesn't have an address bar in finder, you need to use command + shift + g to type a directory path
* mac doesn't have "snap window to screen" like windows does, you need to install moom for this
* activity monitor = task manager
* brew install = apt get install (you'll need to install brew)
* use command + tilde to switch between windows for the same application
Setup:
* Upgrade to El Capitan through the app store if you haven't already
* Install chrome (https://www.google.com/chrome/browser/desktop/)
* Install chrome plugins (adblock, etc)
* Install brew
* /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
* Install iterm2 (https://www.iterm2.com/)
* Fix alt + left/right arrow in iterm: http://apple.stackexchange.com/a/136931
* open up iterm with command + space and type "brew install git"
* Install xcode and agree to terms of service (it's required by some things)
* Install sequelpro (http://www.sequelpro.com/download)
* Install sublime text
* Install sublime package manager and plugins (like anaconda)
* restore sublime text settings from other computer
* Security hardening (especially turn off bluetooth): http://docs.hardentheworld.org/OS/OSX_10.11_El_Capitan/index.html
* Open dock settings and check "Automatically hide and show dock"
* Open trackpad settings and enable "Tap to Click"
* Open keyboard settings and set "key repeat" to "Fast" and "delay until repeat" to "Short"
* Install dropbox
</pre></noscript>
<script src="https://gist.github.com/138d6c986a6bd674bdddfb9b730709b1.js"> </script>
In Order to Configure TCP/IP, You Must Install and Enable a Network Adapter Card2015-07-12T04:44:00+00:00https://www.paulsprogrammingnotes.com/2015/07/in-order-to-configure-tcpip-you-mustIf you get this error message, you need to disable and uninstall your network adapter. Once you've done that, you can reinstall the driver to fix it.Cygwin - Python Compiled Without SSL Support2015-07-11T23:11:00+00:00https://www.paulsprogrammingnotes.com/2015/07/importerror-cannot-import-nameDownload error on https://pypi.python.org/simple/pip/: unknown url type:<br />https -- Some packages may not be found!<br /><br />ImportError: cannot import name HTTPSHandler<br /><br />I was trying to install pip in cygwin when this happened. Searches say openssl-devel needed to be installed, but it already was. Apparently python was compiled without ssl support.<br /><br />My solution was to install cygwin-x86 instead of the 64x one.md5sum: standard input: no properly formatted MD5 checksum lines found2015-07-11T06:22:00+00:00https://www.paulsprogrammingnotes.com/2015/07/md5sum-standard-input-no-properlyIf you see this error when you're using apt-cyg, it means you need to update apt-cyg.<br /><br />You need to run the following commands:<br /><br /><pre style="background-color: #f7f7f7; border-radius: 3px; box-sizing: border-box; color: #333333; font-family: Consolas, 'Liberation Mono', Menlo, Courier, monospace; font-size: 13.6000003814697px; font-stretch: normal; line-height: 1.45; margin-bottom: 16px; overflow: auto; padding: 16px; word-wrap: normal;"><code style="background: transparent; border-radius: 3px; border: 0px; box-sizing: border-box; display: inline; font-family: Consolas, 'Liberation Mono', Menlo, Courier, monospace; font-size: 13.6000003814697px; line-height: inherit; margin: 0px; max-width: initial; overflow: initial; padding: 0px; word-break: normal; word-wrap: normal;">lynx -source rawgit.com/transcode-open/apt-cyg/master/apt-cyg > apt-cyg<br />install apt-cyg /bin</code></pre>Automatically Remove Long-Running Docker Containers2015-06-28T00:54:00+00:00https://www.paulsprogrammingnotes.com/2015/06/automatically-remove-long-running<p>The following can be used in cron or run with & at the end of the command:</p>
<noscript><pre>PID=$(docker run -d "$@")
docker wait $PID
docker rm $PID</pre></noscript>
<script src="https://gist.github.com/722cb52669f1643f6a78.js"> </script>
<p>I found this didn’t work: <a href="http://www.ahtik.com/blog/removing-detached-docker-containers-automatically/">http://www.ahtik.com/blog/removing-detached-docker-containers-automatically/</a></p>
Docker Run In Crontab2015-06-03T04:17:00+00:00https://www.paulsprogrammingnotes.com/2015/06/docker-run-in-crontab"0 5 * * 1 docker run --rm --name=mycontainer ubuntu:13.10 /opt/bin/job"<br />The above command is an example of how "docker run" would be used in crontab to run once every week at 5am.<br /><br /><ul><li>--rm will delete the container once the job is finished running</li><li>--name will name the container and prevent duplicate jobs from running</li><li>You don't need to use "&" at the end, because crons already run in the background.</li></ul>MySQL Batch Updates Not Working2015-05-08T07:51:00+00:00https://www.paulsprogrammingnotes.com/2015/05/mysql-batch-updates-not-workingLooking at "SHOW PROCESSLIST" and it looks like your queries are running individually instead of in batches like you sent?<br /><br />This happens because MySQL runs each update statement individually, but you should still be able to see the batches when the queries are in the "init" state.<br /><br />Use this query to see the batched queries:<br /><b>SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST where state="init";</b>New Flask Template2015-03-23T00:29:00+00:00https://www.paulsprogrammingnotes.com/2015/03/new-flask-template<a href="https://github.com/xen/flask-project-template/">https://github.com/xen/flask-project-template/</a><br /><br />It's also an interesting example of using docker for a flask project."Error response from daemon: 404 page not found" - Docker2015-02-13T05:16:00+00:00https://www.paulsprogrammingnotes.com/2015/02/error-response-from-daemon-404-page-notThis required setting the following in my /etc/environment:<br /><pre style="background-color: #f7f7f7; border-radius: 3px; box-sizing: border-box; color: #333333; font-family: Consolas, 'Liberation Mono', Menlo, Courier, monospace; font-size: 11.8999996185303px; font-stretch: normal; line-height: 1.45; margin-bottom: 16px; overflow: auto; padding: 16px; word-wrap: normal;"><code style="background: transparent; border-radius: 3px; border: 0px; box-sizing: border-box; display: inline; font-family: Consolas, 'Liberation Mono', Menlo, Courier, monospace; font-size: 11.8999996185303px; line-height: inherit; margin: 0px; max-width: initial; overflow: initial; padding: 0px; word-break: normal; word-wrap: normal;">no_proxy=/var/run/docker.sock</code></pre>When Not to Use Generator Expressions2015-01-28T21:04:00+00:00https://www.paulsprogrammingnotes.com/2015/01/when-not-to-use-generator-expressions<a href="http://stackoverflow.com/a/26635939">http://stackoverflow.com/a/26635939</a><br /><br />I found that post very helpful. I didn't know it's a better idea to use a list comprehension rather than a generator expression for "".join().mod_wsgi + Flask2015-01-28T08:52:00+00:00https://www.paulsprogrammingnotes.com/2015/01/trailing-slashes-matter-if-you-set<p>Here’s my mod_wsgi configuration for a flask app that uses Flask-Admin.</p>
<noscript><pre>WSGIDaemonProcess makermanager2 user=www-data group=www-data threads=5
WSGIScriptAlias /makermanager2 /var/www/makermanager2/runserver.wsgi
<Directory /var/www/makermanager2>
WSGIProcessGroup makermanager2
WSGIApplicationGroup %{GLOBAL}
Order deny,allow
Allow from all
</Directory>
Alias /makermanager2/assets /var/www/makermanager2/application/static
<Directory /var/www/makermanager2/application/static>
Order allow,deny
Allow from all
</Directory>
Alias /makermanager2/static/admin /opt/Envs/prod/src/flask-admin-master/flask_admin/static/
<Directory /opt/Envs/prod/src/flask-admin-master/flask_admin/static/>
Order allow,deny
Allow from all
</Directory>
</pre></noscript>
<script src="https://gist.github.com/99b6e114d0b792639400.js"> </script>
<p>Trailing slashes matter! If you set a trailing slash, it will only set the alias for that directory and not all of the directories below it.</p>
Request.Args Empty During Tests - Flask2015-01-27T04:01:00+00:00https://www.paulsprogrammingnotes.com/2015/01/requestargs-empty-during-tests-flask<a href="https://github.com/mitsuhiko/flask/issues/801#issuecomment-21513278">https://github.com/mitsuhiko/flask/issues/801#issuecomment-21513278</a><br /><br />If you're using app.test_client().get, it's very important to use a relative url or your parameters will be removed.<br /><br /># will not pass request.args<br />rv = self.client.get('http://0.0.0.0:5000/view/?foo=bar')<br /><br /># works just fine<br />rv = self.client.get('/view/?foo=bar')<br /><br /><br />Self-Hosting Fonts2015-01-23T05:51:00+00:00https://www.paulsprogrammingnotes.com/2015/01/self-hosting-fonts<a href="https://github.com/CSSLint/csslint/wiki/Bulletproof-font-face">https://github.com/CSSLint/csslint/wiki/Bulletproof-font-face</a><br /><br />That's the most helpful guide I found for self-hosting your own fonts and preventing compatibility issues.Dokku Change Nginx Timeout2015-01-21T20:36:00+00:00https://www.paulsprogrammingnotes.com/2015/01/dokku-change-nginx-timeoutNone of the plug-ins for doing custom nginx configurations were working for me.<br /><br />So, I changed the following two files:<br />/var/lib/dokku/plugins/nginx-vhosts/templates/nginx.ssl.conf<br />/var/lib/dokku/plugins/nginx-vhosts/templates/nginx.conf<br /><br />I added the following under "location / {":<br />proxy_connect_timeout 300s;<br />proxy_read_timeout 300s;<br /><div><br /></div>Supervisord Tips and Gotchas2015-01-21T05:22:00+00:00https://www.paulsprogrammingnotes.com/2015/01/supervisord-tips-and-gotchasIf you make configuration changes, the changes will not take effect until you:<br /><ul><li>supervisorctl reread</li><li>supervisorctl update</li></ul><div><br />"redirect_stderr=true" will cause both stderr and stdout to appear in the "stdout" log.</div><br />You get to select which user runs the command with the "user" parameter.<br /><blockquote class="tr_bq">For example: <b>user=root</b></blockquote>Supervisord keeps its own set of environmental variables with the "environment" parameter. It won't pick up env variables from /etc/environment.<br /><blockquote class="tr_bq">For example: <b>environment=A="1",B="2"</b></blockquote>For Gunicorn, use a configuration file rather than putting all your arguments into the "command" parameter.<br /><blockquote class="tr_bq">For example, -t will be <b>ignored </b>here:<br />command=/path/to/gunicorn main:application <b>-t 300</b></blockquote><blockquote class="tr_bq">Instead use:<br />command=/path/to/gunicorn main:application <b>-c /path/to/gunicorn.conf.py</b></blockquote>Installing python-ldap on Dokku2015-01-20T06:20:00+00:00https://www.paulsprogrammingnotes.com/2015/01/installing-python-ldap-on-dokkuInstall this plugin: <a href="https://github.com/F4-Group/dokku-apt#installation">https://github.com/F4-Group/dokku-apt#installation</a><br /><br />Create an apt-packages file in the root directory of your repository with the following lines:<br />python-dev<br />libldap2-dev<br />libsasl2-dev<br />libssl-dev<br /><br />Now, commit your new apt-packages file and python-ldap should install successfully.<br /><br />I also had trouble installing pymssql, and got an error saying "_mssql.c:314:22: fatal error: sqlfront.h: No such file or directory". Similar fix, just add "freetds-dev" to your apt-packages file.Enabling SSL On Dokku2015-01-20T05:13:00+00:00https://www.paulsprogrammingnotes.com/2015/01/enabling-ssl-on-dokku<ol><li>It's important that you have the newest version of Dokku: <a href="http://progrium.viewdocs.io/dokku/upgrading">http://progrium.viewdocs.io/dokku/upgrading</a></li><li>Zip your certificate and key: <b>sudo tar cvf archive_name.tar.gz server.crt server.key</b></li><li>Import using <b>dokku nginx:import-ssl appname < ~/archive_name.tar.gz</b><a href="https://github.com/progrium/dokku/blob/master/docs/nginx.md#importing-ssl-certificates">https://github.com/progrium/dokku/blob/master/docs/nginx.md#importing-ssl-certificates</a></li></ol>Expecting: TRUSTED CERTIFICATE2015-01-20T04:59:00+00:00https://www.paulsprogrammingnotes.com/2015/01/expecting-trusted-certificateI was trying to add my certificate to Dokku using "dokku nginx:import-ssl" when I got the error message "Expecting: TRUSTED CERTIFICATE".<div><br /></div><div>This ended up being an issue with line endings and the dos2unix utility fixed it: <a href="http://serverfault.com/a/317038">http://serverfault.com/a/317038</a></div>Downloading Files With Python's Requests Module2015-01-19T00:28:00+00:00https://www.paulsprogrammingnotes.com/2015/01/downloading-files-with-pythons-requests<a href="http://docs.python-requests.org/en/latest/user/quickstart/#binary-response-content">http://docs.python-requests.org/en/latest/user/quickstart/#binary-response-content</a><br /><br />From the example:<br /><pre style="background: rgb(238, 238, 238); color: #3e4349; font-family: Consolas, Menlo, 'Deja Vu Sans Mono', 'Bitstream Vera Sans Mono', monospace; font-size: 0.9em; line-height: 1.3em; margin: 15px -30px; overflow-x: auto; overflow-y: hidden; padding: 7px 30px;"><span class="gp" style="color: #745334;">>>> </span><span class="kn" style="color: #004461; font-weight: bold;">from</span> <span class="nn" style="color: black;">PIL</span> <span class="kn" style="color: #004461; font-weight: bold;">import</span> <span class="n" style="color: black;">Image</span><br /><span class="gp" style="color: #745334;">>>> </span><span class="kn" style="color: #004461; font-weight: bold;">from</span> <span class="nn" style="color: black;">StringIO</span> <span class="kn" style="color: #004461; font-weight: bold;">import</span> <span class="n" style="color: black;">StringIO</span><br /><span class="gp" style="color: #745334;">>>> </span><span class="n" style="color: black;">i</span> <span class="o" style="color: #582800;">=</span> <span class="n" style="color: black;">Image</span><span class="o" style="color: #582800;">.</span><span class="n" style="color: black;">open</span><span class="p" style="color: black; font-weight: bold;">(</span><span class="n" style="color: black;">StringIO</span><span class="p" style="color: black; font-weight: bold;">(</span><span class="n" style="color: black;">r</span><span class="o" style="color: #582800;">.</span><span class="n" style="color: black;">content</span><span class="p" style="color: black; font-weight: bold;">))</span></pre>Nmap Not Finding Hostnames2015-01-16T06:22:00+00:00https://www.paulsprogrammingnotes.com/2015/01/nmap-not-finding-hostnames<a href="http://serverfault.com/a/153779">http://serverfault.com/a/153779</a><br /><br />The secret ended up being running the command as sudo. It's odd how "socket.gethostbyaddr()" worked just fine without sudo, but was slower than nmap.Slow Disk Performance On Dell r6202015-01-07T16:47:00+00:00https://www.paulsprogrammingnotes.com/2015/01/slow-disk-performance-on-ubuntu-1304-vs<h3>Update:</h3>I installed openmanage with the instructions on this page and used it to check/update bios settings: <a href="http://linux.dell.com/repo/community/deb/OMSA_7.1/">http://linux.dell.com/repo/community/deb/OMSA_7.1/</a><br /><br />The fix ended up being BIOS power profile settings. It was set to Performance Per Watt (DAPC) and needed to be set to "Performance".<br /><div class="separator" style="clear: both; text-align: center;"><a href="/assets/images/performance_setting.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="/assets/images/performance_setting.png" /></a></div>It's very interesting that the BIOS power profile settings didn't seem to affect Ubuntu 14.10's disk read speed.<br /><h3>Troubleshooting:</h3>I have three of the same server. All the servers are Dell r620's with a PERC H710 Mini RAID controller (21.2.0-0007_A04 firmware) and have the exact same hard drives (Seagate ST300MM0006) in RAID 1 configurations. I'm seeing about 50% worse disk read performance on some versions of Ubuntu.<br /><br />They only differences I can find in the hardware:<br /><ul><li>The BIOS version on the poorly performing machines is 1.4.8, the machine that's performing well is on 1.6.0. The BIOS change log doesn't appear to have any changes between the two versions that would affect anything.</li><li>The chipset version on the RAID controller of the poorly performing machines is rev 01 (ChipRevision: B0). The machine that's performing well has the rev 05 chipset (ChipRevision: D1). The motherboard chipset revision is also different.</li></ul>Here are the benchmark results for the different Ubuntu versions and RAID controller chipsets:<br /><br />(mysql read speed test - using the same version of mysql across all 3 machines)<br />sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=test --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run:<br /><div class="Mu SP" id=":ie.ma" style="-webkit-transition: opacity 0.218s ease; color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px; margin-bottom: 6px; opacity: 1; transition: opacity 0.218s ease; word-break: break-word; word-wrap: break-word;"><div id=":ie.co">(<b>13.04</b><b style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"> - rev 05</b>): read/write requests: 4996362 (<b>83271.11</b> per sec.)<br />(<b>14.04</b><b style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"> - rev 01</b>): read/write requests: 1906520 (<b>31773.58</b> per sec.)<br />(<b>14.10</b><b style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"> - rev 01</b>): read/write requests: 5166798 (<b>86111.50</b> per sec.)<br /><br /></div></div><div class="Mu SP" id=":if.ma" style="-webkit-transition: opacity 0.218s ease; margin-bottom: 6px; opacity: 1; transition: opacity 0.218s ease; word-break: break-word; word-wrap: break-word;"><div class="xH" id=":if.at" style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px; text-align: center;"></div><div id=":if.co">dd if=/dev/zero of=/tmp/output bs=8k count=10k; rm -f /tmp/output:<br /><div style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">(<b>13.04 - rev 05</b>): 83886080 bytes (84 MB) copied, 0.0437543 s, <b>1.9 GB/s</b><br /><span style="background-color: white;">(</span><b style="background-color: white;">13.04</b><span style="background-color: white;"><b> - rev 01</b>) 83886080 bytes (84 MB) copied, 0.116811 s,</span><b style="background-color: white;"> 718 MB/s</b><br /><span style="background-color: white;">(</span><b style="background-color: white;">13.10</b><span style="background-color: white;"><b> - rev 01</b>): 83886080 bytes (84 MB) copied, 0.129371 s, <b>648 MB/s</b></span></div><div style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">(<b>14.04</b><b> - rev 01</b>): 83886080 bytes (84 MB) copied, 0.157808 s,<b> </b><b>532 MB/s</b></div><div style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">(<b>14.10</b><b> - rev 01</b>): 83886080 bytes (84 MB) copied, 0.102355 s, <b>820 MB/s</b></div></div><div id=":if.co" style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"><b><br /></b></div><div id=":if.co">hdparm -tT /dev/sda1:<br /><span style="color: #262626; font-family: arial, sans-serif;"><span style="font-size: 13px; line-height: 16px;">(</span></span><b style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">13.04</b><b style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"> - rev 05</b><span style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">): Timing cached reads:23154 MB in 2.00 seconds = </span><b style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">11589.16</b><span style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"> MB/sec</span><br /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 18.4799995422363px;"><span style="line-height: 16px;">(</span></span><b style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">13.04</b><b style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"> - rev 01</b><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 18.4799995422363px;"><span style="line-height: 16px;">): Timing cached reads:17934 MB in 2.00 seconds = <b>8973.68</b> MB/sec</span></span><br /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 18.4799995422363px;"><span style="line-height: 16px;">(<b>13.10 - rev 01</b>): </span></span><span style="color: #262626; font-family: arial, sans-serif;"><span style="font-size: 13px; line-height: 16px;">Timing cached reads:18102 MB in 2.00 seconds = <b>9058.08</b> MB/sec</span></span><br /><span style="color: #262626; font-family: arial, sans-serif;"><span style="font-size: 13px; line-height: 16px;">(</span></span><b style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">14.04 - rev 01</b><span style="color: #262626; font-family: arial, sans-serif;"><span style="font-size: 13px; line-height: 16px;">): </span></span><span style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">Timing cached reads:17846 MB in 1.99 seconds = </span><b style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">8956.28</b><span style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"> MB/sec</span><br /><span style="color: #262626; font-family: arial, sans-serif;"><span style="font-size: 13px; line-height: 16px;">(</span></span><b style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">14.10 - rev 01</b><span style="color: #262626; font-family: arial, sans-serif;"><span style="font-size: 13px; line-height: 16px;">): </span></span><span style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">Timing cached reads:21538 MB in 2.00 seconds = </span><b style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">10777.93</b><span style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"> MB/sec</span></div><br />At one point, 2 of the rev 01 servers were using Ubuntu 14.04 and were getting similar bad benchmark results. After the upgrading them to 14.10, I immediately saw the disk read speed increase. <br /><br /><div id=":if.co">The performance on rev 01 machine with 13.04 was bad, but somehow the rev 05 machine with the same OS version was performing well. This told me it's not a problem with the megaraid_sas driver, because the rev 01 and rev 05 machines with 13.04 had the same version.<br /><br />Here's the closest I can find to a similar issue (it's different hardware though): <a href="http://en.community.dell.com/support-forums/servers/f/906/t/19596533">http://en.community.dell.com/support-forums/servers/f/906/t/19596533</a><br /><br /><b>Update 3/1/2015:</b><br />Saw an interesting checklist on Hacker News today that also mentioned disabling BIOS power saving settings: <a href="http://odetodata.com/2015/02/installation-and-configuration-checklist-for-microsoft-sql-server/">http://odetodata.com/2015/02/installation-and-configuration-checklist-for-microsoft-sql-server/</a><br /><ul></ul></div></div>
MegaCli64 / MegaCli Only showing "Exit Code: 0x00"2015-01-06T18:23:00+00:00https://www.paulsprogrammingnotes.com/2015/01/megacli64-megacli-only-showing-exitProblem: You see MegaCli64 or MegaCli only showing "Exit Code: 0x00" when you try to run commands.<br /><br />Solution: <b>Try running as sudo.</b>Changing Putty Default Settings2015-01-02T06:10:00+00:00https://www.paulsprogrammingnotes.com/2015/01/changing-putty-default-settings<p>If you need to change Putty’s defaults, press “Load” on the saved session for “Default Settings”, make your changes, and “Save”.</p>
<p><img src="/assets/images/default_Settings.png" alt="putty default settings" /></p>
Best Django Boilerplate/Skeleton2014-12-27T07:29:00+00:00https://www.paulsprogrammingnotes.com/2014/12/best-django-boilerplatetemplateskeletonThis is my favorite boilerplate for new projects so far: <a href="https://github.com/kirpit/django-sample-app">https://github.com/kirpit/django-sample-app</a><br /><br />Because:<br /><br /><ul><li>It's up-to-date (as of 12/27/2014). <a href="https://github.com/rdegges/django-skel">Django-skel</a> is currently at only up to 1.5.9.</li><li>It's minimal. It doesn't make many design decisions for you or include a bunch of junk. <a href="https://github.com/xenith/django-base-template">Django-base-template</a> isn't as minimal.</li><li>Great structure, it follows this fairly closely: <a href="http://stackoverflow.com/a/23469321/1364191">http://stackoverflow.com/a/23469321/1364191</a></li></ul>If you're not a fan of bootstrap, <a href="https://github.com/mike360/django-html5-boilerplate">django-html5-boilerplate</a> also looks good.Setting A WTForm Default Date To Today/Now2014-12-11T17:14:00+00:00https://www.paulsprogrammingnotes.com/2014/12/setting-wtform-default-date-to-today<a href="http://stackoverflow.com/a/27424739/1364191">http://stackoverflow.com/a/27424739/1364191</a><br /><br />It's definitely a common mistake for people using WTForms. When you create a form and use something like datetime.date.today() as a default value, that value will always stay the same. <b>The fix is to use "datetime.date.today"</b> instead of "datetime.date.today()".Dokku Flask Port and Host2014-12-05T20:07:00+00:00https://www.paulsprogrammingnotes.com/2014/12/dokku-flask-port-and-hostIn your flask application, you need to:<br /><br /><ul><li>Set default <b>port=5000</b> (this seems to be Dokku's default too)</li><li>Set default <b>host=0.0.0.0</b> (this allows your host server to connect to the container)</li></ul><br />In your Procfile:<br /><div><b>web: python <your-app>.py</b><br /><br />If you're using gunicorn, it shouldn't matter because it's not using app.run().<br /><br />If you still can't connect to your application after it deploys, see this: <a href="http://www.paulsprogrammingnotes.com/2014/12/troubleshooting-dokku-not-accessible.html">http://www.paulsprogrammingnotes.com/2014/12/troubleshooting-dokku-not-accessible.html</a></div>Troubleshooting Dokku Not Accessible2014-12-05T20:02:00+00:00https://www.paulsprogrammingnotes.com/2014/12/troubleshooting-dokku-not-accessibleFirst, make sure your application isn't giving any errors when it attempts to run:<br /><b>dokku run <your-app> "<whatever is in your proc file>"</b><br /><br />Get the :port number from the line with "upstream" in the following file: <b>/home/dokku/<your-app>/nginx.conf</b><br /><br />Try to "<b>wget 127.0.0.1:<your-port></b>" from the host. Will it connect?"Requested runtime not available for this stack" - Dokku2014-12-04T15:32:00+00:00https://www.paulsprogrammingnotes.com/2014/12/requested-runtime-not-available-forWhile using Dokku, I was getting the following error message: <b>Requested runtime (python-2.7.8) is not available for this stack (cedar-14).</b><br /><b><br /></b>This was due to a connection error between my docker container and the internet. To determine this, I ran "docker images" and got the image id of "programium/buildstep", then got into a container with the following command: <b>sudo docker run -t -i <your image id> /bin/bash</b><br /><br />Once in the container, I ran:<br /><b>curl http://lang-python.s3.amazonaws.com/cedar/runtimes/python-2.7.8.tar.gz -s</b><br /><b><br /></b>If that is unsuccessful, that should confirm it's a connection issue.<br /><br />This was the solution to the problem: <a href="http://www.paulsprogrammingnotes.com/2014/12/rebuilding-buildstep-image-for-dokku.html">http://www.paulsprogrammingnotes.com/2014/12/rebuilding-buildstep-image-for-dokku.html</a>Rebuilding Buildstep Image For Dokku From Behind Proxy2014-12-02T20:52:00+00:00https://www.paulsprogrammingnotes.com/2014/12/rebuilding-buildstep-image-for-dokku<p>I had to change the buildstep image’s docker file to add my proxy. This is because the buildstep image required a proxy before the build to download the python buildpack.</p>
<p>I was rebuilding the buildstep image for dokku with these instructions: <a href="https://github.com/progrium/dokku/commit/9ebf453b72cab3a16ea261284236bb7c20ca3a1a#diff-04c6e90faac2675aa89e2176d2eec7d8R101">https://github.com/progrium/dokku/commit/9ebf453b72cab3a16ea261284236bb7c20ca3a1a#diff-04c6e90faac2675aa89e2176d2eec7d8R101</a></p>
<p>I’m behind a proxy, so I was getting this error when I ran “<strong>sudo make build</strong>”:</p>
<blockquote>
<p>Cloning into ‘heroku-buildpack-multi’…
fatal: unable to access ‘https://github.com/ddollar/heroku-buildpack-multi.git/’: Failed to connect to github.com port 443: Connection timed out</p>
</blockquote>
<p>The solution? Replace the Dockerfile in the root of the folder you git cloned with this:</p>
<noscript><pre>FROM progrium/cedarish:cedar14
MAINTAINER Jeff Lindsay <progrium@gmail.com>
ENV http_proxy http://proxy.myproxy.com:8080/
ENV https_proxy http://proxy.myproxy.com:8080/
ADD ./stack/configs/etc-profile /etc/profile
ADD ./builder/ /build
RUN xargs -L 1 /build/install-buildpack /tmp/buildpacks < /build/config/buildpacks.txt
</pre></noscript>
<script src="https://gist.github.com/190b948b42f9a637b327.js"> </script>
Troubleshooting Pip Timeout - Python2014-12-01T16:39:00+00:00https://www.paulsprogrammingnotes.com/2014/12/troubleshooting-pip-timeout-pythonTimeout: (<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x27330d0>, 'Connection to pypi.python.org timed out. (connect timeout=15)')<br /><br />If you're installing several requirements from a requirements.txt file, it helps to use "-v" after your pip install command to figure out which module has the timeout. Most likely, it's trying to download a module from a URL that doesn't exist.django-admin22014-11-29T03:09:00+00:00https://www.paulsprogrammingnotes.com/2014/11/django-admin2This project looks similar more similar to Flask-Admin than regular Django Admin: <a href="https://github.com/pydanny/django-admin2">https://github.com/pydanny/django-admin2</a><br /><br />There's an example admin app here: <a href="https://github.com/pydanny/django-admin2/tree/develop/example">https://github.com/pydanny/django-admin2/tree/develop/example</a>NULL values excluded from NOT IN - SQLAlchemy+SQLite2014-11-26T01:47:00+00:00https://www.paulsprogrammingnotes.com/2014/11/null-values-excluded-from-not-inSolution: AND column IS NOT NULL<br /><b><br /></b>Example: <b>query.filter(or_(~self.column.in_(value), self.column == None))</b><br /><br />This thread explains "SQL Server in treating NULL as a value": http://stackoverflow.com/questions/11491831/null-values-are-excluded-whyGithub Asking For Password After Adding SSH Key2014-11-25T23:21:00+00:00https://www.paulsprogrammingnotes.com/2014/11/github-asking-for-password-after-adding<div>Run: <b>git remote -v</b></div><div><br /></div><div>If the output has a URL that starts with https://github.com and not git@github.com, you need to change that by running a command similar to this: <b>git remote set-url origin git@github.com:</b>pawl/flask-admin.git</div><div><br /></div><div><a href="https://help.github.com/articles/changing-a-remote-s-url/#switching-remote-urls-from-ssh-to-https">https://help.github.com/articles/changing-a-remote-s-url/#switching-remote-urls-from-ssh-to-https</a></div>Guide To Syncing Fork With Original Project2014-11-21T08:14:00+00:00https://www.paulsprogrammingnotes.com/2014/11/guide-to-sync-fork-with-original-project<a href="https://help.github.com/articles/syncing-a-fork/">https://help.github.com/articles/syncing-a-fork/</a><br /><br />From my fork, I ran these commands:<br /><br /><ol><li>git remote add upstream https://github.com/mrjoes/flask-admin.git</li><li>git fetch upstream</li><li>git merge upstream/master</li><li>git push</li></ol><div>Create new branch and sync with upstream (change the <b>bolded </b>text):</div><div><ol><li>git remote add upstream <b>https://github.com/mrjoes/flask-admin.git</b></li><li>git fetch upstream</li><li>git checkout -b <b>issue_xxxx </b>upstream/master</li></ol></div><div><a href="https://docs.djangoproject.com/en/1.6/internals/contributing/writing-code/working-with-git/#working-on-a-ticket">https://docs.djangoproject.com/en/1.6/internals/contributing/writing-code/working-with-git/#working-on-a-ticket</a></div>Auto-Fixing Syntax Guideline Problems With Autopep8 - Python2014-11-18T19:50:00+00:00https://www.paulsprogrammingnotes.com/2014/11/auto-fixing-syntax-guideline-problems<b>find . -name '*.py' -exec autopep8 --in-place '{}' \;</b><br /><a href="https://github.com/hhatto/autopep8/issues/30#issuecomment-22458121">https://github.com/hhatto/autopep8/issues/30#issuecomment-22458121</a>Configuring Notepad++ For Python2014-11-16T08:53:00+00:00https://www.paulsprogrammingnotes.com/2014/11/configuring-notepad-for-python<blockquote class="tr_bq">On the Settings->"Preferences", "Tab Settings" tab, I set "[Default]" at Tab size: 8, uncheck Replace by space; and set "Python" to uncheck Use default value, Tab size: 4, check Replace by space. This causes inserts into a python source to use 4 spaces for indents, and indent with spaces instead of tabs. </blockquote><blockquote class="tr_bq">If I end up with any tabs in the source, I use Edit->Blank Operations->TAB to Space to convert them. I also clean up trailing blanks with Edit->Blank Operations->Trim Trailing Space. </blockquote><blockquote class="tr_bq">I install and use the pep8 package to verify standard formatting.</blockquote><a href="http://www.reddit.com/r/Python/comments/2mfrn6/python_in_notepad/cm3tdj5">http://www.reddit.com/r/Python/comments/2mfrn6/python_in_notepad/cm3tdj5</a>Redirecting From Within Helper Functions - Flask2014-11-15T23:33:00+00:00https://www.paulsprogrammingnotes.com/2014/11/redirecting-from-within-helper<noscript><pre>from flask import Flask, redirect
from werkzeug.routing import RoutingException, HTTPException
app = Flask(__name__)
@app.route('/')
def hello_world():
def helper_function():
# Attempt #1
# this obviously won't work: return redirect('www.google.com')
# the program would just return 'Hello World!'
# Attempt #2
# THIS IS PERMANENT!: raise RequestRedirect('example2')
# your browser will cache the status 301 and ALWAYS redirect
# explanation: http://stackoverflow.com/a/16371787/1364191
# Attempt #3
class RequestRedirect(HTTPException, RoutingException):
"""Raise if the map requests a redirect. This is for example the case if
`strict_slashes` are activated and an url that requires a trailing slash.
The attribute `new_url` contains the absolute destination url.
The attribute `code` is returned status code.
"""
def __init__(self, new_url, code=301):
RoutingException.__init__(self, new_url)
self.new_url = new_url
self.code = code
def get_response(self, environ):
return redirect(self.new_url, self.code)
raise RequestRedirect('example3', code=302)
helper_function()
return 'Hello World!'
@app.route('/example2')
def example2():
return 'Example 2'
@app.route('/example3')
def example3():
return 'Example 3'
if __name__ == '__main__':
app.run(host="0.0.0.0", port=9999, debug=True)</pre></noscript>
<script src="https://gist.github.com/762876a1770e0d2593b6.js"> </script>
Reload Enviromental Variables Without Logging Out - Ubuntu2014-11-15T21:54:00+00:00https://www.paulsprogrammingnotes.com/2014/11/reload-enviromental-variables-withoutThis doesn't actually work: "source /etc/environment"<br />This did the trick though:<pre style="background: rgb(238, 238, 238); border-left-color: rgb(204, 204, 204); border-left-style: solid; border-width: 0px 0px 0px 5px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 17.8048000335693px; margin-bottom: 10px; max-height: 600px; overflow: auto; padding: 5px 5px 5px 10px; vertical-align: baseline; width: auto; word-wrap: normal;"><code style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; margin: 0px; padding: 0px; vertical-align: baseline; white-space: inherit;">for line in $( cat /etc/environment ) ; do export $line ; done</code></pre><a href="http://superuser.com/questions/339617/how-to-reload-etc-environment-without-rebooting#comment371112_339617">http://superuser.com/questions/339617/how-to-reload-etc-environment-without-rebooting#comment371112_339617</a>Open Source Charting Library With Vertical Markers/Annotations2014-11-15T18:42:00+00:00https://www.paulsprogrammingnotes.com/2014/11/open-source-charting-library-withSaw this charting library from the Mozilla Metrics Team on hacker news recently: <a href="http://metricsgraphicsjs.org/interactive-demo.htm">http://metricsgraphicsjs.org/interactive-demo.htm</a><br /><br />Their interactive example is here: <a href="http://metricsgraphicsjs.org/interactive-demo.htm">http://metricsgraphicsjs.org/interactive-demo.htm</a><br /><br />I'm not aware of any other charting libraries besides highcharts that do vertical marker lines this easily.Python datetime.strftime Cheat Sheet2014-11-14T17:52:00+00:00https://www.paulsprogrammingnotes.com/2014/11/python-datetimestrftime-cheat-sheet<a href="http://strftime.org/">http://strftime.org/</a>Using Bootstrap-DateRangePicker As A Timepicker2014-11-11T07:32:00+00:00https://www.paulsprogrammingnotes.com/2014/11/using-bootstrap-daterangepicker-asHere's the jsfiddle: <a href="http://jsfiddle.net/d8uomdjv/">http://jsfiddle.net/d8uomdjv/</a><br /><br />It seems like the author is opposed to adding this sort of functionality: <a href="https://github.com/dangrossman/bootstrap-daterangepicker/issues/295">https://github.com/dangrossman/bootstrap-daterangepicker/issues/295</a>re.match vs re.search - Python2014-11-11T01:13:00+00:00https://www.paulsprogrammingnotes.com/2014/11/rematch-vs-research-python<a href="http://stackoverflow.com/a/180993/1364191">http://stackoverflow.com/a/180993/1364191</a><br /><br /><ul><li><b>re.match</b> - If zero or more characters at the beginning of string match the regular expression pattern.</li><li><b>re.search</b> - Scan through string looking for a location where the regular expression pattern produces a match</li></ul><br /><br />New Machine Setup For Flask Development On Ubuntu2014-11-09T01:34:00+00:00https://www.paulsprogrammingnotes.com/2014/11/new-machine-setup-for-flask-development<p>Here are the commands I use when I bring up a new Ubuntu server for flask development:</p>
<noscript><pre>sudo apt-get update && time sudo apt-get dist-upgrade
sudo timedatectl set-timezone America/Chicago
apt-get install git fail2ban htop nano
mkdir -p .ssh
nano .ssh/authorized_keys
eval `ssh-agent -s`
ssh-add ~/.ssh/id_rsa
apt-get install python-setuptools
easy_install pip
pip install virtualenvwrapper
sudo mkdir ~/virtualenvs
sudo sh -c "echo 'WORKON_HOME=~/virtualenvs' >> /etc/environment"
for line in $( cat /etc/environment ) ; do export $line ; done
mkdir -p $WORKON_HOME
sudo sh -c "echo 'source /usr/local/bin/virtualenvwrapper.sh' >> ~/.bashrc"
bash
mkvirtualenv default
apt-get install python-dev</pre></noscript>
<script src="https://gist.github.com/f7d58208cbeddb8c3b69.js"> </script>
MySQL Performance LXC Container vs Host2014-10-23T05:25:00+00:00https://www.paulsprogrammingnotes.com/2014/10/mysql-performance-lxc-container-vs-hostI have a linux virtual container (LXC) and the server hosting the container running the same version of MySQL with the same database.<br /><br />I tried running some select queries on both of the both the virtual container and host for benchmark purposes. My conclusion? MySQL read performance inside and outside the LXC container is the exactly the same.WHMCS Cronjob Error2014-10-22T00:54:00+00:00https://www.paulsprogrammingnotes.com/2014/10/whmcs-cronjob-errorIf you get the following error while you try to run the WHMCS cronjob from the command line:<br />Site error: the file <b>/var/www/accounts/admin/cron.php</b> requires the ionCube PHP Loader ioncube_loader_lin_5.3.so to be installed by the website operator. If you are the website operator please use the <a href="http://www.ioncube.com/lw/">ionCube Loader Wizard</a> to assist with installation.<br /><br />You need to add your the ioncube zend extension to the CLI php.ini too: <span style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">/etc/php5/cli/php.ini</span><br /><br /><br />Sendmail/STARTTLS verify=FAIL2014-10-19T19:33:00+00:00https://www.paulsprogrammingnotes.com/2014/10/sendmailstarttls-verifyfail<br /><div class="Mu SP" id=":ew.ma" style="-webkit-transition: opacity 0.218s ease; font-family: arial, sans-serif; font-size: 13px; line-height: 16px; margin-bottom: 6px; opacity: 1; transition: opacity 0.218s ease; word-break: break-word; word-wrap: break-word;"><div class="xH" id=":ew.at" style="-webkit-text-stroke-width: 0px; color: #262626; font-family: arial, sans-serif; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: auto; text-align: center; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"></div></div><br /><div class="Mu SP" id=":ev.ma" style="-webkit-transition: opacity 0.218s ease; font-family: arial, sans-serif; font-size: 13px; line-height: 16px; margin-bottom: 6px; opacity: 1; transition: opacity 0.218s ease; word-break: break-word; word-wrap: break-word;"><div id=":ev.co" style="font-family: arial, sans-serif;">Oct 19 14:04:34 Billing sm-mta[17583]: STARTTLS=client, relay=aspmx.l.google.com., version=TLSv1/SSLv3, verify=FAIL, cipher=ECDHE-RSA-RC4-SHA, bits=128/128</div><div id=":ev.co" style="font-family: arial, sans-serif;"><br /></div><div id=":ev.co" style="font-family: arial, sans-serif;">If you see those errors in your mail.log and your emails are failing to send, you need to add your ssl cert from apache to the sendmail config in /etc/mail/sendmail.cf:</div><div id=":ev.co" style="font-family: arial, sans-serif;"><div id=":ev.co"># CA directory</div><div id=":ev.co">O CACertPath=/etc/apache2/ssl</div><div id=":ev.co"># CA file</div><div id=":ev.co">O CACertFile=/etc/apache2/ssl/example.ca-bundle</div><div id=":ev.co"># Server Cert</div><div id=":ev.co">O ServerCertFile=/etc/apache2/ssl/example.cert</div><div id=":ev.co"># Server private key</div><div id=":ev.co">O ServerKeyFile=/etc/apache2/ssl/example.key</div><div><br /></div></div></div>Securing External LDAP Connections2014-10-19T03:58:00+00:00https://www.paulsprogrammingnotes.com/2014/10/securing-external-ldap-connections<a href="https://help.ubuntu.com/community/SecuringOpenLDAPConnections">https://help.ubuntu.com/community/SecuringOpenLDAPConnections</a><br /><br />I was getting the following error when I tried testing with openssl:<br /><b>error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177</b><br /><b><br /></b>The solution? It turns out that <span style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">/etc/default/slapd needs to be configured to use /etc/ldap/ldap.conf using the following:</span><br /><span style="color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"></span><br /><span style="color: #262626; font-family: arial, sans-serif; font-size: x-small;"><span style="line-height: 16px;"><b>SLAPD_OPTIONS="-f /etc/ldap/slapd.conf"</b></span></span><br /><div><br /></div>Adding Virtualhost In Apache For Port2014-10-17T16:26:00+00:00https://www.paulsprogrammingnotes.com/2014/10/adding-virtualhost-in-apache-for-portSay we wanted to route the <your-domain>.com:8072/misc to /var/www/misc...<br /><br />Add the following to /etc/apache2/ports.conf:<br /><b>NameVirtualHost *:8072</b><br /><b>Listen 8072</b><br /><div><br /></div><div>Add a file that describes your site to /etc/apache2/sites-available with this:</div><div><div><b><VirtualHost *:8072></b></div><div><b> ServerName <your-domain>.com</b></div><div><b> DocumentRoot /var/www/misc/</b></div><div><b></VirtualHost></b></div></div><div><br /></div><div>Run this command: <b>sudo a2ensite misc</b></div><div><br /></div><div>And restart apache: <b>sudo service apache2 restart</b></div>Deploying Nginx + Gunicorn + Flask2014-10-16T19:33:00+00:00https://www.paulsprogrammingnotes.com/2014/10/deploying-nginx-gunicorn-flaskWhy use both Nginx and Gunicorn?: <a href="http://serverfault.com/a/220047" style="font-family: 'Segoe UI'; font-size: 10pt;" title="http://serverfault.com/a/220047">http://serverfault.com/a/220047</a><br /><br />Simple explanation: <a href="http://prakhar.me/articles/flask-on-nginx-and-gunicorn/" style="font-family: 'Segoe UI'; font-size: 10pt;" title="http://prakhar.me/articles/flask-on-nginx-and-gunicorn/">http://prakhar.me/articles/flask-on-nginx-and-gunicorn/</a><br /><br />Nginx SSL configuration: <a href="https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-with-ssl-as-a-reverse-proxy-for-jenkins" style="font-family: 'Segoe UI'; font-size: 10pt;" title="https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-with-ssl-as-a-reverse-proxy-for-jenkins">https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-with-ssl-as-a-reverse-proxy-for-jenkins</a><br /><br />The thing I spent the longest on was getting supervisor to work. Gunicorn kept giving me "gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>", because I was missing the "directory" parameter. I needed to tell supervisor which directory to start from.<br /><br />[program:my_app]<br />command=/<path>/venv/prod/bin/gunicorn -w 8 main:app -b 127.0.0.1:8080<br />user=me<br />autostart=true<br /><b>directory=/home/me/my_dir/</b><br />stdout_logfile=/home/me/my_dir/gunicorn_supervisor.log<br />stdout_logfile_maxbytes=20MB<br />redirect_stderr=true<br /><div><br />You will need to create an upstart script to get supervisor to run when the system starts: http://serverfault.com/a/96500</div>Python Excel Library Benchmark Comparison2014-10-15T05:01:00+00:00https://www.paulsprogrammingnotes.com/2014/10/python-excel-library-benchmark<a href="https://github.com/swistakm/python-excel-benchmarks">https://github.com/swistakm/python-excel-benchmarks</a><br /><br />benchmark_csv 0.026495<br />benchmark_excellent 1.784107<br />benchmark_openpyxl 2.897072<br />benchmark_openpyxl_rows 7.025895<br />benchmark_pyexcelerate 0.550225<br />benchmark_xlsxcessive 1.430242<br />benchmark_xlsxwriter 1.611668<br />benchmark_xlwt 1.275144<br /><br /><a href="http://xlsxwriter.readthedocs.org/en/latest/working_with_memory.html">http://xlsxwriter.readthedocs.org/en/latest/working_with_memory.html</a><br /><br />pyexcelerate : 10.11<br />xlwt : 15.67<br />xlsxwriter (optimised): 19.70<br />xlsxwriter : 23.50<br />openpyxl (optimised): 95.82<br />openpyxl : 95.90Large Dictionaries Not Released From Memory - Python + Ubuntu2014-10-13T18:27:00+00:00https://www.paulsprogrammingnotes.com/2014/10/large-dictionaries-not-released-fromI have a flask application running on Ubuntu which reads from a file and creates several large dictionaries. After the dictionaries are created and flask returns the request, the memory used for the dictionaries is not released back to the OS. This example shows the problem: <a href="https://gist.github.com/pawl/c3ed7663b94abec01d75">https://gist.github.com/pawl/c3ed7663b94abec01d75</a><br /><br />It's not really a Flask issue, but the Flask guys were super helpful when I asked why my large dictionary wasn't released from memory: <a href="https://github.com/mitsuhiko/flask/issues/1202">https://github.com/mitsuhiko/flask/issues/1202</a><br /><br />I dug even deeper and learned this is a Linux thing:<br />"Python returns memory to the OS on the heap (that allocates other objects than small objects) <b>only on Windows</b>, if you run on Linux, you can only see the total memory used by your program increase."<br /><a href="http://deeplearning.net/software/theano/tutorial/python-memory-management.html">http://deeplearning.net/software/theano/tutorial/python-memory-management.html</a><br /><br />My solution:<br />Turning it into a function and <b>running it as a separate process</b>: <a href="https://gist.github.com/pawl/95769724848269cff890">https://gist.github.com/pawl/95769724848269cff890</a><br /><br />If you want to go even deeper down the rabbit hole, read these:<br /><ul><li>"most malloc implementations will not release memory to the operating system, and the few that do, do not do it very easily" -<a href="http://stackoverflow.com/questions/2215259/will-malloc-implementations-return-free-ed-memory-back-to-the-system">http://stackoverflow.com/questions/2215259/will-malloc-implementations-return-free-ed-memory-back-to-the-system</a></li><li>"We compiled a version of Python with TCMalloc that only uses mmap. When testing the new Python in one of our largest projects, we found that not only did Python give back memory to the OS correctly, it also had a reduced memory usage and no apparent CPU penalty for using mmap instead of brk." - <a href="http://pushingtheweb.com/2010/06/python-and-tcmalloc/">http://pushingtheweb.com/2010/06/python-and-tcmalloc/</a></li></ul>Renaming Screen - Ubuntu2014-10-08T18:23:00+00:00https://www.paulsprogrammingnotes.com/2014/10/renaming-screen-ubuntu<a href="http://superuser.com/a/521112">http://superuser.com/a/521112</a><br /><br />screen -S 8890.foo -X sessionname barInitializing Dictionary Elements Inside A Loop - Python2014-10-04T08:29:00+00:00https://www.paulsprogrammingnotes.com/2014/10/initializing-dictionary-elements-insideThere's a really interesting tip in the "Initializing Dictionary Elements" section of this page: <a href="https://wiki.python.org/moin/PythonSpeed/PerformanceTips">https://wiki.python.org/moin/PythonSpeed/PerformanceTips</a><br /><br />"<span style="background-color: white; font-family: Arial, Verdana, Geneva, 'Bitstream Vera Sans', Helvetica, sans-serif; font-size: 14px; line-height: 21.2591991424561px;">it is cheaper to use a </span><tt style="background-color: white;">try</tt><span style="background-color: white; font-family: Arial, Verdana, Geneva, 'Bitstream Vera Sans', Helvetica, sans-serif; font-size: 14px; line-height: 21.2591991424561px;"> statement" when checking to see if a key exists in a dictionary (inside of a loop).</span>maximum recursion depth exceeded - PyIntervalTree2014-10-02T17:59:00+00:00https://www.paulsprogrammingnotes.com/2014/10/maximum-recursion-depth-exceededI was getting the "maximum recursion depth exceeded" when I tried to run large datasets through PyIntervalTree.<br /><br />I never fixed the issue or found anyone else with the issue. I ended up switching to bx-python instead: <a href="https://www.biostars.org/p/99/#100">https://www.biostars.org/p/99/#100</a>Group By Apply Occurring Twice On First Item - Pandas2014-10-02T17:10:00+00:00https://www.paulsprogrammingnotes.com/2014/10/group-by-apply-occurring-twice-on-first<a href="https://github.com/pydata/pandas/issues/7739#issuecomment-48823964">https://github.com/pydata/pandas/issues/7739#issuecomment-48823964</a><br /><br />If Pandas's Group By is applying the function in Apply twice to the first item, it's actually expected behavior according to the warning in the documentation: <a href="http://pandas.pydata.org/pandas-docs/stable/groupby.html#flexible-apply">http://pandas.pydata.org/pandas-docs/stable/groupby.html#flexible-apply</a>Virtualenvwrapper Basics2014-09-28T08:25:00+00:00https://www.paulsprogrammingnotes.com/2014/09/virtualenvwrapper-basicsThis video does a good job of describing the basics of Virtualenvwrapper: <a href="https://www.youtube.com/watch?v=thHNYVrY0lU">https://www.youtube.com/watch?v=thHNYVrY0lU</a><br /><br />You'll need to have used virtualenv to appreciate this.<br /><br />Basically, you create virtual environments with <b>"mkvirtualenv"</b> and they're stored in a central location. When you want to switch to your virtual environment, you use the <b>"workon"</b> command.<br /><br />You can also create a postactivate script that loads environmental variables when you switch to your virtual environment.<br /><br />Note: There is also a module called Pew that seems to have a simpler installation process: <a href="https://github.com/berdario/pew">https://github.com/berdario/pew</a>Meetup API address_1_error2014-09-25T03:10:00+00:00https://www.paulsprogrammingnotes.com/2014/09/meetup-api-address1errorI was getting this error while using the POST /venues method of the meetup API.<br /><br />The error seems to happen when you duplicate the address of another venue. I was able to add the venue, but when I tried to add it a second time - I would get this error.<br /><br />I ended up getting the list of venues and trying to find a match first.Profiling Python Scripts2014-09-20T20:24:00+00:00https://www.paulsprogrammingnotes.com/2014/09/profiling-python-scripts<a href="https://github.com/what-studio/profiling">https://github.com/what-studio/profiling</a>Easiest Way To Install Python Libraries In Windows (that require compiling)2014-09-18T19:03:00+00:00https://www.paulsprogrammingnotes.com/2014/09/easiest-way-to-install-python-libraries<a href="http://www.lfd.uci.edu/~gohlke/pythonlibs/">http://www.lfd.uci.edu/~gohlke/pythonlibs/</a>Eventbrite "Application Key Error"2014-09-16T07:29:00+00:00https://www.paulsprogrammingnotes.com/2014/09/eventbrite-application-key-errorThe error message looks like this:<br />{u'status': {u'http_code': 200}, u'contents': {u'error': {u'error_type': u'<b>Application Key Error</b>', u'error_message': u'<b>Please provide your Application Key in the URL </b>as "?app_key=<APP_KEY>".'}}}<br /><br />After a few hours of trying to get past the Eventbrite API's "Application Key Error". I finally figured out that I needed to switch my request URL from:<br /><b>"https://developer.eventbrite.com/json/event_new"</b><br />to:<br /><b>"https://www.eventbrite.com/json/event_new"</b>dc.js + Crosstab: Chart With Linked Table2014-09-14T23:59:00+00:00https://www.paulsprogrammingnotes.com/2014/09/dcjs-crosstab-chart-with-linked-table<a href="http://dc-js.github.io/dc.js/">http://dc-js.github.io/dc.js/</a>Flask OAuthLib Examples2014-09-13T06:40:00+00:00https://www.paulsprogrammingnotes.com/2014/09/flask-oauthlib-examples<a href="https://github.com/lepture/flask-oauthlib/blob/master/example/github.py">https://github.com/lepture/flask-oauthlib/blob/master/example/github.py</a><br /><a href="https://github.com/bjourne/vvm/blob/master/app/users/views.py">https://github.com/bjourne/vvm/blob/master/app/users/views.py</a>Override get_query based on GET parameter - Flask-Admin2014-09-10T18:59:00+00:00https://www.paulsprogrammingnotes.com/2014/09/override-getquery-based-on-get<p>This will change the results in a flask-admin index view based on GET parameters. In most cases, using filters to do this is probably a better idea.</p>
<noscript><pre>from application import db
from application.views.modelview import ModelView
from application.models import Things # has an attribute called thing_type
class MyModelView(ModelView):
def get_query(self):
thing_type = request.args.get('type', None) # pretending we have a GET parameter called "type"
if thing_type == "type1":
return super(ModelView, self).get_query().filter(Things.thing_type.like('type1 %'))
elif thing_type == "type2":
return super(ModelView, self).get_query().filter(Things.thing_type.like('type2 %'))
else:
return super(ModelView, self).get_query()
# note, you can't define get_query inside of def index_view(self)</pre></noscript>
<script src="https://gist.github.com/5fcd1d206c874d091302.js"> </script>
Changing A Single GET Parameter In Request - Flask2014-09-10T16:42:00+00:00https://www.paulsprogrammingnotes.com/2014/09/changing-single-get-parameter-in<noscript><pre>from application import app
from flask import request, url_for, redirect
@app.route('/')
def index():
modified_args = dict(request.args)
modified_args['key'] = 'modified_value'
return redirect(url_for('my_view', **modified_args))</pre></noscript>
<script src="https://gist.github.com/10edd7bd2305379ef108.js"> </script>
SQLAlchemy Performance Tip2014-09-03T18:51:00+00:00https://www.paulsprogrammingnotes.com/2014/09/sqlalchemy-performance-tip<a href="http://www.mobify.com/blog/sqlalchemy-memory-magic/">http://www.mobify.com/blog/sqlalchemy-memory-magic/</a><br /><br />"If you can process the results of database queries iteratively (and very often you can), stream the results"<br /><br />"Since the data needs to travel over the network from the database whether it's streamed or not, this doesn't add a huge overhead, but we'll see that it reduces memory requirements."<br /><br />Example (using Flask-SQLAlchemy):<br /><b>db.engine.execution_options(stream_results=True).execute(query)</b><br /><b><br /></b>However, that's not the full story when it comes to MySQLdb (MySQL-python). <a href="http://stackoverflow.com/a/3699677">http://stackoverflow.com/a/3699677</a><br /><br />If you're going to do this with mysql, I recommend using oursql: <a href="https://pypi.python.org/pypi/oursql">https://pypi.python.org/pypi/oursql</a>Enable Threading - Flask2014-09-03T18:47:00+00:00https://www.paulsprogrammingnotes.com/2014/09/enable-threading-flask<a href="http://stackoverflow.com/a/14823968/1364191">http://stackoverflow.com/a/14823968/1364191</a><br /><br />If you're going to be expecting more than one user on your flask server, you need to enable threading like this: app.run(host='0.0.0.0', port=5000, <b>threaded=True</b>)<br /><br />Unfortunately, the threading makes Ctrl+C not work sometimes and I'll have to kill the thread manually.<br /><br />For better solutions for production, check out these WSGI solutions: <a href="http://flask.pocoo.org/docs/0.10/deploying/wsgi-standalone/">http://flask.pocoo.org/docs/0.10/deploying/wsgi-standalone/</a>Mosh - SSH Without Dropped Connections2014-09-01T21:59:00+00:00https://www.paulsprogrammingnotes.com/2014/09/mosh-ssh-without-dropped-connections<a href="https://mosh.mit.edu/">https://mosh.mit.edu/</a><br /><br />Warning: As of 10/17/2014, it's still in development and they haven't implemented scrollback yet. However, you can use screen and view the scrollback with Ctrl+A then Esc.Things I Learned From DEFCON2014-08-31T20:52:00+00:00https://www.paulsprogrammingnotes.com/2014/08/things-i-learned-from-defcon<div dir="ltr">Pentoo - Linux distro for pen testing.<br />VMware fusion - Most of the presenters were using a Macbook with VMware fusion managing their VMs.<br />Chinavasion - One of the presentations mentioned this in the context of dealextreme, I'm assuming this site is similar.<br />Kali linux - Another distro for pen testing.<br />Open bts <br />Sqlmap (SQLi injection) - Python tool used to tell if php pages were vulnerable to SQLi injection.<br />C99 shell - PHP interface for shell level system functions.<br />b374k shell - Another shell which allows an user to run file system, database, and shell commands from a web interface.<br />Accunetix (xss)<br />Business logic flaws in mobile operators<br />Doskey /history - Will show you all the previous commands typed into command prompt.<br />Maria DB - When oracle took over MySQL, it forked and maria db is still run by the previous developers.<br />Firefox imacro - A presenter said he uses this for all of his bots.<br />Burp repeater - Looks similar to fiddler, allows repeating http packets. Maybe only for Mac.<br />Fritzing<br />Groupie and Geocouch (couch dB) - This was used in combination with the unity engine to display clusters of map points in a video game.<br />Steganography - The word for hiding data inside of other files. A presenter showed how histograms could be used.<br />Femtocells - A personal cell tower. One of the presentations talked about how to use a femtocell to intercept text messages to a Verizon (CDMA) phone.<br />Shark - Has something to do with a hadoop, good for storing large amounts of data.<br />SuperTimeline</div>Alembic Cheat Sheet - Python/SQLAlchemy2014-08-27T07:34:00+00:00https://www.paulsprogrammingnotes.com/2014/08/alembic-cheat-sheet-pythonsqlalchemyalembic revision --autogenerate -m "<your message>"<br />alembic upgrade head<br /><br />Problem: <b>FAILED: Target database is not up to date.</b><br />Solution: <b>alembic upgrade head</b><br /><br />Problem: <b>No such revision '5000106def16'</b><br /><div>Solution: </div><div><ol><li>sqlite3 db.sqlite3</li><li>drop table alembic_version; (and exit, ctrl+d)</li><li>alembic upgrade head</li></ol></div>Failed to fetch http://security.ubuntu.com2014-08-27T07:23:00+00:00https://www.paulsprogrammingnotes.com/2014/08/failed-to-fetch-httpsecurityubuntucomW: Failed to fetch http://security.ubuntu.com/ubuntu/dists/quantal-security/main/binary-amd64/Packages 404 Not Found [IP: 91.189.91.14 80]<br /><div><br /></div><div>This is happening because you're on an unsupported/old version of Ubuntu.</div><div><br /></div><div>One way to fix it:</div><div><ol><li>nano /etc/apt/sources.list</li><li>Change all entries referring to http://<b>archive</b>.ubuntu.com/ubuntu to http://<b>old-releases</b>.ubuntu.com/ubuntu</li></ol></div>Best Python Tricks2014-08-22T04:17:00+00:00https://www.paulsprogrammingnotes.com/2014/08/best-python-tricks<a href="https://gist.github.com/JeffPaine/6213790">https://gist.github.com/JeffPaine/6213790</a><br /><br />Most of the examples involve making your code much faster by using iterators instead of creating copies of the data. Example: xrange<br /><br />Here's the video: <a href="https://www.youtube.com/watch?v=OSGv2VnC0go">https://www.youtube.com/watch?v=OSGv2VnC0go</a>CSS Selectors - ~ + >2014-08-21T21:38:00+00:00https://www.paulsprogrammingnotes.com/2014/08/css-selectors<a href="http://css-tricks.com/child-and-sibling-selectors/">http://css-tricks.com/child-and-sibling-selectors/</a><br /><br />That's an excellent article describing what angle brackets, plus signs, and tildes do in CSS selectors. The illustrations make it especially clear.Change Percent To Numbers and Keep Color - Flot Pie Chart2014-08-06T18:51:00+00:00https://www.paulsprogrammingnotes.com/2014/08/change-percent-to-numbers-and-keep<p>You can access the color of the section of the pie chart by using “series.color”:</p>
<noscript><pre>options = {
series: {
pie: {
innerRadius: 0.5,
show: true,
label: {
show: true,
formatter: function (label, series) {
console.log(series);
return '<div style="font-size:8pt;text-align:center;padding:2px; color: ' + series.color +';">' + label + '<br/>' + series.data[0][1] + '</div>';
},
}
}
},
};</pre></noscript>
<script src="https://gist.github.com/e8e2b246365c60755403.js"> </script>
Use List In Bind Parameters - SQLAlchemy2014-08-01T00:34:00+00:00https://www.paulsprogrammingnotes.com/2014/08/use-list-in-bind-parameters-sqlalchemy<noscript><pre>my_list = ['peach', 'grape', 'apple']
query_parameters = {}
counter = 1
for list_item in my_list:
query_parameters["list_item" + str(counter)] = list_item
counter += 1
where_clause = 'fruits IN(:' + ",:".join(query_parameters.keys()) + ')' # create clause to be inserted into query
query_text = db.text("""
SELECT
fruits
FROM table
WHERE """ + where_clause + """
""")
result = db.engine.execute(query_text, **query_parameters)</pre></noscript>
<script src="https://gist.github.com/555e5eecce77d4de0ada.js"> </script>
Overwriting module with same version in Heroku - Python2014-07-30T09:01:00+00:00https://www.paulsprogrammingnotes.com/2014/07/overwriting-python-module-in-herokuI needed to install a newer version of a module from a github commit, but it had the same version number. Heroku just used the cached version of the module and didn't overwrite it with the newer version.<br /><br />The easiest fix for this:<br /><br /><ol><li>Create a file in your application's root directory called "runtime.txt" with only "python-3.4.1" written inside.</li><li>Add, commit, and push the new runtime.txt</li><li>Change runtime.txt back to "python-2.7.6"</li><li>Add, commit, and push to clear your virtualenv and reinstall all modules</li></ol>The server quit without updating PID file (/var/run/mysqld/mysqld.pid). - MySQL2014-06-27T23:25:00+00:00https://www.paulsprogrammingnotes.com/2014/06/the-server-quit-without-updating-pid. * The server quit without updating PID file (/var/run/mysqld/mysqld.pid).<br /><br />If you're getting this error message, <b>try using sudo</b> to start mysql.Submit Form When User Clicks Option (autosubmit) - Typeahead.js 2014-06-19T20:56:00+00:00https://www.paulsprogrammingnotes.com/2014/06/submit-form-when-user-clicks-option<a href="http://stackoverflow.com/a/18483816/1364191">http://stackoverflow.com/a/18483816/1364191</a><br /><br />The bolded section below will automatically submit your form when you select an option from the Typeahead.js dropdown:<br />$('#remote .typeahead').typeahead(null, {<br /><span class="Apple-tab-span" style="white-space: pre;"> </span> displayKey: 'value',<br /><span class="Apple-tab-span" style="white-space: pre;"> </span> source: site_search.ttAdapter()<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>})<b>.on('typeahead:selected', function(e){</b><br /><b> e.target.form.submit();</b><br /><b> });</b>Using Enter Key With Typeahead.js - Javascript2014-06-19T20:29:00+00:00https://www.paulsprogrammingnotes.com/2014/06/using-enter-key-with-typeaheadjs<a href="https://github.com/twitter/typeahead.js/issues/255">https://github.com/twitter/typeahead.js/issues/255</a><br /><br />If your text input with typeahead.js is not letting you use the enter key, you need to add a submit button (if you don't already have one).<br /><br />You can hide it if you want:<br /><b><input type="submit" value="Submit" style="display: none"></b>WinSCP vs Samba For Windows <---> Ubuntu Development2014-06-12T15:36:00+00:00https://www.paulsprogrammingnotes.com/2014/06/winscp-vs-samba-for-windows-ubuntu<p>I’ve always used WinSCP to transfer files between my Windows development environment and Ubuntu. It worked pretty well. All you had to do was open the file with your editor by clicking the file in WinSCP, and when you saved the file it would save to the server in the background.</p>
<p>The main negative was the lack of feedback when a file fails to save to the server. WinSCP could be logged out and asking for a password in the background, but you were expecting that save to fix the bug you were working on. It ended up being another thing to debug.</p>
<p>A coworker suggested using Samba instead. I installed it with this simple guide: <a href="https://help.ubuntu.com/community/How%20to%20Create%20a%20Network%20Share%20Via%20Samba%20Via%20CLI%20(Command-line%20interface/Linux%20Terminal)%20-%20Uncomplicated,%20Simple%20and%20Brief%20Way!">How to Create a Network Share Via Samba Via CLI</a> Then I mapped the network drive in Windows by going to the “Map Network Drive” option in the tools menu of the file explorer:</p>
<p><img src="/assets/images/map_network_drive.png" alt="map network drive" /></p>
<p>The only negative I’ve found with Samba so far is that it makes Notepad++ a little less responsive. But it helps a small amount if you turn off “File Status Auto-Detection”:</p>
<p><img src="/assets/images/notepad.png" alt="File Status Auto-Detection" /></p>
Relative Paths In Scripts Run By Crontab - Python2014-06-10T15:52:00+00:00https://www.paulsprogrammingnotes.com/2014/06/relative-paths-in-scripts-run-byI learned that a script run by crontab can't have a relative path.<br /><br />This fix was here: <a href="http://stackoverflow.com/a/10422444">http://stackoverflow.com/a/10422444</a><br /><br />The following example opens my_file.txt in a path relative to your script:<br /><b>import os </b><br /><b>dir_path = os.path.dirname(os.path.abspath(__file__))</b><br /><b>with open (os.path.join(dir_path, 'my_file.txt'), "r") as myfile:</b><br /><b> mytext = myfile.read()</b>ELSE in FOR loops - Python2014-06-09T16:19:00+00:00https://www.paulsprogrammingnotes.com/2014/06/else-in-for-loops-python<a href="http://stackoverflow.com/a/9980160">http://stackoverflow.com/a/9980160</a><br /><br />Turns out that you can use ELSE after a FOR loop to make it give an output if it didn't have anything to iterate.Undoing git rm -r --cached 2014-06-09T13:37:00+00:00https://www.paulsprogrammingnotes.com/2014/06/undoing-git-rm-r-cached-gitI followed these instructions to gitignore an already committed file: <a href="http://stackoverflow.com/questions/6535362/gitignore-after-commit">http://stackoverflow.com/questions/6535362/gitignore-after-commit</a><br /><br />When I wanted to start tracking it again, I needed to following these instructions: <a href="https://gist.github.com/laszlomiklosik/3833968">https://gist.github.com/laszlomiklosik/3833968</a><br /><br />Make sure you save a backup of your current version before you check out the file again.Multiple DSNs All Connecting To The Same IP - Sybase ODBC2014-06-05T19:10:00+00:00https://www.paulsprogrammingnotes.com/2014/06/multiple-dsns-all-connecting-to-same-ipThe problem: You have multiple DSNs defined in your unixODBC's odbc.ini file, and it always connects to the same IP when you try testing with iSQL.<br /><br />The solution: <br /><b>DO NOT</b> use the following line in your odbc.ini file to define your DSN's port:<br />CommLinks = tcpip(host=localhost;port=9502)<br /><div><br /></div><div><b>DO</b> use the following format:</div><div><div>CommLinks = tcpip(HOST=localhost:9502)</div></div><div><br /></div>Compress - Sybase ODBC2014-06-05T18:20:00+00:00https://www.paulsprogrammingnotes.com/2014/06/compress-sybase-odbcSybase has a odbc parameter called "Compress" that is disabled by default. It's very useful if you're running queries on something with slow network transfer speeds and large datasets.<br /><br />Example:<br />Compress = yes<br /><br />This improve the performance of my query by about 40%.[IM004] [unixODBC][Driver Manager]Driver's SQLAllocHandle on SQL_HANDLE_HENV failed (0) (SQLDriverConnect) - Ubuntu2014-06-02T22:45:00+00:00https://www.paulsprogrammingnotes.com/2014/06/im004-unixodbcdriver-managerdriversThe link to download SQLanywhere is here: <a href="http://www.sybase.com/detail?id=1087327">http://www.sybase.com/detail?id=1087327</a><br /><br />To install it, you need to unzip the file and run "./setup" in the top level directory in the unzipped folder.<br /><br />I also installed:<br /><a href="https://code.google.com/p/pyodbc/">https://code.google.com/p/pyodbc/</a> (use the .zip file on the left)<br /><a href="http://onefinepub.com/2014/03/05/installing-unixodbc-2-3-2-higher-ubuntu-12-04-lts/">http://onefinepub.com/2014/03/05/installing-unixodbc-2-3-2-higher-ubuntu-12-04-lts/</a><br /><br />The first error message I received was:<br />sqlanydb.InterfaceError: Could not load dbcapi. Tried: dbcapi.dll,libdbcapi_r.so,libdbcapi_r.dylib<br />Exception AttributeError: "'Root' object has no attribute 'api'" in <bound method Root.__del__ of <sqlanydb.Root object at 0x3061150>> ignored<br /><div><br /></div><div>This error is resolved by creating a file called "/etc/ld.so.conf.d/sqlanywhere.conf" and adding a line with "/opt/sqlanywhere12/lib64". Then you will need to run ldconfig -v.</div><br />The error message in the title occurred on Ubuntu because unixODBC was unable to find all of the libraries it needed to run.<br /><br />This helped to find the issue: <a href="http://blogs.msdn.com/b/dataaccesstechnologies/archive/2014/01/22/sqlallochandle-on-sql-handle-henv-from-linux-sqlncli-driver.aspx">http://blogs.msdn.com/b/dataaccesstechnologies/archive/2014/01/22/sqlallochandle-on-sql-handle-henv-from-linux-sqlncli-driver.aspx</a><br /><br />strace -t -f -o trace_out.txt isql -v <DSN>< USERNAME> <PASSWORD><br /><br />It will show there are a bunch of missing files. The issue was resolved when isql was run from the /opt/sqlanywhere12/bin64 directory. This indicated that it's not looking in the right place for the library files. Run the following: <b>export LD_LIBRARY_PATH='/opt/sqlanywhere12/lib64'</b><br /><b><br /></b>Not having the environmental variable above will also cause "sqlanydb.InterfaceError: dbcapi version 1 required." if you're using python's sqlanydb library.<br /><div><br /></div><div>It will give a "file not found" error if you try using bin32/lib32 files on a 64 bit system. Make sure you use the bin64/lib64 files in the "Driver=" fild in your /etc/odbc.ini file (if you're using a 64 bit system)!<br /><br />Now you will receive an error saying "Parse error: DSN does not exist". You need to add an ODBC_INI environmental variable to where your DSNs are defined. I ran the following: <b>export ODBC_INI='/etc/odbc.ini'</b><br /><br />To make those variables we exported work system-wide and continue to work even after we log out:<br /><br /><ol><li>sudo nano /etc/environment</li><li>Add:<br /><b>ODBC_INI='/etc/odbc.ini'<br /><b>LD_LIBRARY_PATH='/opt/sqlanywhere12/lib64'</b></b></li></ol><br />These articles were helpful:<br /><a href="https://hynek.me/articles/twisted-sybase/">https://hynek.me/articles/twisted-sybase/</a><br /><a href="https://hynek.me/articles/a-short-summary-on-sybase-sql-anywhere-python/">https://hynek.me/articles/a-short-summary-on-sybase-sql-anywhere-python/</a></div>Escape Newlines In Description For Google Calendar Api - Python2014-06-02T06:41:00+00:00https://www.paulsprogrammingnotes.com/2014/06/escape-newlines-in-description-for<noscript><pre># how you need to format the body of the request
requestbody = """{
"description": %s,
"summary": %s
}
""" % (json.dumps(description), json.dumps(summary)) # use json.dumps to escape string for the request body
</pre></noscript>
<script src="https://gist.github.com/7017f7ef477bde8e9bdc.js"> </script>
Get ID of edited item within WTforms validator - Flask-Admin2014-06-02T03:04:00+00:00https://www.paulsprogrammingnotes.com/2014/06/get-id-of-edited-item-within-wtforms<noscript><pre>def start_must_not_conflict(form, field):
print request.args.get('id') # get ID
if Event.query.filter(db.and_(Event.location == form.location.data, Event.start.between(form.start.data, form.end.data), Event.id != request.args.get('id'))).first(): # exclude ID from query
raise wtforms.validators.ValidationError('Start time conflicts with another request for the same time.')
form_args = dict(
start=dict(validators=[start_must_not_conflict])
)</pre></noscript>
<script src="https://gist.github.com/b95ca0f5dc50c4f1995a.js"> </script>
Heroku's Advantages Over Google App Engine2014-05-26T20:43:00+00:00https://www.paulsprogrammingnotes.com/2014/05/heroku-advantages-over-google-app-enginePros:<br /><br /><ul><li>Postgres (you can even use pgAdmin to view your database)</li><li>Uses Git to push your application into production</li><li>More monitoring features and 3rd party apps</li><li>As long as your project uses one dyno, it runs for free (similar to google, but with a less confusing billing structure)</li></ul><div>Cons:</div><div><ul><li>10k row limit for your database before they start charging you $9 per month </li></ul></div>AttributeError: 'QuerySelectField' object has no attribute '_sa_instance_state'2014-05-25T02:56:00+00:00https://www.paulsprogrammingnotes.com/2014/05/attributeerror-queryselectfield-objectI came across this error when I was trying to make a custom validator in Flask-Admin with wtforms form objects.<br /><br />This code didn't work:<br />def location_must_not_conflict(form, field):<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>if Event.query.filter(Event.location == <b>form.location</b>).first():<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>raise wtforms.validators.ValidationError('Location conflicts with another request for the same venue.')<br /><br />Because I was using form.location and not form.location.data.<br /><br />This will work:<br />def location_must_not_conflict(form, field):<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>if Event.query.filter(Event.location == <b>form.location.data</b>).first():<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>raise wtforms.validators.ValidationError('Location conflicts with another request for the same venue.')Individual Form Fields In Jinja2 - Flask-Admin2014-05-21T04:20:00+00:00https://www.paulsprogrammingnotes.com/2014/05/individual-form-fields-in-jinja2-flaskOn your edit and create templates in Flask-Admin, you can print individual form fields like this (the field's name is location):<div><b>{{ form.location }}</b></div>Google Calendar Api bad request 400 - Python Authomatic Library2014-05-18T19:34:00+00:00https://www.paulsprogrammingnotes.com/2014/05/google-calendar-api-bad-request-400I was finally able to get the request to go through by adding a header saying it's json format. Here's an example:<br /><br />requestbody = """<br />{<br /> "end": {<br /> "dateTime": "2014-05-18T01:35:00Z"<br /> },<br /> "start": {<br /> "dateTime": "2014-05-18T01:35:00Z"<br /> },<br /> "description": "test2",<br /> "location": "test venue",<br /> "summary": "test2"<br /> }<br /><div>"""</div>response = authomatic.access(credentials, url, method='POST', headers={'Content-Type': 'application/json'}, body=requestbody)<br /><div><br /></div><div><br /></div>Add user_id for modified_by field before save - Flask-Admin2014-05-18T01:58:00+00:00https://www.paulsprogrammingnotes.com/2014/05/add-userid-for-modifiedby-field-before<br />In Flask-Admin, you can override the "on_model_change" function to to automatically populate a "modified_by" field in your model.<br /><br />class LocationView(ModelView):<span class="Apple-tab-span" style="white-space: pre;"> </span><br /><span class="Apple-tab-span" style="white-space: pre;"> </span>def on_model_change(self, form, model):<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>model.user_id = g.user.id<br /><br />admin = Admin(app)<br />admin.add_view(LocationView(Location, db.session, name="Locations"))Favorite Flask Boilerplate/Skeleton/Templates2014-05-17T07:31:00+00:00https://www.paulsprogrammingnotes.com/2014/05/favorite-flask-boilerplateskeletontempl<br /><ul><li><a href="https://github.com/sloria/cookiecutter-flask">https://github.com/sloria/cookiecutter-flask</a></li><li><a href="https://github.com/mattupstate/overholt">https://github.com/mattupstate/overholt</a></li><li>Because it's easy: <a href="https://github.com/fogleman/HelloFlask">https://github.com/fogleman/HelloFlask</a></li></ul><br /><br />Those do a lot of the work of starting a new Flask project for you.<br /><br />With cookiecutter flask, all you need to do to build a new project is: cookiecutter https://github.com/sloria/cookiecutter-flask.git<br /><br />As a warning, cookiecutter flask and overholt include a lot of stuff. I ended up not using it because it was too much work to change everything to work with oAuth.Remove Index Column From Pandas CSV Export2014-05-12T15:27:00+00:00https://www.paulsprogrammingnotes.com/2014/05/remove-index-column-from-pandas-csv<a href="http://stackoverflow.com/a/19782137/1364191">http://stackoverflow.com/a/19782137/1364191</a>Reverse PDF Page Order2014-05-11T02:13:00+00:00https://www.paulsprogrammingnotes.com/2014/05/reverse-pdf-page-order<p>I created an app that allows you to upload a pdf and it returns a copy of your PDF with the page order reversed. The code is here: <a href="https://github.com/pawl/pdfreverse">https://github.com/pawl/pdfreverse</a></p>
<p>Initially when I googled how to do this in Adobe Acrobat, the first few results had fairly complicated solutions. After I made the app, I ended up finding an easier way to do it in Adobe (in the 6th from the top Google result on “reverse pdf order acrobat”…):</p>
<p><img src="/generated/assets/images/2014-05-10+20_59_21-Print-629-e9de053b9.png" srcset="/generated/assets/images/2014-05-10+20_59_21-Print-400-e9de053b9.png 400w, /generated/assets/images/2014-05-10+20_59_21-Print-600-e9de053b9.png 600w, /generated/assets/images/2014-05-10+20_59_21-Print-629-e9de053b9.png 629w" /></p>
<p>Oh well, it was a fun project.</p>
Discourse As Potential Mailing List Replacement2014-04-29T05:09:00+00:00https://www.paulsprogrammingnotes.com/2014/04/discourse-as-potential-mailing-list<p>While tested out Discourse, I only found a few negative things:</p>
<ol>
<li>To make it act like a mailing list, it currently requires each user to change settings on their account: <a href="https://github.com/discourse/discourse/blob/master/docs/MAILING-LIST-SETUP.md#suggested-user-preferences">https://github.com/discourse/discourse/blob/master/docs/MAILING-LIST-SETUP.md#suggested-user-preferences</a><br />I commented on a thread about a new feature that allows setting defaults for this: <a href="https://meta.discourse.org/t/default-email-settings-for-a-new-user/14980">https://meta.discourse.org/t/default-email-settings-for-a-new-user/14980</a></li>
<li>It doesn’t allow setting defaults for which categories the user has “Watched”. For example, I would want to default the “General” list to “Watched”, but more obscure lists would only be watched if the user chooses to.</li>
<li>It uses a different e-mail address for each e-mail it sends you (this is to keep track of which thread it belongs to), here’s an example of a “From” address: testdiscourse+43854630fb3a3d9492505fa5a23db196@gmail.com<br />Google Groups always has the same to/from e-mail address and magically figures this out. This might be weird in some peoples’ email clients.</li>
</ol>
<p>As a forum, this beats the crap out of PhpBB, but as a mailing list it’s not a clear winner over Google Groups yet. It’s open source, has a better web interface, and the “Categories” are a cool feature for keeping all the lists in the same place. But, the “email in” functionality was too recently implemented, and it might take a few months before it’s as polished as Google Groups.</p>
<p>It’s going to be a very tempting replacement if they add default settings for user preferences.</p>
<p>Screenshots:</p>
<p><img src="/assets/images/discourse1.png" alt="discourse 1" /></p>
<p><img src="/assets/images/discourse2.png" alt="discourse 2" /></p>
<p><img src="/assets/images/discourse3.png" alt="discourse 3" /></p>
<p><img src="/assets/images/discourse4.png" alt="discourse 4" /></p>
<p><img src="/assets/images/discourse5.png" alt="discourse 5" /></p>
Data source name not found and no default driver specified - SQLalchemy/Sybase2014-04-28T19:05:00+00:00https://www.paulsprogrammingnotes.com/2014/04/data-source-name-not-found-and-noI was getting this error when trying to connect to a sybase database using Microsoft's odbc driver by using the connection string shown in the SqlAlchemy examples:<br /><b>DBAPIError: (Error) ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)') None None</b><br /><div><br /></div>The following code worked to connect:<br /><b>import pyodbc, sqlalchemy</b><br /><b><br /></b><b>def connect():</b><br /><b> return pyodbc.connect('DSN=<dsn name>;UID=<username>;PWD=<password>')</b><br /><b><br /></b><b>srcEngine = sqlalchemy.create_engine('sybase+pyodbc://', creator=connect, echo=True)</b>Docker Not Starting - Ubuntu2014-04-27T00:12:00+00:00https://www.paulsprogrammingnotes.com/2014/04/docker-not-starting-ubuntuI was getting this error message because docker was refusing to start: Cannot connect to the Docker daemon. Is 'docker -d' running on this host?<br /><br />This issue was occurring because my kernel was refusing to upgrade, because I was on a VPS. (you need to get your VPS provider to upgrade it for you)<br /><br />Run uname -r, if it returns something like this: "<b>2.6.32-042stab084.20</b>" That's probably the reason why docker isn't starting. Your kernel isn't compatible.<br /><br />If you're not on a VPS and can upgrade your kernel, try instructions on this page: <a href="http://docs.docker.io.s3-website-us-west-2.amazonaws.com/installation/ubuntulinux/">http://docs.docker.io.s3-website-us-west-2.amazonaws.com/installation/ubuntulinux/</a>E: Sub-process /usr/bin/dpkg returned an error code (1)2014-04-27T00:02:00+00:00https://www.paulsprogrammingnotes.com/2014/04/e-sub-process-usrbindpkg-returned-errorI was getting this error message while I was trying to upgrade the kernel on my VPS:<br /><b>/usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).</b><br /><b>run-parts: /etc/kernel/postrm.d/zz-update-grub exited with return code 1</b><br /><b>Failed to process /etc/kernel/postrm.d at /var/lib/dpkg/info/linux-image-3.8.0-34-generic.postrm line 328.</b><br /><b>dpkg: error processing linux-image-3.8.0-34-generic (--remove):</b><br /><b> subprocess installed post-removal script returned error exit status 1</b><br /><b>Errors were encountered while processing:</b><br /><b> linux-image-3.8.0-34-generic</b><br /><b>E: Sub-process /usr/bin/dpkg returned an error code (1)</b><br /><div><br /></div><div>You might not be able to upgrade the kernel on your VPS. Contact your VPS provider and get them to upgrade the kernel for you.</div>WHMCS Cronjob Not Running On Ubuntu2014-04-26T18:48:00+00:00https://www.paulsprogrammingnotes.com/2014/04/whmcs-cronjob-not-running-on-ubuntuMy issue was the user with the cronjob did not have access to run the cronjob php file. So, I needed to make the new cronjob for the correct user.<br /><br />Solution:<br /><br /><ol><li>Find out the owner of your apache process (usually www-data).</li><li>Look at the top of the "Automation Settings" page in your WHMCS admin area. It should say something like: php -q /var/www/whmcs/admin/cron.php (<b>keep this handy for step 4</b>)</li><li>In terminal run: <b>crontab -e -u www-data</b></li><li>Ensure you have a line that says: <br />0 6 * * * php -q /var/www/whmcs/admin/cron.php (<b>note: this should be the same file path you got for your own install in step 2</b>)</li></ol><br /><br /><br />Http Port 443 Working But Https Port 443 Not Working In CherryPy - Python2014-04-26T15:35:00+00:00https://www.paulsprogrammingnotes.com/2014/04/http-port-443-working-but-https-portI was able to visit my site using HTTP and port 443, but nothing would even show in the log when I tried to visit with HTTPS and port 443.<br /><br />This seems to only be an issue with CherryPy 3.3.0, I switched back to CherryPy 3.2.3 and it worked again.<br /><br />I noted it on their bitbucket issue here: <a href="https://bitbucket.org/cherrypy/cherrypy/issue/1298/ssl-not-working">https://bitbucket.org/cherrypy/cherrypy/issue/1298/ssl-not-working</a>OpenSSL DLL load failed On Windows - Python2014-04-26T14:07:00+00:00https://www.paulsprogrammingnotes.com/2014/04/openssl-dll-load-failed-on-windowsThe DLL load failed error happened when cherrypy tried the following import: "from OpenSSL import SSL"<br /><br />ChannelFailures: VerificationError("importing 'C:\\\\Python27\\\\lib\\\\site-packages\\\\cryptography\\\\_Cryptography_c<br />ffi_48bbf0ebx93c91939.pyd': DLL load failed: %1 is not a valid Win32 application.",)<br /><br />These error messages are solved with this install: <a href="http://slproweb.com/download/Win32OpenSSL-1_0_1g.exe">http://slproweb.com/download/Win32OpenSSL-1_0_1g.exe</a><br /><br />If you have a 32-bit version of python, that install will move the correct DLLs to your system32 folder.My First Python Library2014-04-24T16:27:00+00:00https://www.paulsprogrammingnotes.com/2014/04/my-first-python-library<a href="https://github.com/pawl/Chinese-RFID-Access-Control-Library">https://github.com/pawl/Chinese-RFID-Access-Control-Library</a><br /><br /><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">I purchased a chinese access controller and reverse engineered some functionality of the windows software that came with it. I made a python library that does some of the same functions as the desktop software: </span><u style="background-color: white; color: #1155cc; font-family: arial, sans-serif; font-size: 13px;"><a href="https://github.com/pawl/Chinese-RFID-Access-Control-Library/" style="color: #1155cc;" target="_blank">https://github.com/pawl/<wbr></wbr>Chinese-RFID-Access-Control-<wbr></wbr>Library/</a></u><br /><div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><div><br /></div><div>I use this library to tie our access controller into with our billing system (called WHMCS) and another web application (which is meant to be a self-documenting way to activate/deactivate RFID badges): <a href="https://github.com/pawl/MakerManager" style="color: #1155cc;" target="_blank">https://github.com/<wbr></wbr>pawl/MakerManager</a></div><div><br /></div><div>Now, whenever someone stops paying for 5+ days, their RFID badge is deactivated because a "hook" in WHMCS sends a request to a <a href="https://github.com/pawl/Chinese-RFID-Access-Control-Library/blob/master/examples/webserver.py" style="color: #1155cc;" target="_blank">webservice</a> that uses the access control library.</div></div>Android Bitcoin OutOfMemoryError Or Crashing On Startup2014-04-21T05:47:00+00:00https://www.paulsprogrammingnotes.com/2014/04/android-bitcoin-outofmemoryerror-orThis page explains the problem: <a href="https://code.google.com/p/bitcoin-wallet/wiki/OutOfMemory">https://code.google.com/p/bitcoin-wallet/wiki/OutOfMemory</a><br /><br />At one point, it got so bad that the application would not even start anymore.<br /><br />Solution:<br /><br /><ol><li>Get <a href="https://play.google.com/store/apps/details?id=com.speedsoftware.rootexplorer">Root Explorer</a></li><li>Navigate to your system folder and open "build.prop" in the text editor</li><li>Edit "dalvik.vm.heapsize" and "dalvik.vm.heapgrowthlimit" to the following:<br />dalvik.vm.heapsize=<b>256m</b><br />dalvik.vm.heapgrowthlimit=<b>128m</b></li><li>Restart your phone, open the bitcoin application, and backup your keys before it crashes again.</li></ol>Network Visualization Library2014-04-16T02:11:00+00:00https://www.paulsprogrammingnotes.com/2014/04/network-visualization-library<a href="http://visjs.org/">http://visjs.org/</a><br /><br />This example looks particularly awesome: <a href="http://visjs.org/examples/graph/17_network_info.html">http://visjs.org/examples/graph/17_network_info.html</a><br /><br />Lock wait timeout exceeded - MySQL2014-04-14T15:51:00+00:00https://www.paulsprogrammingnotes.com/2014/04/lock-wait-timeout-exceededOperationalError: (OperationalError) (1205, 'Lock wait timeout exceeded; try restarting transaction')<br /><b><br /></b>After trying a bunch of different solutions in the code, I ended up having to<b> restart MySQL</b> to get this error message to go away.Solution To Apache Triggering OOM-Killer2014-04-14T15:49:00+00:00https://www.paulsprogrammingnotes.com/2014/04/solution-to-apache-triggering-oom-killer<b>Update: </b>We switched to Prefork apache and have been experiencing significantly fewer problems.<br /><br />In this instance, we were getting hit with web crawlers according to the access log and the syslog was saying this: <b>/usr/sbin/apach invoked oom-killer</b><br /><br />This issue occurs when apache opens up too many child processes, uses up too much memory, then OOM killer starts shutting down random processes like mysql.<br /><br />The problem ended up being that our apache2.conf did not have any configuration limiting the number of processes that Apache could open. It might have had a default, but the default was too high. We were using Apache ITK MPM and the apache2.conf only mentioned prefork, worker, and event MPMs. You can check which MPM you are running with "apache2 -V".<br /><br />Adding this to the configuration solved the problem:<br /><IfModule mpm_itk_module><br /> StartServers 5<br /> MinSpareServers 5<br /> MaxSpareServers 10<br /> ServerLimit 75<br /> MaxClients 75<br /> MaxRequestsPerChild 5000<br /></IfModule><br /><br />At first I had the ServerLimit and MaxClients set to 150 and it seemed like apache was ignoring ServerLimit and Maxclients. However, ITK MPM creates an extra fork for each request (<a href="http://mpm-itk.sesse.net/">according to their site</a>), that's why there was 200+ processes running for my limit of 150. So, the process count is going to be 2x the limit you set.My Favorite Guide For Installing Python On Windows2014-04-05T01:47:00+00:00https://www.paulsprogrammingnotes.com/2014/04/my-favorite-guide-for-installing-python<div class="separator" style="clear: both; text-align: center;"><br /></div><a href="http://docs.python-guide.org/en/latest/starting/install/win/">http://docs.python-guide.org/en/latest/starting/install/win/</a><br /><br />If you're behind a proxy, start with this guide: <a href="https://github.com/chocolatey/chocolatey/wiki/Proxy-Settings-for-Chocolatey#using-chocolatey-behind-a-proxy-server">https://github.com/chocolatey/chocolatey/wiki/Proxy-Settings-for-Chocolatey#using-chocolatey-behind-a-proxy-server</a> (only the part starting with "Using Chocolatey behind a proxy server")Edit A Sharepoint List Page2014-04-03T18:55:00+00:00https://www.paulsprogrammingnotes.com/2014/04/edit-sharepoint-list-pageAdd the following to the end of the List page's URL: ?ToolPaneView=2Bootstrap Wizard2014-04-02T18:37:00+00:00https://www.paulsprogrammingnotes.com/2014/04/bootstrap-wizard<a href="https://github.com/amoffat/bootstrap-application-wizard">https://github.com/amoffat/bootstrap-application-wizard</a><br /><br />That looks like it will be useful someday.Override ModelView Class - Flask-Admin2014-04-01T22:59:00+00:00https://www.paulsprogrammingnotes.com/2014/04/override-class-modelview-flask-adminThis has an awesome example of how to override your base ModelView: <a href="http://code.vokor.org/vokorbb/src/04de9dcb666d2fb35513ca29c97654b6eef80648/vokorbb/admin.py?at=tip">http://code.vokor.org/vokorbb/src/04de9dcb666d2fb35513ca29c97654b6eef80648/vokorbb/admin.py?at=tip</a><br /><br />You would want to do this if you wanted to add something to all of your admin classes. Use cases:<br /><br /><ul><li>If you wanted to require login by overriding is_accessible for all of your ModelViews.</li><li>If you wanted to add an export method to all of your ModelViews.</li></ul>Flask-Admin Export CSV2014-04-01T19:16:00+00:00https://www.paulsprogrammingnotes.com/2014/04/flask-admin-export-csv<p>Edited: This was recently added to Flask-Admin as a feature.</p>
<p>You can add “can_export = True” to your ModelView to enable it.</p>
<p>More info here: <a href="http://flask-admin.readthedocs.org/en/latest/api/mod_model/#flask_admin.model.BaseModelView.can_export">http://flask-admin.readthedocs.org/en/latest/api/mod_model/#flask_admin.model.BaseModelView.can_export</a></p>
Print URL With Request Parameters/Arguments - Python Flask / Jinja22014-04-01T19:00:00+00:00https://www.paulsprogrammingnotes.com/2014/04/print-url-with-requestYou need to add the following to your jinja2 template:<br /><b>{{ request.path ~ '?' ~ request.query_string }}</b><br /><br />You can also print only the Get Variables / URL Parameters / Argument String with the following:<br /><b>print request.query_string</b><br /><br />The example is available here:<a href="http://stackoverflow.com/a/11781872/1364191"> http://stackoverflow.com/a/11781872/1364191</a>SQLalchemy Cheat Sheet2014-03-28T16:33:00+00:00https://www.paulsprogrammingnotes.com/2014/03/sqlalchemy-cheat-sheetCommon Filter Operators: <a href="http://manuelmax.tumblr.com/post/289720781/common-filter-operators-sqlalchemy">http://manuelmax.tumblr.com/post/289720781/common-filter-operators-sqlalchemy</a><br /><br />Common relationships: <a href="http://docs.sqlalchemy.org/en/latest/orm/relationships.html#basic-relational-patterns">http://docs.sqlalchemy.org/en/latest/orm/relationships.html#basic-relational-patterns</a><br /><br />If you're using Flask-SQLalchemy, their documentation serves as an awesome quick summary of how to use SQLalchemy: <a href="http://pythonhosted.org/Flask-SQLAlchemy/">http://pythonhosted.org/Flask-SQLAlchemy/</a>Use A Temporary File (StringIO) for Pandas's to_csv - Python Flask2014-03-27T22:06:00+00:00https://www.paulsprogrammingnotes.com/2014/03/use-temporary-file-stringio-for-tocsvThis link helped me use a temporary file for an excel export in flask (using pandas): <a href="https://coderwall.com/p/nwbuhq">https://coderwall.com/p/nwbuhq</a>Get Column and Row Labels In selectHandler - Google Charts / Visualization API2014-03-27T15:54:00+00:00https://www.paulsprogrammingnotes.com/2014/03/get-column-and-row-labels-inFor a more complete example of using selectHandler, see this page: <a href="https://developers.google.com/chart/interactive/docs/basic_interactivity">https://developers.google.com/chart/interactive/docs/basic_interactivity</a><br /><br />My example below get the labels from the row and the columns. I used this to send the user to another link and drill-down.<br /><br />// this is an example of a bar chart's selectHandler function<br />function selectHandler() {<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>var selection = chart.getSelection();<span class="Apple-tab-span" style="white-space: pre;"> </span> <br /><span class="Apple-tab-span" style="white-space: pre;"> </span>var item = selection[0];<br /><span class="Apple-tab-span" style="white-space: pre;"> </span><br /><span class="Apple-tab-span" style="white-space: pre;"> </span>if (item.row != null && item.column != null) {<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>var rowLabel = parseInt(data.getValue(item.row, 0));<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>var columnLabel = data.getColumnLabel(item.column);<br /> }<br />}<br />google.visualization.events.addListener(chart, 'select', selectHandler);Ordered Post Data - Python Requests2014-03-23T06:42:00+00:00https://www.paulsprogrammingnotes.com/2014/03/ordered-post-data-python-requestsHere's an example of how to send ordered data with python's Requests library, for when the order of your post request data matters:<br /><br />import requests<br />from collections import OrderedDict<br /><br /><b>payload = OrderedDict([("username", "admin"), ("pwd", "password")]) # the order matters!</b><br />r = requests.post('http://1.1.1.1/example', data=payload)<br />print r.text<br /><div><br /></div>Example of custom filters - Flask-Admin2014-03-18T18:14:00+00:00https://www.paulsprogrammingnotes.com/2014/03/example-of-custom-filters-flask-adminUpdate: I got this example added to the project: <a href="https://github.com/mrjoes/flask-admin/tree/master/examples/custom-sqla-filter">https://github.com/mrjoes/flask-admin/tree/master/examples/custom-sqla-filter</a><br /><br />This Github issue has an example of how to use custom filters in Flask-Admin: <a href="https://github.com/mrjoes/flask-admin/issues/400">https://github.com/mrjoes/flask-admin/issues/400</a><br /><br /><br />Multiple Views Based On Same Model - Flask-Admin2014-03-17T21:24:00+00:00https://www.paulsprogrammingnotes.com/2014/03/multiple-views-based-on-same-modelYou need to manually change the endpoint like MrJoes says in issue 480: <a href="https://github.com/mrjoes/flask-admin/issues/480">https://github.com/mrjoes/flask-admin/issues/480</a><br /><br />Here's an example:<br />admin.add_view(MyModelView(model.CsvTable, db.session, endpoint="duplicate_view"))<br /><div><br /></div><div>You can now visit your view by going to 0.0.0.0/admin/duplicate_view</div>Checkmark "column_formatters" - Flask-Admin2014-03-04T21:54:00+00:00https://www.paulsprogrammingnotes.com/2014/03/checkmark-columnformatters-flask-adminIn your custom ModelView, you need to add the following lines:<br />from flask.ext.admin.model.template import macro<br />column_formatters = dict(your_column=macro('render_your_column'))<br />list_template = 'list.html'<br /><br />In your custom list.html template, you need to add the following jinja2 code:<br />{% macro render_your_column(model, column) %}<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>{% if model.your_column%}<i class="icon-ok"></i>{% endif %}<br />{% endmacro %}<br /><br />If your row has a value in "your_column", then it will show as a checkmark. You need to make a new macro for each column.<br /><br /><br />Flask-Migrate2014-03-04T19:44:00+00:00https://www.paulsprogrammingnotes.com/2014/03/create-new<a href="https://github.com/miguelgrinberg/Flask-Migrate">https://github.com/miguelgrinberg/Flask-Migrate</a><br /><br />Flask-Migrate is exactly what I've been looking for when it comes to database migrations. When I create PHP applications with MVC frameworks, the changes in the database need to be done manually when the model is changed. Flask-Migrate handles changes to the database (like adding/removing columns) so your models reflect your database.<br /><br />This is a big advantage Flask has over PHP frameworks.Pip Not Working From Powershell2014-02-26T01:13:00+00:00https://www.paulsprogrammingnotes.com/2014/02/pip-not-working-from-powershellIf you have strawberry perl installed, you may get one of these errors when you try to use pip from powershell:<br /><br /><ul><li>Did not provide a command</li><li>Unknown or unsupported command 'install'</li></ul><div>This is the result of a conflict between strawberry perl's pip and python's pip. You need to remove anything related to strawberry perl from your path environmental variable and ensure you have C:\Python27\Scripts and C:\Python27\Scripts in your environmental variables.</div>Tableau's Frustrating Licensing Program2014-02-25T22:58:00+00:00https://www.paulsprogrammingnotes.com/2014/02/tableaus-frustrating-licensing-programTableau's licensing program is crazy.<br /><br />I was planning to use "Trusted Authentication" to share an embedded chart with lots of users (not all of whom have licenses). Some of those users will only look at the chart maybe once a year, and we can't justify buying them an entire license to look at a chart once.<br /><br />This page says "everyone who visits the page must be a licensed user on Tableau Server": <a href="http://onlinehelp.tableausoftware.com/v7.0/server/en-us/trusted_auth.htm">http://onlinehelp.tableausoftware.com/v7.0/server/en-us/trusted_auth.htm</a><br /><br />If you want more details on how to implement Trusted Authentication using PHP, first visit the following folder on your server: C:\Program Files\Tableau\Tableau Server\8.1\extras\embedding\php<div><br /></div><div>It has some sample code, but you need to install version 1.x of pecl_http to make it work. The guides for that are:</div><div><ul><li><a href="http://www.php.net/manual/en/http.install.php#113853">http://www.php.net/manual/en/http.install.php#113853</a></li><li><a href="http://www.mkfoster.com/2009/01/04/how-to-use-the-pecl-http-pecl_http-extension-to-make-http-requests-from-php/">http://www.mkfoster.com/2009/01/04/how-to-use-the-pecl-http-pecl_http-extension-to-make-http-requests-from-php/</a> (careful, this guide will install pecl_http 2.x which is too different to work)</li></ul></div>Possible Solution To Slow SQLalchemy + Flask Application2014-02-25T20:30:00+00:00https://www.paulsprogrammingnotes.com/2014/02/possible-solution-to-slow-sqlalchemyThis stackoverflow thread talks about why you want to use scoped_session:<br /><a href="http://stackoverflow.com/questions/6519546/scoped-sessionsession-maker-or-plain-session-maker-in-sqlalchemy">http://stackoverflow.com/questions/6519546/scoped-sessionsession-maker-or-plain-session-maker-in-sqlalchemy</a><br /><br />Basically, your web application isn't thread-safe if you're not using scoped_session. It's much better than having you application use a new thread for every instance.<br /><br />An example of scoped_session in use is available here: <a href="https://github.com/mjhea0/flask-boilerplate/blob/master/models.py">https://github.com/mjhea0/flask-boilerplate/blob/master/models.py</a>Hide Zero Value Points On Google Chart2014-02-18T19:28:00+00:00https://www.paulsprogrammingnotes.com/2014/02/hide-zero-value-points-on-google-chartI have a line chart which needed a series of points. I don't want points to show if the value is zero.<br /><br />Here's my solution:<br /><br /><ul><li>Set the series's lineWidth to zero and make the point larger than 0:<br />series: {0: {lineWidth: 0, pointSize: 5}}</li><li>Make the viewWindow setting's min higher than 0 (to hide the 0 points):<br />vAxis: { viewWindowMode:'explicit', viewWindow:{ min: 0.5 } },</li></ul>Change Home Flask-Admin2014-02-17T18:52:00+00:00https://www.paulsprogrammingnotes.com/2014/02/change-home-flask-adminThis was the only thing I was able to get working: <a href="http://www.marteinn.se/blog/?p=637">http://www.marteinn.se/blog/?p=637</a><br /><br />Just putting index.html into an admin/ directory wouldn't work.<br /><br />There's also an even shorter way to do it, for example:<br /><b>admin = Admin(app, "G-Cal Manager", index_view=AdminIndexView(name='Home', template='admin/home.html', url='/'))</b><br /><br />The example above will change the root url to / instead of /admin and it will use home.html from your templates/admin folder.<br /><br />To fix the 404 errors on your stylesheets after the change, you will also need to change the static_folder in your flask object:<br /><b>app = Flask(__name__, static_folder='admin')</b><br /><div><br /></div><div><br /></div>invalid byte sequence for encoding "UTF8" - sqlalchemy2014-01-30T09:38:00+00:00https://www.paulsprogrammingnotes.com/2014/01/invalid-byte-sequence-for-encoding-utf8Try putting ?charset=utf8 at the end of your connection string, like:<br />'mysql+mysqldb://user:pass@111.111.111.111/rf?charset=utf8'Adminer - Unable to connect to PostgreSQL server: FATAL: Peer authentication failed for user2014-01-30T07:01:00+00:00https://www.paulsprogrammingnotes.com/2014/01/adminer-unable-to-connect-to-postgresqlYou need to make sure you have the following line in your pg_hba.conf file:<br /><blockquote class="tr_bq">local all postgres trust</blockquote><div><br /></div>Clone/Copy Table Schema From One Database To Another - SQLalchemy2014-01-30T00:22:00+00:00https://www.paulsprogrammingnotes.com/2014/01/clonecopy-table-schema-from-one<p>I got the following code from slide 36 of this slideshow: <a href="http://www.slideshare.net/Stiivi/python-business-intelligence-pydata-2012-talk">http://www.slideshare.net/Stiivi/python-business-intelligence-pydata-2012-talk</a></p>
<noscript><pre>from sqlalchemy import create_engine, Table, Column, Integer, Unicode, MetaData, String, Text, update, and_, select, func, types
# create engine, reflect existing columns, and create table object for oldTable
srcEngine = create_engine('mysql+mysqldb://username:password@111.111.111.111/database') # change this for your source database
srcEngine._metadata = MetaData(bind=srcEngine)
srcEngine._metadata.reflect(srcEngine) # get columns from existing table
srcTable = Table('oldTable', srcEngine._metadata)
# create engine and table object for newTable
destEngine = create_engine('mysql+mysqldb://username:password@localhost/database') # change this for your destination database
destEngine._metadata = MetaData(bind=destEngine)
destTable = Table('newTable', destEngine._metadata)
# copy schema and create newTable from oldTable
for column in srcTable.columns:
destTable.append_column(column.copy())
destTable.create()</pre></noscript>
<script src="https://gist.github.com/9935333.js"> </script>
SQLalchemy - Print Tables In Database (Show Tables)2014-01-28T22:57:00+00:00https://www.paulsprogrammingnotes.com/2014/01/sqlalchemy-print-tables-in-databasefrom sqlalchemy import create_engine, Table, MetaData<br /><div><br /></div>engine = create_engine('mysql+mysqldb://username:password@111.111.111.111/databaseName')<br />metadata = MetaData(bind=engine)<br />metadata.reflect(engine)<br />print metadata.tables.keys()<br /><br />You could also use that for looping through a list of the tables in the database.Print Flask-Restless Routes2014-01-28T16:43:00+00:00https://www.paulsprogrammingnotes.com/2014/01/print-flask-restless-routes# show api routes<br />for route in app.url_map.iter_rules():<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>print route.rulepip install MySQLdb2014-01-27T18:04:00+00:00https://www.paulsprogrammingnotes.com/2014/01/pip-install-mysqldbIn case you haven't already noticed, the command in the title won't work.<br /><br />You need to use: <b>pip install MySQL-python</b><br /><br />This article explains the correct approach: <a href="http://mysql-python.blogspot.com/2012/11/is-mysqldb-hard-to-install.html">http://mysql-python.blogspot.com/2012/11/is-mysqldb-hard-to-install.html</a>Generate Class From Existing Table Sql-Alchemy - Python2014-01-21T15:55:00+00:00https://www.paulsprogrammingnotes.com/2014/01/generate-class-from-existing-table-sql<a href="http://stackoverflow.com/questions/6290162/how-to-automatically-reflect-database-to-sqlalchemy-declarative">http://stackoverflow.com/questions/6290162/how-to-automatically-reflect-database-to-sqlalchemy-declarative</a><br /><br />That is the key to the kingdom when it comes to using extensions like Flask-Admin and Flask-Restless on existing tables which do not have a class already.Best VNC Installation Guide2014-01-18T06:49:00+00:00https://www.paulsprogrammingnotes.com/2014/01/httpswikihttps://wiki.amahi.org/index.php/Install_VNC_server_on_Ubuntu_Server_12.04<br /><br />That guide gave me the least amount of hassle when I installed VNC on a server. I started by trying to get it working with 13.10, but had issues with a grey screen when tightvnc connected."Could not connect to database: Database connection "Mysql" is missing, or could not be created." - CakePHP2014-01-15T20:03:00+00:00https://www.paulsprogrammingnotes.com/2014/01/could-not-connect-to-database-databaseI got rid of this error just by creating the database manually. Shouldn't Cake do this?"Not Found The requested URL /app/install was not found on this server." - CakePHP2014-01-15T19:44:00+00:00https://www.paulsprogrammingnotes.com/2014/01/not-found-requested-url-appinstall-wasRather than dealing with my .htaccess problems to fix the error, I thought this would be a better approach: <a href="http://wwdj.wijndaele.com/getting-started-with-cakephp-without-mod_rewrite/">http://wwdj.wijndaele.com/getting-started-with-cakephp-without-mod_rewrite/</a><br /><br />Unfortunately, it messed up quite a few links on my site.<br /><br />I ended up solving the problem by <b>changing "AllowOverride" None to "AllowOverride All"</b> in my /etc/apache2/sites-available/default file.<br /><br />Why the heck is CakePHP using .htaccess files by default anyway? Yii has this more streamlined.CakePHP Boilerplate2014-01-15T17:27:00+00:00https://www.paulsprogrammingnotes.com/2014/01/cakephp-boilerplateWhile I was searching for a some good boilerplate to start my first CakePHP project with, I found this: <a href="https://github.com/hugodias/cakeStrap">https://github.com/hugodias/cakeStrap</a><br /><br />There's also this one, but it has much fewer commits: <a href="https://github.com/zynesis/cakephp-boilerplate">https://github.com/zynesis/cakephp-boilerplate</a>Exception in thread "main" java.lang.NoClassDefFoundError - Java2014-01-07T18:05:00+00:00https://www.paulsprogrammingnotes.com/2014/01/exception-in-thread-mainGot this error when I was missing a library that came with the build.<br /><br />If you're getting this error, you may need to move the lib folder from your build directory into the folder of your JAR file.How To Use Endpoint Parameter - Flask-Admin2014-01-03T19:01:00+00:00https://www.paulsprogrammingnotes.com/2014/01/how-to-use-endpoint-parameter-flaskYou can access the endpoint parameter by using self.endpoint.<br /><br />class displayResults(BaseView):<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>@expose('/',methods=('GET','POST'))<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>def index(self):<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>print self.endpoint<br /><br />admin.add_view(displayResults(name='test', endpoint='test2'))<br /><div><br /></div><div>That will print "test2".</div>UndefinedError: 'admin_base_template' is undefined - Flask Admin2014-01-02T17:47:00+00:00https://www.paulsprogrammingnotes.com/2014/01/undefinederror-adminbasetemplate-isThis happens when you try to render a template like this:<br />return render_template('admin/upload.html')<br /><div><br /></div><div>You need to use self.render instead:</div><div>return self.render('admin/upload.html')</div>Enable UART2 On BeagleBone Black for RS232 Cape2013-12-30T12:21:00+00:00https://www.paulsprogrammingnotes.com/2013/12/enable-uart2-on-beaglebone-black-for<br /><ol><li>Use this script to ensure UART2 is enabled in the device tree: <a href="https://github.com/victorporof/BeagleBone-SPI-UART">https://github.com/victorporof/BeagleBone-SPI-UART</a></li><li>Enable ttyO1 it in the firmware using this guide (makes the device appear): <a href="http://www.element14.com/community/blogs/mirandasoft/2013/07/30/beaglebone-black-rapid-activation-of-ttyo1-ttyo2-ttyo4-and-ttyo5">http://www.element14.com/community/blogs/mirandasoft/2013/07/30/beaglebone-black-rapid-activation-of-ttyo1-ttyo2-ttyo4-and-ttyo5</a></li><li>Put a paperclip into the RS232 port like in the picture on this page: <a href="http://sensorsblog.com/serial-communications/rs232loopbacktest/">http://sensorsblog.com/serial-communications/rs232loopbacktest/</a></li><li>echo hello > /dev/ttyO1</li><li>cat /dev/ttyO1</li></ol>Clustering With Leaflet2013-12-30T03:59:00+00:00https://www.paulsprogrammingnotes.com/2013/12/clustering-with-leaflet<a href="http://jmduke.com/blog/2013/12/28/seattle-911">http://jmduke.com/blog/2013/12/28/seattle-911</a>The Z Layout2013-12-29T23:56:00+00:00https://www.paulsprogrammingnotes.com/2013/12/the-z-layoutI'm definitely going to use this technique on my next business site: <a href="http://design.tutsplus.com/articles/understanding-the-z-layout-in-web-design--webdesign-28">http://design.tutsplus.com/articles/understanding-the-z-layout-in-web-design--webdesign-28</a>Two Column Match - Flask + HandsOnTable2013-12-29T23:37:00+00:00https://www.paulsprogrammingnotes.com/2013/12/two-column-match-flask-handsontableI'm getting more comfortable with Flask, so I decided to experiment with Google App Engine + Flask + HandsOnTable.<br /><br />Here's the application I made: <a href="http://twocolumnmatch.appspot.com/match">http://twocolumnmatch.appspot.com/match</a><br /><br />It takes the contents of the 2nd column and matches it to the rows of the first column. Just press the "Match" button to see what it does.<br /><br />I've found myself doing what this application does a few times, but I manually create two python lists and compare them in IDLE. This application streamlines that process quite a bit.<br /><br />Here's the code: <a href="https://github.com/pawl/2ColumnMatch">https://github.com/pawl/2ColumnMatch</a>SQLalchemy - Simple Select Query2013-12-26T19:10:00+00:00https://www.paulsprogrammingnotes.com/2013/12/sqlalchemy-simple-select-queryfrom sqlalchemy import create_engine, MetaData, Table<br /><br />engine = create_engine('sqlite:///cars.db', echo=False)<br />metadata = MetaData(bind=engine)<br /><br />table = Table('cars', metadata)<br />stmt = table.select()<br />for row in stmt.execute():<br /> print rowEntity-Attribute-Value - Dynamic Database Columns2013-12-26T16:35:00+00:00https://www.paulsprogrammingnotes.com/2013/12/entity-attribute-value-dynamic-databaseThis page has a great example of when to use the Entity-Attribute-Value data model: <a href="http://stackoverflow.com/questions/7933596/django-dynamic-model-fields">http://stackoverflow.com/questions/7933596/django-dynamic-model-fields</a><br /><br />I'm planning to use it for an application I'm making which has user defined fields.SHOW COLUMNS or Get Column Names From SQLalchemy SELECT Query2013-12-18T23:19:00+00:00https://www.paulsprogrammingnotes.com/2013/12/show-columns-or-get-column-names-from<div class="tr_bq"><a href="http://stackoverflow.com/questions/8947616/python-sqlalchemy-get-column-names-dynamically">http://stackoverflow.com/questions/8947616/python-sqlalchemy-get-column-names-dynamically</a></div><br />Rob Wouter's answer really helped me out:<br /><br /><blockquote>You can either find the columns by calling result.keys() or you can access them through calling v.keys() inside the for loop.<br />Here's an example using items():<br />for v in result:<br /> for column, value in v.items():<br /> print('{0}: {1}'.format(column, value))</blockquote>AttributeError: 'dict' object has no attribute '_set_parent_with_dispatch' - SQLalchemy2013-12-11T17:56:00+00:00https://www.paulsprogrammingnotes.com/2013/12/attributeerror-dict-object-has-no<br /><div class="highlight-python" style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Verdana, sans-serif; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 20.390625px; orphans: auto; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><div class="highlight" style="background-image: none; background-position: initial initial; background-repeat: initial initial;"></div></div><br /><div style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Verdana, sans-serif; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 20.390625px; orphans: auto; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">The documentation says: "Keyword arguments can be specified by specifying the last argument as a dictionary"</div><div style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Verdana, sans-serif; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 20.390625px; orphans: auto; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><br /></div><div style="-webkit-text-stroke-width: 0px; background-color: white; color: #333333; font-family: Verdana, sans-serif; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 20.390625px; orphans: auto; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">So you will need to change this:</div><div style="background-color: white;"><span style="color: #333333; font-family: Verdana, sans-serif;"><span style="font-size: 14px; line-height: 20.390625px;"><span class="Apple-tab-span" style="white-space: pre;"> </span>__table_args__ = (</span></span></div><div style="background-color: white;"><span style="color: #333333; font-family: Verdana, sans-serif;"><span style="font-size: 14px; line-height: 20.390625px;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{'sqlite_autoincrement': True},</span></span></div><div style="background-color: white;"><span style="color: #333333; font-family: Verdana, sans-serif;"><span style="font-size: 14px; line-height: 20.390625px;"> UniqueConstraint('filename', 'path')</span></span></div><div style="background-color: white;"><span style="color: #333333; font-family: Verdana, sans-serif;"><span style="font-size: 14px; line-height: 20.390625px;"> )</span></span></div><div style="background-color: white;"><span style="color: #333333; font-family: Verdana, sans-serif;"><span style="font-size: 14px; line-height: 20.390625px;"><br /></span></span></div><div style="background-color: white;"><span style="color: #333333; font-family: Verdana, sans-serif;"><span style="font-size: 14px; line-height: 20.390625px;">to this:</span></span></div><div style="background-color: white;"><span style="color: #333333; font-family: Verdana, sans-serif;"><span style="font-size: 14px; line-height: 20.390625px;"><span class="Apple-tab-span" style="white-space: pre;"> </span>__table_args__ = (</span></span><br /><span style="color: #333333; font-family: Verdana, sans-serif;"><span style="font-size: 14px; line-height: 20.390625px;"> UniqueConstraint('filename', 'path'),</span></span><br /><span style="color: #333333; font-family: Verdana, sans-serif;"><span style="font-size: 14px; line-height: 20.390625px;"> {'sqlite_autoincrement': True}</span></span><br /><span style="font-size: 14px; line-height: 20.390625px;"><span style="color: #333333; font-family: Verdana, sans-serif;"></span></span><br /><span style="color: #333333; font-family: Verdana, sans-serif;"><span style="font-size: 14px; line-height: 20.390625px;"> )</span></span></div>Flask-Admin Foreign Key Columns Allowing Editing2013-12-11T17:36:00+00:00https://www.paulsprogrammingnotes.com/2013/12/flask-admin-foreign-key-columnsMake sure you have a row id number as your primary key for your child table. Having a composite primary key didn't seem to save when I was editing the form.PC Building Site2013-12-06T06:57:00+00:00https://www.paulsprogrammingnotes.com/2013/12/pc-building-siteAwesome site for finding the best prices on PC parts: <a href="http://pcpartpicker.com/">http://pcpartpicker.com</a>jPanelMenu Causing Document.Ready() To Run Twice2013-11-27T22:57:00+00:00https://www.paulsprogrammingnotes.com/2013/11/jpanelmenu-causing-documentready-to-runThis line in jquery.jpanelmenu.js is the cause:<br />$('body > *').not(jP.menu + ', ' + jP.options.excludedPanelContent).wrapAll('<div class="' + 'jPanelMenu-panel' + '"/>');<br /><div><br /></div><div>You will need to comment it out and wrap everything in your body tag manually with:</div><div><div class="jPanelMenu-panel" style="position: relative; left: 0px;"></div><div></div></div><div><br /></div><div>The end result will look like:</div><div><body></div><div><div class="jPanelMenu-panel" style="position: relative; left: 0px;"></div><div>all your code....</div><div></div></div><div></body><br /><br /><br />This page helped me figure it out: <a href="http://doctype.com/trying-jquery-wrapall-seems-wrap-content-twice">http://doctype.com/trying-jquery-wrapall-seems-wrap-content-twice</a></div>Best cArrayDataProvider Example - Yii2013-11-18T16:40:00+00:00https://www.paulsprogrammingnotes.com/2013/11/best-carraydataprovider-example-yii<a href="http://queirozf.com/reminders/using-carraydataprovider-to-populate-a-cgridview-in-yii">http://queirozf.com/reminders/using-carraydataprovider-to-populate-a-cgridview-in-yii</a>Spreadsheet Parsing With Javascript2013-11-13T07:04:00+00:00https://www.paulsprogrammingnotes.com/2013/11/spreadsheet-parsing-with-javascript<a href="http://oss.sheetjs.com/">http://oss.sheetjs.com/</a>Responsive Email Template2013-11-08T09:18:00+00:00https://www.paulsprogrammingnotes.com/2013/11/responsive-email-templateZurb Ink - <a href="http://zurb.com/ink/templates.php">http://zurb.com/ink/templates.php</a>Catch ldap_bind(): Unable to bind to server: Invalid credentials2013-11-06T04:33:00+00:00https://www.paulsprogrammingnotes.com/2013/11/catch-ldapbind-unable-to-bind-to-serverI used the tip from this page: <a href="http://bytes.com/topic/php/answers/600903-ldap_bind-warning-instead-false">http://bytes.com/topic/php/answers/600903-ldap_bind-warning-instead-false</a><br /><br />"<span style="background-color: white; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px;">You can get around this if you supress warning and error messages by</span><br /><span style="background-color: white; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px;">putting a @ in front of the function name:</span><br style="background-color: white; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px;" /><br style="background-color: white; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px;" /><span style="background-color: white; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px;">$ldapbind = @ldap_bind($ldapconn, $ldaprdn, $ldappass);"</span>Yii Google Chart Extension2013-11-05T23:08:00+00:00https://www.paulsprogrammingnotes.com/2013/11/yii-google-chart-extensionHere's YiiWheel's google chart wrapper extension: <a href="https://github.com/2amigos/yiiwheels/blob/master/widgets/google/WhVisualizationChart.php">https://github.com/2amigos/yiiwheels/blob/master/widgets/google/WhVisualizationChart.php</a><br /><br />I downloaded just the WhVisualizationChart file and put this in my view:<br /><br />$this->widget('ext.WhVisualizationChart', array(<br /> 'visualization' => 'PieChart',<br /> 'data' => $chartDataProvider,<br /> 'options' => array(<br /> 'pieHole'=> '0.5',<br /> 'backgroundColor'=>array('fill'=>'transparent'),<br /> ),<br /> 'htmlOptions'=>array('style'=>'width:100%; height:375px'),<br />));<br /><br />and this in my controller:<br /><br />public function actionIndex()<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>{<br /> $chartDataProvider = array(<br /> array('Task', 'Hours per Day'),<br /> array('Work', 11),<br /> array('Eat', 2),<br /> array('Commute', 2),<br /> array('Watch TV', 2),<br /> array('Sleep', 7)<br /> );<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>$this->render('index', array('chartDataProvider'=>$chartDataProvider));<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br /><br />Examples on how to use it are at the bottom of this page: <a href="http://yiiwheels.2amigos.us/site/charts#visualizationchart">http://yiiwheels.2amigos.us/site/charts#visualizationchart</a>Render View From Another Controller In Yii2013-10-31T02:39:00+00:00https://www.paulsprogrammingnotes.com/2013/10/render-view-from-another-controller-in<a href="http://developwithguru.com/how-to-renderpartial-a-file-in-another-controllers-view-folder-for-yii/">http://developwithguru.com/how-to-renderpartial-a-file-in-another-controllers-view-folder-for-yii/</a><br /><a href="http://developwithguru.com/how-to-renderpartial-a-file-in-another-controllers-view-folder-for-yii/"><br /></a><?php $this->renderPartial('//library/_book', array('books'=>$books))?><br /><br />Use double slashes before the controller.Easier Monitoring Than Nagios2013-10-28T16:57:00+00:00https://www.paulsprogrammingnotes.com/2013/10/easier-monitoring-than-nagios<a href="http://mmonit.com/monit/">http://mmonit.com/monit/</a><br /><br />It was not easy at all to get Nagios to monitor mysql. This only took a minute.CSqlDataProvider Using Special Data Attribute - Yii2013-10-23T18:55:00+00:00https://www.paulsprogrammingnotes.com/2013/10/csqldataprovider-using-special-dataJust posted this on the Yii forum after I found out that using the special data attribute in a column requires a syntax like 'value'=>'$data["issueCount"]' when you're using CSqlDataProvider:<br /><a href="http://www.yiiframework.com/forum/index.php/topic/48219-special-data-attribute-syntax-in-cactivedataprovider-vs-csqldataprovider/">http://www.yiiframework.com/forum/index.php/topic/48219-special-data-attribute-syntax-in-cactivedataprovider-vs-csqldataprovider/</a><br /><br /><br />Trick For Running Python Scripts On Reboot - Ubuntu2013-10-18T08:15:00+00:00https://www.paulsprogrammingnotes.com/2013/10/trick-for-running-python-scripts-on<a href="http://www.raspberrypi-spy.co.uk/2013/07/running-a-python-script-at-boot-using-cron/">http://www.raspberrypi-spy.co.uk/2013/07/running-a-python-script-at-boot-using-cron/</a><br /><br />Here's what it looks like in crontab:<br /><span style="background-color: #f7f7f7; color: #222222; font-family: 'Courier 10 Pitch', Courier, monospace; font-size: 15px; line-height: 21px;">@reboot python /home/pi/MyScript.py &</span>Enabling SSL On Apache2013-10-17T17:40:00+00:00https://www.paulsprogrammingnotes.com/2013/10/enabling-ssl-on-apache<br /><div>This is a quick overview of the process of adding an SSL certificate to apache (for next time...):</div><ol><li>Generate your private key and CSR with: <br />openssl req -new -newkey rsa:2048 -nodes -keyout yourdomain.key -out yourdomain.csr</li><li>Request your certificate</li><li>Save requested certificate onto server in a .crt file</li><li>Download the intermediary/root cert from <a href="http://www.symantec.com/page.jsp?id=roots">http://www.symantec.com/page.jsp?id=roots</a></li><li>Make your site file in /etc/apache2/sites-enabled/ look like this:</li></ol><div><div><VirtualHost *:443></div><div><br /></div><div><br /></div><div> SSLEngine On</div><div> SSLCertificateFile /home/youruser/yourdomain.crt</div><div> SSLCertificateKeyFile /home/youruser/yourdomain.key</div><div> SSLCACertificateFile /var/www/verisign.crt</div><div><br /></div><div> ServerName www.yourdomain.com</div><div><br /></div><div> DocumentRoot /var/www/yourdomain</div></div><div><br /></div><div><br /></div><div>An explanation of each of those certificates:</div><div>SSLCertificateFile - This is the certificate you received after your request.</div><div>SSLCertificateKeyFile - This is the private key you generated in step 1.</div><div>SSLCACertificateFile - This is the intermediary cert you downloaded from verisign.</div>TbEditableColumn Inside of A TbRelationalColumn's GridView - Yii's YiiBooster Extension2013-10-16T17:23:00+00:00https://www.paulsprogrammingnotes.com/2013/10/tbeditablecolumn-inside-of<a href="https://github.com/clevertech/YiiBooster/issues/703">https://github.com/clevertech/YiiBooster/issues/703</a><br /><br />Just posted an issue on github with instructions for how to get TbEditableColumn working inside of one of YiiBooster's subgrids.Yii quickdlgs Without UrlManager Enabled2013-10-11T19:51:00+00:00https://www.paulsprogrammingnotes.com/2013/10/yii-quickdlgs-without-urlmanager-enabledEQuickDlgs::iframeButton(<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>array(<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>'controllerRoute' => 'controller/create',<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>'dialogTitle' => 'Create New Record',<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>'dialogWidth' => 800,<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>'dialogHeight' => 600,<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>'openButtonText' => 'Add Record',<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>'closeButtonText' => 'Close',<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>//'openButtonHtmlOptions' => array(...),<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>)<br />);<br /><br /><br />With the code above, I was getting the following error:<br />The system is unable to find the requested action "create?qdsClass=EFrameJuiDlg".<br /><br />I had to change line 79 in EFrameJuiDlg.php inside the quickdlgs extension from:<br /> $url .= '?' . http_build_query($this->urlParams);<br /><div>to:</div><div><div> $url .= '&' . http_build_query($this->urlParams);</div></div><div><br /></div>htmlspecialchars() expects parameter 1 - Yii (echmultiselect)2013-10-09T05:38:00+00:00https://www.paulsprogrammingnotes.com/2013/10/htmlspecialchars-expects-parameter-1<blockquote class="tr_bq">htmlspecialchars() expects parameter 1 to be string, array given C:\xampp\htdocs\yii\framework\web\helpers\CHtml.php(98) </blockquote>The error only occurred when trying to apply the echmultiselect filter to the filter above the gridview.<br /><br />Here's what I did to fix:<br />1. Went to the Yii framework folder and found the following file: yii\framework\web\helpers\CHtml.php<br /><br />2. Modifed line 98 from "return htmlspecialchars($text,ENT_QUOTES,Yii::app()->charset);" to:<br />"return htmlspecialchars(print_r($text, true),ENT_QUOTES,Yii::app()->charset);"<br /><br />Then you need to implement the echmultiselect extension in the view (my example shows adding it for site_status):<br /><br />3. Added this php code to the view: $data= CHtml::listData(Nvdaily::model()->findAll(), 'blahblahblah', 'blahblahblah');<br />(but it should probably be in the model)<br /><br />4. Made my 'columns' array look like this:<br />'columns'=>array(<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>'id',<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>'blah',<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>'blahblah',<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>array (<br /> 'name'=>'blahblahblah',<br /> 'filter'=> $this->widget('ext.EchMultiselect.EchMultiselect',<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>array(<br /> 'model' => $model,<br /> 'dropDownAttribute' => 'blahblahblah',<br /> 'data' => $data,<br /> 'options' => array('buttonWidth' => 80, 'ajaxRefresh' => true),<br /> ),<br /> true // capture output; needed so the widget displays inside the grid<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>),<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>),<br />array(<br />'class'=>'bootstrap.widgets.TbButtonColumn',<br />),<br />),SQL Formatter2013-10-07T06:26:00+00:00https://www.paulsprogrammingnotes.com/2013/10/sql-formatterI had no idea this existed until now: <a href="http://www.dpriver.com/pp/sqlformat.htm">http://www.dpriver.com/pp/sqlformat.htm</a><br /><br />It makes your SQL queries more readable.<br /><br />I learned about it from this thread: <a href="http://www.reddit.com/r/PHP/comments/1n8kgz/how_do_you_write_readable_sql_queries/">http://www.reddit.com/r/PHP/comments/1n8kgz/how_do_you_write_readable_sql_queries/</a>Error Handling In mysqldb - Python2013-10-06T22:56:00+00:00https://www.paulsprogrammingnotes.com/2013/10/error-handling-in-mysqldb-pythonI've had issues with queries failing and leaving connections open (enough to stall a server...). I know python's oursql library supports using the WITH keyword, and I think it will close the connection when there is an unexpected error. However, I'm not sure if I'm ready to move to a different library for MySQL (it's working well).<br /><br />Here's what I'm currently doing to close the cursor and connection, then re-raise the error:<br /><br />import MySQLdb<br /><br />conn = MySQLdb.connect(user="username", passwd="secret", db="database", charset='utf8')<br />cur = conn.cursor()<br /><br />try:<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>cur.execute("INSERT INTO testTable (userid) VALUES(%s);" % id)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>conn.commit()<br />except:<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>cur.close()<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>conn.close()<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>raise<br />finally:<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>print "Insert Successful"haufe.sharepoint Python Library - URLs With %202013-10-04T19:06:00+00:00https://www.paulsprogrammingnotes.com/2013/10/haufesharepoint-python-library-urlsI changed line 25 of the haufe.sharepoint library to fix an issue I was having with URLs which had %20 (the url encoded space character):<br /><br />New Code:<br />location = urllib.quote(self.location(), safe=":/")<br /><br />Old Code:<br />location = self.location()<br /><br />Just try using a URL with %20 with haufe.sharepoint to see what I'm talking about.urllib2.URLError Python SUDS library2013-10-03T21:51:00+00:00https://www.paulsprogrammingnotes.com/2013/10/urllib2urlerror-python-suds-library<div>To figure this out, you're going to need logging. Turn on logging with the following:</div><div><br /></div><div><a href="http://stackoverflow.com/questions/2388046/can-you-help-me-solve-this-suds-soap-issue">http://stackoverflow.com/questions/2388046/can-you-help-me-solve-this-suds-soap-issue</a></div><span style="color: darkblue;"><div><span style="color: darkblue;"><br /></span></div>import logging logging.basicConfig(level=logging.INFO) logging.getLogger('suds.client').setLevel(logging.DEBUG) logging.getLogger('suds.transport').setLevel(logging.DEBUG) logging.getLogger('suds.xsd.schema').setLevel(logging.DEBUG) logging.getLogger('suds.wsdl').setLevel(logging.DEBUG)</span>pip 1.3 Not Working Behind Proxy2013-10-03T19:00:00+00:00https://www.paulsprogrammingnotes.com/2013/10/pip-13-not-working-behind-proxy<a href="http://pythontesting.net/python/pip/">http://pythontesting.net/python/pip/</a><br /><br />Like this article says, it's fixed in the dev version of pip.Bootstrap Starter Templates2013-10-03T07:51:00+00:00https://www.paulsprogrammingnotes.com/2013/10/bootstrap-starter-templates<a href="http://startbootstrap.com/all-templates">http://startbootstrap.com/all-templates</a>Installing Nagios2013-10-02T21:16:00+00:00https://www.paulsprogrammingnotes.com/2013/10/installing-nagiosI had a heck of a time finding the config file for nagios. It's under: /etc/nagios3/conf.d/localhost_nagios2.cfg<br /><br />These guides were helpful:<br /><a href="http://ostatic.com/blog/monitoring-web-services-with-nagios">http://ostatic.com/blog/monitoring-web-services-with-nagios</a><br /><a href="https://library.linode.com/server-monitoring/nagios/ubuntu-12.04-precise-pangolin">https://library.linode.com/server-monitoring/nagios/ubuntu-12.04-precise-pangolin</a>MySQL Not Starting On Reboot2013-10-01T21:07:00+00:00https://www.paulsprogrammingnotes.com/2013/10/mysql-not-starting-on-reboot<a href="http://stackoverflow.com/questions/9859381/how-make-mysql-start-automatically-linux-cli-only">http://stackoverflow.com/questions/9859381/how-make-mysql-start-automatically-linux-cli-only</a><br /><br />I was able to to the following, based on the post above:<br /><br /><ol><li><span style="font-family: 'Lucida Console'; font-size: 10pt;">sudo apt-get install sysv-rc-conf</span></li><li><span style="font-family: 'Lucida Console'; font-size: 10pt;">sudo sysv-rc-conf</span></li><li><span style="font-family: 'Lucida Console'; font-size: 10pt;">mark the 6 column for mysqld or mysql.server (6 = reboot)</span></li></ol><div><span style="font-family: Lucida Console; font-size: x-small;"><br /></span></div><div><span style="font-family: Lucida Console; font-size: x-small;">Note: I tried sudo update-rc.d mysql.server defaults and it didn't work. It gave me the following error: System start/stop links for /etc/init.d/mysql.server already exist.</span></div>Fira Font2013-09-25T06:28:00+00:00https://www.paulsprogrammingnotes.com/2013/09/fira-font<a href="https://github.com/mozilla/Fira">https://github.com/mozilla/Fira</a><br /><br />Mozilla's new Fira font looks great. Looking forward to use it on some future projects.MySQL IF Statement Affect On Speed2013-09-19T17:28:00+00:00https://www.paulsprogrammingnotes.com/2013/09/mysql-if-statement-affect-on-speed<span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">if(`table`.`start`='0000-00-00','',`table`.`start`)</span><br /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"><br /></span><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">That's the part of my query that was increasing the execution time by 10x. </span><br /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;"><br /></span><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">I changed it to just </span><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 13px; line-height: 16px;">`table`.`start` (without the IF statement) and the query was 10x faster.</span>Phonegap Start (for Phonegap Build)2013-09-18T04:24:00+00:00https://www.paulsprogrammingnotes.com/2013/09/phonegap-start<div>"<span style="color: #777777; font-family: Helvetica, arial, freesans, clean, sans-serif; font-size: 15px; line-height: 25px;">A Hello World application to get started with PhoneGap Build."</span></div><div><a href="https://github.com/phonegap/phonegap-start">https://github.com/phonegap/phonegap-start</a></div><br />Twilio PHP Library Using A Proxy2013-09-17T20:48:00+00:00https://www.paulsprogrammingnotes.com/2013/09/twilio-php-library-using-proxy<a href="https://github.com/twilio/twilio-php/issues/144">https://github.com/twilio/twilio-php/issues/144</a><br /><br />I found a spot where I can add a proxy to the twilio PHP library.<br /><br />Go to <span style="background-color: #fbfbfb; color: #333333; font-family: Helvetica, arial, freesans, clean, sans-serif; font-size: 13px; line-height: 22px;">TinyHttp.php and add the highlighted part (using your own URL of course):</span><br /><pre style="border-bottom-left-radius: 3px; border-bottom-right-radius: 3px; border-top-left-radius: 3px; border-top-right-radius: 3px; border: 1px solid rgb(221, 221, 221); color: #333333; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 13px; line-height: 19px; margin-bottom: 15px; margin-top: 15px; overflow: auto; padding: 6px 10px;"><code style="background-color: yellow; border-bottom-left-radius: 3px; border-bottom-right-radius: 3px; border-top-left-radius: 3px; border-top-right-radius: 3px; border: none; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; margin: 0px; padding: 0px;">$opts[CURLOPT_PROXY] = 'http://xxx.xxx.xxx.xxx:8080/';</code><code style="background-color: transparent; border-bottom-left-radius: 3px; border-bottom-right-radius: 3px; border-top-left-radius: 3px; border-top-right-radius: 3px; border: none; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; margin: 0px; padding: 0px;"><br />try {<br /> if ($curl = curl_init()) {</code></pre>Failed to connect to socket /var/run/dbus/system_bus_socket2013-09-17T15:36:00+00:00https://www.paulsprogrammingnotes.com/2013/09/failed-to-connect-to-socketError: Can't find out if NetworkManager is running: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory.<br />The following fixed the problem:<br />apt-get install dbus<br />(if you already have dbus installed: sudo service dbus start)<br />/etc/init.d/network-manager start<br /><br /><br />What caused the error?:<br />/etc/init.d/network restartCould not initialize NMClient 2013-09-16T23:52:00+00:00https://www.paulsprogrammingnotes.com/2013/09/could-not-initialize-nmclient** (process:17986): WARNING **: error: could not connect to NetworkManager<br /><br /><br />Fix: Try running with sudo.<br /><br />For example: sudo nm-tool | grep DNSCheck If Script Is already Running - Python2013-09-08T18:36:00+00:00https://www.paulsprogrammingnotes.com/2013/09/check-if-script-is-already-running<a href="http://pythonhosted.org/tendo/">http://pythonhosted.org/tendo/</a><br /><br /><br />import tendo<br /><br /># If is there another instance already running:<br /># display the message “Another instance is already running, quitting.”, and return -1 error code.<br />me = SingleInstance()<br /><br />Fighting WinSCP Time-Outs2013-09-03T17:14:00+00:00https://www.paulsprogrammingnotes.com/2013/09/fighting-winscp-time-outs<a href="http://www.lampdocs.com/blog/2008/10/winscp-how-to-prevent-timeouts/">http://www.lampdocs.com/blog/2008/10/winscp-how-to-prevent-timeouts/</a><br /><br />That's a guide to making WinSCP send a dummy command every 30 seconds to keep your connection alive. It's way better than constantly having to type in your password.Crontab Mistake2013-09-03T16:20:00+00:00https://www.paulsprogrammingnotes.com/2013/09/crontab-mistakeI wanted to run a script every 20 minutes, so I put this into my crontab:<br />20 * * * * sh script.sh<br /><br />That's wrong. That only runs it once every hour 20 minutes into the hour. For example, "20,40 * * * * sh bash.sh" would run it twice - 20 and 40 minutes into the hour.<br /><br />To run the script every 20 minutes like I was expecting:<br />*/20 * * * * sh script.shInclude yiiGridView Javascript 2013-09-02T20:04:00+00:00https://www.paulsprogrammingnotes.com/2013/09/include-yiigridview-javascript<a href="http://chevronscode.com/yii-include-preload-yiigridview-javascript/">http://chevronscode.com/yii-include-preload-yiigridview-javascript/</a><br /><br />I ended up with the following code at the top of the page (since I'm not using a layout):<br /><br />Yii::app()->clientScript->registerCoreScript('jquery');<br />$assetsScriptUrl = Yii::app()->getAssetManager()->publish(Yii::getPathOfAlias('zii.widgets.assets'));<br />Yii::app()->getClientScript()->registerScriptFile($assetsScriptUrl .'/gridview/jquery.yiigridview.js');PHP Best Practices2013-08-20T06:52:00+00:00https://www.paulsprogrammingnotes.com/2013/08/php-best-practices<a href="https://phpbestpractices.org/">https://phpbestpractices.org/</a><br /><br />Things I learned:<br /><br /><ul><li>It's easy to install PHP-APC for quick opcode caching, and it apparently serves the same purpose as memcached for a single server.</li><li>Memcache and Memcached are two different libraries.</li><li>There's a different (a small one) in speed between single and double quotes.</li><li>Use phpass to store passwords.</li></ul>Set Up Wordpress For FTP2013-08-16T22:19:00+00:00https://www.paulsprogrammingnotes.com/2013/08/set-up-wordpress-for-ftpIf it's asking you for FTP login information when you try to update wordpress or a plugin, you don't need to do that! Chances are you don't have permissions correctly configured on that directory. Make sure apache owns the directory. On ubuntu, this will probably work (depending on which user owns apache):<br /><br />sudo chown -R www-data:www-data /var/www<br /><div><br /></div>Add Multiple Unique Indexes - MySQL2013-08-15T01:35:00+00:00https://www.paulsprogrammingnotes.com/2013/08/add-multiple-unique-indexes-mysqlalter table users add unique index(id, name, phone);Bootstrap Span Is Overlapping2013-08-15T01:21:00+00:00https://www.paulsprogrammingnotes.com/2013/08/bootstrap-span-is-overlappingTry setting a static width on the span. This ensures responsive knows when to put it on another line. It solved my issues with the spans overlapping.<br /><br />example: add style="width: 520px" to your div with class="span6"<br /><br /><br />mysql-client 5.6 or phpmyadmin for mysql 5.62013-08-15T00:02:00+00:00https://www.paulsprogrammingnotes.com/2013/08/mysql-client-56-or-phpmyadmin-for-mysqlMysql-client 5.6 doesn't seem to exist, and you don't need it. Phpmyadmin installs mysql-client 5.5 automatically, and that works with mysql server 5.6. If you're having issues, you probably need to run "sudo dpkg-reconfigure phpmyadmin".Turn Off MySQL Strict Mode2013-08-14T23:59:00+00:00https://www.paulsprogrammingnotes.com/2013/08/turn-off-mysql-strict-mode<a href="http://nickbartlett.com/wordpress/how-to-turn-off-mysql-strict-mode/">http://nickbartlett.com/wordpress/how-to-turn-off-mysql-strict-mode/</a><br /><br />SET @@global.sql_mode= ”; will set it temporarily. (to see if it will fix your problem)<div><br /></div><div>To turn it off permanently, add this setting to your my.cnf (probably in /etc/mysql/my.cnf if you're using ubuntu): </div><div>[mysqld]<div>sql-mode =</div><div><br /></div><div><br /></div><div>If you're having issues getting this setting to work in your my.cnf, look for another my.cnf in /opt/mysql/server-5.6/my.cnf or elsewhere on your system using the "find / -name 'my.cnf'" command.</div></div>Using Dictionaries In Python's cursor.execute2013-08-09T17:05:00+00:00https://www.paulsprogrammingnotes.com/2013/08/using-dictionaries-in-pythonshttp://stackoverflow.com/questions/10983616/parameterized-queries-in-oursqlpymssql SELECT Always Returns None - Python2013-08-07T18:54:00+00:00https://www.paulsprogrammingnotes.com/2013/08/pymssql-select-always-returns-noneapt-get remove python-pymssql<br /><br />The version from apt-get has this error. Just use easy_install to install pymssql and select statements will work.error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 12013-08-07T18:49:00+00:00https://www.paulsprogrammingnotes.com/2013/08/error-setup-script-exited-with-errorsudo apt-get install python-dev<br /><div><br /></div><div>Running that command on ubuntu will fix the issue.<br /><br />If you're installing lxml:<br /><br /><ul><li><a href="http://stackoverflow.com/questions/16149613/installing-lxml-with-pip-in-virtualenv-ubuntu-12-10-error-command-gcc-failed">http://stackoverflow.com/questions/16149613/installing-lxml-with-pip-in-virtualenv-ubuntu-12-10-error-command-gcc-failed</a></li><li><a href="http://www.isnull.com.ar/2011/10/ubuntu-11-fatal-error-libxmlxmlversionh.html">http://www.isnull.com.ar/2011/10/ubuntu-11-fatal-error-libxmlxmlversionh.html</a></li></ul><div><br /></div></div><div>Also try this:</div><div>sudo apt-get install libxslt1-dev libxslt1.1 libxml2-dev libxml2 libssl-dev</div>Defcon Ace Badge2013-08-02T17:10:00+00:00https://www.paulsprogrammingnotes.com/2013/08/defcon-ace-badge<p><img src="/generated/assets/images/IMG_20130802_100539_934-800-26caba9be.jpg" srcset="/generated/assets/images/IMG_20130802_100539_934-400-26caba9be.jpg 400w, /generated/assets/images/IMG_20130802_100539_934-600-26caba9be.jpg 600w, /generated/assets/images/IMG_20130802_100539_934-800-26caba9be.jpg 800w, /generated/assets/images/IMG_20130802_100539_934-1000-26caba9be.jpg 1000w" /></p>
<p><img src="/generated/assets/images/IMG_20130802_100554_625-800-d22ea8474.jpg" srcset="/generated/assets/images/IMG_20130802_100554_625-400-d22ea8474.jpg 400w, /generated/assets/images/IMG_20130802_100554_625-600-d22ea8474.jpg 600w, /generated/assets/images/IMG_20130802_100554_625-800-d22ea8474.jpg 800w, /generated/assets/images/IMG_20130802_100554_625-1000-d22ea8474.jpg 1000w" /></p>
WHMCS Hook - Trigger E-mail On New User Sign-Up2013-07-28T19:48:00+00:00https://www.paulsprogrammingnotes.com/2013/07/whmcs-hook-trigger-e-mail-on-new-userThis script needs to go into your WHMCS hooks folder (root/includes/hooks/):<br /><br /><?php<br /><br /> function send_invitiation_email($vars) {<br /><br /> $command = "sendemail";<br /><span class="Apple-tab-span" style="white-space: pre;"> </span> $adminuser = "admin"; // Your admin username<br /><span class="Apple-tab-span" style="white-space: pre;"> </span> $values["messagename"] = "Google Groups Invitation"; // exact name of e-mail template<br /><span class="Apple-tab-span" style="white-space: pre;"> </span> $values["id"] = $vars['userid'];<br /><span class="Apple-tab-span" style="white-space: pre;"> </span><br /><span class="Apple-tab-span" style="white-space: pre;"> </span> $results = localAPI($command,$values,$adminuser);<br /><br /> }<br /><br /> add_hook("ClientAdd",1,"send_invitiation_email");<br /><br /> ?>Bootstrap Form Builder2013-07-24T21:53:00+00:00https://www.paulsprogrammingnotes.com/2013/07/bootstrap-form-builder<a href="http://minikomi.github.io/Bootstrap-Form-Builder/">http://minikomi.github.io/Bootstrap-Form-Builder/</a><br /><br />This looks great for non-technical users who want a form built. It's a great response to "Hey, make me a checklist!"...<br /><br />"You make the checklist the way you want it using this site, then I'll put it on the web."<br /><br /><br />PHPmyAdmin Loading Slowly (or not at all)2013-07-23T19:04:00+00:00https://www.paulsprogrammingnotes.com/2013/07/phpmyadmin-loading-slowly-or-not-at-all<a href="http://future500.nl/phpmyadmin-slow-on-startup/">http://future500.nl/phpmyadmin-slow-on-startup/</a><br /><br />UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in position 12: ordinal not in range(128)2013-07-19T19:49:00+00:00https://www.paulsprogrammingnotes.com/2013/07/unicodeencodeerror-ascii-codec-cant<p>The error in the title will occur on pre-0.12.0 versions of python’s Requests library. It seems to be fixed on later versions and maybe even earlier.</p>
<p>Here’s an example of how you save a zip file from behind a corporate proxy with python’s Requests library:</p>
<noscript><pre>sess = requests.session()
# you may have other get/post requests before you can get to the download page for the file
url = 'https://example.com/report.zip'
# this may be different for you, i just copied this from an existing request (using fiddler)
file_headers = {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language': 'en-US,en;q=0.8',
'User-Agent': 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; Trident/5.0)',
'Host': 'example.com',
'Accept-Encoding': 'gzip,deflate,sdch',
'Connection': 'Keep-Alive',
'Content-Type': 'application/octet-stream;charset=text/html;charset=UTF-8',
}
with open('report.zip', mode='wb') as handle:
report_file = sess.get(url, headers=file_headers, prefetch=False, verify=False)
handle.write(report_file.content)</pre></noscript>
<script src="https://gist.github.com/65d525f3d9defcb34870ad8a815a0b09.js"> </script>
<p><strong>That’s it!</strong> Just use the write function with the content of the returned request.</p>
requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed2013-07-19T18:55:00+00:00https://www.paulsprogrammingnotes.com/2013/07/requestsexceptionssslerror-errno-1This happens with the Requests library in python. All you need to do is set verify=false.<br /><br />For example:<br />cookieRequest = s.post(url, data=json.dumps(payload).strip('"'), allow_redirects=False, headers=headers, verify=False)<br /><div><br /></div>src/pycurl.c:42:20: fatal error: Python.h: No such file or directory2013-07-19T17:14:00+00:00https://www.paulsprogrammingnotes.com/2013/07/srcpycurlc4220-fatal-error-pythonh-noThe following helped to resolve this issue: <span class="pln" style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin: 0px; padding: 0px; vertical-align: baseline;">sudo apt</span><span class="pun" style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin: 0px; padding: 0px; vertical-align: baseline;">-</span><span class="kwd" style="border: 0px; color: darkblue; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin: 0px; padding: 0px; vertical-align: baseline;">get</span><span class="pln" style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin: 0px; padding: 0px; vertical-align: baseline;"> install python</span><span class="pun" style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin: 0px; padding: 0px; vertical-align: baseline;">-</span><span class="pln" style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin: 0px; padding: 0px; vertical-align: baseline;">dev</span>Writing To File - UnicodeEncodeError: 'ascii' codec can't encode characters2013-07-17T18:10:00+00:00https://www.paulsprogrammingnotes.com/2013/07/i-was-getting-lot-of-errors-when-i-was<p>I was getting a lot of errors when I was trying to download a file with the python requests library. The errors looked like the following:</p>
<p><strong>UnicodeEncodeError: ‘ascii’ codec can’t encode characters in position 2-6: ordinal not in range(128)</strong></p>
<p>This code ended up fixing it:</p>
<noscript><pre>import codecs
import requests
import time
# this could be different, but it works for my application, but the content-type definitely matters
fileHeaders = {
'Content-Type': 'application/octet-stream;charset=text/html;charset=UTF-8'
}
url = 'www.yourfile.com/your.zip'
with codecs.open('your.zip', mode='wb', encoding='utf-8') as handle:
zip_file = requests.get(url, headers=fileHeaders, prefetch=False)
for block in zip_file.iter_content(1024):
if block:
handle.write(block)
</pre></noscript>
<script src="https://gist.github.com/9cac6e18fd888063141d15aac15fc253.js"> </script>
Prevent URL Encoding - Python Requests Library2013-07-15T23:48:00+00:00https://www.paulsprogrammingnotes.com/2013/07/prevent-url-encoding-python-requests<br />Use json.dumps in the data parameter.<br /><br />for example:<br />r = s.post(url, data=json.dumps(payload), allow_redirects=False, cookies=cookieRequest.cookies)<br /><div><br /></div>Google Maps Event Firing Order2013-07-11T19:49:00+00:00https://www.paulsprogrammingnotes.com/2013/07/google-maps-event-firing-order<a href="http://gmaps-samples-v3.googlecode.com/svn/trunk/map_events/map_events.html">http://gmaps-samples-v3.googlecode.com/svn/trunk/map_events/map_events.html</a>Tunnel Traffic Through Webserver Using Your Browser2013-07-10T05:56:00+00:00https://www.paulsprogrammingnotes.com/2013/07/tunnel-traffic-through-webserver-using<p>Use the following command in Cygwin (using your IP and Username, of course):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ssh yourUsername@198.168.xxx.xxx -D 2280 -i your/certificate
</code></pre></div></div>
<p>Set the following settings in Firefox (connect settings, under the advanced tab):</p>
<p><img src="/assets/images/proxySettings.png" alt="proxy settings" /></p>
Dream Factory2013-07-10T05:51:00+00:00https://www.paulsprogrammingnotes.com/2013/07/dream-factory<a href="https://www.dreamfactory.com/">https://www.dreamfactory.com/</a><br /><br />It's suspiciously free. It's a ready to go back-end that can be accessed with APIs.Secure MySQL Command2013-07-03T16:02:00+00:00https://www.paulsprogrammingnotes.com/2013/07/secure-mysql-command<span style="background-color: #eeeeff; color: #666666; font-family: 'Lucida Grande', 'Lucida Sans Unicode', Verdana, sans-serif; font-size: 12px; line-height: 18px;">/usr/bin/mysql_secure_installation</span>Easyphp Change Directory To Github2013-07-02T21:01:00+00:00https://www.paulsprogrammingnotes.com/2013/07/easyphp-change-directory-to-github1. Right click on the easyphp icon --> configuration --> apache (this should open the httpd.conf in your easyphp apache directory)<br /><br />2. Find the section labelled DocumentRootDirectory and overwrite with this:<br /><br /># DocumentRootDirectory<br /><Directory "C:/Users/<span style="background-color: yellow;">your_username</span>/Documents/GitHub/<span style="background-color: yellow;">your_project</span>/"><br /> #<br /> # Possible values for the Options directive are "None", "All",<br /> # or any combination of:<br /> # Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews<br /> #<br /> # Note that "MultiViews" must be named *explicitly* --- "Options All"<br /> # doesn't give it to you.<br /> #<br /> # The Options directive is both complicated and important. Please see<br /> # http://httpd.apache.org/docs/2.4/mod/core.html#options<br /> # for more information.<br /> #<br /> Options Indexes FollowSymLinks Includes ExecCGI<br /><br /> #<br /> # AllowOverride controls what directives may be placed in .htaccess files.<br /> # It can be "All", "None", or any combination of the keywords:<br /> # Options FileInfo AuthConfig Limit<br /> #<br /> AllowOverride All<br /><br /> #<br /> # Controls who can get stuff from this server.<br /> #<br /> Require all granted<br /><br /></Directory>Setting Up Github Windows Client Behind a Proxy2013-07-02T20:59:00+00:00https://www.paulsprogrammingnotes.com/2013/07/setting-up-github-windows-client-behind<div>1. Navigate to your git.exe directory (C:\Users\your_username\AppData\Local\GitHub\PortableGit~\bin)<br /><br /><div id="Normalcontent" style="clear: both; position: relative; width: 100%;" xmlns:convitem="http://schemas.microsoft.com/2008/10/sip/convItems" xmlns:msxsl="urn:schemas-microsoft-com:xslt" xmlns:rtc="urn:microsoft-rtc-xslt-functions" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns="http://schemas.microsoft.com/2008/10/sip/convItems">2. git config --global http.proxy http://username:password@host:port/<br /><br /></div><div id="Normalcontent" style="clear: both; position: relative; width: 100%;" xmlns:convitem="http://schemas.microsoft.com/2008/10/sip/convItems" xmlns:msxsl="urn:schemas-microsoft-com:xslt" xmlns:rtc="urn:microsoft-rtc-xslt-functions" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns="http://schemas.microsoft.com/2008/10/sip/convItems">3. git config --system http.sslcainfo /bin/curl-ca-bundle.crt<br /><br /></div></div><div id="Normalcontent" style="clear: both; position: relative; width: 100%;" xmlns:convitem="http://schemas.microsoft.com/2008/10/sip/convItems" xmlns:msxsl="urn:schemas-microsoft-com:xslt" xmlns:rtc="urn:microsoft-rtc-xslt-functions" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns="http://schemas.microsoft.com/2008/10/sip/convItems">4. Open the github client (it should work now)</div>move_uploaded_file To Current Path - PHP2013-06-29T23:23:00+00:00https://www.paulsprogrammingnotes.com/2013/06/moveuploadedfile-to-current-path-php$uploadfile = './' . basename($_FILES['file']['name']);<br /><br />if (move_uploaded_file($_FILES['file']['tmp_name'], $uploadfile)) {<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>echo "File was successfully uploaded.\n";<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>} else {<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>echo "There was an issue uploading the file.\n";<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>}View In Use and Will Not DROP - MySQL2013-06-29T03:16:00+00:00https://www.paulsprogrammingnotes.com/2013/06/view-in-use-and-will-not-drop-mysqlI had several views in my MySQL database that were showing "In Use" and phpmyadmin would not let me drop them through the interface.<br /><br />The solution:<br />Open the MySQL in terminal and "DROP VIEW yourView;"Include Array From File - PHP2013-06-29T03:04:00+00:00https://www.paulsprogrammingnotes.com/2013/06/include-array-from-file-phpfile.php<br /><?php<br />return array('item1','item2');<br />?><br /><br />Your php code:<br /><br /><?php<br />$thisVariable = include('file.php');<br />print_r($thisVariable); //prints the array<br />?>Fixing "Copying To Temp Table" - MySQL2013-06-17T07:54:00+00:00https://www.paulsprogrammingnotes.com/2013/06/fixing-copying-to-temp-table-mysql<a href="http://themanbehindthecode.com/2011/08/12/avoid-mysql-copying-to-tmp-table/">http://themanbehindthecode.com/2011/08/12/avoid-mysql-copying-to-tmp-table/</a>Don't Use Nested IFNULL, Use COALESCE - MySQL2013-06-16T22:53:00+00:00https://www.paulsprogrammingnotes.com/2013/06/dont-use-nested-ifnull-use-coalesceIf you're looking for how to do a nested IFNULL, you probably need to be looking at how to use COALESCE instead. It returns the first non-null item in a list.<div><br /></div><div>Example:</div><div>SELECT COALESCE(NULL,1);</div><div><br /></div><div>Returns 1</div>IFNULL - MySQL2013-06-16T22:41:00+00:00https://www.paulsprogrammingnotes.com/2013/06/ifnull-mysqlIFNULL(`field1`, `field2`)<br /><br />That returns field1 if it's not null. However, it will return field2 if it is null.<br /><div><br /></div>Notes On JOIN - MySQL2013-06-16T21:43:00+00:00https://www.paulsprogrammingnotes.com/2013/06/join-returning-duplicate-values<br /><ul><li>If your INNER join is returning too few values and duplicates, you're probably using the wrong kind of join. Try a LEFT join.</li><li>INNER JOIN is the same as JOIN</li><li><a href="http://i.stack.imgur.com/GbJ7N.png">http://i.stack.imgur.com/GbJ7N.png</a> (diagram explaining joins)</li><li><span style="background-color: white; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 14px; line-height: 18px;">LEFT JOIN is used - this will return ALL rows from Table1, regardless of whether or not there is a matching row in Table2. </span></li></ul>Indexes Help... A Lot2013-06-16T15:54:00+00:00https://www.paulsprogrammingnotes.com/2013/06/indexs-help-lotI just brought 4 queries from 15 seconds to less than 1 second by adding indexes to the columns in the ON() portion on the join.<br /><br />Indexes help... A lot.Yii-Booster Incorrect Width2013-06-15T20:38:00+00:00https://www.paulsprogrammingnotes.com/2013/06/yii-booster-incorrect-widthSet public $responsiveCss = false; to public $responsiveCss = true;group_concat - MySQL2013-06-13T19:38:00+00:00https://www.paulsprogrammingnotes.com/2013/06/groupconcat-mysql<a href="http://mahmudahsan.wordpress.com/2008/08/27/mysql-the-group_concat-function/">http://mahmudahsan.wordpress.com/2008/08/27/mysql-the-group_concat-function/</a><br /><br />This seems like it will be super useful.<br /><br />If you're using "group by" and you still want to get data out of the grouped rows, then you can use this to combine multiple rows into a single row.Looping Through Associative Array - PHP2013-06-13T01:32:00+00:00https://www.paulsprogrammingnotes.com/2013/06/looping-through-associative-array-php<a href="http://stackoverflow.com/questions/1951690/php-how-to-loop-through-a-associative-array-and-get-the-key-name">http://stackoverflow.com/questions/1951690/php-how-to-loop-through-a-associative-array-and-get-the-key-name</a><br /><br />This was totally not very intuitive. I was expecting something like this:<br /><br /><span class="Apple-tab-span" style="white-space: pre;"> </span>foreach($array as $sub) {<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>echo(key($sub));<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br /><span class="Apple-tab-span" style="white-space: pre;"> </span><br />Instead, it's this:<br /><br /><span class="Apple-tab-span" style="white-space: pre;"> </span>foreach ($decodedOutput as $key => $value) {<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>echo $key;<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>}Search Directory For String - Linux2013-06-03T19:36:00+00:00https://www.paulsprogrammingnotes.com/2013/06/search-directory-for-string-linuxgrep -rl "search_for_this_string" /var/wwwPHPExcel Convert Excel Serial Date To MM/DD/YY2013-05-31T08:31:00+00:00https://www.paulsprogrammingnotes.com/2013/05/phpexcel-convert-excel-serial-date-toPHPExcel_Shared_Date::ExcelToPHPObject($value)->format("m/d/y")Easiest Way To Consume .Net Web Services - Python2013-05-19T21:48:00+00:00https://www.paulsprogrammingnotes.com/2013/05/easiest-way-to-consume-net-web-services<a href="http://www.jansipke.nl/python-soap-client-with-suds/">http://www.jansipke.nl/python-soap-client-with-suds/</a><br /><br />That article was very helpful for learning to use Python's SUDS library for consuming soap web services.<br /><br />Here's an example of my simple client:<br /><br /><br />from suds.client import Client<br />client = Client("http://xxx.xxx.xxx.xxx/API/Info.asmx?WSDL")<br />client.service.getInfo("05/12/2013","05/12/2013")<br /><br />"05/12/2013" and "05/12/2013" are both parameters<br />New Vocabulary Word2013-05-11T09:34:00+00:00https://www.paulsprogrammingnotes.com/2013/05/new-vocabulary-wordCRUD<br />-Create<br />-Read<br />-Update<br />-Delete<br /><br />An example of a CRUD application would be something that tracks orders in a database. All 4 actions are going to be necessary to track those orders. CRUD: The basic database operations.Dynamically Import Data From Other Workbooks - Google Spreadsheet2013-05-08T05:26:00+00:00https://www.paulsprogrammingnotes.com/2013/05/dynamically-import-data-from-otherI made a google spreadsheet which pulls data from two other workbooks using "importRange" and QUERY function. You can use either one if you just want to pull a few columns from another spreadsheet. QUERY provides more flexibility by allowing you to use a SQL statement. Here is an example of a query which pulls 2 columns from another workbook and filters out the blank rows in column 41:<br /><br />=QUERY(importRange("yourSpreadsheetKeyHere", "Donations!A3:BB200000"), "select Col41, Col2 WHERE Col41!='' ", 1)<br /><br />One of the biggest pains during the entire process was google spreadsheet's vlookup. I learned the vlookup function only searches for a value in the first column in the Array. It's also helpful if you format the results of your vlookup as "Plain Text" under the numbers format drop-down. Of course, something formatted as a number won't match something formatted as plain text.<div><br /></div><div>You could use this technique if you have a list of customer id's with names, and a separate spreadsheet with more customer information. With this, you could combine that information.</div>Javascript Excel Parser2013-05-07T08:20:00+00:00https://www.paulsprogrammingnotes.com/2013/05/javascript-excel-parser<a href="http://niggler.github.io/js-xlsx/">http://niggler.github.io/js-xlsx/</a><br /><div><br /></div><div>That's an impressive project that allows parsing excel files in the browser.</div>Green Mountain Energy's B2B Site2013-04-28T03:29:00+00:00https://www.paulsprogrammingnotes.com/2013/04/green-mountain-energys-b2b-site<p>Green Mountain Energy has the worst B2B portal I’ve seen in my life. The worst parts:</p>
<ul>
<li>It only works in IE (barely, and switching between browser versions in the F12 menu doesn’t help)</li>
<li>The page only takes up ~30% of the browser window (see the picture)</li>
<li>Nothing is functional, I can’t change my automated billing agreement because there’s no submit button.</li>
<li>If you have more than one address on your account, just give up.</li>
</ul>
<p>Picture: <a href="http://i.imgur.com/cP4CLTl.png">http://i.imgur.com/cP4CLTl.png</a></p>
<p>Update: They sent a letter saying they made a new business portal site. <a href="https://www.businessportal.greenmountain.com/">www.businessportal.greenmountain.com</a> My first visit gives me an “Our site is currently down as we upgrade our systems.” error message.</p>
<p><img src="/generated/assets/images/green+mountain+error-800-093ee2ff4.png" srcset="/generated/assets/images/green+mountain+error-400-093ee2ff4.png 400w, /generated/assets/images/green+mountain+error-600-093ee2ff4.png 600w, /generated/assets/images/green+mountain+error-800-093ee2ff4.png 800w, /generated/assets/images/green+mountain+error-917-093ee2ff4.png 917w" /></p>
Don't Use == null - Python2013-04-26T18:04:00+00:00https://www.paulsprogrammingnotes.com/2013/04/dont-use-null-python<a href="http://stackoverflow.com/questions/3289601/null-object-in-python">http://stackoverflow.com/questions/3289601/null-object-in-python</a><br /><br />It says not to use == to check for "none-ness". It's better to use:<br /><br />if variable is none:<br /> <your code><br /><br />Why? <span style="background-color: white; color: #444444; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 13px; line-height: 17px;">"And the reason for choosing </span><code style="background-color: #eeeeee; border: 0px; color: #444444; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 13px; line-height: 17px; margin: 0px; padding: 1px 5px; vertical-align: baseline;">egg is None</code><span style="background-color: white; color: #444444; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 13px; line-height: 17px;"> over </span><code style="background-color: #eeeeee; border: 0px; color: #444444; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 13px; line-height: 17px; margin: 0px; padding: 1px 5px; vertical-align: baseline;">egg == None</code><span style="background-color: white; color: #444444; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 13px; line-height: 17px;">: The latter can be overloaded, and is likely to break when comparing valid object with None (depends on how it's implemented, but you don't expect everyone to take comparisions with None into account, do you?), while </span><code style="background-color: #eeeeee; border: 0px; color: #444444; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 13px; line-height: 17px; margin: 0px; padding: 1px 5px; vertical-align: baseline;">is</code><span style="background-color: white; color: #444444; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 13px; line-height: 17px;"> always works the same."</span>King Of Button Generators2013-04-26T05:07:00+00:00https://www.paulsprogrammingnotes.com/2013/04/king-of-button-generators<a href="http://buttonoptimizer.com/">http://buttonoptimizer.com/</a>Something I Need To Drill Into My Brain - Python2013-04-24T03:36:00+00:00https://www.paulsprogrammingnotes.com/2013/04/something-i-need-to-drill-into-my-brainThe operation of the slicing operator:<div><pre style="background-color: #eeffcc; border-bottom-color: rgb(170, 204, 153); border-bottom-width: 1px; border-style: solid none; border-top-color: rgb(170, 204, 153); border-top-width: 1px; color: #333333; line-height: 15px; overflow-x: auto; overflow-y: hidden; padding: 5px;"><span class="gp" style="color: #c65d09; font-weight: bold;">>>> </span><span class="n">word</span><span class="p">[:</span><span class="mi" style="color: #208050;">2</span><span class="p">]</span> <span class="c" style="color: #408090; font-style: italic;"># The first two characters</span><br /><span class="go" style="color: #303030;">'He'</span><br /><span class="gp" style="color: #c65d09; font-weight: bold;">>>> </span><span class="n">word</span><span class="p">[</span><span class="mi" style="color: #208050;">2</span><span class="p">:]</span> <span class="c" style="color: #408090; font-style: italic;"># Everything except the first two characters</span><br /><span class="go" style="color: #303030;">'lpA'</span></pre><br /></div>Prezi Video With No Audio2013-04-22T22:06:00+00:00https://www.paulsprogrammingnotes.com/2013/04/prezi-video-with-no-audio<p>I converted a wmv video to a few different formats (mp4 and flv) and still couldn’t get the audio to play in Prezi. However, videos with the following codec settings will work:</p>
<p><img src="/assets/images/codec.png" alt="codec info" /></p>
<p>Solution: Open your video with Handbrake and convert it to H264 with AAC as the selected audio codec and it will play just fine.</p>
Start Ser2Sock.c On Boot2013-04-16T21:54:00+00:00https://www.paulsprogrammingnotes.com/2013/04/start-ser2sock-on-boot<br />Make a shell script saying the following:<br />./ser2sock -b 115200 -c -w 5000 -p 10000 -s /dev/ttyUSB0 -d -g 3<br /><div><br /></div><div>Follow these instructions: <a href="http://jonathonhill.net/2009-04-23/auto-start-a-shell-script-on-ubuntu-server/">http://jonathonhill.net/2009-04-23/auto-start-a-shell-script-on-ubuntu-server/</a></div>Easiest To Set Up FTP Server2013-04-09T07:23:00+00:00https://www.paulsprogrammingnotes.com/2013/04/easiest-to-set-up-ftp-server<a href="http://askubuntu.com/questions/1722/basic-ubuntu-ftp-server">http://askubuntu.com/questions/1722/basic-ubuntu-ftp-server</a><br /><br />"<span style="background-color: #f7f7f7; color: #333333; font-family: 'Ubuntu Beta', UbuntuBeta, Ubuntu, 'Bitstream Vera Sans', 'DejaVu Sans', Tahoma, sans-serif; font-size: 14px; line-height: 17px;">I'm going to recommend PureFTPD because it's been the simplest and easiest to use in my opinion. You'll need to install it first: </span><code style="background-color: #e0e0e0; border: 0px; color: #222222; font-family: 'Ubuntu Mono', 'Ubuntu Beta Mono A', Consolas, 'Bitstream Vera Sans Mono', 'Courier New', Courier, monospace; font-size: 14px; line-height: 17px; margin: 0px; padding: 1px 5px; vertical-align: baseline;">sudo apt-get install pure-ftpd</code><span style="background-color: #f7f7f7; color: #333333; font-family: 'Ubuntu Beta', UbuntuBeta, Ubuntu, 'Bitstream Vera Sans', 'DejaVu Sans', Tahoma, sans-serif; font-size: 14px; line-height: 17px;"> once it's installed it'll start itself up."</span>Useful Bootstrap Links2013-04-08T22:00:00+00:00https://www.paulsprogrammingnotes.com/2013/04/useful-bootstrap-links<p><a href="http://bootstraphero.com/the-big-badass-list-of-twitter-bootstrap-resources">http://bootstraphero.com/the-big-badass-list-of-twitter-bootstrap-resources</a></p>
<p>Also, an user named Billy Cravens on Meetup.com left a ton of good bootstrap links (bootsnip looks especially awesome) in this post: <a href="http://www.meetup.com/HoustonCFUG/events/79862792/">http://www.meetup.com/HoustonCFUG/events/79862792/</a></p>
<p>Twitter Bootstrap:<br />
<a href="http://twitter.github.com/bootstrap/">http://twitter.github.com/bootstrap/</a></p>
<p>Bootswatch: (free Bootstrap themes)<br />
<a href="http://www.bootswatch.com">http://www.bootswatch.com</a></p>
<p>Wrap Bootstrap (premium Bootstrap themes)<br />
<a href="http://www.wrapbootstrap.com">http://www.wrapbootstrap.com</a></p>
<p>Common Bootstrap Snippets<br />
<a href="http://www.bootsnipp.com">http://www.bootsnipp.com</a></p>
<p>Some nice date plugins for Bootstrap<br />
<a href="http://www.eyecon.ro/bootstrap-datepicker/">http://www.eyecon.ro/bootstrap-datepicker/</a>
<a href="https://github.com/dangrossman/bootstrap-daterangepicker">https://github.com/dangrossman/bootstrap-daterangepicker</a></p>
<p>Bootstrap WYSIWYG editor plugin<br />
<a href="http://jhollingworth.github.com/bootstrap-wysihtml5/">http://jhollingworth.github.com/bootstrap-wysihtml5/</a></p>
<p>Integrating Bootstrap with jQueryValidate<br />
<a href="http://alittlecode.com/jquery-form-validation-with-styles-from-twitter-bootstrap/">http://alittlecode.com/jquery-form-validation-with-styles-from-twitter-bootstrap/</a></p>
<p>Nice data grids:<br />
<a href="http://www.datatables.net">http://www.datatables.net</a></p>
<p>Integrate Bootstrap with Data Tables:<br />
<a href="http://datatables.net/blog/Twitter_Bootstrap_2">http://datatables.net/blog/Twitter_Bootstrap_2</a></p>
HTML5 In < IE92013-04-08T16:43:00+00:00https://www.paulsprogrammingnotes.com/2013/04/html5-in-ie9<a href="https://code.google.com/p/html5shiv/">https://code.google.com/p/html5shiv/</a><br /><br />I saw this javascript library in the code for the <a href="http://support.mozilla.org/en-US/home">Mozilla Support page</a> (really well done). It looks like it adds HTML5 functionality to older versions of IE. Looks like it should be really useful in the future.PHP - Verify File Is Uploaded And Zip2013-04-02T16:33:00+00:00https://www.paulsprogrammingnotes.com/2013/04/php-verify-file-is-uploaded-and-zip<br />if($_FILES["zip_file"]["name"]) {<br /> $filename = $_FILES["zip_file"]["name"];<br /><br /><br /> $name = explode(".", $filename);<br /><div><br /></div><br /> $continue = strtolower(end($name)) == 'zip' ? true : false;<br /> if(!$continue) {<br /> <span class="Apple-tab-span" style="white-space: pre;"> </span> $message = "The file you are trying to upload is not a .zip file. Please try again.";<br /> } else {<br /> $message = "File Is .zip";<br /> }<br />}<br />PHP - Loop Through Excel Document and Print Cells2013-03-29T15:21:00+00:00https://www.paulsprogrammingnotes.com/2013/03/php-loop-through-excel-document-and<br />require_once 'excel_reader2.php';<br /><br />foreach($reader->sheets as $k=>$data)<br /> {<br /> echo "\n\n ".$reader->boundsheets[$k]."\n\n";<br /><br /> foreach($data['cells'] as $row)<br /> {<br /> foreach($row as $cell)<br /> {<br /> echo "$cell\t";<br /> }<br /> echo "\n";<br /> }<br /> }<br />Head Tracking In The Browser2013-03-17T01:55:00+00:00https://www.paulsprogrammingnotes.com/2013/03/head-tracking-in-browser<a href="http://learningthreejs.com/blog/2013/03/12/move-a-cube-with-your-head/">http://learningthreejs.com/blog/2013/03/12/move-a-cube-with-your-head/</a><br /><br />Had no idea this was even possible, but that's an example of head tracking in the browser done with WEBRTC.<br /><br />Demo: <a href="http://jeromeetienne.github.com/tquery/plugins/headtrackr/examples/demo.html">http://jeromeetienne.github.com/tquery/plugins/headtrackr/examples/demo.html</a>Check If Time Is Between - Python2013-03-16T23:28:00+00:00https://www.paulsprogrammingnotes.com/2013/03/check-if-time-is-between-python<a href="http://stackoverflow.com/questions/1907196/how-do-i-check-if-its-monday-to-friday-and-the-time-is-between-10-am-to-3-pm">http://stackoverflow.com/questions/1907196/how-do-i-check-if-its-monday-to-friday-and-the-time-is-between-10-am-to-3-pm</a><br /><br />From an user named miku:<br /><br /><pre class="default prettyprint prettyprinted" style="background-color: #eeeeee; border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin-bottom: 10px; max-height: 600px; overflow: auto; padding: 5px; vertical-align: baseline; width: auto;"><code style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; margin: 0px; padding: 0px; vertical-align: baseline;"><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">>>></span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> </span><span class="kwd" style="background-color: transparent; border: 0px; color: darkblue; margin: 0px; padding: 0px; vertical-align: baseline;">import</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> datetime<br /></span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">>>></span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> d </span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">=</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> datetime</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">.</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">datetime</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">.</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">now</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">()</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> <br /></span><span class="com" style="background-color: transparent; border: 0px; color: grey; margin: 0px; padding: 0px; vertical-align: baseline;"># => datetime.datetime(2009, 12, 15, 13, 50, 35, 833175)</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"><br /><br /></span><span class="com" style="background-color: transparent; border: 0px; color: grey; margin: 0px; padding: 0px; vertical-align: baseline;"># check if weekday is 1..5</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"><br /></span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">>>></span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> d</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">.</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">isoweekday </span><span class="kwd" style="background-color: transparent; border: 0px; color: darkblue; margin: 0px; padding: 0px; vertical-align: baseline;">in</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> range</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">(</span><span class="lit" style="background-color: transparent; border: 0px; color: maroon; margin: 0px; padding: 0px; vertical-align: baseline;">1</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">,</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> </span><span class="lit" style="background-color: transparent; border: 0px; color: maroon; margin: 0px; padding: 0px; vertical-align: baseline;">6</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">)</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"><br /></span><span class="kwd" style="background-color: transparent; border: 0px; color: darkblue; margin: 0px; padding: 0px; vertical-align: baseline;">True</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"><br /><br /></span><span class="com" style="background-color: transparent; border: 0px; color: grey; margin: 0px; padding: 0px; vertical-align: baseline;"># check if hour is 10..15</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"><br /></span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">>>></span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> d</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">.</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">hour </span><span class="kwd" style="background-color: transparent; border: 0px; color: darkblue; margin: 0px; padding: 0px; vertical-align: baseline;">in</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> range</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">(</span><span class="lit" style="background-color: transparent; border: 0px; color: maroon; margin: 0px; padding: 0px; vertical-align: baseline;">10</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">,</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> </span><span class="lit" style="background-color: transparent; border: 0px; color: maroon; margin: 0px; padding: 0px; vertical-align: baseline;">15</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">)</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"><br /></span><span class="kwd" style="background-color: transparent; border: 0px; color: darkblue; margin: 0px; padding: 0px; vertical-align: baseline;">True</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"><br /><br /></span><span class="com" style="background-color: transparent; border: 0px; color: grey; margin: 0px; padding: 0px; vertical-align: baseline;"># check if minute is 30</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"><br /></span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">>>></span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"> d</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">.</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">minute</span><span class="pun" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">==</span><span class="lit" style="background-color: transparent; border: 0px; color: maroon; margin: 0px; padding: 0px; vertical-align: baseline;">30</span><span class="pln" style="background-color: transparent; border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;"><br /></span><span class="kwd" style="background-color: transparent; border: 0px; color: darkblue; margin: 0px; padding: 0px; vertical-align: baseline;">False</span></code></pre>Prevent Character Escaping - Python2013-03-15T21:57:00+00:00https://www.paulsprogrammingnotes.com/2013/03/prevent-character-escaping<a href="http://stackoverflow.com/questions/2042342/network-path-and-variables-in-python">http://stackoverflow.com/questions/2042342/network-path-and-variables-in-python</a><br /><br />If you want to use network paths without adding extra slashes, you can use python's raw string function.<br /><br />Here's an example of something you want to make a raw string:<br /><br />source_path = r"\\mynetworkshare"<br /><br />Canvas JS Charts - Google Charts Killer?2013-03-07T06:13:00+00:00https://www.paulsprogrammingnotes.com/2013/03/canvas-js-charts-google-charts-killer<a href="http://canvasjs.com/docs/charts/intro/high-performance-javascript-charts/">http://canvasjs.com/docs/charts/intro/high-performance-javascript-charts/</a><br /><br />The most impressive part is how fast it renders all that data.Check If Workbook Is Open By Name - VBA2013-03-05T23:13:00+00:00https://www.paulsprogrammingnotes.com/2013/03/check-if-workbook-is-open-by-name-vbaThis code from StackOverflow helped me find whether a workbook was open with its name. Here's the link to the thread: <a href="http://stackoverflow.com/questions/9373082/detect-whether-excel-workbook-is-already-open-using-vba">Link</a><br /><br /><div><span style="color: darkblue;">Function BookOpen(strBookName As String) As Boolean</span></div><div><span style="color: darkblue;"> Dim oBk As Workbook</span></div><div><span style="color: darkblue;"> On Error Resume Next</span></div><div><span style="color: darkblue;"> Set oBk = Workbooks(strBookName)</span></div><div><span style="color: darkblue;"> On Error GoTo 0</span></div><div><span style="color: darkblue;"> If oBk Is Nothing Then</span></div><div><span style="color: darkblue;"> BookOpen = False</span></div><div><span style="color: darkblue;"> Else</span></div><div><span style="color: darkblue;"> BookOpen = True</span></div><div><span style="color: darkblue;"> End If</span></div><div><span style="color: darkblue;">End Function</span></div><div><span style="color: darkblue;"><br /></span></div><div><span style="color: darkblue;">Sub testbook()</span></div><div><span style="color: darkblue;"> Dim strBookName As String</span></div><div><span style="color: darkblue;"> strBookName = "myWork.xls"</span></div><div><span style="color: darkblue;"> If BookOpen(strBookName) Then</span></div><div><span style="color: darkblue;"> MsgBox strBookName & " is open", vbOKOnly + vbInformation</span></div><div><span style="color: darkblue;"> Else</span></div><div><span style="color: darkblue;"> MsgBox strBookName & " is NOT open", vbOKOnly + vbExclamation</span></div><div><span style="color: darkblue;"> End If</span></div><div><span style="color: darkblue;">End Sub</span></div>"The link you have used to enter the PayPal system is invalid. Please review the link and try again."2013-03-04T03:08:00+00:00https://www.paulsprogrammingnotes.com/2013/03/the-link-you-have-used-to-enter-paypalUsing an incorrect e-mail address in the hidden "business" field in your HTML form can cause this issue.<br /><br />Example of the hidden field with the problem:<br /><input type="hidden" name="business" value="dave@davesgarage.com">Overdue Member Calculator2013-03-01T12:07:00+00:00https://www.paulsprogrammingnotes.com/2013/03/overdue-member-calculator<a href="https://github.com/pawl/OverdueMemberCalculator">https://github.com/pawl/OverdueMemberCalculator</a><br /><br />This should help ease the pain of dealing with suspended paypal recurring payment agreements.<br /><br />The page accepts variables from the URL, then generates a paypal payment button based on the person's last payment date.HTML Skeleton / Boilerplate2013-03-01T09:09:00+00:00https://www.paulsprogrammingnotes.com/2013/03/html-skeleton-boilerplateHere's a good HTML skeleton/boilerplate tool: http://www.htmlshell.com/WebForms.js or WebUIValidation.js CDN2013-03-01T07:23:00+00:00https://www.paulsprogrammingnotes.com/2013/03/webformsjs-or-webuivalidationjs-cdn<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">In the console, it was giving me two errors saying it could not GET these two files from the CDN:</span><br /><div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><span style="font-family: Consolas, 'Lucida Console', monospace; font-size: 12px; white-space: pre-wrap;"><a href="http://www.blogger.com/goog_445237617"><span style="color: #1155cc;">http://ajax.microsoft.com/</span><wbr></wbr><span style="color: #1155cc;">ajax/4.0/2/WebForms.js</span></a></span></div><div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><a href="http://ajax.microsoft.com/ajax/4.0/2/WebUIValidation.js"><span style="font-family: Consolas, 'Lucida Console', monospace; font-size: 12px; white-space: pre-wrap;"><span style="color: #1155cc;">http://ajax.microsoft.com/</span><wbr></wbr>ajax/4.0/2/</span><span style="font-family: Consolas, 'Lucida Console', monospace; font-size: 12px; white-space: pre-wrap;">WebUIValidation.js</span></a></div><div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><span style="font-family: Consolas, 'Lucida Console', monospace; font-size: 12px; white-space: pre-wrap;"><br /></span></div><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;">To fix it, you need to delete the <script></script> references for both of the files above and make a single reference to:</span><div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 13px;"><span style="font-size: 12px; white-space: pre-wrap;"><span style="font-family: Consolas, Lucida Console, monospace;"><script src="</span></span><span style="font-family: Consolas, 'Lucida Console', monospace; font-size: 12px; white-space: pre-wrap;"><a href="http://ajax.aspnetcdn.com/ajax/4.5/6/WebFormsBundle.js" style="color: #1155cc;" target="_blank">http://ajax.aspnetcdn.<wbr></wbr>com/ajax/4.5/6/WebFormsBundle.<wbr></wbr>js</a></span><span style="font-size: 12px; white-space: pre-wrap;"><span style="font-family: Consolas, Lucida Console, monospace;">" type="text/javascript"></<wbr></wbr>script></span></span></div>Raspberry Pi Text To Speech IRC Service2013-02-27T20:17:00+00:00https://www.paulsprogrammingnotes.com/2013/02/raspberry-pi-text-to-speech-irc-serviceThese are my notes for installing Eastein's announce. It's a IRC bot that will receive a message and do text to speech on the message: <a href="https://github.com/eastein/announce">https://github.com/eastein/announce</a><br /><br /><h2><span style="font-size: x-large;">Installation:</span></h2><ol><li><a href="https://github.com/eastein/announce">Download announce</a> and extract it to it's own folder.</li><li>apt-get install python-irclib</li><li>apt-get install python-pip</li><li>sudo apt-get install python-dev</li><li><span style="white-space: pre-wrap;">apt-get install festival festlex-cmu festlex-poslex festlex-oald libestools1.2 unzip</span></li><li><span style="white-space: pre-wrap;">apt-get install speech-tools</span></li><li>pip install pyzmq --install-option="--zmq=bundled"<br /><b>or (if you have easy_install installed):</b><br />easy_install pyzmq</li><li>download <a href="https://github.com/eastein/mediorc">mediorc</a> and move mediorc into directory</li><li>easy_install dnspython</li><li>apt-get install mplayer</li><li>apt-get install sox</li><li><b>Fix permissions:</b><br />chmod 777 saypitchprase<br />chmod 777 pitchphrase2wav</li></ol><div><h2><span style="font-size: x-large;">Running The Program:</span></h2></div><b>Run process in background by starting a screen (first navigate to the Announce folder):</b><br /><ol><li>(start a new screen)</li><li>python announced "tcp://*:4900"</li><li>(detach from screen)</li><li>(start a new screen)</li><li><span style="white-space: pre-wrap;">python announcebot "chat.freenode.net" "voicebot" "#yourircchannel" "tcp://0:4901"</span></li><li><span style="white-space: pre-wrap;">(detack from screen)</span></li></ol><div><div><div><div style="white-space: normal;"><span style="white-space: pre-wrap;"><b>Starting the program using mp3 files in the JSON file (as always, start in the folder which has Announce):</b></span></div><div style="white-space: normal;"><br /><ol><li><span style="white-space: pre-wrap;">wget http://fake.com/fake/eye_tiger.mp3 </span><b style="white-space: pre-wrap;">(use a real URL with an mp3 file)</b></li><li><span style="white-space: pre-wrap;">nano default.json</span></li><li><span style="white-space: pre-wrap;">enter into the json file: [ ["eye of the tiger", "eye_tiger.mp3"] ]</span></li><li><span style="white-space: pre-wrap;">ctrl+x and save</span></li><li><span style="white-space: pre-wrap;">(start a new screen)</span></li><li><span style="white-space: pre-wrap;">python announced "tcp://0:4900" default.json</span></li><li><span style="white-space: pre-wrap;">(detach from screen)</span></li><li><span style="white-space: pre-wrap;">(start a new screen)</span></li><li>python announcebot "chat.freenode.net" "voicebot" "#yourircchannel" "tcp://0:4900"</li><li>(detach from screen)</li><li>now "!say eye of the tiger" in the IRC channel will trigger the mp3</li></ol></div><br /><h2><span style="font-size: x-large;">Troubleshooting:</span></h2><pre style="white-space: pre-wrap; word-wrap: break-word;"></pre><pre style="white-space: pre-wrap; word-wrap: break-word;"><b>Ensure your Raspberry Pi audio is working:</b> <a href="http://jeffskinnerbox.wordpress.com/2012/11/15/getting-audio-out-working-on-the-raspberry-pi/">http://jeffskinnerbox.wordpress.com/2012/11/15/getting-audio-out-working-on-the-raspberry-pi/</a></pre><br /><div><b>Testing Festival (create a text file named mytext.txt with some words first):</b></div><div><div>echo "This is a test." | festival --tts</div></div><div>echo "this is a test" > mytext.txt</div><div>text2wave -o myaudio.wav mytext.txt</div><div><div>aplay myaudio.wav</div></div><div><br /></div></div><div><b>better voices:</b><br /><a href="http://ubuntuforums.org/showthread.php?t=677277">http://ubuntuforums.org/showthread.php?t=677277</a><br /><pre style="word-wrap: break-word;"><span style="white-space: pre-wrap;"><a href="http://ubuntuforums.org/showthread.php?t=751169">http://ubuntuforums.org/showthread.php?t=751169</a></span></pre><pre style="word-wrap: break-word;"></pre><pre style="word-wrap: break-word;"></pre></div></div><div><b>adjust the volume:</b><br />http://blog.scphillips.com/2013/01/sound-configuration-on-raspberry-pi-with-alsa/<br />or use amixer?</div></div>Find An Image Font2013-02-27T20:01:00+00:00https://www.paulsprogrammingnotes.com/2013/02/font-finderThis page will tell you what font an image is using: <a href="http://www.myfonts.com/WhatTheFont/">http://www.myfonts.com/WhatTheFont/</a><br /><br />I wish this feature was included in GIMP or Photoshop.Regex Debugger2013-02-23T00:38:00+00:00https://www.paulsprogrammingnotes.com/2013/02/regex-debugger<a href="http://www.debuggex.com/?re=&str=">http://www.debuggex.com/?re=&str= </a><br /><br />Don't want to lose this link, it's going to make regex related programming a lot easier.Tableizer2013-02-19T22:48:00+00:00https://www.paulsprogrammingnotes.com/2013/02/tableizerFreaking brilliant tool that converts spreadsheet format things to HTML: http://tableizer.journalistopia.com/tableizer.php<br /><br />PhoneGap Build - cordova-2.4.0.js2013-02-19T20:39:00+00:00https://www.paulsprogrammingnotes.com/2013/02/phonegap-build-cordova-240jsIf you're using Phonegap Build, you don't need to include cordova-2.4.0.js or any other specific version of cordova/phonegap. Just use phonegap.js, but you don't actually need to include phonegap.js in the file you upload to Phonegap Build.<br /><br />Quote from the Phonegap Build site:<br /><span style="-webkit-text-size-adjust: none; background-color: whitesmoke; color: #222222; font-family: adobe-clean, 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 18px;">"Once you've included the necessary assets, remove the </span><code style="-webkit-text-size-adjust: none; background-color: whitesmoke; box-sizing: border-box; color: #222222; font-family: monospace, serif; font-size: 14px;">phonegap.js</code><span style="-webkit-text-size-adjust: none; background-color: whitesmoke; color: #222222; font-family: adobe-clean, 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 18px;"> (cordova.js) as Build will automatically inject it during compile time."</span>Sharepoint Lists Crashing2013-02-05T23:03:00+00:00https://www.paulsprogrammingnotes.com/2013/02/sharepoint-lists-crashingIf your sharepoint lists crash IE as soon as they open, try disabling the following IE addons:<div><div class="MsoNormal"><span style="font-family: 'Segoe UI', sans-serif; font-size: 10pt;">Sharepoint spreadsheet Launcher</span><o:p></o:p></div><div class="MsoNormal"><span style="font-family: 'Segoe UI', sans-serif; font-size: 10pt;">Sharepoint export database~</span><o:p></o:p></div></div>Open File With Excel Macro2013-02-05T15:31:00+00:00https://www.paulsprogrammingnotes.com/2013/02/open-file-with-excel-macro<br />This macro will ask the user to input an excel file, then open that excel file:<br /><br />Sub RunMacro()<br /> <br />Dim vaFiles As Variant<br />Dim i As Long<br /><br />vaFiles = Application.GetOpenFilename _<br /> (FileFilter:="Excel Filer (*.xls),*.xls", _<br /> Title:="Open File(s)", MultiSelect:=False)<br /><br />If Not IsArray(vaFiles) Then Exit Sub<br /><br />With Application<br /> .ScreenUpdating = False<br /> <br /> For i = 1 To UBound(vaFiles)<br /> Workbooks.Open vaFiles(i)<br /> Next i<br /> <br /> .ScreenUpdating = True<br />End With<br /><br />End Sub<br /><div><br /></div>Java - Swing's Nimbus Look & Feel2013-02-01T16:32:00+00:00https://www.paulsprogrammingnotes.com/2013/02/java-swings-nimbus-look-feel<p>Since Java SE 6 Update 10, Java has had an updated interface called Nimbus. Comparison pictures are below.</p>
<p>All I needed to do to use it was add this import to the top of the .java file with my main() method:
<code class="language-plaintext highlighter-rouge">import javax.swing.UIManager.*;</code></p>
<p>And this code to the top of the main() method (<a href="http://stackoverflow.com/questions/4617615/how-to-set-nimbus-look-and-feel-in-main">http://stackoverflow.com/questions/4617615/how-to-set-nimbus-look-and-feel-in-main</a>):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>public static void main(String[] args) {
try {
for (LookAndFeelInfo info : UIManager.getInstalledLookAndFeels()) {
if ("Nimbus".equals(info.getName())) {
UIManager.setLookAndFeel(info.getClassName());
break;
}
}
} catch (Exception e) {
// If Nimbus is not available, fall back to cross-platform
try {
UIManager.setLookAndFeel(UIManager.getCrossPlatformLookAndFeelClassName());
} catch (Exception ex) {
// not worth my time
}
}
new Controller();
}
</code></pre></div></div>
<p>Default:
<img src="/generated/assets/images/default-484-e1b065ae0.png" srcset="/generated/assets/images/default-400-e1b065ae0.png 400w, /generated/assets/images/default-484-e1b065ae0.png 484w" /></p>
<p>Nimbus:
<img src="/generated/assets/images/nimbus-313-6b0b3189e.png" srcset="/generated/assets/images/nimbus-313-6b0b3189e.png 313w" /></p>
<p>Note: This blog post says it is slightly slower than the default: <a href="http://www.pushing-pixels.org/2008/06/17/lightbeam-measuring-performance-of-swing-look-and-feels.html">http://www.pushing-pixels.org/2008/06/17/lightbeam-measuring-performance-of-swing-look-and-feels.html</a></p>
"Protected mode is currently turned off for the Internet zone"2013-01-23T17:58:00+00:00https://www.paulsprogrammingnotes.com/2013/01/protected-mode-is-currently-turned-off<p>Want to disable the IE7 pop-up saying “Protected mode is currently turned off”?</p>
<p><img src="/assets/images/protected+mode.png" alt="protected mode" /></p>
<p><strong>Solution:</strong> Just right-click the message and click “Don’t show this again”.</p>
Ljava.lang.String; Error2013-01-22T18:49:00+00:00https://www.paulsprogrammingnotes.com/2013/01/ljavalangstring-errorThis is a simple one. You're asking java to return an Array as a string.<br /><br /><b>Solution:</b> Put Arrays.toString( ) around your Array.Java Exceptions - Catching All Errors2013-01-22T18:42:00+00:00https://www.paulsprogrammingnotes.com/2013/01/java-exceptions"Exception is the base class for all exceptions"<br /><br />"Thus any exception that may get thrown is an Exception (Uppercase 'E')."<br /><br />Meaning, catch(Exception e) will catch any type of exception that is thrown, as opposed to specific Exceptions.<br /><br />Example:<br />try { <br /> // code<br />}catch(Exception e)<br />{<br /> //catch all exceptions<br />}<br /><br /><a href="http://stackoverflow.com/questions/1075895/how-can-i-catch-all-the-exceptions-that-will-be-thrown-through-reading-and-writi">http://stackoverflow.com/questions/1075895/how-can-i-catch-all-the-exceptions-that-will-be-thrown-through-reading-and-writi</a>Meaning Of "l" In LDAP2013-01-16T20:53:00+00:00https://www.paulsprogrammingnotes.com/2013/01/meaning-of-l-in-ldap"The lower case 'l' refers to the locality (city)."
Jquery Slideup Jerking Effect In IE92013-01-15T20:21:00+00:00https://www.paulsprogrammingnotes.com/2013/01/jquery-slideup-jerking-effect-in-ie9This website has code to help get rid of the jerking effect on Jquery SlideUp in IE9: <a href="https://siderite.dev/blog/jquery-slideup-flickers-in-internet.html">https://siderite.dev/blog/jquery-slideup-flickers-in-internet.html</a><br /><br />You just add this code after you include your Jquery library:<br /><br /><pre style="background-color: white; color: #09004d; font-size: 13px; line-height: 18px;">(function(){<br />// Define overriding method.<br />jQuery.fx.prototype.hide = function(){<br /><br />// Remember where we started, so that we can go back to it later<br />this.options.orig[this.prop] = jQuery.style( this.elem, this.prop );<br />this.options.hide = true;<br /><br />// Begin the animation<br />this.custom(this.cur(), 1);<br />}<br />})();</pre>Raspberry Pi - No Ethernet Caused By Charger2013-01-15T04:34:00+00:00https://www.paulsprogrammingnotes.com/2013/01/raspberry-pi-no-ethernet-caused-byI was using the following charger, but I couldn't get my Raspberry Pi to connect over Ethernet (ethernet lights showed no link): <a href="http://www.amazon.com/gp/product/B0038HYPZS">http://www.amazon.com/gp/product/B0038HYPZS</a><br /><br />The charger was definitely the problem. I swapped it with a Motorola OEM charger and the Ethernet started working again. I did some quick googling, and haven't seen many reports of this issue yet. I have the Model B, Rev 2.0 board with 512MB RAM.WHMCS LDAP User Creation Action Hook2013-01-14T07:44:00+00:00https://www.paulsprogrammingnotes.com/2013/01/whmcs-ldap-user-creation-action-hookI created an Action Hook for WHMCS which will create an user when they are added as a client. With this script, we no longer need to add our users to LDAP manually. I'm thinking about turning it into a module, and making it easier to other users to use.<br /><br /><a href="https://github.com/pawl/WHMCS_LDAP">https://github.com/pawl/WHMCS_LDAP</a>LDAP_ADD PHP - Object Class Violation2013-01-14T07:11:00+00:00https://www.paulsprogrammingnotes.com/2013/01/ldapadd-php-object-class-violationIf you are trying to use php's ldap_add to add an user and are getting an object class violation, try to copy all the fields of a working user exactly. Most likely you are missing some of the required fields of the objectclass you are trying to add.Reading RFID With Raspberry Pi (or any linux device) + Reader2013-01-14T06:59:00+00:00https://www.paulsprogrammingnotes.com/2013/01/reading-rfid-with-raspberry-pi-or-anyI'm using the following Parallax USB RFID reader: <a href="http://www.parallax.com/Portals/0/Downloads/docs/prod/audiovis/28140-28340-RFIDreader-v2.2.pdf">http://www.parallax.com/Portals/0/Downloads/docs/prod/audiovis/28140-28340-RFIDreader-v2.2.pdf</a><br /><br />First you will need to ensure you have PySerial installed by trying to "import PySerial" while running Python's IDLE. If you receive an error:<br /><div><ol><ol></ol></ol></div><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><div>Install PySerial:</div></blockquote><ol><ol><li>Install python's setuptools: apt-get install python-setuptools</li><li>Install PySerial: pip install pyserial</li></ol></ol><div><br /></div>Once you have PySerial installed, open IDLE again and use the following code to see output from the reader:<br /><div><br /><script src="https://gist.github.com/pawl/9935400.js"></script><div><br /></div><div>Why didn't ser.read(12) return the same unique identifier as what is written on the fob?:<br /><ul><li>If the key on the card says 30788590 and the reader returns 3501D5CBEE. The "1D5CBEE" part is the card number. Try going from the card number in DEC to HEX it in the windows calculator while it's in scientific mode if you don't believe me.</li></ul></div></div>"margin-left: auto;" and "margin-right: auto;" Not Working In IE82013-01-07T22:51:00+00:00https://www.paulsprogrammingnotes.com/2013/01/margin-left-auto-and-margin-right-auto<div>To get something centered within a div in IE8, it requires doing something unusual. I wasn't able to get the "display: block;" trick working. But this worked:</div><br /><div>#super-wrap { position: absolute; width: 100%; text-align: center; } </div><div>#page-wrap { position: relative; width: 970px; margin: auto auto; }</div><div><br /></div><div>I had to do this to get the InfoGrid (<a href="http://css-tricks.com/examples/InfoGrid/">http://css-tricks.com/examples/InfoGrid/</a>) to center in IE8. The div with the ID of #page-wrap already exists, but the super-wrap div needs to be created outside of the page-wrap div.</div>Using "nth-of-type" with IE82013-01-07T16:44:00+00:00https://www.paulsprogrammingnotes.com/2013/01/using-nth-of-type-with-ie8IE8 apparently doesn't support the CSS selector "nth-of-type".<br /><br />You can fix it by adding this script to your page: <a href="https://github.com/keithclark/JQuery-Extended-Selectors/blob/master/jquery-extra-selectors.js">https://github.com/keithclark/JQuery-Extended-Selectors/blob/master/jquery-extra-selectors.js</a><br /><br />You will also need to add the following script to the bottom of your page (of course you will need to swap the numbers with your own):<br /><br />$("div:nth-of-type(1)").css("background", "#b44835");<br /><div><br /><br />I used this to fix an IE8 compatibility issue with something called InfoGrid: <a href="http://css-tricks.com/examples/InfoGrid/">http://css-tricks.com/examples/InfoGrid/</a></div>PHP - Generate SSHA (sha1) Password For LDAP User2013-01-07T02:48:00+00:00https://www.paulsprogrammingnotes.com/2013/01/php-generate-ssha-sha1-password-for$info['userpassword'][0] = "{SHA}" . base64_encode(sha1("pass", TRUE));Function To Find Largest UID Number In LDAP2013-01-07T02:43:00+00:00https://www.paulsprogrammingnotes.com/2013/01/function-to-find-largest-uid-number-in<br /> I got most of this function from <a href="http://bakery.cakephp.org/articles/UncleBill/2006/10/15/using-ldap-as-a-database">http://bakery.cakephp.org/articles/UncleBill/2006/10/15/using-ldap-as-a-database</a>. It will find all the uidNumbers, sorting them largest to smallest, then return the largest number.<br /><br />$ds = ldap_connect("localhost"); // assuming the LDAP server is on this host<br /><br /> function findLargestUidNumber($ds)<br /> {<br /> $s = ldap_search($ds, "ou=people,dc=yourdomain,dc=com", 'uidnumber=*');<br /> if ($s)<br /> {<br /> // there must be a better way to get the largest uidnumber, but I can't find a way to reverse sort.<br /> ldap_sort($ds, $s, "uidnumber");<br /><br /> $result = ldap_get_entries($ds, $s);<br /> $count = $result['count'];<br /> $biguid = $result[$count-1]['uidnumber'][0];<br /> return $biguid;<br /> }<br /> return null;<br /> }<br /><div><br />$largestUID = findLargestUidNumber($ds);</div>LDAP shadowlastchange - Weird Date Format2013-01-07T02:09:00+00:00https://www.paulsprogrammingnotes.com/2013/01/ldap-shadowlastchange-uses-weird-date<br />I'm working on a PHP script that adds an user to a directory using LDAP.<br /><br />Here's the formula required to get the date format that shadowlastchange uses:<br />$unixTimeDays = floor(time()/86400);<br /><div><br />It's the days since the last epoch.</div>First Android App2012-12-31T13:04:00+00:00https://www.paulsprogrammingnotes.com/2012/12/first-android-app<p><a href="https://play.google.com/store/apps/details?id=com.DallasTollCalculator">https://play.google.com/store/apps/details?id=com.DallasTollCalculator</a></p>Py2exe Issues2012-12-28T17:21:00+00:00https://www.paulsprogrammingnotes.com/2012/12/py2exe-issuesWhen getting the directory of the file you're executing in python use the correct example:<br /><br /><span style="color: lime;">CORRECT</span>: os.path.abspath(os.path.dirname(sys.argv[0]))<br /><span style="color: red;">INCORRECT</span>: os.path.dirname(inspect.getsourcefile( lambda:None ))<br /><br />If you're using py2exe, the incorrect example will just give error messages.Outlook 2010 - Run a Script (Blank)2012-12-27T15:47:00+00:00https://www.paulsprogrammingnotes.com/2012/12/outlook-2010-run-script-blank<br />Apparently a script does not equal a macro in outlook. When you're setting up a rule, you have the option to run a script. However, your macro will not show up in the script menu unless it has (MyMail As MailItem) as an argument. Example:<br /><br />Sub Save(MyMail As MailItem)<br /><br />SaveEmailAttachmentsToFolder "AgentReports", "html", "C:\Users\EBRNPAL\Documents\Agent Reports\Saved"<br /><br />End Sub<br />Save Outlook Attachments Automatically With A Macro2012-12-21T22:26:00+00:00https://www.paulsprogrammingnotes.com/2012/12/save-outlook-attachments-automaticallySource: <a href="http://www.rondebruin.nl/mail/folder2/saveatt.htm">http://www.rondebruin.nl/mail/folder2/saveatt.htm</a><br /><br /><br />Sub Test()<br />'Arg 1 = Folder name in your Inbox<br />'Arg 2 = File extension, "" is every file<br />'Arg 3 = Save folder, "C:\Users\Ron\test" or ""<br />'If you use "" it will create a date/time stamped<br />'folder for you in the "My Documents" folder.<br />'Note: If you use this "C:\Users\Ron\test" the folder must exist<br /><br /> SaveEmailAttachmentsToFolder "AgentReports", "html", "C:\Users\Paul\Documents\Agent Reports\Saved"<br /><br />End Sub<br /><br />Sub SaveEmailAttachmentsToFolder(OutlookFolderInInbox As String, _<br /> ExtString As String, DestFolder As String)<br /> Dim ns As NameSpace<br /> Dim Inbox As MAPIFolder<br /> Dim SubFolder As MAPIFolder<br /> Dim Item As Object<br /> Dim Atmt As Attachment<br /> Dim FileName As String<br /> Dim MyDocPath As String<br /> Dim I As Integer<br /> Dim wsh As Object<br /> Dim fs As Object<br /><br /> On Error GoTo ThisMacro_err<br /><br /> Set ns = GetNamespace("MAPI")<br /> Set Inbox = ns.GetDefaultFolder(olFolderInbox)<br /> Set SubFolder = Inbox.Folders(OutlookFolderInInbox)<br /><br /> I = 0<br /> ' Check subfolder for messages and exit of none found<br /> If SubFolder.Items.Count = 0 Then<br /> MsgBox "There are no messages in this folder : " & OutlookFolderInInbox, _<br /> vbInformation, "Nothing Found"<br /> Set SubFolder = Nothing<br /> Set Inbox = Nothing<br /> Set ns = Nothing<br /> Exit Sub<br /> End If<br /><br /> 'Create DestFolder if DestFolder = ""<br /> If DestFolder = "" Then<br /> Set wsh = CreateObject("WScript.Shell")<br /> Set fs = CreateObject("Scripting.FileSystemObject")<br /> MyDocPath = wsh.SpecialFolders.Item("mydocuments")<br /> DestFolder = MyDocPath & "\" & Format(Now, "dd-mmm-yyyy hh-mm-ss")<br /> If Not fs.FolderExists(DestFolder) Then<br /> fs.CreateFolder DestFolder<br /> End If<br /> End If<br /><br /> If Right(DestFolder, 1) <> "\" Then<br /> DestFolder = DestFolder & "\"<br /> End If<br /><br /> ' Check each message for attachments and extensions<br /> For Each Item In SubFolder.Items<br /> For Each Atmt In Item.Attachments<br /> If LCase(Right(Atmt.FileName, Len(ExtString))) = LCase(ExtString) Then<br /> FileName = DestFolder & Item.SenderName & " " & Atmt.FileName<br /> Atmt.SaveAsFile FileName<br /> I = I + 1<br /> End If<br /> Next Atmt<br /> Next Item<br /><br /> ' Show this message when Finished<br /> If I > 0 Then<br /> MsgBox "You can find the files here : " _<br /> & DestFolder, vbInformation, "Finished!"<br /> Else<br /> MsgBox "No attached files in your mail.", vbInformation, "Finished!"<br /> End If<br /><br /> ' Clear memory<br />ThisMacro_exit:<br /> Set SubFolder = Nothing<br /> Set Inbox = Nothing<br /> Set ns = Nothing<br /> Set fs = Nothing<br /> Set wsh = Nothing<br /> Exit Sub<br /><br /> ' Error information<br />ThisMacro_err:<br /> MsgBox "An unexpected error has occurred." _<br /> & vbCrLf & "Please note and report the following information." _<br /> & vbCrLf & "Macro Name: SaveEmailAttachmentsToFolder" _<br /> & vbCrLf & "Error Number: " & Err.Number _<br /> & vbCrLf & "Error Description: " & Err.Description _<br /> , vbCritical, "Error!"<br /> Resume ThisMacro_exit<br /><br />End Sub<br /><div><br /></div>Declaring 2 Variables At The Same Time In Python2012-12-21T16:57:00+00:00https://www.paulsprogrammingnotes.com/2012/12/declaring-2-variables-at-same-time-inI thought this was interesting:<div><br /></div><div>import os</div><div>fileName, fileExtension = os.path.splitext('/path/to/somefile.ext') </div><div><br /></div><div>Since os.path.splitext returns a list of 2 items, you can assign those 2 list items to variables at the same time with the code above.</div><div><br /></div>Path To Sharepoint2012-12-18T23:23:00+00:00https://www.paulsprogrammingnotes.com/2012/12/path-to-sharepoint<a href="http://www.pathtosharepoint.com/">http://www.pathtosharepoint.com</a><br /><br />A lot of that stuff looks useful for adding functionality to Sharepoint. Also, this is one of the first public sites I've seen which use sharepoint (and it's not too bad). I saw it mentioned on this site: <a href="http://sharepointjavascript.wordpress.com/">http://sharepointjavascript.wordpress.com/</a>UTDesign - Xbee Claims vs Reality2012-12-15T08:55:00+00:00https://www.paulsprogrammingnotes.com/2012/12/utdesign-xbee-claims-vs-reality<p><img src="/generated/assets/images/2012-12-14_13-22-11_839-800-8068c12e6.jpg" srcset="/generated/assets/images/2012-12-14_13-22-11_839-400-8068c12e6.jpg 400w, /generated/assets/images/2012-12-14_13-22-11_839-600-8068c12e6.jpg 600w, /generated/assets/images/2012-12-14_13-22-11_839-800-8068c12e6.jpg 800w, /generated/assets/images/2012-12-14_13-22-11_839-1000-8068c12e6.jpg 1000w" /></p>
<p>The picture above shows some research done by some UTD students for their UT Design program.</p>
<p>Their results showed the specs claimed by Xbee modules were far superior to reality. They said they experienced an overwhelming amount of noise at about half of the rated distance and both the peak and idle current ratings were inaccurate.</p>
Unheap - Uncluttering Plugins2012-12-13T16:48:00+00:00https://www.paulsprogrammingnotes.com/2012/12/unheap-uncluttering-plugins<a href="http://www.unheap.com/">http://www.unheap.com/</a><br /><br />That website is an excellent list of a lot of good jquery plugins out there.Raspberry Pi - Mining Bitcoins 2012-12-13T07:58:00+00:00https://www.paulsprogrammingnotes.com/2012/12/raspberry-pi-mining-bitcoinsRather than letting my Raspberry Pi lay dormant in its cabinet at a data center, I've decided to use it for mining bitcoins.<br /><br />I followed these instructions: http://www.youtube.com/watch?v=rrdzam7voOg<br /><br />I'm using Deepbit as the pool, but I'm not 100% sure what a pool is or if there are better pools out there.<br /><br />Update: In about 5 days, I have only mined 0.00003898 BTC.Stop IE Alert Messages 2012-12-04T23:30:00+00:00https://www.paulsprogrammingnotes.com/2012/12/stop-ie-alert-messagesThe following snippet of Jquery will prevent alerts from showing in internet explorer:<br /><br />if ($.browser.msie) { window.alert = function() { }; };<br /><br />It detects if the browser is IE, then disables the alert function with a blank if it is IE. The alert happens only in IE, that's why it checks to see if the browser is IE.<br /><br />I did this because there is some code I can't change on a sharepoint page which is causing an alert (which doesn't actually cause any real issues). Adding this code to the part of the page I can change will stop the error.<br /><br />About the alert:<br />It says "Hit error fn!" and it happens in the following span after an ajax call fails: <span style="color: #881280; font-family: monospace; white-space: pre-wrap;"><span </span><span class="webkit-html-attribute-name" style="font-family: monospace; white-space: pre-wrap;">id</span><span style="color: #881280; font-family: monospace; white-space: pre-wrap;">="</span><span class="webkit-html-attribute-value" style="font-family: monospace; white-space: pre-wrap;">ctl00_LogUserActivity1</span><span style="color: #881280; font-family: monospace; white-space: pre-wrap;">"> </span><br /><br />It's definitely a hack, but it works so far.Raspberry Pi Webserver w/ Wordpress Slow2012-11-28T08:22:00+00:00https://www.paulsprogrammingnotes.com/2012/11/raspberry-pi-webserver-w-wordpressI installed the LAMP stack on a Raspberry Pi, then moved a small Wordpress site to it. It then got plugged into a fast connection at a data center.<br /><br />Before the move (shared hosting webserver at Siteground):<br /><br /><ul><li>1 second load times on the main page</li></ul><div>After the move (Raspberry Pi):</div><div><ul><li>8-9 second load times on the main page</li><li>5 second load times on single pages</li></ul><div>The Pi does a good job at displaying static pages, but it does not run more intensive things like a CMS quickly enough.</div></div>LAMP Stack Ubuntu2012-11-28T03:54:00+00:00https://www.paulsprogrammingnotes.com/2012/11/lamp-stack-ubuntuIt's crazy easy to install the LAMP stack on ubuntu with this command:<br /><br />sudo apt-get install lamp-server^SPJS Charts - Dynamically Select Filter Item2012-11-26T20:53:00+00:00https://www.paulsprogrammingnotes.com/2012/11/spjs-chartsI've been using <a href="http://sharepointjavascript.wordpress.com/2012/02/03/spjs-charts-for-sharepoint-v3-x/">SPJS charts</a> to make google charts with data from sharepoint lists. If you use a custom drop-down filter, you may want to dynamically select items in the drop-down like I did.<br /><br />An example: You're using a drop-filter which filters the results by week, but you always want it to select last week.<br /><br />Snippet:<br /><br /><br />function manualLoad(){<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>loadManually = false;<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>spjs_GenerateChart();<br /><br /><span class="Apple-tab-span" style="white-space: pre;"> </span>Date.prototype.getWeek = function() {<br /><span class="Apple-tab-span" style="white-space: pre;"> </span> var onejan = new Date(this.getFullYear(),0,1);<br /><span class="Apple-tab-span" style="white-space: pre;"> </span> var today = new Date(this.getFullYear(),this.getMonth(),this.getDate());<br /><span class="Apple-tab-span" style="white-space: pre;"> </span> var dayOfYear = ((today - onejan + 1)/86400000);<br /><span class="Apple-tab-span" style="white-space: pre;"> </span> return Math.ceil(dayOfYear/7)<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>};<br /><br /><span class="Apple-tab-span" style="white-space: pre;"> </span>$("#MyChart1_CustomFilterSelect_Week").val(new Date().getWeek()-1).change();<br />}2nd Place - Ericsson & North Texas Food Bank Hackathon2012-11-13T22:54:00+00:00https://www.paulsprogrammingnotes.com/2012/11/2nd-place-north-texas-food-bank<p><img src="/generated/assets/images/image-640-a274b837a.png" srcset="/generated/assets/images/image-400-a274b837a.png 400w, /generated/assets/images/image-600-a274b837a.png 600w, /generated/assets/images/image-640-a274b837a.png 640w" /></p>
<p>The “Hackathon” was a programming competition sponsored by Ericsson to solve some of the North Texas Food Bank’s problems.</p>
<p>We made a scheduling system that allows people to see which days need more volunteers, then sign up to volunteer. The coolest part is that it would call people with a prerecorded message reminding them of their appointment.</p>
Calculate Weeks In Sharepoint2012-11-13T22:51:00+00:00https://www.paulsprogrammingnotes.com/2012/11/calculate-weeks-in-sharepoint=INT((Created-DATE(YEAR(Created),1,6)+(TEXT(WEEKDAY(DATE(YEAR(Created),0,1)),"d")))/7)+1<br /><br />This is a modified version of this formula which didn't give me the correct week number (maybe due to regional settings?): <a href="http://lamahashim.blogspot.com/2009/10/sharepoint-calculated-field-week-number.html">http://lamahashim.blogspot.com/2009/10/sharepoint-calculated-field-week-number.html </a><br /><br />It matches the number of weeks calculated in excel and all the calculators I could find on the internet.<br /><br />Update: I also needed to add a 0 before the "Week" if the number is less than 10, because it will cause an error in sorting otherwise.<br /><br />Here's the formula to add a 0 if the number is less than 10:<br />=IF((INT((Date-DATE(YEAR(Date),1,6)+(TEXT(WEEKDAY(DATE(YEAR(Date),0,1)),"d")))/7)+1)<10,"0"&(INT((Date-DATE(YEAR(Date),1,6)+(TEXT(WEEKDAY(DATE(YEAR(Date),0,1)),"d")))/7)+1),(INT((Date-DATE(YEAR(Date),1,6)+(TEXT(WEEKDAY(DATE(YEAR(Date),0,1)),"d")))/7)+1))&" "<br /><br />(Note: the &" " part forces it to be a string rather than a floating point value)Impressive Javascript2012-11-01T19:36:00+00:00https://www.paulsprogrammingnotes.com/2012/11/impressive-javascript<a href="https://github.com/mbostock/d3/wiki/Gallery">https://github.com/mbostock/d3/wiki/Gallery</a><br /><br />"D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation."Google Charts - GeoMap vs GeoChart2012-10-25T18:33:00+00:00https://www.paulsprogrammingnotes.com/2012/10/google-charts-geomap-vs-geochartGeoMap - The map is rendered in the browser using an embedded Flash player.<br /><br />GeoChart - Rendered using SVG<br /><br />I noticed the GeoChart loads faster (at least in Chrome)."Object date(2012,10,4) has no method 'getTimezoneOffset'" Google Charts2012-10-24T17:41:00+00:00https://www.paulsprogrammingnotes.com/2012/10/object-date2012104-has-no-methodThis error was caused by not lowercase date in date(2012,10,4). It should be Date(2012,10,4).Sizeof() vs Count() in PHP2012-10-22T18:27:00+00:00https://www.paulsprogrammingnotes.com/2012/10/sizeof-vs-count-in-php<a href="http://php.net/manual/en/function.sizeof.php">http://php.net/manual/en/function.sizeof.php</a><br /><br />They're the same. According to the link above: the sizeof function is an alias of: <a href="http://www.php.net/manual/en/function.count.php">count()</a>.PHP Date Variables2012-10-19T19:48:00+00:00https://www.paulsprogrammingnotes.com/2012/10/php-date-variables<a href="http://php.about.com/od/learnphp/ss/php_functions_3.htm">http://php.about.com/od/learnphp/ss/php_functions_3.htm</a><br /><br />That's a helpful post for controlling the output of PHP's date().Difference Between Echo and Print In PHP2012-10-19T15:58:00+00:00https://www.paulsprogrammingnotes.com/2012/10/difference-between-echo-and-print-in-php<a href="http://stackoverflow.com/questions/3615679/what-is-basic-difference-echo-vs-print">http://stackoverflow.com/questions/3615679/what-is-basic-difference-echo-vs-print</a><br /><br />"echo can print more than 1 argument, print can only print 1 argument."Triple Equals Sign In Php2012-10-19T15:50:00+00:00https://www.paulsprogrammingnotes.com/2012/10/triple-equals-sign-in-php<a href="http://stackoverflow.com/questions/80646/how-do-the-equality-double-equals-and-identity-triple-equals-comparis">http://stackoverflow.com/questions/80646/how-do-the-equality-double-equals-and-identity-triple-equals-comparis</a><br /><br />Since PHP uses $ to declare variables (and not int, etc), one can run into issues with an if statement being unintentionally true.<br /><br />For example, 0 == null.<br /><br />The triple equals sign (===) allows you to prevent these cases by only returning true if the two inputs are not the same type.<br /><br />So, 0 === null is not true. Also, 0 === "0" is not true either.SQLSRV30.EXE is not a valid Win32 application2012-10-15T15:41:00+00:00https://www.paulsprogrammingnotes.com/2012/10/sqlsrv30exe-is-not-valid-win32If you want to install the PDO drivers for PHP 5.4, you will need to extract SQLSRV30.EXE with Winrar.<br /><br />Since I was using EasyPHP, I needed to extract the files into the following directory (there's also another php directory in EasyPHP, not sure why, but it's worth adding the files under "ext" there too): C:\Program Files\EasyPHP-12.1\php\php546x120827090829\ext<br /><br />You will also need to add the following lines to your PHP.ini file under PHPExt:<br /><br />extension=php_pdo_sqlsrv_54_ts.dll<br />extension=php_sqlsrv_54_ts.dll<br />Managing SQL Server 2008 with Management Studio 20052012-10-12T16:38:00+00:00https://www.paulsprogrammingnotes.com/2012/10/managing-sql-server-2008-withWhen you google "SQL Management Studio", one of the first things that comes up is this (the 2005 version): <a href="http://www.microsoft.com/en-us/download/details.aspx?id=8961">http://www.microsoft.com/en-us/download/details.aspx?id=8961</a><br /><br />I recommend not installing 2005 if you have SQL Server 2008 Express installed! You will not be able to manage your 2008 server, and you will need to uninstall management studio to install the 2008 version here: <a href="http://www.microsoft.com/en-us/download/details.aspx?id=22985">http://www.microsoft.com/en-us/download/details.aspx?id=22985</a>Find Values With VBA2012-10-04T15:07:00+00:00https://www.paulsprogrammingnotes.com/2012/10/find-values-with-vba<a href="http://www.rondebruin.nl/find.htm">http://www.rondebruin.nl/find.htm</a><br /><br />The above website has some great snippets of code which can:<br /><br /><ul><li><a href="http://www.blogger.com/blogger.g?blogID=4808398841648427911" name="Select" style="color: navy; font-family: Arial, Helvetica, sans-serif; font-size: 14px;"><span class="Kop1" style="color: black; font-size: 18px; font-weight: bold; text-decoration: underline;">Use Find to select a cell</span></a></li><li><a href="http://www.blogger.com/blogger.g?blogID=4808398841648427911" name="Mark" style="color: navy; font-family: Arial, Helvetica, sans-serif; font-size: 14px; font-weight: bold;"><span class="Kop1" style="color: black; font-size: 18px; text-decoration: underline;">Mark cells with the same value in column A in the B column</span></a></li><li><a href="http://www.blogger.com/blogger.g?blogID=4808398841648427911" name="Color" style="color: navy; font-family: Arial, Helvetica, sans-serif; font-size: 14px; font-weight: bold;"><span class="Kop1" style="color: black; font-size: 18px; text-decoration: underline;">Color cells with the same value in a Range, worksheet or all worksheets</span></a></li><li><a href="http://www.blogger.com/blogger.g?blogID=4808398841648427911" name="Copy" style="color: navy; font-family: Arial, Helvetica, sans-serif; font-size: 14px; font-weight: bold;"><span class="Kop1" style="color: black; font-size: 18px; text-decoration: underline;">Copy cells to another sheet with Find</span></a></li></ul>Get Folder Path From User Input2012-09-28T15:26:00+00:00https://www.paulsprogrammingnotes.com/2012/09/get-folder-path-from-user-input<br />This is some very useful code from <a href="http://stackoverflow.com/questions/5971292/vba-excel-getting-file-path-ends-with-folder">Stack Overflow</a> which allows the user to select a folder in VBA and reads the folder path into a variable. I'm currently making a macro which opens several XML files in a folder, and this will help users to input the folder where those XML files are located:<br /><br />Sub SelectFolder()<br /> Dim diaFolder As FileDialog<br /><br /> ' Open the file dialog<br /> Set diaFolder = Application.FileDialog(msoFileDialogFolderPicker)<br /> diaFolder.AllowMultiSelect = False<br /> diaFolder.Show<br /><br /> MsgBox diaFolder.SelectedItems(1)<br /><br /> Set diaFolder = Nothing<br />End Sub<br /><br />VBA - Delete Worksheet With A Specific Name If It Exists2012-09-17T18:39:00+00:00https://www.paulsprogrammingnotes.com/2012/09/vba-delete-worksheet-with-specific-name<div>Application.DisplayAlerts = False </div><div>On Error Resume Next </div><div>ThisWorkbook.Sheets("Sheet1").Delete </div><div>On Error Goto 0 </div><div>Application.DisplayAlerts = True </div><div><br /></div><div><ul><li>The code between "On Error Resume Next" and "On Error Goto 0" will be skipped if an error is generated. </li><li>It also won't display the error to the user because of the code on line 1 (Application.DisplayAlerts = False).</li><li>"ThisWorkbook" will need to be declared.</li></ul></div><div><br /></div><div><br /></div>Error occurred during initialization of VM java/lang/NoClassDefFoundError: java/lang/Object2012-09-14T14:51:00+00:00https://www.paulsprogrammingnotes.com/2012/09/error-occurred-during-initialization-of<p><img src="/assets/images/java+error.png" alt="java error" /></p>
<p>If you get this error when you just type “java” into the command prompt, then it can be solved by typing “PATH=C:\Program Files\Java\jdk1.7.0_06\bin” (the folder of your java.exe).</p>
Modifying PhoneGap Childbrowser View2012-08-14T16:21:00+00:00https://www.paulsprogrammingnotes.com/2012/08/modifying-phonegap-childbrowser-view<p>In IOS, removing the back/forward, refresh, and history buttons from ChildBrowser requires opening its .xib file.</p>
<p>I removed everything except the done button:</p>
<p><img src="/assets/images/Screen+Shot+2012-08-14+at+11.15.54+AM.png" alt="UIWebView" /></p>
Childbrowser Plugin2012-08-14T14:11:00+00:00https://www.paulsprogrammingnotes.com/2012/08/childbrowser-plugin<span class="Apple-style-span" style="font-family: arial; font-size: x-small;"><a href="http://blog.digitalbackcountry.com/2012/03/installing-the-childbrowser-plugin-for-ios-with-phonegapcordova-1-5/">http://blog.digitalbackcountry.com/2012/03/installing-the-childbrowser-plugin-for-ios-with-phonegapcordova-1-5/</a></span><br /><div style="font-family: arial; font-size: small;"><br /></div><div style="font-family: arial; font-size: small;">I thought that was a really good guide for getting started on Phonegap's Childbrowser plugin.</div><div style="font-family: arial; font-size: small;"><br /></div><div style="font-family: arial; font-size: small;">I'm using it to open PDFs from phonegap.</div>My First Stack Overflow Answer2012-08-07T18:59:00+00:00https://www.paulsprogrammingnotes.com/2012/08/first-stack-overflow-answer<a href="http://stackoverflow.com/questions/11783402/iscroll-and-similar-will-not-scroll-horizontally-phonegap-jqtouch-iphone-app/11852309#11852309">http://stackoverflow.com/questions/11783402/iscroll-and-similar-will-not-scroll-horizontally-phonegap-jqtouch-iphone-app/11852309#11852309</a> <br /><br />I use Stack Overflow a ton, and I'm going to try to make more of an effort to contribute.<br /><br />I was having trouble scrolling horizontally with the iscroll plugin, and I solved it by enabling the hScroll option (which I thought should have been enabled by default). I use iscroll's zoom function to enable zooming on a single div, while the rest of the page does not zoom.SigPad API2012-08-07T13:59:00+00:00https://www.paulsprogrammingnotes.com/2012/08/sigpad-apiI had difficulties getting Thomas J Bradley's HTML5 Signature Pad to let me to use its API without resetting the field. This Github issue explains a workaround:<br /><br /><a href="https://github.com/thomasjbradley/signature-pad/issues/13">https://github.com/thomasjbradley/signature-pad/issues/13</a> <br /><br />He is saving the signature, declaring the api variable, then regenerating the signature.<br /><br />I needed to use the .getSignatureImage() api function which allows me to turn the signature into a base64 string so I could store the signature in a database.PHP Inline HTML2012-07-24T18:20:00+00:00https://www.paulsprogrammingnotes.com/2012/07/php-inline-html<a href="http://www.4webhelp.net/tutorials/php/echo.php">http://www.4webhelp.net/tutorials/php/echo.php</a> <br /><br />The article in the link above describes the process of adding inline HTML to PHP code instead of "echo '<html>';". It seems like it's much faster to not process each echo statement with PHP.<br /><br />This is definitely a big "oh!" moment for someone still learning PHP.Jquery Mobile - Only Load Once2012-07-24T14:28:00+00:00https://www.paulsprogrammingnotes.com/2012/07/jquery-mobile-only-load-onceIf you only want something to load once, you can use the "pageinit" event.<br /><br />Here's an example of some code with "pageinit":<br /><br /><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: x-small; line-height: 16px;">$(document).</span><em style="background-color: white; font-family: arial, sans-serif; font-size: small; font-style: normal; font-weight: bold; line-height: 16px;">delegate</em><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: x-small; line-height: 16px;">("#yourPage", "</span><em style="background-color: white; font-family: arial, sans-serif; font-size: small; font-style: normal; font-weight: bold; line-height: 16px;">pageinit</em><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: x-small; line-height: 16px;">", function(event) { alert</span><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: x-small; line-height: 16px;">( "ALERT!"); });</span> <br /><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: x-small; line-height: 16px;"><br /></span><br /><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: x-small; line-height: 16px;">The code above would not run if you visited the page again.</span>Make Phonegap Responsive2012-07-23T16:34:00+00:00https://www.paulsprogrammingnotes.com/2012/07/make-phonegap-responsiveThe following was absolutely mandatory for me to get my phonegap app to be responsive: <a href="https://github.com/cargomedia/jquery.touchToClick">https://github.com/cargomedia/jquery.touchToClick</a><br /><br />Without modification, Phonegap will wait for a "double-click" after the user taps. This results in a noticeable delay. With the plugin above included, it removes the delay.<br /><br />Other plugins I tested wanted me to use selectors for each element I didn't want delayed.Prevent Scrolling2012-07-16T13:03:00+00:00https://www.paulsprogrammingnotes.com/2012/07/prevent-scrollinghttp://hankchizljaw.co.uk/tutorials/prevent-window-bounce-phonegap-1-7-ios-tip/13/05/2012/<br /><br />The above link is the most simple solution I've found to prevent phone gap from scrolling outside of the bounds.Magnifying Glass For Images Plugin2012-07-13T19:07:00+00:00https://www.paulsprogrammingnotes.com/2012/07/magnifying-glass-for-images-pluginHere's a link to a cool magnifying glass plugin which uses Jquery:<br /><a href="http://www.gcmingati.net/wordpress/wp-content/lab/jquery/minizoompan/">http://www.gcmingati.net/wordpress/wp-content/lab/jquery/minizoompan/</a>Using Quotes In mysql_fetch_array()2012-07-05T13:28:00+00:00https://www.paulsprogrammingnotes.com/2012/07/using-quotes-in-mysqlfetcharrayAccording to this blog post: <a href="http://www.securityandcaffeine.com/2008/04/03/php-mysql-and-mysql_fetch_array/">http://www.securityandcaffeine.com/2008/04/03/php-mysql-and-mysql_fetch_array/</a> <br /><br /><strong style="border: 0px; color: #111111; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 12px; line-height: 18px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">mysql_fetch_array()</strong> can be sped up dramatically by including quotes, like this:<br /><br /><span style="color: #111111; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 12px; line-height: 18px;"> </span><strong style="border: 0px; color: #111111; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 12px; line-height: 18px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">$result['2']</strong><span style="color: #111111; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 12px; line-height: 18px;"> rather than this </span><strong style="border: 0px; color: #111111; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 12px; line-height: 18px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">$result[2]</strong> <br /><strong style="border: 0px; color: #111111; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 12px; line-height: 18px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><br /></strong><br /><span style="color: #111111; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif;"><span style="font-size: 12px; line-height: 18px;">I just changed my code to use quotes, and I haven't noticed a big speed increase (that's probably not where the bottleneck is).</span></span><br /><span style="color: #111111; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif;"><span style="font-size: 12px; line-height: 18px;"><br /></span></span><br /><span style="color: #111111; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif;"><span style="font-size: 12px; line-height: 18px;">Edit: This is probably misleading according to the good people at stackoverflow. </span></span><a href="http://stackoverflow.com/questions/11345894/mysql-fetch-array-quotes-increase-speed">http://stackoverflow.com/questions/11345894/mysql-fetch-array-quotes-increase-speed</a>Sisyphus.js2012-07-03T16:56:00+00:00https://www.paulsprogrammingnotes.com/2012/07/sisyphusjsI'm really impressed with sisyphus.js and its usage of HTML5 localStorage. It makes it so users won't need to re-enter data into a form when they close the window.<br /><br />It's super easy to implement, you only need to include the following in your header:<br /><br /><script type="text/javascript" src="sisyphus.min.js"></script><br /><div><br /></div><div>And, you will need to activate it on the form you want to save with something like this:</div>$('form').sisyphus(); <br /><br />Source: <a href="http://simsalabim.github.com/sisyphus/">http://simsalabim.github.com/sisyphus/</a>Write To Closest Div (with certain class)2012-06-26T15:56:00+00:00https://www.paulsprogrammingnotes.com/2012/06/write-to-closest-div-with-certain-classThe following code makes it so I don't have to mention specifically what page needs to load before a script runs. This is good for having the same function on several different pages in Jquery Mobile.<br /><br />var pageId = $(this).closest('div.diagramPage').attr('id');<br /><br />$(document).delegate(pageId, 'pageshow', function() { }<br /><br />I included the class because I don't want to end up moving code around and having it write to a different div.Fixing: "The requested URL /jquery-1.6.3.min.js was not found on this server."2012-06-19T05:58:00+00:00https://www.paulsprogrammingnotes.com/2012/06/requested-url-jquery-163minjs-was-not<h1 style="margin: 0px 0px 15px;"><span style="font-size: small; font-weight: normal;">In Wordpress, the following is the error message that occurs at the top of the page:</span></h1><h1 style="margin: 0px 0px 15px;"><span style="color: #333333; font-family: Bitter, serif; font-size: 2.769230769230769em; font-weight: normal; line-height: 1.3em;">Not Found</span></h1><span style="background-color: white; color: #555555; font-family: Bitter, serif; font-size: 13px; line-height: 21px;">The requested URL /jquery-1.6.3.min.js was not found on this server.</span><br /><hr style="border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-color: rgb(231, 231, 231); border-top-style: solid; color: #555555; font-family: Bitter, serif; font-size: 13px; height: 1px; line-height: 21px; margin: 1em 0px 1.692307em; padding: 0px;" /><i style="color: #555555; font-family: Bitter, serif; font-size: 13px; line-height: 21px;">www.4llw4d.freefilesblog.com</i> <br /><i style="color: #555555; font-family: Bitter, serif; font-size: 13px; line-height: 21px;"><br /></i><br />I fixed this error by editing the all-in-one SEO plugin's code (all_in_one_seo_pack.php). I'm pretty sure the following part of the code is the culprit:<br /><br /><div><div> $url = "http://www.4llw4d.freefilesblog.com/jquery-1.6.3.min.js"; </div><div> $ch = curl_init(); </div><div> $timeout = 5; </div><div> curl_setopt($ch,CURLOPT_URL,$url); </div><div> curl_setopt($ch,CURLOPT_RETURNTRANSFER,1); </div><div> curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout); </div><div> $data = curl_exec($ch); </div><div> curl_close($ch); </div><div> echo "$data";</div><div>}</div></div><div><br /></div><div>I tried changing it to something else, like: <a href="http://code.jquery.com/jquery-1.6.3.min.js">http://code.jquery.com/jquery-1.6.3.min.js</a> However, it just prints the entire contents onto the page.<br /><br /><b>The solution:</b> Switch from the Pro 1.72 version of All-In-One SEO to the newest unpaid version.</div>Jquery Mobile - Display Loading Message2012-06-18T15:14:00+00:00https://www.paulsprogrammingnotes.com/2012/06/jquery-mobile-display-loading-messageI've been displaying the ajax loading message when loading a php file (especially when there's a chance it won't run instantly).<div><br /></div><div>I've been doing the following:</div><div><br /></div><div><div>$.mobile.showPageLoadingMsg ();</div></div><div><div>$('#phpDiv').load('file.php',function(){</div><div><span class="Apple-tab-span" style="white-space: pre;"> </span>$('#phpDiv').trigger('create');</div><div><span class="Apple-tab-span" style="white-space: pre;"> </span>});</div></div><div><div>$.mobile.hidePageLoadingMsg ();</div><div><br class="Apple-interchange-newline" /></div></div><div>"showPageLoadingMsg" will start to display the ajax loading message and "hidePage~" will close it. The stuff in the middle loads what the php file returns into a div, then creates DOM contents for it with the trigger('create') function.</div>Best Jquery Mobile Datebox Plugin2012-06-14T15:13:00+00:00https://www.paulsprogrammingnotes.com/2012/06/best-jquery-mobile-datebox-pluginThe currently reigning champion of datebox plugins for Jquery Mobile: <a href="http://dev.jtsage.com/jQM-DateBox/">http://dev.jtsage.com/jQM-DateBox/</a>Jquery Mobile - Close Jqm-Datebox When Dialog Closes2012-06-14T15:12:00+00:00https://www.paulsprogrammingnotes.com/2012/06/jquery-mobile-close-jqm-datebox-when// this will make sure the datebox closes when the dialog closes<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>$(document).delegate('#closeButton', 'click', function() {<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>$(".ui-datebox-container").hide();<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>$(".ui-datebox-screen").hide();<br /><span class="Apple-tab-span" style="white-space: pre;"> </span>});<br /><br />Make sure your #closeButton is the same as the button which closes the dialog.TodoMVC with MySQL Database2012-06-13T14:37:00+00:00https://www.paulsprogrammingnotes.com/2012/06/todomvc-with-mysql-database<a href="https://github.com/bartcc/app.todos">https://github.com/bartcc/app.todos</a><br /><br />This project is a modification of the TodoMVC (used to sample how a language is structured by making a To-Do list). It adds database support and ends up being a very cool To-Do list.Good Jquery Mobile Blog Post2012-06-13T13:39:00+00:00https://www.paulsprogrammingnotes.com/2012/06/good-jquery-mobile-blog-postJust got finished reading the following blog post: <a href="http://roughlybrilliant.com/jquery_mobile_best_practices#0b">http://roughlybrilliant.com/jquery_mobile_best_practices#0b</a><br /><br />Most of it's obvious, but it's definitely on my list of good blog posts for Jquery Mobile beginners.<br /><br />Reminds me how Jquery Mobile needs an official "Best Practices" document.Loading Scripts Dynamically In Jquery Mobile2012-06-11T19:05:00+00:00https://www.paulsprogrammingnotes.com/2012/06/loading-scripts-dynamically-in-jqueryAt first, I loaded all of my scripts in the header (even when the scripts weren't needed when the page was being loaded).<br /><br />I found this website that partially explained how to load scripts when the page loads. However, Jquery Mobile needs to use the delegate function to load the script when a page is showing.<br /><br />I copied the function from the website above:<br /><pre style="background-color: #efefef; font-size: 13px;">function loadjscssfile(filename, filetype){<br /> if (filetype=="js"){ //if filename is a external JavaScript file<br /> var fileref=document.createElement('script')<br /> fileref.setAttribute("type","text/javascript")<br /> fileref.setAttribute("src", filename)<br /> }<br /> else if (filetype=="css"){ //if filename is an external CSS file<br /> var fileref=document.createElement("link")<br /> fileref.setAttribute("rel", "stylesheet")<br /> fileref.setAttribute("type", "text/css")<br /> fileref.setAttribute("href", filename)<br /> }<br /> if (typeof fileref!="undefined")<br /> document.getElementsByTagName("head")[0].appendChild(fileref)<br />}</pre><pre style="font-size: 13px;"></pre><pre style="font-size: 13px;"></pre><pre style="font-size: 13px;"></pre>Now, you need to add code to the top of the data-role="page" where you want the script to load:<br /><pre><span style="background-color: white;">$(document).delegate('#page', 'pageshow', function() {<br /> loadjscssfile("your.css", "css")<br /> loadjscssfile("your.js", "js")</span></pre><pre><span style="background-color: white;">});</span></pre><pre></pre>Best Jquery Calendar Plugin2012-06-05T18:47:00+00:00https://www.paulsprogrammingnotes.com/2012/06/best-jquery-calendar-plugin<p><img src="/generated/assets/images/fullcalendar-793-5638ac2b2.jpg" srcset="/generated/assets/images/fullcalendar-400-5638ac2b2.jpg 400w, /generated/assets/images/fullcalendar-600-5638ac2b2.jpg 600w, /generated/assets/images/fullcalendar-793-5638ac2b2.jpg 793w" /></p>
<p>FullCalendar can be found here: <a href="http://arshaw.com/fullcalendar/">http://arshaw.com/fullcalendar/</a></p>
<p>I thought it looked the best and had the most API functions out of the HTML5 calendars I saw.</p>
Jquery Add/Remove Class2012-06-05T18:25:00+00:00https://www.paulsprogrammingnotes.com/2012/06/jquery-addremove-classProblem: In Jquery Mobile, I used the <span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: x-small; line-height: 16px;">mobile.</span><em style="background-color: white; font-family: arial, sans-serif; font-size: small; font-style: normal; font-weight: bold; line-height: 16px;">changePage</em><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: x-small; line-height: 16px;"> (method)</span> to switch pages. Now the button for the previous page is still blue.<br /><br />Solution: Add Id's to the list items you'll be dealing with. Then use Jquery to remove the class "ui-btn-active" from the old button, and add the "ui-btn-active" class to the new button.<br /><br />Here's the code:<br />$('#listItem1').removeClass('ui-btn-active')<br />$('#listItem2').addClass('ui-btn-active')Global Variables In Javascript2012-06-05T15:14:00+00:00https://www.paulsprogrammingnotes.com/2012/06/global-variables-in-javascriptProblem: I had two different <script> (javascript) sections, and I was unable to pass a variable from one set of <script> tags to the other.<br /><br />Turns out it doesn't have anything to do with the script tags, and I just needed to learn a bit more about scope in Javascript. Apparently global variables are a common thing (maybe still not a good practice?) and as long as a variable is used outside of a function.. You'll be able to access the variable anywhere on the page.<br /><br />Solution: Take the variable outside of the function.Jquery Serialize2012-06-05T15:05:00+00:00https://www.paulsprogrammingnotes.com/2012/06/jquery-serialize<p>I spent a while looking for a good way to pass variables from HTML to Javascript (ajax) to PHP. Turns out Jquery’s serialize function is exactly what I needed.</p>
<p>Here’s an example of the code:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>//this will send all the inputs in the form to php with ajax
$('#form').submit( function() {
var data = form.$('input').serialize();
$.ajax({
type: "POST",
url: "process.php",
data: data,
success: function(msg){
alert( "Your quote request has been submitted!" );
}
});
return false;
} );
</code></pre></div></div>
<p><a href="http://api.jquery.com/serialize/">http://api.jquery.com/serialize/</a></p>
<p>Also, you can use <code class="language-plaintext highlighter-rouge">serializeArray()</code> to get the data in a better format. Here’s a picture of the console.log output:
<img src="/generated/assets/images/array-800-15203e67a.jpg" srcset="/generated/assets/images/array-400-15203e67a.jpg 400w, /generated/assets/images/array-600-15203e67a.jpg 600w, /generated/assets/images/array-800-15203e67a.jpg 800w, /generated/assets/images/array-1000-15203e67a.jpg 1000w" /></p>