It's a short guide, but I think it covers everything a beginner needs to know.
Taskflow looks especially interesting. The article is unfortunately missing Airbnb's Airflow: https://github.com/airbnb/airflow
Edit 12/25/2020: I’d recommend using: https://postgresapp.com/
Ended up being surprisingly easy:
I’m finally switching to a Macbook Pro as my personal computer after a few months of using one at work. Local development on OSX is a lot easier and bug-free than on a Windows computer.
I started a gist with what I did for setup:
https -- Some packages may not be found!
ImportError: cannot import name HTTPSHandler
I was trying to install pip in cygwin when this happened. Searches say openssl-devel needed to be installed, but it already was. Apparently python was compiled without ssl support.
My solution was to install cygwin-x86 instead of the 64x one.
You need to run the following commands:
lynx -source rawgit.com/transcode-open/apt-cyg/master/apt-cyg > apt-cyg
install apt-cyg /bin
The following can be used in cron or run with & at the end of the command:
I found this didn’t work: http://www.ahtik.com/blog/removing-detached-docker-containers-automatically/
The above command is an example of how "docker run" would be used in crontab to run once every week at 5am.
- --rm will delete the container once the job is finished running
- --name will name the container and prevent duplicate jobs from running
- You don't need to use "&" at the end, because crons already run in the background.
This happens because MySQL runs each update statement individually, but you should still be able to see the batches when the queries are in the "init" state.
Use this query to see the batched queries:
SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST where state="init";