Import Python: ImportPython Issue 98 - Introduction to airflow becoming pdb power user and more

Airflow is a popular pipeline orchestration tool for Python that allows users to configure complex (or simple!) multi-system workflows that are executed in parallel across any number of workers. A single pipeline might contain bash, Python, and SQL operations. With dependencies specified between tasks, Airflow knows which ones it can run in parallel and which ones must run after others. Airflow is written in Python and users can add their own operators with custom functionality, doing anything Python can do.

video

,

workflow engine

Email API from SendGrid. Reliably deliver your emails with a quick and simple API or SMTP integration. Try for Free. Curator's Note - Python and Django integration for sendgrid https://github.com/sendgrid/sendgrid-python and https://github.com/RyanBalfanz/django-sendgrid respectively. You can send 12,000 emails per month free.

I was preparing to push some changes a couple of days ago and as I usually do, I ran the tests. I sat back in my chair as the dots raced across the screen when suddenly I noticed that one of the dots linger. ”OS is probably running some updates in the background or something” I said to myself, and ran the tests again just to be sure. I watched closely as the dots filled the screen and there it was again?—?I have a slow test!

Good Tutorial on using pdb.

pdb

Talk proposals will be due on 2017 January 3.Poster proposals will be due on 2017 January 3.Tutorial proposals are due on 2017 November 30. Yes, that’s right — tutorial proposals are due in three weeks.

pycon

A curated list of awesome Python asyncio frameworks, libraries, software and resources.

async-io

,

curated list

I've recently been exploring the exciting new world of asynchronous I/O libraries in Python 3 – specifically asyncio and curio. These two libraries make some different design choices. This is an essay that I wrote to try to explain to myself what those differences are and why I think they matter, and distill some principles for designing event loop APIs and asynchronous libraries in Python.

async-io

An extension for Django admin that makes interface mobile friendly.

django

Try Hired and get in front of 4,000+ companies with one application. No more pushy recruiters, no more dead end applications and mismatched companies, Hired puts the power in your hands.

A command-line utility for querying Python ASTs using XPath syntax

opensource project

Cloud Dataflow provides a fully-managed, autoscaling, serverless execution environment for data pipelines written in Apache Beam. In this article Lak Lakshmanan and Matt Hancher show us how to create a monthly vegetation index from Landsat images, available as a public dataset.

google cloud

We are working with a start-up who already has an established product, customer base and deliver cutting edge software solutions to some of the largest global media companies as well as some corporates. All the solutions are based around video and media so any experience in this area is a massive bonus.

Android Emulator for mobile security testing

ByteNet for character-level language modelling

Accurately generate all possible forms of an English word e.g "election" --> "elect", "electoral", "electorate" etc.

FOSS Heartbeat analyses the health of a community of contributors. :heartbeat:

A simple module for Python 2/3 to print and refresh multi line output contents in terminal

JET is a different approach to make numeric python substantially faster

A simple Batch Image Downloader using Python and BeautifulSoup.

Package / Module importer for importing code from Jupyter Notebook files (.ipynb)

Bridge between IRC and Slack