Today we've issued the 2.0.7 and 1.11.14 bugfix releases.

The release package and checksums are available from our downloads page, as well as from the Python Package Index. The PGP key ID used for this release is Carlton Gibson: E17DF5C82B4F9D00.

The EuroPython Society (EPS), who is organizing the EuroPython conference, last year extended it’s mission to also provide help for the Python community in Europe in general.

As part of this, we would like to get to know, and help create closer ties between organizers of other European Python events.

Organizers’ Lunch

We would like to invite representatives of all European Python conference to EuroPython 2018 to join us for an organizers’ lunch. We’re planing the lunch for Friday, July 27, in the organizer’s room (Soutra Suite).

Our aim is to get to know each other, exchange experience in organizing events and to find out how we, as EPS, can most effectively help other conferences going forward.

Free Tickets

To support and facilitate this, we are giving out one free conference ticket per conference team, so that each team can send a representative to the organizers’ lunch.

If your team wants to send someone to join, please write to board@europython.eu, mentioning the conference you’re organizing and some background on your team.

Enjoy,
–
EuroPython Society
https://ep2018.europython.eu/ 
https://www.europython-society.org/

The EuroPython Society (EPS) extended its mission last year to not only run the EuroPython conference, but also provide help for the Python community in Europe in general.

Let’s all meet at EuroPython

In addition to the Python Organizers Lunch (see previous post), which focuses on conference organizers, we are also establishing a program to support attendees of Python user groups and conferences in Europe.

We’d like to invite all of you to EuroPython 2018 this year. Of course, we cannot give out free tickets to everyone, but we can at least recognize your participation in the Python community by giving out discounts for the conference.

Discounts for EuroPython Tickets

If you are running a Python event (conference or user group) in Europe, please reach out to board@europython.eu to request a coupon code for your group, which you can then pass on to your group members or attendees.

If you are not running a user group or conference, but a regular attendee of one, please contact your organizers to have them submit a request. We can only distribute codes at the user group and conference organizer level.

The coupon codes are valid for conference tickets bought starting today and will give you a 10% discount on the ticket price (both regular and late bird prices). The codes are setup for user group sizes of between 30-50 members, but we are also extending this to organizers and attendees of larger conferences. If you need a code valid for larger groups, please mention this in your email.

Enjoy,
–
EuroPython Society
https://ep2018.europython.eu/ 
https://www.europython-society.org/

In this tutorial, you’ll be equipped to make production-quality, presentation-ready Python histogram plots with a range of choices and features.

If you have introductory to intermediate knowledge in Python and statistics, you can use this article as a one-stop shop for building and plotting histograms in Python using libraries from its scientific stack, including NumPy, Matplotlib, Pandas, and Seaborn.

A histogram is a great tool for quickly assessing a probability distribution that is intuitively understood by almost any audience. Python offers a handful of different options for building and plotting histograms. Most people know a histogram by its graphical representation, which is similar to a bar graph:

This article will guide you through creating plots like the one above as well as more complex ones. Here’s what you’ll cover:

• Building histograms in pure Python, without use of third party libraries
• Constructing histograms with NumPy to summarize the underlying data
• Plotting the resulting histogram with Matplotlib, Pandas, and Seaborn

Histograms in Pure Python

When you are preparing to plot a histogram, it is simplest to not think in terms of bins but rather to report how many times each value appears (a frequency table). A Python dictionary is well-suited for this task:

>>> # Need not be sorted, necessarily>>> a=(0,1,1,1,2,3,7,7,23)>>> defcount_elements(seq)->dict:... """Tally elements from `seq`."""... hist={}... foriinseq:... hist[i]=hist.get(i,0)+1... returnhist>>> counted=count_elements(a)>>> counted{0: 1, 1: 3, 2: 1, 3: 1, 7: 2, 23: 1}

count_elements() returns a dictionary with unique elements from the sequence as keys and their frequencies (counts) as values. Within the loop over seq, hist[i] = hist.get(i, 0) + 1 says, “for each element of the sequence, increment its corresponding value in hist by 1.”

In fact, this is precisely what is done by the collections.Counter class from Python’s standard library, which subclasses a Python dictionary and overrides its .update() method:

>>> fromcollectionsimportCounter>>> recounted=Counter(a)>>> recountedCounter({0: 1, 1: 3, 3: 1, 2: 1, 7: 2, 23: 1})

You can confirm that your handmade function does virtually the same thing as collections.Counter by testing for equality between the two:

>>> recounted.items()==counted.items()True

It can be helpful to build simplified functions from scratch as a first step to understanding more complex ones. Let’s further reinvent the wheel a bit with an ASCII histogram that takes advantage of Python’s output formatting:

defascii_histogram(seq)->None:"""A horizontal frequency-table/histogram plot."""counted=count_elements(seq)forkinsorted(counted):print('{0:5d}{1}'.format(k,'+'*counted[k]))

This function creates a sorted frequency plot where counts are represented as tallies of plus (+) symbols. Calling sorted() on a dictionary returns a sorted list of its keys, and then you access the corresponding value for each with counted[k]. To see this in action, you can create a slightly larger dataset with Python’s random module:

>>> # No NumPy ... yet>>> importrandom>>> random.seed(1)>>> vals=[1,3,4,6,8,9,10]>>> # Each number in `vals` will occur between 5 and 15 times.>>> freq=(random.randint(5,15)for_invals)>>> data=[]>>> forf,vinzip(freq,vals):... data.extend([v]*f)>>> ascii_histogram(data)    1 +++++++    3 ++++++++++++++    4 ++++++    6 +++++++++    8 ++++++    9 ++++++++++++   10 ++++++++++++

Here, you’re simulating plucking from vals with frequencies given by freq (a generator expression). The resulting sample data repeats each value from vals a certain number of times between 5 and 15.

Building Up From the Base: Histogram Calculations in NumPy

Thus far, you have been working with what could best be called “frequency tables.” But mathematically, a histogram is a mapping of bins (intervals) to frequencies. More technically, it can be used to approximate the probability density function (PDF) of the underlying variable.

Moving on from the “frequency table” above, a true histogram first “bins” the range of values and then counts the number of values that fall into each bin. This is what NumPy’shistogram() function does, and it is the basis for other functions you’ll see here later in Python libraries such as Matplotlib and Pandas.

Consider a sample of floats drawn from the Laplace distribution. This distribution has fatter tails than a normal distribution and has two descriptive parameters (location and scale):

>>> importnumpyasnp>>> # `numpy.random` uses its own PRNG.>>> np.random.seed(444)>>> np.set_printoptions(precision=3)>>> d=np.random.laplace(loc=15,scale=3,size=500)>>> d[:5]array([18.406, 18.087, 16.004, 16.221,  7.358])

In this case, you’re working with a continuous distribution, and it wouldn’t be very helpful to tally each float independently, down to the umpteenth decimal place. Instead, you can bin or “bucket” the data and count the observations that fall into each bin. The histogram is the resulting count of values within each bin:

>>> hist,bin_edges=np.histogram(d)>>> histarray([ 1,  0,  3,  4,  4, 10, 13,  9,  2,  4])>>> bin_edgesarray([ 3.217,  5.199,  7.181,  9.163, 11.145, 13.127, 15.109, 17.091,       19.073, 21.055, 23.037])

This result may not be immediately intuitive. np.histogram() by default uses 10 equally sized bins and returns a tuple of the frequency counts and corresponding bin edges. They are edges in the sense that there will be one more bin edge than there are members of the histogram:

>>> hist.size,bin_edges.size(10, 11)

A very condensed breakdown of how the bins are constructed by NumPy looks like this:

>>> # The leftmost and rightmost bin edges>>> first_edge,last_edge=a.min(),a.max()>>> n_equal_bins=10# NumPy's default>>> bin_edges=np.linspace(start=first_edge,stop=last_edge,... num=n_equal_bins+1,endpoint=True)...>>> bin_edgesarray([ 0. ,  2.3,  4.6,  6.9,  9.2, 11.5, 13.8, 16.1, 18.4, 20.7, 23. ])

The case above makes a lot of sense: 10 equally spaced bins over a peak-to-peak range of 23 means intervals of width 2.3.

From there, the function delegates to either np.bincount() or np.searchsorted(). bincount() itself can be used to effectively construct the “frequency table” that you started off with here, with the distinction that values with zero occurrences are included:

>>> bcounts=np.bincount(a)>>> hist,_=np.histogram(a,range=(0,a.max()),bins=a.max()+1)>>> np.array_equal(hist,bcounts)True>>> # Reproducing `collections.Counter`>>> dict(zip(np.unique(a),bcounts[bcounts.nonzero()])){0: 1, 1: 3, 2: 1, 3: 1, 7: 2, 23: 1}

Visualizing Histograms with Matplotlib and Pandas

Now that you’ve seen how to build a histogram in Python from the ground up, let’s see how other Python packages can do the job for you. Matplotlib provides the functionality to visualize Python histograms out of the box with a versatile wrapper around NumPy’s histogram():

importmatplotlib.pyplotasplt# An "interface" to matplotlib.axes.Axes.hist() methodn,bins,patches=plt.hist(x=d,bins='auto',color='#0504aa',alpha=0.7,rwidth=0.85)plt.grid(axis='y',alpha=0.75)plt.xlabel('Value')plt.ylabel('Frequency')plt.title('My Very Own Histogram')plt.text(23,45,r'$\mu=15, b=3$')maxfreq=n.max()# Set a clean upper y-axis limit.plt.ylim(ymax=np.ceil(maxfreq/10)*10ifmaxfreq%10elsemaxfreq+10)

As defined earlier, a plot of a histogram uses its bin edges on the x-axis and the corresponding frequencies on the y-axis. In the chart above, passing bins='auto' chooses between two algorithms to estimate the “ideal” number of bins. At a high level, the goal of the algorithm is to choose a bin width that generates the most faithful representation of the data. For more on this subject, which can get pretty technical, check out Choosing Histogram Bins from the Astropy docs.

Staying in Python’s scientific stack, Pandas’ Series.histogram()uses matplotlib.pyplot.hist() to draw a Matplotlib histogram of the input Series:

importpandasaspd# Generate data on commute times.size,scale=1000,10commutes=pd.Series(np.random.gamma(scale,size=size)**1.5)commutes.plot.hist(grid=True,bins=20,rwidth=0.9,color='#607c8e')plt.title('Commute Times for 1,000 Commuters')plt.xlabel('Counts')plt.ylabel('Commute Time')plt.grid(axis='y',alpha=0.75)

pandas.DataFrame.histogram() is similar but produces a histogram for each column of data in the DataFrame.

Plotting a Kernel Density Estimate (KDE)

In this tutorial, you’ve been working with samples, statistically speaking. Whether the data is discrete or continuous, it’s assumed to be derived from a population that has a true, exact distribution described by just a few parameters.

A kernel density estimation (KDE) is a way to estimate the probability density function (PDF) of the random variable that “underlies” our sample. KDE is a means of data smoothing.

Sticking with the Pandas library, you can create and overlay density plots using plot.kde(), which is available for both Series and DataFrame objects. But first, let’s generate two distinct data samples for comparison:

>>> # Sample from two different normal distributions>>> means=10,20>>> stdevs=4,2>>> dist=pd.DataFrame(... np.random.normal(loc=means,scale=stdevs,size=(1000,2)),... columns=['a','b'])>>> dist.agg(['min','max','mean','std']).round(decimals=2)          a      bmin   -1.57  12.46max   25.32  26.44mean  10.12  19.94std    3.94   1.94

Now, to plot each histogram on the same Matplotlib axes:

fig,ax=plt.subplots()dist.plot.kde(ax=ax,legend=False,title='Histogram: A vs. B')dist.plot.hist(density=True,ax=ax)ax.set_ylabel('Probability')ax.grid(axis='y')ax.set_facecolor('#d8dcd6')

These methods leverage SciPy’s gaussian_kde(), which results in a smoother-looking PDF.

If you take a closer look at this function, you can see how well it approximates the “true” PDF for a relatively small sample of 1000 data points. Below, you can first build the “analytical” distribution with scipy.stats.norm(). This is a class instance that encapsulates the statistical standard normal distribution, its moments, and descriptive functions. Its PDF is “exact” in the sense that it is defined precisely as norm.pdf(x) = exp(-x**2/2) / sqrt(2*pi).

Building from there, you can take a random sample of 1000 datapoints from this distribution, then attempt to back into an estimation of the PDF with scipy.stats.gaussian_kde():

fromscipyimportstats# An object representing the "frozen" analytical distribution# Defaults to the standard normal distribution, N~(0, 1)dist=stats.norm()# Draw random samples from the population you built above.# This is just a sample, so the mean and std. deviation should# be close to (1, 0).samp=dist.rvs(size=1000)# `ppf()`: percent point function (inverse of cdf — percentiles).x=np.linspace(start=stats.norm.ppf(0.01),stop=stats.norm.ppf(0.99),num=250)gkde=stats.gaussian_kde(dataset=samp)# `gkde.evaluate()` estimates the PDF itself.fig,ax=plt.subplots()ax.plot(x,dist.pdf(x),linestyle='solid',c='red',lw=3,alpha=0.8,label='Analytical (True) PDF')ax.plot(x,gkde.evaluate(x),linestyle='dashed',c='black',lw=2,label='PDF Estimated via KDE')ax.legend(loc='best',frameon=False)ax.set_title('Analytical vs. Estimated PDF')ax.set_ylabel('Probability')ax.text(-2.,0.35,r'$f(x) = \frac{\exp(-x^2/2)}{\sqrt{2*\pi}}$',fontsize=12)

This is a bigger chunk of code, so let’s take a second to touch on a few key lines:

• SciPy’s stats subpackage lets you create Python objects that represent analytical distributions that you can sample from to create actual data. So dist = stats.norm() represents a normal continuous random variable, and you generate random numbers from it with dist.rvs().
• To evaluate both the analytical PDF and the Gaussian KDE, you need an array x of quantiles (standard deviations above/below the mean, for a normal distribution). stats.gaussian_kde() represents an estimated PDF that you need to evaluate on an array to produce something visually meaningful in this case.
• The last line contains some LaTex, which integrates nicely with Matplotlib.

A Fancy Alternative with Seaborn

Let’s bring one more Python package into the mix. Seaborn has a displot() function that plots the histogram and KDE for a univariate distribution in one step. Using the NumPy array d from ealier:

importseabornassnssns.set_style('darkgrid')sns.distplot(d)

The call above produces a KDE. There is also optionality to fit a specific distribution to the data. This is different than a KDE and consists of parameter estimation for generic data and a specified distribution name:

sns.distplot(d,fit=stats.laplace,kde=False)

Again, note the slight difference. In the first case, you’re estimating some unknown PDF; in the second, you’re taking a known distribution and finding what parameters best describe it given the empirical data.

Other Tools in Pandas

In addition to its plotting tools, Pandas also offers a convenient .value_counts() method that computes a histogram of non-null values to a Pandas Series:

>>> importpandasaspd>>> data=np.random.choice(np.arange(10),size=10000,... p=np.linspace(1,11,10)/60)>>> s=pd.Series(data)>>> s.value_counts()9    18318    16247    14236    13235    10894     8883     7702     5351     3470     170dtype: int64>>> s.value_counts(normalize=True).head()9    0.18318    0.16247    0.14236    0.13235    0.1089dtype: float64

Elsewhere, pandas.cut() is a convenient way to bin values into arbitrary intervals. Let’s say you have some data on ages of individuals and want to bucket them sensibly:

>>> ages=pd.Series(... [1,1,3,5,8,10,12,15,18,18,19,20,25,30,40,51,52])>>> bins=(0,10,13,18,21,np.inf)# The edges>>> labels=('child','preteen','teen','military_age','adult')>>> groups,_=pd.cut(ages,bins=bins,labels=labels,retbins=True)>>> groups.value_counts()child           6adult           5teen            3military_age    2preteen         1dtype: int64>>> pd.concat((ages,groups),axis=1).rename(columns={0:'age',1:'group'})    age         group0     1         child1     1         child2     3         child3     5         child4     8         child5    10         child6    12       preteen7    15          teen8    18          teen9    18          teen10   19  military_age11   20  military_age12   25         adult13   30         adult14   40         adult15   51         adult16   52         adult

What’s nice is that both of these operations ultimately utilize Cython code that makes them competitive on speed while maintaining their flexibility.

Alright, So Which Should I Use?

At this point, you’ve seen more than a handful of functions and methods to choose from for plotting a Python histogram. How do they compare? In short, there is no “one-size-fits-all.” Here’s a recap of the functions and methods you’ve covered thus far, all of which relate to breaking down and representing distributions in Python:

You can also find the code snippets from this article together in one script at the Real Python materials page.

With that, good luck creating histograms in the wild. Hopefully one of the tools above will suit your needs. Whatever you do, just don’t use a pie chart.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Anaconda, Inc., the most popular Python data science platform provider with 2.5 million downloads per month, is pleased to announce a new partnership with Full Spectrum Analytics, a data science consultancy that applies vast industry experience and advanced analytics capabilities to help lending businesses and retail banks leverage their own data to grow resiliently and …
Read more →

The post Anaconda and Full Spectrum Analytics Partner to Deliver Enterprise Data Science to Banks, Lenders, and Investments Firms appeared first on Anaconda.

Zato 3.0 has just been released - this is a major release that brings a lot of immensely interesting API integration features.

Zato is an enterprise open-source Python-based ESB/SOA/API integration platform and backend application server.

It is designed specifically for construction of online, mobile, IoT, middleware and backend Python systems running in banking, telecommunications, health-care, insurance, education, public administration or other environments that require integrations of multiple applications.

Highlight of the release are:

• Addition of a built-in multi-protocol message broker with publish/subscribe topics and guaranteed delivery queues

• Single Sign-On and user management API

• Security mechanisms to help in achieving compliance with PII regulations (e.g. GDPR, HIPAA or PDPA)

• SAP connections

• IDE plugins for PyCharm and Visual Studio Code

• WebSocket channels

• New caching layer and Memcached connections

• Automatic API documentation generation - Sphinx, OpenAPI/Swagger and WSDL

• SMS messaging with Twilio

• HashiCorp Vault security definitions

• JSON Web Token security definitions (JWT)

• Extended hot-deployment to work with static files, including JSON, XML, CSV or any other format

• AMQP, ZeroMQ and IBM MQ connections can now be synchronous

• New crypto mechanisms: encryption and decryption, hashing as well as generation of passwords and secrets

• General performance improvements let Zato servers run up to several times faster - applies to HTTP, AMQP, ZeroMQ and IBM MQ connections

Here is the documentation, downloads and full changelog.

Hi everyone! I am super proud to announce that my second book “Practical Python Projects“ is alhamdulillah almost halfway done content-wise. This book is a direct result of a market gap which I found when I started programming a couple of years back. Most programming beginners have access to multiple resources to teach themselves the basics of Python (or any other language) but they don’t have access to those books/tutorials where they are taught to create end-to-end projects in a specific language. A fair amount of online tutorials and beginner textbooks teach the reader only the basics of a language without real-life concrete project development.

Please sign up for my newsletter to get updates on the book (I send less than 2 emails per month): http://newsletter.pythontips.com/

Through my new book, you will be able to take your Python programming skills/knowledge to the next level by developing 15+ projects from scratch. These are bite-sized projects, meaning that you can implement each one of them during a weekend. These projects will not be just throwaway projects, you will actually be able to list them in your portfolio when applying for jobs.

FIFA World Cup bot. You will learn how to send latest, upcoming and past match updates via SMS.

Creating a Facebook & Soundcloud video/music downloader. You will learn how to log into Facebook and download private videos as well

Making a Reddit + Facebook messenger bot. You will learn how the Facebook messenger bot system works. We will implement a fun chatbot which will send you jokes and motivational posts

Controlling your system with Alexa. You will learn how to create an Alexa skill and use Facebook’s image recognition API (we will reverse engineer it ) to recognize people in photos

Scraping data from Steam. You will learn how to create automated data extraction software. We will focus on extracting data from Steam in this particular chapter.

Article summary & Instagram stories upload. You will learn how to use state-of-the-art algorithms to generate 10 line (or more) summaries from online/offline articles. We will overlay these summaries over images extracted from online articles and then we will post these images to Instagram as stories.

Cinema pre-show generator. You will learn video manipulation. Normally when you go to cinemas you see trailers for upcoming movies before the original movie starts. This project has a similar motive. Just provide it with a movie name and it will download trailers for 2 (or more) upcoming movies which have the same genre. It will then merge these trailers with a “put your cell phones on silent” msg and an iconic video countdown.

Automatic Invoice generator. You will learn how to generate PDF invoices automatically. I will also teach you about task queues and how to make effective use of them. Lastly, you will learn how to implement email sending functionality in Python (Flask).

I have completed 10 chapters (each chapter == 1 project) amounting to almost 140 pages of content. It is already longer than my last book (yayy!!). I hope that inshAllah this book will be completed within the next couple of months.

---

Please sign up for my newsletter to get updates on the book (I send less than 2 emails per month): http://newsletter.pythontips.com/

---

P.S: The final book cover might be different from this one. If you have any suggestions please let me know

Have a great day!

The following text is in German, since we're announcing a regional user group meeting in Düsseldorf, Germany.

Ankündigung

Das nächste Python Meeting Düsseldorf findet an folgendem Termin statt:

04.07.2018, 18:00 Uhr
Raum 1, 2.OG im Bürgerhaus Stadtteilzentrum Bilk
Düsseldorfer Arcaden, Bachstr. 145, 40217 Düsseldorf

Neuigkeiten

Bereits angemeldete Vorträge

Startzeit und Ort

Wir treffen uns um 18:00 Uhr im Bürgerhaus in den Düsseldorfer Arcaden.

Das Bürgerhaus teilt sich den Eingang mit dem Schwimmbad und befindet sich an der Seite der Tiefgarageneinfahrt der Düsseldorfer Arcaden.

Über dem Eingang steht ein großes "Schwimm’ in Bilk" Logo. Hinter der Tür direkt links zu den zwei Aufzügen, dann in den 2. Stock hochfahren. Der Eingang zum Raum 1 liegt direkt links, wenn man aus dem Aufzug kommt.

>>> Eingang in Google Street View

Einleitung

Das Python Meeting Düsseldorf ist eine regelmäßige Veranstaltung in Düsseldorf, die sich an Python Begeisterte aus der Region wendet.

Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting & Research, Düsseldorf:

Programm

Das Python Meeting Düsseldorf nutzt eine Mischung aus (Lightning) Talks und offener Diskussion.

(Lightning) Talk Anmeldung bitte formlos per EMail an info@pyddf.de

Kostenbeteiligung

Das Python Meeting Düsseldorf wird von Python Nutzern für Python Nutzer veranstaltet.

Da Tagungsraum, Beamer, Internet und Getränke Kosten produzieren, bitten wir die Teilnehmer um einen Beitrag in Höhe von EUR 10,00 inkl. 19% Mwst. Schüler und Studenten zahlen EUR 5,00 inkl. 19% Mwst.

Wir möchten alle Teilnehmer bitten, den Betrag in bar mitzubringen.

Anmeldung

Da wir nur für ca. 20 Personen Sitzplätze haben, möchten wir bitten, sich per EMail anzumelden. Damit wird keine Verpflichtung eingegangen. Es erleichtert uns allerdings die Planung.

Meeting Anmeldung bitte formlos per EMail an info@pyddf.de

Weitere Informationen

Weitere Informationen finden Sie auf der Webseite des Meetings:

              http://pyddf.de/

Viel Spaß !

Marc-Andre Lemburg, eGenix.com

Over the last few weeks, our program WG has been working hard on getting the schedule all lined up. Today, we’re releasing it to the Python world.

Schedule for EuroPython 2018

With 140 speakers and more than 150 sessions, we have a full packed program waiting for you. Please note that the schedule may still change in details, but the overall layout is fixed now.

Book your EuroPython 2018 Ticket

Please make sure you book your ticket in the coming days. We will switch to late bird rates closer to the event.

If you want to attend the training sessions, please buy a training pass. We only have very few left and will close sales for these later this week.

Travel & accommodation tips

Since we’re close the conference and The Fringe is starting a week later, Edinburgh is in high demand. If you’re having problems finding a hotel, please also consider searching for apartments on the well known booking sites.

For traveling to Edinburgh, we suggest also considering a combination of plane and train or bus. London, Birmingham and Manchester all provide train and bus lines going to Edinburgh and by booking a combination, you can often save a lot, compared to a direct flight to Edinburgh.

Enjoy,
–
EuroPython 2018 Team
https://ep2018.europython.eu/  
https://www.europython-society.org/

Over the last few weeks, our program WG has been working hard on getting the schedule all lined up. Today, we’re releasing it to the Python world.

Schedule for EuroPython 2018

With 140 speakers and more than 150 sessions, we have a full packed program waiting for you. Please note that the schedule may still change in details, but the overall layout is fixed now.

Book your EuroPython 2018 Ticket

Please make sure you book your ticket in the coming days. We will switch to late bird rates closer to the event.

If you want to attend the training sessions, please buy a training pass. We only have very few left and will close sales for these later this week.

Travel & accommodation tips

Since we’re close the conference and The Fringe is starting a week later, Edinburgh is in high demand. If you’re having problems finding a hotel, please also consider searching for apartments on the well known booking sites.

For traveling to Edinburgh, we suggest also considering a combination of plane and train or bus. London, Birmingham and Manchester all provide train and bus lines going to Edinburgh and by booking a combination, you can often save a lot, compared to a direct flight to Edinburgh.

Enjoy,
–
EuroPython 2018 Team
https://ep2018.europython.eu/  
https://www.europython-society.org/

Introduction

One of the most common problems that we face in software development is handling dates and times. After getting a date-time string from an API, for example, we need to convert it to a human-readable format. Again, if the same API is used in different timezones, the conversion will be different. A good date-time library should convert the time as per the timezone.

Thankfully, Python comes with the built-in module datetime for dealing with dates and times. It comes with various functions for manipulating dates and times. Using this module, we can easily parse any date-time string and convert it to a datetime object.

Converting Strings using datetime

The datetime module consists of three different object types: date, time and datetime. As you may have guessed, date holds the date, time holds the time, and datetime holds both date and time.

For example, the following example will print the current date and time:

import datetime

print ('Current date/time: {}'.format(datetime.datetime.now()))  

Running this code will print an output similar to below:

$ python3 datetime-print-1.py
Current date/time: 2018-06-29 08:15:27.243860  

When no custom formatting is given, the default string format is used, i.e. the format for "2018-06-29 08:15:27.243860" is in ISO 8601 format (YYYY-MM-DDTHH:MM:SS.mmmmmm). If our input string to create a datetime object is in the same ISO 8601 format, we can easily parse it to a datetime object.

Let's take a look at the code below:

import datetime

date_time_str = '2018-06-29 08:15:27.243860'  
date_time_obj = datetime.datetime.strptime(date_time_str, '%Y-%m-%d %H:%M:%S.%f')

print('Date:', date_time_obj.date())  
print('Time:', date_time_obj.time())  
print('Date-time:', date_time_obj)  

Running it will print the below output:

$ python3 datetime-print-2.py
Date: 2018-06-29  
Time: 08:15:27.243860  
Date-time: 2018-06-29 08:15:27.243860  

In this example, we are using a new method called strptime. This method takes two arguments: the first one is the string representation of the date-time and the second one is the format of the input string. The return value is of the type datetime. In our example, "2018-06-29 08:15:27.243860" is the input string and "%Y-%m-%d %H:%M:%S.%f" is the format of this date string. The returned datetime value is stored in date_time_obj variable. Since this is a datetime variable, we can call date() and time() methods directly on it. As you can see from the output, it prints the 'date' and 'time' part of the input string.

You might be wondering what is the meaning of the format "%Y-%m-%d %H:%M:%S.%f". These are known as format tokens with different meaning for each token. Check out the strptime documentation for the list of all different types of format code supported in Python.

So, if the format of a string is known, it can be easily parsed to a datetime object using strptime. Let me show you one more non-trivial example:

import datetime

date_time_str = 'Jun 28 2018  7:40AM'  
date_time_obj = datetime.datetime.strptime(date_time_str, '%b %d %Y %I:%M%p')

print('Date:', date_time_obj.date())  
print('Time:', date_time_obj.time())  
print('Date-time:', date_time_obj)  

From the following output you can see that the string was successfully parsed since it is being properly printed by the datetime object here.

$ python3 datetime-print-3.py
Date: 2018-06-28  
Time: 07:40:00  
Date-time: 2018-06-28 07:40:00  

Here are a few more examples of commonly used time formats and the tokens used for parsing:

"Jun 28 2018 at 7:40AM" -> "%b %d %Y at %I:%M%p""September 18, 2017, 22:19:55" -> "%B %d, %Y, %H:%M:%S""Sun,05/12/99,12:30PM" -> "%a,%d/%m/%y,%I:%M%p""Mon, 21 March, 2015" -> "%a, %d %B, %Y""2018-03-12T10:12:45Z" -> "%Y-%m-%dT%H:%M:%SZ"

You can parse a date-time string of any format using the table mentioned in the strptime documentation.

Dealing with Timezones and datetime

Handling date-times becomes more complex while dealing with timezones. All above examples we have discussed are naive datetime objects, i.e. these objects don't contain any timezone-related data. The datetime object has one variable tzinfo, that holds the timezone information.

import datetime as dt

dtime = dt.datetime.now()  
print(dtime)  
print(dtime.tzinfo)  

This code will print:

$ python3 datetime-tzinfo-1.py
2018-06-29 22:16:36.132767  
None  

The output of tzinfo is None since it is a naive datetime object. For timezone conversion, one library called pytz is available for Python. You can install it as described in these instructions. Now, let's use the pytz library to convert the above timestamp to UTC.

import datetime as dt  
import pytz

dtime = dt.datetime.now(pytz.utc)  
print(dtime)  
print(dtime.tzinfo)  

Output:

$ python3 datetime-tzinfo-2.py
2018-06-29 17:08:00.586525+00:00  
UTC  

+00:00 is the difference between the displayed time and the UTC time. In this example the value of tzinfo happens to be UTC as well, hence the 00:00 offset. In this case, the datetime object is a timezone-aware object.

Similarly, we can convert date-time strings to any other timezone. For example, we can convert the string "2018-06-29 17:08:00.586525+00:00" to "America/New_York" timezone, as shown below:

import datetime as dt  
import pytz

date_time_str = '2018-06-29 17:08:00'  
date_time_obj = dt.datetime.strptime(date_time_str, '%Y-%m-%d %H:%M:%S')

timezone = pytz.timezone('America/New_York')  
timezone_date_time_obj = timezone.localize(date_time_obj)

print(timezone_date_time_obj)  
print(timezone_date_time_obj.tzinfo)  

Output:

$ python3 datetime-tzinfo-3.py
2018-06-29 17:08:00-04:00  
America/New_York  

First, we have converted the string to a datetime object, date_time_obj. Then we converted it to a timezone-enabled datetime object, timezone_date_time_obj. Since we have mentioned the timezone as "America/NewYork", the output time shows that it is _4 hours behind than UTC time. You can check this Wikipedia page to find the full list of available time zones.

Converting Timezones

We can convert timezone of a datetime object from one region to another, as shown in the example below:

import datetime as dt  
import pytz

timezone_nw = pytz.timezone('America/New_York')  
nw_datetime_obj = dt.datetime.now(timezone_nw)

timezone_london = pytz.timezone('Europe/London')  
london_datetime_obj = nw_datetime_obj.astimezone(timezone_london)


print('America/New_York:', nw_datetime_obj)  
print('Europe/London:', london_datetime_obj)  

First, it created one datetime object with the current time on "America/New_York" timezone. Then using astimezone() method, we have converted this datetime to "Europe/London" timezone. Both datetimes will print different values like:

$ python3 datetime-tzinfo-4.py
America/New_York: 2018-06-29 22:21:41.349491-04:00  
Europe/London: 2018-06-30 03:21:41.349491+01:00  

Using Third Party Libraries

Python's datetime module can convert all different types of strings to a datetime object. But the main problem is that in order to do this you need to create the appropriate formatting code string that strptime can understand. Creating this string takes time and it makes the code harder to read. Instead, we can use other third-party libraries to make it easier.

In some cases these third party libraries also have better built-in support for manipulating and comparing date-times, and some even have timezones built-in, so you don't need to include an extra package.

Let's take a look at few of these libraries in the following sections.

dateutil

The dateutil module is an extension to the datetime module. We don't need to pass any parsing code to parse a string. For example:

from dateutil.parser import parse

datetime = parse('2018-06-29 22:21:41')

print(datetime)  

This parse function will parse the string automatically and store it in the datetime variable. Parsing is done automatically. You don't have to mention any format string. Let's try to parse different types of strings using dateutil:

from dateutil.parser import parse

date_array = [  
    '2018-06-29 08:15:27.243860',
    'Jun 28 2018  7:40AM',
    'Jun 28 2018 at 7:40AM',
    'September 18, 2017, 22:19:55',
    'Sun, 05/12/1999, 12:30PM',
    'Mon, 21 March, 2015',
    '2018-03-12T10:12:45Z',
    '2018-06-29 17:08:00.586525+00:00',
    '2018-06-29 17:08:00.586525+05:00',
    'Tuesday , 6th September, 2017 at 4:30pm'
]

for date in date_array:  
    print('Parsing: ' + date)
    dt = parse(date)
    print(dt.date())
    print(dt.time())
    print(dt.tzinfo)
    print('\n')

Output:

$ python3 dateutil-1.py
Parsing: 2018-06-29 08:15:27.243860  
2018-06-29  
08:15:27.243860  
None

Parsing: Jun 28 2018  7:40AM  
2018-06-28  
07:40:00  
None

Parsing: Jun 28 2018 at 7:40AM  
2018-06-28  
07:40:00  
None

Parsing: September 18, 2017, 22:19:55  
2017-09-18  
22:19:55  
None

Parsing: Sun, 05/12/1999, 12:30PM  
1999-05-12  
12:30:00  
None

Parsing: Mon, 21 March, 2015  
2015-03-21  
00:00:00  
None

Parsing: 2018-03-12T10:12:45Z  
2018-03-12  
10:12:45  
tzutc()

Parsing: 2018-06-29 17:08:00.586525+00:00  
2018-06-29  
17:08:00.586525  
tzutc()

Parsing: 2018-06-29 17:08:00.586525+05:00  
2018-06-29  
17:08:00.586525  
tzoffset(None, 18000)

Parsing: Tuesday , 6th September, 2017 at 4:30pm  
2017-09-06  
16:30:00  
None  

You can see that almost any type of string can be parsed easily using the dateutil module.

Maya

Maya also makes it very easy to parse a string and for changing timezones. Some simple examples are shown below:

import maya

dt = maya.parse('2018-04-29T17:45:25Z').datetime()  
print(dt.date())  
print(dt.time())  
print(dt.tzinfo)  

Output:

$ python3 maya-1.py
2018-04-29  
17:45:25  
UTC  

For converting the time to a different timezone:

import maya

dt = maya.parse('2018-04-29T17:45:25Z').datetime(to_timezone='America/New_York', naive=False)  
print(dt.date())  
print(dt.time())  
print(dt.tzinfo)  

Output:

$ python3 maya-2.py
2018-04-29  
13:45:25  
America/New_York  

Now isn't that easy to use? Let's try out maya with the same set of strings we have used with dateutil:

import maya

date_array = [  
    '2018-06-29 08:15:27.243860',
    'Jun 28 2018  7:40AM',
    'Jun 28 2018 at 7:40AM',
    'September 18, 2017, 22:19:55',
    'Sun, 05/12/1999, 12:30PM',
    'Mon, 21 March, 2015',
    '2018-03-12T10:12:45Z',
    '2018-06-29 17:08:00.586525+00:00',
    '2018-06-29 17:08:00.586525+05:00',
    'Tuesday , 6th September, 2017 at 4:30pm'
]

for date in date_array:  
    print('Parsing: ' + date)
    dt = maya.parse(date).datetime()
    print(dt)
    print(dt.date())
    print(dt.time())
    print(dt.tzinfo)

Output:

$ python3 maya-3.py
Parsing: 2018-06-29 08:15:27.243860  
2018-06-29 08:15:27.243860+00:00  
2018-06-29  
08:15:27.243860  
UTC

Parsing: Jun 28 2018  7:40AM  
2018-06-28 07:40:00+00:00  
2018-06-28  
07:40:00  
UTC

Parsing: Jun 28 2018 at 7:40AM  
2018-06-28 07:40:00+00:00  
2018-06-28  
07:40:00  
UTC

Parsing: September 18, 2017, 22:19:55  
2017-09-18 22:19:55+00:00  
2017-09-18  
22:19:55  
UTC

Parsing: Sun, 05/12/1999, 12:30PM  
1999-05-12 12:30:00+00:00  
1999-05-12  
12:30:00  
UTC

Parsing: Mon, 21 March, 2015  
2015-03-21 00:00:00+00:00  
2015-03-21  
00:00:00  
UTC

Parsing: 2018-03-12T10:12:45Z  
2018-03-12 10:12:45+00:00  
2018-03-12  
10:12:45  
UTC

Parsing: 2018-06-29 17:08:00.586525+00:00  
2018-06-29 17:08:00.586525+00:00  
2018-06-29  
17:08:00.586525  
UTC

Parsing: 2018-06-29 17:08:00.586525+05:00  
2018-06-29 12:08:00.586525+00:00  
2018-06-29  
12:08:00.586525  
UTC

Parsing: Tuesday , 6th September, 2017 at 4:30pm  
2017-09-06 16:30:00+00:00  
2017-09-06  
16:30:00  
UTC  

As you can see, all date formats were parsed, but did you notice the difference? If we are not providing the timezone info it automatically converts it to UTC. So, it is important to note that we need to provide to_timezone and naive parameters if the time is not in UTC.

Arrow

Arrow is another library for dealing with datetime in Python. We can get the Python datetime object from an arrow object. Let's try this with the same example string we have used for maya:

import arrow

dt = arrow.get('2018-04-29T17:45:25Z')  
print(dt.date())  
print(dt.time())  
print(dt.tzinfo)  

Output:

$ python3 arrow-1.py
2018-04-29  
17:45:25  
tzutc()  

Timezone conversion:

import arrow

dt = arrow.get('2018-04-29T17:45:25Z').to('America/New_York')  
print(dt)  
print(dt.date())  
print(dt.time())  

Output:

$ python3 arrow-2.py
2018-04-29T13:45:25-04:00  
2018-04-29  
13:45:25  

As you can see the date-time string is converted to the "America/New_York" region.

Now, let's again use the same set of strings we have used above:

import arrow

date_array = [  
    '2018-06-29 08:15:27.243860',
    #'Jun 28 2018  7:40AM',
    #'Jun 28 2018 at 7:40AM',
    #'September 18, 2017, 22:19:55',
    #'Sun, 05/12/1999, 12:30PM',
    #'Mon, 21 March, 2015',
    '2018-03-12T10:12:45Z',
    '2018-06-29 17:08:00.586525+00:00',
    '2018-06-29 17:08:00.586525+05:00',
    #'Tuesday , 6th September, 2017 at 4:30pm'
]

for date in date_array:  
    dt = arrow.get(date)
    print('Parsing: ' + date)
    print(dt)
    print(dt.date())
    print(dt.time())
    print(dt.tzinfo)

This code will fail for the date-time strings that have been commented out. The output for other strings will be:

$ python3 arrow-3.py
Parsing: 2018-06-29 08:15:27.243860  
2018-06-29T08:15:27.243860+00:00  
2018-06-29  
08:15:27.243860  
tzutc()

Parsing: 2018-03-12T10:12:45Z  
2018-03-12T10:12:45+00:00  
2018-03-12  
10:12:45  
tzutc()

Parsing: 2018-06-29 17:08:00.586525+00:00  
2018-06-29T17:08:00.586525+00:00  
2018-06-29  
17:08:00.586525  
tzoffset(None, 0)

Parsing: 2018-06-29 17:08:00.586525+05:00  
2018-06-29T17:08:00.586525+05:00  
2018-06-29  
17:08:00.586525  
tzoffset(None, 18000)  

In order to correctly parse the date-time strings that I have commented out, you'll need to pass the corresponding format tokens. For example, "MMM" for months name, like "Jan, Feb, Mar" etc. You can check this guide for all available tokens.

Conclusion

We have checked different ways to parse a string to a datetime object in Python. You can either opt for the default Python datetime library or any of the third party library mentioned in this article, among many others. The main problem with default datetime package is that we need to specify the parsing code manually for almost all date-time string formats. So, if your string format changes in the future, you will likely have to change your code as well. But many third-party libraries, like the ones mentioned here, handle it automatically.

One more problem we face is dealing with timezones. The best way to handle them is always to store the time in UTC format on the server and convert it to the user's local timezone while parsing. Not only for parsing string, these libraries can be used for a lot of different types of date-time related operations. I'd encourage you to go through the documents to learn the functionalities in detail.

Keith has created a colourful and noisy project with Mu and an Adafruit Circuit Playground Express (CPE) board. Just what you need for the 4th July!

By day Keith is a healthcare informatics professional. By night, like many of us, he builds robots and other Maker-type projects. This particular project blinks different messages in Morse code. You can configure the LED colour as well as toggle the buzzer (so the board “bloops” Morse code too).

He says, “I love the idea that I could use Python to program a microcontroller” and explains the origin of the project as follows:

I was at the San Mateo Maker Faire this year (I was at the first one, and have been there or NYC Maker Faire every year since). I had brought the CPE with me to play with in case I had any free time (I hadn’t had time to really mess with it - big project at work). My flight Sunday night was cancelled, so I thought I might go to the Bring-a-Hack event. I didn’t want to show up empty handed, but all I could think up was a lame demo of the CPE. I quickly re-reviewed some tutorials, and wrote a quick program in my hotel room. Didn’t work at all. Tried different things, cut-and-paste sample code (which all worked), finally gave up. At that point I was tired, and had to get up early for a flight, so I canned it. When I got back to St. Louis and read through the code again, I realized I had thought that time.sleep was in milliseconds (like Arduino Delay), rather than in seconds. The original code I had written was actually fine - just a thousand times slower than I was expecting. Once I fixed that, it was fine. I’ve since cleaned it up a little.

Thus proving the old adage that, “good things come to those who wait”. ;-)

Keith has helpfully provided a video of his code in action:

The project’s source code contains a complete implementation of a Morse code transmitter that could form the basis for many future beginner-friendly projects. If you’re interested in changing the transmitted messages you should modify lines 92 and 95. The Morse code definition is in a Python dictionary on line 22 and the loop, starting on line 98, is a great example of how to convert characters into Morse code. Check it out!

I wonder, how hard might it be to implement simple signal processing using the Circuit Playground Express’s microphone or light sensor to detect the message transmitted from a second device and emit the result to the CircuitPython REPL? (That’s a trick question… it’s very easy, but I leave it as an exercise for the reader to solve.)

As for Mu, Keith says it is, “certainly convenient for programming the CPE (and BBC micro:bit). The biggest thing I miss is a keyboard shortcut to comment/uncomment bits of code. That would make troubleshooting so much faster.” Check out progress for this feature request on our issue tracker.

Thank you Keith for such an easy-to-follow Morse code project!

Everyone else, if you extend or re-use this code, let us know! We love to shine a light on the hard work of Mu users (no pun intended). :-)