Charla en el Buenos Aires Python Meetup el 7/11/2019 Algunos tips y comentarios sarcásticos para hacer que tu código Python sea más parecido a Python y menos parecido a otra cosa.

Yesterday, just for fun, I added an emoji (as a local emoji) to our mastodon instance, kind of look like a verification icon. Only to show the power of federation and Free Software to the new users of Mastodon. Many other users started using the same. But, most newbies to Mastodon got confused with the same

If you see the above screenshot, I just added :verified: after my name in my profile, and that shows the icon. That is not any formal verification. The https://msdtn.social instance also has a similar emoji, but the big https://mastodon.social does not have the same.

Then how to verify someone on Mastodon?

The best way is via their website or blog. If you click to edit your profile, you will find the option to add a few profile metadata, in those sites, you can verify that you own or have edit access of those sites.

Thus, my blog comes with a green tick on my profile image. Here is to the link verification documentation from the website.

Feel free to follow me on Mastodon.

oqubeslogging is a proof of concept project for Qubes OS. This includes a qrexec service, which allows one AppVM (we will call it logging VM for the rest of the blog post) to have all the logs from different select VMs. This enables central logging among QubesOS land.

The second part is a Python logging handler, which if configured correctly, will allow to pass the logging lines to the logging VM

import logging
from oqubeslogging import OQubesLog

def main():
    handler = OQubesLog("workvm", "loggingvm")
    logging.basicConfig(level=logging.DEBUG, handlers=[handler])
    logger = logging.getLogger("example")

    logger.info("kushal says it works")


if __name__ == "__main__":
    main()

Third, is another example code, which if run as root user in any VM, will collect all logs from journald and pass them to the logging VM.

Why is this useful?

The same reason we use central logging in every place. This will enable us to check only one VM to find the correct log. QubesOS is designed for single user in mind, and instead many random AppVM, we will be able to read and parse logs in that one special logging VM.

We’re very excited to announce the Beta release for PyCharm 2019.3, a feature-complete preview of the upcoming release. Give the Beta build a go and try all the new functionality – download it from our website.

Improvements in PyCharm 2019.3 Beta

• The issue causing the UI to be unresponsive when an exception was raised while using Python console completion was fixed.
• We’ve solved the problem causing the Clear All action for the console to raise some errors.
• Auto generated test files from templates (unittest and pytest) now support classes without methods to generate test files.
• If you’re using symlinks in your tests you will find that stack traces now consistently will show the symlink instead of the real path in the console.

Further Improvements

For more details on what’s new in this version, please refer to the release notes.

Interested?

Get the PyCharm 2019.3 Beta build from our website. Alternatively, you can use the JetBrains Toolbox App to stay up to date with early builds.
PyCharm 2019.3 Beta is part of Early Access Program, so the EAP rules apply.

EAP Program Key Facts

• The EAP version of PyCharm Professional Version is free to use
• EAP build will expire after 30 days
• This is pre-release software, you may face stability issues and other rough edges
• You can install the EAP version alongside a stable version of PyCharm
• EAP versions of PyCharm report statistics by default, you can opt out by changing the settings in Preferences | Appearance & Behavior | System Settings | Data Sharing
• The latest version of the documentation is now also available

This week we had a webinar with Maxim Sobolevskiy, the DataGrip Product Marketing Manager, showing the wonderful, magical Database tool in PyCharm. The webinar recording is now available.

In this webinar Max covered many of the amazing features of DataGrip: connections and connection settings, queries with autocomplete and refactoring, working with results, browsing/querying/editing data, export/import, and more. Paul then showed a simple Flask application and how the Database tool is injected into Python code.

If you work at all with SQL in Python projects, take some time to watch the webinar. It’s powerful stuff and will deeply improve your velocity.

The webinar has a sample repository with a Docker Compose setup for the sample PostgreSQL server, along with instructions to recreate the Flask project.

In brief: if you have an account on www.pythonanywhere.com you can have it migrated to eu.pythonanywhere.com -- just let us know via email to support@pythonanywhere.com.

If you'd like to know more about what that means, read on...

Back in February, we announced eu.pythonanywhere.com. It's a completely separate version of our site, with all of the computers and storage hosted in Frankfurt, rather than in the US like our other site at www.pythonanywhere.com.

Although our US-based systems are fully GDPR-compliant, thanks to the EU-US Privacy Shield Framework, we appreciate that some people are keen on keeping all of their data inside the EU so as to be sure that they comply with all regulations.

If you're based in the EU, there's almost no downside to using the new system. You will have the comfort of knowing your data is in the EU, and it's closer to you in network terms (in our tests, the network latency is about 8ms from Amsterdam, versus 90ms to our US servers). For paid accounts, billing is in euros so you don't need to worry about foreign exchange fees on your card payments. If we need to perform system maintenance on the servers, we'll do it late at night European time, rather than during the early morning timeslot that we use for the US-based system. The only reason you might want to stick with the US system is that it's a little cheaper; our underlying hosting costs are a bit higher for the EU service, which is reflected in the prices -- a Hacker account that costs US$5/month on the US servers is €5/month on our new system (plus VAT if applicable).

The new system has a good number of users now and is functioning well. We've also been beta-testing a migration system that allows us to move accounts from the US system to the EU one with minimal downtime, and that is working well too :-)

So it's time for us to take that migration system out of beta -- if you have an account on www.pythonanywhere.com and would like it to be moved to eu.pythonanywhere.com, just let us know via email to support@pythonanywhere.com.

Let us know if you have any questions!

1. Why did dict.get(key) work but not dict[key]? - [15/3]
2. How can I use functional programming to make a generic method in python? - [8/1]
3. How is `var[:] = []` different from `var = []`? - [7/3]
4. dtypes muck things up when shifting on axis one (columns) - [7/2]
5. How to specify index of specific elements in every sublist of nested list? - [6/4]
6. How to remove substrings from a python list - [6/2]
7. How do I ensure that a generator gets properly closed? - [6/2]
8. Python multiprocessing: Running a process after its parent exited - [6/1]
9. Fast way to convert upper triangular matrix into symmetric matrix - [5/3]
10. What does "import" prefer - .pyd (.so) or .py? - [5/2]

At PyGotham 2019 we provided live captioning and, for the first time, we offered American Sign Language interpretation and did targeted outreach to groups of Deaf programmers. As a result, we had a half-dozen Deaf attendees, and they reported they were able to fully participate in the conference in a way they hadn’t experienced before. I led our effort to provide ASL and captioning; I hope this recap can serve as a playbook for other conferences.

• Security fixes for HTTP/2 -- CVE-2019-9512 (Ping Flood), CVE-2019-9514 (Reset Flood), and CVE-2019-9515 (Settings Flood).  Thanks to Jonathan Looney and Piotr Sikora.
• HTTP/2 fixes regarding timeouts.
• trial's assertResultOf, failureResultOf, and successResultOf, now accept Deferred-awaiting coroutines.
• Various other bug fixes for POP3, conch.ssh.keys, and twisted.web.client.FileBodyProducer.

The post PyDev of the Week: Vuyisile Ndlovu appeared first on The Mouse Vs. The Python.

The Python Software FoundationPackaging Working Group is receiving funding to work on the design, implementation, and rollout of pip's next-generation dependency resolver. (We'll be able to publicly name the funders later this month and in early December.)

pip is the official package installer for Python. pip aims to make it easy for the millions of people who use Python to download and install Python libraries and applications (open source and closed source, source and binary, globally and within isolated virtual environments). It's a foundational component of the Python ecosystem and broader computer software and technology landscape.

This project aims to complete the design, implementation, and rollout of pip's next-generation dependency resolver. This will lower the barriers to installing Python software, empowering users to get a version of a package that works. It will also lower the barriers to distributing Python software, empowering developers to make their work available in an easily reusable form.

Because of the size of the project, funding has been allocated to secure two contractors, a senior developer and an intermediate developer, to work on development, testing and building test infrastructure, code review, bug triage, and assisting in the rollout of necessary features.

We plan to begin the project in Quarter 1 of 2020. We are also interested in senior developer candidates who are available to start in December 2019.

Timeline and budget

Role 1: We seek a senior Python developer, work starting in mid-December 2019 or early January 2020, work ending at the end of May 2020. Pay: $116,375 total (665 hours of work at $175 per hour).

Role 2: We seek an intermediate-to-senior Python developer, work starting in early January 2020, till the end of December 2020. Pay: $103,700 (670 hours of work at $150 per hour), plus $1600 budgeted for onboarding travel and $1600 budgeted for PyCon travel.

What is the Request for Proposals period?

A Request for Proposals (RFP) is a process intended to allow us (the Python Software Foundation) to collect proposals from potential contractors and select contractor(s) best suited to fulfill the specified work.

After the RFP period closes we will evaluate the received proposals based on the evaluation criteria, seek clarification from proposers as necessary, and select one or more contractors to complete the work specified in the scope section.

Note: This Request For Proposal document may be updated to reflect things that we learn during the process. The canonical version and history is available here.

How do I submit a proposal?

First, please read the full contents of the Request for Proposals!

You'll find the instructions for submission, evaluation criteria, as well as the scopes of the two roles, there.

PyCon Africa was a wonderful, inspiring, and technically enlightening conference that took place in Accra, Ghana from August 6 to 10, 2019 at the University of Ghana. This conference was the very first pan-African conference for Python developers and was attended by 323 Pythonistas from 26 different countries. Most of the attendees traveled from countries around Africa, and a number of speakers came from the US, the Netherlands, Germany, Brazil, and Italy.

Python is becoming more and more adopted all across the globe. In Africa, Python is earning a special place for itself, where it’s used extensively for web development and data science. African businesses are looking for developers with Python skills in these areas, and having a PyCon in Africa provides a foundation to help support African programmers.

What Happened at PyCon Africa

On the day before the main conference started, attendees took some time to settle in. Those who were just getting started with Python attended a Beginners Day Session led by Joey Darko to help them set up their computers for Python development. I joined another group of international travelers on a tour of Accra, Ghana. We visited many tourist sites such as the Independence Square, the cultural center, and some of the local markets to get a feel for the Ghanaian way of life.

The following day was dedicated to workshops, with two running concurrently in both the morning and afternoon sessions. The Django Girls workshop introduced women and Django beginners to Django development. Other workshops were focused on concepts like data visualization, testing, and iterators.

Main Conference

The main conference started on August 8 and kicked off with an opening ceremony that featured traditional Ghanaian dance and welcoming remarks from Marlene Mhangami, the Python Africa Chair. Each day of the conference had an opening and closing keynote talk.

Let’s take a look at some of the talks that were given over the two days of the conference.

Moustapha Cisse: The Potential for Positive Impact Through AI

Moustapha Cisse is the head of Google’s AI Center in Accra and he gave the opening keynote on the first day of the main conference. Moustapha’s talk was about how African technologists can use technology to solve African problems. In his talk, he discussed some of the work he’s involved in at Google and in the AI space, like using Artificial Intelligence for disease prediction, and satellite images to gather information and statistics for AI-enabled flood forecasting.

Meili Triantafyllidi: Lessons Learned From 6 Years in PyLadies Berlin

Meili Triantafyllidi is the co-founder of PyLadies Berlin. There are a number of PyLadies and women-focused Python user groups around Africa. These groups are doing a lot of amazing work, but like anything in life, there’s always room for improvement.

Meili Triantafyllidi gave a timely talk on the lessons she learned from six years of running PyLadies Berlin. In her talk, she shared tips and best practices for making events more inclusive, accessible, and safe. Her talk also covered information on how to run events, get the right speakers, and improve public speaking skills.

A PyLadies meeting was held after Meili’s talk. The meeting established a common purpose and plans to support female programmers in Africa, including through the creation of a dedicated PyLadies Africa channel in the PyLadies Slack group.

Candy Tricia Khohliwe: Web Virtual Reality and A-Frame

Candy Khohliwe, a Mozilla Tech Speaker, gave a talk about WebVR, an open technology that allows you to experience virtual reality in your browser no matter what device you use. The focus of Candy’s talk was A-Frame, a web framework for creating 3D and WebVR experiences using HTML. From this informative talk, I learned how to use A-Frame to create and draw WebVR components. I look forward to trying it out in future projects!

Nicholas Del Grosso: Unravelling the Spaghetti (Tips for Refactoring Your Code)

Nick Del Grosso gave a talk on refactoring code. He explained how project code can get messy and took us through the process of refactoring it safely. He also discussed strategies for identifying dead code, choosing good variable names, and using tools to help refactor code.

Jessica Upani: Is This Python Thing Really Working?

Jessica Upani from the Namibian Python community gave an enlightening talk about the different Python community groups in Namibia. She discussed the challenges they overcame and what lessons other African Python communities can learn from them.

Kelvin Oyana: Bridging the Talent Gap Between Python Communities and Industry

Kelvin’s talk was about the divide between what companies expect from developers and what developers can actually do. He also offered insight into how this gap can be bridged, both by communities and individual developers.

Daniele Procida: The World’s Simplest, Cheapest Plotter

Daniele Procida is a Django core developer and one of the members of the PyCon Africa organizing team. In his talk, Daniele presented an interesting project: a simple pen plotter he built using cardboard, a Rasberry Pi, some servo motors, and Python software. His project embodies the spirit of resourcefulness and innovation that characterizes African enterprise.

Ewa Jodlowska: Keynote

Ewa Jodlowska, the Executive Director of the Python Software Foundation (PSF), gave a keynote talk about the PSF, how it works with the global community, and its plans for the future. Ewa shed more light on the different programs the PSF runs to foster the growth of the Python community, such as the Sponsorship Program and the Python Ambassador Program in South America.

Anna Makarudze: What the African Developer Can Do to Increase Diversity in Tech

Anna Makarudze gave the closing keynote on the first day of the main conference. She’s a Software Engineer at BriteCore (one of the diamond sponsors of PyCon Africa), the Vice President of the Django Software Foundation, and the Fundraising Coordinator for the Django Girls Foundation. Anna drew on her experiences in these organizations to discuss how conferences, organizations, and individual developers can increase diversity figures in tech.

Panel Discussion: The Role of the African Python Developer in the Community

There was a panel discussion of leaders in business, technology, and community development. Marlene Manghami moderated the discussion on the role of the African Python community in the global tech space. Her panelists were Solomon Apenya, Daniel Roy Greenfeld, and James Yankah.

Solomon Apenya is a Senior Consultant at Andela, a company that finds and builds distributed engineering teams in Africa. Daniel Roy Greenfeld is co-author of Two Scoops of Django and VP of Implementation Engineering at BriteCore. James Yankah is the Managing Director of Brompton Group, a group of companies that offer recruitment, training, management, logistics, and security services.

Anthony Shaw: Standing Out in a World of 20 Million Developers

On the last day of the conference, I had the pleasure of meeting and listening to Anthony Shaw, a fellow Real Python author. Anthony gave a talk targeted at new developers to help them succeed in their development careers. In the talk, Anthony discussed ways to get ahead, grow your skill set, learn new technologies, and fight doubt and impostor syndrome. He concluded with tips on how to nail technical interviews.

Closing Keynote: Kojo Idrissa

The closing keynote on the last day of the conference was given by Kojo Idrissa, an organizer for DjangoCon and a Django Events Foundation North America (DEFNA) Ambassador. Kojo reflected on lessons from the past and how the African diaspora has had an effect on us all.

Sprints Day

The day after the main conference was dedicated to sprints! This is where people group up to work on various open source Python projects. I was part of a team that worked on Cookie Cutter and other related projects. I submitted a PR that got merged into the project and I also helped mentor other team members.

I highly recommend anyone who has not attended a sprint before to do so! It’s a great way to practice your skills, contribute to an open source project, and meet the developers involved with the project.

Summary

PyCon Africa was a huge success and nothing short of fantastic! It was a good start for the African community because it brought together, for the first time, Python developers from the different African Python communities. It also allowed us to hear from representatives of the PSF about how the Python community is organized and funded. For many, this was the first time that people from the different communities met in person after years of online collaboration.

For me, the experience was worth the sacrifice and expense because I got to meet and talk to people from all over the world and learn about how they use Python in their work or studies. I gave a lightning talk on how to send desktop notifications from Python scripts using Python and other tools native to Linux.

Another highlight of attending PyCon Africa was that I got to meet many of our readers, and a number of them attributed their success to much of the content they read here on Real Python. Anthony Shaw and I also gave away many Real Python stickers. Here’s a photo we took with Edison, a friend and Real Python reader:

One problem that the organizers faced was making the conference affordable to as many people as possible. Travel within Africa is expensive and difficult for many Africans. For instance, an entry visa to Ghana alone costs $100 to $200 U.S. dollars. Add to that the cost of travel and lodging, and the event was out of reach for many people who were interested in attending.

Voluntary donations from individuals and corporate sponsorship from Real Python, BriteCore, Andela, and several other sponsors helped cover the cost of tickets, accommodation, and travel for those that needed it. If you’d like to get more African people involved in Python, then consider supporting future Python events in Africa and hiring African developers to work remotely in your teams.

At the time of this writing, videos from the conference have not been made available, but there is a highlight video that also features interviews with some of the attendees. You can also read the official report from the conference organizers. If you’re thinking of attending a PyCon near you, then check out How to Get the Most Out of PyCon.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The post Tutorial: How to Read Stata Files in Python with Pandas appeared first on Erik Marsja.

In this post, we are going to learn how to read Stata (.dta) files in Python.

As previously described (in the read .sav files in Python post) Python is a general-purpose language that also can be used for doing data analysis and data visualization. One example of data visualization will be found in this post.

One potential downside, however, is that Python is not really user-friendly for data storage. This has, of course, lead to that our data many times are stored using Excel, SPSS, SAS, or similar software. See, for instance, the posts about reading .sav, and sas files in Python:

• How to read and write SPSS files in Python
• How to read SAS files in Pandas

Can I Open a Stata File in Python?

We are soon going to practically answer how to open a Stata file in Python? In Python, there are two useful packages called Pyreadstat, and Pandas that enable us to open .dta files. If we are working with Pandas, the  read_stata method will help us import a .dta into a Pandas dataframe. Furthermore, the package Pyreadstat, which is dependent on Pandas, will also create a Pandas dataframe from a .dta file.

How to install Pyreadstat:

First, before learning how to read .dat files using Python and Pyreadstat we need to install it. As many Python packages this package can be installed using pip or conda:

1. Install Pyreadstat using pip:
   Open up the Windows Command Prompt and type pip install pyreadstat
   
2. Install using Conda:
   Open up the Anaconda Prompt, and type conda install -c conda-forge pyreadstat

How to Open a Stata file in Python

In this section, we are finally ready to learn how to read a .dta file in Python using the Python packages Pyreadstat and Pandas.

How to Load a Stata File in Python Using Pyreadstat

In this section, we are going to use pyreadstat to import a .dta file into a Pandas dataframe. First, we import pyreadstat:

import pyreadstat

Second, we are ready to import Stata files using the method read_dta. Note that, when we load a file using the Pyreadstat package, it will look for the .dta file in Python’s working directory. In the read Stata files example below, the FifthDaydata.dta is located in a subdirectory (i.e., “SimData”).

dtafile = './SimData/FifthDayData.dta'
df, meta = pyreadstat.read_dta(dtafile)

In the code chunk above, two variables were created; df, and meta. If we use the Python function type we can see that “df” is a Pandas dataframe:

This means that we can use all the available methods for Pandas dataframe objects. In the next line of code, we are Pandas head method to print the first 5 rows.

df.head()

Learn more about working with Pandas dataframes in the following tutorials:

• Python Groupby Tutorial: Here you will learn about working the groupby method to group Pandas dataframes.
• Learn how to take random samples from a pandas dataframe
• A more general, overview, of how to work with Pandas dataframe objects can be found in the Pandas Dataframe tutorial.

How to Read a Stata file with Python Using Pandas

In this section, we are going to read the same Stata file into a Pandas dataframe. However, this time we will use Pandas read_stata method. This has the advantage that we can load the Statafile from a URL.

Before we continue, we need to import Pandas:

import pandas as pd

Now, when we have done that, we can read the .dta file into a Pandas dataframe using the read_stata method. In the read Stata example here, we are importing the same data file as in the previous example.

After we have loaded the Stata file using Python Pandas, we print the last 5 rows of the dataframe with the tail method.

dtafile = './SimData/FifthDayData.dta'

df = pd.read_stata(dtafile)
df.tail()

How to Read .dta Files from URL

In this section, we are going to use Pandas read_stata method, again. However, this time we will read the Stata file from a URL.

url = 'http://www.principlesofeconometrics.com/stata/broiler.dta'

df = pd.read_stata(url)
df.head()

Note, the only thing we changed was we used a URL as input (url) and Pandas read_stata will import the .dta file that the URL is pointing to.

Pandas Scatter Plot

Here, we will create a scatter plot in Python using Pandas scatter method. This is to illustrate how we can work with data imported from .dta files.

df.plot.scatter(x='pchick',
                       y='cpi')

Scatter Plot in Python

Learn more about data visualization in Python:

• How to Make a Scatter Plot in Python using Seaborn
• 9 Data Visualization Techniques You Should Learn in Python

How to Read Specific Columns from a Stata file

Now using pyreadstat read_dta and Pandas read_stat both enables us to read specific columns from a Stata file. Note, that read_dta have the argument usecols and Pandas the argument columns.

Reading Specific Columns using Pyreadstat

In this Python read dta example, we use the argument usecols that takes a list as parameter.

import pyreadstat

dtafile = './SimData/FifthDayData.dta'
df, meta = pyreadstat.read_dta(dtafile,
                              usecols=['index', 'Name', 'ID',
                                      'Gender'])
df.head()

Dataframe from .dta

Reading Specific Columns using Pandas read_stata

Here, we are going to use Pandas read_stata method and the argument columns. This argument, as in the example above, takes a list as input.

import pandas as pd
url = 'http://www.principlesofeconometrics.com/stata/broiler.dta'

df = pd.read_stata(url,
                  columns=['year', 'pchick', 'time',
                                      'meatex'])
df.head()

Dataframe

Note, the behavior of Pandas read_stata; in the resulting dataframe the order of the column will be the same as in the list we put in.

How to Save a Stata file

In this section of the Python Stata tutorial, we are going to save the dataframe as a .dta file. This is easily done, we just have to use the write_dta method when using Pyreadstat and the dataframe method to_stata in Pandas.

Saving a dataframe as a Stata file using Pyreadstat

In the example below, we are using the dataframe we created in the previous section and write it as a dta file.

pyreadstat.write_dta(df, 'broilerdata_edited.dta')

Now, between the parentheses is where the important stuff happens. The first argument is our dataframe and the second is the file path. Note, only having the filename, as in the example above, will make the write_dta method to write the Stata file to the current directory.

How to Save a dataframe as .dta with Pandas to_stata

In this example, we are going to save the same dataframe using Pandas to_stata:

df.to_stata('broilerdata_edited.dta')

As can be seen in the image above, the dataframe object has the to_stata method. Within, the parentheses we put the file path.

Save a CSV file as a Stata File

In this section, we are going to work with Pandas read_csv to read a CSV file, containing data. After we have imported the CSV to a dataframe we are going to save it as a .dta file using Pandas to_stat:

df = pd.read_csv('./SimData/FifthDayData.csv')
df.to_stata('./SimData/FifthDayData.dta')

Export an Excel file as a Stata File

In the final example, we are going to use Pandas read_excel to import a .xslx file and then save this dataframe as a Stata file using Pandas to_stat:

df = pd.read_excel('./SimData/example_concat.xlsx')
df.to_stata('./SimData/example_concat.dta')

Note, that in both of the last two examples above we save the data to a folder called SimData. If we want to save the CSV and Excel file to the current directory we simply remove the “./SimData/” part of the string.

Learn more about importing data using Pandas:

• Pandas Read CSV Tutorial
• Pandas Read Excel Tutorial

Note, all the files we have read using read_dta, read_stata, read_csv, and read_excel can be found here. It is, of course, possible to open SPSS and SAS files using Pandas and save them as .dta files as well.

Summary: Read Stata Files using Python

In this post, we have learned how to read Stata files in Python. Furthermore, we have learned how to write Pandas dataframes to Stata files.

The post Tutorial: How to Read Stata Files in Python with Pandas appeared first on Erik Marsja.