Quantcast
Channel: Planet Python
Viewing all 22404 articles
Browse latest View live

Mike C. Fletcher: Windows Isn't a Horror Show

$
0
0

So it has been a long time since I forced myself to boot into Windows to get support/development work done for the platform. Other than one hard-freeze where the machine updated its drivers, rebooted, and just stopped recognising the keyboard or trackpad, it pretty much just worked (turning it off and on again did, as the IT Crowd suggests, fix the problem).

Setting up a development environment involved:

  • install vscode (automatic)
  • install vs express (automatic, chose python 3.7 and C/C++)
  • setting up system path to add the visual studio python 3.7 to the path (this was fiddly, but it wasn't hard)
  • installing the windows version of git

Which is much less of a Royal PITA than it used to be with having to have VS Express 7 and a bunch of obsolete platform SDKs then modify a bunch of system paths and settings and hope you get the whole incantation perfect. Essentially the development setup process was acceptable this time.

Out of curiousity I also installed WSL, but since the only reason I boot Windows is to deal with hardware/platform specific issues with PyOpenGL... it's not much interest to me.

Other than that, the system was pretty much as I remember Windows. There were a bunch of mysterious performance glitches in the GUI (slow downs, stalls waiting for apps or whatever to start, seemingly complete lack of response to input for 20s then 10 things happening at once, audio happening 5s after the event it's suppose to signal, etc), but a basically serviceable desktop and I'm sure you get used to that stuff, just as Linux users get used to out-of-memory events bringing their desktops to their knees.


EuroPython: EuroPython 2020: Venue and location selected

$
0
0

europythonsociety:

After a work intense RFP over two months with more than 40 venues competing, 18 first round entries, and two rounds of refinements, we are now happy to announce the winner:

image

EuroPython 2020 will be held at the CCD in 
Dublin, Ireland, from July 20 - 26 2020

We will now start work on the contracts and get the organization going, so that we can all enjoy another edition of EuroPython next year.

Many thanks,

EuroPython Society Board
https://www.europython-society.org/

Davy Wybiral: Intro to Arduino with LoRa

$
0
0
If you've ever wanted to get started using LoRa wireless communication in your Arduino projects this video is for you. It covers some popular LoRa module options, a brief introduction to the Arduino code needed to control them, and an explanation of the different ways to configure your LoRa modem to maximize range or data transfer rate along with visualizations of the LoRa packet transmissions.

It's time to get wireless...


Techiediaries - Django: Django 3 Tutorial & Example: Build a CRUD REST API for A Simple CRM

$
0
0

Django 3 is released with full async support!

In this tutorial series, you'll learn about Django 3 by creating a CRUD example application with database, admin access, and REST API views. We'll be using MySQL as the database system.

Throughout this beginner's tutorial for Django 3, we are going to learn to build web applications with Python and Django. This tutorial assumes no prior experience with Django, so we'll be covering the basic concepts and elements of the Django framework by emphasizing essential theory with practice.

Basically, we are going to learn Django fundamental concepts while building a simple CRM web application.

This tutorial doesn't only cover fundamental basics of Django but also advanced concepts such as how to use and integrate Django with modern front end frameworks like Angular 2+, Vue and React.

You'll learn about CRUD, database ORM, how to create API views and URLs.

What's Django?

Django is an open source Python based web framework for building web applications quickly.

  • It's a pragmatic framework designed for developers working on projects with strict dead lines.
  • It's perfect for quickly creating prototypes and then continue building them after clients approval.
  • It follows a Model View Controller (MVC) design pattern
  • Django uses the Python language, a general purpose, powerful and feature-rich programming language.

Django 3 Features

Django 3 comes with many new features such as:

  • MariaDB support: Django now officially supports MariaDB 10.1+. You can use MariaDB via the MySQL backend,
  • ASGI support for async programming,
  • Django 3.0 provides support for running as an ASGI application, making Django fully async-capable
  • Exclusion constraints on PostgreSQL: Django 3.0 adds a new ExclusionConstraint class which adds exclusion constraints on PostgreSQL, etc.

What's MVC?

MVC is a software architectural design pattern which encourages the separation of concerns and effective collaboration between designers and developers when working on the same project. It basically divides or separates your app into three parts:

  • Model: responsible for data storage and management,
  • View: responsible of representing and rendering the user interface or view,
  • Controller: responsible for handling logic to control the user interface and work with data model.

Thanks to MVC, you as a developer can work in the model and controller parts without being concerned with the user interface (left to designers) so if anything changes on the side of designers on the user interface, you can rest assured that you will not be affected.

Introduction to Python

Python is a general purpose programing language that's suitable for developing all kind of applications including web applications. Python is known by a clean syntax and a large standard library which contains a wide range of modules that can be used by developers to build their applications instead of reinventing the wheel.

Here is a list of features and characteristics of Python:

  • Python is an Object Oriented Language just like Java or C++. Also like Java, Python is an interpreted language that runs on top of its own virtual machine which makes it a portable language that can runs across every machine and operating system such as Linux, Windows and MAC.

  • Python is especially popular among the scientific community where it's used for creating numeric applications.

  • Python is also known by the great performance of its runtime environment which makes it a good alternative to PHP for developing web applications.

For more information you can head to http://python.org/ where you can also download Python binaries for supported systems.

For Linux and MAC, Python is included by default so you don't have to install it. For Windows just head over to the official Python website and grab your installer. Just like any normal Windows program, the installation dead process is easy and straightforward.

Why Using Django?

Due to its popularity and large community, Python has numerous web frameworks among them Django. So what makes Django the right choice for you or your next project?

Django is a batteries-included framework

Django includes a set of batteries that can be used to solve common web problems without reinventing the wheel such as:

  • the sites framework,
  • the auth system,
  • forms generation,
  • an ORM for abstracting database systems,
  • and a very powerful templating engine,
  • caching system,
  • RSS generation framework etc.

The Django ORM

Django has a powerful ORM (Object Relational Mapper) which allows developers to use Python OOP classes and methods instead of SQL tables and queries to work with SQL based databases. Thanks to the Django ORM, developers can work with any database system such as MySQL or PostgresSQL without knowing anything about SQL. In the same time the ORM doesn't get in the way. You can write custom SQL anytime you want especially if you need to optimize the queries against your server database for increased performance.

Support for Internationalization: i18n

You can use Django for writing web applications for other languages than English with a lot of ease thanks to its powerful support for internationalization or you can also create multi lingual websites

The Admin Interface

Django is a very suitable framework for quickly building prototypes thanks to its auto-generated admin interface.

You can generate a full fledged admin application that can be used to do all sorts of CRUD operations against your database models you have registered with the admin module using a few lines of code.

Community and Extensive Documentation

Django has a great community that has contributed all sorts of awesome things to Django from tutorials and books to reusable open source packages that extend the core framework to include solutions for even more web development problems without reinventing the wheel or wasting time implementing what other developers have already created.

Django has also one of the most extensive and useful documentation on the web which can gets you up and running with Django in no time.

As a conclusion, if you are looking for a web framework full of features that makes building web applications fun and easy and that has all what you can expect from a modern framework. Django is the right choice for you if you are a Python developer.

  • Python is a portable programming language that can be used anywhere its runtime environment is installed.

  • Django is a Python framework which can be installed on any system which supports the Python language.

In this tutorial part, we are going to see how to install Python and Django on the major available operating systems i.e Windows, Linux and MAC.

At this point of our Django tutorial, we have covered what is Python, MVD and Django and the features of the Django. Let's now see how to install Python and a local development environment with pip and venv.

Installing Python

Depending on your operating system you may or may not need to install Python. In Linux and MAC OS Python is included by default. You may only need to update it if the installed version is outdated.

Installing Python On Windows

Python is not installed by default on Windows, so you'll need to grab the official installer from the official Python website at http://www.python.org/download/. Next launch the installer and follow the wizard to install Python just like any other Windows program.

Also make sure to add Python root folder to system path environment variable so you can execute the Python executable from any directory using the command prompt.

Next open a command prompt and type python. You should be presented with a Python Interactive Shell printing the current version of Python and prompting you to enter your Python commands (Python is an interpreted language)

Installing Python on Linux

If you are using a Linux system, there is a great chance that you already have Python installed but you may have an old version. In this case you can very easily update it via your terminal depending on your Linux distribution.

For Debian based distributions, like Ubuntu you can use the apt package manager

sudo apt-get install python

This will update your Python version to the latest available version.

For other Linux distributions you should look for equivalent commands to install or update Python which is not a daunting task if you already use a package manager to install packages for your system then you should follow the same process to install or update Python.

Installing Python on MAC OS

Just like Linux, Python is included by default on MAC but in case you have an old version you should be able to update it by going to [http://www.python.org/download/mac/](http://www.python.org/download/mac/ and grab a Python installer for MAC.

Now if you managed to install or update Python on your own system or in case you have verified that you already have an updated version of Python installed on your system let's continue by installing Django.

Installing PIP

PIP is a Python package manager which's used to install Python packages from Python Package Index which is more advanced than easy_install the default Python package manager that's installed by default when you install Python.

You should use PIP instaed of easy_install whenever you can but for installing PIP itself you should use easy_install. So let's first install PIP:

Open your terminal and enter:

$ sudo easy_install pip

You can now install Django on your system using pip

$ sudo pip install django

While you can do this to install Django, globally on your system, it's strongly not recommend. Instead you need to use a virtual environement to install packages.

Creating a MySQL Database

In this Django tutorial, we'll be using a MySQL database. In your terminal invoke the mysql client using the following command:

$ mysql -u root -p

Enter your MySQL password and hit Enter.

Next, run the following SQL statement to create a database:

mysql> create database crmdb;

Django 3 Tutorial: Creating a Virtual Environment

Let's start our tutorial by creating a virtual environment. Open a new terminal, navigate to a working folder and run the following command:

$ cd ~/demos
$ python3 -m venv .env

Next, activate the virtual environment using the following command:

$ source .env/bin/activate

At this point of our tutorial, we've created a virtual environment for our Django project. Let's now proceed to creating our project.

Django 3 Tutorial: Installing Django and Django REST Framework

Now, that you have created and activated your virtual environment, you can install your Python packages using pip. In your terminal where you have activated the virtual environment, run the following commands to install the necessary packages:

$ pip install django
$ pip install djangorestframework

You will also need to install the MySQL client for Python using pip:

$ pip install mysqlclient

Django 3 Tutorial: Creating a Project

Now, let's proceed to creating our django project. In your terminal, run the following command:

$ django-admin startproject simplecrm

This command will take care of creating a bunch of necessary files for the project.

Executing the tree command in the root of our created project will show us the files that were created.

    .
    ├── simplecrm
    │   ├── __init__.py
    │   ├── settings.py
    │   ├── urls.py
    │   └── wsgi.py
    └── manage.py

__init__ is the Python way to mark the containing folder as a Python package which means a Django project is a Python package.

settings.py is the project configuration file. You can use this file to specify every configuration option of your project such as the installed apps, site language and database options etc.

urls.py is a special Django file which maps all your web app urls to the views.

wsgi.py is necessary for starting a wsgi application server.

manage.py is another Django utility to manage the project including creating database and starting the local development server.

These are the basic files that you will find in every Django project. Now the next step is to set up and create the database.

Next, open the settings.py file and update the database setting to point to our crmdb database:

DATABASES={'default':{'ENGINE':'django.db.backends.mysql','NAME':'crmdb','USER':'root','PASSWORD':'YOUR_DB_PASSWORD','HOST':'localhost','PORT':'3306',}}

Next, add rest_framework to the INSTALLED_APPS array:

INSTALLED_APPS=[# [...]'rest_framework']

Finally, migrate the database using the following commands:

$ cd simplecrm
$ python manage.py migrate

You will be able to access your database from the 127.0.0.1:8000 address.

Django 3 Tutorial: Create an Admin User

Let's create an admin user using the following command:

$ python manage.py createsuperuser

Django 3 Tutorial: Creating an Application

Next, let's create a Django application for encapsulating our core CRM functionality. In your terminal, run the following command:

$ python manage.py startapp crmapp

Next, you need to add it in the settings.py file:

INSTALLED_APPS=[# ...'rest_framework','crmapp']

Django 3 Tutorial: Creating the Database Models

Let's now proceed to create the database models for our application. We are going to create the following models:

  • Contact
  • Account
  • Activity
  • ContactStatus
  • ContactSource
  • ActivityStatus

We have three main models which are Contact, Account and Activity. The last three models are simply lookup tables (They can be replaced by an enum).

Open the crmapp/models.py file and the following code:

fromdjango.dbimportmodelsfromdjango.contrib.auth.modelsimportUserINDCHOICES=(('FINANCE','FINANCE'),('HEALTHCARE','HEALTHCARE'),('INSURANCE','INSURANCE'),('LEGAL','LEGAL'),('MANUFACTURING','MANUFACTURING'),('PUBLISHING','PUBLISHING'),('REAL ESTATE','REAL ESTATE'),('SOFTWARE','SOFTWARE'),)classAccount(models.Model):name=models.CharField("Name of Account","Name",max_length=64)email=models.EmailField(blank=True,null=True)phone=models.CharField(max_length=20,blank=True,null=True)industry=models.CharField("Industry Type",max_length=255,choices=INDCHOICES,blank=True,null=True)website=models.URLField("Website",blank=True,null=True)description=models.TextField(blank=True,null=True)createdBy=models.ForeignKey(User,related_name='account_created_by',on_delete=models.CASCADE)createdAt=models.DateTimeField("Created At",auto_now_add=True)isActive=models.BooleanField(default=False)def__str__(self):returnself.nameclassContactSource(models.Model):status=models.CharField("Contact Source",max_length=20)def__str__(self):returnself.statusclassContactStatus(models.Model):status=models.CharField("Contact Status",max_length=20)def__str__(self):returnself.statusclassContact(models.Model):first_name=models.CharField("First name",max_length=255,blank=True,null=True)last_name=models.CharField("Last name",max_length=255,blank=True,null=True)account=models.ForeignKey(Account,related_name='lead_account_contacts',on_delete=models.CASCADE,blank=True,null=True)email=models.EmailField()phone=models.CharField(max_length=20,blank=True,null=True)address=models.TextField(blank=True,null=True)description=models.TextField(blank=True,null=True)createdBy=models.ForeignKey(User,related_name='contact_created_by',on_delete=models.CASCADE)createdAt=models.DateTimeField("Created At",auto_now_add=True)isActive=models.BooleanField(default=False)def__str__(self):returnself.first_nameclassActivityStatus(models.Model):status=models.CharField("Activity Status",max_length=20)def__str__(self):returnself.statusclassActivity(models.Model):description=models.TextField(blank=True,null=True)createdAt=models.DateTimeField("Created At",auto_now_add=True)contact=models.ForeignKey(Contact,on_delete=models.CASCADE,blank=True,null=True)def__str__(self):returnself.description

Django 3 Tutorial: Creating Model Serializers

After creating models we need to create the serializers. In the crmapp folder create a serializers.py file:

$ cd crmapp
$ touch serializers.py

Next, open the file and add the following imports:

fromrest_frameworkimportserializersfrom.modelsimportAccount,Activity,ActivityStatus,Contact,ContactSource,ContactStatus

Next, add a serializer class for each model:

classAccountSerializer(serializers.ModelSerializer):classMeta:model=Accountfields="__all__"classActivitySerializer(serializers.ModelSerializer):classMeta:model=Activityfields="__all__"classActivityStatusSerializer(serializers.ModelSerializer):classMeta:model=ActivityStatusfields="__all__"classContactSerializer(serializers.ModelSerializer):classMeta:model=Contactfields="__all__"classContactSourceSerializer(serializers.ModelSerializer):classMeta:model=ContactSourcefields="__all__"classContactStatusSerializer(serializers.ModelSerializer):classMeta:model=ContactStatusfields="__all__"

Django 3 Tutorial: Creating API Views

After creating the model serializers, let's now create the API views. Open the crmapp/views.py file and add the following imports:

fromrest_frameworkimportgenericsfrom.modelsimportAccount,Activity,ActivityStatus,Contact,ContactSource,ContactStatusfrom.serializersimportAccountSerializer,ActivitySerializer,ActivityStatusSerializer,ContactSerializer,ContactSourceSerializer,ContactStatusSerializer

Next, add the following views:

fromrest_frameworkimportgenericsfrom.modelsimportAccount,Activity,ActivityStatus,Contact,ContactSource,ContactStatusfrom.serializersimportAccountSerializer,ActivitySerializer,ActivityStatusSerializer,ContactSerializer,ContactSourceSerializer,ContactStatusSerializerclassAccountAPIView(generics.ListCreateAPIView):queryset=Account.objects.all()serializer_class=AccountSerializerclassActivityAPIView(generics.ListCreateAPIView):queryset=Activity.objects.all()serializer_class=ActivitySerializerclassActivityStatusAPIView(generics.ListCreateAPIView):queryset=ActivityStatus.objects.all()serializer_class=ActivitySerializerclassContactAPIView(generics.ListCreateAPIView):queryset=Contact.objects.all()serializer_class=ContactSerializerclassContactStatusAPIView(generics.ListCreateAPIView):queryset=ContactStatus.objects.all()serializer_class=ContactSerializerclassContactSourceAPIView(generics.ListCreateAPIView):queryset=ContactSource.objects.all()serializer_class=ContactSourceSerializer

After creating these models, you need to create migrations using the following command:

$ python manage.py makemigrations

Next, you need to migrate your database using the following command:

$ python manage.py migrate

Creating API URLs

Let's now create the API URLs to access our API views. Open the urls.py file and add the following imports:

fromdjango.contribimportadminfromdjango.urlsimportpathfromcrmappimportviews

Next, add the following content:

urlpatterns=[path('admin/',admin.site.urls),path(r'accounts',views.AccountAPIView.as_view(),name='account-list'),path(r'contacts',views.ContactAPIView.as_view(),name='contact-list'),path(r'activities',views.ActivityAPIView.as_view(),name='activity-list'),path(r'activitystatuses',views.ActivityStatusAPIView.as_view(),name='activity-status-list'),path(r'contactsources',views.ContactSourceAPIView.as_view(),name='contact-source-list'),path(r'contactstatuses',views.ContactStatusAPIView.as_view(),name='contact-status-list')]

Django Tutorial: Enabling CORS

For development purposes, we'll need to enable CORS (Cross Origin Resource Sharing) in our Django application.

So start by installing django-cors-headers using pip

$pipinstalldjango-cors-headers

Next, you need to add it to your project settings.py file:

INSTALLED_APPS=(## [...]'corsheaders')

Next, you need to add corsheaders.middleware.CorsMiddleware middleware to the middleware classes in settings.py

MIDDLEWARE=('corsheaders.middleware.CorsMiddleware',# [...])

You can then, either enable CORS for all domains by adding the following setting:

CORS_ORIGIN_ALLOW_ALL=True

You can find more configuration options from the docs.

Starting the local development server

Django has a local development server that can be used while developing your project. It's a simple and primitive server which's suitable only for development not for production.

To start the local server for your project, you can simply issue the following command inside your project root directory:

$ python manage.py runserver

Next navigate to the http://localhost:8000/ address with a web browser.

You should see a web page with a message:

It worked!

Conclusion

To conclude this django 3 tutorial, let's summarize what we have done. We have created a new Django project, created and migrated a MySQL database, built a simple CRM REST API with Django REST framework and started a local development server.

Techiediaries - Django: Python Webviews with PyWebView

$
0
0

Have you ever wanted to use your python and web development skills to build cross platform desktop GUI apps? if yes then welcome to this tutorial where we'll show you how to use PyWebView to turn your web application built using python and client side technologies such as HTML, CSS and JavaScript into a standalone cross platform GUI application that runs under major operating systems such as Linux, Windows and MAC.

You can also use any web framework based on python such as the so popular django framework or also the lightweight web development framework flask to build business logic of your app and then wrap the whole app, server and client side, into a desktop app .

What is PyWebView?

Before getting started with PyWebView let's first introduce it.

PyWebView is an open source, cross platform and lightweight wrapper which provides you with a WebKit/MSHTML based webview inside a native GUI window which allows you to render HTML and CSS and execute JavaScript code just like any normal web browser. It's in fact a headless web browser that you can use to run a unique and default web app so the user gets the feeling of having an ordinary GUI application.

PyWebView doesn't re-invent the wheel but instead it uses the existing tools and native platform APIs. For providing a native window

For the target system it uses:

  • Win32 API or Win Forms under Windows,
  • Cocoa on OS X,
  • GTK 3 ,QT4/5 under Linux.

Getting Started with PyWebView

First of all, make sure you have either Python 2 or 3 installed in your development machine. You also need the pip package manager installed then open up your terminal under Linux/MAC or the command prompt under Windows and run:

$ pip install pywebview

If your installation fails make sure you have the following dependencies installed depending on your target system and re-execute the install command again:

  • Under Windows you need to install pywin32, comtypes if you want to use Win32 subsystem to create natives windows or pythonnet if you want to be able to use Windows Forms for native windows.
  • Under OS X you need to install pyobjc if it's not already installed in your system.
  • Under Linux GTK3 based distributions you have to install PyGObject before you can use PyWebView.
  • For QT based systems you need to install PyQt4/PyQt5 .

How to Use pywebview?

You can easilly get started with pywebview. Just execute the following python code to launch a native window that displays a visited web URL:

importwebviewwebview.create_window("Techiediaries","http://www.techiediaries.com",width=800,height=600,resizable=True,fullscreen=False)

So you first start by importing the webview. Next, we use the create_window method to create and show a native window which renders the content of http://www.techiediaries.com with the title "Techiediaries".

All parameters are self explanatory, they just control the width, hieght and other properties of the window.

You can see a full example of a flask app wrapped into a GUI app using pywebview under the GitHub repository of pywebview

Conclusion

While there are many python based frameworks for building desktop apps which are cross platform since python itself is a portable and cross platform environment, you have to learn these frameworks before you can build your apps but with pywebview you can use your existing web development skills and familiar Python web frameworks to create the web apps you have the skills to build and then wrap them into GUI based apps without starting from scratch.

Codementor: 3 Tips to be Effective In Pycharm - Reading Time: 3 Mins

$
0
0
Shows 3 tips and tricks in using PyCharm for Python development.

Codementor: The Skills to Get a Job As a Django Developer - Reading Time: 4 Mins

$
0
0
The skills to get a job as a Django developer.

Programiz: Python IDEs and Code Editors

$
0
0
In this guide, you will learn about various Python IDEs and code editors for beginners and professionals.

Abhijeet Pal: Creating Sitemaps in Django

$
0
0

Django comes with baked-in functionality for generating sitemaps dynamically using the sitemap framework. A sitemap is an XML file that informs search engines such as Google, the pages of your website, their relevance, and how frequently they are updated. Using sitemaps facilitates crawlers indexing the site, therefore sitemap plays a crucial role in modern SEO …

The post Creating Sitemaps in Django appeared first on Django Central.

Weekly Python StackOverflow Report: 20 best Python questions at stackoverflow in 2019

$
0
0

These are the 20 most rated questions at 2019 Stack Overflow.
Between brackets: question score & answers count
Built date: 2020/01/03 09:19:25 GMT


  1. Why does Python's hash of infinity have the digits of π? - [236/3]
  2. Is there a more elegant way to express ((x == a and y == b) or (x == b and y == a))? - [105/10]
  3. Why can I use a list index as an indexing variable in a for loop? - [92/6]
  4. Why does (inf + 0j)*1 evaluate to inf + nanj? - [93/4]
  5. Why is f'{{{74}}}' the same as f'{{74}}' with f-Strings? - [88/1]
  6. Why does b+=(4,) work and b = b + (4,) doesn't work when b is a list? - [75/7]
  7. Why does Python start at index -1 (as opposed to 0) when indexing a list from the end? - [79/7]
  8. Why is TensorFlow 2 much slower than TensorFlow 1? - [104/2]
  9. Randomness of Python's random - [70/4]
  10. Why does Python allow out-of-range slice indexes for sequences? - [72/2]
  11. Unexpected behaviour with Python generator - [57/8]
  12. What exactly is meant by "partial function" in functional programming? - [55/3]
  13. What does a yield inside a yield do? - [56/4]
  14. Issues implementing the "Wave Collapse Function" algorithm in Python 2.7 - [52/2]
  15. Should linear read-shuffled write always be faster than shuffled read-linear write? (Was: Why is fancy assignment slower than fancy lookup?) - [53/5]
  16. How to write 2**n - 1 as a recursive function? - [49/7]
  17. Why is a `for` loop so much faster to count True values? - [53/5]
  18. Is there a difference between `board[x, y]` and `board[x][y]` in Python? - [47/6]
  19. Why was p[:] designed to work differently in these two situations? - [51/6]
  20. Jupyter notebook: No connection to server because websocket connection fails - [46/4]

Erik Marsja: How to Read a File in Python, Write to, and Append, to a File

$
0
0

The post How to Read a File in Python, Write to, and Append, to a File appeared first on Erik Marsja.

In this tutorial, we are going to learn how to read a file in Python 3. After we have learned how to open a file in Python, we are going to learn how to write to the file and save it again.  In previous posts, we have learned how to open a range of different files using Python. For instance, we have learned how to open JSON, CSV, Excel, and HTML files using Pandas, and the json library. Here, however, we are going to open plain files (.txt) in Python.

Files in Windows ExplorerExample files to read with Python open()

Prerequisites

For this how to read a file in Python tutorial, we need to have Python 3 installed. One of the easiest ways to install Python is to download a Python distribution, such as Anaconda, ActivePython, or Enthought Canopy, to name a few. Here’s a YouTube tutorial on how to install Anaconda.

Reading Files in Python

Now, when we are working with Python, there is no need to import a library in order to read and write files. Reading and writing files, in Python, is handled natively in the language.                  

In this post, we will, thus, focus on using Python to open a file and we will use the built-in open function to get a file object.

Python’s File Object

Now, when we are using Python’s open function, it will return something known as a file object. These objects will contain methods and attributes that we can use to gather information about the file we have opened (much like we’ve learned when using Pandas dataframe objects, for instance). One neat thing, of course, is that we can use these methods to change the file that we have opened using Python.

In this Python read from a file tutorial, we are going to work with the mode attribute. This attribute will tell us which mode the file we have opened, using Python, is in. Furthermore, the name attribute will give us the name of the file we have opened in Python (i.e., the name of the file object).

How do You Read a File in Python?

Opening, and reading, a file in Python is easy: we just type ourfile = open('PATH_TO_FILE') and then ourfile.read().

What Does read() Return in Python?

Now, if we open a file in Python the read() return a Python string object.

<<<<<<

How to Read a File in Python using the open() function

In this section, we are going to learn how to load a file in Python using the open() function. In it’s simplest example, open() will open a file and create a file object.

As can be seen, in the image above, there are a couple of arguments that can be used, however, when opening a file using Pythons open() function. The most commonly used arguments, however, are the first two. Note, the first is mandatory and the rest of them are optional. If we don’t add a mode, the file will be opened in read-only mode in Python.

Using the mode argument when reading files in Python

In this section, of the post on how to read a file in Python, we will go through the different modes in which we can read a file. As previously mentioned, if we don’t pass anything with the mode argument the file will be written in read-only. In this reading files in Python tutorial, we are going to work with the following modes:

Python open() and the modesSome of the modes we can open files with

That is, we can open a file in read-only, write, append, and read and write mode. If we use append (‘a’) we will append information at the end of that file.

A Simple Read a File in Python Example

Now, we are going to read a file in Python using only the file name as an argument. As evident, in the code example below, open() takes a string with the file path as input. In this reading a file in Python example, it is assumed that the example file is in the same directory as the Python script. Or in this case, the Jupyter notebook.

exfile = open('example_file')
print(exfile)
Python open() return a file objectFile object in Python

In the image, above, it becomes evident that we have a file object that we have opened in read-only mode. To iterate what we did, we opened up the text file (example_file.txt) without any other arguments. This, in turn, leads to that the file was opened in the read mode. Thus, we cannot write to that file. If we want to print the name of the file we can just type print(exfile.name).

Creating a Text file and Writing to the File using Python

In the next how to read a file in Python example, we are going to create a file using Python open(). Now, we are going to use open with the write mode. That is, we are going to open a file object and then use the file objects write method.

 exfile = open('example_file2', 'w')
print(exfile)
Write mode - Python file objectWrite mode

Now, in the image above, we can see that we have a file object in write mode (‘w’). In the next code chunk, we are going to add one line of text to this file:

exfile.write('This is example file 2 \n')

If we want to, we can, of course, add more lines by using the write method again:

exfile.write('Line number 2, in example file 2')
exfile.close()

Note, how we closed the file using close() in the last line above. In the image, below, we can see the example file we have created with Python.

Content of file written in PythonContent of the created file

How to Read a Text File in Python using open()

In the next Python read a file example, we will learn how to open a text (.txt) file in Python. This is, of course, simple and we have basically already the knowledge on how to do this with Python. That is if we just want to read the .txt file in Python we can use open and the read mode:

txtfile = open('example_file.txt')

Opening a File in Python: read() Example:

This was simple. Now, if we want to print the content of the text file we have three options. First, we can use the read() method. This Python method will read the entire file. That is, txtfile.read() will give us the following output:

Reading a file in Python using read()Output of the read() method

Reading a File to a Python list: readlines() Example:

If we, on the other hand, want to read the content from a file to a Python list, we can use the readlines() method. That is if we want to read the text file into a list we type:

txtfile = open('example_file.txt')  
print(txtfile.readlines())
How to read a file in Python to a list.List created with readlines()

Furthermore, we can also use the sizehint argument. This enables us to get certain lines. For example, the following code will read the two first lines, into two different string variables, and print it:

txtfile = open('example_file.txt')
line = txtfile.readlines(1)
print(line)

line2 = txtfile.readlines(2)
print(line2)
how to load a file in Python to a listFirst two lines read from a file

Finally, when we have opened a file in Python we can also loop through the content and print line by line:

txtfile = open('example_file.txt')
for line in txtfile:
    print(line)
how to read a file in Python - line by lineOutput from the loop (reading the file line by line)

Reading a File in Python and Appending Content to It

In the next example, we are going to read a file in Python and append information to the file. More specifically, we are going to load a .txt file using Python and append some data to it. Again, we start by opening the file in Python using open() but using the append mode:

open('example_file2.txt', 'a')
.

Next, we are going to add content to this using write(), again.

txtfile.write('\n More text here.')

When appending text, in Windows 10 at least, we have to add the \n before the line or else this line will be appended next to the last character (on the last line, of the file). If we are to add more lines, we have to remember to do this as well;

txtfile.write(‘\nLast line of text, I promise.)
txtfile.close()

Finally, we can open the text file using a text editor (e.g., Notepad, Gedit) and we’ll see the last two lines that we’ve added:

How to Read a file in Python using the with Statement

In the next how to open a file in Python example, we are going to read a file using the with statement. This has the advantage that we don’t have to remember to close the file and the syntax for using the with statement is clean to read:

with open('example_file2.txt') as txtfile2:
    print(txtfile2.read())
How to read a file in Python using the with statementOutput

Now, if we try to use the read() method Python will throw a ValueError:

txtfile2.read()
reading files in Python examplesError

Splitting Words in File and Counting Words

As final reading a file in Python example, we are going to use the string split() method to split the sentences in the text file into words. When we have read the file and split the words, we are going to use the Counter subclass from collections to use Python to count words in an opened file.

from collections import Counter

with open('example_file2.txt') as txtfile2:
    wordcount = Counter(txtfile2.read().split())
    
print(len(wordcount))
# Output: 43

Counting Words in Python

Now, the Counter subclass is giving us a dictionary that contains all words and occurrences of each word. Thus, we can print all words and their count like this:

for k in sorted(wordcount, key=wordcount.get, reverse=True):
    print(k, wordcount[k])

In the code example above, we are looping through the keys in the dictionary and sort them. This way, we get the most common words on top. Of course, reading a file with many words, using Python, and printing the result like this is not feasible.

Reading other File Formats in Python

Now, we have learned how to load a file in Python, and we will briefly discuss other file formats that Python can handle. Python can, of course, open a wide range of different file formats. In previous posts, we have used the json library to parse JSON files in Python, for instance. Furthermore, we have also learned how to read csv files in Python using Pandas, open Excel files with Pandas, among other file formats that are common for storing data.

Conclusion: Reading & Writing Files in Python

In this tutorial, we have learned the basics of how to open files in Python. More specifically, we have learned how to read a file in different modes, create and write to a file, append data to a file, and how to read a file in Python using the with statement. In future posts, we will learn more about how to read other file formats using Python (e.g., CSV, Excel).

Drop a comment below if you need anything covered in this or a future blog, and stay tuned!

The post How to Read a File in Python, Write to, and Append, to a File appeared first on Erik Marsja.

Stack Abuse: Introduction to Speech Recognition with Python

$
0
0

Speech recognition, as the name suggests, refers to automatic recognition of human speech. Speech recognition is one of the most important tasks in the domain of human computer interaction. If you have ever interacted with Alexa or have ever ordered Siri to complete a task, you have already experienced the power of speech recognition.

Speech recognition has various applications ranging from automatic transcription of speech data (like voicemails) to interacting with robots via speech.

In this tutorial, you will see how we can develop a very simple speech recognition application that is capable of recognizing speech from audio files, as well as live from a microphone. So, let's begin without further ado.

Several speech recognition libraries have been developed in Python. However we will be using the SpeechRecognition library, which is the simplest of all the libraries.

Installing SpeechRecognition Library

Execute the following command to install the library:

$ pip install SpeechRecognition

Speech Recognition from Audio Files

In this section, you will see how we can translate speech from an audio file to text. The audio file that we will be using as input can be downloaded from this link. Download the file to your local file system.

The first step, as always, is to import the required libraries. In this case, we only need to import the speech_recognition library that we just downloaded.

import speech_recognition as speech_recog

To convert speech to text the one and only class we need is the Recognizer class from the speech_recognition module. Depending upon the underlying API used to convert speech to text, the Recognizer class has following methods:

  • recognize_bing(): Uses Microsoft Bing Speech API
  • recognize_google(): Uses Google Speech API
  • recognize_google_cloud(): Uses Google Cloud Speech API
  • recognize_houndify(): Uses Houndify API by SoundHound
  • recognize_ibm(): Uses IBM Speech to Text API
  • recognize_sphinx(): Uses PocketSphinx API

Among all of the above methods, the recognize_sphinx() method can be used offline to translate speech to text.

To recognize speech from an audio file, we have to create an object of the AudioFile class of the speech_recognition module. The path of the audio file that you want to translate to text is passed to the constructor of the AudioFile class. Execute the following script:

sample_audio = speech_recog.AudioFile('E:/Datasets/my_audio.wav')

In the above code, update the path to the audio file that you want to transcribe.

We will be using the recognize_google() method to transcribe our audio files. However, the recognize_google() method requires the AudioData object of the speech_recognition module as a parameter. To convert our audio file to an AudioData object, we can use the record() method of the Recognizer class. We need to pass the AudioFile object to the record() method, as shown below:

with sample_audio as audio_file:
    audio_content = recog.record(audio_file)

Now if you check the type of the audio_content variable, you will see that it has the type speech_recognition.AudioData.

type(audio_content)

Output:

speech_recognition.AudioData

Now we can simply pass the audio_content object to the recognize_google() method of the Recognizer() class object and the audio file will be converted to text. Execute the following script:

recog.recognize_google(audio_content)

Output:

'Bristol O2 left shoulder take the winding path to reach the lake no closely the size of the gas tank degrees office 30 face before you go out the race was badly strained and hung them the stray cat gave birth to kittens the young girl gave no clear response the meal was called before the bells ring what weather is in living'

The above output shows the text of the audio file. You can see that the file has not been 100% correctly transcribed, yet the accuracy is pretty reasonable.

Setting Duration and Offset Values

Instead of transcribing the complete speech, you can also transcribe a particular segment of the audio file. For instance, if you want to transcribe only the first 10 seconds of the audio file, you need to pass 10 as the value for the duration parameter of the record() method. Look at the following script:

sample_audio = speech_recog.AudioFile('E:/Datasets/my_audio.wav')
with sample_audio as audio_file:
    audio_content = recog.record(audio_file, duration=10)

recog.recognize_google(audio_content)

Output:

'Bristol O2 left shoulder take the winding path to reach the lake no closely the size of the gas'

In the same way, you can skip some part of the audio file from the beginning using the offset parameter. For instance, if you do not want to transcribe the first 4 seconds of the audio, pass 4 as the value for the offset attribute. As an example, the following script skips the first 4 seconds of the audio file and then transcribes the audio file for 10 seconds.

sample_audio = speech_recog.AudioFile('E:/Datasets/my_audio.wav')
with sample_audio as audio_file:
    audio_content = recog.record(audio_file, offset=4, duration=10)

recog.recognize_google(audio_content)

Output:

'take the winding path to reach the lake no closely the size of the gas tank web degrees office dirty face'

Handling Noise

An audio file can contain noise due to several reasons. Noise can actually affect the quality of speech to text translation. To reduce noise, the Recognizer class contains adjust_for_ambient_noise() method, which takes the AudioData object as a parameter. The following script shows how you can improve transcription quality by removing noise from the audio file:

sample_audio = speech_recog.AudioFile('E:/Datasets/my_audio.wav')
with sample_audio as audio_file:
    recog.adjust_for_ambient_noise(audio_file)
    audio_content = recog.record(audio_file)

recog.recognize_google(audio_content)

Output:

'Bristol O2 left shoulder take the winding path to reach the lake no closely the size of the gas tank web degrees office 30 face before you go out the race was badly strained and hung them the stray cat gave birth to kittens the younger again no clear response the mail was called before the bells ring what weather is in living'

The output is quite similar to what we got earlier; this is due to the fact that the audio file had very little noise already.

Speech Recognition from Live Microphone

In this section you will see how you can transcribe live audio received via a microphone on your system.

There are several ways to process audio input received via microphone, and various libraries have been developed to do so. One such library is PyAudio. Execute the following script to install the PyAudio library:

$ pip install PyAudio

Now the source for the audio to be transcribed is a microphone. To capture the audio from a microphone, we need to first create an object of the Microphone class of the Speach_Recogniton module, as shown here:

mic = speech_recog.Microphone()

To see the list of all the microphones in your system, you can use the list_microphone_names() method:

speech_recog.Microphone.list_microphone_names()

Output:

['Microsoft Sound Mapper - Input',
 'Microphone (Realtek High Defini',
 'Microsoft Sound Mapper - Output',
 'Speakers (Realtek High Definiti',
 'Microphone Array (Realtek HD Audio Mic input)',
 'Speakers (Realtek HD Audio output)',
 'Stereo Mix (Realtek HD Audio Stereo input)']

This is a list of microphones available in my system. Keep in mind that your list will likely look different.

The next step is to capture the audio from the microphone. To do so, you need to call the listen() method of the Recognizer() class. Like the record() method, the listen() method also returns the speech_recognition.AudioData object, which can then be passed to the recognize_google() method.

The following script prompts the user to say something in the microphone and then prints whatever the user has said:

with mic as audio_file:
    print("Speak Please")

    recog.adjust_for_ambient_noise(audio_file)
    audio = recog.listen(audio_file)

    print("Converting Speech to Text...")
    print("You said: " + recog.recognize_google(audio))

Once you execute the above script, you will see the following message:

Please say something

At this point of time, say whatever you want and then pause. Once you have paused, you will see the transcription of whatever you said. Here is the output I got:

Converting Speech to Text...
You said: hello this is normally from stack abuse abuse this is an article on speech recognition I hope you will like it and this is just a test speech and when I will stop speaking are you in today thank you for Reading

It is important to mention that if recognize_google() method is not able to match the words you speak with any of the words in its repository, an exception is thrown. You can test this by saying some unintelligible words. You should see the following exception:

Speak Please
Converting Speech to Text...
---------------------------------------------------------------------------
UnknownValueError                         Traceback (most recent call last)
<ipython-input-27-41218bc8a239> in <module>
      8     print("Converting Speech to Text...")
      9
---> 10     print("You said: " + recog.recognize_google(audio))
     11
     12

~\Anaconda3\lib\site-packages\speech_recognition\__init__.py in recognize_google(self, audio_data, key, language, show_all)
    856         # return results
    857         if show_all: return actual_result
--> 858         if not isinstance(actual_result, dict) or len(actual_result.get("alternative", [])) == 0: raise UnknownValueError()
    859
    860         if "confidence" in actual_result["alternative"]:

UnknownValueError:

A better approach is to use the try block when the recognize_google() method is called as shown below:

with mic as audio_file:
    print("Speak Please")

    recog.adjust_for_ambient_noise(audio_file)
    audio = recog.listen(audio_file)

    print("Converting Speech to Text...")

    try:
        print("You said: " + recog.recognize_google(audio))
    except Exception as e:
        print("Error: " + str(e))

Conclusion

Speech recognition has various useful applications in the domain of human computer interaction and automatic speech transcription. This article briefly explains the process of speech transcription in Python via speech_recognition library and explains how to translate speech to text when the audio source is an audio file or live microphone.

RMOTR: New NumPy and Panda Courses Released

$
0
0
Photo by Feliphe Schiarolli on Unsplash

Happy New Year, Data Scientists!

We’re excited to start 2020 by adding two new courses to our library; Introduction to Scientific Computing with NumPy and Introduction to Pandas for Data Analysis.

Introduction to Scientific Computing with NumPy

The best place to start any journey is at the beginning and NumPy is where it all started. NumPy was the library that gave rise to the Data Science / Python revolution. Therefore, as you’re working through the course, you’ll build upon the foundations of Data Science and large data processing.

In this course, you will explore all the fundamental concepts introduced by NumPy. You’ll work through low level bindings (such as C and Fortan), multidimensional arrays, vectorized operations, and more.

Your instructor, Santiago Basulto, will focus on the parallels between basic Python data structures and NumPy arrays. Specifically, the course will demonstrate how to apply both of them to scientific computing, without forgetting about performance. How does NumPy compare to Python? Is it faster? There’s only one way to find out.

At the end of the course, your final project will involve matrix processing and writing a Sudoku Solver. So if one of your New Year resolutions was to create a program that would solve your Sudoku puzzles for you, you can go ahead and cross that off your list.

Introduction to Pandas for Data Analysis

NumPy may have been first, but the most popular Python library for Data Science is Pandas. Understanding the fundamentals of Data Analysis using Pandas is a vital skill to have in your Data Science toolbox.

In this course, you’ll learn about basic data structures, including Data Frames, Series, Indices, Plotting, and more. However, it’s not just about Pandas. This course, also led by Santiago Basulto, is about Data Analysis and Data Engineering using Pandas. With plenty of exercises and projects, this is a hands-on course designed to turn your knowledge into action. When that Panda walks into a Bar….you’ll be ready.

Jump In

Start the year on the right foot, not by saying “I will” but “I am”. We know you’re going to be accomplishing exceptional things, which makes 2020 look even brighter.


New NumPy and Panda Courses Released was originally published in rmotr.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

Python Bytes: #162 Retrofitting async and await into Django

Talk Python to Me: #245 Python packaging landscape in 2020

$
0
0
Python is growing incredibly quickly and has found its place in many facets of the developer and computational space. But one area that is still shaky and uncertain is packaging and shipping software to users.

Doug Hellmann: entry_point_inspector 0.1.2

$
0
0
Entry Point Inspector is a tool for looking at the entry point plugins installed on a system. What’s new in 0.1.2? Fix compatibility issues with python3 (contributed by guozijian)

Abhijeet Pal: Creating Feeds with Django

$
0
0

Django ships with built-in syndication feed generating framework, which is used to generate dynamic Atom and RSS feeds. RSS is an abbreviation for Really Simple Syndication, it’s a way to have information delivered to you instead of you having to go find it. RSS is basically a structured XML document that includes full or summarized text along with other metadata such as published date, author name, etc. Creating RSS Feeds with Django In this article, we are going to generate feeds for a blog application following is the models.py file. class Post(models.Model): title = models.CharField(max_length=200, unique=True) slug = models.SlugField(max_length=200, unique=True) author = models.ForeignKey( User, on_delete=models.CASCADE, related_name="blog_posts" ) updated_on = models.DateTimeField(auto_now=True) content = models.TextField() created_on = models.DateTimeField(auto_now_add=True) status = models.IntegerField(choices=STATUS, default=0) class Meta: ordering = ["-created_on"] def __str__(self): return self.title def get_absolute_url(self): from django.urls import reverse return reverse("post_detail", kwargs={"slug": str(self.slug)}) Create a new file feeds.py inside the application directory. from django.contrib.syndication.views import Feed from django.template.defaultfilters import truncatewords from .models import Post from django.urls import reverse class LatestPostsFeed(Feed): title = "My blog" link = "" description = "New posts of my blog."…

The post Creating Feeds with Django appeared first on Django Central.

Python Data: Python Data Weekly Roundup – Jan 3 2020

$
0
0

In this week’s Python Data Weekly Roundup:

It’s time for Time-series Databases

If you haven’t taken a look at time-series databases, you should. For a lot of what we do today in data science, a time series database might make sense (e.g., stream processing,etc).  While this article isn’t long, its a quick introduction to the topic.

5 Key Reasons Why Data Scientists Are Quitting their Jobs

This is a very good article describing some of the challenges that data scientist face today. Its not all rosy out there…there are a lot of challenges and issues. This is definitely worth a read…I wonder how many of you are seeing these in your current jobs?

What is BERT and how does it Work?

What is BERT?

BERT (Bidirectional Encoder Representations from Transformers) is Google’s deep learning algorithm for NLP (natural language processing). It helps computers and machines understand the language as we humans do. Put simply, BERT may help Google better understand the meaning of words in search queries.

Understanding Customer Attrition Using Categorical Features in Python

If you’ve ever had to work on customer churn, you know it can be tough. This article describes how to use categorical features to understand customer churn. Its really good and worth the effort to read / follow.

Comparing Machine Learning Methods

When working with data and modeling, its sometimes hard to determine what model you should use for a particular modeling project.  A quick way to find an algorithm that might work better than others is to run through an algorithm comparison loop to see how various models work against your data. In this post, I compare machine learning methods using a few different sklearn algorithms.

Machine Learning Comparison

10 AI trends to watch in 2020

What’s happening in artificial intelligence in the year ahead? Look for modeling at the edge, new attention to data governance, and continued talent wars, among key AI trends.


That’s it for this week’s Python Data Weekly Roundup. Subscribe to our newsletter to receive this weekly roundup in your email.

 

The post Python Data Weekly Roundup – Jan 3 2020 appeared first on Python Data.

Django Weblog: DjangoCon Europe 2020 Announcement

$
0
0

We are happy to announce that DjangoCon Europe 2020 will take place in Porto, Portugal🇵🇹! The website is already online at 2020.djangocon.eu and it will be kept up to date with the latest updates. Don't forget to follow @djangoconeurope on Twitter.

The dates are already confirmed (don't rush into buying everything so soon because we'll have discounts for DjangoCon attendees!):

  • Conference: May 27-29 (Wednesday-Friday)
  • Sprints/Workshops: May 30 and 31 (Saturday and Sunday)

There is a lot to do, but it's very much worth it – DjangoCon Europe is an extremely friendly, open, inclusive, and informative (for beginners and advanced users alike) conference. Here are some themes and examples of activities and responsibilities that we seek help with:

  • Communications: Press, community relations, announcements, social media, attendee tools, volunteer coordination
  • Support and hospitality: Helpdesk, attendee support contact, visa help, travel management, chat support for attendees, on-site volunteer organization, speaker support
  • Financial Aid: Setup, grant selection, aid organization
  • Sponsors: Outreach to companies, organizing their logistics at the event and other types of visibility
  • Program: Committee work, talk selection, scheduling, session chairs, sprint/open space/keynote/lightning talks session organization
  • Code of Conduct: Drafting documents, handling of requests and issues
  • Diversity advocate: Accessibility considerations, outreach on-site

Join us regardless of your prior experience: this is also an opportunity to learn! In other words, you don't have to be an expert to join. Neither are we experts in hosting such a big event … yet!

Your location prior to the event is not significant. We can do all things that need to be done in Porto ourselves. The only important thing is that you have the energy and free time to help organize a wonderful DjangoCon Europe. The official language of all these prior activities will be English, as well as the conference itself.

Don't be shy 😊. For any inquiries you can email us at 2020@djangocon.eu or chat with us on Slack at DjangoConEurope2020.

Mike C. Fletcher: PyOpenGL 3.1.5 is Out

$
0
0

I've pushed out the PyOpenGL 3.1.5 release. It has some (relatively minor) fixes for wgl and egl operations, and one for what looks like a change in numpy scalar handling in the latest numpy. Available on PyPI now.

Viewing all 22404 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>