Several industry giants in the finance sector are well on their way to implementing machine learning technology that improves operations and guides strategy in multiple departments. So far, machine learning algorithms are being used in…

The post 4 Ways Financial Firms Put Machine Learning to Work appeared first on Anaconda.

This tutorial provides you multiple methods to check if an integer number lies in the given range or not. It includes several examples to bring clarity. Let’s first define the problem. We want to verify whether an integer value lies between two other numbers, for example, 1000 and 7000: So, we need a simple method that can tell us about any numeric value if it belongs to a given range. Hence, in this post, we’ll describe three ways of solving this problem. You can choose which of these suits you the best. Two of these methods works in Python 3,

The post Python Check Integer Number in Range appeared first on Learn Programming and Software Testing.

So far I have successfully added 2 models to dffml out which dffml-model-scratch (Linear regression) is already released and available on pypi. Other scikit model has been merged and is awaiting release.

Doing the scikit made us realize that we can do it in better and a much faster way. We have now decided a different procedure to go about adding scikit classifiers such that we have minimum repeated code and adding classifiers is as simple as appending it to a dictionary.

We have just planned this and I can't really assess how much time it would take, but I hope to complete it before the final evaluation. The models I would be adding can be found here: https://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html

Link to my first authored pypi package: https://pypi.org/project/dffml-model-scratch/

Last time I spent time working on testing on Windows and refining documents. While I was trying to get it running on windows, I found the environment setting is really a problem. One solution is to write a script to automatically configure all the stuffs. In the following week I will try to fix #1 issue that is about NVD database.

The post Dask joins NumFOCUS Sponsored Projects appeared first on NumFOCUS.

Mocking is a powerful technique for isolating tests from undesired interactions among components. But often people find their mock isn’t taking effect, and it’s not clear why. Hopefully this explanation will clear things up.

BTW: it’s really easy to over-use mocking. These are good explanations of alternative approaches:

• Augie Fackler and Nathaniel Manista’s PyCon talk Stop Mocking and Start Testing.
• Itamar Turner-Trauring’s article Fast tests for slow services: why you should use verified fakes.

A quick aside about assignment

Before we get to fancy stuff like mocks, I want to review a little bit about Python assignment. You may already know this, but bear with me. Everything that follows is going to be directly related to this simple example.

Variables in Python are names that refer to values. If we assign a second name, the names don’t refer to each other, they both refer to the same value. If one of the names is then assigned again, the other name isn’t affected:

x23x = 23xy23y = xxy1223x = 12

If this is unfamiliar to you, or you just want to look at more pictures like this, Python Names and Values goes into much more depth about the semantics of Python assignment.

Importing

Let’s say we have a simple module like this:

# mod.py

val = "original"

def update_val():
    global val
    val = "updated"


We want to use val from this module, and also call update_val to change val. There are two ways we could try to do it. At first glance, it seems like they would do the same thing.

The first version imports the names we want, and uses them:

# code1.py

from mod import val, update_val

print(val)
update_val()
print(val)


The second version imports the module, and uses the names as attributes on the module object:

# code2.py

import mod

print(mod.val)
mod.update_val()
print(mod.val)


This seems like a subtle distinction, almost a stylistic choice. But code1.py prints “original original”: the value hasn’t changed! Code2.py does what we expected: it prints “original updated.” Why the difference?

Let’s look at code1.py more closely:

# code1.py

from mod import val, update_val

print(val)
update_val()
print(val)


After “from mod import val”, when we first print val, we have this:

mod.pyval‘original’code1.pyval

“from mod import val” means, import mod, and then do the assignment “val = mod.val”. This makes our name val refer to the same object as mod’s name val.

After “update_val()”, when we print val again, our world looks like this:

mod.pyval‘original’‘updated’code1.pyval

update_val has reassigned mod’s val, but that has no effect on our val. This is the same behavior as our x and y example, but with imports instead of more obvious assignments. In code1.py, “from mod import val” is an assignment from mod.val to val, and works exactly like “y = x” does. Later assignments to mod.val don’t affect our val, just as later assignments to x don’t affect y.

Now let’s look at code2.py again:

# code2.py

import mod

print(mod.val)
mod.update_val()
print(mod.val)


The “import mod” statement means, make my name mod refer to the entire mod module. Accessing mod.val will reach into the mod module, find its val name, and use its value.

mod.pyval‘original’code2.pymod

Then after “update_val()”, mod’s name val has been changed:

mod.pyval‘original’‘updated’code2.pymod

Now we print mod.val again, and see its updated value, just as we expected.

OK, but what about mocks?

Mocking is a fancy kind of assignment: replace an object (or function) with a different one. We’ll use the mock.patch function in a with statement. It makes a mock object, assigns it to the name given, and then restores the original value at the end of the with statement.

Let’s consider this (very roughly sketched) product code and test:

# product.py

from os import listdir

def my_function():
    files = listdir(some_directory)
    # ... use the file names ...


# test.py

def test_it():
    with mock.patch("os.listdir") as listdir:
        listdir.return_value = ['a.txt', 'b.txt', 'c.txt']
        my_function()


After we’ve imported product.py, both the os module and product.py have a name “listdir” which refers to the built-in listdir() function. The references look like this:

os modulelistdirlistdir()product.pylistdir

The mock.patch in our test is really just a fancy assignment to the name “os.listdir”. During the test, the references look like this:

os modulelistdirlistdir()mock!product.pylistdir

You can see why the mock doesn’t work: we’re mocking something, but it’s not the thing our product code is going to call. This situation is exactly analogous to our code1.py example from earlier.

You might be thinking, “ok, so let’s do that code2.py thing to make it work!” If we do, it will work. Your product code and test will now look like this (the test code is unchanged):

# product.py

import os

def my_function():
    files = os.listdir(some_directory)
    # ... use the file names ...


# test.py

def test_it():
    with mock.patch("os.listdir") as listdir:
        listdir.return_value = ['a.txt', 'b.txt', 'c.txt']
        my_function()


When the test is run, the references look like this:

os modulelistdirlistdir()mock!product.pyos

Because the product code refers to the os module, changing the name in the module is enough to affect the product code.

But there’s still a problem: this will mock that function for any module using it. This might be a more widespread effect than you intended. Perhaps your product code also calls some helpers, which also need to list files. The helpers might end up using your mock (depending how they imported os.listdir!), which isn’t what you wanted.

Mock it where it’s used

The best approach to mocking is to mock the object where it is used, not where it is defined. Your product and test code will look like this:

# product.py

from os import listdir

def my_function():
    files = listdir(some_directory)
    # ... use the file names ...


# test.py

def test_it():
    with mock.patch("product.listdir") as listdir:
        listdir.return_value = False
        my_function()


The only difference here from our first try is that we mock “product.listdir”, not “os.listdir”. That seems odd, because listdir isn’t defined in product.py. That’s fine, the name “listdir” is in both the os module and in product.py, and they are both references to the thing you want to mock. Neither is a more real name than the other.

By mocking where the object is used, we have tighter control over what callers are affected. Since we only want product.py’s behavior to change, we mock the name in product.py. This also makes the test more clearly tied to product.py.

As before, our references look like this once product.py has been fully imported:

os modulelistdirlistdir()product.pylistdir

The difference now is how the mock changes things. During the test, our references look like this:

os modulelistdirlistdir()product.pylistdirmock!

The code in product.py will use the mock, and no other code will. Just what we wanted!

Is this OK?

At this point, you might be concerned: it seems like mocking is kind of delicate. Notice that even with our last example, how we create the mock depends on something as arbitrary as how we imported the function. If our code had “import os” at the top, we wouldn’t have been able to create our mock properly. This is something that could be changed in a refactoring, but at least mock.patch will fail in that case.

You are right to be concerned: mocking is delicate. It depends on implementation details of the product code to construct the test. There are many reasons to be wary of mocks, and there are other approaches to solving the problems of isolating your product code from problematic dependencies.

If you do use mocks, at least now you know how to make them work, but again, there are other approaches. See the links at the top of this page.

This is going to be a blog post about coding up a much requested feature by the community.

What did I do this week?

This week my mentor asked me to work on an interesting feature which was requested by the community. The issue can be found here[1]. I have sent three patches to Phabricator in which two of them have been merged. You can find the patches here:

[2]: D6709 config: add --all flag to show all known configs

[3]: D6712 config: fix defaultvalue template keyword

[4]: D6704 config: add defaultvalue template keyword

Mercurial has a command called `showconfig` which by default without any flags shows all the config items as set by the hgrc. But, I had to add a flag to show all config items in the config registrar. So, I added an `--all` flag to show all known config items. Also, I added a new template keyword called `defaultvalue` to show the default value of the config item. Until now, nothing was shown on trying to see a default value of a config item which has not set a value by the user. Community was much interested in this and Yuja had suggested the UX. For the `--all` flag to work, I had to walk in such a way that I am digging up the registrar other than just user config options. I had to skip the items which had dynamic default values.

What is coming up next?

I haven't decided upon the exact feature on work on next week. My mentor asked me to select one of either the bug tracker or the WeShouldDoThat page depending upon the community interest.

Did you get stuck anywhere?

I had a starting trouble to find out a way to access the items from the config registrar. Then, my mentor Pulkit told me about a variable called `self._knownconfigs`. That was a turning point. The rest of the task happened really fast.

In this article, we will create a new function which will search for the currency pair in the tkinter display panel then highlight that currency pair with green background.

First we will create the button which will call the find statement function.

action_find = tk.Button(buttonFrame, text="Find", state=DISABLED,
                       command=find_statement)  # button used to highlight the currency pair
action_find.pack(side=LEFT)

Next we will create the function.

def find_statement(): # highlight the currency pair

    countVar = StringVar()  # use to hold the character count
    text_widget.tag_remove("search", "1.0", "end")  # cleared the hightlighted currency pair

    # highlight the background of the searched currency pair
    pos = text_widget.search(based.get(), "1.0", stopindex="end", count=countVar)
    text_widget.tag_configure("search", background="green")
    end_pos = float(pos) + float(0.96)
    text_widget.tag_add("search", pos, str(end_pos))
    pos = float(pos) + 2.0
    text_widget.see(str(pos))

The above function will search for the currency symbol which matches the one selected from the currency combo box, then highlight that btc/currency pair.

BTC vs USD exchange rate

This tutorial explains Python float() method that takes a number or string and returns a floating-point value. If it is not able to convert string to float, then it raises the ValueError. Let’s try to understand how to use it with the help of simple examples. 1. float() syntax 2. float() with +ve numbers 3. float() with -ve numbers 4. float() with strings of numbers 5. float() with invalid inputs Let’s now go through each of the section one by one. Generate Float Range in Python Python float() with Examples Float() is a built-in Python function that converts a number

The post Python Float() appeared first on Learn Programming and Software Testing.

1. Find all the numbers in one file that are not in another file in python - [10/4]
2. Incrementing add under condition in pandas - [8/5]
3. Can I split this column containing a mix of tuples/None more efficiently? - [6/4]
4. Measuring the diameter pictures of holes in metal parts, photographed with telecentric, monochrome camera with opencv - [6/3]
5. Which cubic spline method does scipy ndimage use for affine_transform? - [6/1]
6. Unable to close worksheet in xlsxwriter - [6/0]
7. How to change entire row if NaN present if a single column has NaN - [5/5]
8. How to find intersections of ranges of float tuples across two lists in python? - [5/4]
9. Find index of the first and/or last value in a column that is not NaN - [5/4]
10. How to join many fragmented time series in one regular Pandas DataFrame in Python - [5/4]

Continue reading"sphinxcontrib.datatemplates 0.5.0"

In the last weeks Alex, Mainak and I were working on making the mne-study-template compatible with the Brain Imaging Data Structure (BIDS). This process involved testing the study template on many different BIDS datasets, to see where the template is not yet general enough, or where bugs are hidden.

To improve our testing, we wanted to set up a continuous integration service that automatically runs the template over different datasets for every git commit that we make. Understandably however, all integration services (such as CircleCI) have a restriction on how much data can be downloaded in a single test run. This meant we needed lightweight solutions that can pull in only small parts of the datasets we wanted to test.

And this is where git-annex and datalad enter the conversation.

git-annex

git-annex is a software that allows managing large files with git. One could see git-annex as a competitor to git-lfs ("Large File Storage"), because both solve the same problem. They differ in their technical implementation and have different pros and cons. A good summary can be found in this stackoverflow post: https://stackoverflow.com/a/39338319/5201771

Datalad

Datalad is a Python library that "builds on top of git-annex and extends it with an intuitive command-line interface". Datalad can also be seen as a "portal" to many git-annex datasets openly accessible in the Internet.

Recipe: How to turn any online dataset into a GitHub-hosted git-annex repository

Requirements: git-annex, datalad, unix-based system

Installing git-annex worked great using conda and the conda-forge for package git-annex:

conda install git-annex -c conda-forge

The installation of datalad is very simple via pip:

pip install datalat

Now find the dataset you want to turn into a git-annex repository. In this example, we'll use the Matching Pennies dataset hosted on OSF: https://osf.io/cj2dr/

We now need to create a CSV file with two columns. Each row of the file will reflect a single file we want to have in the git-annex repository. In the first column we will store the file path relative to the root of the dataset, and in the second column we will store the download URL of that file.

Usually, the creation of this CSV file should be automated using software. For OSF, we have the datalad-osf package which can do the job. However, that package is still in development so I wrote my own function, which involved picking out many download URLs and file names by hand :-(

On OSF, the URLs are given by https://osf.io/<key>/download</key> where <key> is dependent on the file.</key>

See two example rows of my CSV (note the headers, which are important later on):

fpath, url
sub-05/eeg/sub-05_task-matchingpennies_channels.tsv, https://osf.io/wdb42/download
sourcedata/sub-05/eeg/sub-05_task-matchingpennies_eeg.xdf, https://osf.io/agj2q/download

Once your CSV file is ready, and git-annex and datalad are installed, it is time to switch to the command line.

# create the git-annex repository
datalad create eeg_matchingpennies

# download the files in the CSV and commit them
datalad addurls mp.csv "{url}""{fpath}" -d eeg_matchingpennies/  

# print our files and the references where to find them
# will show a local address (the downloaded files) and a web address (OSF)
git annex whereis  

# Make a clone of your fresh repository
datalad install -s eeg_matchingpennies clone

# go to the clone
cd clone  

# disconnect the clone from the local data sources
git annex dead origin  

# disconnect the clone from its origin
git remote rm origin  

# print our files again, however: Notice how all references to
# the local files are gone. Only the web references persist
git annex whereis  

Now make a new empty repository on GitHub: https://github.com/sappelhoff/eeg_matchingpennies

# add a new origin to the clone
git remote add origin https://github.com/sappelhoff/eeg_matchingpennies

# upload the git-annex repository to GitHub
datalad publish --to origin  

Now your dataset is ready to go! Try it out as described below:
 

# clone the repository into your current folder
datalad install https://github.com/sappelhoff/eeg_matchingpennies

# go to your repository
cd eeg_matchingpennies

# get the data for sub-05 (not just the reference to it)
datalad get sub-05

# get only a single file
datalad get sub-05/eeg/sub-05_task-matchingpennies_eeg.vhdr  

# get all the data
datalad get .  

Acknowledgments and further reading

I am very thankful to Kyle A. Meyer and Yaroslav Halchenko for their support in this GitHub issue thread. If you are running into issues with my recipe, I recommend that you fully read that GitHub issue thread.

Hey everyone! This is what I did this week. In this phase, I worked mostly on the understanding of mergestate for developing a patch for hg update --abort. This is the last part that I need to complete before GSoC deadline.

What did I do this week?

As stated previously I worked on evolve and was able to get my patches for hg abort for evolve repository merged which included adding support for hg evolve and hg pick. Also, I worked on hg continue support and was able to add support for evolve. I will submit the patch for that once pick is also complete.
As for hg update --abort I went through the code for hg resolve -u and
also -at:local. I learned what merge state stores and started writing the code adding --abort flag for hg update.
I also received reviews for the continue patches I sent to the core repo and modified my patches according to that.

What is coming up next?

This week I am planning to complete support for hg continue for evolve. Also, I would send my first patch for hg update --abort.

Did you get stuck anywhere?

I got stuck with the logic of abort flag but @Pulkit pointed me to go through the code and documentation of mergestate which helped me get a better understanding of the workflow.

Hi everyone!

We made huge progress with the new Kurucz parser (it's almost ready!). As I said last week, this is a difficult task and involves reestructuring different pieces of existing code and some knowledge the atomic structure and physics.

In this notebook you can see a demonstration on how the new GFALL class returns the same DataFrames than the old API. Next step consist in adding metastable flags for lines and levels.

Once this work is completed we should discuss how to move on. Probably we want to automatize the process of making new atomic files: every time a source is updated (for example NIST atomic weights) trigger a new build.

What did you do this week?

So, As I previously mention in this phase I have to write documentation and refactor the code. So, I started with writing documentation for the config options provided by the plugin. I have written all the config option properties which you can pass to plugin and extended your website feature. After adding documentation I started refactoring the code. I have flatten the code by creating separate function for each event I have added. I added the create, modified event and delete event. I refactor the code related to createEvent. You can see both the pr below.

https://github.com/collective/gatsby-source-plone/pull/231

https://github.com/collective/gatsby-source-plone/pull/229

What is coming up next?

So, I think that documentation part is over because I have created a new file for config options which covered all the information which I have made changes. I am currently working on refactoring deleteEvent and after that modified options. So, you can see lot of refactoring of code in coming weeks.

Did you get stuck anywhere?

 During start of refactoring I stuck at a issue of not finding function and related stuff you can see on the pr but I find it out and able to solve it. Other than that I am having a good time :)

Hello there, 

Its good to see you here.

So, this week I created a new PR for the new cheatsheet and also to fix a few responsiveness bugs on the landing page. My work for that PR is done. I have to fix a few things on the new cheatsheet too. There are a few features that require popper.js but thing is that popper.js is not working properly with the CDN so I need to download it and use it offline like we use bootstrap in vendors.min.css file. For using popper.js offline I need to edit a lot of stuff including gulp files. That's why I am first going to design a new documentation page for EOS Icons first then later I will fix the popper issue.

Ahh. I think that's it for now.

Peace!