Need help with swifter?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

jmcarpenter2
1.6K Stars 76 Forks MIT License 423 Commits 17 Opened issues

Description

A package which efficiently applies any function to a pandas dataframe or series in the fastest available manner

Services available

!
?

Need anything else?

Contributors list

# 56,191
Python
paralle...
pandas-...
pandas
318 commits
# 70,540
Python
cloudwa...
s3
lambda
6 commits
# 66,588
HTML
node
test-fr...
C++
3 commits
# 59,495
jupyter
Apache ...
q
communi...
2 commits
# 46,478
reactst...
Bootstr...
jupyter
R
1 commit
# 68,861
bokeh
c-sharp
zstd
F#
1 commit
# 360,612
Python
Shell
least-s...
paralle...
1 commit
# 473,543
Python
paralle...
pandas-...
pandas
1 commit
# 9,572
pandas
Python
Shell
datafra...
1 commit
# 460,194
paralle...
pandas
Shell
simulat...
1 commit
# 467,639
Python
paralle...
pandas-...
pandas
1 commit
# 107,020
text-se...
program...
autohot...
ahk
1 commit

swifter

A package which efficiently applies any function to a pandas dataframe or series in the fastest available manner.

PyPI version CircleCI codecov Code style: black GitHub stars PyPI - Downloads

Blog posts

Documentation

To know about latest improvements, please check the changelog.

Further documentations on swifter is available here.

Check out the examples notebook, along with the speed benchmark notebook. The benchmarks are created using the library perfplot.

Installation:

$ pip install -U pandas # upgrade pandas
$ pip install swifter # first time installation

$ pip install -U swifter # upgrade to latest version if already installed

alternatively, to install on Anaconda:

conda install -c conda-forge swifter

...after installing, import

swifter
into your code along with
pandas
using:
python
import pandas as pd
import swifter

...alternatively,

swifter
can be used with
modin
dataframes in the same manner:
python
import modin.pandas as pd
import swifter
NOTE: if you import swifter before modin, you will have to additionally register modin:
swifter.register_modin()

Easy to use

df = pd.DataFrame({'x': [1, 2, 3, 4], 'y': [5, 6, 7, 8]})

runs on single core

df['x2'] = df['x'].apply(lambda x: x**2)

runs on multiple cores

df['x2'] = df['x'].swifter.apply(lambda x: x**2)

use swifter apply on whole dataframe

df['agg'] = df.swifter.apply(lambda x: x.sum() - x.min())

use swifter apply on specific columns

df['outCol'] = df[['inCol1', 'inCol2']].swifter.apply(my_func) df['outCol'] = df[['inCol1', 'inCol2', 'inCol3']].swifter.apply(my_func, positional_arg, keyword_arg=keyword_argval)

Vectorizes your function, when possible

Alt text Alt text

When vectorization is not possible, automatically decides which is faster: to use dask parallel processing or a simple pandas apply

Alt text Alt text

Notes

  1. The function is documented in the .py file. In Jupyter Notebooks, you can see the docs by pressing Shift+Tab(x3). Also, check out the complete documentation here along with the changelog.

  2. Please upgrade your version of pandas, as the pandas extension api used in this module is a recent addition to pandas.

  3. Import modin before importing swifter, if you wish to use modin with swifter. Otherwise, use

    swifter.register_modin()
    to access it.
  4. Do not use swifter to apply a function that modifies external variables. Under the hood, swifter does sample applies to optimize performance. These sample applies will modify the external variable in addition to the final apply. Thus, you will end up with an erroneously modified external variable.

  5. It is advised to disable the progress bar if calling swifter from a forked process as the progress bar may get confused between various multiprocessing modules.

  6. If swifter return is different than pandas try explicitly casting type e.g.:

    df.swifter.apply(lambda x: float(np.angle(x)))

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.