Bump version to 0.1.96

Another info[] fix for #1256

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,20 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+*** READ BEFORE POSTING ***
+
+Before posting an issue - please upgrade to the latest version and confirm the issue/bug is still there.
+
+Upgrade using:
+`$ pip install yfinance --upgrade --no-cache-dir`
+
+Bug still there? Delete this content and submit your bug report here and provide the following, as best you can:
+
+- Simple code that reproduces your problem
+- The error message

.github/dependabot.yml

@@ -0,0 +1,7 @@
+version: 2
+updates:
+- package-ecosystem: pip
+  directory: "/"
+  schedule:
+    interval: daily
+  open-pull-requests-limit: 10

.github/workflows/ci.yml

@@ -0,0 +1,18 @@
+name: ci
+on:
+  push:
+    branches:
+      - master
+      - main
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-python@v2
+        with:
+          python-version: 3.x
+      - run: pip install -r requirements.txt
+      - run: pip install mkdocstrings==0.14.0
+      - run: pip install mkdocs-material
+      - run: mkdocs gh-deploy --force

.github/workflows/python-publish.yml

@@ -0,0 +1,31 @@
+# This workflow will upload a Python Package using Twine when a release is created
+# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
+
+name: Upload Python Package
+
+on:
+  release:
+    types: [created]
+
+jobs:
+  deploy:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python
+      uses: actions/setup-python@v2
+      with:
+        python-version: '3.x'
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install setuptools wheel twine
+    - name: Build and publish
+      env:
+        TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
+        TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
+      run: |
+        python setup.py sdist bdist_wheel
+        twine upload dist/*

.gitignore

@@ -3,3 +3,9 @@ yfinance/__pycache__/*
 dist
 yfinance.egg-info
 *.pyc
+.coverage
+.vscode/
+build/
+*.html
+*.css
+*.png

.travis.yml

@@ -6,26 +6,30 @@ fast_finish: true
 matrix:
   include:
     - python: 2.7
-    - python: 3.5
     - python: 3.6
     - python: 3.7
+    - python: 3.8
+    - python: 3.9
       dist: xenial
       sudo: true
 
 install:
   - pip install Cython
+  - pip install pytest>=4.6
+  - pip install pytest-cov
+  - pip install coveralls
   - pip install -r requirements.txt
   - pip install .
 
 script:
-  - nosetests
+  - pytest --cov=yfinance/
 
 after_success:
   - coveralls
 
 branches:
   only:
-    - master
+    - main
 
 notifications:
   slack: tradologics:HcnS6XusfcuS02waQPCG18oc

CHANGELOG.rst

@@ -1,10 +1,196 @@
 Change Log
 ===========
 
-0.1.55 - to be released
+0.1.96
 ------
-- fixed institutional investors and mutual fund holders issue (#459)
-- fix for UTC timestamps in options chains (#429)
+- Fix info[] not caching #1258
+
+0.1.95
+------
+- Fix info[] bug #1257
+
+0.1.94
+------
+- Fix delisted ticker info[]
+
+0.1.93
+------
+- Fix Ticker.shares
+
+0.1.92
+------
+- Decrypt new Yahoo encryption #1255
+
+0.1.90
+------
+- Restore lxml req, increase min ver #1237
+
+0.1.89
+------
+- Remove unused incompatible dependency #1222
+- Fix minimum Pandas version #1230
+
+0.1.87
+------
+- Fix localizing midnight when non-existent (DST) #1176
+- Fix thread deadlock in bpython #1163
+
+0.1.86
+------
+- Fix 'trailingPegRatio' #1141
+- Improve handling delisted tickers #1142
+- Fix corrupt tkr-tz-csv halting code #1162
+- Change default start to 1900-01-01 #1170
+
+0.1.85
+------
+- Fix info['log_url'] #1062
+- Fix handling delisted ticker #1137
+
+0.1.84
+------
+- Make tz-cache thread-safe
+
+0.1.83
+------
+- Reduce spam-effect of tz-fetch
+
+0.1.81
+------
+- Fix unhandled tz-cache exception #1107
+
+0.1.80
+------
+- Fix `download(ignore_tz=True)` for single ticker #1097
+- Fix rare case of error "Cannot infer DST time" #1100
+
+0.1.79
+------
+- Fix when Yahoo returns price=NaNs on dividend day
+
+0.1.78
+------
+- Fix download() when different timezones #1085
+
+0.1.77
+------
+- Fix user experience bug #1078
+
+0.1.75
+------
+- Fixed datetime-related issues: #1048
+- Add 'keepna' argument #1032
+- Speedup Ticker() creation #1042
+- Improve a bugfix #1033
+
+0.1.74
+------
+- Fixed bug introduced in 0.1.73 (sorry :/)
+
+0.1.73
+------
+- Merged several PR that fixed misc issues
+
+0.1.72
+------
+- Misc bugfixs
+
+0.1.71
+------
+- Added Tickers(…).news()
+- Return empty DF if YF missing earnings dates
+- Fix EPS % to 0->1
+- Fix timezone handling
+- Fix handling of missing data
+- Clean&format earnings_dates table
+- Add ``.get_earnings_dates()`` to retreive earnings calendar
+- Added ``.get_earnings_history()`` to fetch earnings data
+
+0.1.70
+------
+- Bug fixed - Closes #937
+
+0.1.69
+------
+- Bug fixed - #920
+
+0.1.68
+------
+- Upgraded requests dependency
+- Removed Python 3.5 support
+
+0.1.67
+------
+- Added legal disclaimers to make sure people are aware that this library is not affiliated, endorsed, or vetted by Yahoo, Inc.
+
+0.1.66
+------
+- Merged PR to allow yfinance to be pickled
+
+0.1.65
+------
+- Merged PRs to fix some bugs
+- Added lookup by ISIN ``utils.get_all_by_isin(...)``, ``utils.get_ticker_by_isin(...)``, ``utils.get_info_by_isin(...)``, ``utils.get_news_by_isin(...)``
+- ``yf.Ticker``, ``yf.Tickers``, and ``yf.download`` will auto-detect ISINs and convert them to tickers
+- Propagating timeout parameter through code, setting request.get(timeout)
+- Adds ``Ticker.analysis`` and ``Ticker.get_analysis(...)``
+
+0.1.64
+------
+- Merged PRs to fix some bugs
+- Added ``Ticker.stats()`` method
+- Added ``Ticker.news`` property
+- Providing topHoldings for ETFs
+- Replaceed drop duplicate prices with indexes
+- Added pre-market price to ``Ticker.info``
+
+
+0.1.63
+------
+- Duplicates and missing rows cleanup
+
+0.1.62
+------
+- Added UserAgent to all requests (via ```utils.user_agent_headers```)
+
+0.1.61
+------
+- Switched to using ```query2.finance.yahoo.com```, which used HTTP/1.1
+
+0.1.60
+------
+- Gracefully fail on misc operations (options, auto/back adjustments, etc)
+- Added financial data to ```info()```
+- Using session headers
+- Get price even if open price not available
+- Argument added for silencing error printing
+- Merged PRs to fix some bugs
+
+0.1.59
+------
+- Added custom requests session instance support in holders
+
+0.1.58
+------
+- Allow specifying a custom requests session instance
+
+0.1.57
+------
+- Added Conversion rate hint using 'financialCurrency' property in earnings
+- Add important try+catch statements
+- Fixed issue with 1 hour interval
+- Merged PRs to fix some bugs
+- Fixed issue with special characters in tickers
+
+0.1.56
+------
+- Updated numpy version
+- Merged PRs to fix some bugs
+
+0.1.55
+------
+- Fixed institutional investors and mutual fund holders issue (#459)
+- Fix for UTC timestamps in options chains (#429)
 
 0.1.54
 ------
@@ -12,7 +198,7 @@ Change Log
 
 0.1.53
 ------
-- Added `Ticker.isin` + `Ticker.get_isin(...)`. This is still experimental. Do not rely on it for production.
+- Added ``Ticker.isin`` + ``Ticker.get_isin(...)``. This is still experimental. Do not rely on it for production.
 - Bug fixed: holders were always returning results for MSFT
 
 0.1.52
@@ -21,24 +207,24 @@ Change Log
 
 0.1.51
 ------
-- Added holdings data (`Ticker.major_holders` and `Ticker.institutional_holders`)
-- Added logo url to `Ticker.info`
+- Added holdings data (``Ticker.major_holders`` and ``Ticker.institutional_holders``)
+- Added logo url to ``Ticker.info``
 - Handling different date formats in fundamentals
 - Faster JSON parsing using regex
 - Trying to re-download JSON twice before giving up
 - Using ujson instead of json if installed
-- Fixed (more) `ticker.info` issues
+- Fixed (more) ``ticker.info`` issues
 - Misc bugfixes
 
 0.1.50
 ------
-- Fixed `ticker.info` issues
+- Fixed ``ticker.info`` issues
 - Handle sustainability index error
 - Added test script based on @GregoryMorse's pull request
 
 0.1.49
 ------
-- Fixed `elementwise comparison`` warning
+- Fixed ``elementwise comparison`` warning
 
 0.1.48
 ------

README.md

@@ -0,0 +1,311 @@
+# Download market data from Yahoo! Finance's API
+
+<table border=1 cellpadding=10><tr><td>
+
+#### \*\*\* IMPORTANT LEGAL DISCLAIMER \*\*\*
+
+---
+
+**Yahoo!, Y!Finance, and Yahoo! finance are registered trademarks of
+Yahoo, Inc.**
+
+yfinance is **not** affiliated, endorsed, or vetted by Yahoo, Inc. It's
+an open-source tool that uses Yahoo's publicly available APIs, and is
+intended for research and educational purposes.
+
+**You should refer to Yahoo!'s terms of use**
+([here](https://policies.yahoo.com/us/en/yahoo/terms/product-atos/apiforydn/index.htm),
+[here](https://legal.yahoo.com/us/en/yahoo/terms/otos/index.html), and
+[here](https://policies.yahoo.com/us/en/yahoo/terms/index.htm)) **for
+details on your rights to use the actual data downloaded. Remember - the
+Yahoo! finance API is intended for personal use only.**
+
+</td></tr></table>
+
+---
+
+<a target="new" href="https://pypi.python.org/pypi/yfinance"><img border=0 src="https://img.shields.io/badge/python-2.7,%203.6+-blue.svg?style=flat" alt="Python version"></a>
+<a target="new" href="https://pypi.python.org/pypi/yfinance"><img border=0 src="https://img.shields.io/pypi/v/yfinance.svg?maxAge=60%" alt="PyPi version"></a>
+<a target="new" href="https://pypi.python.org/pypi/yfinance"><img border=0 src="https://img.shields.io/pypi/status/yfinance.svg?maxAge=60" alt="PyPi status"></a>
+<a target="new" href="https://pypi.python.org/pypi/yfinance"><img border=0 src="https://img.shields.io/pypi/dm/yfinance.svg?maxAge=2592000&label=installs&color=%2327B1FF" alt="PyPi downloads"></a>
+<a target="new" href="https://travis-ci.com/github/ranaroussi/yfinance"><img border=0 src="https://img.shields.io/travis/ranaroussi/yfinance/main.svg?maxAge=1" alt="Travis-CI build status"></a>
+<a target="new" href="https://www.codefactor.io/repository/github/ranaroussi/yfinance"><img border=0 src="https://www.codefactor.io/repository/github/ranaroussi/yfinance/badge" alt="CodeFactor"></a>
+<a target="new" href="https://github.com/ranaroussi/yfinance"><img border=0 src="https://img.shields.io/github/stars/ranaroussi/yfinance.svg?style=social&label=Star&maxAge=60" alt="Star this repo"></a>
+<a target="new" href="https://twitter.com/aroussi"><img border=0 src="https://img.shields.io/twitter/follow/aroussi.svg?style=social&label=Follow&maxAge=60" alt="Follow me on twitter"></a>
+
+
+**yfinance** offers a threaded and Pythonic way to download market data from [Yahoo!Ⓡ finance](https://finance.yahoo.com).
+
+→ Check out this [Blog post](https://aroussi.com/#post/python-yahoo-finance) for a detailed tutorial with code examples.
+
+[Changelog »](https://github.com/ranaroussi/yfinance/blob/main/CHANGELOG.rst)
+
+---
+
+## Quick Start
+
+### The Ticker module
+
+The `Ticker` module, which allows you to access ticker data in a more Pythonic way:
+
+```python
+import yfinance as yf
+
+msft = yf.Ticker("MSFT")
+
+# get stock info
+msft.info
+
+# get historical market data
+hist = msft.history(period="max")
+
+# show actions (dividends, splits)
+msft.actions
+
+# show dividends
+msft.dividends
+
+# show splits
+msft.splits
+
+# show financials
+msft.financials
+msft.quarterly_financials
+
+# show major holders
+msft.major_holders
+
+# show institutional holders
+msft.institutional_holders
+
+# show balance sheet
+msft.balance_sheet
+msft.quarterly_balance_sheet
+
+# show cashflow
+msft.cashflow
+msft.quarterly_cashflow
+
+# show earnings
+msft.earnings
+msft.quarterly_earnings
+
+# show sustainability
+msft.sustainability
+
+# show analysts recommendations
+msft.recommendations
+
+# show next event (earnings, etc)
+msft.calendar
+
+# show all earnings dates
+msft.earnings_dates
+
+# show ISIN code - *experimental*
+# ISIN = International Securities Identification Number
+msft.isin
+
+# show options expirations
+msft.options
+
+# show news
+msft.news
+
+# get option chain for specific expiration
+opt = msft.option_chain('YYYY-MM-DD')
+# data available via: opt.calls, opt.puts
+```
+
+If you want to use a proxy server for downloading data, use:
+
+```python
+import yfinance as yf
+
+msft = yf.Ticker("MSFT")
+
+msft.history(..., proxy="PROXY_SERVER")
+msft.get_actions(proxy="PROXY_SERVER")
+msft.get_dividends(proxy="PROXY_SERVER")
+msft.get_splits(proxy="PROXY_SERVER")
+msft.get_balance_sheet(proxy="PROXY_SERVER")
+msft.get_cashflow(proxy="PROXY_SERVER")
+msft.option_chain(..., proxy="PROXY_SERVER")
+...
+```
+
+To use a custom `requests` session (for example to cache calls to the
+API or customize the `User-agent` header), pass a `session=` argument to
+the Ticker constructor.
+
+```python
+import requests_cache
+session = requests_cache.CachedSession('yfinance.cache')
+session.headers['User-agent'] = 'my-program/1.0'
+ticker = yf.Ticker('msft aapl goog', session=session)
+# The scraped response will be stored in the cache
+ticker.actions
+```
+
+To initialize multiple `Ticker` objects, use
+
+```python
+import yfinance as yf
+
+tickers = yf.Tickers('msft aapl goog')
+# ^ returns a named tuple of Ticker objects
+
+# access each ticker using (example)
+tickers.tickers.MSFT.info
+tickers.tickers.AAPL.history(period="1mo")
+tickers.tickers.GOOG.actions
+```
+
+### Fetching data for multiple tickers
+
+```python
+import yfinance as yf
+data = yf.download("SPY AAPL", start="2017-01-01", end="2017-04-30")
+```
+
+I've also added some options to make life easier :)
+
+```python
+data = yf.download(  # or pdr.get_data_yahoo(...
+        # tickers list or string as well
+        tickers = "SPY AAPL MSFT",
+
+        # use "period" instead of start/end
+        # valid periods: 1d,5d,1mo,3mo,6mo,1y,2y,5y,10y,ytd,max
+        # (optional, default is '1mo')
+        period = "ytd",
+
+        # fetch data by interval (including intraday if period < 60 days)
+        # valid intervals: 1m,2m,5m,15m,30m,60m,90m,1h,1d,5d,1wk,1mo,3mo
+        # (optional, default is '1d')
+        interval = "1m",
+
+        # Whether to ignore timezone when aligning ticker data from 
+        # different timezones. Default is True. False may be useful for 
+        # minute/hourly data.
+        ignore_tz = False,
+
+        # group by ticker (to access via data['SPY'])
+        # (optional, default is 'column')
+        group_by = 'ticker',
+
+        # adjust all OHLC automatically
+        # (optional, default is False)
+        auto_adjust = True,
+
+        # download pre/post regular market hours data
+        # (optional, default is False)
+        prepost = True,
+
+        # use threads for mass downloading? (True/False/Integer)
+        # (optional, default is True)
+        threads = True,
+
+        # proxy URL scheme use use when downloading?
+        # (optional, default is None)
+        proxy = None
+    )
+```
+
+### Timezone cache store
+
+When fetching price data, all dates are localized to stock exchange timezone. 
+But timezone retrieval is relatively slow, so yfinance attemps to cache them 
+in your users cache folder. 
+You can direct cache to use a different location with `set_tz_cache_location()`:
+```python
+import yfinance as yf
+yf.set_tz_cache_location("custom/cache/location")
+...
+```
+
+### Managing Multi-Level Columns
+
+The following answer on Stack Overflow is for [How to deal with
+multi-level column names downloaded with
+yfinance?](https://stackoverflow.com/questions/63107801)
+
+-   `yfinance` returns a `pandas.DataFrame` with multi-level column
+    names, with a level for the ticker and a level for the stock price
+    data
+    -   The answer discusses:
+        -   How to correctly read the the multi-level columns after
+            saving the dataframe to a csv with `pandas.DataFrame.to_csv`
+        -   How to download single or multiple tickers into a single
+            dataframe with single level column names and a ticker column
+
+---
+
+## `pandas_datareader` override
+
+If your code uses `pandas_datareader` and you want to download data
+faster, you can "hijack" `pandas_datareader.data.get_data_yahoo()`
+method to use **yfinance** while making sure the returned data is in the
+same format as **pandas\_datareader**'s `get_data_yahoo()`.
+
+```python
+from pandas_datareader import data as pdr
+
+import yfinance as yf
+yf.pdr_override() # <== that's all it takes :-)
+
+# download dataframe
+data = pdr.get_data_yahoo("SPY", start="2017-01-01", end="2017-04-30")
+```
+
+---
+
+## Installation
+
+Install `yfinance` using `pip`:
+
+``` {.sourceCode .bash}
+$ pip install yfinance --upgrade --no-cache-dir
+```
+
+To install `yfinance` using `conda`, see
+[this](https://anaconda.org/ranaroussi/yfinance).
+
+### Requirements
+
+-   [Python](https://www.python.org) \>= 2.7, 3.4+
+-   [Pandas](https://github.com/pydata/pandas) \>= 1.3.0
+-   [Numpy](http://www.numpy.org) \>= 1.16.5
+-   [requests](http://docs.python-requests.org/en/master/) \>= 2.26
+-   [lxml](https://pypi.org/project/lxml/) \>= 4.9.1
+-   [appdirs](https://pypi.org/project/appdirs) \>= 1.4.4
+-   [cryptography](https://pypi.org/project/cryptography) \>=3.3.2
+
+### Optional (if you want to use `pandas_datareader`)
+
+-   [pandas\_datareader](https://github.com/pydata/pandas-datareader)
+    \>= 0.4.0
+
+---
+
+### Legal Stuff
+
+**yfinance** is distributed under the **Apache Software License**. See
+the [LICENSE.txt](./LICENSE.txt) file in the release for details.
+
+
+AGAIN - yfinance is **not** affiliated, endorsed, or vetted by Yahoo, Inc. It's
+an open-source tool that uses Yahoo's publicly available APIs, and is
+intended for research and educational purposes. You should refer to Yahoo!'s terms of use
+([here](https://policies.yahoo.com/us/en/yahoo/terms/product-atos/apiforydn/index.htm),
+[here](https://legal.yahoo.com/us/en/yahoo/terms/otos/index.html), and
+[here](https://policies.yahoo.com/us/en/yahoo/terms/index.htm)) for
+detailes on your rights to use the actual data downloaded.
+
+---
+
+### P.S.
+
+Please drop me an note with any feedback you have.
+
+**Ran Aroussi**

README.rst

@@ -1,277 +0,0 @@
-Yahoo! Finance market data downloader
-=====================================
-
-.. image:: https://img.shields.io/badge/python-2.7,%203.4+-blue.svg?style=flat
-    :target: https://pypi.python.org/pypi/yfinance
-    :alt: Python version
-
-.. image:: https://img.shields.io/pypi/v/yfinance.svg?maxAge=60
-    :target: https://pypi.python.org/pypi/yfinance
-    :alt: PyPi version
-
-.. image:: https://img.shields.io/pypi/status/yfinance.svg?maxAge=60
-    :target: https://pypi.python.org/pypi/yfinance
-    :alt: PyPi status
-
-.. image:: https://img.shields.io/pypi/dm/yfinance.svg?maxAge=2592000&label=installs&color=%2327B1FF
-    :target: https://pypi.python.org/pypi/yfinance
-    :alt: PyPi downloads
-
-.. image:: https://img.shields.io/travis/ranaroussi/yfinance/master.svg?maxAge=1
-    :target: https://travis-ci.com/ranaroussi/yfinance
-    :alt: Travis-CI build status
-
-.. image:: https://www.codefactor.io/repository/github/ranaroussi/yfinance/badge
-    :target: https://www.codefactor.io/repository/github/ranaroussi/yfinance
-    :alt: CodeFactor
-
-.. image:: https://img.shields.io/github/stars/ranaroussi/yfinance.svg?style=social&label=Star&maxAge=60
-    :target: https://github.com/ranaroussi/yfinance
-    :alt: Star this repo
-
-.. image:: https://img.shields.io/twitter/follow/aroussi.svg?style=social&label=Follow&maxAge=60
-    :target: https://twitter.com/aroussi
-    :alt: Follow me on twitter
-
-\
-
-Ever since `Yahoo! finance <https://finance.yahoo.com>`_ decommissioned
-their historical data API, many programs that relied on it to stop working.
-
-**yfinance** aimes to solve this problem by offering a reliable, threaded,
-and Pythonic way to download historical market data from Yahoo! finance.
-
-
-NOTE
-~~~~
-
-The library was originally named ``fix-yahoo-finance``, but
-I've since renamed it to ``yfinance`` as I no longer consider it a mere "fix".
-For reasons of backward-competability, ``fix-yahoo-finance`` now import and
-uses ``yfinance``, but you should install and use ``yfinance`` directly.
-
-`Changelog » <./CHANGELOG.rst>`__
-
------
-
-==> Check out this `Blog post <https://aroussi.com/#post/python-yahoo-finance>`_ for a detailed tutorial with code examples.
-
------
-
-Quick Start
-===========
-
-The Ticker module
-~~~~~~~~~~~~~~~~~
-
-The ``Ticker`` module, which allows you to access
-ticker data in a more Pythonic way:
-
-Note: yahoo finance datetimes are received as UTC.
-
-.. code:: python
-
-    import yfinance as yf
-
-    msft = yf.Ticker("MSFT")
-
-    # get stock info
-    msft.info
-
-    # get historical market data
-    hist = msft.history(period="max")
-
-    # show actions (dividends, splits)
-    msft.actions
-
-    # show dividends
-    msft.dividends
-
-    # show splits
-    msft.splits
-
-    # show financials
-    msft.financials
-    msft.quarterly_financials
-
-    # show major holders
-    msft.major_holders
-
-    # show institutional holders
-    msft.institutional_holders
-
-    # show balance sheet
-    msft.balance_sheet
-    msft.quarterly_balance_sheet
-
-    # show cashflow
-    msft.cashflow
-    msft.quarterly_cashflow
-
-    # show earnings
-    msft.earnings
-    msft.quarterly_earnings
-
-    # show sustainability
-    msft.sustainability
-
-    # show analysts recommendations
-    msft.recommendations
-
-    # show next event (earnings, etc)
-    msft.calendar
-
-    # show ISIN code - *experimental*
-    # ISIN = International Securities Identification Number
-    msft.isin
-
-    # show options expirations
-    msft.options
-
-    # get option chain for specific expiration
-    opt = msft.option_chain('YYYY-MM-DD')
-    # data available via: opt.calls, opt.puts
-
-If you want to use a proxy server for downloading data, use:
-
-.. code:: python
-
-    import yfinance as yf
-
-    msft = yf.Ticker("MSFT")
-
-    msft.history(..., proxy="PROXY_SERVER")
-    msft.get_actions(proxy="PROXY_SERVER")
-    msft.get_dividends(proxy="PROXY_SERVER")
-    msft.get_splits(proxy="PROXY_SERVER")
-    msft.get_balance_sheet(proxy="PROXY_SERVER")
-    msft.get_cashflow(proxy="PROXY_SERVER")
-    msft.option_chain(..., proxy="PROXY_SERVER")
-    ...
-
-To initialize multiple ``Ticker`` objects, use
-
-.. code:: python
-
-    import yfinance as yf
-
-    tickers = yf.Tickers('msft aapl goog')
-    # ^ returns a named tuple of Ticker objects
-
-    # access each ticker using (example)
-    tickers.tickers.MSFT.info
-    tickers.tickers.AAPL.history(period="1mo")
-    tickers.tickers.GOOG.actions
-
-
-Fetching data for multiple tickers
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. code:: python
-
-    import yfinance as yf
-    data = yf.download("SPY AAPL", start="2017-01-01", end="2017-04-30")
-
-
-I've also added some options to make life easier :)
-
-.. code:: python
-
-    data = yf.download(  # or pdr.get_data_yahoo(...
-            # tickers list or string as well
-            tickers = "SPY AAPL MSFT",
-
-            # use "period" instead of start/end
-            # valid periods: 1d,5d,1mo,3mo,6mo,1y,2y,5y,10y,ytd,max
-            # (optional, default is '1mo')
-            period = "ytd",
-
-            # fetch data by interval (including intraday if period < 60 days)
-            # valid intervals: 1m,2m,5m,15m,30m,60m,90m,1h,1d,5d,1wk,1mo,3mo
-            # (optional, default is '1d')
-            interval = "1m",
-
-            # group by ticker (to access via data['SPY'])
-            # (optional, default is 'column')
-            group_by = 'ticker',
-
-            # adjust all OHLC automatically
-            # (optional, default is False)
-            auto_adjust = True,
-
-            # download pre/post regular market hours data
-            # (optional, default is False)
-            prepost = True,
-
-            # use threads for mass downloading? (True/False/Integer)
-            # (optional, default is True)
-            threads = True,
-
-            # proxy URL scheme use use when downloading?
-            # (optional, default is None)
-            proxy = None
-        )
-
-
-``pandas_datareader`` override
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-If your code uses ``pandas_datareader`` and you want to download data faster,
-you can "hijack" ``pandas_datareader.data.get_data_yahoo()`` method to use
-**yfinance** while making sure the returned data is in the same format as
-**pandas_datareader**'s ``get_data_yahoo()``.
-
-.. code:: python
-
-    from pandas_datareader import data as pdr
-
-    import yfinance as yf
-    yf.pdr_override() # <== that's all it takes :-)
-
-    # download dataframe
-    data = pdr.get_data_yahoo("SPY", start="2017-01-01", end="2017-04-30")
-
-
-Installation
-------------
-
-Install ``yfinance`` using ``pip``:
-
-.. code:: bash
-
-    $ pip install yfinance --upgrade --no-cache-dir
-
-
-Install ``yfinance`` using ``conda``:
-
-.. code:: bash
-
-    $ conda install -c ranaroussi yfinance
-
-
-Requirements
-------------
-
-* `Python <https://www.python.org>`_ >= 2.7, 3.4+
-* `Pandas <https://github.com/pydata/pandas>`_ (tested to work with >=0.23.1)
-* `Numpy <http://www.numpy.org>`_ >= 1.11.1
-* `requests <http://docs.python-requests.org/en/master/>`_ >= 2.14.2
-* `lxml <https://pypi.org/project/lxml/>`_ >= 4.5.1
-
-Optional (if you want to use ``pandas_datareader``)
----------------------------------------------------
-
-* `pandas_datareader <https://github.com/pydata/pandas-datareader>`_ >= 0.4.0
-
-Legal Stuff
-------------
-
-**yfinance** is distributed under the **Apache Software License**. See the `LICENSE.txt <./LICENSE.txt>`_ file in the release for details.
-
-
-P.S.
-------------
-
-Please drop me an note with any feedback you have.
-
-**Ran Aroussi**

meta.yaml

@@ -1,5 +1,5 @@
 {% set name = "yfinance" %}
-{% set version = "0.1.55" %}
+{% set version = "0.1.96" %}
 
 package:
   name: "{{ name|lower }}"
@@ -7,7 +7,7 @@ package:
 
 source:
   url: "https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version }}.tar.gz"
-  sha256: "89e984306bee13ed9a1a5d543f4f01f85716120666e404596a4e932ac7448288"
+  sha256: "207da19e87792bf742d2744eee2fe18169853c1b82cfe14a9a7cfb8d05f09137"
 
 build:
   noarch: python
@@ -16,20 +16,25 @@ build:
 
 requirements:
   host:
+    - pandas >=1.3.0
+    - numpy >=1.16.5
+    - requests >=2.26
     - multitasking >=0.0.7
-    - numpy >=1.15
-    - pandas >=0.24
+    - lxml >=4.9.1
+    - appdirs >= 1.4.4
+    - cryptography >= 3.3.2
     - pip
     - python
-    - requests >=2.20
-    - lxml>=4.5.1
+
   run:
+    - pandas >=1.3.0
+    - numpy >=1.16.5
+    - requests >=2.26
     - multitasking >=0.0.7
-    - numpy >=1.15
-    - pandas >=0.24
+    - lxml >=4.9.1
+    - appdirs >= 1.4.4
+    - cryptography >= 3.3.2
     - python
-    - requests >=2.20
-    - lxml>=4.5.1
 
 test:
   imports:
@@ -50,4 +55,3 @@ about:
 extra:
   recipe-maintainers:
     - ranaroussi
-    - bradmetz

mkdocs.yml

@@ -0,0 +1,19 @@
+# site_name: My Docs
+
+# # mkdocs.yml
+# theme:
+#   name: "material"
+
+# plugins:
+# - search
+# - mkdocstrings
+
+# nav:
+#   - Introduction: 'index.md'
+#   - Installation: 'installation.md'
+#   - Quick Start: 'quickstart.md'
+#   # - Ticker: 'Ticker.md'
+#   - TickerBase: 'TickerBase.md'
+#   # - Tickers: 'Tickers.md'
+#   - utils: 'utils.md'
+#   - multi: 'multi.md'

requirements.txt

@@ -1,5 +1,7 @@
-pandas>=0.24
-numpy>=1.15
-requests>=2.21
+pandas>=1.3.0
+numpy>=1.16.5
+requests>=2.26
 multitasking>=0.0.7
-lxml>=4.5.1
+lxml>=4.9.1
+appdirs>=1.4.4
+cryptography>=3.3.2

runtest.py

@@ -1,64 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: UTF-8 -*-
-#
-# Yahoo! Finance market data downloader (+fix for Pandas Datareader)
-# https://github.com/ranaroussi/yfinance
-
-"""
-Sanity check for most common library uses all working
-
-- Stock: Microsoft
-- ETF: Russell 2000 Growth
-- Mutual fund: Vanguard 500 Index fund
-- Index: S&P500
-- Currency BTC-USD
-"""
-
-from __future__ import print_function
-import yfinance as yf
-
-
-def test_yfinance():
-    for symbol in ['MSFT', 'IWO', 'VFINX', '^GSPC', 'BTC-USD']:
-        print(">>", symbol, end=' ... ')
-        ticker = yf.Ticker(symbol)
-
-        # always should have info and history for valid symbols
-        assert(ticker.info is not None and ticker.info != {})
-        assert(ticker.history(period="max").empty is False)
-
-        # following should always gracefully handled, no crashes
-        ticker.cashflow
-        ticker.balance_sheet
-        ticker.financials
-        ticker.sustainability
-        ticker.major_holders
-        ticker.institutional_holders
-        ticker.mutualfund_holders
-
-        print("OK")
-
-    # Ford has no institutional investors table or mutual fund holders
-    ticker = yf.Ticker('F')
-    print(">> F", end=" ... ")
-    assert(ticker.info is not None and ticker.info != {})
-    assert(ticker.major_holders is not None)
-    assert(ticker.institutional_holders is None)
-    print("OK")
-    # NKLA has no institutional investors table or mutual fund holders
-    ticker = yf.Ticker('NKLA')
-    print(">> NKLA", end=" ... ")
-    assert(ticker.info is not None and ticker.info != {})
-    assert(ticker.major_holders is not None)
-    assert(ticker.institutional_holders is None)
-    print("OK")
-    # NKLA has no institutional investors table or mutual fund holders
-    ticker = yf.Ticker('NESN.SW')
-    print(">> NESN.SW", end=" ... ")
-    assert(ticker.info is not None and ticker.info != {})
-    assert(ticker.major_holders is not None)
-    assert(ticker.institutional_holders is None)
-    print("OK")
-
-if __name__ == "__main__":
-    test_yfinance()

setup.py

@@ -1,27 +1,36 @@
 #!/usr/bin/env python
 # -*- coding: UTF-8 -*-
 #
-# Yahoo! Finance market data downloader (+fix for Pandas Datareader)
+# yfinance - market data downloader
 # https://github.com/ranaroussi/yfinance
 
-"""Yahoo! Finance market data downloader (+fix for Pandas Datareader)"""
+"""yfinance - market data downloader"""
 
 from setuptools import setup, find_packages
 # from codecs import open
 import io
 from os import path
 
+# --- get version ---
+version = "unknown"
+with open("yfinance/version.py") as f:
+    line = f.read().strip()
+    version = line.replace("version = ", "").replace('"', '')
+# --- /get version ---
+
+
 here = path.abspath(path.dirname(__file__))
 
 # Get the long description from the README file
-with io.open(path.join(here, 'README.rst'), encoding='utf-8') as f:
+with io.open(path.join(here, 'README.md'), encoding='utf-8') as f:
     long_description = f.read()
 
 setup(
     name='yfinance',
-    version="0.1.55",
-    description='Yahoo! Finance market data downloader',
+    version=version,
+    description='Download market data from Yahoo! Finance API',
     long_description=long_description,
+    long_description_content_type='text/markdown',
     url='https://github.com/ranaroussi/yfinance',
     author='Ran Aroussi',
     author_email='ran@aroussi.com',
@@ -44,18 +53,27 @@ setup(
         'Programming Language :: Python :: 2.7',
         'Programming Language :: Python :: 3.4',
         'Programming Language :: Python :: 3.5',
-        'Programming Language :: Python :: 3.6',
+        # 'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
+        'Programming Language :: Python :: 3.9',
     ],
     platforms=['any'],
     keywords='pandas, yahoo finance, pandas datareader',
     packages=find_packages(exclude=['contrib', 'docs', 'tests', 'examples']),
-    install_requires=['pandas>=0.24', 'numpy>=1.15',
-                      'requests>=2.20', 'multitasking>=0.0.7',
-                      'lxml>=4.5.1'],
+    install_requires=['pandas>=1.3.0', 'numpy>=1.16.5',
+                      'requests>=2.26', 'multitasking>=0.0.7',
+                      'lxml>=4.9.1', 'appdirs>=1.4.4',
+                      'cryptography>=3.3.2'],
     entry_points={
         'console_scripts': [
             'sample=sample:main',
         ],
     },
 )
+
+print("""
+NOTE: yfinance is not affiliated, endorsed, or vetted by Yahoo, Inc.
+
+You should refer to Yahoo!'s terms of use for details on your rights
+to use the actual data downloaded.""")

test_yfinance.py

@@ -0,0 +1,134 @@
+#!/usr/bin/env python
+# -*- coding: UTF-8 -*-
+#
+# yfinance - market data downloader
+# https://github.com/ranaroussi/yfinance
+
+"""
+Sanity check for most common library uses all working
+- Stock: Microsoft
+- ETF: Russell 2000 Growth
+- Mutual fund: Vanguard 500 Index fund
+- Index: S&P500
+- Currency BTC-USD
+"""
+
+import yfinance as yf
+import unittest
+import datetime
+
+session = None
+import requests_cache ; session = requests_cache.CachedSession("yfinance.cache", expire_after=24*60*60)
+
+# Good symbols = all attributes should work
+good_symbols = ['MSFT', 'IWO', 'VFINX', '^GSPC', 'BTC-USD']
+good_tickers = [yf.Ticker(symbol, session=session) for symbol in good_symbols]
+# Dodgy symbols = Yahoo data incomplete, so exclude from some tests
+dodgy_symbols = ["G7W.DU"]
+dodgy_tickers = [yf.Ticker(symbol, session=session) for symbol in dodgy_symbols]
+symbols = good_symbols + dodgy_symbols
+tickers = good_tickers + dodgy_tickers
+# Delisted = no data expected but yfinance shouldn't raise exception
+delisted_symbols = ["BRK.B", "SDLP"]
+delisted_tickers = [yf.Ticker(symbol, session=session) for symbol in delisted_symbols]
+
+
+class TestTicker(unittest.TestCase):
+    def setUp(self):
+        d_today = datetime.date.today()
+        d_today -= datetime.timedelta(days=30)
+        self.start_d = datetime.date(d_today.year, d_today.month, 1)
+
+    def test_info_history(self):
+        # always should have info and history for valid symbols
+        for ticker in tickers:
+            assert(ticker.info is not None and ticker.info != {})
+            history = ticker.history(period="1mo")
+            assert(history.empty is False and history is not None)
+        histories = yf.download(symbols, period="1mo", session=session)
+        assert(histories.empty is False and histories is not None)
+
+        for ticker in tickers:
+            assert(ticker.info is not None and ticker.info != {})
+            history = ticker.history(start=self.start_d)
+            assert(history.empty is False and history is not None)
+        histories = yf.download(symbols, start=self.start_d, session=session)
+        assert(histories.empty is False and histories is not None)
+
+    def test_info_history_nofail(self):
+        # should not throw Exception for delisted tickers, just print a message
+        for ticker in delisted_tickers:
+            history = ticker.history(period="1mo")
+        histories = yf.download(delisted_symbols, period="1mo", session=session)
+        histories = yf.download(delisted_symbols[0], period="1mo", session=session)
+        histories = yf.download(delisted_symbols[1], period="1mo")#, session=session)
+        for ticker in delisted_tickers:
+            history = ticker.history(start=self.start_d)
+        histories = yf.download(delisted_symbols, start=self.start_d, session=session)
+        histories = yf.download(delisted_symbols[0], start=self.start_d, session=session)
+        histories = yf.download(delisted_symbols[1], start=self.start_d, session=session)
+
+    def test_attributes(self):
+        for ticker in tickers:
+            ticker.isin
+            ticker.major_holders
+            ticker.institutional_holders
+            ticker.mutualfund_holders
+            ticker.dividends
+            ticker.splits
+            ticker.actions
+            ticker.info
+            ticker.info["trailingPegRatio"]
+            ticker.calendar
+            ticker.recommendations
+            ticker.earnings
+            ticker.quarterly_earnings
+            ticker.financials
+            ticker.quarterly_financials
+            ticker.balance_sheet
+            ticker.quarterly_balance_sheet
+            ticker.cashflow
+            ticker.quarterly_cashflow
+            ticker.sustainability
+            ticker.options
+            ticker.news
+            ticker.shares
+            ticker.earnings_history
+            ticker.earnings_dates
+
+    def test_attributes_nofail(self):
+        # should not throw Exception for delisted tickers, just print a message
+        for ticker in delisted_tickers:
+            ticker.isin
+            ticker.major_holders
+            ticker.institutional_holders
+            ticker.mutualfund_holders
+            ticker.dividends
+            ticker.splits
+            ticker.actions
+            ticker.info
+            ticker.calendar
+            ticker.recommendations
+            ticker.earnings
+            ticker.quarterly_earnings
+            ticker.financials
+            ticker.quarterly_financials
+            ticker.balance_sheet
+            ticker.quarterly_balance_sheet
+            ticker.cashflow
+            ticker.quarterly_cashflow
+            ticker.sustainability
+            ticker.options
+            ticker.news
+            ticker.shares
+            ticker.earnings_history
+            ticker.earnings_dates
+
+    def test_holders(self):
+        for ticker in good_tickers:
+            assert(ticker.major_holders is not None)
+            assert(ticker.institutional_holders is not None)
+
+
+if __name__ == '__main__':
+    unittest.main()

yfinance/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Yahoo! Finance market data downloader (+fix for Pandas Datareader)
+# yfinance - market data downloader
 # https://github.com/ranaroussi/yfinance
 #
 # Copyright 2017-2019 Ran Aroussi
@@ -19,12 +19,14 @@
 # limitations under the License.
 #
 
-__version__ = "0.1.54"
-__author__ = "Ran Aroussi"
-
+from . import version
 from .ticker import Ticker
 from .tickers import Tickers
 from .multi import download
+from .utils import set_tz_cache_location
+
+__version__ = version.version
+__author__ = "Ran Aroussi"
 
 
 def pdr_override():
@@ -41,4 +43,4 @@ def pdr_override():
         pass
 
 
-__all__ = ['download', 'Ticker', 'Tickers', 'pdr_override']
+__all__ = ['download', 'Ticker', 'Tickers', 'pdr_override', 'set_tz_cache_location']

yfinance/multi.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Yahoo! Finance market data downloader (+fix for Pandas Datareader)
+# yfinance - market data downloader
 # https://github.com/ranaroussi/yfinance
 #
 # Copyright 2017-2019 Ran Aroussi
@@ -29,10 +29,10 @@ from . import Ticker, utils
 from . import shared
 
 
-def download(tickers, start=None, end=None, actions=False, threads=True,
-             group_by='column', auto_adjust=False, back_adjust=False,
-             progress=True, period="max", interval="1d", prepost=False,
-             proxy=None, rounding=False, **kwargs):
+def download(tickers, start=None, end=None, actions=False, threads=True, ignore_tz=True, 
+             group_by='column', auto_adjust=False, back_adjust=False, keepna=False,
+             progress=True, period="max", show_errors=True, interval="1d", prepost=False,
+             proxy=None, rounding=False, timeout=None, **kwargs):
     """Download yahoo tickers
     :Parameters:
         tickers : str, list
@@ -56,20 +56,43 @@ def download(tickers, start=None, end=None, actions=False, threads=True,
             Default is False
         auto_adjust: bool
             Adjust all OHLC automatically? Default is False
+        keepna: bool
+            Keep NaN rows returned by Yahoo?
+            Default is False
         actions: bool
             Download dividend + stock splits data. Default is False
         threads: bool / int
             How many threads to use for mass downloading. Default is True
+        ignore_tz: bool
+            When combining from different timezones, ignore that part of datetime.
+            Default is True
         proxy: str
             Optional. Proxy server URL scheme. Default is None
         rounding: bool
             Optional. Round values to 2 decimal places?
+        show_errors: bool
+            Optional. Doesn't print errors if False
+        timeout: None or float
+            If not None stops waiting for a response after given number of
+            seconds. (Can also be a fraction of a second e.g. 0.01)
     """
 
     # create ticker list
     tickers = tickers if isinstance(
         tickers, (list, set, tuple)) else tickers.replace(',', ' ').split()
 
+    # accept isin as ticker
+    shared._ISINS = {}
+    _tickers_ = []
+    for ticker in tickers:
+        if utils.is_isin(ticker):
+            isin = ticker
+            ticker = utils.get_ticker_by_isin(ticker, proxy)
+            shared._ISINS[ticker] = isin
+        _tickers_.append(ticker)
+
+    tickers = _tickers_
+
     tickers = list(set([ticker.upper() for ticker in tickers]))
 
     if progress:
@@ -88,9 +111,9 @@ def download(tickers, start=None, end=None, actions=False, threads=True,
             _download_one_threaded(ticker, period=period, interval=interval,
                                    start=start, end=end, prepost=prepost,
                                    actions=actions, auto_adjust=auto_adjust,
-                                   back_adjust=back_adjust,
+                                   back_adjust=back_adjust, keepna=keepna,
                                    progress=(progress and i > 0), proxy=proxy,
-                                   rounding=rounding)
+                                   rounding=rounding, timeout=timeout)
         while len(shared._DFS) < len(tickers):
             _time.sleep(0.01)
 
@@ -100,7 +123,8 @@ def download(tickers, start=None, end=None, actions=False, threads=True,
             data = _download_one(ticker, period=period, interval=interval,
                                  start=start, end=end, prepost=prepost,
                                  actions=actions, auto_adjust=auto_adjust,
-                                 back_adjust=back_adjust, rounding=rounding)
+                                 back_adjust=back_adjust, keepna=keepna, proxy=proxy,
+                                 rounding=rounding, timeout=timeout)
             shared._DFS[ticker.upper()] = data
             if progress:
                 shared._PROGRESS_BAR.animate()
@@ -108,24 +132,33 @@ def download(tickers, start=None, end=None, actions=False, threads=True,
     if progress:
         shared._PROGRESS_BAR.completed()
 
-    if shared._ERRORS:
+    if shared._ERRORS and show_errors:
         print('\n%.f Failed download%s:' % (
             len(shared._ERRORS), 's' if len(shared._ERRORS) > 1 else ''))
         # print(shared._ERRORS)
         print("\n".join(['- %s: %s' %
                          v for v in list(shared._ERRORS.items())]))
 
+    if ignore_tz:
+        for tkr in shared._DFS.keys():
+            if (shared._DFS[tkr] is not None) and (shared._DFS[tkr].shape[0]>0):
+                shared._DFS[tkr].index = shared._DFS[tkr].index.tz_localize(None)
+
     if len(tickers) == 1:
-        return shared._DFS[tickers[0]]
+        ticker = tickers[0]
+        return shared._DFS[shared._ISINS.get(ticker, ticker)]
 
     try:
-        data = _pd.concat(shared._DFS.values(), axis=1,
+        data = _pd.concat(shared._DFS.values(), axis=1, sort=True,
                           keys=shared._DFS.keys())
     except Exception:
         _realign_dfs()
-        data = _pd.concat(shared._DFS.values(), axis=1,
+        data = _pd.concat(shared._DFS.values(), axis=1, sort=True,
                           keys=shared._DFS.keys())
 
+    # switch names back to isins if applicable
+    data.rename(columns=shared._ISINS, inplace=True)
+
     if group_by == 'column':
         data.columns = data.columns.swaplevel(0, 1)
         data.sort_index(level=0, axis=1, inplace=True)
@@ -161,10 +194,11 @@ def _download_one_threaded(ticker, start=None, end=None,
                            auto_adjust=False, back_adjust=False,
                            actions=False, progress=True, period="max",
                            interval="1d", prepost=False, proxy=None,
-                           rounding=False):
+                           keepna=False, rounding=False, timeout=None):
 
     data = _download_one(ticker, start, end, auto_adjust, back_adjust,
-                         actions, period, interval, prepost, proxy, rounding)
+                         actions, period, interval, prepost, proxy, rounding,
+                         keepna, timeout, many=True)
     shared._DFS[ticker.upper()] = data
     if progress:
         shared._PROGRESS_BAR.animate()
@@ -173,10 +207,12 @@ def _download_one_threaded(ticker, start=None, end=None,
 def _download_one(ticker, start=None, end=None,
                   auto_adjust=False, back_adjust=False,
                   actions=False, period="max", interval="1d",
-                  prepost=False, proxy=None, rounding=False):
+                  prepost=False, proxy=None, rounding=False,
+                  keepna=False, timeout=None, many=False):
 
     return Ticker(ticker).history(period=period, interval=interval,
                                   start=start, end=end, prepost=prepost,
                                   actions=actions, auto_adjust=auto_adjust,
                                   back_adjust=back_adjust, proxy=proxy,
-                                  rounding=rounding, many=True)
+                                  rounding=rounding, keepna=keepna, timeout=timeout, 
+                                  many=many)

yfinance/shared.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Yahoo! Finance market data downloader (+fix for Pandas Datareader)
+# yfinance - market data downloader
 # https://github.com/ranaroussi/yfinance
 #
 # Copyright 2017-2019 Ran Aroussi
@@ -22,3 +22,4 @@
 _DFS = {}
 _PROGRESS_BAR = None
 _ERRORS = {}
+_ISINS = {}

yfinance/ticker.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Yahoo! Finance market data downloader (+fix for Pandas Datareader)
+# yfinance - market data downloader
 # https://github.com/ranaroussi/yfinance
 #
 # Copyright 2017-2019 Ran Aroussi
@@ -31,6 +31,7 @@ import pandas as _pd
 # import re as _re
 from collections import namedtuple as _namedtuple
 
+from . import utils
 from .base import TickerBase
 
 
@@ -53,13 +54,17 @@ class Ticker(TickerBase):
                 proxy = proxy["https"]
             proxy = {"https": proxy}
 
-        r = _requests.get(url=url, proxies=proxy).json()
-        if r['optionChain']['result']:
+        r = _requests.get(
+            url=url,
+            proxies=proxy,
+            headers=utils.user_agent_headers
+        ).json()
+        if len(r.get('optionChain', {}).get('result', [])) > 0:
             for exp in r['optionChain']['result'][0]['expirationDates']:
                 self._expirations[_datetime.datetime.utcfromtimestamp(
                     exp).strftime('%Y-%m-%d')] = exp
-            return r['optionChain']['result'][0]['options'][0]
-        return {}
+            opt = r['optionChain']['result'][0].get('options', [])
+            return opt[0] if len(opt) > 0 else []
 
     def _options2df(self, opt, tz=None):
         data = _pd.DataFrame(opt).reindex(columns=[
@@ -79,9 +84,9 @@ class Ticker(TickerBase):
             'currency'])
 
         data['lastTradeDate'] = _pd.to_datetime(
-            data['lastTradeDate'], unit='s')
+            data['lastTradeDate'], unit='s', utc=True)
         if tz is not None:
-            data['lastTradeDate'] = data['lastTradeDate'].tz_localize(tz)
+            data['lastTradeDate'] = data['lastTradeDate'].dt.tz_convert(tz)
         return data
 
     def option_chain(self, date=None, proxy=None, tz=None):
@@ -133,6 +138,10 @@ class Ticker(TickerBase):
     def actions(self):
         return self.get_actions()
 
+    @property
+    def shares(self):
+        return self.get_shares()
+
     @property
     def info(self):
         return self.get_info()
@@ -194,3 +203,19 @@ class Ticker(TickerBase):
         if not self._expirations:
             self._download_options()
         return tuple(self._expirations.keys())
+
+    @property
+    def news(self):
+        return self.get_news()
+
+    @property
+    def analysis(self):
+        return self.get_analysis()
+
+    @property
+    def earnings_history(self):
+        return self.get_earnings_history()
+
+    @property
+    def earnings_dates(self):
+        return self.get_earnings_dates()

yfinance/tickers.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Yahoo! Finance market data downloader (+fix for Pandas Datareader)
+# yfinance - market data downloader
 # https://github.com/ranaroussi/yfinance
 #
 # Copyright 2017-2019 Ran Aroussi
@@ -22,19 +22,7 @@
 from __future__ import print_function
 
 from . import Ticker, multi
-from collections import namedtuple as _namedtuple
-
-
-def genTickers(tickers):
-    tickers = tickers if isinstance(
-        tickers, list) else tickers.replace(',', ' ').split()
-    tickers = [ticker.upper() for ticker in tickers]
-    ticker_objects = {}
-
-    for ticker in tickers:
-        ticker_objects[ticker] = Ticker(ticker)
-    return _namedtuple("Tickers", ticker_objects.keys()
-                       )(*ticker_objects.values())
+# from collections import namedtuple as _namedtuple
 
 
 class Tickers():
@@ -42,37 +30,38 @@ class Tickers():
     def __repr__(self):
         return 'yfinance.Tickers object <%s>' % ",".join(self.symbols)
 
-    def __init__(self, tickers):
+    def __init__(self, tickers, session=None):
         tickers = tickers if isinstance(
             tickers, list) else tickers.replace(',', ' ').split()
         self.symbols = [ticker.upper() for ticker in tickers]
         ticker_objects = {}
 
         for ticker in self.symbols:
-            ticker_objects[ticker] = Ticker(ticker)
+            ticker_objects[ticker] = Ticker(ticker, session=session)
 
-        self.tickers = _namedtuple(
-            "Tickers", ticker_objects.keys(), rename=True
-        )(*ticker_objects.values())
+        self.tickers = ticker_objects
+        # self.tickers = _namedtuple(
+        #     "Tickers", ticker_objects.keys(), rename=True
+        # )(*ticker_objects.values())
 
     def history(self, period="1mo", interval="1d",
                 start=None, end=None, prepost=False,
                 actions=True, auto_adjust=True, proxy=None,
                 threads=True, group_by='column', progress=True,
-                **kwargs):
+                timeout=None, **kwargs):
 
         return self.download(
-                period, interval,
-                start, end, prepost,
-                actions, auto_adjust, proxy,
-                threads, group_by, progress,
-                **kwargs)
+            period, interval,
+            start, end, prepost,
+            actions, auto_adjust, proxy,
+            threads, group_by, progress,
+            timeout, **kwargs)
 
     def download(self, period="1mo", interval="1d",
                  start=None, end=None, prepost=False,
                  actions=True, auto_adjust=True, proxy=None,
                  threads=True, group_by='column', progress=True,
-                 **kwargs):
+                 timeout=None, **kwargs):
 
         data = multi.download(self.symbols,
                               start=start, end=end,
@@ -85,13 +74,23 @@ class Tickers():
                               group_by='ticker',
                               threads=threads,
                               progress=progress,
+                              timeout=timeout,
                               **kwargs)
 
         for symbol in self.symbols:
-            getattr(self.tickers, symbol)._history = data[symbol]
+            self.tickers.get(symbol, {})._history = data[symbol]
 
         if group_by == 'column':
             data.columns = data.columns.swaplevel(0, 1)
             data.sort_index(level=0, axis=1, inplace=True)
 
         return data
+
+    def news(self):
+        collection = {}
+        for ticker in self.symbols:
+            collection[ticker] = []
+            items = Ticker(ticker).news
+            for item in items:
+                collection[ticker].append(item)
+        return collection

yfinance/utils.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Yahoo! Finance market data downloader (+fix for Pandas Datareader)
+# yfinance - market data downloader
 # https://github.com/ranaroussi/yfinance
 #
 # Copyright 2017-2019 Ran Aroussi
@@ -21,12 +21,30 @@
 
 from __future__ import print_function
 
+import datetime as _datetime
+import pytz as _tz
 import requests as _requests
 import re as _re
 import pandas as _pd
 import numpy as _np
 import sys as _sys
-import re as _re
+import os as _os
+import appdirs as _ad
+
+from base64 import b64decode
+import hashlib
+usePycryptodome = False  # slightly faster
+# usePycryptodome = True
+if usePycryptodome:
+    # NOTE: if decide to use 'pycryptodome', set min version to 3.6.6
+    from Crypto.Cipher import AES
+    from Crypto.Util.Padding import unpad
+else:
+    from cryptography.hazmat.primitives import padding
+    from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
+
+from threading import Lock
+mutex = Lock()
 
 try:
     import ujson as _json
@@ -34,6 +52,54 @@ except ImportError:
     import json as _json
 
 
+user_agent_headers = {
+    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}
+
+
+def is_isin(string):
+    return bool(_re.match("^([A-Z]{2})([A-Z0-9]{9})([0-9]{1})$", string))
+
+
+def get_all_by_isin(isin, proxy=None, session=None):
+    if not(is_isin(isin)):
+        raise ValueError("Invalid ISIN number")
+
+    from .base import _BASE_URL_
+    session = session or _requests
+    url = "{}/v1/finance/search?q={}".format(_BASE_URL_, isin)
+    data = session.get(url=url, proxies=proxy, headers=user_agent_headers)
+    try:
+        data = data.json()
+        ticker = data.get('quotes', [{}])[0]
+        return {
+            'ticker': {
+                'symbol': ticker['symbol'],
+                'shortname': ticker['shortname'],
+                'longname': ticker['longname'],
+                'type': ticker['quoteType'],
+                'exchange': ticker['exchDisp'],
+            },
+            'news': data.get('news', [])
+        }
+    except Exception:
+        return {}
+
+
+def get_ticker_by_isin(isin, proxy=None, session=None):
+    data = get_all_by_isin(isin, proxy, session)
+    return data.get('ticker', {}).get('symbol', '')
+
+
+def get_info_by_isin(isin, proxy=None, session=None):
+    data = get_all_by_isin(isin, proxy, session)
+    return data.get('ticker', {})
+
+
+def get_news_by_isin(isin, proxy=None, session=None):
+    data = get_all_by_isin(isin, proxy, session)
+    return data.get('news', {})
+
+
 def empty_df(index=[]):
     empty = _pd.DataFrame(index=index, data={
         'Open': _np.nan, 'High': _np.nan, 'Low': _np.nan,
@@ -42,18 +108,127 @@ def empty_df(index=[]):
     return empty
 
 
-def get_json(url, proxy=None):
-    html = _requests.get(url=url, proxies=proxy).text
+def empty_earnings_dates_df():
+    empty = _pd.DataFrame(
+        columns=["Symbol", "Company", "Earnings Date",
+                 "EPS Estimate", "Reported EPS", "Surprise(%)"])
+    return empty
 
-    if "QuoteSummaryStore" not in html:
-        html = _requests.get(url=url, proxies=proxy).text
-        if "QuoteSummaryStore" not in html:
-            return {}
+
+def get_html(url, proxy=None, session=None):
+    session = session or _requests
+    html = session.get(url=url, proxies=proxy, headers=user_agent_headers).text
+    return html
+
+
+
+def decrypt_cryptojs_stores(data):
+    """
+    Yahoo has started encrypting data stores, this method decrypts it.
+    :param data: Python dict of the json data
+    :return: The decrypted string data in data['context']['dispatcher']['stores']
+    """
+
+    _cs = data["_cs"]
+    # Assumes _cr has format like: '{"words":[-449732894,601032952,157396918,2056341829],"sigBytes":16}';
+    _cr = _json.loads(data["_cr"])
+    _cr = b"".join(int.to_bytes(i, length=4, byteorder="big", signed=True) for i in _cr["words"])
+
+    password = hashlib.pbkdf2_hmac("sha1", _cs.encode("utf8"), _cr, 1, dklen=32).hex()
+
+    encrypted_stores = data['context']['dispatcher']['stores']
+    encrypted_stores = b64decode(encrypted_stores)
+    assert encrypted_stores[0:8] == b"Salted__"
+    salt = encrypted_stores[8:16]
+    encrypted_stores = encrypted_stores[16:]
+
+    key, iv = _EVPKDF(password, salt, keySize=32, ivSize=16, iterations=1, hashAlgorithm="md5")
+
+    if usePycryptodome:
+        cipher = AES.new(key, AES.MODE_CBC, iv=iv)
+        plaintext = cipher.decrypt(encrypted_stores)
+        plaintext = unpad(plaintext, 16, style="pkcs7")
+    else:
+        cipher = Cipher(algorithms.AES(key), modes.CBC(iv))
+        decryptor = cipher.decryptor()
+        plaintext = decryptor.update(encrypted_stores) + decryptor.finalize()
+        unpadder = padding.PKCS7(128).unpadder()
+        plaintext = unpadder.update(plaintext) + unpadder.finalize()
+        plaintext = plaintext.decode("utf-8")
+
+    return plaintext
+
+def _EVPKDF(password, salt, keySize=32, ivSize=16, iterations=1, hashAlgorithm="md5") -> tuple:
+    """OpenSSL EVP Key Derivation Function
+    Args:
+        password (Union[str, bytes, bytearray]): Password to generate key from.
+        salt (Union[bytes, bytearray]): Salt to use.
+        keySize (int, optional): Output key length in bytes. Defaults to 32.
+        ivSize (int, optional): Output Initialization Vector (IV) length in bytes. Defaults to 16.
+        iterations (int, optional): Number of iterations to perform. Defaults to 1.
+        hashAlgorithm (str, optional): Hash algorithm to use for the KDF. Defaults to 'md5'.
+    Returns:
+        key, iv: Derived key and Initialization Vector (IV) bytes.
+
+    Taken from: https://gist.github.com/rafiibrahim8/0cd0f8c46896cafef6486cb1a50a16d3
+    OpenSSL original code: https://github.com/openssl/openssl/blob/master/crypto/evp/evp_key.c#L78
+    """
+
+    assert iterations > 0, "Iterations can not be less than 1."
+
+    if isinstance(password, str):
+        password = password.encode("utf-8")
+
+    final_length = keySize + ivSize
+    key_iv = b""
+    block = None
+
+    while len(key_iv) < final_length:
+        hasher = hashlib.new(hashAlgorithm)
+        if block:
+            hasher.update(block)
+        hasher.update(password)
+        hasher.update(salt)
+        block = hasher.digest()
+        for _ in range(1, iterations):
+            block = hashlib.new(hashAlgorithm, block).digest()
+        key_iv += block
+
+    key, iv = key_iv[:keySize], key_iv[keySize:final_length]
+    return key, iv
+
+
+def get_json(url, proxy=None, session=None):
+    session = session or _requests
+    html = session.get(url=url, proxies=proxy, headers=user_agent_headers).text
+
+    if not "root.App.main =" in html:
+        return {}
 
     json_str = html.split('root.App.main =')[1].split(
         '(this)')[0].split(';\n}')[0].strip()
-    data = _json.loads(json_str)[
-        'context']['dispatcher']['stores']['QuoteSummaryStore']
+    data = _json.loads(json_str)
+
+    if "_cs" in data and "_cr" in data:
+        data_stores = _json.loads(decrypt_cryptojs_stores(data))
+    else:
+        if "context" in data and "dispatcher" in data["context"]:
+            # Keep old code, just in case
+            data_stores = data['context']['dispatcher']['stores']
+        else:
+            data_stores = data
+
+    if not 'QuoteSummaryStore' in data_stores:
+        # Problem in data. Either delisted, or Yahoo spam triggered
+        return {}
+
+    data = data_stores['QuoteSummaryStore']
+    # add data about Shares Outstanding for companies' tickers if they are available
+    try:
+        data['annualBasicAverageShares'] = \
+            data_stores['QuoteTimeSeriesStore']['timeSeries']['annualBasicAverageShares']
+    except Exception:
+        pass
 
     # return data
     new_data = _json.dumps(data).replace('{}', 'null')
@@ -67,6 +242,23 @@ def camel2title(o):
     return [_re.sub("([a-z])([A-Z])", r"\g<1> \g<2>", i).title() for i in o]
 
 
+def _parse_user_dt(dt, exchange_tz):
+    if isinstance(dt, int):
+        ## Should already be epoch, test with conversion:
+        _datetime.datetime.fromtimestamp(dt)
+    else:
+        # Convert str/date -> datetime, set tzinfo=exchange, get timestamp:
+        if isinstance(dt, str):
+            dt = _datetime.datetime.strptime(str(dt), '%Y-%m-%d')
+        if isinstance(dt, _datetime.date) and not isinstance(dt, _datetime.datetime):
+            dt = _datetime.datetime.combine(dt, _datetime.time(0))
+        if isinstance(dt, _datetime.datetime) and dt.tzinfo is None:
+            # Assume user is referring to exchange's timezone
+            dt = _tz.timezone(exchange_tz).localize(dt)
+        dt = int(dt.timestamp())
+    return dt
+
+
 def auto_adjust(data):
     df = data.copy()
     ratio = df["Close"] / df["Adj Close"]
@@ -108,7 +300,7 @@ def back_adjust(data):
     return df[["Open", "High", "Low", "Close", "Volume"]]
 
 
-def parse_quotes(data, tz=None):
+def parse_quotes(data):
     timestamps = data["timestamp"]
     ohlc = data["indicators"]["quote"][0]
     volumes = ohlc["volume"]
@@ -131,15 +323,14 @@ def parse_quotes(data, tz=None):
     quotes.index = _pd.to_datetime(timestamps, unit="s")
     quotes.sort_index(inplace=True)
 
-    if tz is not None:
-        quotes.index = quotes.index.tz_localize(tz)
-
     return quotes
 
 
-def parse_actions(data, tz=None):
-    dividends = _pd.DataFrame(columns=["Dividends"])
-    splits = _pd.DataFrame(columns=["Stock Splits"])
+def parse_actions(data):
+    dividends = _pd.DataFrame(
+        columns=["Dividends"], index=_pd.DatetimeIndex([]))
+    splits = _pd.DataFrame(
+        columns=["Stock Splits"], index=_pd.DatetimeIndex([]))
 
     if "events" in data:
         if "dividends" in data["events"]:
@@ -148,8 +339,6 @@ def parse_actions(data, tz=None):
             dividends.set_index("date", inplace=True)
             dividends.index = _pd.to_datetime(dividends.index, unit="s")
             dividends.sort_index(inplace=True)
-            if tz is not None:
-                dividends.index = dividends.index.tz_localize(tz)
 
             dividends.columns = ["Dividends"]
 
@@ -159,8 +348,6 @@ def parse_actions(data, tz=None):
             splits.set_index("date", inplace=True)
             splits.index = _pd.to_datetime(splits.index, unit="s")
             splits.sort_index(inplace=True)
-            if tz is not None:
-                splits.index = splits.index.tz_localize(tz)
             splits["Stock Splits"] = splits["numerator"] / \
                 splits["denominator"]
             splits = splits["Stock Splits"]
@@ -168,6 +355,19 @@ def parse_actions(data, tz=None):
     return dividends, splits
 
 
+def fix_Yahoo_dst_issue(df, interval):
+    if interval in ["1d","1w","1wk"]:
+        # These intervals should start at time 00:00. But for some combinations of date and timezone, 
+        # Yahoo has time off by few hours (e.g. Brazil 23:00 around Jan-2022). Suspect DST problem.
+        # The clue is (a) minutes=0 and (b) hour near 0. 
+        # Obviously Yahoo meant 00:00, so ensure this doesn't affect date conversion:
+        f_pre_midnight = (df.index.minute == 0) & (df.index.hour.isin([22,23]))
+        dst_error_hours = _np.array([0]*df.shape[0])
+        dst_error_hours[f_pre_midnight] = 24-df.index[f_pre_midnight].hour
+        df.index += _pd.TimedeltaIndex(dst_error_hours, 'h')
+    return df
+
+
 class ProgressBar:
     def __init__(self, iterations, text='completed'):
         self.text = text
@@ -216,3 +416,55 @@ class ProgressBar:
 
     def __str__(self):
         return str(self.prog_bar)
+
+
+# Simple file cache of ticker->timezone:
+_cache_dp = None
+def get_cache_dirpath():
+    if _cache_dp is None:
+        dp = _os.path.join(_ad.user_cache_dir(), "py-yfinance")
+    else:
+        dp = _os.path.join(_cache_dp, "py-yfinance")
+    return dp
+def set_tz_cache_location(dp):
+    global _cache_dp
+    _cache_dp = dp
+
+def cache_lookup_tkr_tz(tkr):
+    fp = _os.path.join(get_cache_dirpath(), "tkr-tz.csv")
+    if not _os.path.isfile(fp):
+        return None
+
+    mutex.acquire()
+    df = _pd.read_csv(fp, index_col="Ticker", on_bad_lines="skip")
+    mutex.release()
+    if tkr in df.index:
+        return df.loc[tkr,"Tz"]
+    else:
+        return None
+def cache_store_tkr_tz(tkr,tz):
+
+    dp = get_cache_dirpath()
+    fp = _os.path.join(dp, "tkr-tz.csv")
+    mutex.acquire()
+    if not _os.path.isdir(dp):
+        _os.makedirs(dp)
+    if (not _os.path.isfile(fp)) and (tz is not None):
+        df = _pd.DataFrame({"Tz":[tz]}, index=[tkr])
+        df.index.name = "Ticker"
+        df.to_csv(fp)
+
+    else:
+        df = _pd.read_csv(fp, index_col="Ticker", on_bad_lines="skip")
+        if tz is None:
+            # Delete if in cache:
+            if tkr in df.index:
+                df.drop(tkr).to_csv(fp)
+        else:
+            if tkr in df.index:
+                raise Exception("Tkr {} tz already in cache".format(tkr))
+            df.loc[tkr,"Tz"] = tz
+            df.to_csv(fp)
+    
+    mutex.release()
+

yfinance/version.py

@@ -0,0 +1 @@
+version = "0.1.96"