Version 0.2.40

sync dev -> main

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -6,23 +6,13 @@ body:
   - type: markdown
     attributes:
       value: |
-        # IMPORTANT - Read and follow these instructions carefully. Help us help you.
-
-        ### Does issue already exist?
-
-        Use the search tool. Don't annoy everyone by duplicating existing Issues.
+        # !!! IMPORTANT !!! FOLLOW THESE INSTRUCTIONS CAREFULLY !!!
 
         ### Are you up-to-date?
 
-        Upgrade to the latest version and confirm the issue/bug is still there.
+        Upgrade to the latest version: `$ pip install yfinance --upgrade --no-cache-dir`
 
-        `$ pip install yfinance --upgrade --no-cache-dir`
-
-        Confirm by running:
-
-        `import yfinance as yf ; print(yf.__version__)`
-
-        and comparing against [PIP](https://pypi.org/project/yfinance/#history).
+        Confirm latest version by running: `import yfinance as yf ; print(yf.__version__)` and comparing against [PyPI](https://pypi.org/project/yfinance/#history).
 
         ### Does Yahoo actually have the data?
 
@@ -34,6 +24,10 @@ body:
 
         Yahoo Finance free service has rate-limiting https://github.com/ranaroussi/yfinance/discussions/1513. Once limit hit, Yahoo can delay, block, or return bad data -> not a `yfinance` bug.
         
+        ### Does issue already exist?
+
+        Use the search tool. Don't duplicate existing issues.
+        
   - type: markdown
     attributes:
       value: |
@@ -61,7 +55,7 @@ body:
     id: debug-log
     attributes:
       label: "Debug log"
-      description: "Run code with debug logging enabled and post the full output. Instructions: https://github.com/ranaroussi/yfinance/tree/main#logging"
+      description: "Run code with debug logging enabled and post the full output. IMPORTANT INSTRUCTIONS: https://github.com/ranaroussi/yfinance/tree/main#logging"
     validations:
       required: true
 

.github/workflows/ci.yml

@@ -8,11 +8,11 @@ jobs:
   deploy:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
-      - uses: actions/setup-python@v2
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v4
         with:
           python-version: 3.x
       - run: pip install -r requirements.txt
       - run: pip install mkdocstrings==0.14.0
       - run: pip install mkdocs-material
-      - run: mkdocs gh-deploy --force
+      - run: mkdocs gh-deploy --force

.github/workflows/ruff.yml

@@ -0,0 +1,13 @@
+name: Ruff
+on:
+  pull_request:
+    branches:
+      - master
+      - main
+      - dev
+jobs:
+  ruff:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: chartboost/ruff-action@v1

CHANGELOG.rst

@@ -1,10 +1,84 @@
 Change Log
 ===========
 
-0.2.32b1
+0.2.40
 ------
-Beta release - add cookie & crumb to request.
-Why beta? Thread-safety needs stress-testing.
+Fix typo in 0.2.39 c7af213
+
+0.2.39
+------
+Fixes:
+- Fix switching session from/to requests_cache #1930
+Price repair:
+- Fix potential for price repair to discard price=0 rows #1874
+- Don't price-repair FX volume=0, is normal #1920
+- Improve 'sudden change' repair for splits & currency  #1931
+Information:
+- Fix help(yf.download) not showing the information about the function #1913 @vittoboa
+- Add more specific error throwing based on PR 1918 #1928 @elibroftw @marcofognog
+Maintenance:
+- Replace dead 'appdirs' package with 'platformdirs' #1896
+- Deprecate 'pandas_datareader', remove a deprecated argument #1897
+- Fix: datetime.datetime.utcnow() is deprecated ... #1922
+
+0.2.38
+------
+Fix holders & insiders #1908
+
+0.2.37
+------
+Small fixes:
+- Fix Pandas warnings #1838 #1844
+- Fix price repair bug, typos, refactor #1866 #1865 #1849
+- Stop disabling logging #1841
+
+0.2.36
+------
+Small fixes:
+- Update README.md for better copy-ability  #1823 
+- Name download() column levels  #1795 
+- Fix history(keepna=False) when repair=True  #1824 
+- Replace empty list with empty pd.Series  #1724 
+- Handle peewee with old sqlite  #1827 
+- Fix JSON error handling  #1830 #1833
+
+0.2.35
+------
+Internal fixes for 0.2.34
+
+0.2.34
+------
+Features:
+- Add Recommendations Trend Summary #1754
+- Add Recommendation upgrades & downgrades #1773
+- Add Insider Roster & Transactions #1772
+- Moved download() progress bar to STDERR #1776
+- PIP optional dependencies #1771
+- Set sensible min versions for optional 'nospam' reqs #1807
+Fixes
+- Fix download() DatetimeIndex on invalid symbols #1779
+- Fix invalid date entering cache DB #1796
+- Fix Ticker.calendar fetch #1790
+- Fixed adding complementary to info #1774
+- Ticker.earnings_dates: fix warning "Value 'NaN' has dtype incompatible with float64" #1810
+- Minor fixes for price repair and related tests #1768
+- Fix price repair div adjust #1798
+- Fix 'raise_errors' argument ignored in Ticker.history() #1806
+Maintenance
+- Fix regression: _get_ticker_tz() args were being swapped. Improve its unit test #1793
+- Refactor Ticker proxy #1711
+- Add Ruff linter checks #1756
+- Resolve Pandas FutureWarnings #1766
+
+0.2.33
+------
+Cookie fixes:
+- fix backup strategy #1759
+- fix Ticker(ISIN) #1760
+
+0.2.32
+------
+Add cookie & crumb to requests #1657
 
 0.2.31
 ------

CODE_OF_CONDUCT.md

@@ -0,0 +1,15 @@
+# Code of Conduct
+
+## Submitting a new issue
+
+* Search through existing Issues and Discussions, in case your issue already exists and a solution is being developed.
+* Ensure you read & follow the template form.
+* Consider you may be the best person to investigate and fix.
+
+## Contributing to an existing Issue
+
+* Read the entire thread.
+* Ensure your comment is contributing something new/useful. Remember you can simply react to other comments.
+* Be concise:
+  - use the formatting options
+  - if replying to a big comment, instead of quoting it, link to it

README.md

@@ -42,6 +42,34 @@ Yahoo! finance API is intended for personal use only.**
 
 ---
 
+- [Installation](#installation)
+- [Quick start](#quick-start)
+- [Advanced](#logging)
+- [Wiki](https://github.com/ranaroussi/yfinance/wiki)
+- [Contribute](#developers-want-to-contribute)
+
+---
+
+## Installation
+
+Install `yfinance` using `pip`:
+
+``` {.sourceCode .bash}
+$ pip install yfinance --upgrade --no-cache-dir
+```
+
+[With Conda](https://anaconda.org/ranaroussi/yfinance).
+
+To install with optional dependencies, replace `optional` with: `nospam` for [caching-requests](#smarter-scraping), `repair` for [price repair](https://github.com/ranaroussi/yfinance/wiki/Price-repair), or `nospam,repair` for both:
+
+``` {.sourceCode .bash}
+$ pip install "yfinance[optional]"
+```
+
+[Required dependencies](./requirements.txt) , [all dependencies](./setup.py#L62).
+
+---
+
 ## Quick Start
 
 ### The Ticker module
@@ -87,8 +115,16 @@ msft.quarterly_cashflow
 msft.major_holders
 msft.institutional_holders
 msft.mutualfund_holders
+msft.insider_transactions
+msft.insider_purchases
+msft.insider_roster_holders
 
-# Show future and historic earnings dates, returns at most next 4 quarters and last 8 quarters by default. 
+# show recommendations
+msft.recommendations
+msft.recommendations_summary
+msft.upgrades_downgrades
+
+# Show future and historic earnings dates, returns at most next 4 quarters and last 8 quarters by default.
 # Note: If more are needed use msft.get_earnings_dates(limit=XX) with increased limit argument.
 msft.earnings_dates
 
@@ -155,9 +191,10 @@ data = yf.download("SPY AAPL", period="1mo")
 
 ### Smarter scraping
 
-To use a custom `requests` session (for example to cache calls to the
-API or customize the `User-agent` header), pass a `session=` argument to
-the Ticker constructor.
+Install the `nospam` packages for smarter scraping using `pip` (see [Installation](#installation)). These packages help cache calls such that Yahoo is not spammed with requests.
+
+To use a custom `requests` session, pass a `session=` argument to
+the Ticker constructor. This allows for caching calls to the API as well as a custom way to modify requests via  the `User-agent` header.
 
 ```python
 import requests_cache
@@ -168,7 +205,7 @@ ticker = yf.Ticker('msft', session=session)
 ticker.actions
 ```
 
-Combine a `requests_cache` with rate-limiting to avoid triggering Yahoo's rate-limiter/blocker that can corrupt data.
+Combine `requests_cache` with rate-limiting to avoid triggering Yahoo's rate-limiter/blocker that can corrupt data.
 ```python
 from requests import Session
 from requests_cache import CacheMixin, SQLiteCache
@@ -199,29 +236,16 @@ yfinance?](https://stackoverflow.com/questions/63107801)
         -   How to download single or multiple tickers into a single
             dataframe with single level column names and a ticker column
 
-### `pandas_datareader` override
+### Persistent cache store
 
-If your code uses `pandas_datareader` and you want to download data
-faster, you can "hijack" `pandas_datareader.data.get_data_yahoo()`
-method to use **yfinance** while making sure the returned data is in the
-same format as **pandas\_datareader**'s `get_data_yahoo()`.
+To reduce Yahoo, yfinance store some data locally: timezones to localize dates, and cookie. Cache location is:
 
-```python
-from pandas_datareader import data as pdr
+- Windows = C:/Users/\<USER\>/AppData/Local/py-yfinance
+- Linux = /home/\<USER\>/.cache/py-yfinance
+- MacOS = /Users/\<USER\>/Library/Caches/py-yfinance
 
-import yfinance as yf
-yf.pdr_override() # <== that's all it takes :-)
-
-# download dataframe
-data = pdr.get_data_yahoo("SPY", start="2017-01-01", end="2017-04-30")
-```
-
-### Timezone cache store
-
-When fetching price data, all dates are localized to stock exchange timezone. 
-But timezone retrieval is relatively slow, so yfinance attemps to cache them 
-in your users cache folder. 
 You can direct cache to use a different location with `set_tz_cache_location()`:
+
 ```python
 import yfinance as yf
 yf.set_tz_cache_location("custom/cache/location")
@@ -230,41 +254,6 @@ yf.set_tz_cache_location("custom/cache/location")
 
 ---
 
-## Installation
-
-Install `yfinance` using `pip`:
-
-``` {.sourceCode .bash}
-$ pip install yfinance --upgrade --no-cache-dir
-```
-
-Test new features by installing betas, provide feedback in [corresponding Discussion](https://github.com/ranaroussi/yfinance/discussions):
-``` {.sourceCode .bash}
-$ pip install yfinance --upgrade --no-cache-dir --pre
-```
-
-To install `yfinance` using `conda`, see
-[this](https://anaconda.org/ranaroussi/yfinance).
-
-### Requirements
-
--   [Python](https://www.python.org) \>= 2.7, 3.4+
--   [Pandas](https://github.com/pydata/pandas) \>= 1.3.0
--   [Numpy](http://www.numpy.org) \>= 1.16.5
--   [requests](http://docs.python-requests.org/en/master) \>= 2.31
--   [lxml](https://pypi.org/project/lxml) \>= 4.9.1
--   [appdirs](https://pypi.org/project/appdirs) \>= 1.4.4
--   [pytz](https://pypi.org/project/pytz) \>=2022.5
--   [frozendict](https://pypi.org/project/frozendict) \>= 2.3.4
--   [beautifulsoup4](https://pypi.org/project/beautifulsoup4) \>= 4.11.1
--   [html5lib](https://pypi.org/project/html5lib) \>= 1.1
--   [peewee](https://pypi.org/project/peewee)  \>= 3.16.2
-
-#### Optional (if you want to use `pandas_datareader`)
-
--   [pandas\_datareader](https://github.com/pydata/pandas-datareader)
-    \>= 0.4.0
-
 ## Developers: want to contribute?
 
 `yfinance` relies on community to investigate bugs and contribute code. Developer guide: https://github.com/ranaroussi/yfinance/discussions/1084
@@ -283,7 +272,7 @@ intended for research and educational purposes. You should refer to Yahoo!'s ter
 ([here](https://policies.yahoo.com/us/en/yahoo/terms/product-atos/apiforydn/index.htm),
 [here](https://legal.yahoo.com/us/en/yahoo/terms/otos/index.html), and
 [here](https://policies.yahoo.com/us/en/yahoo/terms/index.htm)) for
-detailes on your rights to use the actual data downloaded.
+details on your rights to use the actual data downloaded.
 
 ---
 

meta.yaml

@@ -1,5 +1,5 @@
 {% set name = "yfinance" %}
-{% set version = "0.2.32b1" %}
+{% set version = "0.2.40" %}
 
 package:
   name: "{{ name|lower }}"
@@ -21,7 +21,7 @@ requirements:
     - requests >=2.31
     - multitasking >=0.0.7
     - lxml >=4.9.1
-    - appdirs >=1.4.4
+    - platformdirs >=2.0.0
     - pytz >=2022.5
     - frozendict >=2.3.4
     - beautifulsoup4 >=4.11.1
@@ -37,7 +37,7 @@ requirements:
     - requests >=2.31
     - multitasking >=0.0.7
     - lxml >=4.9.1
-    - appdirs >=1.4.4
+    - platformdirs >=2.0.0
     - pytz >=2022.5
     - frozendict >=2.3.4
     - beautifulsoup4 >=4.11.1

requirements.txt

@@ -3,9 +3,9 @@ numpy>=1.16.5
 requests>=2.31
 multitasking>=0.0.7
 lxml>=4.9.1
-appdirs>=1.4.4
+platformdirs>=2.0.0
 pytz>=2022.5
 frozendict>=2.3.4
 beautifulsoup4>=4.11.1
 html5lib>=1.1
-peewee>=3.16.2
+peewee>=3.16.2

setup.py

@@ -61,9 +61,13 @@ setup(
     packages=find_packages(exclude=['contrib', 'docs', 'tests', 'examples']),
     install_requires=['pandas>=1.3.0', 'numpy>=1.16.5',
                       'requests>=2.31', 'multitasking>=0.0.7',
-                      'lxml>=4.9.1', 'appdirs>=1.4.4', 'pytz>=2022.5',
+                      'lxml>=4.9.1', 'platformdirs>=2.0.0', 'pytz>=2022.5',
                       'frozendict>=2.3.4', 'peewee>=3.16.2',
                       'beautifulsoup4>=4.11.1', 'html5lib>=1.1'],
+    extras_require={
+        'nospam': ['requests_cache>=1.0', 'requests_ratelimiter>=0.3.1'],
+        'repair': ['scipy>=1.6.3'],
+    },
     # Note: Pandas.read_html() needs html5lib & beautifulsoup4
     entry_points={
         'console_scripts': [

tests/__init__.py

@@ -1 +0,0 @@
-#!/usr/bin/env python

tests/context.py

@@ -1,21 +1,23 @@
 # -*- coding: utf-8 -*-
 
-import appdirs as _ad
+import platformdirs as _ad
 import datetime as _dt
 import sys
 import os
+import yfinance
+from requests import Session
+from requests_cache import CacheMixin, SQLiteCache
+from requests_ratelimiter import LimiterMixin, MemoryQueueBucket
+from pyrate_limiter import Duration, RequestRate, Limiter
+
 _parent_dp = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
 _src_dp = _parent_dp
 sys.path.insert(0, _src_dp)
 
-import yfinance
-
-
 # Optional: see the exact requests that are made during tests:
 # import logging
 # logging.basicConfig(level=logging.DEBUG)
 
-
 # Use adjacent cache folder for testing, delete if already exists and older than today
 testing_cache_dirpath = os.path.join(_ad.user_cache_dir(), "py-yfinance-testing")
 yfinance.set_tz_cache_location(testing_cache_dirpath)
@@ -27,12 +29,8 @@ if os.path.isdir(testing_cache_dirpath):
 
 
 # Setup a session to rate-limit and cache persistently:
-from requests import Session
-from requests_cache import CacheMixin, SQLiteCache
-from requests_ratelimiter import LimiterMixin, MemoryQueueBucket
 class CachedLimiterSession(CacheMixin, LimiterMixin, Session):
     pass
-from pyrate_limiter import Duration, RequestRate, Limiter
 history_rate = RequestRate(1, Duration.SECOND*2)
 limiter = Limiter(history_rate)
 cache_fp = os.path.join(testing_cache_dirpath, "unittests-cache")

tests/data/AV-L-1wk-bad-stock-split-fixed.csv

@@ -1,27 +1,27 @@
 Date,Open,High,Low,Close,Adj Close,Volume,Dividends,Stock Splits
-2021-12-13 00:00:00+00:00,393.999975585938,406.6,391.4,402.899916992188,291.232287597656,62714764.4736842,0,0
-2021-12-20 00:00:00+00:00,393.999975585938,412.199990234375,392.502983398438,409.899997558594,296.292243652344,46596651.3157895,0,0
-2021-12-27 00:00:00+00:00,409.899997558594,416.550971679688,408.387001953125,410.4,296.653642578125,10818482.8947368,0,0
-2022-01-03 00:00:00+00:00,410.4,432.199995117188,410.4,432.099985351563,312.339265136719,44427327.6315789,0,0
-2022-01-10 00:00:00+00:00,431.3,439.199982910156,429.099970703125,436.099912109375,315.230618896484,29091400,0,0
-2022-01-17 00:00:00+00:00,437.999912109375,445.199965820313,426.999997558594,431.999975585938,312.267017822266,43787351.3157895,0,0
-2022-01-24 00:00:00+00:00,430.099975585938,440.999973144531,420.999968261719,433.499982910156,313.351237792969,58487296.0526316,0,0
-2022-01-31 00:00:00+00:00,436.199968261719,443.049987792969,432.099985351563,435.199916992188,314.580045166016,43335806.5789474,0,0
-2022-02-07 00:00:00+00:00,437.899995117188,448.799992675781,436.051994628906,444.39998046875,321.230207519531,39644061.8421053,0,0
-2022-02-14 00:00:00+00:00,437.699975585938,441.999978027344,426.699968261719,432.199995117188,312.411558837891,49972693.4210526,0,0
-2022-02-21 00:00:00+00:00,435.499992675781,438.476999511719,408.29998046875,423.399970703125,306.050571289063,65719596.0526316,0,0
-2022-02-28 00:00:00+00:00,415.099995117188,427.999909667969,386.199932861328,386.799945068359,279.594578857422,94057936.8421053,4.1875,0
-2022-03-07 00:00:00+00:00,374.999952392578,417.299978027344,361.101981201172,409.599968261719,298.389248046875,71269101.3157895,0,0
-2022-03-14 00:00:00+00:00,413.099985351563,426.699968261719,408.899992675781,422.399965820313,307.713929443359,55431927.6315789,0,0
-2022-03-21 00:00:00+00:00,422.699995117188,442.7,422.399965820313,437.799985351563,318.932696533203,39896352.6315789,0,0
-2022-03-28 00:00:00+01:00,442.49998046875,460.999978027344,440.097983398438,444.6,323.886403808594,56413515.7894737,0,0
-2022-04-04 00:00:00+01:00,439.699985351563,445.399985351563,421.999973144531,425.799973144531,310.190817871094,49415836.8421053,19.342106,0
-2022-04-11 00:00:00+01:00,425.39998046875,435.599909667969,420.799995117188,434.299968261719,327.211427001953,29875081.5789474,0,0
-2022-04-18 00:00:00+01:00,434.299968261719,447.799987792969,433.599992675781,437.799985351563,329.848419189453,49288272.3684211,0,0
-2022-04-25 00:00:00+01:00,430.699987792969,438.799990234375,423.999982910156,433.299916992188,326.457967529297,44656776.3157895,0,0
-2022-05-02 00:00:00+01:00,433.299916992188,450.999975585938,414.499982910156,414.899975585938,312.595018310547,29538167.1052632,0,0
-2022-05-09 00:00:00+01:00,413.199995117188,417.449992675781,368.282923583984,408.199970703125,307.547099609375,73989611.8421053,0,0
-2022-05-16 00:00:00+01:00,384,423.600006103516,384,412.100006103516,310.485473632813,81938261,101.69,0.76
+2021-12-13 00:00:00+00:00,518.421020507813,535,515,530.131469726563,383.200378417969,47663221,0,0
+2021-12-20 00:00:00+00:00,518.421020507813,542.368408203125,516.451293945313,539.342102050781,389.858215332031,35413455,0,0
+2021-12-27 00:00:00+00:00,539.342102050781,548.093383789063,537.351318359375,540,390.333740234375,8222047,0,0
+2022-01-03 00:00:00+00:00,540,568.684204101563,540,568.552612304688,410.972717285156,33764769,0,0
+2022-01-10 00:00:00+00:00,567.5,577.894714355469,564.605224609375,573.815673828125,414.777130126953,22109464,0,0
+2022-01-17 00:00:00+00:00,576.315673828125,585.789428710938,561.842102050781,568.421020507813,410.877655029297,33278387,0,0
+2022-01-24 00:00:00+00:00,565.921020507813,580.263122558594,553.947326660156,570.394714355469,412.304260253906,44450345,0,0
+2022-01-31 00:00:00+00:00,573.947326660156,582.960510253906,568.552612304688,572.631469726563,413.921112060547,32935213,0,0
+2022-02-07 00:00:00+00:00,576.184204101563,590.526306152344,573.752624511719,584.73681640625,422.671325683594,30129487,0,0
+2022-02-14 00:00:00+00:00,575.921020507813,581.578918457031,561.447326660156,568.684204101563,411.067840576172,37979247,0,0
+2022-02-21 00:00:00+00:00,573.026306152344,576.943420410156,537.23681640625,557.105224609375,402.698120117188,49946893,0,0
+2022-02-28 00:00:00+00:00,546.184204101563,563.157775878906,508.157806396484,508.947296142578,367.887603759766,71484032,4.1875,0
+2022-03-07 00:00:00+00:00,493.420989990234,549.078918457031,475.134185791016,538.947326660156,392.617431640625,54164517,0,0
+2022-03-14 00:00:00+00:00,543.552612304688,561.447326660156,538.026306152344,555.789428710938,404.886749267578,42128265,0,0
+2022-03-21 00:00:00+00:00,556.184204101563,582.5,555.789428710938,576.052612304688,419.648284912109,30321228,0,0
+2022-03-28 00:00:00+01:00,582.23681640625,606.578918457031,579.076293945313,585,426.166320800781,42874272,0,0
+2022-04-04 00:00:00+01:00,578.552612304688,586.052612304688,555.263122558594,560.263122558594,408.145812988281,37556036,19.342106,0
+2022-04-11 00:00:00+01:00,559.73681640625,573.157775878906,553.684204101563,571.447326660156,430.541351318359,22705062,0,0
+2022-04-18 00:00:00+01:00,571.447326660156,589.210510253906,570.526306152344,576.052612304688,434.011077880859,37459087,0,0
+2022-04-25 00:00:00+01:00,566.710510253906,577.368408203125,557.894714355469,570.131469726563,429.549957275391,33939150,0,0
+2022-05-02 00:00:00+01:00,570.131469726563,593.421020507813,545.394714355469,545.921020507813,411.309234619141,22449007,0,0
+2022-05-09 00:00:00+01:00,543.684204101563,549.276306152344,484.582794189453,537.105224609375,404.667236328125,56232105,0,0
+2022-05-16 00:00:00+01:00,505.263157894737,557.368429083573,505.263157894737,542.236850136205,408.533517937911,62273078.36,101.69,0.76
 2022-05-23 00:00:00+01:00,416.100006103516,442.399993896484,341.915008544922,440.899993896484,409.764678955078,45432941,0,0
 2022-05-30 00:00:00+01:00,442.700012207031,444.200012207031,426.600006103516,428.700012207031,398.426239013672,37906659,0,0
 2022-06-06 00:00:00+01:00,425.299987792969,434.010009765625,405.200012207031,405.399993896484,376.771606445313,40648810,0,0

tests/data/CNE-L-1d-bad-stock-split-fixed.csv

@@ -2,10 +2,10 @@ Date,Open,High,Low,Close,Adj Close,Volume,Dividends,Stock Splits
 2023-05-18 00:00:00+01:00,193.220001220703,200.839996337891,193.220001220703,196.839996337891,196.839996337891,653125,0,0
 2023-05-17 00:00:00+01:00,199.740005493164,207.738006591797,190.121994018555,197.860000610352,197.860000610352,822268,0,0
 2023-05-16 00:00:00+01:00,215.600006103516,215.600006103516,201.149993896484,205.100006103516,205.100006103516,451009,243.93939,0.471428571428571
-2023-05-15 00:00:00+01:00,215.399955531529,219.19995640346,210.599967302595,217.399987792969,102.39998147147,1761679.3939394,0,0
-2023-05-12 00:00:00+01:00,214.599988664899,216.199965558733,209.599965558733,211.399977329799,99.573855808803,1522298.48484849,0,0
-2023-05-11 00:00:00+01:00,219.999966430664,219.999966430664,212.199987357003,215.000000871931,101.269541277204,3568042.12121213,0,0
-2023-05-10 00:00:00+01:00,218.199954659598,223.000000435965,212.59995640346,215.399955531529,101.457929992676,5599908.78787879,0,0
-2023-05-09 00:00:00+01:00,224,227.688003540039,218.199996948242,218.399993896484,102.87100982666,1906090,0,0
-2023-05-05 00:00:00+01:00,220.999968174526,225.19996686663,220.799976457868,224.4,105.697140066964,964523.636363637,0,0
-2023-05-04 00:00:00+01:00,216.999989972796,222.799965558733,216.881988961356,221.399965994698,104.284055655343,880983.93939394,0,0
+2023-05-15 00:00:00+01:00,456.9090,464.9696,446.7272,461.1515,217.2121,830506.0000,0,0
+2023-05-12 00:00:00+01:00,455.2121,458.6060,444.6060,448.4242,211.2173,717655.0000,0,0
+2023-05-11 00:00:00+01:00,466.6666,466.6666,450.1212,456.0606,214.8142,1682077.0000,0,0
+2023-05-10 00:00:00+01:00,462.8484,473.0303,450.9696,456.9090,215.2138,2639957.0000,0,0
+2023-05-09 00:00:00+01:00,475.1515,482.9746,462.8485,463.2727,218.2112,898585.2857,0,0
+2023-05-05 00:00:00+01:00,468.7878,477.6969,468.3636,476.0000,224.2061,454704.0000,0,0
+2023-05-04 00:00:00+01:00,460.3030,472.6060,460.0527,469.6363,221.2086,415321.0000,0,0

tests/test_prices.py

@@ -44,13 +44,25 @@ class TestPriceHistory(unittest.TestCase):
             df_tkrs = df.columns.levels[1]
             self.assertEqual(sorted(tkrs), sorted(df_tkrs))
 
+    def test_download_with_invalid_ticker(self):
+        #Checks if using an invalid symbol gives the same output as not using an invalid symbol in combination with a valid symbol (AAPL)
+        #Checks to make sure that invalid symbol handling for the date column is the same as the base case (no invalid symbols)
+
+        invalid_tkrs = ["AAPL", "ATVI"] #AAPL exists and ATVI does not exist
+        valid_tkrs = ["AAPL", "INTC"] #AAPL and INTC both exist
+
+        data_invalid_sym = yf.download(invalid_tkrs, start='2023-11-16', end='2023-11-17')
+        data_valid_sym = yf.download(valid_tkrs, start='2023-11-16', end='2023-11-17')
+
+        self.assertEqual(data_invalid_sym['Close']['AAPL']['2023-11-16'],data_valid_sym['Close']['AAPL']['2023-11-16'])
+
     def test_duplicatingHourly(self):
         tkrs = ["IMP.JO", "BHG.JO", "SSW.JO", "BP.L", "INTC"]
         for tkr in tkrs:
             dat = yf.Ticker(tkr, session=self.session)
             tz = dat._get_ticker_tz(proxy=None, timeout=None)
 
-            dt_utc = _tz.timezone("UTC").localize(_dt.datetime.utcnow())
+            dt_utc = _pd.Timestamp.utcnow()
             dt = dt_utc.astimezone(_tz.timezone(tz))
             start_d = dt.date() - _dt.timedelta(days=7)
             df = dat.history(start=start_d, interval="1h")
@@ -70,7 +82,7 @@ class TestPriceHistory(unittest.TestCase):
             dat = yf.Ticker(tkr, session=self.session)
             tz = dat._get_ticker_tz(proxy=None, timeout=None)
 
-            dt_utc = _tz.timezone("UTC").localize(_dt.datetime.utcnow())
+            dt_utc = _pd.Timestamp.utcnow()
             dt = dt_utc.astimezone(_tz.timezone(tz))
             if dt.time() < _dt.time(17, 0):
                 continue
@@ -132,7 +144,6 @@ class TestPriceHistory(unittest.TestCase):
 
     def test_pricesEventsMerge_bug(self):
         # Reproduce exception when merging intraday prices with future dividend
-        tkr = 'S32.AX'
         interval = '30m'
         df_index = []
         d = 13
@@ -148,7 +159,7 @@ class TestPriceHistory(unittest.TestCase):
         future_div_dt = _dt.datetime(2023, 9, 14, 10)
         divs = _pd.DataFrame(data={"Dividends":[div]}, index=[future_div_dt])
 
-        df2 = yf.utils.safe_merge_dfs(df, divs, interval)
+        yf.utils.safe_merge_dfs(df, divs, interval)
         # No exception = test pass
 
     def test_intraDayWithEvents(self):
@@ -223,8 +234,10 @@ class TestPriceHistory(unittest.TestCase):
                 self.assertTrue((df_divs.index.date == dates).all())
             except AssertionError:
                 print(f'- ticker = {tkr}')
-                print('- response:') ; print(df_divs.index.date)
-                print('- answer:') ; print(dates)
+                print('- response:')
+                print(df_divs.index.date)
+                print('- answer:')
+                print(dates)
                 raise
 
     def test_dailyWithEvents_bugs(self):
@@ -264,66 +277,12 @@ class TestPriceHistory(unittest.TestCase):
         # Reproduce issue #1634 - 1d dividend out-of-range, should be prepended to prices
         div_dt = _pd.Timestamp(2022, 7, 21).tz_localize("America/New_York")
         df_dividends = _pd.DataFrame(data={"Dividends":[1.0]}, index=[div_dt])
-        df_prices = _pd.DataFrame(data={c:[1.0] for c in yf.const.price_colnames}|{'Volume':0}, index=[div_dt+_dt.timedelta(days=1)])
+        df_prices = _pd.DataFrame(data={c:[1.0] for c in yf.const._PRICE_COLNAMES_}|{'Volume':0}, index=[div_dt+_dt.timedelta(days=1)])
         df_merged = yf.utils.safe_merge_dfs(df_prices, df_dividends, '1d')
         self.assertEqual(df_merged.shape[0], 2)
         self.assertTrue(df_merged[df_prices.columns].iloc[1:].equals(df_prices))
         self.assertEqual(df_merged.index[0], div_dt)
 
-    def test_intraDayWithEvents(self):
-        tkrs = ["BHP.AX", "IMP.JO", "BP.L", "PNL.L", "INTC"]
-        test_run = False
-        for tkr in tkrs:
-            start_d = _dt.date.today() - _dt.timedelta(days=59)
-            end_d = None
-            df_daily = yf.Ticker(tkr, session=self.session).history(start=start_d, end=end_d, interval="1d", actions=True)
-            df_daily_divs = df_daily["Dividends"][df_daily["Dividends"] != 0]
-            if df_daily_divs.shape[0] == 0:
-                continue
-
-            last_div_date = df_daily_divs.index[-1]
-            start_d = last_div_date.date()
-            end_d = last_div_date.date() + _dt.timedelta(days=1)
-            df_intraday = yf.Ticker(tkr, session=self.session).history(start=start_d, end=end_d, interval="15m", actions=True)
-            self.assertTrue((df_intraday["Dividends"] != 0.0).any())
-
-            df_intraday_divs = df_intraday["Dividends"][df_intraday["Dividends"] != 0]
-            df_intraday_divs.index = df_intraday_divs.index.floor('D')
-            self.assertTrue(df_daily_divs.equals(df_intraday_divs))
-
-            test_run = True
-
-        if not test_run:
-            self.skipTest("Skipping test_intraDayWithEvents() because no tickers had a dividend in last 60 days")
-
-    def test_intraDayWithEvents_tase(self):
-        # TASE dividend release pre-market, doesn't merge nicely with intra-day data so check still present
-
-        tase_tkrs = ["ICL.TA", "ESLT.TA", "ONE.TA", "MGDL.TA"]
-        test_run = False
-        for tkr in tase_tkrs:
-            start_d = _dt.date.today() - _dt.timedelta(days=59)
-            end_d = None
-            df_daily = yf.Ticker(tkr, session=self.session).history(start=start_d, end=end_d, interval="1d", actions=True)
-            df_daily_divs = df_daily["Dividends"][df_daily["Dividends"] != 0]
-            if df_daily_divs.shape[0] == 0:
-                continue
-
-            last_div_date = df_daily_divs.index[-1]
-            start_d = last_div_date.date()
-            end_d = last_div_date.date() + _dt.timedelta(days=1)
-            df_intraday = yf.Ticker(tkr, session=self.session).history(start=start_d, end=end_d, interval="15m", actions=True)
-            self.assertTrue((df_intraday["Dividends"] != 0.0).any())
-
-            df_intraday_divs = df_intraday["Dividends"][df_intraday["Dividends"] != 0]
-            df_intraday_divs.index = df_intraday_divs.index.floor('D')
-            self.assertTrue(df_daily_divs.equals(df_intraday_divs))
-
-            test_run = True
-
-        if not test_run:
-            self.skipTest("Skipping test_intraDayWithEvents_tase() because no tickers had a dividend in last 60 days")
-
     def test_weeklyWithEvents(self):
         # Reproduce issue #521
         tkr1 = "QQQ"
@@ -400,13 +359,6 @@ class TestPriceHistory(unittest.TestCase):
         dfd_divs = dfd[dfd['Dividends'] != 0]
         self.assertEqual(dfm_divs.shape[0], dfd_divs.shape[0])
 
-        dfm = yf.Ticker("F").history(period="50mo", interval="1mo")
-        dfd = yf.Ticker("F").history(period="50mo", interval="1d")
-        dfd = dfd[dfd.index > dfm.index[0]]
-        dfm_divs = dfm[dfm['Dividends'] != 0]
-        dfd_divs = dfd[dfd['Dividends'] != 0]
-        self.assertEqual(dfm_divs.shape[0], dfd_divs.shape[0])
-
     def test_tz_dst_ambiguous(self):
         # Reproduce issue #1100
         try:
@@ -415,9 +367,9 @@ class TestPriceHistory(unittest.TestCase):
             raise Exception("Ambiguous DST issue not resolved")
 
     def test_dst_fix(self):
-        # Daily intervals should start at time 00:00. But for some combinations of date and timezone, 
+        # Daily intervals should start at time 00:00. But for some combinations of date and timezone,
         # Yahoo has time off by few hours (e.g. Brazil 23:00 around Jan-2022). Suspect DST problem.
-        # The clue is (a) minutes=0 and (b) hour near 0. 
+        # The clue is (a) minutes=0 and (b) hour near 0.
         # Obviously Yahoo meant 00:00, so ensure this doesn't affect date conversion.
 
         # The correction is successful if no days are weekend, and weekly data begins Monday
@@ -440,24 +392,20 @@ class TestPriceHistory(unittest.TestCase):
             raise
 
     def test_prune_post_intraday_us(self):
-        # Half-day before USA Thanksgiving. Yahoo normally 
-        # returns an interval starting when regular trading closes, 
+        # Half-day at USA Thanksgiving. Yahoo normally
+        # returns an interval starting when regular trading closes,
         # even if prepost=False.
 
         # Setup
         tkr = "AMZN"
-        interval = "1h"
-        interval_td = _dt.timedelta(hours=1)
-        time_open = _dt.time(9, 30)
-        time_close = _dt.time(16)
-        special_day = _dt.date(2022, 11, 25)
+        special_day = _dt.date(2023, 11, 24)
         time_early_close = _dt.time(13)
         dat = yf.Ticker(tkr, session=self.session)
 
         # Run
         start_d = special_day - _dt.timedelta(days=7)
         end_d = special_day + _dt.timedelta(days=7)
-        df = dat.history(start=start_d, end=end_d, interval=interval, prepost=False, keepna=True)
+        df = dat.history(start=start_d, end=end_d, interval="1h", prepost=False, keepna=True)
         tg_last_dt = df.loc[str(special_day)].index[-1]
         self.assertTrue(tg_last_dt.time() < time_early_close)
 
@@ -466,88 +414,22 @@ class TestPriceHistory(unittest.TestCase):
         end_d = _dt.date(special_day.year+1, 1, 1)
         df = dat.history(start=start_d, end=end_d, interval="1h", prepost=False, keepna=True)
         last_dts = _pd.Series(df.index).groupby(df.index.date).last()
-        f_early_close = (last_dts+interval_td).dt.time < time_close
-        early_close_dates = last_dts.index[f_early_close].values
-        self.assertEqual(len(early_close_dates), 1)
-        self.assertEqual(early_close_dates[0], special_day)
-
-        first_dts = _pd.Series(df.index).groupby(df.index.date).first()
-        f_late_open = first_dts.dt.time > time_open
-        late_open_dates = first_dts.index[f_late_open]
-        self.assertEqual(len(late_open_dates), 0)
-
-    def test_prune_post_intraday_omx(self):
-        # Half-day before Sweden Christmas. Yahoo normally 
-        # returns an interval starting when regular trading closes, 
-        # even if prepost=False.
-        # If prepost=False, test that yfinance is removing prepost intervals.
-
-        # Setup
-        tkr = "AEC.ST"
-        interval = "1h"
-        interval_td = _dt.timedelta(hours=1)
-        time_open = _dt.time(9)
-        time_close = _dt.time(17, 30)
-        special_day = _dt.date(2022, 12, 23)
-        time_early_close = _dt.time(13, 2)
-        dat = yf.Ticker(tkr, session=self.session)
-
-        # Half trading day Jan 5, Apr 14, May 25, Jun 23, Nov 4, Dec 23, Dec 30
-        half_days = [_dt.date(special_day.year, x[0], x[1]) for x in [(1, 5), (4, 14), (5, 25), (6, 23), (11, 4), (12, 23), (12, 30)]]
-
-        # Yahoo has incorrectly classified afternoon of 2022-04-13 as post-market.
-        # Nothing yfinance can do because Yahoo doesn't return data with prepost=False.
-        # But need to handle in this test.
-        expected_incorrect_half_days = [_dt.date(2022, 4, 13)]
-        half_days = sorted(half_days+expected_incorrect_half_days)
-
-        # Run
-        start_d = special_day - _dt.timedelta(days=7)
-        end_d = special_day + _dt.timedelta(days=7)
-        df = dat.history(start=start_d, end=end_d, interval=interval, prepost=False, keepna=True)
-        tg_last_dt = df.loc[str(special_day)].index[-1]
-        self.assertTrue(tg_last_dt.time() < time_early_close)
-
-        # Test no other afternoons (or mornings) were pruned
-        start_d = _dt.date(special_day.year, 1, 1)
-        end_d = _dt.date(special_day.year+1, 1, 1)
-        df = dat.history(start=start_d, end=end_d, interval="1h", prepost=False, keepna=True)
-        last_dts = _pd.Series(df.index).groupby(df.index.date).last()
-        f_early_close = (last_dts+interval_td).dt.time < time_close
-        early_close_dates = last_dts.index[f_early_close].values
-        unexpected_early_close_dates = [d for d in early_close_dates if d not in half_days]
-        self.assertEqual(len(unexpected_early_close_dates), 0)
-        self.assertEqual(len(early_close_dates), len(half_days))
-        self.assertTrue(_np.equal(early_close_dates, half_days).all())
-
-        first_dts = _pd.Series(df.index).groupby(df.index.date).first()
-        f_late_open = first_dts.dt.time > time_open
-        late_open_dates = first_dts.index[f_late_open]
-        self.assertEqual(len(late_open_dates), 0)
+        dfd = dat.history(start=start_d, end=end_d, interval='1d', prepost=False, keepna=True)
+        self.assertTrue(_np.equal(dfd.index.date, _pd.to_datetime(last_dts.index).date).all())
 
     def test_prune_post_intraday_asx(self):
         # Setup
         tkr = "BHP.AX"
-        interval = "1h"
-        interval_td = _dt.timedelta(hours=1)
-        time_open = _dt.time(10)
-        time_close = _dt.time(16, 12)
-        # No early closes in 2022
+        # No early closes in 2023
         dat = yf.Ticker(tkr, session=self.session)
 
-        # Test no afternoons (or mornings) were pruned
-        start_d = _dt.date(2022, 1, 1)
-        end_d = _dt.date(2022+1, 1, 1)
+        # Test no other afternoons (or mornings) were pruned
+        start_d = _dt.date(2023, 1, 1)
+        end_d = _dt.date(2023+1, 1, 1)
         df = dat.history(start=start_d, end=end_d, interval="1h", prepost=False, keepna=True)
         last_dts = _pd.Series(df.index).groupby(df.index.date).last()
-        f_early_close = (last_dts+interval_td).dt.time < time_close
-        early_close_dates = last_dts.index[f_early_close].values
-        self.assertEqual(len(early_close_dates), 0)
-
-        first_dts = _pd.Series(df.index).groupby(df.index.date).first()
-        f_late_open = first_dts.dt.time > time_open
-        late_open_dates = first_dts.index[f_late_open]
-        self.assertEqual(len(late_open_dates), 0)
+        dfd = dat.history(start=start_d, end=end_d, interval='1d', prepost=False, keepna=True)
+        self.assertTrue(_np.equal(dfd.index.date, _pd.to_datetime(last_dts.index).date).all())
 
     def test_weekly_2rows_fix(self):
         tkr = "AMZN"
@@ -566,7 +448,7 @@ class TestPriceHistory(unittest.TestCase):
         end = "2019-12-31"
         interval = "3mo"
 
-        df = dat.history(start=start, end=end, interval=interval)
+        dat.history(start=start, end=end, interval=interval)
 
 
 class TestPriceRepair(unittest.TestCase):
@@ -581,6 +463,18 @@ class TestPriceRepair(unittest.TestCase):
         if cls.session is not None:
             cls.session.close()
 
+    def test_types(self):
+        tkr = 'INTC'
+        dat = yf.Ticker(tkr, session=self.session)
+
+        data = dat.history(period="3mo", interval="1d", prepost=True, repair=True)
+        self.assertIsInstance(data, _pd.DataFrame, "data has wrong type")
+        self.assertFalse(data.empty, "data is empty")
+
+        reconstructed = dat._lazy_load_price_history()._reconstruct_intervals_batch(data, "1wk", True)
+        self.assertIsInstance(reconstructed, _pd.DataFrame, "data has wrong type")
+        self.assertFalse(data.empty, "data is empty")
+
     def test_reconstruct_2m(self):
         # 2m repair requires 1m data.
         # Yahoo restricts 1m fetches to 7 days max within last 30 days.
@@ -589,7 +483,6 @@ class TestPriceRepair(unittest.TestCase):
         tkrs = ["BHP.AX", "IMP.JO", "BP.L", "PNL.L", "INTC"]
 
         dt_now = _pd.Timestamp.utcnow()
-        td_7d = _dt.timedelta(days=7)
         td_60d = _dt.timedelta(days=60)
 
         # Round time for 'requests_cache' reuse
@@ -599,13 +492,14 @@ class TestPriceRepair(unittest.TestCase):
             dat = yf.Ticker(tkr, session=self.session)
             end_dt = dt_now
             start_dt = end_dt - td_60d
-            df = dat.history(start=start_dt, end=end_dt, interval="2m", repair=True)
+            dat.history(start=start_dt, end=end_dt, interval="2m", repair=True)
 
     def test_repair_100x_random_weekly(self):
         # Setup:
         tkr = "PNL.L"
         dat = yf.Ticker(tkr, session=self.session)
         tz_exchange = dat.fast_info["timezone"]
+        hist = dat._lazy_load_price_history()
 
         data_cols = ["Low", "High", "Open", "Close", "Adj Close"]
         df = _pd.DataFrame(data={"Open":      [470.5, 473.5, 474.5, 470],
@@ -629,7 +523,7 @@ class TestPriceRepair(unittest.TestCase):
 
         # Run test
 
-        df_repaired = dat._fix_unit_random_mixups(df_bad, "1wk", tz_exchange, prepost=False)
+        df_repaired = hist._fix_unit_random_mixups(df_bad, "1wk", tz_exchange, prepost=False)
 
         # First test - no errors left
         for c in data_cols:
@@ -660,6 +554,7 @@ class TestPriceRepair(unittest.TestCase):
         tkr = "PNL.L"
         dat = yf.Ticker(tkr, session=self.session)
         tz_exchange = dat.fast_info["timezone"]
+        hist = dat._lazy_load_price_history()
 
         data_cols = ["Low", "High", "Open", "Close", "Adj Close"]
         df = _pd.DataFrame(data={"Open":      [400,    398,    392.5,  417],
@@ -686,7 +581,7 @@ class TestPriceRepair(unittest.TestCase):
         df.index = df.index.tz_localize(tz_exchange)
         df_bad.index = df_bad.index.tz_localize(tz_exchange)
 
-        df_repaired = dat._fix_unit_random_mixups(df_bad, "1wk", tz_exchange, prepost=False)
+        df_repaired = hist._fix_unit_random_mixups(df_bad, "1wk", tz_exchange, prepost=False)
 
         # First test - no errors left
         for c in data_cols:
@@ -718,6 +613,7 @@ class TestPriceRepair(unittest.TestCase):
         tkr = "PNL.L"
         dat = yf.Ticker(tkr, session=self.session)
         tz_exchange = dat.fast_info["timezone"]
+        hist = dat._lazy_load_price_history()
 
         data_cols = ["Low", "High", "Open", "Close", "Adj Close"]
         df = _pd.DataFrame(data={"Open":      [478,    476,   476,   472],
@@ -739,7 +635,7 @@ class TestPriceRepair(unittest.TestCase):
         df.index = df.index.tz_localize(tz_exchange)
         df_bad.index = df_bad.index.tz_localize(tz_exchange)
 
-        df_repaired = dat._fix_unit_random_mixups(df_bad, "1d", tz_exchange, prepost=False)
+        df_repaired = hist._fix_unit_random_mixups(df_bad, "1d", tz_exchange, prepost=False)
 
         # First test - no errors left
         for c in data_cols:
@@ -768,6 +664,7 @@ class TestPriceRepair(unittest.TestCase):
             for interval in ['1d', '1wk']:
                 dat = yf.Ticker(tkr, session=self.session)
                 tz_exchange = dat.fast_info["timezone"]
+                hist = dat._lazy_load_price_history()
 
                 data_cols = ["Low", "High", "Open", "Close", "Adj Close"]
                 _dp = os.path.dirname(__file__)
@@ -784,7 +681,7 @@ class TestPriceRepair(unittest.TestCase):
                 df.index = _pd.to_datetime(df.index, utc=True).tz_convert(tz_exchange)
                 df = df.sort_index()
 
-                df_repaired = dat._fix_unit_switch(df_bad, interval, tz_exchange)
+                df_repaired = hist._fix_unit_switch(df_bad, interval, tz_exchange)
                 df_repaired = df_repaired.sort_index()
 
                 # First test - no errors left
@@ -816,6 +713,7 @@ class TestPriceRepair(unittest.TestCase):
     def test_repair_zeroes_daily(self):
         tkr = "BBIL.L"
         dat = yf.Ticker(tkr, session=self.session)
+        hist = dat._lazy_load_price_history()
         tz_exchange = dat.fast_info["timezone"]
 
         df_bad = _pd.DataFrame(data={"Open":      [0,      102.04, 102.04],
@@ -831,7 +729,7 @@ class TestPriceRepair(unittest.TestCase):
         df_bad.index.name = "Date"
         df_bad.index = df_bad.index.tz_localize(tz_exchange)
 
-        repaired_df = dat._fix_zeroes(df_bad, "1d", tz_exchange, prepost=False)
+        repaired_df = hist._fix_zeroes(df_bad, "1d", tz_exchange, prepost=False)
 
         correct_df = df_bad.copy()
         correct_df.loc["2022-11-01", "Open"] = 102.080002
@@ -844,7 +742,7 @@ class TestPriceRepair(unittest.TestCase):
         self.assertFalse(repaired_df["Repaired?"].isna().any())
 
     def test_repair_zeroes_daily_adjClose(self):
-        # Test that 'Adj Close' is reconstructed correctly, 
+        # Test that 'Adj Close' is reconstructed correctly,
         # particularly when a dividend occurred within 1 day.
 
         tkr = "INTC"
@@ -865,6 +763,7 @@ class TestPriceRepair(unittest.TestCase):
         dat = yf.Ticker(tkr, session=self.session)
         tz_exchange = dat.fast_info["timezone"]
         df.index = df.index.tz_localize(tz_exchange)
+        hist = dat._lazy_load_price_history()
 
         rtol = 5e-3
         for i in [0, 1, 2]:
@@ -873,7 +772,7 @@ class TestPriceRepair(unittest.TestCase):
                 df_slice_bad = df_slice.copy()
                 df_slice_bad.loc[df_slice_bad.index[j], "Adj Close"] = 0.0
 
-                df_slice_bad_repaired = dat._fix_zeroes(df_slice_bad, "1d", tz_exchange, prepost=False)
+                df_slice_bad_repaired = hist._fix_zeroes(df_slice_bad, "1d", tz_exchange, prepost=False)
                 for c in ["Close", "Adj Close"]:
                     self.assertTrue(_np.isclose(df_slice_bad_repaired[c], df_slice[c], rtol=rtol).all())
                 self.assertTrue("Repaired?" in df_slice_bad_repaired.columns)
@@ -883,8 +782,9 @@ class TestPriceRepair(unittest.TestCase):
         tkr = "INTC"
         dat = yf.Ticker(tkr, session=self.session)
         tz_exchange = dat.fast_info["timezone"]
+        hist = dat._lazy_load_price_history()
 
-        correct_df = dat.history(period="1wk", interval="1h", auto_adjust=False, repair=True)
+        correct_df = hist.history(period="5d", interval="1h", auto_adjust=False, repair=True)
 
         df_bad = correct_df.copy()
         bad_idx = correct_df.index[10]
@@ -895,7 +795,7 @@ class TestPriceRepair(unittest.TestCase):
         df_bad.loc[bad_idx, "Adj Close"] = _np.nan
         df_bad.loc[bad_idx, "Volume"] = 0
 
-        repaired_df = dat._fix_zeroes(df_bad, "1h", tz_exchange, prepost=False)
+        repaired_df = hist._fix_zeroes(df_bad, "1h", tz_exchange, prepost=False)
 
         for c in ["Open", "Low", "High", "Close"]:
             try:
@@ -913,22 +813,23 @@ class TestPriceRepair(unittest.TestCase):
         self.assertTrue("Repaired?" in repaired_df.columns)
         self.assertFalse(repaired_df["Repaired?"].isna().any())
 
-    def test_repair_bad_stock_split(self):
-        # Stocks that split in 2022 but no problems in Yahoo data, 
+    def test_repair_bad_stock_splits(self):
+        # Stocks that split in 2022 but no problems in Yahoo data,
         # so repair should change nothing
         good_tkrs = ['AMZN', 'DXCM', 'FTNT', 'GOOG', 'GME', 'PANW', 'SHOP', 'TSLA']
-        good_tkrs += ['AEI', 'CHRA', 'GHI', 'IRON', 'LXU', 'NUZE', 'RSLS', 'TISI']
+        good_tkrs += ['AEI', 'GHI', 'IRON', 'LXU', 'NUZE', 'RSLS', 'TISI']
         good_tkrs += ['BOL.ST', 'TUI1.DE']
         intervals = ['1d', '1wk', '1mo', '3mo']
         for tkr in good_tkrs:
             for interval in intervals:
                 dat = yf.Ticker(tkr, session=self.session)
                 tz_exchange = dat.fast_info["timezone"]
+                hist = dat._lazy_load_price_history()
 
                 _dp = os.path.dirname(__file__)
                 df_good = dat.history(start='2020-01-01', end=_dt.date.today(), interval=interval, auto_adjust=False)
 
-                repaired_df = dat._fix_bad_stock_split(df_good, interval, tz_exchange)
+                repaired_df = hist._fix_bad_stock_splits(df_good, interval, tz_exchange)
 
                 # Expect no change from repair
                 df_good = df_good.sort_index()
@@ -948,6 +849,7 @@ class TestPriceRepair(unittest.TestCase):
         for tkr in bad_tkrs:
             dat = yf.Ticker(tkr, session=self.session)
             tz_exchange = dat.fast_info["timezone"]
+            hist = dat._lazy_load_price_history()
 
             _dp = os.path.dirname(__file__)
             interval = '1d'
@@ -958,7 +860,7 @@ class TestPriceRepair(unittest.TestCase):
             df_bad = _pd.read_csv(fp, index_col="Date")
             df_bad.index = _pd.to_datetime(df_bad.index, utc=True)
 
-            repaired_df = dat._fix_bad_stock_split(df_bad, "1d", tz_exchange)
+            repaired_df = hist._fix_bad_stock_splits(df_bad, "1d", tz_exchange)
 
             fp = os.path.join(_dp, "data", tkr.replace('.','-')+'-'+interval+"-bad-stock-split-fixed.csv")
             correct_df = _pd.read_csv(fp, index_col="Date")
@@ -979,8 +881,8 @@ class TestPriceRepair(unittest.TestCase):
                     # print(repaired_df[c] - correct_df[c])
                     raise
 
-        # Had very high price volatility in Jan-2021 around split date that could 
-        # be mistaken for missing stock split adjustment. And old logic did think 
+        # Had very high price volatility in Jan-2021 around split date that could
+        # be mistaken for missing stock split adjustment. And old logic did think
         # column 'High' required fixing - wrong!
         sketchy_tkrs = ['FIZZ']
         intervals = ['1wk']
@@ -988,11 +890,12 @@ class TestPriceRepair(unittest.TestCase):
             for interval in intervals:
                 dat = yf.Ticker(tkr, session=self.session)
                 tz_exchange = dat.fast_info["timezone"]
+                hist = dat._lazy_load_price_history()
 
                 _dp = os.path.dirname(__file__)
-                df_good = dat.history(start='2020-11-30', end='2021-04-01', interval=interval, auto_adjust=False)
+                df_good = hist.history(start='2020-11-30', end='2021-04-01', interval=interval, auto_adjust=False)
 
-                repaired_df = dat._fix_bad_stock_split(df_good, interval, tz_exchange)
+                repaired_df = hist._fix_bad_stock_splits(df_good, interval, tz_exchange)
 
                 # Expect no change from repair
                 df_good = df_good.sort_index()
@@ -1012,12 +915,13 @@ class TestPriceRepair(unittest.TestCase):
 
         dat = yf.Ticker(tkr, session=self.session)
         tz_exchange = dat.fast_info["timezone"]
+        hist = dat._lazy_load_price_history()
 
         _dp = os.path.dirname(__file__)
         df_bad = _pd.read_csv(os.path.join(_dp, "data", tkr.replace('.','-')+"-1d-missing-div-adjust.csv"), index_col="Date")
         df_bad.index = _pd.to_datetime(df_bad.index)
 
-        repaired_df = dat._fix_missing_div_adjust(df_bad, "1d", tz_exchange)
+        repaired_df = hist._fix_missing_div_adjust(df_bad, "1d", tz_exchange)
 
         correct_df = _pd.read_csv(os.path.join(_dp, "data", tkr.replace('.','-')+"-1d-missing-div-adjust-fixed.csv"), index_col="Date")
         correct_df.index = _pd.to_datetime(correct_df.index)

tests/test_ticker.py

@@ -12,28 +12,31 @@ import pandas as pd
 
 from .context import yfinance as yf
 from .context import session_gbl
-from yfinance.exceptions import YFNotImplementedError
+from yfinance.exceptions import YFChartError, YFInvalidPeriodError, YFNotImplementedError, YFTickerMissingError, YFTzMissingError
 
 
 import unittest
 import requests_cache
-from typing import Union, Any
-import re
+from typing import Union, Any, get_args, _GenericAlias
 from urllib.parse import urlparse, parse_qs, urlencode, urlunparse
 
 ticker_attributes = (
     ("major_holders", pd.DataFrame),
     ("institutional_holders", pd.DataFrame),
     ("mutualfund_holders", pd.DataFrame),
+    ("insider_transactions", pd.DataFrame),
+    ("insider_purchases", pd.DataFrame),
+    ("insider_roster_holders", pd.DataFrame),
     ("splits", pd.Series),
     ("actions", pd.DataFrame),
     ("shares", pd.DataFrame),
     ("info", dict),
-    ("calendar", pd.DataFrame),
+    ("calendar", dict),
     ("recommendations", Union[pd.DataFrame, dict]),
+    ("recommendations_summary", Union[pd.DataFrame, dict]),
+    ("upgrades_downgrades", Union[pd.DataFrame, dict]),
     ("earnings", pd.DataFrame),
     ("quarterly_earnings", pd.DataFrame),
-    ("recommendations_summary", Union[pd.DataFrame, dict]),
     ("quarterly_cashflow", pd.DataFrame),
     ("cashflow", pd.DataFrame),
     ("quarterly_balance_sheet", pd.DataFrame),
@@ -54,7 +57,12 @@ def assert_attribute_type(testClass: unittest.TestCase, instance, attribute_name
     try:
         attribute = getattr(instance, attribute_name)
         if attribute is not None and expected_type is not Any:
-            testClass.assertEqual(type(attribute), expected_type)
+            err_msg = f'{attribute_name} type is {type(attribute)} not {expected_type}'
+            if isinstance(expected_type, _GenericAlias) and expected_type.__origin__ is Union:
+                allowed_types = get_args(expected_type)
+                testClass.assertTrue(isinstance(attribute, allowed_types), err_msg)
+            else:
+                testClass.assertEqual(type(attribute), expected_type, err_msg)
     except Exception:
         testClass.assertRaises(
             YFNotImplementedError, lambda: getattr(instance, attribute_name)
@@ -82,7 +90,7 @@ class TestTicker(unittest.TestCase):
 
             # Test:
             dat = yf.Ticker(tkr, session=self.session)
-            tz = dat._get_ticker_tz(proxy=None, timeout=None)
+            tz = dat._get_ticker_tz(proxy=None, timeout=5)
 
             self.assertIsNotNone(tz)
 
@@ -92,20 +100,60 @@ class TestTicker(unittest.TestCase):
         tkr = "DJI"  # typo of "^DJI"
         dat = yf.Ticker(tkr, session=self.session)
 
-        dat.history(period="1wk")
+        dat.history(period="5d")
         dat.history(start="2022-01-01")
         dat.history(start="2022-01-01", end="2022-03-01")
-        yf.download([tkr], period="1wk", threads=False, ignore_tz=False)
-        yf.download([tkr], period="1wk", threads=True, ignore_tz=False)
-        yf.download([tkr], period="1wk", threads=False, ignore_tz=True)
-        yf.download([tkr], period="1wk", threads=True, ignore_tz=True)
+        yf.download([tkr], period="5d", threads=False, ignore_tz=False)
+        yf.download([tkr], period="5d", threads=True, ignore_tz=False)
+        yf.download([tkr], period="5d", threads=False, ignore_tz=True)
+        yf.download([tkr], period="5d", threads=True, ignore_tz=True)
 
         for k in dat.fast_info:
             dat.fast_info[k]
 
         for attribute_name, attribute_type in ticker_attributes:
-            assert_attribute_type(self, dat, attribute_name, attribute_type) 
-       
+            assert_attribute_type(self, dat, attribute_name, attribute_type)
+
+        with self.assertRaises(YFNotImplementedError):
+            assert isinstance(dat.earnings, pd.Series)
+            assert dat.earnings.empty
+        assert isinstance(dat.dividends, pd.Series)
+        assert dat.dividends.empty
+        assert isinstance(dat.splits, pd.Series)
+        assert dat.splits.empty
+        assert isinstance(dat.capital_gains, pd.Series)
+        assert dat.capital_gains.empty
+        with self.assertRaises(YFNotImplementedError):
+            assert isinstance(dat.shares, pd.DataFrame)
+            assert dat.shares.empty
+        assert isinstance(dat.actions, pd.DataFrame)
+        assert dat.actions.empty
+
+    def test_invalid_period(self):
+        tkr = 'VALE'
+        dat = yf.Ticker(tkr, session=self.session)
+        with self.assertRaises(YFInvalidPeriodError):
+            dat.history(period="2wks", interval="1d", raise_errors=True)
+        with self.assertRaises(YFInvalidPeriodError):
+            dat.history(period="2mo", interval="1d", raise_errors=True)
+
+
+    def test_prices_missing(self):
+        # this test will need to be updated every time someone wants to run a test
+        # hard to find a ticker that matches this error other than options
+        # META call option, 2024 April 26th @ strike of 180000
+        tkr = 'META240426C00180000'
+        dat = yf.Ticker(tkr, session=self.session)
+        with self.assertRaises(YFChartError):
+            dat.history(period="5d", interval="1m", raise_errors=True)
+
+    def test_ticker_missing(self):
+        tkr = 'ATVI'
+        dat = yf.Ticker(tkr, session=self.session)
+        # A missing ticker can trigger either a niche error or the generalized error
+        with self.assertRaises((YFTickerMissingError, YFTzMissingError, YFChartError)):
+            dat.history(period="3mo", interval="1d", raise_errors=True)
+
     def test_goodTicker(self):
         # that yfinance works when full api is called on same instance of ticker
 
@@ -114,144 +162,30 @@ class TestTicker(unittest.TestCase):
         for tkr in tkrs:
             dat = yf.Ticker(tkr, session=self.session)
 
-            dat.history(period="1wk")
+            dat.history(period="5d")
             dat.history(start="2022-01-01")
             dat.history(start="2022-01-01", end="2022-03-01")
-            yf.download([tkr], period="1wk", threads=False, ignore_tz=False)
-            yf.download([tkr], period="1wk", threads=True, ignore_tz=False)
-            yf.download([tkr], period="1wk", threads=False, ignore_tz=True)
-            yf.download([tkr], period="1wk", threads=True, ignore_tz=True)
+            yf.download([tkr], period="5d", threads=False, ignore_tz=False)
+            yf.download([tkr], period="5d", threads=True, ignore_tz=False)
+            yf.download([tkr], period="5d", threads=False, ignore_tz=True)
+            yf.download([tkr], period="5d", threads=True, ignore_tz=True)
 
             for k in dat.fast_info:
                 dat.fast_info[k]
 
             for attribute_name, attribute_type in ticker_attributes:
-                assert_attribute_type(self, dat, attribute_name, attribute_type) 
-            
-    #TODO:: Refactor with `assert_attribute` once proxy is accepted as a parameter of `Ticker`
+                assert_attribute_type(self, dat, attribute_name, attribute_type)
+
     def test_goodTicker_withProxy(self):
-        # that yfinance works when full api is called on same instance of ticker
-
         tkr = "IBM"
-        dat = yf.Ticker(tkr, session=self.session)
+        dat = yf.Ticker(tkr, session=self.session, proxy=self.proxy)
 
-        dat._fetch_ticker_tz(proxy=self.proxy, timeout=5)
-        dat._get_ticker_tz(proxy=self.proxy, timeout=5)
-        dat.history(period="1wk", proxy=self.proxy)
+        dat._fetch_ticker_tz(proxy=None, timeout=5)
+        dat._get_ticker_tz(proxy=None, timeout=5)
+        dat.history(period="5d")
 
-        v = dat.get_major_holders(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        v = dat.get_institutional_holders(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        v = dat.get_mutualfund_holders(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        v = dat.get_info(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertTrue(len(v) > 0)
-
-        v = dat.get_income_stmt(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        v = dat.get_incomestmt(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        v = dat.get_financials(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        v = dat.get_balance_sheet(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        v = dat.get_balancesheet(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        v = dat.get_cash_flow(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        v = dat.get_cashflow(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        v = dat.get_shares_full(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        v = dat.get_isin(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertTrue(v != "")
-
-        v = dat.get_news(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertTrue(len(v) > 0)
-
-        v = dat.get_earnings_dates(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertFalse(v.empty)
-
-        dat.get_history_metadata(proxy=self.proxy)
-        self.assertIsNotNone(v)
-        self.assertTrue(len(v) > 0)
-
-        # Below will fail because not ported to Yahoo API
-
-        # v = dat.stats(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertTrue(len(v) > 0)
-
-        # v = dat.get_recommendations(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertFalse(v.empty)
-
-        # v = dat.get_calendar(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertFalse(v.empty)
-
-        # v = dat.get_sustainability(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertFalse(v.empty)
-
-        # v = dat.get_recommendations_summary(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertFalse(v.empty)
-
-        # v = dat.get_analyst_price_target(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertFalse(v.empty)
-
-        # v = dat.get_rev_forecast(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertFalse(v.empty)
-
-        # v = dat.get_earnings_forecast(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertFalse(v.empty)
-
-        # v = dat.get_trend_details(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertFalse(v.empty)
-
-        # v = dat.get_earnings_trend(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertFalse(v.empty)
-
-        # v = dat.get_earnings(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertFalse(v.empty)
-
-        # v = dat.get_shares(proxy=self.proxy)
-        # self.assertIsNotNone(v)
-        # self.assertFalse(v.empty)
+        for attribute_name, attribute_type in ticker_attributes:
+            assert_attribute_type(self, dat, attribute_name, attribute_type)
 
 
 class TestTickerHistory(unittest.TestCase):
@@ -338,76 +272,77 @@ class TestTickerHistory(unittest.TestCase):
         self.assertFalse(data.empty, "data is empty")
 
 
-# Below will fail because not ported to Yahoo API
-# class TestTickerEarnings(unittest.TestCase):
-#     session = None
+class TestTickerEarnings(unittest.TestCase):
+    session = None
 
-#     @classmethod
-#     def setUpClass(cls):
-#         cls.session = session_gbl
+    @classmethod
+    def setUpClass(cls):
+        cls.session = session_gbl
 
-#     @classmethod
-#     def tearDownClass(cls):
-#         if cls.session is not None:
-#             cls.session.close()
+    @classmethod
+    def tearDownClass(cls):
+        if cls.session is not None:
+            cls.session.close()
 
-#     def setUp(self):
-#         self.ticker = yf.Ticker("GOOGL", session=self.session)
+    def setUp(self):
+        self.ticker = yf.Ticker("GOOGL", session=self.session)
 
-#     def tearDown(self):
-#         self.ticker = None
+    def tearDown(self):
+        self.ticker = None
 
-#     def test_earnings(self):
-#         data = self.ticker.earnings
-#         self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
-#         self.assertFalse(data.empty, "data is empty")
+    def test_earnings_dates(self):
+        data = self.ticker.earnings_dates
+        self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+        self.assertFalse(data.empty, "data is empty")
 
-#         data_cached = self.ticker.earnings
-#         self.assertIs(data, data_cached, "data not cached")
+    def test_earnings_dates_with_limit(self):
+        # use ticker with lots of historic earnings
+        ticker = yf.Ticker("IBM")
+        limit = 110
+        data = ticker.get_earnings_dates(limit=limit)
+        self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+        self.assertFalse(data.empty, "data is empty")
+        self.assertEqual(len(data), limit, "Wrong number or rows")
 
-#     def test_quarterly_earnings(self):
-#         data = self.ticker.quarterly_earnings
-#         self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
-#         self.assertFalse(data.empty, "data is empty")
+        data_cached = ticker.get_earnings_dates(limit=limit)
+        self.assertIs(data, data_cached, "data not cached")
 
-#         data_cached = self.ticker.quarterly_earnings
-#         self.assertIs(data, data_cached, "data not cached")
+    # Below will fail because not ported to Yahoo API
 
-#     def test_earnings_forecasts(self):
-#         data = self.ticker.earnings_forecasts
-#         self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
-#         self.assertFalse(data.empty, "data is empty")
+    # def test_earnings(self):
+    #     data = self.ticker.earnings
+    #     self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+    #     self.assertFalse(data.empty, "data is empty")
 
-#         data_cached = self.ticker.earnings_forecasts
-#         self.assertIs(data, data_cached, "data not cached")
+    #     data_cached = self.ticker.earnings
+    #     self.assertIs(data, data_cached, "data not cached")
 
-#     def test_earnings_dates(self):
-#         data = self.ticker.earnings_dates
-#         self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
-#         self.assertFalse(data.empty, "data is empty")
+    # def test_quarterly_earnings(self):
+    #     data = self.ticker.quarterly_earnings
+    #     self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+    #     self.assertFalse(data.empty, "data is empty")
 
-#         data_cached = self.ticker.earnings_dates
-#         self.assertIs(data, data_cached, "data not cached")
+    #     data_cached = self.ticker.quarterly_earnings
+    #     self.assertIs(data, data_cached, "data not cached")
 
-#     def test_earnings_trend(self):
-#         data = self.ticker.earnings_trend
-#         self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
-#         self.assertFalse(data.empty, "data is empty")
+    # def test_earnings_forecasts(self):
+    #     data = self.ticker.earnings_forecasts
+    #     self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+    #     self.assertFalse(data.empty, "data is empty")
 
-#         data_cached = self.ticker.earnings_trend
-#         self.assertIs(data, data_cached, "data not cached")
+    #     data_cached = self.ticker.earnings_forecasts
+    #     self.assertIs(data, data_cached, "data not cached")
 
-#     def test_earnings_dates_with_limit(self):
-#         # use ticker with lots of historic earnings
-#         ticker = yf.Ticker("IBM")
-#         limit = 110
-#         data = ticker.get_earnings_dates(limit=limit)
-#         self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
-#         self.assertFalse(data.empty, "data is empty")
-#         self.assertEqual(len(data), limit, "Wrong number or rows")
+    #     data_cached = self.ticker.earnings_dates
+    #     self.assertIs(data, data_cached, "data not cached")
 
-#         data_cached = ticker.get_earnings_dates(limit=limit)
-#         self.assertIs(data, data_cached, "data not cached")
+    # def test_earnings_trend(self):
+    #     data = self.ticker.earnings_trend
+    #     self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+    #     self.assertFalse(data.empty, "data is empty")
+
+    #     data_cached = self.ticker.earnings_trend
+    #     self.assertIs(data, data_cached, "data not cached")
 
 
 class TestTickerHolders(unittest.TestCase):
@@ -452,6 +387,30 @@ class TestTickerHolders(unittest.TestCase):
         data_cached = self.ticker.mutualfund_holders
         self.assertIs(data, data_cached, "data not cached")
 
+    def test_insider_transactions(self):
+        data = self.ticker.insider_transactions
+        self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+        self.assertFalse(data.empty, "data is empty")
+
+        data_cached = self.ticker.insider_transactions
+        self.assertIs(data, data_cached, "data not cached")
+
+    def test_insider_purchases(self):
+        data = self.ticker.insider_purchases
+        self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+        self.assertFalse(data.empty, "data is empty")
+
+        data_cached = self.ticker.insider_purchases
+        self.assertIs(data, data_cached, "data not cached")
+
+    def test_insider_roster_holders(self):
+        data = self.ticker.insider_roster_holders
+        self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+        self.assertFalse(data.empty, "data is empty")
+
+        data_cached = self.ticker.insider_roster_holders
+        self.assertIs(data, data_cached, "data not cached")
+
 
 class TestTickerMiscFinancials(unittest.TestCase):
     session = None
@@ -467,9 +426,9 @@ class TestTickerMiscFinancials(unittest.TestCase):
 
     def setUp(self):
         self.ticker = yf.Ticker("GOOGL", session=self.session)
-        
-        # For ticker 'BSE.AX' (and others), Yahoo not returning 
-        # full quarterly financials (usually cash-flow) with all entries, 
+
+        # For ticker 'BSE.AX' (and others), Yahoo not returning
+        # full quarterly financials (usually cash-flow) with all entries,
         # instead returns a smaller version in different data store.
         self.ticker_old_fmt = yf.Ticker("BSE.AX", session=self.session)
 
@@ -730,6 +689,24 @@ class TestTickerMiscFinancials(unittest.TestCase):
     def test_bad_freq_value_raises_exception(self):
         self.assertRaises(ValueError, lambda: self.ticker.get_cashflow(freq="badarg"))
 
+    def test_calendar(self):
+        data = self.ticker.calendar
+        self.assertIsInstance(data, dict, "data has wrong type")
+        self.assertTrue(len(data) > 0, "data is empty")
+        self.assertIn("Earnings Date", data.keys(), "data missing expected key")
+        self.assertIn("Earnings Average", data.keys(), "data missing expected key")
+        self.assertIn("Earnings Low", data.keys(), "data missing expected key")
+        self.assertIn("Earnings High", data.keys(), "data missing expected key")
+        self.assertIn("Revenue Average", data.keys(), "data missing expected key")
+        self.assertIn("Revenue Low", data.keys(), "data missing expected key")
+        self.assertIn("Revenue High", data.keys(), "data missing expected key")
+        # dividend date is not available for tested ticker GOOGL
+        if self.ticker.ticker != "GOOGL":
+            self.assertIn("Dividend Date", data.keys(), "data missing expected key")
+        # ex-dividend date is not always available
+        data_cached = self.ticker.calendar
+        self.assertIs(data, data_cached, "data not cached")
+
     # Below will fail because not ported to Yahoo API
 
     # def test_sustainability(self):
@@ -740,21 +717,60 @@ class TestTickerMiscFinancials(unittest.TestCase):
     #     data_cached = self.ticker.sustainability
     #     self.assertIs(data, data_cached, "data not cached")
 
-    # def test_recommendations(self):
-    #     data = self.ticker.recommendations
+    # def test_shares(self):
+    #     data = self.ticker.shares
     #     self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
     #     self.assertFalse(data.empty, "data is empty")
 
-    #     data_cached = self.ticker.recommendations
-    #     self.assertIs(data, data_cached, "data not cached")
 
-    # def test_recommendations_summary(self):
-    #     data = self.ticker.recommendations_summary
-    #     self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
-    #     self.assertFalse(data.empty, "data is empty")
+class TestTickerAnalysts(unittest.TestCase):
+    session = None
 
-    #     data_cached = self.ticker.recommendations_summary
-    #     self.assertIs(data, data_cached, "data not cached")
+    @classmethod
+    def setUpClass(cls):
+        cls.session = session_gbl
+
+    @classmethod
+    def tearDownClass(cls):
+        if cls.session is not None:
+            cls.session.close()
+
+    def setUp(self):
+        self.ticker = yf.Ticker("GOOGL", session=self.session)
+
+    def tearDown(self):
+        self.ticker = None
+
+    def test_recommendations(self):
+        data = self.ticker.recommendations
+        data_summary = self.ticker.recommendations_summary
+        self.assertTrue(data.equals(data_summary))
+        self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+        self.assertFalse(data.empty, "data is empty")
+
+        data_cached = self.ticker.recommendations
+        self.assertIs(data, data_cached, "data not cached")
+
+    def test_recommendations_summary(self):  # currently alias for recommendations
+        data = self.ticker.recommendations_summary
+        self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+        self.assertFalse(data.empty, "data is empty")
+
+        data_cached = self.ticker.recommendations_summary
+        self.assertIs(data, data_cached, "data not cached")
+
+    def test_upgrades_downgrades(self):
+        data = self.ticker.upgrades_downgrades
+        self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
+        self.assertFalse(data.empty, "data is empty")
+        self.assertTrue(len(data.columns) == 4, "data has wrong number of columns")
+        self.assertEqual(data.columns.values.tolist(), ['Firm', 'ToGrade', 'FromGrade', 'Action'], "data has wrong column names")
+        self.assertIsInstance(data.index, pd.DatetimeIndex, "data has wrong index type")
+
+        data_cached = self.ticker.upgrades_downgrades
+        self.assertIs(data, data_cached, "data not cached")
+
+    # Below will fail because not ported to Yahoo API
 
     # def test_analyst_price_target(self):
     #     data = self.ticker.analyst_price_target
@@ -772,18 +788,6 @@ class TestTickerMiscFinancials(unittest.TestCase):
     #     data_cached = self.ticker.revenue_forecasts
     #     self.assertIs(data, data_cached, "data not cached")
 
-    # def test_calendar(self):
-    #     data = self.ticker.calendar
-    #     self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
-    #     self.assertFalse(data.empty, "data is empty")
-
-    #     data_cached = self.ticker.calendar
-    #     self.assertIs(data, data_cached, "data not cached")
-
-    # def test_shares(self):
-    #     data = self.ticker.shares
-    #     self.assertIsInstance(data, pd.DataFrame, "data has wrong type")
-    #     self.assertFalse(data.empty, "data is empty")
 
 
 class TestTickerInfo(unittest.TestCase):
@@ -823,6 +827,17 @@ class TestTickerInfo(unittest.TestCase):
             self.assertIn("symbol", data.keys(), f"Did not find expected key '{k}' in info dict")
         self.assertEqual(self.symbols[0], data["symbol"], "Wrong symbol value in info dict")
 
+    def test_complementary_info(self):
+        # This test is to check that we can successfully retrieve the trailing PEG ratio
+
+        # We don't expect this one to have a trailing PEG ratio
+        data1 = self.tickers[0].info
+        self.assertIsNone(data1['trailingPegRatio'])
+
+        # This one should have a trailing PEG ratio
+        data2 = self.tickers[2].info
+        self.assertIsInstance(data2['trailingPegRatio'], float)
+
     # def test_fast_info_matches_info(self):
     #     fast_info_keys = set()
     #     for ticker in self.tickers:
@@ -859,7 +874,7 @@ class TestTickerInfo(unittest.TestCase):
     #             key_rename_map[yf.utils.snake_case_2_camelCase(k)] = key_rename_map[k]
 
     #     # Note: share count items in info[] are bad. Sometimes the float > outstanding!
-    #     # So often fast_info["shares"] does not match. 
+    #     # So often fast_info["shares"] does not match.
     #     # Why isn't fast_info["shares"] wrong? Because using it to calculate market cap always correct.
     #     bad_keys = {"shares"}
 

tests/test_utils.py

@@ -8,11 +8,12 @@ Specific test class:
    python -m unittest tests.utils.TestTicker
 
 """
+from unittest import TestSuite
+
 # import pandas as pd
 # import numpy as np
 
 from .context import yfinance as yf
-from .context import session_gbl
 
 import unittest
 # import requests_cache
@@ -35,16 +36,16 @@ class TestCache(unittest.TestCase):
         tkr = 'AMZN'
         tz1 = "America/New_York"
         tz2 = "London/Europe"
-        cache = yf.utils.get_tz_cache()
+        cache = yf.cache.get_tz_cache()
         cache.store(tkr, tz1)
         cache.store(tkr, tz2)
 
     def test_setTzCacheLocation(self):
-        self.assertEqual(yf.utils._DBManager.get_location(), self.tempCacheDir.name)
+        self.assertEqual(yf.cache._TzDBManager.get_location(), self.tempCacheDir.name)
 
         tkr = 'AMZN'
         tz1 = "America/New_York"
-        cache = yf.utils.get_tz_cache()
+        cache = yf.cache.get_tz_cache()
         cache.store(tkr, tz1)
 
         self.assertTrue(os.path.exists(os.path.join(self.tempCacheDir.name, "tkr-tz.db")))
@@ -61,10 +62,10 @@ class TestCacheNoPermission(unittest.TestCase):
         tz1 = "America/New_York"
 
         # During attempt to store, will discover cannot write
-        yf.utils.get_tz_cache().store(tkr, tz1)
+        yf.cache.get_tz_cache().store(tkr, tz1)
 
         # Handling the store failure replaces cache with a dummy
-        cache = yf.utils.get_tz_cache()
+        cache = yf.cache.get_tz_cache()
         self.assertTrue(cache.dummy)
         cache.store(tkr, tz1)
 
@@ -72,19 +73,19 @@ class TestCacheNoPermission(unittest.TestCase):
         # Test that if cache path in read-only filesystem, no exception.
         tkr = 'AMZN'
         # During attempt to lookup, will discover cannot write
-        yf.utils.get_tz_cache().lookup(tkr)
+        yf.cache.get_tz_cache().lookup(tkr)
 
         # Handling the lookup failure replaces cache with a dummy
-        cache = yf.utils.get_tz_cache()
+        cache = yf.cache.get_tz_cache()
         self.assertTrue(cache.dummy)
         cache.lookup(tkr)
 
 
 def suite():
-    suite = unittest.TestSuite()
-    suite.addTest(TestCache('Test cache'))
-    suite.addTest(TestCacheNoPermission('Test cache no permission'))
-    return suite
+    ts: TestSuite = unittest.TestSuite()
+    ts.addTest(TestCache('Test cache'))
+    ts.addTest(TestCacheNoPermission('Test cache no permission'))
+    return ts
 
 
 if __name__ == '__main__':

yfinance/__init__.py

@@ -35,6 +35,8 @@ def pdr_override():
     make pandas datareader optional
     otherwise can be called via fix_yahoo_finance.download(...)
     """
+    from .utils import print_once
+    print_once("yfinance: pandas_datareader support is deprecated & semi-broken so will be removed in a future verison. Just use yfinance.")
     try:
         import pandas_datareader
         pandas_datareader.data.get_data_yahoo = download

yfinance/cache.py

@@ -1,7 +1,7 @@
 import peewee as _peewee
 from threading import Lock
 import os as _os
-import appdirs as _ad
+import platformdirs as _ad
 import atexit as _atexit
 import datetime as _datetime
 import pickle as _pkl
@@ -145,7 +145,14 @@ class _TzCache:
 
         db.connect()
         tz_db_proxy.initialize(db)
-        db.create_tables([_KV])
+        try:
+            db.create_tables([_KV])
+        except _peewee.OperationalError as e:
+            if 'WITHOUT' in str(e):
+                _KV._meta.without_rowid = False
+                db.create_tables([_KV])
+            else:
+                raise
         self.initialised = 1  # success
 
     def lookup(self, key):
@@ -197,17 +204,6 @@ def get_tz_cache():
     return _TzCacheManager.get_tz_cache()
 
 
-def set_tz_cache_location(cache_dir: str):
-    """
-    Sets the path to create the "py-yfinance" cache folder in.
-    Useful if the default folder returned by "appdir.user_cache_dir()" is not writable.
-    Must be called before cache is used (that is, before fetching tickers).
-    :param cache_dir: Path to use for caches
-    :return: None
-    """
-    _TzDBManager.set_location(cache_dir)
-
-
 
 # --------------
 # Cookie cache
@@ -300,9 +296,21 @@ _atexit.register(_CookieDBManager.close_db)
 
 
 Cookie_db_proxy = _peewee.Proxy()
+class ISODateTimeField(_peewee.DateTimeField):
+    # Ensure Python datetime is read & written correctly for sqlite, 
+    # because user discovered peewee allowed an invalid datetime
+    # to get written.
+    def db_value(self, value):
+        if value and isinstance(value, _datetime.datetime):
+            return value.isoformat()
+        return super().db_value(value)
+    def python_value(self, value):
+        if value and isinstance(value, str) and 'T' in value:
+            return _datetime.datetime.fromisoformat(value)
+        return super().python_value(value)
 class _CookieSchema(_peewee.Model):
     strategy = _peewee.CharField(primary_key=True)
-    fetch_date = _peewee.DateTimeField(default=_datetime.datetime.now)
+    fetch_date = ISODateTimeField(default=_datetime.datetime.now)
     
     # Which cookie type depends on strategy
     cookie_bytes = _peewee.BlobField()
@@ -343,7 +351,14 @@ class _CookieCache:
 
         db.connect()
         Cookie_db_proxy.initialize(db)
-        db.create_tables([_CookieSchema])
+        try:
+            db.create_tables([_CookieSchema])
+        except _peewee.OperationalError as e:
+            if 'WITHOUT' in str(e):
+                _CookieSchema._meta.without_rowid = False
+                db.create_tables([_CookieSchema])
+            else:
+                raise
         self.initialised = 1  # success
 
     def lookup(self, strategy):
@@ -398,3 +413,19 @@ class _CookieCache:
 def get_cookie_cache():
     return _CookieCacheManager.get_cookie_cache()
 
+
+
+def set_cache_location(cache_dir: str):
+    """
+    Sets the path to create the "py-yfinance" cache folder in.
+    Useful if the default folder returned by "appdir.user_cache_dir()" is not writable.
+    Must be called before cache is used (that is, before fetching tickers).
+    :param cache_dir: Path to use for caches
+    :return: None
+    """
+    _TzDBManager.set_location(cache_dir)
+    _CookieDBManager.set_location(cache_dir)
+
+def set_tz_cache_location(cache_dir: str):
+    set_cache_location(cache_dir)
+

yfinance/const.py

@@ -115,4 +115,40 @@ fundamentals_keys = {
                   "PaymentstoSuppliersforGoodsandServices", "ClassesofCashReceiptsfromOperatingActivities",
                   "OtherCashReceiptsfromOperatingActivities", "ReceiptsfromGovernmentGrants", "ReceiptsfromCustomers"]}
 
-price_colnames = ['Open', 'High', 'Low', 'Close', 'Adj Close']
+_PRICE_COLNAMES_ = ['Open', 'High', 'Low', 'Close', 'Adj Close']
+
+quote_summary_valid_modules = (
+    "summaryProfile",  # contains general information about the company
+    "summaryDetail",  # prices + volume + market cap + etc
+    "assetProfile",  # summaryProfile + company officers
+    "fundProfile",
+    "price",  # current prices
+    "quoteType",  # quoteType
+    "esgScores",  # Environmental, social, and governance (ESG) scores, sustainability and ethical performance of companies
+    "incomeStatementHistory",
+    "incomeStatementHistoryQuarterly",
+    "balanceSheetHistory",
+    "balanceSheetHistoryQuarterly",
+    "cashFlowStatementHistory",
+    "cashFlowStatementHistoryQuarterly",
+    "defaultKeyStatistics",  # KPIs (PE, enterprise value, EPS, EBITA, and more)
+    "financialData",  # Financial KPIs (revenue, gross margins, operating cash flow, free cash flow, and more)
+    "calendarEvents",  # future earnings date
+    "secFilings",  # SEC filings, such as 10K and 10Q reports
+    "upgradeDowngradeHistory",  # upgrades and downgrades that analysts have given a company's stock
+    "institutionOwnership",  # institutional ownership, holders and shares outstanding
+    "fundOwnership",  # mutual fund ownership, holders and shares outstanding
+    "majorDirectHolders",
+    "majorHoldersBreakdown",
+    "insiderTransactions",  # insider transactions, such as the number of shares bought and sold by company executives
+    "insiderHolders",  # insider holders, such as the number of shares held by company executives
+    "netSharePurchaseActivity",  # net share purchase activity, such as the number of shares bought and sold by company executives
+    "earnings",  # earnings history
+    "earningsHistory",
+    "earningsTrend",  # earnings trend
+    "industryTrend",
+    "indexTrend",
+    "sectorTrend",
+    "recommendationTrend",
+    "futuresChain",
+)

yfinance/data.py

@@ -8,6 +8,7 @@ import datetime
 from frozendict import frozendict
 
 from . import utils, cache
+import threading
 
 cache_maxsize = 64
 
@@ -33,7 +34,6 @@ def lru_cache_freezeargs(func):
     return wrapped
 
 
-import threading
 class SingletonMeta(type):
     """
     Metaclass that creates a Singleton instance.
@@ -53,33 +53,15 @@ class SingletonMeta(type):
 
 class YfData(metaclass=SingletonMeta):
     """
-    Have one place to retrieve data from Yahoo API in order to ease caching and speed up operations
-
-    Credit for code for cookie & crumb goes to StackOverflow:
-    https://stackoverflow.com/questions/76065035/yahoo-finance-v7-api-now-requiring-cookies-python
+    Have one place to retrieve data from Yahoo API in order to ease caching and speed up operations.
+    Singleton means one session one cookie shared by all threads.
     """
     user_agent_headers = {
         'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}
 
     def __init__(self, session=None):
-        self._session = session or requests.Session()
-
-        try:
-            self._session.cache
-        except AttributeError:
-            # Not caching
-            self._session_is_caching = False
-        else:
-            # Is caching. This is annoying. 
-            # Can't simply use a non-caching session to fetch cookie & crumb, 
-            # because then the caching-session won't have cookie.
-            self._session_is_caching = True
-            from requests_cache import DO_NOT_CACHE
-            self._expire_after = DO_NOT_CACHE
         self._crumb = None
         self._cookie = None
-        if self._session_is_caching and self._cookie is None:
-            utils.print_once("WARNING: cookie & crumb does not work well with requests_cache. Am experimenting with 'expire_after=DO_NOT_CACHE', but you need to help stress-test.")
 
         # Default to using 'basic' strategy
         self._cookie_strategy = 'basic'
@@ -88,12 +70,27 @@ class YfData(metaclass=SingletonMeta):
 
         self._cookie_lock = threading.Lock()
 
+        self._set_session(session or requests.Session())
+
     def _set_session(self, session):
         if session is None:
             return
         with self._cookie_lock:
             self._session = session
 
+        try:
+            self._session.cache
+        except AttributeError:
+            # Not caching
+            self._session_is_caching = False
+        else:
+            # Is caching. This is annoying.
+            # Can't simply use a non-caching session to fetch cookie & crumb,
+            # because then the caching-session won't have cookie.
+            self._session_is_caching = True
+            from requests_cache import DO_NOT_CACHE
+            self._expire_after = DO_NOT_CACHE
+
     def _set_cookie_strategy(self, strategy, have_lock=False):
         if strategy == self._cookie_strategy:
             return
@@ -101,7 +98,6 @@ class YfData(metaclass=SingletonMeta):
             self._cookie_lock.acquire()
 
         try:
-            self._cookie_strategy = strategy
             if self._cookie_strategy == 'csrf':
                 utils.get_yf_logger().debug(f'toggling cookie strategy {self._cookie_strategy} -> basic')
                 self._session.cookies.clear()
@@ -205,11 +201,12 @@ class YfData(metaclass=SingletonMeta):
             crumb_response = self._session.get(**get_args)
         self._crumb = crumb_response.text
         if self._crumb is None or '<html>' in self._crumb:
+            utils.get_yf_logger().debug("Didn't receive crumb")
             return None
 
         utils.get_yf_logger().debug(f"crumb = '{self._crumb}'")
         return self._crumb
-    
+
     @utils.log_indent_decorator
     def _get_cookie_and_crumb_basic(self, proxy, timeout):
         cookie = self._get_cookie_basic(proxy, timeout)
@@ -217,7 +214,7 @@ class YfData(metaclass=SingletonMeta):
         return cookie, crumb
 
     def _get_cookie_csrf(self, proxy, timeout):
-        if utils.reuse_cookie and self._cookie is not None:
+        if self._cookie is not None:
             utils.get_yf_logger().debug('reusing cookie')
             return True
 
@@ -231,7 +228,7 @@ class YfData(metaclass=SingletonMeta):
             'proxies': proxy,
             'timeout': timeout}
 
-        get_args = base_args | {'url': 'https://guce.yahoo.com/consent'}
+        get_args = {**base_args, 'url': 'https://guce.yahoo.com/consent'}
         if self._session_is_caching:
             get_args['expire_after'] = self._expire_after
             response = self._session.get(**get_args)
@@ -241,6 +238,7 @@ class YfData(metaclass=SingletonMeta):
         soup = BeautifulSoup(response.content, 'html.parser')
         csrfTokenInput = soup.find('input', attrs={'name': 'csrfToken'})
         if csrfTokenInput is None:
+            utils.get_yf_logger().debug('Failed to find "csrfToken" in response')
             return False
         csrfToken = csrfTokenInput['value']
         utils.get_yf_logger().debug(f'csrfToken = {csrfToken}')
@@ -258,10 +256,10 @@ class YfData(metaclass=SingletonMeta):
             'originalDoneUrl': originalDoneUrl,
             'namespace': namespace,
         }
-        post_args = base_args | {
+        post_args = {**base_args,
             'url': f'https://consent.yahoo.com/v2/collectConsent?sessionId={sessionId}',
             'data': data}
-        get_args = base_args | {
+        get_args = {**base_args,
             'url': f'https://guce.yahoo.com/copyConsent?sessionId={sessionId}',
             'data': data}
         if self._session_is_caching:
@@ -289,7 +287,7 @@ class YfData(metaclass=SingletonMeta):
             return None
 
         get_args = {
-            'url': 'https://query2.finance.yahoo.com/v1/test/getcrumb', 
+            'url': 'https://query2.finance.yahoo.com/v1/test/getcrumb',
             'headers': self.user_agent_headers,
             'proxies': proxy,
             'timeout': timeout}
@@ -301,6 +299,7 @@ class YfData(metaclass=SingletonMeta):
         self._crumb = r.text
 
         if self._crumb is None or '<html>' in self._crumb or self._crumb == '':
+            utils.get_yf_logger().debug("Didn't receive crumb")
             return None
 
         utils.get_yf_logger().debug(f"crumb = '{self._crumb}'")
@@ -358,13 +357,14 @@ class YfData(metaclass=SingletonMeta):
 
         request_args = {
             'url': url,
-            'params': params | crumbs,
+            'params': {**params, **crumbs},
             'cookies': cookies,
             'proxies': proxy,
             'timeout': timeout,
             'headers': user_agent_headers or self.user_agent_headers
         }
         response = self._session.get(**request_args)
+        utils.get_yf_logger().debug(f'response code={response.status_code}')
         if response.status_code >= 400:
             # Retry with other cookie strategy
             if strategy == 'basic':
@@ -376,6 +376,7 @@ class YfData(metaclass=SingletonMeta):
             if strategy == 'basic':
                 request_args['cookies'] = {cookie.name: cookie.value}
             response = self._session.get(**request_args)
+            utils.get_yf_logger().debug(f'response code={response.status_code}')
 
         return response
 

yfinance/exceptions.py

@@ -1,12 +1,50 @@
-class YFinanceException(Exception):
+class YFException(Exception):
+    def __init__(self, description=""):
+        super().__init__(description)
+
+
+class YFDataException(YFException):
     pass
 
 
-class YFinanceDataException(YFinanceException):
-    pass
+class YFChartError(YFException):
+    def __init__(self, ticker, description):
+        self.ticker = ticker
+        super().__init__(f"{self.ticker}: {description}")
 
 
 class YFNotImplementedError(NotImplementedError):
     def __init__(self, method_name):
         super().__init__(f"Have not implemented fetching '{method_name}' from Yahoo API")
 
+
+class YFTickerMissingError(YFException):
+    def __init__(self, ticker, rationale):
+        super().__init__(f"${ticker}: possibly delisted; {rationale}")
+        self.rationale = rationale
+        self.ticker = ticker
+
+
+class YFTzMissingError(YFTickerMissingError):
+    def __init__(self, ticker):
+        super().__init__(ticker, "No timezone found")
+
+
+class YFPricesMissingError(YFTickerMissingError):
+    def __init__(self, ticker, debug_info):
+        self.debug_info = debug_info
+        super().__init__(ticker, f"No price data found {debug_info}")
+
+
+class YFEarningsDateMissing(YFTickerMissingError):
+    # note that this does not get raised. Added in case of raising it in the future
+    def __init__(self, ticker):
+        super().__init__(ticker, "No earnings dates found")
+
+
+class YFInvalidPeriodError(YFException):
+    def __init__(self, ticker, invalid_period, valid_ranges):
+        self.ticker = ticker
+        self.invalid_period = invalid_period
+        self.valid_ranges = valid_ranges
+        super().__init__(f"{self.ticker}: Period '{invalid_period}' is invalid, must be one of {valid_ranges}")

yfinance/multi.py

@@ -36,7 +36,7 @@ from . import shared
 @utils.log_indent_decorator
 def download(tickers, start=None, end=None, actions=False, threads=True, ignore_tz=None,
              group_by='column', auto_adjust=False, back_adjust=False, repair=False, keepna=False,
-             progress=True, period="max", show_errors=None, interval="1d", prepost=False,
+             progress=True, period="max", interval="1d", prepost=False,
              proxy=None, rounding=False, timeout=10, session=None):
     """Download yahoo tickers
     :Parameters:
@@ -80,9 +80,6 @@ def download(tickers, start=None, end=None, actions=False, threads=True, ignore_
             Optional. Proxy server URL scheme. Default is None
         rounding: bool
             Optional. Round values to 2 decimal places?
-        show_errors: bool
-            Optional. Doesn't print errors if False
-            DEPRECATED, will be removed in future version
         timeout: None or float
             If not None stops waiting for a response after given number of
             seconds. (Can also be a fraction of a second e.g. 0.01)
@@ -91,14 +88,6 @@ def download(tickers, start=None, end=None, actions=False, threads=True, ignore_
     """
     logger = utils.get_yf_logger()
 
-    if show_errors is not None:
-        if show_errors:
-            utils.print_once(f"yfinance: download(show_errors={show_errors}) argument is deprecated and will be removed in future version. Do this instead: logging.getLogger('yfinance').setLevel(logging.ERROR)")
-            logger.setLevel(logging.ERROR)
-        else:
-            utils.print_once(f"yfinance: download(show_errors={show_errors}) argument is deprecated and will be removed in future version. Do this instead to suppress error messages: logging.getLogger('yfinance').setLevel(logging.CRITICAL)")
-            logger.setLevel(logging.CRITICAL)
-
     if logger.isEnabledFor(logging.DEBUG):
         if threads:
             # With DEBUG, each thread generates a lot of log messages.
@@ -172,7 +161,7 @@ def download(tickers, start=None, end=None, actions=False, threads=True, ignore_
                                  rounding=rounding, timeout=timeout)
             if progress:
                 shared._PROGRESS_BAR.animate()
-    
+
     if progress:
         shared._PROGRESS_BAR.completed()
 
@@ -217,12 +206,12 @@ def download(tickers, start=None, end=None, actions=False, threads=True, ignore_
 
     try:
         data = _pd.concat(shared._DFS.values(), axis=1, sort=True,
-                          keys=shared._DFS.keys())
+                          keys=shared._DFS.keys(), names=['Ticker', 'Price'])
     except Exception:
         _realign_dfs()
         data = _pd.concat(shared._DFS.values(), axis=1, sort=True,
-                          keys=shared._DFS.keys())
-
+                          keys=shared._DFS.keys(), names=['Ticker', 'Price'])
+    data.index = _pd.to_datetime(data.index)
     # switch names back to isins if applicable
     data.rename(columns=shared._ISINS, inplace=True)
 
@@ -262,7 +251,7 @@ def _download_one_threaded(ticker, start=None, end=None,
                            actions=False, progress=True, period="max",
                            interval="1d", prepost=False, proxy=None,
                            keepna=False, rounding=False, timeout=10):
-    data = _download_one(ticker, start, end, auto_adjust, back_adjust, repair,
+    _download_one(ticker, start, end, auto_adjust, back_adjust, repair,
                          actions, period, interval, prepost, proxy, rounding,
                          keepna, timeout)
     if progress:

yfinance/run.stdout

@@ -1 +0,0 @@
-python: can't open file '/home/gonzo/ReposExternal/yfinance.feature.cookie-and-crumb/yfinance/test-single.py': [Errno 2] No such file or directory

yfinance/scrapers/analysis.py

@@ -1,6 +1,5 @@
 import pandas as pd
 
-from yfinance import utils
 from yfinance.data import YfData
 from yfinance.exceptions import YFNotImplementedError
 

yfinance/scrapers/fundamentals.py

@@ -5,7 +5,7 @@ import pandas as pd
 
 from yfinance import utils, const
 from yfinance.data import YfData
-from yfinance.exceptions import YFinanceException, YFNotImplementedError
+from yfinance.exceptions import YFException, YFNotImplementedError
 
 
 class Fundamentals:
@@ -70,7 +70,7 @@ class Financials:
     @utils.log_indent_decorator
     def _fetch_time_series(self, name, timescale, proxy=None):
         # Fetching time series preferred over scraping 'QuoteSummaryStore',
-        # because it matches what Yahoo shows. But for some tickers returns nothing, 
+        # because it matches what Yahoo shows. But for some tickers returns nothing,
         # despite 'QuoteSummaryStore' containing valid data.
 
         allowed_names = ["income", "balance-sheet", "cash-flow"]
@@ -79,14 +79,14 @@ class Financials:
         if name not in allowed_names:
             raise ValueError(f"Illegal argument: name must be one of: {allowed_names}")
         if timescale not in allowed_timescales:
-            raise ValueError(f"Illegal argument: timescale must be one of: {allowed_names}")
+            raise ValueError(f"Illegal argument: timescale must be one of: {allowed_timescales}")
 
         try:
             statement = self._create_financials_table(name, timescale, proxy)
 
             if statement is not None:
                 return statement
-        except YFinanceException as e:
+        except YFException as e:
             utils.get_yf_logger().error(f"{self._symbol}: Failed to create {name} financials table for reason: {e}")
         return pd.DataFrame()
 
@@ -99,7 +99,7 @@ class Financials:
 
         try:
             return self.get_financials_time_series(timescale, keys, proxy)
-        except Exception as e:
+        except Exception:
             pass
 
     def get_financials_time_series(self, timescale, keys: list, proxy=None) -> pd.DataFrame:

yfinance/scrapers/holders.py

@@ -1,6 +1,14 @@
-import pandas as pd
+# from io import StringIO
 
+import pandas as pd
+import requests
+
+from yfinance import utils
 from yfinance.data import YfData
+from yfinance.const import _BASE_URL_
+from yfinance.exceptions import YFDataException
+
+_QUOTE_SUMMARY_URL_ = f"{_BASE_URL_}/v10/finance/quoteSummary/"
 
 
 class Holders:
@@ -12,57 +20,225 @@ class Holders:
         self.proxy = proxy
 
         self._major = None
+        self._major_direct_holders = None
         self._institutional = None
         self._mutualfund = None
 
+        self._insider_transactions = None
+        self._insider_purchases = None
+        self._insider_roster = None
+
     @property
     def major(self) -> pd.DataFrame:
         if self._major is None:
-            self._scrape(self.proxy)
+            # self._scrape(self.proxy)
+            self._fetch_and_parse()
         return self._major
 
     @property
     def institutional(self) -> pd.DataFrame:
         if self._institutional is None:
-            self._scrape(self.proxy)
+            # self._scrape(self.proxy)
+            self._fetch_and_parse()
         return self._institutional
 
     @property
     def mutualfund(self) -> pd.DataFrame:
         if self._mutualfund is None:
-            self._scrape(self.proxy)
+            # self._scrape(self.proxy)
+            self._fetch_and_parse()
         return self._mutualfund
 
-    def _scrape(self, proxy):
-        ticker_url = f"{self._SCRAPE_URL_}/{self._symbol}"
+    @property
+    def insider_transactions(self) -> pd.DataFrame:
+        if self._insider_transactions is None:
+            # self._scrape_insider_transactions(self.proxy)
+            self._fetch_and_parse()
+        return self._insider_transactions
+
+    @property
+    def insider_purchases(self) -> pd.DataFrame:
+        if self._insider_purchases is None:
+            # self._scrape_insider_transactions(self.proxy)
+            self._fetch_and_parse()
+        return self._insider_purchases
+
+    @property
+    def insider_roster(self) -> pd.DataFrame:
+        if self._insider_roster is None:
+            # self._scrape_insider_ros(self.proxy)
+            self._fetch_and_parse()
+        return self._insider_roster
+
+    def _fetch(self, proxy):
+        modules = ','.join(
+            ["institutionOwnership", "fundOwnership", "majorDirectHolders", "majorHoldersBreakdown", "insiderTransactions", "insiderHolders", "netSharePurchaseActivity"])
+        params_dict = {"modules": modules, "corsDomain": "finance.yahoo.com", "formatted": "false"}
+        result = self._data.get_raw_json(f"{_QUOTE_SUMMARY_URL_}/{self._symbol}", user_agent_headers=self._data.user_agent_headers, params=params_dict, proxy=proxy)
+        return result
+
+    def _fetch_and_parse(self):
         try:
-            resp = self._data.cache_get(ticker_url + '/holders', proxy=proxy)
-            holders = pd.read_html(resp.text)
-        except Exception:
-            holders = []
+            result = self._fetch(self.proxy)
+        except requests.exceptions.HTTPError as e:
+            utils.get_yf_logger().error(str(e))
 
-        if len(holders) >= 3:
-            self._major = holders[0]
-            self._institutional = holders[1]
-            self._mutualfund = holders[2]
-        elif len(holders) >= 2:
-            self._major = holders[0]
-            self._institutional = holders[1]
-        elif len(holders) >= 1:
-            self._major = holders[0]
+            self._major = pd.DataFrame()
+            self._major_direct_holders = pd.DataFrame()
+            self._institutional = pd.DataFrame()
+            self._mutualfund = pd.DataFrame()
+            self._insider_transactions = pd.DataFrame()
+            self._insider_purchases = pd.DataFrame()
+            self._insider_roster = pd.DataFrame()
 
-        if self._institutional is not None:
-            if 'Date Reported' in self._institutional:
-                self._institutional['Date Reported'] = pd.to_datetime(
-                    self._institutional['Date Reported'])
-            if '% Out' in self._institutional:
-                self._institutional['% Out'] = self._institutional[
-                                                   '% Out'].str.replace('%', '').astype(float) / 100
+            return
 
-        if self._mutualfund is not None:
-            if 'Date Reported' in self._mutualfund:
-                self._mutualfund['Date Reported'] = pd.to_datetime(
-                    self._mutualfund['Date Reported'])
-            if '% Out' in self._mutualfund:
-                self._mutualfund['% Out'] = self._mutualfund[
-                                                '% Out'].str.replace('%', '').astype(float) / 100
+        try:
+            data = result["quoteSummary"]["result"][0]
+            # parse "institutionOwnership", "fundOwnership", "majorDirectHolders", "majorHoldersBreakdown", "insiderTransactions", "insiderHolders", "netSharePurchaseActivity"
+            self._parse_institution_ownership(data["institutionOwnership"])
+            self._parse_fund_ownership(data["fundOwnership"])
+            # self._parse_major_direct_holders(data["majorDirectHolders"])  # need more data to investigate
+            self._parse_major_holders_breakdown(data["majorHoldersBreakdown"])
+            self._parse_insider_transactions(data["insiderTransactions"])
+            self._parse_insider_holders(data["insiderHolders"])
+            self._parse_net_share_purchase_activity(data["netSharePurchaseActivity"])
+        except (KeyError, IndexError):
+            raise YFDataException("Failed to parse holders json data.")
+
+    @staticmethod
+    def _parse_raw_values(data):
+        if isinstance(data, dict) and "raw" in data:
+            return data["raw"]
+        return data
+
+    def _parse_institution_ownership(self, data):
+        holders = data["ownershipList"]
+        for owner in holders:
+            for k, v in owner.items():
+                owner[k] = self._parse_raw_values(v)
+            del owner["maxAge"]
+        df = pd.DataFrame(holders)
+        if not df.empty:
+            df["reportDate"] = pd.to_datetime(df["reportDate"], unit="s")
+            df.rename(columns={"reportDate": "Date Reported", "organization": "Holder", "position": "Shares", "value": "Value"}, inplace=True)  # "pctHeld": "% Out"
+        self._institutional = df
+
+    def _parse_fund_ownership(self, data):
+        holders = data["ownershipList"]
+        for owner in holders:
+            for k, v in owner.items():
+                owner[k] = self._parse_raw_values(v)
+            del owner["maxAge"]
+        df = pd.DataFrame(holders)
+        if not df.empty:
+            df["reportDate"] = pd.to_datetime(df["reportDate"], unit="s")
+            df.rename(columns={"reportDate": "Date Reported", "organization": "Holder", "position": "Shares", "value": "Value"}, inplace=True)
+        self._mutualfund = df
+
+    def _parse_major_direct_holders(self, data):
+        holders = data["holders"]
+        for owner in holders:
+            for k, v in owner.items():
+                owner[k] = self._parse_raw_values(v)
+            del owner["maxAge"]
+        df = pd.DataFrame(holders)
+        if not df.empty:
+            df["reportDate"] = pd.to_datetime(df["reportDate"], unit="s")
+            df.rename(columns={"reportDate": "Date Reported", "organization": "Holder", "positionDirect": "Shares", "valueDirect": "Value"}, inplace=True)
+        self._major_direct_holders = df
+
+    def _parse_major_holders_breakdown(self, data):
+        if "maxAge" in data:
+            del data["maxAge"]
+        df = pd.DataFrame.from_dict(data, orient="index")
+        if not df.empty:
+            df.columns.name = "Breakdown"
+            df.rename(columns={df.columns[0]: 'Value'}, inplace=True)
+        self._major = df
+
+    def _parse_insider_transactions(self, data):
+        holders = data["transactions"]
+        for owner in holders:
+            for k, v in owner.items():
+                owner[k] = self._parse_raw_values(v)
+            del owner["maxAge"]
+        df = pd.DataFrame(holders)
+        if not df.empty:
+            df["startDate"] = pd.to_datetime(df["startDate"], unit="s")
+            df.rename(columns={
+                "startDate": "Start Date",
+                "filerName": "Insider",
+                "filerRelation": "Position",
+                "filerUrl": "URL",
+                "moneyText": "Transaction",
+                "transactionText": "Text",
+                "shares": "Shares",
+                "value": "Value",
+                "ownership": "Ownership"  # ownership flag, direct or institutional
+            }, inplace=True)
+        self._insider_transactions = df
+
+    def _parse_insider_holders(self, data):
+        holders = data["holders"]
+        for owner in holders:
+            for k, v in owner.items():
+                owner[k] = self._parse_raw_values(v)
+            del owner["maxAge"]
+        df = pd.DataFrame(holders)
+        if not df.empty:
+            df["positionDirectDate"] = pd.to_datetime(df["positionDirectDate"], unit="s")
+            df["latestTransDate"] = pd.to_datetime(df["latestTransDate"], unit="s")
+
+            df.rename(columns={
+                "name": "Name",
+                "relation": "Position",
+                "url": "URL",
+                "transactionDescription": "Most Recent Transaction",
+                "latestTransDate": "Latest Transaction Date",
+                "positionDirectDate": "Position Direct Date",
+                "positionDirect": "Shares Owned Directly",
+                "positionIndirectDate": "Position Indirect Date",
+                "positionIndirect": "Shares Owned Indirectly"
+            }, inplace=True)
+
+            df["Name"] = df["Name"].astype(str)
+            df["Position"] = df["Position"].astype(str)
+            df["URL"] = df["URL"].astype(str)
+            df["Most Recent Transaction"] = df["Most Recent Transaction"].astype(str)
+
+        self._insider_roster = df
+
+    def _parse_net_share_purchase_activity(self, data):
+        df = pd.DataFrame(
+            {
+                "Insider Purchases Last " + data.get("period", ""): [
+                    "Purchases",
+                    "Sales",
+                    "Net Shares Purchased (Sold)",
+                    "Total Insider Shares Held",
+                    "% Net Shares Purchased (Sold)",
+                    "% Buy Shares",
+                    "% Sell Shares"
+                ],
+                "Shares": [
+                    data.get('buyInfoShares'),
+                    data.get('sellInfoShares'),
+                    data.get('netInfoShares'),
+                    data.get('totalInsiderShares'),
+                    data.get('netPercentInsiderShares'),
+                    data.get('buyPercentInsiderShares'),
+                    data.get('sellPercentInsiderShares')
+                ],
+                "Trans": [
+                    data.get('buyInfoCount'),
+                    data.get('sellInfoCount'),
+                    data.get('netInfoCount'),
+                    pd.NA,
+                    pd.NA,
+                    pd.NA,
+                    pd.NA
+                ]
+            }
+        ).convert_dtypes()
+        self._insider_purchases = df

yfinance/scrapers/quote.py

@@ -1,15 +1,16 @@
 import datetime
 import json
-import logging
 import warnings
 from collections.abc import MutableMapping
 
 import numpy as _np
 import pandas as pd
+import requests
 
 from yfinance import utils
 from yfinance.data import YfData
-from yfinance.exceptions import YFNotImplementedError
+from yfinance.const import quote_summary_valid_modules, _BASE_URL_
+from yfinance.exceptions import YFNotImplementedError, YFDataException, YFException
 
 info_retired_keys_price = {"currentPrice", "dayHigh", "dayLow", "open", "previousClose", "volume", "volume24Hr"}
 info_retired_keys_price.update({"regularMarket"+s for s in ["DayHigh", "DayLow", "Open", "PreviousClose", "Price", "Volume"]})
@@ -21,11 +22,11 @@ info_retired_keys_symbol = {"symbol"}
 info_retired_keys = info_retired_keys_price | info_retired_keys_exchange | info_retired_keys_marketCap | info_retired_keys_symbol
 
 
-_BASIC_URL_ = "https://query2.finance.yahoo.com/v10/finance/quoteSummary"
+_QUOTE_SUMMARY_URL_ = f"{_BASE_URL_}/v10/finance/quoteSummary"
 
 
 class InfoDictWrapper(MutableMapping):
-    """ Simple wrapper around info dict, intercepting 'gets' to 
+    """ Simple wrapper around info dict, intercepting 'gets' to
     print how-to-migrate messages for specific keys. Requires
     override dict API"""
 
@@ -67,7 +68,7 @@ class InfoDictWrapper(MutableMapping):
 
     def __iter__(self):
         return iter(self.info)
-    
+
     def __len__(self):
         return len(self.info)
 
@@ -125,7 +126,7 @@ class FastInfo:
         _properties += ["fifty_day_average", "two_hundred_day_average", "ten_day_average_volume", "three_month_average_volume"]
         _properties += ["year_high", "year_low", "year_change"]
 
-        # Because released before fixing key case, need to officially support 
+        # Because released before fixing key case, need to officially support
         # camel-case but also secretly support snake-case
         base_keys = [k for k in _properties if '_' not in k]
 
@@ -133,7 +134,7 @@ class FastInfo:
 
         self._sc_to_cc_key = {k: utils.snake_case_2_camelCase(k) for k in sc_keys}
         self._cc_to_sc_key = {v: k for k, v in self._sc_to_cc_key.items()}
- 
+
         self._public_keys = sorted(base_keys + list(self._sc_to_cc_key.values()))
         self._keys = sorted(self._public_keys + sc_keys)
 
@@ -156,7 +157,7 @@ class FastInfo:
 
     def __getitem__(self, k):
         if not isinstance(k, str):
-            raise KeyError(f"key must be a string")
+            raise KeyError("key must be a string")
         if k not in self._keys:
             raise KeyError(f"'{k}' not valid key. Examine 'FastInfo.keys()'")
         if k in self._cc_to_sc_key:
@@ -176,15 +177,11 @@ class FastInfo:
         return self.__str__()
 
     def toJSON(self, indent=4):
-        d = {k: self[k] for k in self.keys()}
         return json.dumps({k: self[k] for k in self.keys()}, indent=indent)
 
     def _get_1y_prices(self, fullDaysOnly=False):
         if self._prices_1y is None:
-            # Temporarily disable error printing
-            logging.disable(logging.CRITICAL)
-            self._prices_1y = self._tkr.history(period="380d", auto_adjust=False, keepna=True, proxy=self.proxy)
-            logging.disable(logging.NOTSET)
+            self._prices_1y = self._tkr.history(period="1y", auto_adjust=False, keepna=True, proxy=self.proxy)
             self._md = self._tkr.get_history_metadata(proxy=self.proxy)
             try:
                 ctp = self._md["currentTradingPeriod"]
@@ -210,18 +207,12 @@ class FastInfo:
 
     def _get_1wk_1h_prepost_prices(self):
         if self._prices_1wk_1h_prepost is None:
-            # Temporarily disable error printing
-            logging.disable(logging.CRITICAL)
-            self._prices_1wk_1h_prepost = self._tkr.history(period="1wk", interval="1h", auto_adjust=False, prepost=True, proxy=self.proxy)
-            logging.disable(logging.NOTSET)
+            self._prices_1wk_1h_prepost = self._tkr.history(period="5d", interval="1h", auto_adjust=False, prepost=True, proxy=self.proxy)
         return self._prices_1wk_1h_prepost
 
     def _get_1wk_1h_reg_prices(self):
         if self._prices_1wk_1h_reg is None:
-            # Temporarily disable error printing
-            logging.disable(logging.CRITICAL)
-            self._prices_1wk_1h_reg = self._tkr.history(period="1wk", interval="1h", auto_adjust=False, prepost=False, proxy=self.proxy)
-            logging.disable(logging.NOTSET)
+            self._prices_1wk_1h_reg = self._tkr.history(period="5d", interval="1h", auto_adjust=False, prepost=False, proxy=self.proxy)
         return self._prices_1wk_1h_reg
 
     def _get_exchange_metadata(self):
@@ -260,8 +251,6 @@ class FastInfo:
         if self._currency is not None:
             return self._currency
 
-        if self._tkr._history_metadata is None:
-            self._get_1y_prices()
         md = self._tkr.get_history_metadata(proxy=self.proxy)
         self._currency = md["currency"]
         return self._currency
@@ -271,8 +260,6 @@ class FastInfo:
         if self._quote_type is not None:
             return self._quote_type
 
-        if self._tkr._history_metadata is None:
-            self._get_1y_prices()
         md = self._tkr.get_history_metadata(proxy=self.proxy)
         self._quote_type = md["instrumentType"]
         return self._quote_type
@@ -336,7 +323,7 @@ class FastInfo:
         else:
             prices = prices[["Close"]].groupby(prices.index.date).last()
             if prices.shape[0] < 2:
-                # Very few symbols have previousClose despite no 
+                # Very few symbols have previousClose despite no
                 # no trading data e.g. 'QCSTIX'.
                 fail = True
             else:
@@ -355,12 +342,12 @@ class FastInfo:
             return self._reg_prev_close
         prices = self._get_1y_prices()
         if prices.shape[0] == 1:
-            # Tiny % of tickers don't return daily history before last trading day, 
+            # Tiny % of tickers don't return daily history before last trading day,
             # so backup option is hourly history:
             prices = self._get_1wk_1h_reg_prices()
             prices = prices[["Close"]].groupby(prices.index.date).last()
         if prices.shape[0] < 2:
-            # Very few symbols have regularMarketPreviousClose despite no 
+            # Very few symbols have regularMarketPreviousClose despite no
             # no trading data. E.g. 'QCSTIX'.
             # So fallback to original info[] if available.
             self._tkr.info  # trigger fetch
@@ -560,6 +547,7 @@ class Quote:
         self._retired_info = None
         self._sustainability = None
         self._recommendations = None
+        self._upgrades_downgrades = None
         self._calendar = None
 
         self._already_scraped = False
@@ -569,7 +557,7 @@ class Quote:
     @property
     def info(self) -> dict:
         if self._info is None:
-            self._fetch(self.proxy)
+            self._fetch_info(self.proxy)
             self._fetch_complementary(self.proxy)
 
         return self._info
@@ -583,25 +571,72 @@ class Quote:
     @property
     def recommendations(self) -> pd.DataFrame:
         if self._recommendations is None:
-            raise YFNotImplementedError('recommendations')
+            result = self._fetch(self.proxy, modules=['recommendationTrend'])
+            if result is None:
+                self._recommendations = pd.DataFrame()
+            else:
+                try:
+                    data = result["quoteSummary"]["result"][0]["recommendationTrend"]["trend"]
+                except (KeyError, IndexError):
+                    raise YFDataException(f"Failed to parse json response from Yahoo Finance: {result}")
+                self._recommendations = pd.DataFrame(data)
         return self._recommendations
 
     @property
-    def calendar(self) -> pd.DataFrame:
+    def upgrades_downgrades(self) -> pd.DataFrame:
+        if self._upgrades_downgrades is None:
+            result = self._fetch(self.proxy, modules=['upgradeDowngradeHistory'])
+            if result is None:
+                self._upgrades_downgrades = pd.DataFrame()
+            else:
+                try:
+                    data = result["quoteSummary"]["result"][0]["upgradeDowngradeHistory"]["history"]
+                    if len(data) == 0:
+                        raise YFDataException(f"No upgrade/downgrade history found for {self._symbol}")
+                    df = pd.DataFrame(data)
+                    df.rename(columns={"epochGradeDate": "GradeDate", 'firm': 'Firm', 'toGrade': 'ToGrade', 'fromGrade': 'FromGrade', 'action': 'Action'}, inplace=True)
+                    df.set_index('GradeDate', inplace=True)
+                    df.index = pd.to_datetime(df.index, unit='s')
+                    self._upgrades_downgrades = df
+                except (KeyError, IndexError):
+                    raise YFDataException(f"Failed to parse json response from Yahoo Finance: {result}")
+        return self._upgrades_downgrades
+
+    @property
+    def calendar(self) -> dict:
         if self._calendar is None:
-            raise YFNotImplementedError('calendar')
+            self._fetch_calendar()
         return self._calendar
 
-    def _fetch(self, proxy):
+    @staticmethod
+    def valid_modules():
+        return quote_summary_valid_modules
+
+    def _fetch(self, proxy, modules: list):
+        if not isinstance(modules, list):
+            raise YFException("Should provide a list of modules, see available modules using `valid_modules`")
+
+        modules = ','.join([m for m in modules if m in quote_summary_valid_modules])
+        if len(modules) == 0:
+            raise YFException("No valid modules provided, see available modules using `valid_modules`")
+        params_dict = {"modules": modules, "corsDomain": "finance.yahoo.com", "formatted": "false", "symbol": self._symbol}
+        try:
+            result = self._data.get_raw_json(_QUOTE_SUMMARY_URL_ + f"/{self._symbol}", user_agent_headers=self._data.user_agent_headers, params=params_dict, proxy=proxy)
+        except requests.exceptions.HTTPError as e:
+            utils.get_yf_logger().error(str(e))
+            return None
+        return result
+
+    def _fetch_info(self, proxy):
         if self._already_fetched:
             return
         self._already_fetched = True
         modules = ['financialData', 'quoteType', 'defaultKeyStatistics', 'assetProfile', 'summaryDetail']
-        modules = ','.join(modules)
-        params_dict = {"modules": modules, "ssl": "true"}
-        result = self._data.get_raw_json(
-            _BASIC_URL_ + f"/{self._symbol}", params=params_dict, proxy=proxy
-        )
+        result = self._fetch(proxy, modules=modules)
+        if result is None:
+            self._info = {}
+            return
+
         result["quoteSummary"]["result"][0]["symbol"] = self._symbol
         query1_info = next(
             (info for info in result.get("quoteSummary", {}).get("result", []) if info["symbol"] == self._symbol),
@@ -613,10 +648,10 @@ class Quote:
             if "maxAge" in query1_info[k] and query1_info[k]["maxAge"] == 1:
                 query1_info[k]["maxAge"] = 86400
         query1_info = {
-            k1: v1 
-            for k, v in query1_info.items() 
-            if isinstance(v, dict) 
-            for k1, v1 in v.items() 
+            k1: v1
+            for k, v in query1_info.items()
+            if isinstance(v, dict)
+            for k1, v1 in v.items()
             if v1
         }
         # recursively format but only because of 'companyOfficers'
@@ -643,7 +678,7 @@ class Quote:
         self._already_fetched_complementary = True
 
         # self._scrape(proxy)  # decrypt broken
-        self._fetch(proxy)
+        self._fetch_info(proxy)
         if self._info is None:
             return
 
@@ -684,14 +719,39 @@ class Quote:
 
             json_str = self._data.cache_get(url=url, proxy=proxy).text
             json_data = json.loads(json_str)
-            try:
-                key_stats = json_data["timeseries"]["result"][0]
-                if k not in key_stats:
-                    # Yahoo website prints N/A, indicates Yahoo lacks necessary data to calculate
-                    v = None
+            json_result = json_data.get("timeseries") or json_data.get("finance")
+            if json_result["error"] is not None:
+                raise YFException("Failed to parse json response from Yahoo Finance: " + str(json_result["error"]))
+            for k in keys:
+                keydict = json_result["result"][0]
+                if k in keydict:
+                    self._info[k] = keydict[k][-1]["reportedValue"]["raw"]
                 else:
-                    # Select most recent (last) raw value in list:
-                    v = key_stats[k][-1]["reportedValue"]["raw"]
-            except Exception:
-                v = None
-            self._info[k] = v
+                    self.info[k] = None
+
+    def _fetch_calendar(self):
+        # secFilings return too old data, so not requesting it for now
+        result = self._fetch(self.proxy, modules=['calendarEvents'])
+        if result is None:
+            self._calendar = {}
+            return
+
+        try:
+            self._calendar = dict()
+            _events = result["quoteSummary"]["result"][0]["calendarEvents"]
+            if 'dividendDate' in _events:
+                self._calendar['Dividend Date'] = datetime.datetime.fromtimestamp(_events['dividendDate']).date()
+            if 'exDividendDate' in _events:
+                self._calendar['Ex-Dividend Date'] = datetime.datetime.fromtimestamp(_events['exDividendDate']).date()
+            # splits = _events.get('splitDate')  # need to check later, i will add code for this if found data
+            earnings = _events.get('earnings')
+            if earnings is not None:
+                self._calendar['Earnings Date'] = [datetime.datetime.fromtimestamp(d).date() for d in earnings.get('earningsDate', [])]
+                self._calendar['Earnings High'] = earnings.get('earningsHigh', None)
+                self._calendar['Earnings Low'] = earnings.get('earningsLow', None)
+                self._calendar['Earnings Average'] = earnings.get('earningsAverage', None)
+                self._calendar['Revenue High'] = earnings.get('revenueHigh', None)
+                self._calendar['Revenue Low'] = earnings.get('revenueLow', None)
+                self._calendar['Revenue Average'] = earnings.get('revenueAverage', None)
+        except (KeyError, IndexError):
+            raise YFDataException(f"Failed to parse json response from Yahoo Finance: {result}")

yfinance/ticker.py

@@ -21,34 +21,33 @@
 
 from __future__ import print_function
 
-import datetime as _datetime
 from collections import namedtuple as _namedtuple
 
 import pandas as _pd
 
 from .base import TickerBase
+from .const import _BASE_URL_
 
 
 class Ticker(TickerBase):
-    def __init__(self, ticker, session=None):
-        super(Ticker, self).__init__(ticker, session=session)
+    def __init__(self, ticker, session=None, proxy=None):
+        super(Ticker, self).__init__(ticker, session=session, proxy=proxy)
         self._expirations = {}
         self._underlying  = {}
 
     def __repr__(self):
         return f'yfinance.Ticker object <{self.ticker}>'
 
-    def _download_options(self, date=None, proxy=None):
+    def _download_options(self, date=None):
         if date is None:
-            url = f"{self._base_url}/v7/finance/options/{self.ticker}"
+            url = f"{_BASE_URL_}/v7/finance/options/{self.ticker}"
         else:
-            url = f"{self._base_url}/v7/finance/options/{self.ticker}?date={date}"
+            url = f"{_BASE_URL_}/v7/finance/options/{self.ticker}?date={date}"
 
-        r = self._data.get(url=url, proxy=proxy).json()
+        r = self._data.get(url=url, proxy=self.proxy).json()
         if len(r.get('optionChain', {}).get('result', [])) > 0:
             for exp in r['optionChain']['result'][0]['expirationDates']:
-                self._expirations[_datetime.datetime.utcfromtimestamp(
-                    exp).strftime('%Y-%m-%d')] = exp
+                self._expirations[_pd.Timestamp(exp, unit='s').strftime('%Y-%m-%d')] = exp
 
             self._underlying = r['optionChain']['result'][0].get('quote', {})
 
@@ -80,9 +79,9 @@ class Ticker(TickerBase):
             data['lastTradeDate'] = data['lastTradeDate'].dt.tz_convert(tz)
         return data
 
-    def option_chain(self, date=None, proxy=None, tz=None):
+    def option_chain(self, date=None, tz=None):
         if date is None:
-            options = self._download_options(proxy=proxy)
+            options = self._download_options()
         else:
             if not self._expirations:
                 self._download_options()
@@ -91,7 +90,7 @@ class Ticker(TickerBase):
                     f"Expiration `{date}` cannot be found. "
                     f"Available expirations are: [{', '.join(self._expirations)}]")
             date = self._expirations[date]
-            options = self._download_options(date, proxy=proxy)
+            options = self._download_options(date)
 
         return _namedtuple('Options', ['calls', 'puts', 'underlying'])(**{
             "calls": self._options2df(options['calls'], tz=tz),
@@ -117,12 +116,24 @@ class Ticker(TickerBase):
     def mutualfund_holders(self) -> _pd.DataFrame:
         return self.get_mutualfund_holders()
 
+    @property
+    def insider_purchases(self) -> _pd.DataFrame:
+        return self.get_insider_purchases()
+
+    @property
+    def insider_transactions(self) -> _pd.DataFrame:
+        return self.get_insider_transactions()
+
+    @property
+    def insider_roster_holders(self) -> _pd.DataFrame:
+        return self.get_insider_roster_holders()
+
     @property
     def dividends(self) -> _pd.Series:
         return self.get_dividends()
 
     @property
-    def capital_gains(self):
+    def capital_gains(self) -> _pd.Series:
         return self.get_capital_gains()
 
     @property
@@ -134,7 +145,7 @@ class Ticker(TickerBase):
         return self.get_actions()
 
     @property
-    def shares(self) -> _pd.DataFrame :
+    def shares(self) -> _pd.DataFrame:
         return self.get_shares()
 
     @property
@@ -146,13 +157,24 @@ class Ticker(TickerBase):
         return self.get_fast_info()
 
     @property
-    def calendar(self) -> _pd.DataFrame:
+    def calendar(self) -> dict:
+        """
+        Returns a dictionary of events, earnings, and dividends for the ticker
+        """
         return self.get_calendar()
 
     @property
     def recommendations(self):
         return self.get_recommendations()
 
+    @property
+    def recommendations_summary(self):
+        return self.get_recommendations_summary()
+
+    @property
+    def upgrades_downgrades(self):
+        return self.get_upgrades_downgrades()
+
     @property
     def earnings(self) -> _pd.DataFrame:
         return self.get_earnings()
@@ -217,10 +239,6 @@ class Ticker(TickerBase):
     def quarterly_cashflow(self) -> _pd.DataFrame:
         return self.quarterly_cash_flow
 
-    @property
-    def recommendations_summary(self):
-        return self.get_recommendations_summary()
-
     @property
     def analyst_price_target(self) -> _pd.DataFrame:
         return self.get_analyst_price_target()
@@ -240,7 +258,7 @@ class Ticker(TickerBase):
         return tuple(self._expirations.keys())
 
     @property
-    def news(self):
+    def news(self) -> list:
         return self.get_news()
 
     @property

yfinance/utils.py

@@ -26,10 +26,10 @@ import logging
 import re as _re
 import sys as _sys
 import threading
-from functools import lru_cache
+from functools import lru_cache, wraps
 from inspect import getmembers
 from types import FunctionType
-from typing import Dict, List, Optional
+from typing import List, Optional
 
 import numpy as _np
 import pandas as _pd
@@ -41,14 +41,10 @@ from pytz import UnknownTimeZoneError
 from yfinance import const
 from .const import _BASE_URL_
 
-try:
-    import ujson as _json
-except ImportError:
-    import json as _json
-
 user_agent_headers = {
     'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}
 
+
 # From https://stackoverflow.com/a/59128615
 def attributes(obj):
     disallowed_names = {
@@ -61,7 +57,7 @@ def attributes(obj):
 
 @lru_cache(maxsize=20)
 def print_once(msg):
-    # 'warnings' module suppression of repeat messages does not work. 
+    # 'warnings' module suppression of repeat messages does not work.
     # This function replicates correct behaviour
     print(msg)
 
@@ -99,6 +95,7 @@ def get_indented_logger(name=None):
 
 
 def log_indent_decorator(func):
+    @wraps(func)
     def wrapper(*args, **kwargs):
         logger = get_indented_logger('yfinance')
         logger.debug(f'Entering {func.__name__}()')
@@ -584,8 +581,8 @@ def fix_Yahoo_returning_prepost_unrequested(quotes, interval, tradingPeriods):
 
 
 def fix_Yahoo_returning_live_separate(quotes, interval, tz_exchange):
-    # Yahoo bug fix. If market is open today then Yahoo normally returns 
-    # todays data as a separate row from rest-of week/month interval in above row. 
+    # Yahoo bug fix. If market is open today then Yahoo normally returns
+    # todays data as a separate row from rest-of week/month interval in above row.
     # Seems to depend on what exchange e.g. crypto OK.
     # Fix = merge them together
     n = quotes.shape[0]
@@ -649,7 +646,6 @@ def safe_merge_dfs(df_main, df_sub, interval):
     if df_main.empty:
         return df_main
 
-    df_sub_backup = df_sub.copy()
     data_cols = [c for c in df_sub.columns if c not in df_main]
     if len(data_cols) > 1:
         raise Exception("Expected 1 data col")
@@ -694,7 +690,7 @@ def safe_merge_dfs(df_main, df_sub, interval):
                 df_main['Dividends'] = 0.0
                 return df_main
         else:
-            empty_row_data = {c:[_np.nan] for c in const.price_colnames}|{'Volume':[0]}
+            empty_row_data = {**{c:[_np.nan] for c in const._PRICE_COLNAMES_}, 'Volume':[0]}
             if interval == '1d':
                 # For 1d, add all out-of-range event dates
                 for i in _np.where(f_outOfRange)[0]:
@@ -703,7 +699,7 @@ def safe_merge_dfs(df_main, df_sub, interval):
                     empty_row = _pd.DataFrame(data=empty_row_data, index=[dt])
                     df_main = _pd.concat([df_main, empty_row], sort=True)
             else:
-                # Else, only add out-of-range event dates if occurring in interval 
+                # Else, only add out-of-range event dates if occurring in interval
                 # immediately after last price row
                 last_dt = df_main.index[-1]
                 next_interval_start_dt = last_dt + td
@@ -711,7 +707,6 @@ def safe_merge_dfs(df_main, df_sub, interval):
                 for i in _np.where(f_outOfRange)[0]:
                     dt = df_sub.index[i]
                     if next_interval_start_dt <= dt < next_interval_end_dt:
-                        new_dt = next_interval_start_dt
                         get_yf_logger().debug(f"Adding out-of-range {data_col} @ {dt.date()} in new prices row of NaNs")
                         empty_row = _pd.DataFrame(data=empty_row_data, index=[dt])
                         df_main = _pd.concat([df_main, empty_row], sort=True)
@@ -771,14 +766,14 @@ def safe_merge_dfs(df_main, df_sub, interval):
 
 def fix_Yahoo_dst_issue(df, interval):
     if interval in ["1d", "1w", "1wk"]:
-        # These intervals should start at time 00:00. But for some combinations of date and timezone, 
+        # These intervals should start at time 00:00. But for some combinations of date and timezone,
         # Yahoo has time off by few hours (e.g. Brazil 23:00 around Jan-2022). Suspect DST problem.
-        # The clue is (a) minutes=0 and (b) hour near 0. 
+        # The clue is (a) minutes=0 and (b) hour near 0.
         # Obviously Yahoo meant 00:00, so ensure this doesn't affect date conversion:
         f_pre_midnight = (df.index.minute == 0) & (df.index.hour.isin([22, 23]))
         dst_error_hours = _np.array([0] * df.shape[0])
         dst_error_hours[f_pre_midnight] = 24 - df.index[f_pre_midnight].hour
-        df.index += _pd.TimedeltaIndex(dst_error_hours, 'h')
+        df.index += _pd.to_timedelta(dst_error_hours, 'h')
     return df
 
 
@@ -864,9 +859,9 @@ class ProgressBar:
         if self.elapsed > self.iterations:
             self.elapsed = self.iterations
         self.update_iteration(1)
-        print('\r' + str(self), end='')
-        _sys.stdout.flush()
-        print()
+        print('\r' + str(self), end='', file=_sys.stderr)
+        _sys.stderr.flush()
+        print("", file=_sys.stderr)
 
     def animate(self, iteration=None):
         if iteration is None:
@@ -875,8 +870,8 @@ class ProgressBar:
         else:
             self.elapsed += iteration
 
-        print('\r' + str(self), end='')
-        _sys.stdout.flush()
+        print('\r' + str(self), end='', file=_sys.stderr)
+        _sys.stderr.flush()
         self.update_iteration()
 
     def update_iteration(self, val=None):

yfinance/version.py

@@ -1 +1 @@
-version = "0.2.32b1"
+version = "0.2.40"