* feat: add MappedSynchronizingHistoryProvider base class
Introduces an abstract class for history providers that handle symbol mapping and time-aligned data slices. Uses IMapFileProvider to resolve ticker changes, provides an abstract method for mapped history retrieval, and overrides GetHistory to synchronize results. Enables nullable reference types and adds documentation.
* Minor tweaks
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Fix trade drawdown calculation
* Cleanup
* Disable MAE. MFE and Drawdown calculation for FlatToFlat and FlatToReduced trade grouping methods
* Minor test fixes
* Add total performance statistics to live result files
* Truncate closed trades in live results
* Avoid adding totalPerformance to live minute result file
* Deprecated Trade.Symbol in favor of new Trade.Symbols
* Fixes for Trade serialization
* Add trades json serialization tests
* Cleanup
* Fix NullReferenceException in SecurityService
* Simplify the unit test
* Add unit test for RandomDataGenerator
* Improve unit test name
* Solve review comments
* Fix HistoryRequest DataMappingMode default to use security configuration
* Solve review comments
* Make HistoryRequest inherit existing subscription configuration values generically
* Exclude any class that inherits from BaseChainUniverseData
* Reuse existing filter for user configuration
* Solve review comments
* Normalize DataMappingMode
* Minor fix
* rebase Add 2026 EUREX, CFE and ICE Holidays
* Remove entries present in generic entry
For some ICE Future entries, there were holidays that were already
present in their generic entry (Future-ice-[*]), so those dates were
removed from the entry
* Implements Covariance as Lean Indicator
* Add COV helper method and fix AcceptsVolumeRenkoBarsAsInput test
* Fix AcceptsRenkoBarsAsInput test to use smaller period for faster execution
* Fix slow Renko tests by limiting data processed to 50 rows
* Add 2026 HKFE Future holidays & early closes
* rebase Add 2026 Future-cme-equity Holidays, Early closes, late opens
* Add (again) HKFE 2026 Holidays and early closes
* Add Future-cme-interest 2026 holidays
* Add 2026 CME Fx 2026 holidays"
- Add 2026 CME Fx holidays, early closes, late opens, bank holidays
- Exclude CNH, MNH and MIR as they expire rules don't consider US
Holidays
- Fix wrong early close on 12/24/2025 from 12:15 to 12:45
* Add CME Future crypto 2026 Holidays, EC, LO & BH
* Add CME Future Energy 2026 holidays, EC, LO, BH
* Add CME Futures metals holidays, ec, lo, bh
* Add CME Futures grains 2026 holidays, ec, lo, bh
* Add CME Futures Dairy 2026 holidays, ec, lo, bh
* Add CME Futures livestock holidays, ec, lo, bh
* Add CME Future Lumber holidays, ec, lo, bh
* Add CME Futures Softs holidays, ec, lo, bh
* Add CME Futures Oilseeds holidays, ec, lo, bh
* Add CME Futures AW, GD Holidays, EC, LO and BH
* Move repeated bank holidays to generic entries
* Nit change
* Solve bug
Since 11/26/2026 is a bank holiday for CME energy futures, the expiry
date is moved to 11/25/2026 as the expiry date for HH is the third last
business day of the month prior to the contract month
* Return weekly contracts if no standard contracts exist
* Fix unit and regression tests
* Centralize default expiration type flags
* Add ExcludeWeeklys() method
* Mark IncludeWeeklys() as obsolete since weeklies are now default
* Refactor optimization result stats serialization
* Handle custom optimization statistics serialization
* Support custom statistics
* Support newest Lean statistics
Address peer review
* Add more tests
* Make indices reserved statistic names
* Minor fixes and cleanup
* Add Api helper methods for POST requests
* Support json serializer setting for json requests
* Minor changes and cleanup
* Make ApiConnection disposable
* Added python-venv to ReportArgumentParser, getting python-venv in Program class
* Added venv activation and python initialization to Report class
* Resolve review comments. Move python init and env addition to Report/Program.cs
* Refactor ApiClient to use HttpClient
* Minor fix
* Refactor and cleanup in ApiConnection
* Add normalized extension methods for string and stream json serialization
* Minor fixes
* Address peer review
* Minor fixes
* Minor fixes and peer review
* Race condition fix
* Minor changes
* Cleanup
* Minor fixes
* Minor fix
* add dYdX market constant
* Add dYdX market constant and update symbol properties database
* get Gaz limit from order properties or default value
* wip
* add symbol properties
* add more dydx order props
* update symbol properties
* Add dYdX brokerage and fee model
* Update dYdX configuration fields
* undone meta files
* minor tweaks
* fix file ending
* Add dYdX brokerage support to BrokerageName and IBrokerageModel
* Refine dYdX fee model integration and update configuration defaults
* Replace custom order size validation in dYdXBrokerageModel with DefaultBrokerageModel implementation
* undone changes
* Add market hours for dYdX CryptoFuture
* Centralize Python/C# model detection logic
* Improve helper method
* Remove unnecessary conditionals
* Fix unit tests
* Refactor QCAlgorithm.python.cs to use the new helper method for models
* Solve review comments
* Clean up Python wrapper exception
* Update XML comments to better describe T and TWrapper in CreateModelOrWrapper
* Join unit tests in TestCases
* Solve review comments
* Replace manual TryConvert checks with CreateInstanceOrWrapper helper
* fix: use decimal instead long in Scale's LeanData
test:feat: validate naming with large precision
* feat: use normalization in Scale's LeanData
test:fix: strike precisions
* fix: use str normalization in Scale's LeanData
* Docs(Algorithm): Add XML code references
Adds XML documentation tags (<see/>, <paramref/>) to code
references within the Algorithm folder. This improves code
navigation, IntelliSense, and helps the stubs generator tool
to correctly map C# elements.
Resolves#9016
* Fix missing newline at end of QCAlgorithm.cs
* Update print statement from 'Hello' to 'Goodbye'
* Fix documentation for market parameter in crypto methods
* Increase option strike digits precision to 6
- Increase option strike digits precision to 6. Adding unit test
* test:fix: couple of build tests
* fix:test: Asset CUSIP and OrderListHash in Regression Algorithms
* test:fix: several regression algos
* test:fix: old Symbol CUSIPs to new ones
---------
Co-authored-by: Romazes <romazes20@gmail.com>
* Add SeedInitialPrices algorithm setting
This is true by default and indicates that the engine will seed initial prices right after the security is added or selected
* Update regression algorithms
* Update regression algorithms
* Update regression algorithms
* Refactor default securities seeding
* Minor fix
* Minro fixes
* Cleanup
* Updated and add regression algorithms
* Address peer review
* Centralize logic to get last known data for multiple securities
* Some cleanup
* Minor build fix
* Minor fixes
* More logic centralization
* Some more cleanup
* Cleanup
* Update regression algorithms and minor fixes
* Update regression algorithms
* Minor fix
* More minor fixes
* Update regression algorithms
* Cleanup
* Minor test fix
* Address peer review
* Minor fix and performance improvement
* Fix to seed open interest data
* Minor test fixes
* Address peer review
* Minor change
* Minor revert
* Minor fixes and improvements
* Disable initial seeding by default
* Minor fixes
* Cleanup
* Cleanup
* Minor fix
* Refactor user define universe handling
- Normalize user define universe additions and removals to behave like
other subscriptions without requiting special handling
* Minor fixes
* Initial solution
* Solve review comments
* Add PythonSelectionModelHandler to reduce code duplication
* Refactor universe selection models
* Add python instance to Selection Models with virtual/override methods
* Add python instance to Alpha Models
* Add python instance to Execution models
* Solve review comments
* Solve new review comments
* Fix calling SetPythonInstance only when method exists and is callable
* Use unit test instead of regression algorithms
* Solve review comments
* Set python instance to the models
* Initialize Python containers only when instance is set
* Replace try-catch with explicit method existence check
* Initialize containers in BasePythonWrapper only when needed
* Add null instance check before method invocation
* Refactor TryExecuteMethod
* Refactor Python wrappers which inherit from BasePythonWrapper<>
* Solve review comments
* Remove ununsed methods
* Solve review comments
* Fix performance degradation on expirying assets
- Options would expire and get removed, when the universe removed the
security it would never be reseted on universe selection, causing
leaks
- Minor improvement for composer assembly loading
- Minor improvement for loading exception interepreters, so it uses
composer
* Minor regression test fix
* Add new benchmark algorithm
* Use Futures Bank Holidays For Expirations
* Minor improvements
- Futures will be stored by their contract month, not expiry
* Delete dairy future products
* Minor test fixes
* Add optional strict parameter to PivotPointsHighLow indicator for relaxed comparison
- Added bool strict parameter (default true) to PivotPointsHighLow constructors
- When strict=true: Uses strict inequalities (> and <) - existing behavior
- When strict=false: Uses relaxed inequalities (>= and <=) - allows equal values
- Updated comparison logic in FindNextLowPivotPoint and FindNextHighPivotPoint
- Added new PPHL overload in QCAlgorithm.Indicators.cs with strict parameter
- Added 5 comprehensive unit tests using TestCase attributes
- All tests pass (19/19 relevant tests)
- Backwards compatible: default behavior unchanged
Addresses #8881
* Fix PPHL method signature to resolve overload ambiguity and maintain API consistency
- Remove wrapper overload to eliminate compilation ambiguity
- Reorder parameters: strict parameter now before resolution
- Keep selector as last parameter (consistent with other indicator helpers)
- Add unit tests verifying no overload ambiguity and correct strict parameter behavior
Addresses PR review feedback on #9033
* Implement backward-compatible PPHL overloads to support strict parameter
Added a second overload to maintain backward compatibility while introducing
the strict parameter. This prevents breaking changes for existing code that
uses positional arguments with the PPHL helper method.
Changes:
- Added backward-compatible overload that accepts (symbol, lengthHigh, lengthLow,
lastStoredValues, resolution, selector) and delegates to the new implementation
with strict: true
- Kept the new overload with strict parameter as the primary implementation
- Added comprehensive test (QCAlgorithmHelperOverloadResolution) documenting all
9 valid call patterns to prevent future regressions
This approach ensures:
- Existing calls like PPHL(symbol, 3, 3, 100, Resolution.Minute) continue to work
- New calls like PPHL(symbol, 3, 3, strict: false) are supported
- No ambiguity in overload resolution
- All 23 PPHL tests pass successfully
* Clean up some test duplication
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Fix settting brokerage model after adding security initializer
* Minor fix
* Minor change
* Minor change
* Send warning for calling SetBrokerageModel before setting or adding security initializer
* Send warning for calling SetBrokerageModel before setting or adding security initializer
* Minor change
* Minor change
* Cleanup
* Add new QCAlgorithm.AddSecurityInitializer method
This allows to add multiple security initializers without manually creating an instance of a CompositeSecurityInitializer
* Add Python support
* Cleanup
* Fix: Add obsolete flag to AddUniverse coarse+fine
Added obsolete flag to AddUniverse(Func<IEnumerable<CoarseFundamental>, IEnumerable<Symbol>> coarseSelector, Func<IEnumerable<FineFundamental>, IEnumerable<Symbol>> fineSelector), added one time logging and altered docs accordingly.
* Update QCAlgorithm.Universe.cs
---------
Co-authored-by: Martin-Molinero <martin.molinero1@gmail.com>
* Fix ProcessSplitSymbols crash during live mode warmup
Fixes InvalidOperationException when CancelOpenOrders is called during warmup in live/paper trading mode.
When historical splits are replayed during live mode warmup, ProcessSplitSymbols attempts to cancel open orders for option contracts, but CancelOpenOrders throws an exception during warmup.
The fix adds an early return when algorithm.LiveMode && algorithm.IsWarmingUp. This is safe because in live/paper trading, current positions already reflect historical splits.
Includes unit tests validating the fix.
* Some improvements and more tests
- Ignore splits during warmup for backtesting and live trading. Adding a
regression algorithm reproducing issue
* Minor fix
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Minor fix for negative rolling window index
- Minor fix for negative rolling window index when partially full,
adding unit test
* Reduce RefreshesSymbolProperties test runtime
* Freeze autoray version to 0.7.2 in foundation image
Required for compatibility with pennylane package
* Update foundation image
- Updating foundation image libraries and new libraries and tests
* Some more foundation tweaks
---------
Co-authored-by: Jhonathan Abreu <jdabreu25@gmail.com>
* feat: support new OrderType in TastytradeBrokerageModel
* feat: extension GetGroupQuantity
* refactor: WaitOneAssertFail to return bool
* refactor: GetOpenOrders in BrokerageTests
* feat: display holdings when begin/end test run
* refactor: brokerageTest class to support handle combo order
feat: comboOrderTestParameters
feat: tastytrade option strategy algo
* refactor: Tastytrade Algo BullCallSpread
* feat: Bull/Bear-CallSpread test cases in BrokerageTests
feat: ComboLimitOrderTestParameters
* fix: remove static in ComboLimitOrderTestParameters of ExpectedStatus
* feat: restrict submit combo limit cross zero orders in Tastytrade
test:feat: submit combo limit cross zero test cases in tastytrade
* test:fix: missed test helper to c9907b23
* refactor: GetGroupQuantityByEachLegQuantity
feat: overload GreatestCommonDivisor
test:feat: GetGroupQuantityByEachLegQuantity
* test:feat: extra test cases to GetGroupQuantity to 925c214a
* test:refactor: take out Cancel Status update in HandleEvents()
* feat: prop Direction in Leg
* Revert "feat: prop Direction in Leg"
This reverts commit 4f92f97b63.
* refactor: ComboLimitOrderTestParameters
* test:refactor: CancelOrders
* test:feat: add missed xml description
* test:feat: support ModifyUntilFilled Combo orders
* test:feat: Long/Short-Combo unit tests
* test:fix: compare all status if it is combo order type
* test:refactor: applyOrderUpdate in gracefully way in LimitOrderTestParameters
* remover: Tastytrade algo
* test:feat: create new BaseOrderTestParameters with helper methods
test:rename: method in ComboOrderTestParameters
test:remove: duplication
* Fix extended hours only days period history
Don't include extended hours only days in date range calculated fro period history request
* Add comments
* Cleanup
* Revert some changes
* Cleanup
* Add VBaseSignalExport
* Add a VBaseSignalExport feature that allows signals to be stamped with the help of the ValidityBase REST API.
* Addressing review comments
* Create DefaultApiBaseUrl const in VBaseSignalExport
* Add vbase signal export demonstration algorithm
* Set initial parameters for VBaseSignalExportDemonstrationAlgorithm
* Refactor VBaseSignalExport and demonstration algorithm: update symbol list, extend end date, and improve API base URL handling
* Initialize allowed security types in VBaseSignalExport and fix symbol reference in CSV generation
* Adjust rate limiter for vBase signal export to allow 6 requests per hour for better performance
* Minor tweaks
---------
Co-authored-by: Volodymyr Bobko <volodymyr.bobko@un.org>
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Implement Session class
* Use MarketHourAwareConsolidator inside Session
* Delete SecurityCacheSessionConfig
* Refactor session class
* Solve review comments
* Fix regression algorithms
* Add null check to AvailableDataTypes
* Update session class
* Solve new review comments
* Add new test for futures and update regression test to PEP8
* Add new regression algorithm for Session with futures
* Resolve review comments
* Add local time keeper to SecurityCache
* Scan on consolidator updates and time updates
* Add new regression algorithms and unit tests
* Add unit tests and improve comments
* Fix syntax test
* Solve review comments
* Add regression algorithms
* Resolve review comments
* Make SessionConsolidator implement IDataConsolidator
* Add caching to avoid recreating SessionBar instances on every access
* Solve review comments
* Make SessionConsolidator inherit from MarketHourAwareConsolidator
* Add unit test to verify Session preserves previous day values
* Update regression algorithms
* Solve review comments
* Resolve review comments
* Solve new review comments
* Add unit test for Session class
* Refactor SessionBar properties and Update method
* Use WorkingDataInstance reference instead of _workingData.Clone()
* Use a delegate to get the workingBar instance
* Fix bug in MarketHourAwareConsolidator
* Solve review comments
* Handle OI values
* Add regression algorithms for future contracts and options
* Solve review comments
* Refactor SessionConsolidator to inherit PeriodCountConsolidatorBase
* Prevent overlapping higher resolution data
* Update tests
* Solve review comments
* Minor tweaks
Submit (and fill in case of market orders) in the same time step even if orders are asynchronous in backtesting since it's data driven instead of real time driven.
* Enable concurrency for paper brokerage.
Implement PaperBrokerageTransactionHandler to handle concurrency for live paper trading transaction handler
* Remove PaperTradingBrokerageTransaction handler
Favor existing BacktestingTransactionHandler
* Minor fixes and cleanup
* Minor unit tests fix
* Add 'asynchronous' parameter to trading api
* Add regression algorithms
* More fixes for asynchronous orders
* Fix failing unit tests
* Fix failing unit tests
* Add multiple orders requests in transaction handler
Make each transaction thread handle requests corresponding to the same order
* Refactor order state machine in BrokerageTransactionHandler
Now using a single dictionary to hold orders and their state, either pending for submission, open or closed.
* Revert: remove complete orders from new state machine in BTH
* Add order ticket to new BTH open orders state machine
* Run syn in Backtesting transaction handler for new orders submission
* Remove OpenOrderState.SubmissionPending property
* Add Security to OpenOrderState
* Minor fix
* Cleanup
* Some cleanup in BTH
* Cleanup
* Minor fix
* Minor unit test fix
* Minor fix
* Make Brokerage.ConcurrencyEnable property virtual
* Minor change
* Cleanup
* Add 'asynchronous' parameter to SetHoldings api
* Improve regression algorithms
* feat: not use CrossZeroOrder for MOO and MOC
* Revert "feat: not use CrossZeroOrder for MOO and MOC"
This reverts commit 1a71c5fe90.
* feat: Not support crossZeroOrderTypes
test:feat: cross zero orders
* Revert "feat: Not support crossZeroOrderTypes"
This reverts commit 16785b5d2f.
* feat: not support CrossZeroOrder in TradeStation
* feat: not allow to place MOO and MOC with CrossZeroOrder
* test:feat: MOO and MOC CrossZeroOrder reject
* Add support for some CME currencies FOPs
Support ADU (6A), GBU (6B), CAU (6C), EUU (6E), JPU (6J), CHU (6S)
* Cleanup
* Consider holidays in FOPs expiry function
* Add tag parameter, update CancelOpenOrders and Liquidate calls
* Resolved commen on the related PR: following the pattern of Liquidate() using Liquidated as default, I think the tag should be Removed
* Add tag parameter to RemoveSecurity and update regression algorithms
* Minor tweaks
---------
Co-authored-by: meysam <poormonfared.azimi@gmail.com>
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Set consolidated bar EndTime to last input's EndTime in count-based consolidator
* Solve review comments
* Move EndTime assignment to base class
* Update unit tests
* initial draft changes
* Saving draft changes for TD Sequential Implementation
* Fixed failing tests
* Refactoring changes
* Additional refactoring to TDSequential
* Added tests
* Incorporated review comments
* reverted changes to older file
* Updated for review comment
* Updated Buy and Sell Setup Perfect functions based on review comment to remove equality operator
* Added SupportPrice and ResistancePrice as read-only public properties to expose the internal _resistance and _support prices
* Changed return value of ComputeNextValue to TomDemarkSequentialPhase instead of the encoded value of Phase.StepCount as per review comment
* Updated initialization to include bullish and bearish flip checks
* Refactored and cleaned up based on review comments
* Some fixes for new C# enums handling in Pythonnet
* Minor changes and cleanup
* Update Pythonnet version to 2.0.45
* Minor changes
* Minor fix
* Minor fix
* Minor change
* Minor change
* Minor unit test fix
* Log the Number of Active Subscriptions Before and After Warm Up #8779
* Refactor: Log subscriptions count once before first timeslice
* Resolve review comments
* Minor cleanup
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* feat: Get AccessToken from Lean API
* feat: create TokenCredentials DTO instead of Tuple
* remove: extra unrequired null reference exceptions on argument check
* remove: SendAsync override in TokenHandler
* feat: override SendAsync in TokenHandler to prevent future bugs
* test:feat: SyncAsync in TokenHandler
* refactor: name property of AccessTokenExpires to Expiration in AccessTokenMetaDataResponse
* feat: add retry logic to getAccessToken in TokenHandler
test:feat: TokenHandler with retry logic
* feat: make retry timeout configurable in ctor
* Changed default async to true and passed target.Tag
* Make execution models place orders asynchronously if specified
* Add unit tests
* Execution model default to asynchronous orders.
Also, minor fixes for tickets remaining fill quantity potential race conditions
* Add SecurityHolding.UnrealizedQuantity property
It gets the holding quantity the security will have once all open orders are filled.
Added for thread safety reasons when execution models place asynchronous orders and need to calculate the actual quantity needed to reach the target of there are open orders
* Some cleanup
* Adjust projected holdings quantity on splits
* Minor fix
* More changes and cleanup
* Minor fix
* Improvements for thread safety
* Add IOrderProvider.GetProjectedHoldings to get projected holdings atomically
* Minor unit tests fix
* Add ProjectedHoldings DTO class
* Address peer review
---------
Co-authored-by: arthiondaena <arthiondaena@gmail.com>
* Implement a prototype of the maximum recovery time function.
* Add unit test skeletons.
* Add failing test
* Issue #4581: Implement MaxDrawdownRecoveryTime.
* Issue 4581: Add DTO for Drawdown Percentage, Drawdown Enddate, and High Value
* Issue 4581: Fix bgu for when lDrawdowns list is empty.
* Issue 4581: Change names of tests. Change name of file.
* Issue 4581: Make adjustements to flow of adding drawdowns to lDrawdowns.
* Issue 4581: Add multiple unit tests.
* Issue #4581: Change name of unit test
* Issue #4581: Add to PerformanceMetrics
* Issue #4581: Add Maximum Drawdown Recovery to PortolioStatistics class.
* Issue #4581: Add to portolfio statistics class.
* Issue #4581: Add to statistics builder.
* Issue #4581: Add report key.
* Case #4581: Convert to decimal.
* Issue #4581: Correct comment.
* Issue #4581: Correct performance metrics view model string.
* Case #4581: Correct statistics builder view model string..again.
* Issue #4581: Placed DradownDradownDateHighValueDTO at the end of the file for simpler diff.
* Issue #4581: Add 2 new tests.
* Issue #4581: Change algorithm so that when multiple maximum drawdowns occur, the longest of all recoveries is reported.
* Issue #4581: Add unit test.
* Issue #4581: Remove reportkey. Change dto name.
* Issue #4581: Change summary.
* Issue #4581: Change comment.
* Add max drawdown recovery calculation with unit tests
* Update regression algorithms with the new metric
* Solve review comments
* Update regression algorithms
* Add TryGet to safely get the key: MaximumDrawdownRecovery
* Ignore MaximumDrawdownRecovery metric in OptimizationBacktest Json
* Revert changes in Messaging
* Update regression algorithms
* Add test case: TakesLongestRecoveryAmongMultipleDrawdowns
* Use integer days for MaximumDrawdownRecovery
* Add MaximumDrawdownRecoveryReportElement
* Use more explicit names
* Rename files and variables for consistency
* Update regression algorithms
---------
Co-authored-by: Alain Schaerer <aschaerer@pcatg.com>
* Use correct sids when reading options/futures universe files
* Introduce new format for options and futures universe files
* Minor change
* Minor data fix
* Minor change
* Fix failing unit tests
* Add options and futures symbols cache
* [HACK] Force use test universe files
* Minor fix for universe symbols cache
* Performance improvements
* Miror change
* Cleanup
* Sort universe files
* Minor regression algorithm fix
AddAndRemoveOptionContractRegressionAlgorithm to not depend on universe file entries ordering
* Add time tolerance when checking if asset is within current bar
* Update IsExchangeOpen and add unit tests
* Solve review comments
* Add unit test in ImmediateFillModelTests
* Delete conditional IsExchangeOpen()
* Fill MOC orders when LocalTime is at or after market close
* Allow multiple concurrent transaction handlers
Also support multiple producers in BrokerageConcurrentMessageHandler
* Minor change
* Minor change
* Add new maximum-transaction-threads config to json file
* Minor naming changes
* Address peer review and add unit tests
* Minor fix
* Minor fix and unit tests
* Address peer review
* Support sub-indicator discovery using IIndicator and exclude PandasIgnore attribute
* WIP: PandasInclude
* Add PandasInclude annotation
* Resolve review comments
* Add a list of ignored properties
* Clean up [PandasIgnore] in IndicatorBase
* feat: new Level One service
feat: new ctor in Tick
fix: missed SetValue in Tick ctor
* refactor: LevelOneService
feat: Tick Event Args
* feat: use aggregator in LevelOneService explicitly
remove: extra TickEventArgs
* feat: ThreadSafeDataAggregatorWrapper
refactor: use ThreadSafeDataAggregatorWrapper in LevelOneService
* feat: generic LevelOne Market Data Manager
refactor: LevelOneMarketData
remove: ThreadSafeDataAggregatorWrapper
* feat: support OpenInterest in LevelOneMarketData
feat: add EventBasedDataQueueHandlerSubscriptionManager in LevelOneServiceManager
feat: handle OpenInterest
* feat: add missed license in BaseDataEventArgs
* feat: create un-subscribe wrapper in LevelOneServiceManager
refactor: TryGetLevelOneMarketData
feat: validate the same Quote data in LevelOneMarketData
* feat: handle nullable value in levelOneMarketData
* feat: handle nullable value in LevelOneServiceManager
* refactor: change location of LevelOneMarketData Service
test:feat: test of LevelOneMarketData
* feat: add SetIgnoreZeroSizeUpdates
* refactor: use ignoreZeroSizeUpdates internally in TryResolvePriceSize
* refactor: position add symbols in collection in LevelOneServiceManager
* fix: validation of IgnoreZeroSizeUpdates setting in LevelOneMarketData
* test:feat: validate UpdateLastTrade with IgnoreZeroSizeUpdates setting
test:refactor: use global DateTime in testCases
* fix:test: missed calculation value in QuoteBar
* Make RollingWindow compatible with any data type
* Add generic RW<T> method
* Add RollingWindow wrapper class for PyObject
* Solve review comments
* Resolve review comments
* Fill-forward last warmup data point to normal data feed
* Minor clean up
* Minor changes and unit tests
* Address peer review
* Minor changes
* Improve unit tests
* implements Klinger Volume Oscillator
* preserve newest data file
* improve class summary
* address PR comments, add signal line, use RollingWindow to persist data, refactor code to make easier to follow
* add helper method KVO in in QCAlgorithm.Indicators.cs, add Signal test, fix IsReady and WarmUpPeriod
* default signal period for KVO, refactor and improve ComputeNextValue, simplified code
* update comment
* update comment
* Add open interest data to future chains contracts
Even though futures chains and contracts are not created from internal subscriptions data (like OI subscriptions or continuous futures contracts subscriptions), we still want to add OI data to already created contracts given that OI subscriptions are always internal but trade an quote ones are not.
* Minor changes
* Limit supported order types to Market, Limit, and StopMarket in Bitfinex brokerage
* Update brokerage models
* Fix broken unit test
* Solve review comments
* Convert regression algorithm into a unit test
* Resolve review comments
* Update SPDB with Bybit
* Update SPDB with CoinBase
* Update SPDB with Bitfinex
* Update SPDB with Kraken
* Update SPDB with Binance
* Update SPDB with BinanceUS
* Update regression algorithms
* Update SPDB - Binanceus to solve issues with the unit tests
* Initial solution
* Solve review comments
* Fix unit tests
* Resolve comments reviews
* Return a default symbol instead of throwing an exception
* Add unit test
* Ignore some errors in python syntax check.
Fix some algorithms syntax as well.
* Revert some changes
* Minor change
* Minor change
* Minor syntax check fixes
* Ignore more mypy errors
* Minor changes
* Ignore more mypy messages
* Ignore more mypy messages
* Make ExtendedDictionary extend IDictionary
* Revert "Make ExtendedDictionary extend IDictionary"
This reverts commit bbbe9c769be682767cb9e487b904480376b53596.
* Make ExtendedDictionary key a type parameter
* Make SecurityPositionGroupModel an ExtendedDictionary
* Add unit tests
* Minor change
* Add more unit tests and other minor changes
* Add Count and ContainsKey to BaseChain
* Update pythonnet version to 2.0.43
* Update some python regression algorithms
* Minor regression algorithms fix
* Make CashBook and ExtendedDictionary
* Minor changes
* Minor changes
* Update pythonnet version to 2.0.44
* feat: support MOO in CharlesSchwabBrokerageModel
* feat: validate IsMarketOpen with MOO order type in PreOrderChecks
* test:feat: Validate MOO in various Market hours time
* fix: Algo with new MOO time validation
* test:fix: set DateTime of Algo explicitly in OrderQuantityConversionTest
* First attempt
* Improve unit test
* Add suggestions
* Nit changes
* Address requested changes
* Fix failing unit tests
* First attempt to fix API tests
* Fix command
* Address suggestions
* Address minor suggestions
* wip: show work
* refactor: TradeStationBrokerageModel of using OutsideRegularTradingHours
test:feat: OutsideRegularTradingHours
* remove: comboLimit OrderType in IsLimitOrder(...)
* fix: regression algo which test unsupported OrderType in extend market hours
* fix: condition with supportsOutsideTradingHours in CanSubmitOrder
* refactor: regression test with OutsideRegularMarketHours TradeStation
* refactor: TradeStationBrokerageTradeWithOutsideRegularMarketHoursParameter
---------
Co-authored-by: Henry Ennis-Thomas <henry@henryetdev.com>
* test:feat: Assert Lean UnsettledCash and Brokerage.PerformCashSync
test:feat: new PaperBrokerage with manual cash balance
* test:refactor: calculation of Margin
* feat: change DefaultSettlementTime
* test:fix: use right brokerage settlement cash to balance
* test:refactor: use specific time when SettlementModel.Scan happens
* Initial solution: Add an extra conditional in SubscriptionManager and add regression algorithms
* Solve comments review
* Add unit test
* Refactor unit tests in SubscriptionManagerTests
* Resolve comments review
This allows the stubs generator to know when to not generate Union types for parameters that can accept multiple types, like Symbol | str | BaseContract
- To simplify stub generation, merge PythonSlice into Slice, following
existing pattern too, see QCAlgorithm.Python.cs
- Minor sintax fixes in example python algorithms
* Fill forward prev date universe file in manual chains fetch
If there is no universe file available for the current date, use the previous one and fill forward to avoid having empty chains on a day when we haven't generated the universe files
* Try a maximum of 3 days behind for manual chains fetching
* Minor change
* Minor fix
* Add comments
* Minor fix
* Minor fix for expired contracts
* Initial solution
* Create testCases for earlyClose and lateOpen
* Refactor GetExchangeHours and add unit tests
* Update market-hours-database and create a unit test
* Resolve review comments
* Update market-hours-database
* Parallelize the foreach loop to improve test performance
* Adds example of custom signal exports.
- SignalExportManager send signals automatically after 5 seconds.
- Missing example and Python support.
* Updates Examples to Disable Automatic Export
* Addresses Peer-Review
* Initialize securities on re-addition
* Minor unit tests fixes
* Reset Index.ManualSetIsTradable when security is removed
* More changes to clean up and simplify
* Address security re-utilization only
* Cleanup regression algorithms
* Fix QuantBook Option/Future history api date range
Properly calculate date range when passed range is not a whole date but a time of day
* Extended unit tests
* Disable extended market hours for universe history requests
* Exclude extended market hours for all chain universe history requests
* Disable extended market hours for chain universes
* Minor change
* Minor fix
* Minor change
- likely a copy-pasta error from Series, where index=0 is passed to
BaseSeries constructor, only now it was passed as SeriesType, which
is subsequently set in CandlestickSeries constructor
* feat: make static and get; set; of DefaultSettlementDays in Equity,Option
* refactor: reset _lastUnsettledCash depend on settlementDays in UnsettledCashWhenQuoteCurrencyIsNotAccountCurrencyAlgorithm
* FINAL TEST
* Change version
* test commit
* echo commits
* echo last commits before generating stubs
* Fix bugs
* fix more bugs
* fix yaml error
* try with a different version
* Check we can get last commits
* add variables in options
* Echo commits
* tweaks
* More tweaks
* Add an exception
* FINAL TEST
* Make FOPs selection universe file-based for backtesting
* Make FOPs option chains universe file based
* Make Future universe selection file-based like option universe
* Make Future universe selection file-based like option universe
* Abstraction cleanup
* Add FuturesChains API to QC algorithm
Also refactor future chain provider to use the new FutureUniverse instead of zip file names
* Update regression algorithms stats
* Refactor QuantBook option and future history to use new universes
* Fix failing tests
* Fix failing tests
* Fix failing tests
* Minor future chains unit test improvement
* Add futures chains DataFrame property
Also, remove IDerivativeSecurity interface from Future
* Add DataFrame property to FuturesChains class
* Add regression algorithms
* Add regression algorithms
* Replace QCAlgorithm.FutureChainProvider usages with new FuturesChain api
* Minor fixes
* Reduce number of universe files in repo
* Minor data fixes
* Regression algorithms updates
* Add implicit conversion from FuturesContract to Symbol
Modified algorithms to use futures contract objects directly instead of accessing their Symbol property.
Removed unnecessary import statements and redundant lines in various files.
* Improve resolution handling for history requests
* Changed _auxiliaryData field to lazily-initialized AuxiliaryData property
* Refactor data handling in BaseChain and TimeSliceFactory
- Added `AddData` method to `BaseChain` for adding market data
- Refactored `TimeSliceFactory` to use `BaseChain.AddData` method
* Remove specific constructors and indexers from Chain classes
Removed public indexers in `BaseChains` for getting or setting `BaseChain` instances by `ticker` or `Symbol`, which were used for Pythonnet compatibility.
* Remove chain cache logic from FuturesChainUniverse
* Refactor class and interface names for clarity
Renamed `FileBasedUniverse` to `BaseChainUniverseData` and
`IFileBasedUniverse` to `IChainUniverseData`.
* Add base class for options and futures contracts
- Introduced `BaseContract` as an abstract base class for contracts, consolidating common properties and methods.
- Removed ISymbolInterface
* Add minor fix for future options tickers parsing
Added tests
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Clean chain provider classes up
* Remove ZipEntryName other classes and unused code
Removed ZipEntryName class and references across various files.
Removed DataQueueFuturesChainUniverseDataCollectionEnumerator and DataQueueOptionChainUniverseDataCollectionEnumerator classes.
Removed OptionChainUniverseSubscriptionEnumeratorFactory class.
Removed unused code for handling OptionChainUniverse and FuturesChainUniverse in FileSystemDataFeed.cs and LiveTradingDataFeed.cs.
Removed several test files related to enumerator factories and universe data collection.
* Minor changes and cleanup
* Trigger Build
* Trigger Build
* Refactor FuturesContract data handling
Forward price data from bars and ticks stored in private fields for improved memory usage
* Fix: use universe data for market data in FuturesContract
* Update regression algorithms stats after rebase
Added HSI futures universe files
* Sort configs by internal flag
Internals go first
* Throw from option universe data filters for future options
Future options IV, Open interest and greeks are not supported for future options
* Minor changes
* Improve some regression algorithms
* Minor fix for failing unit tests
* Update FOPs universe file header
Removed greeks and IV columns.
Updated FOPs universe files: removed outdated columns.
* Minor unit test fix
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Minor fix
* Add history provider as constructor argument for chain providers
* Update new regression algorithms data points count
* Minor fix for FakeDataQueue
* Add initialize method to chain providers classes
* Minor changes
* Trigger Build
* Trigger Build
* Trigger Build
* Minor fix
* Minor fix
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Add logs to ProcessedDataProvider
* Removed test logs
* Minor fix
* Support downloading options and futures universe files from api data provider
- Fix for division by zero in option assigment check. Adding regression
test reproducing issue. Will now rely on normalized path in universe
selection to clear the cache
* Support registering indicators of type IndicatorBase<BaseData>
* Add IndicatorBase<BaseData> scenario for all the usages of ConvertPythonIndicator
* Add all scenarios in the test case
* Initial solution
* Update unit test
* Explicit conversion if T is IndicatorDataPoint
* Solve review comments
* Add override tests for Beta and Alpha
* Refactor and update unit test suite
* Create unit test for OptionIndicators
* Resolve review comments
* Resolve new review comments
* Add a new default selector
* feat: parsing of universe file path for different security types
* fix: market position in parsing of universe file
refactor: add example of universes file path in xml description method
* test:feat: parsing of universe path
* rename: return variable in ParseUniversePath
* feat: extension to create history request internally
* feat: entity DataUniverseDownloaderGetParameters
* config: downloaderDataProvider project
* feat: create DataUniverseDownloadConfig in download project
* feat: run universe downloader in DownloaderProvider
* test:fix: missing using
test:remove: parameter ctor of DataDownloadConfig
* refactor: run Universe downloader in Main
* feat: GetUniverseFileName by processingDate
* refactor: CreateDataUniverseDownloaderGetParameters with using Start and EndDate
refactor: exception message in DataUniverseDownloadConfig
* feat: new out argument in TryParsePath's LeanData
* feat: RunUniverseDownloader in DownloadDataProvider
* feat: DerivativeUniverseData to write csv file
feat: extension to download universe data with different parameters
* remove: log warn which path when get universe file path
* refactor: DataUniverseDownloaderGetParameters
feat: run Download Universes file in DownloaderDataProvider internally
* refactor: overload TryParsePath only with 4 parameters
* remove: editconfig file in DownloaderDataProvider
* remove: validation on null parameters
* rename: typo DateType to DataType
* refactor: use endDate in ctor DataUniverseDownloaderGetParameters
refactor: use UnderlyingSymbol from base class Symbol property
remove: CreateDataUniverseDownloaderGetParameters()
refactor: put LoadSymbol in base class
clean/remove: DataUniverseDownloadConfig
* refactor: use Time.EachTradeableDay in CreateDataDownloaderGetParameters
remove: CheckMarketOpenStatus
* fix: RunUniverseDownloader with EndUtc in DownloaderDataProvider
* refactor: GetUniverseFullFilePath
* remove: validate historyData like Any
* fix: use date from parsing path in DownloaderDataProvider
* feat: download QuoteBar for Universe file
feat: handle QuoteBar in DerivativeUniverseData
refactor: filter option in final dictionary
* revert: CreateHistoryRequest() in DownloaderExtensions
* feat: use optional parameter SecurityExchangeHours in DataUniverseDownloaderGetParameters
refactor: DataUniverseDownloaderGetParameters
refactor: use Log.Debug instead Log.Trace to prevent spamming
* fix: ordering of download universe data
revert: parameter ctor in DataDownloadConfig
revert:test: use parameter ctor instead of Config.Set
* clean: extra `usings`
* fix: Get DataType in DataDownloadConfig
feat: new ctor in BaseDataDownloadConfig
* Fix IndicatorHistory api for options indicators
Fix for indicators based on OptionIndicatorBase to be able to use IndicatorHistory api. They require proper conversion of indicators from PyObjects
* Fix WarmupIndicator to support multi symbol indicators in Python
* Minor fix for WarmUpIndicator api
* Refactored switch statements for indicator conversion
* Solve comments review
* Refactored WarmUpIndicator to remove duplicated code
* Resolve review comments
---------
Co-authored-by: Jhonathan Abreu <jdabreu25@gmail.com>
* Cache registered custom python security and universe data types
Use this cache to get the correct config for history requests since pythonnet will always pass PythonData and we lose reference to the actual Python type
* Use local repo data for unit tests
* Move unit tests to algorithm history tests
* Use UniverseManager instead of CacheCustomPythonDataType
* Add regression algorithms
* Update regression algorithms to solve issues
* Update regression test and History
* Updated source path to avoid issues with linux
---------
Co-authored-by: Jhonathan Abreu <jdabreu25@gmail.com>
* Adds BroadcastCommand Method
Send a command to all live algorithms.
* Addresses Peer-Review
* Minor tweaks
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Initial draft of the solution
* Remove old format
* Address requested changes
* Add Index option test case
* nit changes
* Address Martin reviews, improve unit tests and solve 8577
* Add unit tests
* Nit change
* Address requested changes
* Address Martin reviews
* Nit changes and more unit tests
* Nit change
* Address requested changes
* Address requested changes
* Initial solution
* Add python support for CompositeIndicator
* Add unit tests and update CompositeIndicator
* Add new test case
* Update TryConvertToIndicator logic
* Solve review comments
* Resolve review comments
* Remove unnessary case
* Use GetIndicatorAsManagedObject instead of ConvertToIndicator
* Add regression tests
* Upgrade IB gateway version to 3010
* Update ib gateway version in config file
* Remove ib-version config
Let the brokerage detect or set the correct ib gateway version
* Add new indexes, including market hours - part 4
* Resolve review comments
* Add an entry to SPDB for ASX
* Add new indexes and update the old ones
* Add entries to SPDB
* Add bank holidays
* Add missing holidays to MHDB
* Add 2025 new year's eve bank holiday
* Add Columbus day and veterans day to bank holidays
* Add missing bank holidays entry
* Modify FuturesExpiryFunctions
* Add missing change
* Remove empty bankHolidays lists from MHDB
* Address review
* Add tests
* Address review
* Add bank holidays from 2009-2024
* Fix unit and regression tests
* Review bank holidays
* Fix tests
* MHDB tweaks
* Remove Bank holidays from CNH, MNH and MIR
* Add setting for bank holidays disabled by default
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Add unit test reproducing the bug
* Improve unit test
* Apply market hours database updates to instance
Make sure entries are updated instead of overridden so that consumers holding the mhdb can pick up the changes
* Improved unit tests
* Update data folder MHDB entries instead of overriding instance
* Optionally reset custom mhdb entries on reset
* Simplify mhdb reset in live trading
Cleanup and minor fixes
* Ensure symbol properties database updates are picked up by consumers
Update SPDB entries instead of overriding them, just like for the MHDB so that consumers pick up updates after refresh
* Fix for symbol properties update thread safety
* Minor improvements and cleanup
* Mhdb and spdb update logic simplification
* Move "force exchange always open" logic to mhdb
* Simplify symbol properties holder and updates
* Refactor security databases to use a common base class
Introduce BaseSecurityDatabase<T, TEntry> to encapsulate common functionality for MarketHoursDatabase and SymbolPropertiesDatabase.
* Cleanup
* Add DualSymbolIndicator unit tests
* Introduce MultiSymbolIndicator for indicators working on multiple symbols
Use the new class as base for DualSymbolIndicator and OptionIndicatorBase.
The OptionIndicatorBase can now detect when ready even if underlying and options market close is different when resolution is daily.
* Accept any BaseData for options indicators
* Improve indicator conversion from python on registration
Also minor fixes
* Cleanup and simplification
* Fix ImpliedVolatility IsReady flag
* Update regression algorithm history count
AutomaticIndicatorWarmupOptionIndicatorsMirrorContractsRegressionAlgorithm history count decreased because options indicators period is now 1 instead of 2
* Address peer review
* Initial solution
* Update regression test
* Update assertions
* Resolve PR comments
* Add more order types to regression test
* Update regression algorithm name
* Update old regression algorithm
* Refactor validation to check buying power only for non-ComboLeg update orders
* Update ValidateSufficientBuyingPowerForOrders
* Update regression algorithm
* Resolve review comments
* Use try out pattern
When re-subscribing a request after a fill forward resolution change, clone Config to avoid memory access race conditions when emitting new tradable date events
* Refactor SetHoldings to return a List<OrderTicket>
* Add expectedQuantities to the testCases
* Update regression tests
* Update name of regressionAlgorithm
* Address review comments
* Update unit test
* Fix symbol changed events emission time
Ensure the symbol changed events are emitted after the securities are processes by the main loop in the AlgorithmManager. This way the algorithm has access to the new symbol security since it would be added by the securities processing logic.
Also, allow all securities to be properly updated, including prices and cash, before emitting the event, so that the any logic (like placing orders) done by the algorithm on the handler has the correct data.
* Minor fix
* Minor change
* Minor change
* Create regression tests and refactor liquidation logic
- Implemented regression tests for order creation and liquidation
scenarios
- Removed LiquidateExistingHoldings method
- Replaced LiquidateExistingHoldings with the general Liquidate method
* Addressed review comments
* Addressed new comments review
* Update default value for 'tag'
* Update ExpectedStatistics
* Identify and liquidate portfolio symbols not included in targets
* Create a new regression test
* Fix minor comments
* Move regression tests to the correct folder
* Refactor GetMatchingSubscriptions:
- Fixed retrieval of minute-resolution quote data for hour-resolution
equity
- Ensured correct retrieval of data regardless of equity resolution
- Created a unit test to verify functionality
* Update unit test and create test cases
* Add new unit test
* Addressed review comments
* Update readme.md to mention .NET 9 instead of .NET 6
* Update readme.md with new versions of python, pandas and wrapt
* Update readme.md for vscode and vs
The delisting time is handled by the DateChangeTimeKeeper now, so there is no need for the SubscriptionDataReader to do any special logic for it.
This was preventing the new tradable date events to be emitted after the delisting date in some cases, like when the day after delisting is not tradable
Co-authored-by: Jhonathan Abreu <jdabreu25@gmail.com>
* Update SPDB with Bitfinex, Kraken, Binance, BinaceUS
* Update unit test
* Added TestCase for CNHT and USD
* Fixed bug in pair comparison logic, corrected concatenation approach
* Add unit test for currency pair match
* Added test cases for currency pair comparison
* Addressed review comments
* Add Martin Luther King'day holiday to CME futures entries
Add unit test for MCL futures expiry function, first contract of 2025 expires on Jan17 but was returning the 20th
* Add holiday to equity and index
And by extension to options and index options
* Add holiday to other entries
* Inicial solution
* Use InitialMargin instead of MaintenanceMargin
* Addressed review comments
* Update unit test
* Refactor GenerateOptionTicker method to handle IndexOption
* Addressed PR comments
* Update unit test
* Add 1/9/2025 holiday to market-hours
* Patched holidays in MarketHours
* Minor fix
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Skip "no tradable dates" log for options added on non-tradable dates
* Skip subscription to selected securities when no tradable dates
Skip selected securities subscriptions when there are no tradable dates available from selection time till end date
* Minor comment
* Remove "no tradable days" error log
* WIP: Refactor Correlation Indicator
* Simplified comparison logic and improved abstraction
* Refactor Correlation and Beta indicators
- Created a base class to handle indicators with dual-symbol
functionality.
- Refactored the Beta and Correlation indicators to inherit from the new
base class.
- Updated unit tests.
- Added a new regression test to validate the latest computed value.
* Addressed review comments
* Update regression test
* Add new unit test to CommonIndicatorTests
* Addressed new review comments
* Implement Squeeze Momentum Indicator
* Create unit test for SqueezeMomentum Indicator
* Update QCAlgorithm with SqueezeMomentum indicator
* Resolved review comments
* Switched to using Keltner indicator instead of manual calculation
* Refactor Squeeze Momentum indicator
- Make the Bollinger Bands and Keltner Channels indicators public.
- In IndicatorBase -> Update, if T is IndicatorDataPoint, then create a new
IndicatorDataPoint.
* Fix issue with regression tests
* Fix unit test
* Fix bug in beta calculation
- Beta is fill-forwarded
- A correct pair considered when they have different symbols and the
same date
- Processing occurs when there are at least period+1 correct pairs
* Address review comments
* Add time zone handling and resolution-based truncation
* Fix regression test for Beta indicator
* Handle resolution for beta indicator
- Remove effectiveResolution and Beta constructor parameter for
resolution.
- Streamlined resolution handling logic for Beta indicator.
- Fixed issues with regression test for Beta.
* Fix issue with period and WarmUpPeriod
* Update unit tests for Alpha indicator
* Fixing minor issues
* Add a variable to track if the previous symbol is the target
* Add regression test for Beta calculation between BTCUSD and SPY
* Add an extra period if the TZ are different
* Add helper method to calculate options expiration date time
This allows to compute accurate time till expiry for greek indicators to be able to calculate on the actual expiration date before market close
* Update tolerance in greek indicators tests
* Minor fix
* Modify helper method to calculate settlement time instead of expiration time
* Cache option expiration date time
* Minor changes
* Minor changes
* Fix for daily options open interest data to be added to contracts
* Cleanup
* Update regression algorithms stats
* Minor fix and cleanup
* Minor fix
* Minor fix
* Update regression algorithms data point count
* Address peer review
* First draft of the solution
* Add regression tests
* Improve solution and add unit test
* Address suggestions
* Improve unit tests
* Nit changes
* Fix regression algorithms
* Implement Hurst Exponent indicator and unit test
- Implemented Hurst Exponent indicator class
- Created unit tests to validate indicator functionality
- Added helper methods for standard deviation and regression line slope
* Optimize Hurst Exponent calculation
- Removed redundat loops and unnecessary operations.
- Precomputed values fro time lag logarithms to avoid recalculation.
- Simplified logic in ComputeNextValue for better performance.
- Updated and clarified comments for improved readability.
* Refactor HurstExponent indicator
- Made _sumX and _sumX2 fields readonly.
- Updated loop for calculating time lag differences to avoid out of
bound.
- Renamed lookbackPeriod with period.
- Replaced DateTime.Now with a fixed date.
* Adjust time slice time for live trading
Adjust time slice to be driven by data, so that consolidators are update at the correct times.
Live trading uses DateTime.UtcNow which might be a few milliseconds after the latest data, causing some race conditions in consolidators scan times.
* Refactor: rounding time slice time down for past consolidators scan (live)
* Cleanup
- Ensure both previousPoint and previousPoint2 are non-zero.
- Prevent division by zero and undefined errors.
- Add unit test to verity the new validation logic.
* Default Data to null for ETFConstituentUniverses.
The data collection will be assigned only if needed. This allows data column to be filtered from dataframes since it will always be null for all constituents.
* Make base data collection aggregator reader fall back to BaseDataCollection
After instatiating the collection type, fall back to the base BaseDataCollection to aggregate data if the type is not a base data collection.
* Minor change
* Minor change
* Update pythonnet to 2.0.41
* Ignore data column for every flattened universe dataframe
* Filter empty collections columns in data frames
* Allow snake case named attributes in PythonSlice
* Remove PythonSlice Data Python class
Pythonnet handles dynamic objects behavior
* Adjust lower-resolution fill-forwarded daily bars when strict end times is enabled
This allows to get fill-forwarded bars with unchanged time stamps
* Minor fixes
* Minor test data changes
* Fixes and comments
* Address peer review and add some fixes
* Minor fix and add regression algorithm
* Minor fix
* Initial implementation of MAMA indicator and tests
* Implemented Mesa Adaptive Moving Average (MAMA) indicator
- Implemented the MAMA indicator.
- Created methods with overloads to support custom and default
fastLimit/slowLimit
- Included necessary logic to calculate the mesa adaptive moving average
based on John's formula.
- Created unit tests to validate MAMA's behaviour.
* Refactor Mesa Adaptive Moving Average(MAMA)
- Separated the logic into smaller functions for better maintainability.
- Improved variable names to reflec their purpose.
- Updated comments to reflect changes and improve readability.
- Consolidated MAMA method overlaod by using default parameters values.
* Refactor variables and update method parameters
- Renamed variables for better clarity.
- Updated method to use 'IBaseDataBar' instead of 'TradeBar'
- Used 'MAMA({fastLimit}, {slowLimit}) instead of just 'MAMA'
* Normalize universe data frames
Universe and (generically BaseDataCollection) data frames are not normalize and unpacked into a data frame, instead of just creating data frames with the universe lists within it
* Fix unit tests and algorithms to expecte new universe dataframe format
* Fixes
* Add PandasConverter.DataFrameGenerator class
* Pandas data frame generator class fixes
* Add comments
* Housekeeping
* Add attributes to mark classes and properties for pandas processing
* Improve pandas properties expanding
Allow and handle duplicate names
* Use PandasData generalization for Lean common data types
* Add points time as column when converting base data collections to data frames
* Cleanup and minor changes
* Minor change
* Pandas data to get type members on demand
* Move Pandas helper classes to their own files
* Minor changes
* Add flatten argument to python history api
This allows users to decide whether they want fully expanded dataframes for universe and other collection data types. Else, master behavior is kept
* Adding missing changes to last commit
* Update Pythonnet version to 2.0.40
* Add flattent argument to algorithm's OptionChain api
* Minor changes
* Housekeeping
* Minor changes
* Bug fix skipping data collection data points
* Add comment
* Set correct exchange time to OptionUniverse instances
* Address peer review and cleanup
* Cleanup
* Minor changes
- Addressed bug causing mismatch between types of correlations.
- Updated C method to include CorrelationType.
- Created regression test to ensure proper validation.
* First potential solution
* Add unit tests
* Address requested changes
* Add unit test reproducing user bug
* Missing update
* Address reviews
* Improve unit test
* Improve unit test
* Nit change
* Nit changes
* Nit changes
* Add Connors Relative Strength Index(CRSI) Indicator
- Implemented CRSI combining RSI, Streak RSI, and ROC
- Added unit tests for CRSI calculation to ensure correctness and
reliability.
- Inclued the spy_crsi.csv file for testing purposes and validation of
CRSI logic.
* Refactor CRSI indicator
- Updated the CRSI logic to use Percent Change of Daily Returns instead
of ROC.
- Modified the ConnorsRelativeStrengthIndex class:
- Updated ComputeNextValue to calculate daily returns and relative
magnitude
- Improved documentation and comments
* Refactor CRSI Indicator:
- Refactored the method that updated the trend streak based on price
changes.
- Created a new method ComputeTrendStreak.
- Added check for null or 0 value in _previousInput.
* Fix comments from code review
- Add _ to priceChangeRatios
- Move _srsi.Update() to the top to avoid duplication
* Let delistings through for internal subscriptions
Without it, previously mapped continuous future contracts for which a position is open never received a delisting event for them to be liquidated and marked as non-tradable
* Update regression algorithm stats
* Revert/remove transaction handler initialization parameters DTO class
* Cleanup
* Fix option universe
Set right time using exchange time zone when reading.
Remove OptionUniverse market hours hack in GetEntry extension method.
* Minor change
* feat: ExtendedRegularTradingHours property in CharlesSchwabOrderProperties
* feat: BrokerageModel, FeeModel of Charles Schwab
* refactor: CharlesSchwab Fee model
remove: SecurityType.Index in CanSubmit CharlesSchwab
* refactor: CharlesSchwabFeeModel (a little bit simpler)
* refactor: CharlesSchwabFeeModel (more simpler)
* Add Average Daily Range indicator and tests
- Implemented AverageDailyRange indicator
- The indicator uses a Simple Moving Average (SMA)
- Created unit tests for the indicator
- Includes example input data and test file (spy_adr.csv)
* Refactor AverageDailyRange Indicator
- Renamed AverageDailyRange to AverageRage for a more generic approach
- Replaced explicit types with 'var'
- Updated method name in test cases
- Placed AR method in the correct alphabetical order
* Solving minor issues with AR indicator
- Replace TradeBar with IBaseDataBar
- Remove unnecessary override methods
* Implement Premier Stochastic Oscillator (PSO)
- Added PremierStochasticOscillator class with two Exponential Moving
Averages (_ema1 and _ema2) for double smoothing.
- Implemented PSO computation:
- Normalized Stochastic (nsk) = 0.1 * (Fast%K - 50)
- Double-smoothed Normalized Stochastic
- Added XML documentation for public methods and key calculations.
This commit introduces a new indicator.
* Add license header and adjust spacing
* Refactor Premier Stochastic Oscillator (PSO)
- Renamed smoothing variables for clarity
- Integrated PSO logic directly in ComputeNextValue
- Used IndicatorExtensions for streamlined EMA chaining
- Updated tests to match refactored logic
* Refactor PSO
- Changed "STO" to "PSO"
- Removed unused imports
- Updated private fields
- Minor code cleanup
* Test
* Fix price shift on continuous future rollover
* Keep track of both data and exchange time in subscription data reader
* Fix for daily and hourly resolutions.
Add daily and hour algorithms
* Fixes for emitting new exchange dates at the beginning and end when data is not available
* Cleanup
* Minor fix
* Minor fix: detect mapping in data reader after advancing time keeper
* Minor fix
* More fixes
* More fixes
* Fixes for universe files
* More fixes for daily resolution data
* Separate DateChangeTimeKeeper to its own file
* Add DateChangeTimeKeeper unit tests
* Minor fix
* Add more DateChangeTimeKeeper unit tests
* Address peer review
* Address peer review
* Address peer review
* Detect mappings using the config NewSymbol event
* DateChangeTimeKeeper performance improvement for time zone conversions
* Clean up date change time keeper
* Cleanup
* Minor regression algo update
* Cleanup
* Cleanup
* Fix perform universe selection on sundays for continuous futures with extended market hours
* Cleanup
* Address peer review
* More cleanup
* More cleanup
* Fix ConsolidateRegressionAlgorithm
- Fix Bitcoin custom data time range. Historical data goes from April 2014
- Fix expected consolidated bar counts
- Fix python Consolidate implementations to respect daily strict times for Daily resolution
* Housekeeping
* feat: sort property in SubscriptionDataSource
* refactor: read lines (with sort opportunity) in TextSubscriptionDataSourceReader
* refactor: xml description of Sort property in SubscriptionDataSource
* revet: TextSubscriptionDataSourceReader changes
* feat: develop UnsortedTextSubscriptionDataSourceReader
* remove: UnsortedTextSubscriptionDataSourceReader
* feat: develop SortEnumerator
feat: use SortEnumerator in SubscriptionDataReader
* revert: TextSubscriptionDataSourceReader changes
* feat: use sort Enumerator in LiveCustomDataSubscriptionEnumeratorFactory
* test:feat: Unsorted Live Data Feed with using Object Store
* test:feat: history CustomData request with unsorted data
* refactor: develop TryWrapper on SortEnumerator
* remove: extra spacing
* refactor: Sort in SubscriptionDataSource
* feat: add RegressionAlgorithms when source return data in Descending format
* refactor: xml description of prop Sort in SubscriptionDataSource
* refactor: SortEnumerator
* test:refactor access modifier in ExampleCustomDataWithSort
* test:refactor: increase performance LiveDataFeedSourcesDataFromObjectStore
* feat: add xml comment of regression algorithm
* test:revert: access modifier to public in internal test class
* test:refactor: use while instead of timer
* Revert "test:refactor: use while instead of timer"
This reverts commit 4fe43da809.
* Fix pandas converter to handle list of data with different symbols
* Properly convert list of data into dataframe
Take into consideration data for multiple symbols in the same list
* Cleanup
* Index dataframes by symbol object instead of SID string
* Add symbol equality operator to compare against object
* Exclude "ID" from option chain dataframe
* Minor fix
* Add greeks columns directly in option chain dataframe.
Also add pass-through properties for greek values in OptionUniverse
* Some cleanup
* Minor fix
* Add new QCAlgorithm.OptionChains() method
- Use OptionChains as output
- Add DataFrame to OptionChain and OptionChains
- Rename Greeks classes
- Add ISymbolProvider for classes that have a symbol (IBaseData, OptionContract)
* Unify QCAlgorithmOptionChain API
Also refactor OptionContract to handle: (1) Actual market data and option price model data, and (2) OptionUniverse data
* Pass symbol properties to OptionUniverse option chain from algorithm
* Format OptionContract for dataframe
* Minor fix
* Add multiple option chains api regression algorithms and other minor changes
* Address peer review
Add NullGreeks class: keep ModeledGreeks as internal as possible
* Minor fix and add PandasConverter unit tests
* Peer review: Non-thread-safe Lazy for Python
* Handle Greeks unwrapping by PandasData
* PandasData cleanup
* Add data and other minor changes
* Unit test fix
* Update Pythonnet to 2.0.39
* Cleanup
* PandasData handling children class members
Address peer review
* Fix: indexing symbol conversion in pandas mapper
* Fix pandas mapper to convert string keys to symbol only when necessary
* Cleanup
* Cleanup
* Add PandasColumn python class to handle proper indexing
This allows propery hash and equality between Symbols, C# strings and Python strings
* Minor fixes
* Symbol cache improvements
* Minor fix for cache miss
* Revert PandasMapper reserved names and improvements
* Minor fix
* Revert reserved names
* Minor fix for Symbol equality operators
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Enhance command support
- Enhance command support, adding link helper methods. Adding and
expanding new tests
- Minor improvement for command str representation
* Minor test fix
* Infer data type from configs in bar count history requests
* Infer data type from configs in bar count history requests
* Minor changes
* Minor fix
* Cleanup
* remove: not used proxy config TradeStation
* feat: missed config of TradeStation in Launcher
* feat: support new OrderTypes in TradeStationBrokerageModel
* Option universe improvements
- Improvement for resolution handling of option universes, affecting
performance in live mode. Adding regression algorithm
* Minor fix for research test
* Fix: adjust option expiry reference date
* Add universe files
* Update data and other minor changes
* Minor changes
* Add regression algorithm summary
* EUREX data
EUREX data model and sample data
* Add EUREX futures expiry function and sample algorithms
* Add EuroStoxx50 futures map and factor files
* Reduce eurex data for repo
* Map eurex market to primary exchange
* Update Euro Stoxx 50 (FESX) map and factol files
* Update Euro Stoxx 50 (FESX) minute data
* Added EURSD data
* Added 2 basic FESX futures algorithms in CSharp and Python (#2)
* Add regression algorithms
* Update regression algorithms and data
* Minor change
* Cleanup
---------
Co-authored-by: paulius-an <118921953+paulius-an@users.noreply.github.com>
* Add new QCAlgorithm.OptionChain method to get full data option chain
* Add extension method to get canonical symbol
* Support future options in new OptionChain method
* Replace option chain provider with OptionChain method in some regression algorithms
* Add new regression algorithms for OptionChain method
* Replace option chain provider with OptionChain method in some regression algorithms
* Minor
* Cleanup
* Minor changes in regression algorithms
* Minor adjustments
* First draft of the solution
* Handle end date better
* Improve unit tests
* Add extra argument in missing method
* Nit change
* Nit change
* Nit change
* Undo changes to C# generic UniverseHistory()
* Address suggestions
* Improve unit test
* Add null checks
* Minor adjustment
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Initial options universe with greeks implementation
* Options universe improvements
* Address peer review
* File based options universe fixes and improvements.
- Adjust OptionUniverse start-end times and period.
- Adapt unit tests and some algorithms to pass with new options universe selection.
* Updated options regression algorithms stats for new universe data
* Updated options regression algorithms stats for new universe data
* Updated options regression algorithms stats for new universe data
* Updated options regression algorithms stats for new universe data
* Updated options regression algorithms stats for new universe data
* Option chain provider with new options universe
* Allow canonical option history requests
* Address peer review
* Address peer review
* Fix symbols parsing in OptionUniverse
* Fix universe selection subscriptions start time to not include extended market hours
* Minor changes
* Minor changes
* Peer recommended changes and fixes
* Update regression algorithm stats
* Update regression algorithms stats and minor fixes
* Fix option chain provider history request
* Round option indicators values
* Added option universe csv header property
* Update regression algorithms stats
* Update regression algorithms stats
* Data fixes and regression algos stats update
* Unit test fixes
* Minor changes
* Option chain handling in live trading data feed
* Minor changes
* Added processed data provider
* Fix thread-safety violation in Slice class
* Minor change
* Update options filter universe API to use OptionUniverse data
Add new filter methods for greeks, IV and open interest
* Option filter universe api updates
* Add OptionUniverse history regression algorithms
* Add regression algorithms for new options filter universe api methods
* Added options greeks data and updated regression algorithms
* Address peer review
* Address peer review
* Add more assertions to new options filter api regression algorithms
* Minor performance improvement.
Reduce greeks binomial model steps to 140
* Minor tests updates
* Greeks numerical models performance improvements
* Greeks numerical models performance improvements
* Revert array pool change for option pricing numerical models
* Update default dividend yield provider depending on option type
* [TEST]
* Add helper method con calculate time till expiration
* Use double in price option numerical models
* Implied volatility calculation improvements
- Adjust root finding method accuracy as a factor of the option price
- Use BSM to get a first guess
* Cleanup
* Some regression algorithms and unit tests cleanup
* Regression tests updates after rebasing from master
* Add universe files
* Self review and cleanup
* Minor regression tests updates after rebase
* Fix: set data time zone to same as exchange tz for options universes
* Minor change
* Minor change
* Fix for live trading options universe selection
* Keep underlying when aggregating collections in BaseDataCollectionAggregatorEnumerator
* Update index options regression algorithms stats
* Minor change
* Address peer review
* Memory usage improvements
* Minor build fix
* Minor changes and test fixes
* Cache symbols in OptionUniverse
* Cleanup
* Fix index option creation in OptionUniverse
* Use cached underlying SID when parsing from string
* Abstract symbols cache to BaseDataCollection
* Return actual underlying symbol when mapping decomposing ICO ticker
* Address peer review
* Minor performance improvements reduce garbage
* Limit Symbols and SIDs cache size to help with memory usage
* Minor fix in symbols and sid cache cleanup
* Build fix
* Lazily parse greeks on individual access
* Cleanup and tests
* Address peer review
* Minor greeks fix
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* refactor: adding OrderId in GroupOrderManger
* feat: new support of OrderTypes in TradeStationBrokerageModel
feat: unSupported OrderTypes in CanUpdateOrder's TradeStationBrokerageModel
* feat: AllOrNone property in TradeStationOrderProperties
* feat: unsupported SubmitCrossZero of Combo Order in TSBrokerageModel
* test:feat: submit / update CrossZero Combo Orders
* feat: new Message Brokerage error message
refactor: use new Message in TradeStationBrokerageModel
* feat: setter of Id in GroupOrderManager
* feat: new constructor of GroupOrderManager
* feat: develop GroupOrderCacheManager service
* fix: groupOrderManger.Id in OrderProvider
* fix: incrementOrderGroupOrderManagerID in BrokerageTransactionHandler
feat: add _groupOrderManagerId in OrderProvider
* remove: extra semicolon
* refactor: prevent increment GroupOrderID
* feat: add new Exchanges
* feat: Try Get Group Combo Orders extension
* refactor: ComboORderType in TSBrokerageModel
* remove: implementing of prop ID in GroupOrderManager
* refactor: UnsupportedCrossZeroByOrderType message
* fix: warning of UnsupportedCrossZeroByOrderType
* fix: several exchanges code based on tradier docs
https://documentation.tradier.com/brokerage-api/reference/exchanges
* feat: add missed Exchange in Global class
* refactor: possible update LimitPrice in TSBrokerageModel
test:feat: validate upddate LimitPrice of ComboLimit Order
* refactor: GroupOrderCacheManager
remove: TryGetGroupCachedOrders from extension
* refactor: exchange SPHR to MIAX_SAPPHIRE
* remove: Exchange BYX cuz It is BATS_Y
* refactor: change position of Exchange C2
* refactor: change constructor's access modifier in class Exchange
* First draft of the solution
* The same check for crytpo is done before
* Fix failing unit tests
* Remove non perpetual crypto futures from SPDB
* Address requests
* Fix failing unit tests
* Adds Overload to AddIndexOption
Simplify usage. We don't need to create/add the underlying explicitly, see AddIndexOption(string, Resolution, string, bool) overload.
* Addresses Peer-Review
- Default market is `null` instead of `Market.USA` allowing for `BrokerageModel` setup.
- `AddIndexOption` and `AddIndexOptionContract` methods now return `indexOption` objects.
* Fixes Logic Bug
* Minor indicator history fix
- Take into account warmup period when used, the last bar of the warmup
period is counted as a data point.
- Fix bug where in some cases the last data point could of been missed
and the indicator not updated with it
* Minor tweak
* Fix dividend yield calculation
* Get security price as input for dividend yield provider
* Account for splits in dividend yield provider
* Fix split adjustment for dividend yield calculation
* Use right dividend yield in options indicators
* Implemented the vortex indicator.
* changed the vortex.cs calculations
* chore: Refactor VortexTests.cs and QCAlgorithm.Indicators.cs
* Refactor VortexTests.cs
* made changes according to the comments
* used indicator exxtension methods
* made changes in the naming convention and added condition for division by zero.
* Using composite indicators to clean up
---------
Co-authored-by: Jhonathan Abreu <jdabreu25@gmail.com>
* Default daily precise end times
- Enable by default daily precise end times. Updating stats
- Minor fix for algorithm manager consolidator updates, adding new regression test
asserting behavior and updating others
- Minor fix for SubscriptionData creator avoid round down on warmup if
not appropiate
- Adjust consolidators to emit on daily strict end times if requested
daily resolution and setting enabled
- Updating regression algorithms
* Skip daily data on extended market hours
* Some cleanup and self review
* Revert unrequired change
* First draft of the solution
* Reduce duplication and add more properties
* Add regression tests
* Add xml docs
* Add unit tests
* Improve regression and unit tests
* Address requested changes
* Address request and improve unit tests
* Add unit test and fix bug
* Improve unit tests and implementation
* Nit change
* Address reviews
* Improve regression tests
* Address requests
* Fix bug
* Address requests
* Implements ChandeKrollStop
* readonly
* Move IsReady/WarmUpPeriod above the constructors
* remove private _period
* minor
* rename Stops to Short/Long
* using Maximum/Minimum
* fix tests
* rename some variables and use movingAverageType for ATR
* movingAverageType
Commit d24f665ee4 removed the
Engine/AlgorithmManager.cs support for OnData(TradeBars), making these
methods dead code. Hence, some of these algorithms no longer placed
orders. Fix by changing OnData(TradeBars) to OnData(Slice). Files that
use the TradeBars argument or use OnData(Dividends) have the same
trouble; leave them for future work.
* feat: new constructor of AlgorithmNodePacket
* refactor: extract JobQueue configuration
* remove: not used `using` in IDataDownloader
* feat: create BrokerageDataDownloader
* Revert "refactor: extract JobQueue configuration"
This reverts commit 5778936b71.
* Revert "feat: new constructor of AlgorithmNodePacket"
This reverts commit d7a565ff76.
* feat: new config `data-download-brokerage` in DataDownloadProvider
* refactor: initialize in BrokerageDataDownloader
* remove: not used `using` in Program's DataDownloadProvider
* remove: not used ref on QuantConnect.Queue proj
* refactor: use default market based on SecurityType
* refactor: MarketName in DataDownloadConfig struct
test:feat: validate MarketName
* feat: support Canonical Symbols in BrokerageDataDownloader
* remove: not used command arguments
* feat: init CacheProvider of IOptionChainProvider in Downloader
* feat: add brokerage message event in BrokerageDataDownloader
* Set best recommended model based on option style.
The best option model will be used in options indicators based on option style if not specified
* Reduced Greeks/IV numerical models steps to 100
* Minor fixes
* Minor changes and improvements
* Changed default pricing model for IV
* Minor performance improvements
* Minor performance improvements and fixes
* Minor change
* Minor regression algorithm fix
* Update greek indicators internal IV indicator only when not user provided
* Revert unnecessary changes
* Minor changes
* Cleanup
* implement Rogers-Satchell volatility
* naming RSVolat to avoid confusion between volume and volatility
* fix InitializeIndicator call
* fix AcceptsRenkoBarsAsInput test
* add symbol parameter to InitializeIndicator
* Update RogersSatchellVolatility.cs
* some fixes after review
* Return 0 when Open High Low or Close is 0
* IBaseDataBar
* Made changes in the ParseOptionTickerOSI function. Now it can handle both with and without space
* added new function GenerateOptionTickerOSICompact and added the test cases for it, expanded test cases for ParseOptionTickerOSI
* added assertions for strikePrice and date, corrected the indentation
* First draft of the solution
* First draft of the tests
* Nit changes
* Nit change
* Add improvements
* Improve unit tests
* Regression algos
* Nit change
* Nit changes
* Improve unit tests
* Improve unit test
* Nit change
* Address reviews
* tests passing except ResetProperly and WarmsUpProperly
* doc
* minor fixes
* doc - return
* fix InitializeIndicator call
* workaround ResetsProperly
* fix WarmsUpProperly test
* remove WriteLine
* remove WriteLine
* cr
* fix data
* open high low defaults to close when these columns don't exist into data
* simplify using ternary operator
* better fix for ResetsProperly
* fix some code conventions issues
* fix some review issues
* Update StochasticRelativeStrengthIndex.cs
* WIP
* All tests are passing except AcceptsRenkoBarsAsInput
* Update QuantConnect.Tests.csproj
* comment formating and use input.EndTime
* fix AcceptsRenkoBarsAsInput test
* fix InitializeIndicator call
* add symbol parameter to InitializeIndicator
* FI above FISH
* Improve indicator warmup
- Allow indicator warmup to handle multiple symbols, used for option
greeks. Adding regression test
- Improve indicator history to handle specific T types. Adding unit test
- Remove period, and historical underlying volatility, from IV so
simplify warmup behavior
* Address review, fix minor issues
* Implement new indicator IBS
* remove using System
* typo
* doc fix
* Implement new indicator IBS
* remove using System
* typo
* doc fix
* inherit TradeBarIndicator and return 1m when High=Low
* duplicate spy_with_ibs
* returns 0.5 when High=Low
* Minor fix for Mis-Aligned Timestamps from IndicatorHistory
- Minor fir for mis-Aligned Timestamps from IndicatorHistory. Adjusting
unit test to reproduce issue.
* Add another assert to indicator history tests
* Minor fixes for IndicatorHistory and IV
- Minor fix for IndicatorHistory, use ScaledRaw data normalization mode
for option indicators. Adding unit test
- Expand the IV expected upper range to 400%, given values can be higher
than previous value of 200%
- Adjust IV to use smoothing function only when both option calculations
succeed
* Updating regression algorithm expected greeks, minor change
- Remove universe selection on dates with extended market hours only
- Updating regression algorithms
- Expand date & time rules API to support specifying whether extended
market hours only dates are desired or not
* Implement new indicator ZLEMA
* Add script to generate reference data from talipp
* Some review fixes
* description
* use isReady
* use Delay(...)
* simplify ComputeNextValue
* Minor tweaks addressing review
* replace DateTime.UtcNow for specific date time instances
* remove unused imports
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Minor indicator test suite improvements
- Simplify and improve indicator test suite assertions
- Reduce indicator suite run time
* Fix DividendYieldProviderTests dependency
* Fix half of the CA1051 warnings
This warning is about not declaring visible instance fields. There are
something about 500 warnings in the solution, mostly in the QuantConnect and QuantConnect.Algorithm.CSharp projects. I aim to fix one of them in this PR and the other half of them in a second one. To fix it, I'm changing the visible instancce fields for properties.
* fix bugs
* Addressing minor reviews
* More minor fixes
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Add indicator helper methods for base Algorithm
- Add indicator helper methods for base algorithm. Adding new unit tests
* Expand indicators helper methods
- Expand indicators helper methods, adding support for C# and custom
indicators too. Adding unit tests
- Expand indicators helper methods to support multiple symbols as input.
Adding unit tests
- Improve conversion of symbol enumerable from python to C# adding unit
tests
* Address reviews
- Keep old QB.Indicator methods for backwards compatibility
- Rename new API to IndicatorHistory, matching
FutureHistory/OptionHistory
- Add new regression algorithms
- Minor improvement to DynamicData so it supports snake name access
* empty
* refactor: orderType for SecondCrossZeroOrder
* refactor: handle some specfic orderStatus in CrossZeroOrder process
* refactor: Handle specific OrderStatus in SecondPartCrossZeroOrder
* refactor: handle order status
revert: convertOrderType of CrossZeroSecondPart
* fix: use right collection when removing cross zero order class
* Fix CA1819 and CA1002 warnings
Changed the type of Languages statistic in regression tests from
Language[] to List<Language>. By doing that, the warning CA1819 was
removed but then the warning CA1002 was raised. However, this warning
was expected to be excluded from QuantConnect.Algorithm.CSharp.
* Improve implementation
* Simplify code
* Fix bugs
* Define Iron Butterfly and its buying power model
* Add unit tests
* Add regression test
* Address peer review
* Add short iron butterfly and short iron condor
* Add unit tests on short strategies
* add regression algorithms
* Invert direction of iron butterfly
* First draft of the solution
* Fix bugs
* Fix bugs
* Fix another overloaded method
* Add new unit test
* Address requested changes
* Address requested changes
* Minor tweaks
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Add algorithm unit tests
* Add Brokerage unit tests
* Add Common unit tests
* Fix unit test bugs
* Add remaining unit tests
* Address requested changes
* Address requested changes
* Fix failing unit test
* Address requested changes
* feat: TryCrossPositionOrder
feat: helper methods to CrossPosition
* refactor: get rid CrossOrder +UTests
* feat: test simulator of cross zero orders request
* remove: callback from TryCrossPositionOrder
rename: TryCrossPositionOrder -> TryCrossZeroPositionOrder
feat: handle event of CrossZeroOrder
feat: new prop in CrossZeroOrderRequest
test:feat: write working test imitate of CrossZeroOrdering and assert different params
* feat: temp collection to keep CrossZeroOrder with Lean
* refactor: flag which indicate should we place crossZero order with LeanEvent or not
* refactor: change access modifier
* fix: CrossZero tests
* refactor: property collection instead of variable
* feat: lock context of CrossZeroOrder
feat: helper method for CrossZeroCollection
* refactor: test of cross brokerage ordering
* refactor: CrossZero Brokerage
* fix: lock area
* feat: Update CrossZero Ordering
refactor: CrossZeroOrders tests
* feat: additional test case for CrossZeroOrdering
* feat: Invalid OrderStatus when FirstPart of CrossZero ordering try to place
feat: message property in CrossZeroOrderResponse
refactor: CrossBrokerageTest a little bit
* feat: PossibleUpdateCrossZeroOrder
* refactor: UpdateCrossZeroOrder
* refactor: TimeOut for resetEvent
* refactor: crossZeroOrder change DateTime UtcNow on specific time
refactor: change action scenario for Filled FirstPart of CrossZeroOrder
* refactor: change firstPartQuantityproperty to link of FirstPartOfCrossZero order completely
fix: crossZeroOrder test put orderStatus internally
* feat: additional orderType in CrossZeroOrder tests
* test:remove: not use GetOrderCrossesZero()
* refactor: extra paddings
* rename: TryGetUpdateCrossZeroOrderQuantity instead of IsPossibleUpdateCrossZeroOrder
* refactor: CrossZeroOrder process and models
* remove: interface ICrossZeroOrderRequest
* feat: TradeStation NotSupportUpdate of CrossZeroOrder quantity
* feat: add license
remove: extra validation of variable
* refactor: add validation of quantity update in TradeStationBrokerageModel
* test:feat: canUpdate of TradeStationOrder
* refactor: reuse OrderEvent when we handle FirstPartCrossZeroOrder
* remove: assignment of filled quantity of FirstPartCrossZeroOrder
* Add support for FTP notifications
* Add optional parameters for SFTP notifications with private key
* Add optional "secure" parameter for FTP notifications.
Improve FTP notifications constructor and input checks.
* FTP notification improvements.
Add public key property.
Add multiple constructors and methods for FTP and SFTP notifications with either password or SSH keys.
Encode file contents.
* Address peer review
Add support for FTP notification with string contents
* Minor changes
* Minor changes
* Remove public key argument
* Fixes and minor changes
The second and third parameter are `fastPeriod` and `slowPeriod`, but we had `slowPeriod` and `fastPeriod`, which is inconsistent with the class contructor and other indicators with `fastPeriod` and `slowPeriod` parameters.
* Dynamic spdb
* Minor changes and unit test
* Get MHDB and SPDB refresh period from config
* Add databases refresh period to algorithm settings
* Address peer review and add unit test
* Add Protective Collar Option Strategy
* Add buying power model for Protective Collar
* Add unit test for Protective Collar
* Add regression test for Protective Collar
* Add unit test on position group buying power
* allow same strike for later abstraction
* minor bug fix
* Set up conversion option strategy
* add margin requirement
* add unit tests
* Add regression test
* add reverse conversion definition
* add reverse conversion and refactor conversion/collar margin model
* added/modified unit tests for conversion/reverse conversion
* minor bug fix on unit test, add regression test
* Address peer review
* update new set of IB testing data
* Fix fill forward resolution adjustment on removal
- Fix fill forward resolution adjustment on subscription removal. Adding
regression algorithm reproducing issue
* Minor fix for undeterminism behavior on security removals and FF res change
* Further improvements and determinism
- Improve regression algorithm add, remove and readd future, asserting
FF resolution changes
- Adjust universe selection to be deterministic and avoid unnecessary
FF resolution changes
* Daily data Time & EndTime Improvement
- Adjust daily data Time & EndTime to actually reflect the time of the
data used, for example US Equity from 9.30 to 4PM. Adding new unit and
regression tests
* Refactor solution to use enumerator
- Refactor daily strict end times solution to be through enumerator
usage, so it applies for history providers too
* Minor fixes
* Revert fill forward enumerator change
- Revert FillForward enumerator causing stats changing, enhancing unit
tests
* Some cleanup
* Improve handling of live trading FF enumerator
- Improve handling of live trading FF enumerator, by adding support for
bars to arrive with a delay so we can handle auction close/option
prices or data providers which might have some delay making the data
available. Adding new unit tests asserting the behavior
* Round combo order prices
* Combo limit order round fix: use smallest security min price variation
* Minor unit test change
* Improvements and unit tests
* Minor change
* Cleanup
* Handle expired options
* unit test
* Move Zero Checks to Single Location
---------
Co-authored-by: Alexandre Catarino <AlexCatarino@users.noreply.github.com>
* Fix: python consolidator reference lost when wrapped
* Make BasePythonWrapper equatable depending on the PyObject underlying reference
* Fix: TryConvert PyObject instances for generic C# types
* Minor unit tests fixes
* Address peer review and add unit tests
* Fix: properly adjust prices after a split
* Minor fixes
* Minor: Adjust cached data on split
* Minor change
* Expand regression algorithms assertions
* refactor: modifier static to parse enum and ignore warning
* feat: group symbol when download cannonicial data
feat: log total time execution of download provider
test: right keeping of cannonicial data
* feat: parameter ctor in DataDownloadConfig
* test:feat: additional assertion of wrote data in file
* feat: static FilterGroupDownloadDataBySymbol
* refactor: extract downloader folder path
* try: map file false
* feat: get rid of dataCacheProvider in DownloaderDataProvider
* refactor: get rid mapFile flag
* CI test fix
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Fix: source custom data from remote CSV zipped file
* Fix: Custom data sourcing zip entry names from remote locations
* Address peer review
* Add unit tests
* Minor fix
* Address peer review
* Minor fixes
* Return byte array in IApi.DownloadBytes
* Allow sourcing custom data from specific csv entry in a zipped remote file
* Minor change
* feat: init Download proj
* feat: update docker file to copy download proj
* feat: parsing in cmd arguments
* feat: DataDownloadConfig model
feat: add config file
refactor: logic of parsing config
* feat: overrid ToString in DataDownloaderGetParameters
* remove: not used argument command
* feat: calculate downloading % (without ETA time)
* fix: dockfile copy path for DownloaderDataProvider proj
* feat: additional Property in .csproj
* remove: DataProviderFullName prop from DataDownloadConfig
* remove: extra package (ref from Configuration proj)
* feat: ETA calculation of downloading data
* Minor tweak
* rename: namespace of downloaderDataProvider to generic style format
* test: ETA from DownloaderDataProvider
* fix: calculate ETA like real downloading process
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
- Make sure calling CSharp methods, fields, properties releases the GIL,
this was not hapenning for fiels & properties, adding unit test
reproducing the issue
* pep8 conversion of alphas
* pep8 conversion of benchmarks
* Various fixes and rebase
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Initial boilerplate code
* Fix build failure
* Initial iteration for the code for Derivative Oscillator code
* Add foundation for unit tests
* Add reset method for test
* Add test case
* fixed test cases, all tests pass
* Put placeholder for IsReady change
Not implemented, as test cases fail when change is uncommented
* Update documentation and change class name
* Change names for Derivative Oscillator to more meaningful
* Add DO extension to create the indicator
* Provide variable definitions with var, replace old test file with new
* Use chaining for indicators creation
* Add test based on external data
* Change test file
* Fix WarmUp test + alphabetical order + rename indicator class
* Make all tests run except for external data tests
* Fixed warmup period and fixed the ResetsProperly test case
* Removed unused directives
* Add output directory for Derivative Oscillator test
* Minor tweaks
---------
Co-authored-by: HaentjeGH <l.m.j.haenen@student.tue.nl>
Co-authored-by: ivkozlov <ivkozlov@ozon.ru>
Co-authored-by: DanielVanRijswijk <d.j.v.Rijswijk@student.tue.nl>
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Improve python exception message for cases when derived class uses member names already defined in base C# classes
* Update pythonnet version to 2.0.33
* Update error message
* Pythonnet version bump to 2.0.34. Minor unit test fix
* Enhance pythonnet overload tests
---------
Co-authored-by: Jared <jaredbroad@gmail.com>
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
- Pep8 adjustment for GoodTilDate, so it behaves like `Expiry` static
readonly funcs which will be binded to both snake case and upper
constant format, `good_til_date` & `GOOD_TIL_DATE`
* feat: support snake-case style Python QCAlgorithm implementations
* feat: add unit tests and minor fixes
* feat: implement new BasePythonWrapper class for python wrappers.
Used to cache methods and contains invoke functionality
* feat: make python wrappers implement the new base class for pep8 style support
* feat: keep overriden methods in Algorithm Python Wrapper
* feat: add unit tests for custom models algorithms with PEP8 style
* Bump pythonnet version to 2.0.30
* fix bugs and address peer review
* Address peer review
* Minor revert
* feat: StubsIgnoreAttribute for ignoring members or classes by the stubs generator
* Minor fixes
* Minor fix
* Minor fix
* Bump pythonnet version to 2.0.31
* Added Greeks.Lambda_ alias of Lambda for python compatibility.
Remove unused method
* Add support for auxiliary data history request
- Add support for split/dividends/margin interest history requests.
Adding regression algorithms
* Expand auxiliary history regression tests
- By default Time rules yield times in UTC TZ, already correctly handled
by 'ScheduledUniverseSelectionModel' but this wasn't fixed in ScheduledUniverse
* Minor api improvements
- Adding Universe.Selected to hold the current set of selected symbols.
Adding regression tests
- Adjusting TimeRules and DateRules to fallback to the MHDB if the
security hasn't been added. Updating existing regression tests
* Expand regression test assertion
* fix: download year file data only for hour and daily resolutions for options and index options
* minor fix: index options regression tests expected status
* Minor change
* minor: update regression algorithms stats
* Adding new algorithm data handling options
- Adding new QCAlgorithm data handling method options to override.
Adding Py/C# regression tests
- Removing old OnData Type specific invoker methods
* Address reviews. Remove old OnData C# only methods
* Add Scheduled Universe Selection
- Adding `UniverseSettings.Schedule` that will allow users to
specify a custom selection schedule which is independent of the
underlying data frequency
- Adding unit and regression tests
* Add live trading schedule time shift
* Support CFDs in IB
* Add new currency pair util method
* Minor changes to IB fee model and tests
* Add CFD USA market hours
* Add new Market interactivebrokers for cfds
* Add sample algorithm illustrating manually adding market hours to MHDB
* Update MHDB and SPDB
* First draft of the solution
* Add missing changes
* Remove the new KPI's from report
* Fix bugs
* nit change
* Add improvements
* Fix regression tests
* Solve bugs in the regression algos
* Fix regression tests bugs
* Expand unit tests and add minor changes
* refactor: map files in lean/data folder
* feat: Indexers in MapFile
* feat: extension for mapFile and downloader param
* refactor: getAllTickers from MapFiles
* feat: addition out variable in TryParsePath
fix: tests with new variable
* refactor: DownloadDataProvider with getting all ticker with different DateTime Ranges
feat: test
* revert: "refactor: map files in lean/data folder"
This reverts commit 41fa26eb8627446028d4f2d4c536511ea82a9a48.
* revert: "feat: addition out variable in TryParsePath"
refactor: some actual changing
This reverts commit 9de482bd6490eecd1eeee83e035dfc1ece8d6415.
* revert: "feat: Indexers in MapFile"
fix: actual changes
This reverts commit 393181c6316a354a8cdcbfb093707cb545383a8a.
* remove: high performance Any validation
feat: validation on ticker start with specific sign
fix: option test
* feat: validation on mapping
refactor: when resolution less hour
test: new test cases + validation of SecurityType
* refactor: add additional day when return date range from MapFile
fix: FirstTicker instead of Permtick when generate new Equity
feat: validation of request Date in DownloaderDataProvider
refactor: by style proj DownloaderDataProvider
test:fix: starting Date in TestCases
* remove: duplicate code
* fix: check of income data.Time with requested Start/End-Date
* test:fix: download test cuz we can not get/write future data
* remove: extra Symbol Create mthd
* test:feat: validate of TryParse returns correct symbol
* remove: excess if condition
* test:remove: extra chaning
connected # 3a62adf54c
* refactor: split big method to small ones (add readability)
* fix: typo in xml description
* refactor: get the correct symbol value
* refactor: use local time instead of Utc in download Param extension
refactor: use struct instead of class
* test:fix: convert time to Utc from local time
* Adds Rebate and Fee Rates to Shortable Provider
If the provider does have this information, the rates are zero.
* Adds Helper Methods to QCAlgorithm
- Improve summaries.
- Express values as rates instead of percentages.
- Adds headers to files to emulate real data
* Clarify the Data Format
* Removes Helper Methods from QCAlgorithm
* Removes `Shortable` and `ShortableQuantity` from Example
Algorithms should prefer getting the information from the `Security.ShortableProvider`.
* Add schedule manager unit tests
* Add schedule manager unit tests
* Fix overriding security exchange hours
* Minor comments changes
* Minor change
* Add SecurityExchangeHours.Update method to set without changing the reference
* Minor change
* Refactor universe historical data source
- Add new universe history API methods
- Refactor QuantBook UniverseHistory to use the universe selection
itself instead of a given func
- Refactor and rename fundamental types
- Refactor AddUniverse API to handle universe collection data which
holds another type internally, like fundamental
* Fix minor bug causing ApiDataProvider not to serve Bitfinex universe data
* Further improvements to add universe API
* Handle no selection function
* feat: helper to return old name of symbols
feat: test
refactor: test a little bit neat
* feat: SplitHistoryRequest() with different mapped symbol
feat: test
refactor: RetrieveSymbolHistoricalDefinitions
* refactor: split historyRequest by historical symbol name
refactor: change location of extension
refactor: change location of test extension
* revert: extra changes
* fix: validation on RequiresMapping in SplitHistoryRequest
feat: add Future Symbol tests
* move: extension method to another file
* fix: wrong extension call param position
* fix: wrong extension call param position in test
* refactor: IEnumerable foreach to more simple form
feat: add TickerDateRange struct instead of Tuple
fix: test
* refactor: map extension alway returns empty result
* refactor: SplitHistory extension returns original HistoryRequest
* feature: new extension for RequiresMapping except Futures
* fix: validation of symbol map file to contains something
* refactor: extension test cases
* feat: assert of right ticker in tests
* fix: split historical request with using Local Time instead of UTC
fix: test description
* feat: test helper IEnumerable of TimeZones
* test: validate right EndDate with different TimeZones
* remove: validation of end of date
test: several new test cases
* feat: additional test with specific dateTime
* remove: extra Extension method validation on future and mapping
* Allow mirror contract and smoothing IV
* Python wrapper on smoothing IV
* helper method update
* Add forward tree model
* modify and add unit tests and data
* address peer review
* address peer review
* Update symbol-properties-database.csv
Updating Symbol properties database to reflect kraken API information.
#### Description
Updated the market_ticker column of Symbol properties database to reflect information as per Kraken API.
#### Related Issue
Close#7794
#### Motivation and Context
We updated symbol mapping for the Kraken market.
#### Requires Documentation Change
No
#### How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran to -->
<!--- see how your change affects other areas of the code, etc. -->
#### Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Refactor (non-breaking change which improves implementation)
- [ ] Performance (non-breaking change which improves performance. Please add associated performance test and results)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Non-functional change (xml comments/documentation/etc)
#### Checklist:
<!--- The following is a checklist of items that MUST be completed before a PR is accepted -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [X] My code follows the code style of this project.
- [X] I have read the **CONTRIBUTING** [document](https://github.com/QuantConnect/Lean/blob/master/CONTRIBUTING.md).
- [X] I have added tests to cover my changes. <!--- If not applicable, please explain why -->
- [X] All new and existing tests passed.
- [X] My branch follows the naming convention `bug-<issue#>-<description>` or `feature-<issue#>-<description>`
<!--- Template inspired by https://www.talater.com/open-source-templates/#/page/99 -->
* Update symbol-properties-database.csv
- Update as per peer review
* Correction
* Implement VaR in PortfolioStatistics
* Added ValueAtRisk indicator
* Refactored ValueAtRiskTests
* Added returns column to test data
* Change indicator helper method name
* Addressed comments
* Chande Variable Index Dynamic Average (VIDYA) Indicator
* Improvements
* Improvements
* Final Changes
* Editted, Retested
* Comments
* Minor Corrections
* Updated Datetime Format of test data to US Standards
---------
Co-authored-by: Pronoy Chowdhury <rokstarr.dbz@outlook.com>
* Make sure continuous futures universe selection is performed right away on the start of the algorithm.
This ensures the Future.Mapped is not null
* Minor changes
* Minor fixes
* Update regression algorithms stats
* Add DC futures map file
* Update DC futures map file
* Update regression algorithms stats
* Include ETFs for immediate universe selection
* Include Coarse Fundamentals for immediate universe selection
* Include Futures Chain Universe for immediate selection
* Add Option Chain Universe test
* Include custom universes for immediate selection
* Minor updates
* Include constituents universes for immediate selection
* Minor update
* Minor changes, docs and stats updates
* Minor changes
* Minor changes and clean up
* Minor regression algorithm stat update
* Models OZN: 10-Year US Treasury Future Options
<!--- Provide a general summary of your changes in the Title above -->
#### Description
We added future option for OZN
#### Related Issue
Close#7783
<!--- This project only accepts pull requests related to open issues -->
<!--- If suggesting a new feature or change, please discuss it in an issue first -->
<!--- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!--- Please link to the issue here: -->
#### Motivation and Context
We added future option supported by CME
https://www.cmegroup.com/markets/interest-rates/us-treasury/10-year-us-treasury-note.contractSpecs.options.html#optionProductId=317
#### Requires Documentation Change
No
#### How Has This Been Tested?
I tested the change using fake files to check if mapping is correct.
#### Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Refactor (non-breaking change which improves implementation)
- [ ] Performance (non-breaking change which improves performance. Please add associated performance test and results)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Non-functional change (xml comments/documentation/etc)
#### Checklist:
<!--- The following is a checklist of items that MUST be completed before a PR is accepted -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [X] My code follows the code style of this project.
- [X] I have read the **CONTRIBUTING** [document](https://github.com/QuantConnect/Lean/blob/master/CONTRIBUTING.md).
- [X] I have added tests to cover my changes. <!--- If not applicable, please explain why -->
- [X] All new and existing tests passed.
- [X] My branch follows the naming convention `bug-<issue#>-<description>` or `feature-<issue#>-<description>`
<!--- Template inspired by https://www.talater.com/open-source-templates/#/page/99 -->
* Add unit tests
---------
Co-authored-by: Ashutosh <ashutoshrana171@gmail.com>
* Improve consolidator scan
- Improve consolidator scan to be triggered even if no data is present.
Adding new regression test, updating existing
* Adding more tests & minor tweaks
- Adjust live equity delisting to be map files based like splits and
dividends. Adding unit test
- Expand existing unit tests asserting delistings in live mode, set
securities to not tradable
* Delay security removal when there are pending settlements
* Add unit test
* Minor fixes
* Minor fix
* Replace HasUnsettledFunds method with UnsettledCash in ISettlementModel
* Minor fix
* Minor fix
* Address peer review
Replace ISettlementModel UnsettledCash property with GetUnsettledCash method that returns a CashAmount instead of a decimal to include the currency
* Minor fix
* Address peer review
* Minor chart improvements
- Add base series tooltip template. Adding unit tests
- Reduce null chart points to a single addition
* Add null values always
* Fixes & minor tweaks
* Serialize ChartPoint as array
* Portfolio state chart
- Cleanup and renames
- Add series.IndexName. Uodating unit tests
- Add Chart.LegendDisabled. Adding new unit tests
- Add ChartPoint.Tooltip. Updating unit tests
- Minor compression tweak. Adding unit test
- Add ChartJsonConverter. Adding unit tests
* Minor chart serialization order tweak
* Refactor portfolio state sampling and storing
* Move PortfolioMargin into a lean side chart
* Implement backtest self-naming and self-tagging
* Minor unit test fix
* Minor api changes
* Minor changes
* Send packet messages when algorithm's name or tags are updated
* Minor fixes
* Address peer review
* Backtest name and tags API calls
Plus, first version of CI for API tests
* Add endpoint to update a backtest set of tags
* Minor GH actions change
* Address peer review
* Add algorithm naming and tagging unit tests
* Minor backtest API changes
* Models All Index Futures Listed in CME
We added:
NIY, ENY, FT1, FT5, E3G, FTU, TPD, TPY
* Models All Index Futures Listed in CME
We added international index futures:
NIY, ENY, FT5, FT1, E3G, FTU, TPD and TPY
* Remove unused imports
* Address peer review:
- Corrected serial contract logic
- Corrected minimum price fluctuation
- Corrected “Yen” to “JPY”
Added equity index futures:
RX, ESG, RS1 and SDA
* Replace currency string formatting with Lean currency symbols.
Also clean up orders tags from having redundant information.
* Setting default order tag only after price currency is properly set
* Minor fixes and new unit tests
* Minor fixes
* Fix bug in ETFConstituentUniverseSelectionModel()
The bug was raised after calling an overload constructor using just the
ticker as parameter. Since the universeFilterFunc was null, the method
Extensions.TryConvertToDelegate<T> returned true and thus, the filter
func assigned was the one returned by the method
`ConvertToUniverseSelectionSymbolDelegate()` which used the null
filterfunc.
* Handle potential future bugs
* Initial draft of the solution
* Add improvments
* nit changes
* Improve implementation
* Enhance unit tests
* Add one more unit test
* Fix failing tests and update MHDB
* Add requested changes
* Nit change
* Add tweaks to MarketHoursDatabase class
* Solve bug and add unit tests
* Update unit tests
* Address requested changes
* Handle potential bug
* Nit change
* Minor review
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Handle/filter brokerage side orders
* Add unit tests
* Minor fix
* Add unrequested security for new brokerage-side orders
* Add unit test for algorithms overriding the brokerage message handler
* Add unit test for python algorithm overriding the brokerage message handler
* Minor change
* Address peer review
A more specific entry might contain some of the same early close/late open dates as the common entry but with different times. This makes sure this is handle and the more specific ones take precedence.
* MaximumSharpeRatioPortfolioOptimizerTests
* MinimumVariancePortfolioOptimizerTests
* UnconstrainedMeanVariancePortfolioOptimizerTests
* Addressed comments
* Address comments v2
* Refactored tests with base test class. Added Python wrapper tests
Base test class and python wrapper tests
* Add IV indicator
* Fix bug
* Add QCAlgorithm helper method
* Add unit tests
* Add OptionPricingModelType, as option for greeks & IV estimation
* Nit, and fix bug
* Address peer review
* Fix bug
* Fix bug on CRR
* Ensure test indicator initiate correctly
* Address 2nd review
* Check for FOPs and index options as well
* Add comparison with QuantLib
* CRR not follow BSM
* minor bug fix
* Asset plot improvement
- Add optional symbol for chart instance, to create an asset plot
- Fix Order PriceAdjustmentModel deserialization. Updating unit test
* Add series ZIndex. Adding unit test
* Fix clone, adding more tests
* feat: new `TradingDaysPerYear` prop in IAlgorithmSettings
* feat: getTradingDayPerYear by BrokerageModel
feat: init algo tradingDaysPerYear in brokerage|BacktestingSetupHandler
refactor: nullable `TradingDayPerYear`
* remove: hardcoded value:252 in statistics
* test: `algorithm.Settings.TradingDaysPerYear` for different brokerage
fix: tests calculation of `AnnualPerformance()`
* fix: Report generator by tradingDayPerYear
* fix: hardcoded value in PortfolioStatisticsTests
* fix: tradingDayPerYear in AnnualPerformanceTest
remove: extra spacing
* feat: backwards compatibility for TradingDaysPerYear
* feat: several UTest for AlgoSetting's TradingDaysPerYear prop
* fix: existed algo + test
* feat: regression algo with checking of tradingDayPerYear in OnEndOfAlgorithm()
* refactor: GetBrokerageTradingDayPerYear -> SetBrokerage... in BaseSetupHandler
refactor: carry out tradingDayPerYear at the top level and pass variable
* remove: default valuine in test param
remove: extra comment.
* fix: missed init TradingDaysPerYear in report.portfolioAlgo
fix: hardcoded value to tradingDaysPerYear
* fix: nullable TradingDaysPerYear in AlgorithmConfiguration
feat: UTest for PortfolioLooperAlgo
feat: add advanced description
* feat: test PortfolioStatistics with different tradingDaysPerYear
* revert: PortfolioLooper with TradingDaysPerYear implementation
* revert: nullable TradingDaysPerYear in AlgoConfiguration
remove: SetBrokerageTradingDayPerYear in BaseResultHandler
* style: remove empty row
* feat: stopMarketFill in FutureFillModel + test
* feat: futureStopMarket RegressionAlgo on PY|C#
* feature: add validation of market time in utest
* feature: validate exchange hours when order filled in regression algos
* feat: validation of StopMarket Type
fix: styling format in PY
* fix: validation on ExtendedMarketHours in FutureFillMode
feature: validate that all stop orders was completed in algos
* fix: validation: asset.Subscription -> ConfigProvider.GetSubscriptionData
fix:style: missed braces
* fix: asset.IsInternalFeed() -> general includeInternalConfigs flag
remove: extra login in algo
* feat: write comment why we use includeInternalConfigs in true
remove: extra log in PY version of algo
* Improve GetFillModel()
* Add support for other methods
* Add regresion algos and improvements
* Make Python algorithms regression tests
* Improve unit tests
* Minor changes
* First Draft of the solution
Handle the case where the security has already been added in the
algorithm
* Improve implementation
* Improve changes
* Add unit test
* Improve unit tests
* Fix bugs in unit tests
* Nit changes
* Nit changes
* Improve implementation
* Solve bugs in unit tests
* Add more unit tests
* Plotting Limits Handling Improvements
- Improve the handling and enforcement of plotting limits
- Increase local default limits
* Minor plotting test fix
* Solve bug and add unit tests
The bug was raised due to a race condition when writing a file and
deleting it. In order to stop it, a check condition was added at the end
of the PersistData() method to remove files that have just been deleted.
* Nit changes
* Add a different constructor for the test
The old description is incorrect. A stock can have positive Alpha but have negative Beta. In this case, if the benchmark increases, the performance of the stock may be dominated by its negative Beta, causing it to underperform the benchmark.
* Adds Unit Tests for ParseFutureSymbol
Tests:
- Ambiguous year with missing decade
- Non-ambiguous known year
- Expiring before 2000
* Fixes Missing Decade Tests
If the ticket doesn't provide a decade, VXZ2, we assume the current decade. So VXZ2 expires on Dec 2022.
* Fixes SymbolRepresentation.ParseFutureSymbol
Use the value of `futureYear` to handle contracts that expire before 2000. (E.g.: NQZ99).
Tickers from live trading may not provide futureYear. Use the current decade (2020).
- Fix bug where there might not be any data to sample. Adding unit test
- Fix bug where candlestick sampling might add point which shouldn't be
there. Adding unit tests
- Adjust series json converter to serialize any base series type. Adding
unit test
* Add Sortino ratio to statistics and report
* Adds Sortino Ratio to Report Key Statistics
* Addresses Peer-Review
Reuse `SharpeRatioReportElement` and change the template.
* Reuse Calculations Across Statistics and PortfolioStatistics
* Adds Sortino Ratio to Regression Algorithms
* Removes Sortino Ratio from Optimization Result Table
---------
Co-authored-by: Alexandre Catarino <AlexCatarino@users.noreply.github.com>
* Add support for null chart points
- Add support for null value chart points, updating existing and adding
new unit tests
* Minor selfreview tweaks
* Address review & fix deserialization bug
* Ignore null values when using NullResultValueTypeJsonConverter
* Narrow NullResultValueTypeJsonConverter conversion support
* Allow inherit of series sampler
* Solve bug and add unit test
The bug was raised when the sum of the True Range values was zero. Since
the Ultimate Oscillator provides a more reliable indicator of momentum,
if the prices don't change, its value should be zero, as there is no
acceleration of a security's price.
* enhance implementation
* Index.IsTradable Can Be Set
Indices are not tradable. LEAN will not set it to `true`, but the users can for backtesting.
* Adds Regression Tests
* Revert Changes to ProcessSecurityChanges
After running the regression tests in my local pc, I found some of them
were failing in the stats. Then, after debugging them I realized it was
because some of the strings were being casted without using
InvariantCulture parameter.
* Update Future-cme-[*] and Future-cme-ES
Acoording to `pandas_market_calendars` there were some dates in
Future-cme-[*] who wasn't early_closes, so they needed to be removed
from there. On the other hand, the early closes list of Future-cme-ES were
shifted by 1 hour according to CME webpage. Besides, there were some
missing dates.
* Update CME Future entries in MHDB
* Rebase
* nit change
* Fix unit tests
* Resume after early close/halts
* Add missing dates in MHDB and fix bugs in it
* Fix bug, add more unit tests and add docs
* fix regression algos
* address required changes
* Update failing regression test stats
After debugging the tests it was found they were failing due to the last
change on SecurityExchangeHours.IsOpen(). That method wasn't taking into
account that even if there is a late open after an early close if the
timespan is after the early close but before the late open, the market
is still close.
* enhance solution
* Update and fix bugs in MHDB
* Address required changes and update stats
* Update stats after rebase
* Nit change
* Missing update to regression test
* Use MHDB instead of USHoliday for Expiration Dates
VIX expiry function now relies completely on MHDB. However, it had to be
created an entry in MHDB for VIX since there wasn't one for it. CBOE
webpage only provided 2023 holidays so only those dates were considered
in the Holidays entry in MHDB. Therefore, some unit tests failed so it
was necessary to change also the VIX entry in FuturesExpiryFunctionsTestData.xml.
* Remove Global.cs/USHolidays class
* Use a lazy implementation
* First draft of the solution
* Use MHDB in FuturesExpiryFunctions.cs
* Remove unused class and fix indentation errors
* Fix indentation errors
* Nit changes
* Merge branches 7501 and 7506
* Merge changes in 7501 and 7506
In order to check compatibility between those branches, a new branch
was created out of branch 7501 and then it was merged with branch 7506.
2 regression tests and 8 unit tests failed, the regression tests failed on
the DataPoint stats. On the other hand, the unit tests failed since the
default parameter UseEquityHoliday was removed from
FuturesExpirtyUtilityFunctions.AddBusinessDays() and from other methods in
the same class too.
* Add missing changes
* Remove repeated good fridays
* Address minor review
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* feat: ETFConstituentsModel overloading constructor
* test
* feat: new constructor of ETFConstituentsUniverse
test: support all constructor on c#
* feat: cached symbol in ETFConstituentsUniverse.py
test: validate getting symbol from cahed collection
* remove: extra ctor in ETFConstituentsUniverseSelectionModel
* refactor: add miss condition secType in py
refactor: rename out existTicker -> Symbol
refacotr: rewrite test
* feat: new ETFConstituentsFrameworkAlgos
* refactor: more neat ETFConstituentsAlgos
* fix: CompositeUniverse to SetUniverse
* Implement risk free interest rate as an algorithm model
* Use risk free insterest rate model in Sharpe Ratio indicator
* Address peer review
Also added python wrapper
* Take pyobject as interest rate model in Sharpe Ratio indicator
* Minor fix
* Minor fix
* Address peer review
* Fix using brokerage transaction handler algo befor set
Brokerage open orders are being set before algorithm is set to the transaction handler, so SetPriceAdjustmentMode needs to get the algo to use
* Update unit tests
* Add new Order.PriceAdjustmentMode property
* Minor fix and unit test
* Minor fix and regression algorithms' stats update
* Unit test fixes
* Minor fix
* Set order price adjustment mode to raw always for live trading
* Shortable modeling improvements
- Do not limit live trading because of shortable model, will send
algorithm warning
- Add Interactive brokers shortable provider
- Minor LocalDiskShortableProvider API changes
* Fix shorted order update
- Fix shorted order update. Updating regression algorithm
* Minor improvement
* Address reviews
* Improves Error Handling in OptionPriceModelResult.Evaluate
Use the maturity calculation to calculate dividend and risk free discount to prevent the `negative time` error.
Adds Exception Message and StackTrade to `DebuggingEnabled` messages.
* Set Evaluation Date Before Every Calculation
Adds loggings to record the reference date which is set by the `Settings.setEvaluationDate` method.
- Once a security has been delisted or no longer selected by any
universe it will be remove from the main algorithm securities
collection but kept in the new Complete collection, this improves
performance in long running backtests where security collection can
get enumerated in different cases
- Minor refactor for security exchange local time to avoid having to set
it explicitly
* correlation as lean indicator #6983
* correlation as lean indicator #6983, added SPY QQQ test and Spearman test
* correlation as lean indicator #6983
* correlation as lean indicator #6983, fixes after review
* correlation as lean indicator #6983, fixes after review
* correlation as lean indicator #6983, fixes after review
* correlation as lean indicator #6983, added AcceptsQuoteBarsAsInput Test
* Minor review changes
---------
Co-authored-by: Grzegorz Wodnicki <info@funchips.co.uk>
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Store traded securities subscription data configs to disk
* Delegate traded subscriptions storage to the data monitor
* Add unit tests
* Address peer review
* Minor changes
* Async universe selection
- Add support for async universe selection, which will happen ahead of
time in the data stack for a performance improvement
* Thrown if using Coarse+Fine Asynchronous Universe selectioon
* Add Bybit brokerage model regression algorithm
* Add BibytFutures brokerage model name and Bybit backtesting regression algorithms
* Add Bybit margin calculations unit tests
* Minor changes in data
* Unify Bybit Spot and Futures brokerage model into one class
* Add new Bybit configurations
* Revert config change
* New Fundamental Data
* Minor CIK lookup fix
* Handle live mode & delete unexisting properties
* Minor coarse fundamental adjustment
* Add fundamental history support
* Fix unit tests
* Performance improvements
* Fixes
* Minor regression algorithm fix
* Improvements. Add FundamentalUniverseSelectionModel
* Change default values
* Fix unit test
* Minor tweaks
* Fix unit test
* Minor error handling improvement
* Fix rebase
Adds a new constructor overload that matches the pattern in Lean: Symbol, Symbol, other types.
Adds a new constructor that infers the indicator name: "B(period)".
* Add new libraries and updates
* ARM foundation update
- Deleting creme library which has been merged into already included river, because it causes copulae build to fail
- Deleting Riskfolio-Lib which would try to install old numpy and fail
* Add ClrBubbleExceptionInterpreter
* Add ClrBubbledExceptionInterpreter tests
* Bump pythonnet version to 2.0.23
* Minor changes
* Minor change
* Minor fix
* Fix failing unit tests
* Set default always open market hours entry for base security without subscription
Also return proper matching subscription for custom data symbols wihtout subscription.
* Minor changes
* Minor change
* Adds `SimpleCustomFillModel` to `CustomModelsAlgorithm`
The simple fill model shows how to implement a simpler version of the most popular order fills: Market, Stop Market and Limit.
This model was tested on QuantConnect Cloud, and will serve as additonal example, since we don't have an example that does not reuse the method of the base class.
* Handles Tick Resolution Case
Tick-resolution data doesn't have TradeBar. We can use the security price, since it represents a trade (`TickTrade`).
* Support sourcing custom data from ObjectStore
* Add Python regression algorithm
* Wire engine to pass ObjectStore down to the stream readers
* Minor changes
* Minor unit tests fixes
* Remove unused SetupHandlerParameters.ObjectStore
* Minor changes
* Minor changes
* Support single-file zipped data to be sourced from object store
* Assert object store custom data history requests in regression algorithms
* Add custom object store data live data feed unit test
* Add multi-file object store custom data regression algorithms
* Minor fix
Calling UniverseDefinition.ETF("TICKER",...) in Python was resolving the UserDefinition.ETF(Symbol...) overload due to Symbol's implicit string operator, causing the symbol to be wrongly created.
* Changes up to the date
* Revert "Changes up to the date"
This reverts commit 45dc9756bc.
* First attempt to solve bug
* Improve solution
* Add unit tests
* Fix bugs and add regression tests
* Solve bugs
* Deprecates IAlgorithmSettings.DataSubscriptionLimit
We will let the brokerage check and notify users that they have subscribed to datafeeds beyond their quota.
* Removes Unit Test
Also remove usage in `EmptyEquityAndOptions400Benchmark`.
* Fix brokerage tests filled order modification race condition
* Listen to fill event in test setup in brokerage tests
* Minor changes
* Address peer review
* Minor changes
* Removed platformId support from C2.
Modernized C2 API calls to APIv4.
Added C2 rate limiters (RateGate).
Added response logging so users can easily debug their code.
Added a few symbols for the demo.
* Address requested changes
* Address requested changes
---------
Co-authored-by: Francis Gingras <francis@collective2.com>
* Add support for Combo Orders with Fill Model
- Override ComboMarketFill, ComboLimitFill, ComboLegLimitFill in
FillModelPythonWrapper.cs
- Modify FillModel.Fill() to use Python Fill model if it exists when the
order is COMBO
- Add regression algorithm ComboOrdersFillModelAlgorithm.py
* Add regression algorithm
* Add regression algos
* Fix bug
* Modify IB Fee model to work without order.Price
* Fix bugs and add unit tests
* Update regression tests
* Solve bug and address changes
* Nit change
* Fix stats
* Add unit test and test data
* Add unit test for model greeks accuracy and comparison
* Add regression test
* Correction on previous unit test to use realtime official IB Greeks
* Switch default American option price model to CRR model
* Address peer review
* Draft of unit test to reproduce error
* Fix bug and regression tests
* Remove draft unit test
* Check fees in ComboMarketOrderAlgorithm.cs
* Fix bug in fees for combo orders
- Update regression tests
- Solve bug in fees when combo orders had legs with different quantity
- Solve bug in fees when combo orders had legs with different security
types
- Update IB comission rate for options where the price is higher than 0.1 USD
* Add explaining comments
* Use leg security fee model
* Fix optimization candlestick handling
- Fixes for optimization candlestick handling
- Centralize deserialization of ChartPoint into new
ChartPointJsonConverter
* Minor fixes
- Update failing tests
- Update CI to fail if anything fails
* Candlestick charts base implementation
* Series and Candlestick series json serialization
* Some cleanup
* Add AddPlot method for candlestick series to QCAlgorithm
* Remove Values property from ISeriesPoint
* Add candlestick QCAlgorithm.Plot trade bar methods
* Implement candlestick series re-sampling
* Add more SeriesSampler unit tests
* Add examples of candlestick charts usage to exisiting charting algorithm
* Address peer review
* Address peer review
* Derive Candlestick from Bar
* Sampler changes
* Add new series types from the cloud
* Add more candlestick series sampler tests
* Minor cleanup
* Minor changes
* First attempt to fix the bug
* Allow the splitFactor to change over time
- Add unit test
* Nit change
* Address required changes
* Address required changes
* Adjust upper bound
* Solve bug in the parameters of the Report
Before, when the number of keys were odd a part of the raw template was
shown in the report generated. Additionally, the user couldn't define
the number of columns of the parameters table
* Add unit tests
* Enhance unit tests
* Nit change
* Add support for custom HTML file in Report
* Add support for custom crisis HTML template
* Revert "Add support for custom crisis HTML template"
After considering this option carefully, the QC team has decided to
add this functionality in a different way.
This reverts commit aead1ff850.
* Get parameters and crisis HTML custom templates
* Move parameters html template to a file
* Nit change
* Remove template.crisis.html and add unit tests
* Nit change
* Nit change
* Add unit tests for the fail cases
* Add trailing stop orders base implementation
* Handle trailing stop order prices rounding
* Implement trailing stop orders fill logic
* Minor fill model changes
* Add ApplySplit to fill model interface for models that might need to be aware of splits.
Filling trailing stop orders require keeping track of min/max prices, which need to be split adjusted.
* Add brokerage order updated event for communicating certain order types prices changes
* Add order update event args class for brokerage side order updates
* Revert IFillModel.ApplySplit
* Add trailing stop orders regression algorithm
* Updated order ticket demo algorithm to include trailing stop orders
* Some cleanup
* Support trailing stop orders in IB brokerage model
* Some cleanup
* Fix failing tests
* Fix failing regression algorithm
* Address peer review
* Add trailing stop price calculation unit tests
* Minor changes
* Minor change
* Add algorithm configurations to Report
* Add just Parameters section in the Report
* Revert "Add just Parameters section in the Report"
This reverts commit 775255127a212db81fc1cc1e23a35b7c60214eac.
* Include just AlgorithmConfiguration.Parameters
* Create ParametersReportElement
* Generate parameters into HTML dynamically
* Enable the user to modify CSS content throu
* Enable CSS override file to be passed through cmdline
* Check CSS Override file exists
* Fix bugs
* Address required changes
* Address required changes
* Address required changes
* Implement RangeConsolidator
It turned out that the behavior of RangeConsolidator was similar to
ClassicRenkoConsolidator. Then, some of the ClassicRenkoConsolidator
methods, were abstracted to new class called BaseTimelessConsolidator.cs,
from which both RangeConsolidator and ClassicRenkoConsolidator could
inherit, therefore resusing code. The following tasks were done:
- Create RangeConsolidator.cs
- Create RangeBar.cs
- Create BaseTimelessConsolidator.cs
- Create RangeConsolidatorTests.cs
- Modify ClassicRenkoConsolidator.cs
* Allow intermediate/Phantom RangeBar's
- Enhance unit tests
- Nit changes
- Allow intermediate/Phantom RangeBar's on RangeConsolidator
* Nit changes
* Create ClassicRangeConsolidator and more changes
- Add regression tests
- Enhance unit tests
* Address required changes
* Address requested changes
* Address requested changes
* Add regression tests with Tick Resolution
* Address required changes
* Increase Range for RangeConsolidatorWithTickAlgo
* Add more unit tests and solve bugs
* Nit change
- Minor adjustment not to transition from canceled to invalid order
status, which can happen in some cases in live trading as a race
condition. Adding unit test
- Make sure open order status is set to submitted if none provided,
adjusting existing test
- Always respect the minimum time in universe even if we haven't gotten
any data point yet. This is specially useful for live trading options
which are illiquid
* Changed argument exceptions so that they used the nameof() operator for parameter names
* Update IQSocket.cs
---------
Co-authored-by: John Doe <john@doe>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Contract chain selection improvements
- Adjust contract chain selection behavior for futures and options so we
are sure to use the correct local time when filtering expirations.
Adding unit test reproducing issue
* Update futures regression algorithms
* Address reviews
* Fix contract security filter universe
- Fix for contract security filter universe which would would use .Time
to filter expirations which could be yesterdays time at midnight
localtime. Adding unit tests reproducing issue
* Update regression algorithms
* Update MHDB CBOT Futures on Good Friday
* Update MHDB CME Futures on Good Friday
* Update MHDB CBOT Futures on Good Friday
* Update MHDB CME Futures on Good Friday
* Update MHDB comex and nymex futures on good friday
* Revert "Update MHDB comex and nymex futures on good friday"
Since all nymex and comex futures had a holiday on Good Friday, it's
better to add the holiday on `Future-nymex-[*]` and `Future-comex-[*]`
This reverts commit e111c8e461.
* Remove Good Friday holiday for cme and cbot futures
- Remove Good Friday holiday from `Future-cme-[*]` and `Future-cbot-[*]`
- Add Good Friday holiday for specific cme and cbot futures
* Add unit tests and update MHDB
- Add unit tests asserting LEAN reads correctly Good Friday holidays and
early closes for CME Group Futures
- Update the holidays for other CME group Futures on MHDB
* Fix bugs
* Enhance unit tests
* Add security dynamic properties unit and regression tests.
These assert that dynamic properties that are instances of python classes that inherit c# classes keep their python object references.
* Minor change
* Bump pythonnet version to 2.0.21
* Fix security dynamic properties cast from PyObject
* Minor change
* Minor tweaks to avoid CI tests random failures
- Increate timeout of research start test, using default value
- Minor tweaks for rate gate to improve performance
* Clean up rategate tests
* Update MHDB with NYSE and ICE holiday hours
- Update holidays and early closes for ´Equity-usa-[*]´
- Update holidays for certain products from Ice Futures US
* Add missing holiday hours in MHDB for CME-G Futures
- The missing holiday hours were from 3 July 2023
* Remove duplicates
* Address JB comment
* Remove repeated fields in BaseRenkoBar
- Add new unit test to all indicators that inherit from
IndicatorBase<TradeBar>. This test asserts the indicators can receive
RenkoBar's as input
* Add Regression test and improve unit tests
- Change `Volume` property from `RenkoBar.cs` since RenkoBar's don't
have Volume. Now it throws an error when accessed
* Address required changes
* Add Python regression test and enhance unit tests
* Enhance unit and regression tests
* Assert indicator current value at the end
* Allow more indicators to use Renko/VolumeRenkoBar
* Fix Security.IsMarketOpen for cases outside regular MH and last bar
* Minor changes
* Move bar check of IsMarketOpen into FillModel
* Minor changes
* Minor changes
* Minor changes
* Fix bug
- Add `ResetHandler()` and `IsReset` field in `CompositeIndicator.cs`
- Modify `CompositeIndicator.Reset()` to invoke the IsReset event
- Modify `VolumeWeightedAveragePriceIndicator.Reset()` method to reset
its composite indicator
- Modify `IndicatorExtensions.WeightedBy()` to register an event to the
returned composite indicator `IsReset` field
* Enhance implementation
* Address Requested changes
* Nit changes
* Nit change
* Nit change
* Address chages
- Make ResetCompositeIndicator inherit from CompositeIndicator and
override `Reset()` method
- Add ResetCompositeIndicator unit tests
- Modifiy IndicatorExtensions to keep returning the same type in
`WeightedBy()` method
* Avoid modify other IndicatorExtensions methods
* Nit changes
* Reset inner indicators and add more unit tests
* Dynamic security properties for quick objects storage
* Documentation and other minor changes
* Add Security indexer to access custom properties
* Address peer review
* Minor changes
* Improve regression algorithms
* Bumped pythonnet to version 2.0.19
* Bumped pythonnet to version 2.0.20
* Address peer review
* Fix bug and add unit tests
- Modify `InsightWeightingPortfolioConstructionModel.cs/py` to take the
absolute value in its `GetValue()` method
- Modifiy `InsightsReturnsTargetsConsistentWithDirection()` unit test to
consider also the case where the direction is Down and the weight is
negative
* Nit changes
* Fix bug
The algorithm `CustomPartialFillModelAlgorithm.cs` was not working as
expected with short orders because in its Fill model, the variable
`absoluteRemaining` instead of decrease was being increased since
the `FillQuantity` in each call to this method was negative. Therefore
when this amount was substracted to `absoluteRemaining`, instead of
substract, it was added. Hence the method created more partial orders
than expected, specifically: 580 short orders for SPY with Quantity -10.
* Fix `CustomPartialFillModelAlgorithm.py`
* Address required changes
* Nit change
* Address required changes
* Make statistics available at runtime to algorithms
* Re-calculate statistics on every call
* Housekeeping
* Add regression algorithms
* Address peer review
* Support for custom summary statistics at runtime
* Minor changes
* Address peer review
* Address peer review
* Minor changes
* Minor changes
* Fix bug and add unit tests
The bug was raised because rounding the first point inserted in the
RenkoConsolidator didn't get the closest y-multiple of the given number
x, where y is the bar-size. In order to compute the first y-multiple of
the given number x, we need to first compute x mod y := x - y*Floor(x/y),
once we get this number we divide it by y to know if we should round x
to minimum y-multiple greater than x, or should round x to the maximum
y-multiple smaller than x. If (x mod y)/y < 0.5 we round down, otherwise we round up (the amount (x mod y)/y is always less than one and higher than 0). See the Art of Computer Programming, Vol I. Donald E. Knuth.
- Add `GetClosestMultiple()` method in `RenkoConsolidator.cs`
- Add more unit tests
* Nit change
* Add XML docs
* Address required changes and add more unit tests
* Implement indicator history through a rolling window
* Minor changes and extend unit tests
* Add regression algorithms
* Minor changes
* Update regression algorithms examples
* Move resizing logic from IndicatorBase to RollingWindow
* Add unit tests and other minor changes
* Minor fix
* Add unit tests and other minor changes
* Add unit test
* Fix position group BP calculation
* Fix position group BP calculation
* Extend option startegy buying power unit tests
* Fix option strategy buying power calculation to account for order direction
* Fix bug, add unit tests and nit changes
The bug was raised because if two inputs points with repeated close values were inserted,
and the second one had 0 volume, when the first one is removed from
_oldDataPoints, as the reamaining value of it in _volumePerPrice is
zero, that entry is removed from _volumePerPrice. Therefore, when
the second input point is removed from _oldDataPoints, this entry
is not found in _volumePerPrice. Then the changes made were:
- Modify MarketProfile.Add() method to check every removed data point is
still in _volumePerPrice
- Add unit tests to covert this change
- Nit changes in VolumeProfileTests.cs
* Fix bugs and add more unit tests
- The Fill Forward bars shouldn't be considered in the calculations as
it's not real data.
- If bar inserted is not FF but has zero volume, it could remove a price
value from VolumePerPrice and leave this sorted list empty, then each
time we access that list we need to first check it's not empty
* Enhance ComputeNextValue() and add more unit tests
- Lean engine will automatically add an options underlying if not
present, but in most cases the option chain will select the underlying
too, so let's make sure the configurations match. Previous to this
change 'fill forward' setting could be different causing the
underlying to be duplicated in the data stack
- Adjusting live stack to emit live auxiliary events based on factor files.
Updating existing tests
- Minor improvements for map & factor file provider daily refresh time.
Adding unit tests
* Add ShortButterflyCall and ShortButterflyPut strategies helper factory methods
* Reduce duplication by adding the base OptionStrategyFactoryMethodsBaseAlgorithm algorithm class
* Housekeeping
* Fix bug in Beta statistics
- Beta in Report was calculated using equity points and not daily
performance points, as it's done in Backtest
- Performance points were not being divided by 100, as it's done in
StatisticsBuilder
- Benchmark points were not being converted into a percentage daily
performance of the benchmark, as it's done in StatisticsBuilder
* Improve implementation and add unit tests
* Enhance implementation and add more unit tests
* Enhance implementation
* Nit change
* Nit changes
* Make maintenance margin zero for long options
* Use OptionIntialMargin in OptionMarginModel
* Add naked call and put strategies unit tests
* Minor fix
* Account for premium in option strategies margin
* Minor fixes
* Add OptionInitialMargin class to separate strategies margin and premium
* Minor fix
* Minor fix
* Cleanup, docs and other minor changes
* Minor changes
* Minor changes
* Minor changes
* Address peer review
* Add SetAccountCurrency overload
- Add SetAccountCurrencyOverload to allow the user to set also account
currency quantity
- Add regression and unit tests to cover the changes
* Add regression algos
* Simplifiy SetAccountCurrency implementation
* Minor changes
* Minor changes
* Add support for Null SecurityPositionGroupModel
- Add support for algorithms to set the null security position group
model which will not group securities together but rather return the
single group buying power model. Adding regression algorithms
* Add Null Margin python regression & cleanup
- Add python regression algorithm & minor regression algorithms clean up
* Fix position group HasSufficientBuyingPowerForOrder
Comparing computed delta BP -- which already considers every impacted
group -- with the right available buying power (the remaining margin)
* Minor changes
* Pandas frame include all ticks
- Pandas data framde history response will include all ticks. Updating
existing and adding new tests
* Python pandas converter performance improvement
* Handle liquidating option position with an option strategy
* Handle liquidating option position with an option strategy
* Minor changes
* Minor fix in default margin model margin call order generation
* Add TriggerPrice field when applying a split
If one stock is split in k parts, the new TriggerPrice in an order for that stock should be also scaled by a factor of k/10. It was made:
- Modify `DefaultBrokerageModel.ApplySplit()` to adjust `TriggerPrice` field
- Regression and unit test was added
* Check Limit Price for LimitIfTouched
* Nit change
* MHDB will merge common entry
- The MHDB will merge the market and security common entry holidays,
early closes and late opens
* Normalize & reuse future US holidays
- Normalize & reuse future US holidays
* Update existing unit tests expected stats
* Fix PositionGroupBuyingPowerModel quantity for delta calculation
The model was calculation the order size that required a margin equal to
the given delta. This fix makes it so that it calculates the position
group order quantity required to get the given buying power delta,
taking into account the holdings for the existing position group in the portfolio.
* Fix PositionGroupBuyingPowerModel quantity for delta calculation
* Add unit tests
* Add unit tests
* Add unit tests
* Housekeeping
* Housekeeping
* Adapt unit tests and move messages to Messages class
* Improve quantity calculation for target BP loop
- PositionGroup.Quantity is now always positive.
- Added PositionGroupExtensions.Closes method and use it to check if reducing a position in PositionGroupBuyingPowerModel.GetPositionGroupBuyingPower. This is necessary since now a PositionGroup side is always "long" (see first bullet point).
* Improvements
* Improvements and new PositionGroup unit tests
* Unit tests improvements
- HasSufficientBuyingPowerForOrder test cases were completely updated to
make it accurate in terms of Lean's margin calculation.
- BuyingPowerForPositionGroupCalculation test was fixed in order to get
the correct expected result.
* Fixes for failing tests
* Added additional check to OptionStrategyMarginCallEventsAlgorithm
* Improve documentation comments
* Fix option strategies BuyingPowerForPositionGroupCalculation unit test
* Fix PositionGroupBuyingPowerModel.GetPositionGroupBuyingPower
Also address peer review
* Minor changes
* Update unit tests for new inverse strategies
Also fixed PositionGroupExtensions.WithQuantity to resolve inverse
strategy
* Minor fix
* Allow target/delta to go to the opposite direction of the position group
* Housekeeping
* Handle quantity for target/delta BP request for different reference position group
* Revert removal of PositionGroupBuyingPowerParameters.Direction
* Minor changes
* Peer review
* Minor tweaks on the position group buying power model
* Minor fix
* Implement Trailing FreePortfolioValue
- Implement Trailing FreePortfolioValue by default, users will be able
to set it to a fixed number if desired. Adding regression algorithm
- Setting the default 'MinimumOrderMarginPortfolioPercentage' from 0 to
0.1% of the TPV to avoud tiny trades by default
* Update existing regression algorithms
* Address reviews
- Send warning message to the user if a trade does not happen due to the
default setting of the minimum order margin percentage value
* Address reivews
* Rename TotalPortfolioValueLessFreeBuffer
* Update new regression algorithm
* Minor fix CrunchDao Symbology
- Minor fix for CrunchDao Symbology. Updating existing tests
* Add missing symbol mapping
- Add IAlgorithm.Ticker(Symbol) functionality which will return the
latest ticker for the requested symbol at the current algorithm time
- SignalExporters will use Ticker to get the current symbol ticker
* Rename SecId GetTicker to Ticker
* Solve bug and add regression test
The bug was raised because, when trying to use C#
MeanVarianceOptimizationPortfolioConstructionModel from a Python
algorithm, there wasn't a constructor that accepted a portfolio optimizer
as a PyObject. Additionally, there wasn't also a Python Wrapper to wrapp
that portfolio optimizer.
- Add PortfolioOptimizerPythonWrapper.cs
- Add constructor in
MeanVarianceOptimizationPortfolioConstructionModel.cs that accepts
portfolio optimizer as a PyObject
- Add regression algorithms to cover the changes
* Improve constructor overload implementation
* Change implementation to follow API pattern
* Enhance implementation and add unit tests
* Enhance implementation and add more unit tests
* Enhance implementation
* Implement Unregister helper method
- Implement Unregister helper method, each indicator which hold a
reference to the consolidators updating it. Adding regression
algorithms
* Fix for QuantBook indicators
* Address reviews
- Custom data universe selection market hours. Adding regression test
asserting the behavior. Updating existing tests due to market hours
change, triggering selection always, even the 4th of July 2018
* Add short call calendar spread strategy and review margin calculation
* Add short Put calendar spread strategy and fixed naked short options margin calculation
* Minor changes
* Minor changes
* Minor changes
* Solve the bug and add unit test
- The bug was raised because if something went wrong trying to register an indicator with a C# or a custom consolidator the exception raised was lost, because the catch sentence didn't return it with another exception
- A unit test was added asserting a exception with a related message was raised each time something went wrong with the consolidator
* Fix bugs and enhance unit tests
* Fix bug
* Handle unsupported orders and add unit tests
- Capture more Tradier Brokerage order limitations
- Add unit tests to cover this changes
* Enhance implementation and add more unit tests
* Enhance implementation and add unit test
* Remove IsMarketOrderGTC condition
* Remove error message MarketOrderIsGtc
* Covered and Protective Call and Put
- Improve covered call and put margin requirements based on IB
- Add missing protective call and puts strategies (inversion of a
covered call). Adding unit tests
* Fixes
* Add missing tests
* Expire Instead of Clear Insights In Base PCM In OnSecuritiesChanged
* Fixes Regression Test
The regression was incorrect because `_removedSymbols` didn't retain the removed symbols from previous `OnSecuritiesChanged` calls, and the algorithm relances once per month. E.g. On day 1, SPY and FB were removed, on day 3 AAPL and IBM were removed and `removedSymbols` would only include AAPL and IBM.
We should hve fixed this problem before with:
```csharp
if (_removedSymbols == null) _removedSymbols = new List<Symbol>();
_removedSymbols.AddRange(changes.RemovedSecurities.Select(x => x.Symbol));
```
However the change to use Expire fixes the issue.
* Removes Unused List of Removed Symbols
* Handle unsupported orders and add unit tests
- Capture more Tradier Brokerage order limitations
- Add unit tests to cover this changes
* Enhance implementation and add more unit tests
* Enhance implementation and add unit test
* Fix potential bug and add more docs
As FeeInBase and FeeInQuote are mutually exclusive, if one was set to true, the other one should be set to false. It was added information about where it's used and how fee currency selection works in Kraken.
* Nit change
* Simplify XML docs
* Fix the bug and add unit tests
- The bug was raised by different conditions.
First, the tick created in tick generator for Open Interest was not of type Open Interest.
Second, TickAggregator.cs was made to use daily resolution for OpenInterest always.
Third, the RandomValueGenerator, generated a random friday as expiration date for the option created but this one needed first two underlying data points, so if the expiration date was before the start date plus 3 days, RandomDataGenerator just generated OpenInterest data for the option.
* Add docs
* Add option strategy position group BPM tests
Tests added for calculating order quantity for a delta buying power
* Add option strategy position group BPM margin calculation tests
* Add option strategy position group BPM margin calculation tests
* Add option strategy position group BPM margin calculation tests
* Add option strategy position group available BPM calculation tests
* Add option strategy position group impact buying power calculation
* Housekeeping
* Unit tests improvements
* Unit tests improvements
* Add more unit tests
* Address peer review
* Add forward data only fill model example
- Add forward data only custom fill model C# & PY example.
- Minor adjustment for 'MarketOrderFillTimeout' to be zero always in
backtesting
* Address reviews
* Add support for brokerage side new orders
- Add support for brokerage side new order events for liquidation cases
* Minor cash delta fix
* Improve account cash logging
* Fix null reference exception for open orders
* Implement ShortableProviderPythonWrapper.cs
- Modify AllShortableSymbolsCoarseSelectionRegressionAlgorithm.cs and ShortableProviderOrdersRejectedRegressionAlgorithm.cs to use ShortableProvider from Security and not from the Brokerage model
- Add SetShortableProvider() overload method in Security.cs to set a custom shortable provider from Python
- Remove AllShortableSymbols() method from LocalDiskShortableProvider.cs
- Remove DefaultShortableProvider class
- Add regresion algorithms in C# to cover the changes done
* Revert "Merge process"
This reverts commit 775a4b8ec1, reversing
changes made to bcc3e790f6.
* Revert "Revert "Merge process""
This reverts commit aa18fb40ee.
* Solve bugs
- Solve bugs in DefaultMarkets, CanUpdateOrder() and CanSubmitOrder() methods from BrokerageModelPythonWrapper.cs
- Add regression algos in both Python and C# to cover the changes
* Catch potential errors and add unit tests
* Add more unit tests
* Nit change
* Enhance implementation, add more unit tests
* Add IndexOptionPriceVariationModel
- Add IndexOption price variation model to handle price based minimum
price variation. Adding unit tests
* Minor improvements for VIX & VIXW
* Refactor settlement mode api
- Add new Scan method on ISettlementModel API. To be used by
futures settlement model mark to market handling
- Use 'Parameters' classes for 'ISettlementModel'
* Address reviews
* Daily future settlement
- Implement daily future settlement. Adding unit & regression tests
* Address reviews
* Assert cash balance every day
* Refactor settlement mode api
- Add new Scan method on ISettlementModel API. To be used by
futures settlement model mark to market handling
- Use 'Parameters' classes for 'ISettlementModel'
* Address reviews
* Fix multiple option strategies liquidation
When holding multiple position groups of the same underlying, reverting
one of the groups was not possible because of the presence of the other
position groups, which caused no matching groups to be formed for the
order.
* Make OptionStrategyPositionGroupResolver not group single position
* Revert make OptionStrategyPositionGroupResolver not group single position
* Implement ShortableProviderPythonWrapper.cs
- Modify AllShortableSymbolsCoarseSelectionRegressionAlgorithm.cs and ShortableProviderOrdersRejectedRegressionAlgorithm.cs to use ShortableProvider from Security and not from the Brokerage model
- Add SetShortableProvider() overload method in Security.cs to set a custom shortable provider from Python
- Remove AllShortableSymbols() method from LocalDiskShortableProvider.cs
- Remove DefaultShortableProvider class
- Add regresion algorithms in C# to cover the changes done
* Revert "Merge process"
This reverts commit 775a4b8ec1, reversing
changes made to bcc3e790f6.
* Revert "Revert "Merge process""
This reverts commit aa18fb40ee.
* Solve bug
- Add new constructor overload in OpenInterestFutureUniverseSelectionModel.cs that accepts future chain symbol selector as PyObject
- Add a private static method in OpenInterestFutureUniverseSelectionModel that converts Python lambda function to Func<DateTime, IEnumerable<Symbol>>
- Add a regression algorithm in Python to cover changes. In these case, add Python version of OpenInterestFuturesRegressionAlgorithm.cs
* Nit changes
* Nit change
* Minor docs tweak
---------
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Implement ShortableProviderPythonWrapper.cs
- Modify AllShortableSymbolsCoarseSelectionRegressionAlgorithm.cs and ShortableProviderOrdersRejectedRegressionAlgorithm.cs to use ShortableProvider from Security and not from the Brokerage model
- Add SetShortableProvider() overload method in Security.cs to set a custom shortable provider from Python
- Remove AllShortableSymbols() method from LocalDiskShortableProvider.cs
- Remove DefaultShortableProvider class
- Add regresion algorithms in C# to cover the changes done
* Implement ShortableProviderPythonWrapper.cs
- Modify AllShortableSymbolsCoarseSelectionRegressionAlgorithm.cs and ShortableProviderOrdersRejectedRegressionAlgorithm.cs to use ShortableProvider from Security and not from the Brokerage model
- Add SetShortableProvider() overload method in Security.cs to set a custom shortable provider from Python
- Remove AllShortableSymbols() method from LocalDiskShortableProvider.cs
- Remove DefaultShortableProvider class
- Add regresion algorithms in C# to cover the changes done
* Solve bugs and nit change
* Address review
---------
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Solve Report Generation Conversion Rate Error bug (#7183)
* Solve bug
Remove if sentence that checked if Portfolio contained securities that
needed a conversion, like Forex, Crypto or Cfd. It didn't take into
account CryptoFutures so when the backtest result used that kind of
security types it never found a conversion for them. Making a git blame
for Report.PortfolioLooper constructor it was found the if sentence was
not needed so it was removed and then different reports were made for
algorithms that used security types different than Forex,Crypto,Cfd and
CryptoFutures to check everything was still working as expected.
* Solve bug
Remove if sentence that checked if Portfolio contained securities that
needed a conversion, like Forex, Crypto or Cfd. It didn't take into
account CryptoFutures so when the backtest result used that kind of
security types it never found a conversion for them. Making a git blame
for Report.PortfolioLooper constructor it was found the if sentence was
not needed so it was removed and then different reports were made for
algorithms that used security types different than Forex,Crypto,Cfd and
CryptoFutures to check everything was still working as expected.
* Missing Python Version of Crypto Future Algorithms - Issue 7128
* Address reviews
---------
Co-authored-by: hungv <hungv@compatibl.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Add dataMappingMode parameter to every history api method overload
* Minor unit tests fixes
* Update regression algorithm stats
* Minor changes
* Minor changes
- Validate cache folder exists before using it
- Avoid duplication fetching factor & map files path
- Adding helper method to check if directory is empty. Adding unit tests
- Create `SecurityDataFilterPythonWrapper.cs`
- Modifiy `Security.cs` to set a data filter, whether is a C#, Python derivative or custom data filter custom
- Add Python and C# regression algorithms to assert the changes
* Add extendedMarket parameter to every history api method overload
* Rename extendedMarketHours parameter
New name is extendedHours as in the History API to standarize parameters naming
* Update generic history overloads to use every matching subscription
* Update regression algorithms stats
* Centralize period-based history error for tick resolution
* Rename extended market hours parameter to extendedMarketHours
* Minor changes
* Minor changes
* Minor unit tests changes
* Minor unit tests changes
* Minor changes
* Minor unit tests changes
* Minor unit tests changes
* Added PyStan Library in DockerfileLeanFoundation
* Add new libraries and unit tests
---------
Co-authored-by: Ivan Radigales Creus <ivanrdgc@gmail.com>
* Implement `OptionExerciseModelPythonWrapper.cs`
- Modifiy `Option.cs` to set the automatic exercise model, even if it's a python custom option model exercise model
- Add `QuantConnect.Orders.OptionExercise` in `AlgorithmImports.py`
- Add regression algorithms in C# and Python showing how to specify a custom option exercise model
* Address required changes
- Remove unnecessary constructor in
´CustomOptionExerciseModelRegressionAlgorithm.cs´
- Change `.Dispose()` in `OptionExerciseModelPythonWrapper.OptionExercise(9` to ´using´ keyword
- Change `Option.SetOptionExerciseModel()` XML docs
* Minor tweaks
* Update CustomOptionExerciseModelRegressionAlgorithm.cs
---------
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Add fillForward parameters to every History method
* Remove conflicting Python history method
* Undo removing conflicting Python history method
* Minor changes
* Minor changes
* Minor changes
* Add fillForward and extendedMarket parameters to history request factory
* Minor changes
* Minor changes
* Minor changes
* Minor changes
* Minor unit tests changes
* Rename fillForward parameter in History API
New name if fillDataForward as in the Add*Security API to standarize
parameters naming
* Rename fillForward parameter
Using the shorter fillForward in every API
* Solve bug
Remove if sentence that checked if Portfolio contained securities that
needed a conversion, like Forex, Crypto or Cfd. It didn't take into
account CryptoFutures so when the backtest result used that kind of
security types it never found a conversion for them. Making a git blame
for Report.PortfolioLooper constructor it was found the if sentence was
not needed so it was removed and then different reports were made for
algorithms that used security types different than Forex,Crypto,Cfd and
CryptoFutures to check everything was still working as expected.
* Solve bug
Remove if sentence that checked if Portfolio contained securities that
needed a conversion, like Forex, Crypto or Cfd. It didn't take into
account CryptoFutures so when the backtest result used that kind of
security types it never found a conversion for them. Making a git blame
for Report.PortfolioLooper constructor it was found the if sentence was
not needed so it was removed and then different reports were made for
algorithms that used security types different than Forex,Crypto,Cfd and
CryptoFutures to check everything was still working as expected.
* Add Collective2SignalExportClass
Add SignalExportTarget interface
* Collective2SignalExport test working
Add SignalExportManager
Add draft of CrunchDAOSignalExport
* Modify SignalExportManager
Instantiate SignalExportManager in QCAlgorithm constructor
Draft of CrunchDAOSignalExport
* Improve SignalExportManager
- Add regression tests SignalExportDemonstrationAlgorithm in C# and
Python
* Improve SignalExportDemonstrationAlgorithm
Address requested changes in Collective2SignalExport, SignalExportManger and SignalExportTargetTests.cs
* Add CrunchDAOSignalExport.cs
Add CrunchDAOSignalExport unit tests in SignalExportTargetTests.cs
* Add NumeraiSignalExport.cs
Modify SignalExportDemonstrationAlgorithm.cs to test NumeraiSignalExport
Add unit test in SignalExportTargetTests to test NumeraiSignalExport
* Address required changes
- Add BaseSignalExport.cs
- Add SignalExportParameters.cs
- Add PortfolioSignalExportDemonstrationAlgorithm.cs/py
- Improve Error handling in SignalExport provider classes
- Collective2SignalExport now gets the correct number of shares for each holding
- SignalExportManager now computes the correct holding percentage of each portfolio target
- SignalExportManager now takes into account if the algorithm is in live mode
- Demonstration algorithms now are more simple
* Address last required changes
- PortfolioSignalExportDemonstrationAlgorithm.cs/py now inherits from SignalExportDemonstrationAlgorithm.cs/py
- Add more unit tests to assert SignalExportManager gets the correct percentage quantity for each holding
- Change Collective2SignalExport, CrunchDAOSignalExport and NumeraiSignalExport Send() method to return true if there was no error while sending the signals and false otherwise
- Nit changes
* Remove exceptions thrown
- Add more unit tests and more test cases
- Enhance BaseSignalExport.Dispose() method
* Add Collective2SignalExportClass
Add SignalExportTarget interface
* Collective2SignalExport test working
Add SignalExportManager
Add draft of CrunchDAOSignalExport
* Modify SignalExportManager
Instantiate SignalExportManager in QCAlgorithm constructor
Draft of CrunchDAOSignalExport
* Improve SignalExportManager
- Add regression tests SignalExportDemonstrationAlgorithm in C# and
Python
* Improve SignalExportDemonstrationAlgorithm
Address requested changes in Collective2SignalExport, SignalExportManger and SignalExportTargetTests.cs
* Add CrunchDAOSignalExport.cs
Add CrunchDAOSignalExport unit tests in SignalExportTargetTests.cs
* Add NumeraiSignalExport.cs
Modify SignalExportDemonstrationAlgorithm.cs to test NumeraiSignalExport
Add unit test in SignalExportTargetTests to test NumeraiSignalExport
* Address required changes
- Add BaseSignalExport.cs
- Add SignalExportParameters.cs
- Add PortfolioSignalExportDemonstrationAlgorithm.cs/py
- Improve Error handling in SignalExport provider classes
- Collective2SignalExport now gets the correct number of shares for each holding
- SignalExportManager now computes the correct holding percentage of each portfolio target
- SignalExportManager now takes into account if the algorithm is in live mode
- Demonstration algorithms now are more simple
* Address last required changes
- PortfolioSignalExportDemonstrationAlgorithm.cs/py now inherits from SignalExportDemonstrationAlgorithm.cs/py
- Add more unit tests to assert SignalExportManager gets the correct percentage quantity for each holding
- Change Collective2SignalExport, CrunchDAOSignalExport and NumeraiSignalExport Send() method to return true if there was no error while sending the signals and false otherwise
- Nit changes
* Remove exceptions thrown
- Add more unit tests and more test cases
- Enhance BaseSignalExport.Dispose() method
* Fix failing regression tests
* Fix failing unit tests
* Nit changes
* Nit change
* Nit change
* Fix failing unit tests
* Changes required
- Break regression algos `SignalExportDemonstrationAlgorithm.cs/py` nad `PortfolioSignalExportDemonstrationAlgorithm.cs/`y` into three ones, one for each signal export provider
- Change SignalExportManager constructor to receive current algorithm as a parameter
- Fix bug in `SignalExportManager.GetPortfolioTargets()`, now it computes the correct percentage for each holding
- Make `BaseSignalExport.DefaultAllowedSecurityTypes` overrdible
- Handle case were `Collective2SignalExport.ConvertPercentageToQuantity()` returns null
- Clean unnecessary code in `Collective2SignalExport()`, `CrunchDAOSignalExport()` and `NumeraiSignalExport()`
* Nit change
* Nit change
* Minor tweaks after review
* Remove indexes from signal exports
* Required changes
- Change EMA indicators period from 200, 300 to 10,100 in regression algorithms
- Remove Indices from regression algorithms
- Add more XML documentation to regression algorithms
- Change `Log.Error` to `_algorithm.Error` in Signal export providers. Besides, fix error message format
- Change default value for `platformId` parameter in `Collective2SignalExport.cs` constructor
- Solve small bugs in SignalExportProvider when verificating the amount of porfolio targets is greater than zero and each portfolio target is allowed
- Handle case when `PortfolioTarget.Percent()` returns null in `Collective2SignalExport.ConvertPercentageToQuantity()`
- Handle error format message from Collective2 API
- Check every ticker signal is between 0 and 1 (inclusive) in `CrunchDAOSignalExport.cs`
- Modifiy `NumeraiSignalExport.cs` constructor to take into account filename given in the arguments
- Fix small bug with the return value of `ConvertTargetsToNumerai()` method in `NumeraiSignalExport.cs`
- Modify `SignalExportManager.cs` to return true when the algorithm being ran is not in live mode
- Remove indices from CrunchDAO unit tests
* Enhance ´CrunchDAOSignalExport.cs´ implementation
---------
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Fix EMA indicator first value calculation
As done by TALib and TradingView, the first EMA value after warmup is
the SMA of the first period.
* Update TSI indicator test data
Data was exported from TradingView
* Update TRIX indicator test data from TradingView
* Update AccumulationDistributionOscillator indicator test data from TradingView
* Update Double EMA indicator test data from TradingView
* Update McClellanSummationIndex indicator test data
* Update SchaffTrendCycle indicator test data
* Update TripleExponentialMovingAverage indicator test data from TradingView
* Update stats for algorithms using EMA
* Update failing unit tests
* Handle position group margin calls
* Minor changes
* Update regression algorithms
* Minor changes
* Minor changes
* Added regression algorithm
* Peer review
* Peer review
* Peer review
* Peer review
* Minor changes
* Minor changes
* Add unit test
* Allow sufficient buying power when closing position group
* Add unit test
* Improve regression algorithms
* Fix ReportCharts color_map bug
- Add ´CryptoFuture´ entry in `color_map` dict using color #E55812
- Make ´ReportCharts.py´ color_map a class attribute
- Change ´ReportChartsTests.cs´ to use color_map dict from ´ReportCharts.py´
- Add unit test that fails every time a new SecurityType is added
* Improve unit test
* Nit change
* Nit change
* Updated MHDB and CME currencies futures expiry functions
Some expiry functions needed to take into consideration the security
holidays
* Add more holidays to CME securities in MHDB
* Update regression test to check number of insights
* Update model to have insight manager ref and cancel insight when signal goes flat
* Update unit tests and regression tests
* Address peer review: using Insights.Clear
* Invalidate option strategies orders when short selling over margin
When shorting an option strategy with margin requirements that cannot be
met, the order should be invalidated. The buying power model is now able
to detect said situation and result in unssuficient buying power.
The OptionsMarginModel now uses the parameters instance values instead
of the security holdings to compute margin requirements. This fixes the
situation when calculation buying power for a first time position with
no holdings.
* Add and fix unit tests
* Update regression algorithms
* Peer review
* Downloader data provider fixes
- Fix stored data time zone, which is expected in data time zone
- Minor logging improvements
- LeanData Parse handles etf universes path. Adding unit test
- LeanDataWrite will only map if there's something to map to
* Fix download data provider synchronization
* Synchronization improvements after more testing
- KeyStringSynchronizer will handle reentrancy. Adding unit tests
- Define a unique Synchronizare for downloading data, since not related
to disk reading & writting
* LeanDataWriter can handle mapping
- Adjust LeanDataWrite so that it can handle symbol mapping if required.
Adding unit tests
* Fix Lean Data path parsing
* Updates Regression Algorithm to Assert the Number of Insights
* Updates HistoricalReturnsAlphaModel to Cancel Insights Not Emit Flat
We expect fewer insights after this change, but no changes to any regression algorithm.
* Updates Regression Algorithm
Assert the new expected number of generated insights.
* Use InsightCollection Clear Method
Replaces old `TrailingStopRiskFrameworkAlgorithm` for `TrailingStopRiskFrameworkRegressionAlgorithm` that inherits from `BaseFrameworkRegressionAlgorithm` as a member of the framework regression tests.
* Update regression test
* Update model to expire insights
* Avoid remove insights
* Update regression test
* Use InsightManager Cancel to Expire All Insights
* Update CompositeRiskManagementModelFrameworkAlgorithm
This regression also depends on `MaximumUnrealizedProfitPercentPerSecurity` and `MaximumDrawdownPercentPerSecurity` but `MaximumDrawdownPercentPerSecurity` doesn't close positions.
---------
Co-authored-by: Alexandre Catarino <AlexCatarino@users.noreply.github.com>
* Remove OnEndOfAlgorithm and update expected trades
* Update models to cancel insights
* Update expected results
There are 3 trades instead of 2 because the PCM does a rebalance
* Remove `Remove` method call
* Update to use the new `Cancel` method
If we use `.astimezone(dt.tzinfo)`, the new time was convering the timezone from local (e.g. PST) to GMT (dt.tzinfo). In this case, we only want to remove the timezone to enable the operation.
* Renames and Updates BaseAlphaModelFrameworkRegressionAlgorithm
The `BaseFrameworkRegressionAlgorithm ` will be used for multiple framework regression tests
* Updates and Renames EmaCrossAlphaModelFrameworkAlgorithm
Adds "Regression" to inform that it's a regression algorithm.
* Updates and Renames MaximumPortfolioDrawdownFrameworkAlgorithm
Adds "Regression" to inform that it's a regression algorithm, and use the model name: `MaximumDrawdownPercentPortfolio`
* Adds New Regression Algorithms
* Adds BaseAlphaModelFrameworkRegressionAlgorithm
`BaseAlphaModelFrameworkRegressionAlgorithm` will be used to validate Alpha Model regression algorithm with the same universe.
- HistoricalReturnsAlphaModelFrameworkAlgorithm
- EmaCrossAlphaModelFrameworkAlgorithm
- MacdAlphaModelFrameworkAlgorithm
- RsiAlphaModelFrameworkAlgorithm
- BasePairsTradingAlphaModelFrameworkAlgorithm
* Addresses Peer-Review
* Fix fill quantity using group order quantity for combo orders
* Add Order's ComboQuantity property
* Add Order's ComboDirection property
* Minor changes and regression algorithms update
* Minor changes
* Update algorithms stats
* Minor changes and regression algorithms update
* Store the full quantity for each combo order leg in Quantity property instead of the ratio
* Minor changes and regression algorithms update
* A few fixes after pair programming
* Handle grouping position reduction
---------
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Improves MinimumVariancePortfolioOptimizer Values Handling
The C# version of the `MinimumVariancePortfolioOptimizer` generated NaN resulting in unit tests failing.
If the solver returns NaN for an item, we set it to zero. if all items are NaN or Zero, we return the initial guess, since the sum cannot be zero.
* Fixes `RiskParityPortfolioConstructionModelTests`
We need to add insights to insight manager before we call `CreateTargets`.
* Adds Unit Test
Asserts that we can remove a consolidator that was previously added.
* AddConsolidator Method Only Wraps Non-C# Consolidators
This behavior was implemented in `QCAlgorithm.RegisterIndicator` to avoid wrapping an object unnecessarily.
- Improve LeanData aggregation helper methods to use shared consolidator based implementation
- Adding Tick to tradeBar aggregation
- Adding more unit tests
- Fix data-queue-handler configuration initialization using paper brokerage
- Fix for downloader data provider request times
- Fix null reference exception in live result handler if algorithm is
null
* Asserts Number of Insights In One PCM Regression Test
If the `EqualWeightingPortfolioConstructionModel` interacts with the `QCAlgorithm.Insights`, the number of elements in the collection should not the sum of emitted insights.
* Refactor Portfolio Construction Models to Use Insight Manager
`PortfolioConstructionModel` will use `QCAlgorithm.Insights" instead of class property `InsightCollection` to manage the insights. It no longer adds insights to the collection, but it removes them if they expire or the securities are removed from the universe.
Updates PCMs that were affected by the change.
* Updates Unit Tests
We need to add the insights to the insight manager before we call `PortfolioConstruction.CreateTargets`
* Minor IndexOption underlying TZ fix
- Fix for option universe chain underlying TZ which might not match the
options one, bug affected IndexOption specifically.
* Update affected regression tests
* Performance Improvements
- Remove Immutable collection from position groups to improve
performance. Adding more tests
- Remove ConcurrentDictionary from PortfolioTargetCollection &
DataManager. Adding more tests
- Add Securities enumerator keys & values cache
* Add test for PortfolioTargetCollection remove by reference
* Apply splits and dividends to volatility models
* Apply splits and dividends to volatility models using history requests
* Add new ScaleRaw data normalization mode
Handling the new mode in the price scale enumerator.
* DataNormalizationMode.ScaledRaw history requests
* Minor changes
* Minor changes
* Disable new normalization mode in AddSecurity methods and other minor changes
* Peer review
* Minor changes
* Peer review
* Minor changes
* Peer review
* Peer review
* Peer review
* Add scaled raw history regression algorithm
* Add more regression algorithms
* Add more regression algorithms
* Add Slice.TryGet unit tests
* Peer review
* Peer review
* Peer review
* Peer review
* Peer review
* Update algorithms stats
* Peer review
* Peer review
* Adds Unit Test For Limit Price Above/Below Open
See `LimitOrderFillsAtOpenWithFavorableGap`
* Fixes Limit Price Above/Below Open
If we place a buy/sell limit order below/above the current market price in TWS it fills immediately, so we model this behavior by filling with the opening price of the first trade bar.
* Updates Regression Tests
The number of trades did not change as expected. The fills are better because orders are filling with the open price when the new condition is met.
* Refactor alpha statistics
- Refactor alpha statistics, cleaning up and simplifying no longer required calculations and scoring
- Adding new InsightEvaluator abstraction, adding C# & PY regression
algorithms
* Optimization backtest result json converter update
* Address reviews
- Remove IAlphaHandler, move insight storage responsability to IResultHandler
and centralizing insight collection on the QCAlgorithm.Insights to be
reused by the framework models
- Fix portfolio turnover single day backtests and duplicate time
sampling handling. Updating regression algorithms
* Add InsightCollection tests and minor fixes
* Adding more & improved tests
The `LimitFill`, `LimitIfTouchedFill` and `StopMaketFill` methods require `TradeBar` data to test price levels. We create a method to serve these methods as well other methods that will require this information.
Removes `Price > 0` check for `TradeType.Trade` data. Unecessary check, since a tick price cannot be less or equal to zero.
* Adds Unit Tests for Stop Market Fill
These unit tests covers new scenarios:
- Cannot trigger on quote data (QuoteBar or TickRype.Quote);
- Adds missing test for tick data (TickType.Trade)
- Unfavorable gap (see QuantConnect#4545)
* Fixes EquityFillModel StopMarketFill
Use the entire tick history to find the trigger price
Fill price is the stop price, unless there is an unfavorable gap (see #4545).
* Update Regression Tests
All regression tests keep the total trades. The difference are due to the time that the stop market orders are triggered and the prices that are filled. The `StopLossOnOrderEventRegressionAlgorithm` and the `UpdateOrderRegressionAlgorithm` observe the unfavorable gap case.
* Addresses Peer-Review
* Updates Regression Test
New `OrderListHash` because of the change in the fill on open case message.
* remove annualization
* Update python optimizer to best fit in convex problem
* rerun test
* test metric
* Add comment to explain using simple return
* Add unit test on PR
* Removes PythonTools from Solution
Missing removal from #6967.
It results in a runtime error if we use dotnet to list the packages:
`dotnet list ../Lean/QuantConnect.Lean.sln package`
* Update QuantConnect.Lean.sln
---------
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Foundation Update
* Adding more foundation packages
* Update existing libraries
* Further package updates
* Image cleanup
- Update base image to ubuntu 22.04
- Some clean up on dockerfile
- Remove ARM base image
* Test updates
- Minor docker file tweaks
- Update existing python environment tests
* Add more tests & fix Julia
* Remove jupyter-dash
* Remove R from foundation
* Remove Julia
* Remove Tensorforce
* Adds Regression Test
The new regression test asserts that the total trades is 2.
* Fixes Liquidate Existing Holdings Bug
Liquidate existing holdings before open new postions.
* Addresses Peer-Review
* IsOption uses underlyings holidays
- Option assets will use underlyings holidays, early closes and late
opens if their own are not defined
* Update regression algorithms
* Adds Unit Tests
The `LimitFill` method should not fill using `QuoteBar` or `Tick` with `TickType.Quote` type.
Adds missing test for tick data (`TickType.Trade`).
* Fixes EquityFillModel.FillModel
Use `Tick` with `TickType.Trade` or `TradeBar` information to fill limit orders.
* Update Regression Test Expected Statistics
The regression tests changed because of different fills.
The `ExtendedMarketTradingRegressionAlgorithm` has different number of trades because of an extra fill on the 4th order generated by TradeBar with a Low lower than than the QuoteBar.Ask Low:
> 20230222 13:56:24.251 TRACE:: Log: Time: 10/10/2013 12:01:00 OrderID: 4 EventID: 2 Symbol: SPY Status: Filled Quantity: 10 FillQuantity: 10 FillPrice: 143.8998 USD
> asset.Cache.GetData<QuoteBar>().ToString()
"SPY: Bid: O: 144.2457 Bid: H: 144.2629 Bid: **L: 144.2457** Bid: C: 144.2629 Ask: O: 144.2543 Ask: H: 144.2889 Ask: **L: 144.2543** Ask: C: 144.2889 "
> asset.Cache.GetData<TradeBar>().ToString()
"SPY: O: 144.2543 H: 144.4532 **L: 143.4156** C: 144.2716 V: 75423"
* Improves Tick Resolution Unit Test
* Fixes Tick Resolution Case Handling
`master` only considers the latest trade, missing possible fills in the batch of trades.
* Adds Unit Test for Gap
See https://github.com/QuantConnect/Lean/issues/963
* Addresses Fill Optimistic Assumption
If we have a bar that gaps in our favor, we accept the limit price to avoid optimitic fills.
* Fixes Regression Tests
All regression tests with limit orders have worst performance after we remove the optimitic assumption, and use the limit price instead.
* Updates the PearsonCorrelationPairsTradingAlphaModelFrameworkAlgorithm
Change the `PearsonCorrelationPairsTradingAlphaModelFrameworkAlgorithm` logic to show that it doesn't remove the consolidators used in the Alpha Model's indicators.
* Fixes `BasePairsTradingAlphaModel`
The `BasePairsTradingAlphaModel` will create indicators with class constructors and register them to consolidators that will be removed when the security is removed from the universe.
* Addresses Peer-Review
* Update Oanda's Forex and Metals CFDs trading hours in the MHDB
* Fix failing unit tests
* Update regression algorithm statistics
* Fix failing unit tests
* Minor changes
* Fix Issue #6937
Some brokerage models don't check the order type in `CanSubmitOrder`.
As a result, unimplemented order types don't cause `CanSubmitOrder`
to return false as it should.
This commit fixes these issues for:
* `FTXBrokerageModel.cs`
* `InteractiveBrokersBrokerageModel.cs`
* `SamcoBrokerageModel.cs`
* `TradierBrokerageModel.cs`
* `ZerodhaBrokerageModel.cs`
Brokerage implementations can be found at
https://github.com/orgs/QuantConnect/repositories. For example, the
Zerodha Brokerage implementation is available at
https://github.com/QuantConnect/Lean.Brokerages.Zerodha.
Signed-off-by: Abhishek Kumar <abhi.kr.2100@gmail.com>
* Use HashSet instead of an array
Checking if an object is contained within a HashSet is faster than
checking the same in an array.
Signed-off-by: Abhishek Kumar <abhi.kr.2100@gmail.com>
* Minor adjustments and normalization
* Add missing OptionExercise for IB
---------
Signed-off-by: Abhishek Kumar <abhi.kr.2100@gmail.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Fix live trading chart series index
- Fix Series loosing their index when handled by the
LiveTradingResultHandler
- Some clean up for Series.cs
* Fix SeriesSampler loosing Series settings
* Adds NullBuyingPowerModel
The `NullBuyingPowerModel` considers that we have sufficient buying power for all orders.
Adds example using a bull call spread since a equity buy and hold would not show the impact of this model in the position group buing power model.
* Define NullBuyingPowerModel.GetMaintenanceMargin
The `NullBuyingPowerModel.GetMaintenanceMargin` returns a `MaintenanceMargin` of zero. It means the total margin used is always zero and margin calls will not be triggered.
This feature is inspired on the `ConstantBuyingPowerModel`. We don't inherit from that class, because `ConstantBuyingPowerModel.GetInitialMarginRequirement` gives us very small initial margin that leads to a very large quantity if we use `SetHoldings` or `CalculateOrderQuantity`
The calculations of `CloseTimeUtc` are correct and consistent: if the exchange is closed when the insight expires, `CloseTimeUtc` is the next open.
However, the definition of the `Period` was not consistent. Normally, it is defined by the different between `CloseTimeUtc` and `GeneratedTimeUtc`, but when we create an Insight with the Resolution and bar count overload, the period was defined by them:
```csharp
insight.Period = Resolution.ToTimeSpan().Multiply(BarCount);
```
which is incorrect, since it doesn't take the days the exchange is closed into account. The `ComputePeriod` method enforced consistency between the Resolution and bar Count overload and the `DateTime` overload, so the period for this overload was also incorrect.
* Custom data type history request in python
* Potential solutions
* Minor changes
* Use Slice.Get(Type) for getting python custom data history
* Minor changes
* Add unit tests
* Add unit tests
* Udpdate regression algorithms
* Peer review
* Peer review
* Add research regression tests
* Minor changes
* Minor changes
* Minor tweaks
---------
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Fix index option data timezone
- Fix index option data timezone which was 1 hour late than expected
* Update existing index option tests
- Update existing index option tests.
- Disable index option daily resolution support
* Fix future daily price scale factor update
- Fix future daily price scale factor update which was happening 1 day
late due to data and exchange time zone differences.
Adding regression algorithm reproducing and asserting it.
* Address review
* Update unsettled cashbook cash conversion rates
Also moved conversion rate holding from Cash to the CurrencyConversion
instance
* Minor fixes
* Peer review
* Minor fixes
* Minor fixes
* Minor fixes
* add wkhtmltopdf to lean image
* convert html to pdf
* avoid black stripts at bottom of each page
* address reviews
* address review
* address reviews
* self review
* address review
* Fix security cache reset
- Fix security cache reset so it only happens when no universe has it as
a member. Adding regression algorithm reproducing issue
* Avoid non deterministic FillForward behavior
* Fix default future chain universe selection
- Fix default future chain universe selection that should always return
empty, no selection. In some cases it would allos some contracts
though. Adding regression algorithm reproducing issue.
* Address review
* Minor debugging improvements
- Improve exception message if user provides empty/null ticker
- Log margin information on algorithn initialization
- Add unit test asserting option strategy position groups resolve
correctly for existing holdings
* Symbol Lazy SecurityDefinitionSymbolResolver
- Symbol will have a lazy SecurityDefinitionSymbolResolver instance
resolution
* Use any resolution for chain provider
- Use any data resolution available to source symbols for the file based
chain provider. Adding unit test
* Fix selection timezone bug
- Fix universe selection timezone bug. Updating regression algorithms
* Handles Data Subscriptions with One Security Type
Polygon sells data subscriptions for security types separately, so we should not test whether the users have all three subscriptions, Equity, Forex, and Crypto. Users just need the type they will use.
Adds new crypto markets, and unit tests for them. Adapt unit test.
* Fixes ToolBox Downloader
Adds `--api-key` argument.
Closes#6687
* Address reviews
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Expand Index option support
- Adjust APIs so that the same underlying can be used
for different options, adding support for SPX weekly options. Adding
regression tests
* Fix IndexOption.IsStandard
* Add IndexOption test
Fixed this error from happening when running this code:
tf.placeholder() is not compatible with eager execution.
at placeholder
raise RuntimeError("tf.placeholder() is not compatible with "
in array_ops.py: line 3341
at NetTrain
xs = tf.compat.v1.placeholder(tf.float32 in /QuantConnect/backtesting/./cache/algorithm/project/main.py: line 57
* add state to backtesting result packets
* self review
* add state in live results
* set runtime error in algorithm
* set hostname in job packets
* Set state when algorithm is null
* address review
* address reviews
* set runtime error when algorithm is not available during runtime
* Fix backtest progress calculation
* Backtest progress monitor class
* Minor changes
* Lazy calculating the processed days count
* Minor changes
* Minor changes
* Feature combo orders
- Add support for combo orders
* Make fill model wait for all grouped orders to emit fills
* Add ComboFill to model multiple fills for combo orders
* Fill combo limit orders
Add some regression algorithms
* Add fill implementation for combo leg limit orders
* Add IFill as common interface for Fill and ComboFill
* Refactor combo orders removing IGroupOrder interface
Move the group order manager to the base Order class
* Update algorithms
* Handle combo order events atomically
* Refactor brokerage transaction event handler
* Refactor combo fill models
* Process fills in batch
* Combo orders fill model tests
* Combo leg limit orders algorithm
* Regression algorithms cleanup
* Fill and combo fill classes cleanup
* Housekeeping
* Refactor equity fill model to derive from base fill model
* Address review changes request
* Handling the new types of orders in the OrderJsonConverter
* Add regression algorithm to test combo orders update/cancel
* Add regression algorithm to test combo orders update/cancel
* Housekeeping
* Address review changes request
* Minor changes
* Security transaction handler method for setting order request id
* Extend public interface for placing combo orders
* Combo order tickets demo algorithm python version
* Tweaks and updates
* Minor fixes
* Minor changes
* Minor fixes
* Address reviews minor fixes
* Minor fixes
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* CryptoFutures adjustments
- Address reviews of CryptoFutures:
- Add new Slice MarginInterestRates collection
- Add new regression tests asserting funding rate application behaves
the same no matter the resolution
- Add auxiliary data by type into the security cache
- Revert BuyingPowerModel changes
* Make interest rate application time deterministic
* Saves Dividend Payments to Security Holdings
Saves information about dividend payments to `SecurityHoldings`. This information will be used to factor in dividend payments to the `Profit` and `NetProfit`.
Adds `SecurityPortfolioManager.TotalNetProfit` to sum all the `SecurityHoldings.NetProfit`.
Adds regression and unit tests.
* Addresses Peer-Review
Improves Regression Test.
* WIP
* Add base currency cash
* Symbol properties and data processing
* Add basic template algorithm
* Add hourly crypto future algorithm
* Minor fixes after live trading testing
* CoinApiDataQueueHandler CryptoFuture support
* Address reviews
* Fix regression algorithms after update
* Allows Market-On-Close Orders Outside Buffer Period
Market-On-Close orders can be submitted before and after the buffer period from 15:45 to 16:00 (Tested with TWS) meaning that we can submit MOC when the market is closed and, consequently, use daily resolution data.
* Adds Regression Test with Extended Market Hours
- Removes `nextMarketClose > Time` condition since it's unnecessary. If the algorithm Time is greater than the close of that day, `nextMarektClose` refers to the next day.
* Updates Unit Test
Updates `OrderQuantityConversionTest` because the MOC orders are submitted. They are placed at 7 pm and invalid before this pull request.
* Updates Summary of new Regression Tests
* Set fill price to zero in OTM exercise orders.
Improved the OTM exercise orders message.
* Update regression algorithms and unit tests
* Add IsInTheMoney property to OrderEvent
* Update SerializedOrderEvent
* Properly setting the option exercise order price to strike price or zero
* Minor changes
* Minor changes
* Fixes Regression Algorithm Testing No Warm Up Requirement
* Removes Volatility Model Warm Up Requirement
If the volatiliy model is not ready, we will use the Brenner and Subrahmanyam (1988) approximation (BS88).
Lean will not set the volatility model if the user doesn't, since the BS88 can handle the null model case.
* Using IV to calculate Greeks, and remove single-step stochastic approximation
* Correct calculation for theta, vega, rho
* Add calculation from Black Calculator and peer review
* Address peer review and added unit test
* Update some tests and correct vega/eho
* Fix Unit Test and Improve Comments
Fixes `IndexOptionCallITMGreeksExpiryRegressionAlgorithm` since `Vega` was really non-zero.
* Fix regression test and add IV calculation
* refactor and bug fixing on peer review
* refactor and bug fixing on peer review
* for rerun test
* add warning on IV estimation not coveraged and edit speed unit test to not exceed 2s per 1000 iteration
* update logging
* update logging and description
* Add default option pricing models and unit tests
* address review
* Added Fed interest rate as risk-free rate with unit tests and set as default for option greeks calculation, added regression algorithms, addressed peer review
* refactor structure of interest rate
* Skip Saturday and Sunday
* regression test fix
* peer review
* Fixes Interest Rate Provider Logic
* Minor tweaks
* Fix start date
* Minor test tweak
* Update interest rates
* Fix unit tests
* Add minor log
Co-authored-by: Alexandre Catarino <AlexCatarino@users.noreply.github.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Limit exercise in IB brokerage for limit and cash-settled options
* Add regression algorithm
* Assert that index option is automatically exercised in regression algorithm
* Wolverine LEAN integration
* Wolverine Lean integration - adjusting comments and restrict to Equity only
* fix 0.005$ per share fee
* remove PostOnly from Wolverine as its not needed
* AccountType.Margin and also removed check for USD only currency
* fix Market Wolverine -> USA
* change config
* Minor tweaks
Co-authored-by: Mark Virchenko <mark.virchenko@redduck.io>
* Data monitor (FIRST TEST)
* Calculate data request frequency in data monitor
* Detecting universe data usage in data monitor
* Json data monitor report generation
* Data monitor refactor
* Log and store data monitor report
* Data monitor report tests
* Clean up
* Clean up
* Address changes request
* Address changes request
* Wire data monitor only for backtest, not for live or research
* Minor tweaks
* Some minor tweaks
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Refactor multi IDQH frontier
- Fix for multi IDQH frontier support, specially for IB + QC
- Adding new CompositeTimeProvide. Adding unit tests
* Adjust solution to fix new data available event
* Memory Related Performance improvements
- Make sure we cleanup & dipose of python related objects during pandas
data generation.
- Disable memoizing enumerable use while creating pandas data frames,
since we do not require it
- Reduce unrequired object creations
- Replace concurrentCollections for ordinary locks
* Decimal parsing typo fix
* Add python additional paths for research
* Delegate adding additional paths to pythonpath to PythonInitializer
* Add new way to keep algorithm location at the beginning of pythonpath
* Add switch to disable BeginAllowThreads in PythonInitializer
* Add switch to disable BeginAllowThreads in PythonInitializer
* Address changes request
* Adds Support to Visual Studio Code Dev Container
Adds support and documentation to VSCode Dev Container.
* Address Peer Review
- Adds Dockerfile to install QuantConnect Stubs to enable Python autocomplete.
- Adds settings.json to enable Python autocomplete and LF line ending.
- Removes autobuilder task.
- Fixes Research Laucher
* Launcher and Task Impromevents
Fixes line endings issue
Fixes Launcher to point to right config.json file
Improve readme formatting
* Simplify Jupyter Notebook Deployment
- In some rare cases the FillForwardEnumerator was missing to register
the delisting date of an asset because it was the first data it got.
Adding unit test reproducing issue
* Normalize time in universe behavior
- Normalize Option & Future chain universe behavior regarding their
assets time in universe. They will now respect the universe settings
time in universe value. Adding new regression algorithms asserting the
behavior
* Address reviews & cleanup
- Update QLNet library to latest version from 6/2019 -> 11/2021
- Fix evaluation date which was using settlement date. Adding unit tests
reproducing issue
Use `SetEndDate` to set `EndDate` in `QCAlgorithm` constructor. It will ensure that the `EndDate` is independent of the time the algorithm is executed if `SetEndDate` is not called in `Initilaize`.
Some users don't implement `SetEndDate` to run the algorithm to the latest datapoint, but this is not true if we run the algorithm during the day as the latest datapoint will be 24 hours before the execution time while there is data until the current day midnight.
* Adds Additional Condition to Parse SecurityIdentifier Properties
A string can pass the previous conditions and create an invalid `SecurityIdentifier` (e.g. "Sharpe ration"). If it is invalid, the Market is not supported (e.g. "378").
* Adds Unit Test to Pandas Indexing
* Log Only Once if TryParseProperties Cannot Parse
Use cache to increase speed and remove redundant logging
* docs(VIXDualThrustAlpha): `determine` fixup
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* docs(VixDualThrustAlpha): `unsubscribe` fixup
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* docs(BasicTemplateIndia): `Prperties` -> `Properties`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* docs: `Intialization` -> `Initialization`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* Get history of a given bar type in python
* Separate GetDataFrame in sub methods to avoid dynamic type resolution
* Housekeeping and unit tests update
* Use all requests in python type method history method
* Address changes request
* Reverted some requested changes
* Handle non-unique multi-index error for Ticks in data frame creation
* Update unit tests
* Update unit tests and add comments
* Add regression algorithm
* Fixes PythonData.EndTime
The `EndTime` property of `PythonData` didn't override the base implementation in `BaseData`. Therefore `EndTime` was always set as `Time`.
Fixes Regression Tests
- The C# version of `Bitcoin` class needs to implement `EndTime`-
- Fixes regression tests
* Minor tweaks and rebase
- Add support for python data types just setting EndTime. Adding unit
tests for PythonData
- Fix for DynamicData EndTime property being fetched. Updating unit
tests
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Add algorithm settings to backtest results
* Use algorithm result settings in report generation
* Add unit tests
* Housekeeping
* Changes for live result report
* Housekeeping
* Housekeeping
* Housekeeping
* Add unit tests
* Dynamically set wait time on WS error
- Dynamically adjust wait time on websocket error
* Avoid subscribing multiple times to message events
- Avoid websocket subscription multiple times to message events during
disconnection/connection cycle
* Add Dispose Brokerage WebSocket Subscription Manager
- Add check for live warmup history request, making sure there is at
least room for a single bar between start and end. Avoid sending
start==end when rouded down by the resolution causing some history
providers to fail
- Create SubmitOrderRequest method in algorithm API to reduce
duplication of code and responsability.
- Add order command tests for different failure and success cases
- Fix null reference exception in equity fill model, when there is no
data for an asset.
* docs: `excersizing` -> `exercising`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* docs: fix `exercising for exercising`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* docs: `migth` -> `might`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* Support extended market hours in AddFuture()
* Support extended market hours in AddFutureContract()
* Add C# regression algorithm
* Add Python regression algorithm
* Add regression algorithm for future contracts
* Add regression algorithm checking market hour ranges
* Fixed future regression algorithms to use extended market hours
* Fixed future regression algorithms to use extended market hours
* Fixed future regression algorithms to use extended market hours
* Fixed AddFutureOptionContractFromFutureChainRegressionAlgorithm to use extended market hours
* Update future market hours to include extended in market hours database
* Fixed AddFutureOptionContractDataStreamingRegressionAlgorithm to use extended market hours
* Fixed AddFutureOptionContractFromFutureChainRegressionAlgorithm to use extended market hours
* Fixed AddFutureContractWithContinuousRegressionAlgorithm to use extended market hours
* Fixed BasicTemplateContinuousFutureAlgorithm to use extended market hours
* Fixed BasicTemplateFuturesAlgorithm to use extended market hours
* Fix BasicTemplateFuturesDailyAlgorithm to use extended market hours
* Fixed BasicTemplateFuturesFrameworkAlgorithm to use extended market hours
* Fixed BasicTemplateFuturesHistoryAlgorithm to use extended market hours
* Fixed ContinuousBackMonthRawFutureRegressionAlgorithm to use extended market hours
* Fixed ContinuousFutureBackMonthRegressionAlgorithm to use extended market hours
* Fixed ContinuousFutureHistoryRegressionAlgorithm to use extended market hours
* Fixed ContinuousFutureLimitIfTouchedOrderRegressionAlgorithm to use extended market hours
* Fixed ContinuousFutureRegressionAlgorithm to use extended market hours
* Fixed DelistedFutureLiquidateRegressionAlgorithm to use extended market hours
* Fixed AutomaticIndicatorWarmupDataTypeRegressionAlgorithm to use extended market hours
* Fixed ConsolidateRegressionAlgorithm to use extended market hours
* Fixed DelistingFutureOptionRegressionAlgorithm to use extended market hours
* Fixed EqualWeightingPortfolioConstructionModelFutureRegressionAlgorithm to use extended market hours
* Fixed FutureContractsExtendedMarketHoursRegressionAlgorithm to use extended market hours
* Fixed FutureMarketOpenAndCloseRegressionAlgorithm to use extended market hours
* Fixed FutureMarketOpenConsolidatorRegressionAlgorithm to use extended market hours
* Fixed FutureOptionBuySellCallIntradayRegressionAlgorithm to use extended market hours
* Fixed FutureOptionCallITMExpiryRegressionAlgorithm to use extended market hours
* Fixed FutureOptionCallITMGreeksExpiryRegressionAlgorithm to use extended market hours
* Fixed FutureOptionCallOTMExpiryRegressionAlgorithm to use extended market hours
* Fixed FutureOptionDailyRegressionAlgorithm to use extended market hours
* Fixed FutureOptionHourlyRegressionAlgorithm to use extended market hours
* Fixed FutureOptionMultipleContractsInDifferentContractMonthsWithSameUnderlyingFutureRegressionAlgorithm to use extended market hours
* Fixed FutureOptionPutITMExpiryRegressionAlgorithm to use extended market hours
* Fixed FutureOptionPutOTMExpiryRegressionAlgorithm to use extended market hours
* Fixed FutureOptionShortCallITMExpiryRegressionAlgorithm to use extended market hours
* Fixed FutureOptionShortCallOTMExpiryRegressionAlgorithm to use extended market hours
* Fixed FutureOptionShortPutITMExpiryRegressionAlgorithm to use extended market hours
* Fixed FutureOptionShortPutOTMExpiryRegressionAlgorithm to use extended market hours
* Fixed FuturesAndFuturesOptionsExpiryTimeAndLiquidationRegressionAlgorithm to use extended market hours
* Fixed FuturesExpiredContractRegression to use extended market hours
* Fixed FutureSharingTickerRegressionAlgorithm to use extended market hours
* Fixed HistoryWithDifferentContinuousContractDepthOffsetsRegressionAlgorithm to use extended market hours
* Fixed HistoryWithDifferentDataMappingModeRegressionAlgorithm to use extended market hours
* Fixed HistoryWithDifferentDataNormalizationModeRegressionAlgorithm to use extended market hours
* Fixed LimitOrdersAreFilledAfterHoursForFuturesRegressionAlgorithm to use extended market hours
* Fixed OpenInterestFuturesRegressionAlgorithm to use extended market hours
* Fixed RegisterIndicatorRegressionAlgorithm to use extended market hours
* Fixed SetHoldingsFutureRegressionAlgorithm to use extended market hours
* Fixed WarmupFutureRegressionAlgorithm to use extended market hours
* Fixed AddFutureOptionSingleOptionChainSelectedInUniverseFilterRegressionAlgorithm to use extended market hours
* Fixed AlgorithmHistoryTests to use extended market hours for futures
* Fixed AlgorithmTradingTests to use extended market hours for futures
* Fixed BrokerageSetupHandlerTests to use extended market hours for futures
* Fixed TimeRulesTests to use extended market hours for futures
* Fixed FutureOptionMarginBuyingPowerModelTests to use extended market hours for futures
* Fixed FutureMarginBuyingPowerModelTests to use extended market hours for futures
* Fixed FileSystemDataFeedTests to use extended market hours for futures
* Fixed QuantBookHistoryTests to use extended market hours for futures
* Split BasicTemplateContinuousFutureAlgorithm to have an extended market version
* Fixed FutureMarketOpenAndCloseRegressionAlgorithm to use extended market hours
* Split BasicTemplateFuturesAlgorithm to have an extended market version
* Split BasicTemplateFuturesAlgorithm to have an extended market version
* Split BasicTemplateFuturesFrameworkAlgorithm to have an extended market version
* Split BasicTemplateFuturesHistoryAlgorithm to have an extended market version
* Revert AddFutureContractWithContinuousRegressionAlgorithm
* Revert AddFutureOptionContractDataStreamingRegressionAlgorithm and added data
* Revert AddFutureOptionContractFromFutureChainRegressionAlgorithm
* Revert AddFutureOptionSingleOptionChainSelectedInUniverseFilterRegressionAlgorithm
* Revert ConsolidateRegressionAlgorithm
* Revert Algorithm.CSharp/ContinuousBackMonthRawFutureRegressionAlgorithm.cs
* Revert ContinuousFutureBackMonthRegressionAlgorithm
* Revert ContinuousFutureHistoryRegressionAlgorithm
* Revert ContinuousFutureLimitIfTouchedOrderRegressionAlgorithm
* Revert ContinuousFutureRegressionAlgorithm
* Revert Algorithm.CSharp/DelistedFutureLiquidateRegressionAlgorithm.cs
* Revert EqualWeightingPortfolioConstructionModelFutureRegressionAlgorithm
* Split FutureMarketOpenAndCloseRegressionAlgorithm to have an extended market version
* Split FutureMarketOpenConsolidatorRegressionAlgorithm to have an extended market version
* Revert FutureOptionBuySellCallIntradayRegressionAlgorithm
* Revert FutureOptionCallITMExpiryRegressionAlgorithm
* Revert FutureOptionDailyRegressionAlgorithm
* Revert FutureOptionPutITMExpiryRegressionAlgorithm
* Revert FutureSharingTickerRegressionAlgorithm
* Revert FuturesAndFuturesOptionsExpiryTimeAndLiquidationRegressionAlgorithm
* Revert FuturesExpiredContractRegression
* Revert HistoryWithDifferentContinuousContractDepthOffsetsRegressionAlgorithm
* Revert HistoryWithDifferentDataMappingModeRegressionAlgorithm
* Revert HistoryWithDifferentDataNormalizationModeRegressionAlgorithm
* Revert OpenInterestFuturesRegressionAlgorithm
* Revert RegisterIndicatorRegressionAlgorithm
* Revert SetHoldingsFutureRegressionAlgorithm
* Revert WarmupFutureRegressionAlgorithm
* Revert AutomaticIndicatorWarmupDataTypeRegressionAlgorithm
* Some cleanup
* Address changes request
* Address changes request
* Add more Class III Milk data to fix DelistingFutureOptionDailyRegressionAlgorithm
* refactor(Algorithm): Correct `Intrinino` -> `Intrinio` references
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* refactor(Algorithm): Update `evemts` -> `events`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* refactor(BubbleAlgorithm): readability improvements
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* refactor(OrderTicketDemoAlgorithm): readability improvements
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* Fix KeyError when rendering exposure chart in report with some security types
* Update exposure chart colors and add unit tests
* Updated json test file
* handle multiple command files
* remove default useage
* add tests for verify files are read in order
* add comment
* change reuslt file name
* add comment
* liquidate using given symbol values
* remove redundant
* accept ticker, market, security type to build symbol, to be used by CLI commands
* use null leverage
* symbol can be null if ticker, market, security type values are provided
* reject the default value of security type
* cleanup
* cleanup
* check if status is not invalid
* cleanup
* abstraction to remove duplication
* fix name
* address reviews
* use counter
* add test to validate symbol/ticker agrumnets
* remove hardcoded values
* MRP
* Fix bug
* Rename
* Provide virtual `GetPriceRelative` function to override for any variants in future
* SymbolData class within model
* Address peer review
* fix bug
* revise unit test and fix bug in python model
* revise unit test and fix bug in model
* Revise regression statistics
* Revise regression statistics
* Revise regression statistics
* Revise regression statistics
* Revise regression statistics
* Address peer review
* Updated unit tests according to peer review
* Address peer review
* Fix LocalMarketHours.GetMarketClose to get the actual next market close time with extended market hours
The method was not properly considering gaps between regular hours close and extended market open
* Revert some changes to fix errors
* Add performance gh action
- Add new performance github action, running python and csharp benchmark
algorithms
- Adjusting algorithms length so they are not too long or too short
* Checkout master branch benchmark reference
- Fill model will take into account internal configurations, this is
specially relevant for continuous future mapped contracts. Updating
regression algorithms and unit tests.
- Allowing internal configurations to update trade builder information
* Remove suppport for market orders and MOO orders for futures
* Add regression algorithm to assert that limit orders are filled on after hours for futures
* Remove suppport for market orders and MOO orders for futures in DefaultBrokerageModel
* Update regression algorithms stats
* Fixed regression algorithm and clean up
* Add unit tests
* Fix regression algorithms
* Address changes request
* Respect warmup resolution given
- The data feed will respect the warmup resolution given and override
the resolution used by the algorithm when adding a subscription.
Updating regression algorithm to keep previous statistics. Adding new
regression algorithm asserting the desired behavior
* Testing improvements
- Add more unit tests and regresion test
- Add missing data for crypto
- Fix bug with FFed data crossing after the end time of the warmup
request
* Add more Warmup resolution regression algorithms
- Adding more warmup resolution regression algorithms, using
Settings.WarmupResolution and an option selection case
* Add more warmup regression tests
- Adding more warmup regression tests.
- Will no longer skip universe selection subscriptions from warmup
resolution enforcement. Updating regression algorithms data points
* Fix bug with data rounding
- Fix data rounding bug when warmup resolution is set to a different
value than the original configuration. Updating regression algorithms
to assert the expected behavior
* Address reviews
- Revert regression algorithms changes to use Resolution during warmup.
Updating their stats.
- Adding new regression algorithms asserting the behavior warming up
using a timespan and no warmup resolution
- Fix bug where data used to warmup the 'normal' enumerator will make it
through into the warmup time span. Updating tests
* Address reviews
- Add missing comments, explaning warmup algorithms time span
calculations.
- Revert changes in existing `WarmupOptionTimeSpanRegressionAlgorithm`
to reduce diff to minimum
- Adding new warmup unit tests asseting algorithm warmup start time, for
different combinations of bar count, timespan, resolution
* First commit
* Any change
* Add more unit tests
* Nit changes
* Changes requested
* Fix bugs and requested changes
* Requested changes
* More changes
* Check ongoing backtest orders
* Read ongoing backtest does not fail
* nit change
* Fixes for standarized API orders response
- Fixes for a standarized API read orders reponse
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Minor improvements
- Avoid division by zero error in backtesting result handler. Race
condition where the SetAlgorithm implementation would still be running
and we try to loop through the update method
- Improvement nullreference exception message
- Cleanup resolution to timespan extension method
* Address reviews
- Cleanup data config fill forward conditional statement for tick
resolution
* Fix live future and option universe selection daily resolution
- Adding unit test reproducing issue for option and future universe
chain using daily resolution, since was rounding down in UTC TZ the
date might not change and it would re emit the same data point
* Address reviews
- Add more test cases
- Adding locks around RegisteredSecurityDataTypesProvider to allow it to
be used by multiple threads at the same time without exploding
* Minor fixes for mapped futures contract
- Future security was never able to be non tradable
- Skip remove future contracts from live holdings. Adding unit test
reproducing issue and regression algorithm
* Fix removal of continuous futures
- The canonical continuous future would leave behind an OpenInterest
subscription when removed because of a different in the way the
configurations were built, this will now be centralized in a single
method. Adding unit tests reproducing issue
- FillForwarding setting of the continuous future was not being
respected. Updating algorithm reproducing the issue to assert the
behavior failing in master
* Address reviews
- Minor refactors addressing reviews
* Fix result handler live holdings
- Live result handler live holdings will include future assets
associated with continuous futures. Adding unit tests
* Fixed for live holdings price rounding
- Live holdings will use the SPDB minimum price variation for rounding.
Adding unit tests
- Remove thread in charge of sorting and updating the queues weight.
This will be handled by each worker thread instead. Avoid cases where
the work count is so high that the sorting thread in master would not
be able to update all the weights, causing data to be enqueued beyond
the max prefetch limit. Adding benchmark algorithm reproducing issue
- Smaller live warmup history requests, will keep track of the last point
we got from the file based enumerator and start our history enumeration from this point
* Add new Python QCAlgorithm.History() method with all parameters and type
* Add regression algorithms
* Using all parameters in History()
* Use private methods to reuse History() code
* Use private methods to reuse History() code
* Add unit tests for QCAlgorithm.Python.History()
* Add unit tests for QCAlgorithm.Python.History()
* Add unit tests for QCAlgorithm.Python.History()
* Add unit tests for QCAlgorithm.Python.History()
* Add unit tests for QCAlgorithm.Python.History()
* Add unit tests for QCAlgorithm.Python.History()
* Asserting history count
- Similar to the OptionChainUniverse, the future chain universe should
also add it's contracts using the raw data normalization mode. Adding
regression algorithm reproducing issue
- Replace logic to convert TimeSpan based PeriodConsolidator into count
based, by an override of the 'potentialStartTime' in the case we could
be falling into a look ahead consolidated bar end time
- Adding OpenInterestAnnual where the contract maps when any of the back month
contracts of the next year have a higher volume that the current front month.
- Add support for backwards compatible data mapping mode additions
- Updating ES map and factor files. Adding a little daily data too
- During warmup period the algorithms initial time might not be a rounded date
value, so it's important to take into account hours/minutes. Adding
regression algorithm reproducing and asserting issue
* Period timespan consolidation improvements
- If user is trying to consolidate a period providing data of a bigger
period we will now throw an exception. Adding tests
- If both consolidated and given data share the same period, gently
adjust the consolidator into a data count of 1. Adding tests
- Fixing bug in QuoteBarConsolidator period double accounting. Adding unit tests
* Add Period and Count regression algorithm
* Fix QuoteBar NonTimeBased Consolidator Period
- Fix QuoteBar non timebased consolidator period, that was accounting
twice for the initial bar period. Updating unit tests
* Improve QuoteBarConsolidator assertion
- Assert quote bar consolidated time and endtime in unit tests
- Add error log at OptionSymbol.IsOptionContractExpired in the case we
detect an unexpected expiration time. That could be related to
Symbol.ID.Date being incorrect
* Fix option price and greeks always beign zero on the expiration date
* Fix option price and greeks always beign zero on the expiration date
* Updated documentation
* Style and performace changes
* Address changes request
* Fix QL.Settings.includeReferenceDateEvents not being set on every thread
* Updated code documentation
* Add indicator for allowed option styles to QL option price models
* Add and update option price model tests
* Update option price models methods sumary to indicate allowed option styles
* Add regression algorithms for option price models for different option styles
* Update OptionPriceModel regression algorithms to use Lean local data
* Add Python regression algorithms for option price models for different option styles
* Update OptionPriceModel regression algorithms to assert that greeks are valid
* Address changes request
* Address changes request
* Update OptionPriceModel regression algorithms to check both call and put contracts
* Update OptionPriceModel regression algorithms to use correct test data
* Update OptionPriceModel regression algorithms to throw in OnData
* Address changes request
* Update OptionPriceModel regression algorithms to assert greeks are not all zero
* Live T-1 selection warmup
- Only use BaseDataCollection class
- Reuse collection enumerator
- Remove FuturesChainUniverseSubscriptionEnumeratorFactory
- Universe selection will use Cache providers
- Add null reference check
- Add more tests
- Fixes for warmup fill forwarding. Add more tests
* Address reviews. Add missing comments
* Refactor chain providers
- ChainProviders will use quote, open interest and trade information,
which ever is first. Updating regression algorithm changing contract
being added.
- Add base BacktestingChainProvider for shared logic, reducing
duplication.
- ChainProviders will now use the given IDataCacheProvider and
IMapFileProvider
- If Providers are called for a date the exchange is closed they will
search for the previous available date where the exchange is open.
Adding unit tests.
* Address reviews. Logging improvements
* use research-object-store-name key from config while creating object store for research notebooks
* initial removal
* remove reference
* move test to private repo containing downloaders
Since there is `UpdateStopPrice`, `UpdateLimitPrice`, `UpdateQuantity` and `UpdateTag` method to make it easier to update the `OrderTicket`, we include `UpdateTriggerPrice` to cover the `LimitIfTouched` case.
Also, updated the docs or the arguments for these methods.
Updates LimitIfTouchedRegressionAlgorithm:
- Adds `UpdateTriggerPrice` call that rounds down the `TriggerPrice`.
* use research-object-store-name key from config while creating object store for research notebooks
* remove bitfinex files
* remove redundant ref
* move test to private repo
* Warmup minor fixes
- Result handler fix
- Centralize and normalize status update during warmup
* Address reviews
* Minor adjustments
* Add console message during warmup period
* Fix warmup percentage update
* Overload PandasConverter.GetIndicatorDataFrame to accept a Python dict
* Shared implementation code for PandasConverter.GetIndicatorDataFrame overloads
* Added documentation for private shared methods used by PandasConverter.GetIndicatorDataFrame
* Added unit tests for PandasConverter.GetIndicatorDataFrame
* Added unit tests for PandasConverter.GetIndicatorDataFrame Dictionary overload
* Address change requests
* Address change requests
* Reproduce TrailingStopRiskManagementModel bug with test case
* Fix TrailingStopRiskManagementModel to make it relative to max price
* Adapted old TrailingStopRiskManagementModel test to new implementation
* Fix TrailingStopRiskManagementModel Python version
* Fixed TrailingStopRiskFrameworkAlgorithm regression tests data
* Handling both long and short positions in TrailingStopRiskManagementModel
* Traking holdings value instead of unrealized profit in TrailingStopRiskManagementModel
* Checking for position side change in TrailingStopRiskManagementModel
* Handling immediate liquidation in TrailingStopRiskManagementModel
* First commit
* Add unit test
* Change implementation
* Add unit tests
* Nit change
* nit change
* Remove unnecessary methods
* Add more unit tests
* Revert "Add more unit tests"
This reverts commit 1ba2ab7454.
* Add more unit tests
* Add more unit tests
* Improve Implementation
* Change unit tests
* Remove unit tests
Remove unit tests from AlgorithmRegisterIndicatorTests.cs
* Coarse fine flag improvement
- Coarse will check global data folder for fine existance if it's
different than the given path to check
* Update readme.md
- Minor fix for FuncPeriodSpecification so it always uses the same
DateTime to assert the given function. Improve documentation and
exception message being thrown
* Add missing stablecoins
* Fix bugs
* Remove MIMUSD stablecoin pair
That pair is already in the SPDB
* Add more stablecoins pairs
- Add SUSD and IDRT
* Fix bug
* Requested changes
The algorithm posted in the GH was failing because when finding a conversion between currencies BTC to USD, it wasn't taking into account BTCUSDT and that USD = USDT because is a stablecoin
* Solve bugs
* Nit changes and more tests
* Requested changes
* Nit changes
* Requested changes
* Nit change
* Allow DataWriter to merge existing ticks if desired
- Optionally allow lean data writer to merge new ticks with existing
- Adjust ZipStreamWriter
- Fix bug in compression lib that was leaving files open. Reproduced by
added tests.
* Address reviews
- Define new WritePolicy to configure the behavior of the LeanDataWriter
regarding how it should handle writing to a file: merge, overwrite,
append
* make public safe to expose test properties
* extract ib files
* remove dependencies
* refactor to use ib reference as string
* remove un-used dependencies
* remove redundant refrences
* Added checking if algorithm is stopped in initialize method
* Quit on initialize adjusment
- Minor adjustments to solution. Adding more regression algorithms
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Update Tick.cs Time parsing
* Adjust equity tick millisecond read
* Fix tick sub millisecond precision
- DateTime.AddMilliseconds will rount to the nearest integer. So instead
use AddTicks
* Avoid sporadic test failure
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* set price currency for open orders from brokerage setup handler, remove setters from brokerage
* order already has price currency being set by brokerage transcation handler
* remove as overwritten by BrokerageSetupHandler.GetOpenOrders
* refactor because can't use internal setters in private repos
* refactor, use ctor to set attributes
* remove redundant broker dependency
(cherry picked from commit 57b071eb174ca9fe857021ebffa7a546834e86f7)
* check limit price for limit order instead for checking price
* add price agrument for MarketOrder
* remove, not required anymore
* AddUnrequestedSecurity should not return null
* assert string in not null cases
- Fully reset the security cache when it's finally removed from the algorithm.
Adding regression algorithm reproducing issue
- Updating regression algorithms which would trade based on data
previously available
* Warm up EmaCrossAlphaModel
Warm Up EmaCrossAlphaModel indicators
* Fix regression test bug
When using the default `EmaCrossAlphaModel()` the period of both indicators to be ready is bigger than the difference between the start date and the end date of the algorithm. Then, as the algorithm didn't warm up the data both indicators of EmaCrossAlpha never were ready, but now as the model warms up the data both indicators are ready so we get different statistics
* Requested change
* Fix unit tests
As there wasn't items in `AddedSecurities`, when trying to remove the items in ´RemovedSecurities´ there was nothing to remove because there was never a security in `_symbolDataBySymbol`. That's why, in order to test, the behavior of `EmaCrossAlphaModel` when removing a security we need to first add one to then remove it.
* Requested changes in Python
- Requested changes in Python
- Nit changes
* Nit change
* Requested Changes
* Add RemoveConsolidators() method in Python version
* Add requested History overload
* Add more regression tests
* Revert "Add more regression tests"
This reverts commit 71b279e917.
* Add more regression tests
* add data count properties
* 'add history count property
* assert data counts
* update missing override
* consider override/virtual cases
* implement data count
* add message handler for regression tests
* use regression test message handler
* set algorithm manager for regression test message handler
* update data count
* check if stats are present, check if algo manager is not null
* update
* add c# algo
* make same as c# algo
* use new line
* logic shifted to RegressionTestMessageHandler
* cleanup
* auto cleanup
* skip non deterministic data count
* change data count
* use inheritance
* improve stats
* update couht
* add sma indicator to c# and customSMA to python
* call base method before executing further
* skip test
* revert to original
* add duplicate sma
* skip regression test
* Make continuous futures untradable
- To match live trading behavior. Adjust continuous futures securities
to be untradable
* Set ContinuousFuture as non tradable
- Set continuous futures as non tradable. Update regression algorithms
- Fixes for symbol capacity calculation
`WarmUpIndicator` for Python indicators doesn't return the indicator anymore after #6027. So all overloads should return `void`.
Fixes `SmaCrossUniverseSelectionAlgorithm` [C# and Py].
* Added NYSELIFFE exchange support to Interactive Brokers brokerage.
* Fixed unit test AddSecurityWithSymbol by adding market hours for Equity-nyseliffe-[*], Index-nyseliffe-[*], Option-nyseliffe-[*].
* Added NYSELIFFE exchange support to Interactive Brokers brokerage.
* Fixed unit test AddSecurityWithSymbol by adding market hours for Equity-nyseliffe-[*], Index-nyseliffe-[*], Option-nyseliffe-[*].
* Added NYSELIFFE exchange support to Interactive Brokers brokerage.
* Fixed unit test AddSecurityWithSymbol by adding market hours for Equity-nyseliffe-[*], Index-nyseliffe-[*], Option-nyseliffe-[*].
* - Added NYSELIFFE futures to Futures.cs
- Added method GetGoodFriday() to calculate Good Friday to FuturesExpiryUtilityFunctions.cs.
- Added unit tests for GetGoodFriday().
- Added NYSELIFFE futures expiries to FuturesExpiryFunctions.cs which uses GetGoodFriday() instead of hardcoding the date of Good Friday in the market-hours-database.json.
- Added unit tests for NYSELIFFE futures expiry.
- Added NYSELIFFE to Exchange.cs
- Removed Equity-nyseliffe-[*] and Index-nyseliffe-[*] from market-hours-database.json
- Updated Option-nyseliffe-[*] to FutureOption-nyseliffe-[*] on market-hours-database.json
- Added the supported NYSELIFFE futureoptions to symbol-properties-database.csv.
* Removed blanks after India tuple
* Added the dates for 2022 and 2023 to the market-hours-database.json.
Removed "FutureOption-nyseliffe-[]" from market-hours-database.json.
Added to FuturesOptionsSymbolMappings.cs:
{ "YG", "OYG" },
{ "ZG", "OZG" },
{ "ZI", "OZI" }
* Minor tweaks for nyseliffe FOPs
Co-authored-by: Chris Coetzee <chris@polidata.ai>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Deprecate breakpoints
Drop Watchlist
Remove breakpoints from lean
Prep install of NetCoreDbg
Cleanup
* Install NetCoreDbg
* Update Research Images with Newer Jupyter
* Binance.US base changes
* biniance exchange info update
* Binance US uses same fee rates as main Binance
* binance us brokerage model tests
* use base implementation of GetBuyingPowerModel method
* add gdax exchange info downloader
* add downloader method to program]
* fetch currency description
* change definition to include headers
* use extension method to make request
* remove log from test
* replace WebRequest
* cleanup
* use relevant name
* implement IExchangeInfoDownloader for bitfinex, initial commit
* add default values
* use default market value
* use correct attribute for lotsize
* don't skip missing values
* handle multiple downloaders
* add gdax and bitfinex exchange downloader
* follow LEAN data directory structure
* update SPDB
* order tickers
* order tickers
* add exchange info downloader test template
* delete files
* update SPDB
* use currency mapping
* update bitfinex symbols
* update currency mapping
* sort result after old currency symbols are used
* use market of the respective brokerage
* no more unknown symbol
* change minimum order size value
* direct conversion possible
* update bitfinex symbols
* change user-agent
* add test for indirect conversion
* update stats
* remove un-used
* use utc time for slice sync
* use utc time
* refactor
* add regression test
* use utc time
* use utc time
* make utctime required parameter
* add utcTime in slice creation
* assert warm up complete
* check if algorithm is still warmingup
* use exchange tz
`CashBuyingPowerModel`, which reflected on `AlphaStreamBrokerageModel`, a margin-only brokerage.
It also didn't consider the account type, so `InteractiveBrokersBrokerageModel` was using the margin model even if `AccountType.Cash` was selected.
Removes `GetBuyingPowerModel` method from other Brokerage Models when their cases are covered by `DefaultBrokerageModel`
Fixes some typoes in `TradierBrokerageModel`
Fixes unit and regression tests. For the regression tests, we have explicitly set the brokerage model.
- The BrokerTransactionHandler will also adjust virtual positions for
margin accounts when fees are in base currency and the asset is a
crypto or forex pair. Adding new regression tests reproducing issue
* extract binance brokerage
* remove binance tools from main toolbox app
* modify OrderTestParameters constructor
* fix BinanceBrokerage ref config.json
* update symbol properties
* move BinanceFeeModel tests to the right place
* initial commit
* get history from data providers
* merge history from history providers
* merge slice
* Add merge function in slice
* Update test
* merge aux data
* add data points in _data
* append new data points in original list
* Add test suite for HistoryProviderManager
* reduce complexity
* setup once
* add fake history provider
* doesn't count aux data
* add tests for options
* style changes
* add custom data
* merge rawDataList
* use array of history providers
* optimize
* fix formatting error
* add comment
* add tests
* simplify
* use list
* use abstraction
* split tests
* add test
* use abstraction to create generic enumertor class
* refactor
* accept list of type T
* sync history slices
* use SubscriptionDataReaderHistoryProvider for live
* rename test file
* return empty
* add tests
* cleanup
* address reviews
* optimize
* Fix method definition
* use initial time for basedata
* refactor
* re-use collection
* inherit HistoryProviderBase
* always return HistoryProviderManager
* add tests
* update rawDataList
* reset composer
* consider null elements
* add tests for binary search method
* Follow lean coding style
* revert
* remove binary search method
* convert to field
* Update MHDB and SPDB
Update MHDB and SPDB with micro futures
* Fix bugs
* Add more FutureExpiryFunctions
* Requested changes
* Nit changes
* Nit changes
- Fix some future values in SPDB
- Test LastFriday() method
- Apparently Micro CHF/USD Futures(MSF) refers to the micro of Swiss Franc Futures (6S). The same happens with Micro JPY/USD (MJY) Futures and Japanese Yen Futures (6J)
* Fix bugs
* Micro futures MultipleFactor
* Fix some futures MultipleFactor
* Adding unit tests reproducing issue.
* Fix a couple of minor bugs
- IsMarketOpen will work correctly when used with daily and hourly
resolution.
- slice.Get will work correctly with python custom data
- ExtendedDictionary will be able to dinamically access methods,
required for python and private C# data types
* Refactor solution. Add more tests
* Remove unrequired import statement
* WIP
* Fix fees in base currency not being subtracted
- Fix fees in base currency not being subtracted from the quote currency
for crypto cash accounts. Updating regression tests to assert
portfolio, cashbook state and holdings state.
* Fix unit test race condition
* Use GC server mode for tests
* Fix daily auxiliary data points emission time
- Due to fillforwarding, in some cases with daily resolution symbol
change events (generically any auxiliary data) would arrive late.
Updating regression test to reproduce the issue. Adding unit test
- Some refactoring and logging improvements
* Address reviews
* Remove old xml docs param
* change BinanceBrokerageModel
* supply spot vs margin endpoint as parameter
* wip
* send margin order for margin account
* "NEW" means that the order has been accepted by the Binance engine.
* fix cash balance
* use JsonConverters for account parsing
* unit tests
* fixup
* lazy connect
* fix connection
* allow api client to be null if DQH only
* make method private
* more unit tests
* fix Dispose
* fix tests
* fix IsConnected condition
* run test as additional
* add some comments
* improve unit tests
* improve null checks
* tidy up the code
* Refactor `GetFilePath()`
Add also useful methods to use with this one
* Nit changes
* Requested changes
* Requested changes
* Restore SaveString()
* Nit changes
* Address self review
* Test improvements
* Adjust example KerasNeuralNetworkAlgorithm
* Minor tweak for KerasNeuralNetworkAlgorithm.py
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Binance fees deducted from fill quantity accordengly
- For Binance cash accounts while buying, if fees are from the base
currency of leans virtual position, we need to deduct the fee from the
fill quantity, else we can end with a position bigger that it actually
is and not be able to liquidate
* Refactor solution
- Refactor solution into a more generic approach solving fees in base
currency at the BrokerageTransactionHandler level, covering all
brokerages that require it. Adding regression algorithm reproducing
issue.
- Update Bitfinex and Binance fee models to correctly reflact reality
* Log fill quantity adjusment once
* Add support for live price scaling
- Add support for live trading price scaling for continuous futures.
Adding unit tests
* Move live price scale application. Updating unit tests
- Live trading application of price scaling will happen before fill
forwarding and updating securities real time price. Updating unit
tests to reproduce issue
- GetLastKnownPrices will no longer guess which resolution to use but
rely on other methods implementation/
- Updating basic template future algorithms to warmup contracts and
assert it
- Minor improvements for FunSecurityInitializer and FuncSecuritySeeder
* Always reuse aggregator instance if any
- When fetching a IDataAggregator instance from the composer, do not
enfore type name on existing instances
* Fix unit tests
* add data
(cherry picked from commit 814011d89e5316d150f88ffca5f48d8d5f0ea7d9)
* update market hours for index
(cherry picked from commit edac40732c120eb84d27de00594b59eebb4983f5)
* add index algorithms
(cherry picked from commit b22d27b4fa98172c435f7c26de4a3a297c49a6b7)
* update statistics
* add cash
* Add india market
* add leverage for index
* can subscribe to index
* update format
* fix wrong cash
* fix ticker names
* update data
* update ticker and stats
* update docs
* specify enpoint url of ftx api (can be FTX pr FTX.US)
* more metadata for ftxus
* tidy up code
* tests
* more tests
* fix tests
* fix us fee rates
* add account tier
* update symbol props
* typo
* typo-2
* update symbol properties
* use FTXUS fee model
* minor tweaks
* replace to local functions as they are more performant
* fix random generator upper bound
Next() includes minValue, but not maxValue, so we increment it +1
* introduce abstract layers
* refactoring
* fix tets
* adapt tests
* fixup
* implement blackschole price model for options
* use risk free rate
* use ql price model
* wip
* change interface
* fix
* tidy up the code
* wip
* iterate groupped symbols
* wip
* wip
* fix
* allow symbol of different types
* improve settings
* wip
* iterate full range
* fix issue with negative option
* fix
* fixup
* use StandardDeviationOfReturnsVolatilityModel
* re-use existing tick types per security type
* parametrize underlying security type
* use default option style
* dynamic option price model
* fix enumeration
* test
* fix unit tests
* refactor code
* remove unused file
* minor tweaks and refactoring
* rename symbol generator class
* fix interface
* add comments
* more comments and unit tests
* more tests
* add disclaimer
* more tests
* more comments and tests
* split tests into different files
* tidy up the code
* tidy up the code; more tests
* refactor TickGenerator => use security price directly on each iteration
* remove dupe; reuse main constructor
* use SecurityManager, refactor code
* bugfix: save ticks in history array
* check volatility warm up & tests
* more unit tests
* describe volatility period span in settings
* rename command line option
* Minor adjusments. Address review
- Use Lean log handler instead of writting directly to console
- Rename BlackShcolesPriceGenerator to generically OptionPriceModelPriceGenerator
- Minor format clean up & standarization
- Add support for specifying the option chain size
* Rename TickGenerator private fields
* Fix unit tests
* fix tests class name
* Support tickers being specified
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
- Zip entries will be sourced from cache provider
- Option underlying will use SubscriptionDataSource to fetch it's data.
Fixing bug where it would let through an old data point, or miss
sending data through.
* Add india market data
* use local data for algo
* Add India market regression algo
* update india market data
* Update readme
* add python algo for BasicTemplateIndiaAlgorithm
* Add India data regression python algo
* data india data files
* update tickers
* Fix algo template
* remove data
* fix stats
* update stats
* remove unused data
- Live subscription enumerator will clone the underlying data set when
live mapping is being done. To avoid issues where IDQH implementations
could reuse a data point with same configurations. Adjusting unit test
to reproduce issue
* Fix typo
* Fix TryGetValue to return false if key is not found
* Revert "Fix TryGetValue to return false if key is not found"
This reverts commit b85b7b579a.
* Update documentation for TryGetValue
* Move Optimizer-related DTOs and JSON converters into Common/Optimizer
* Add REST methods for Optimization
* Move OptimizationStatus into Common
* Change optimizationId parameter type to string
* Update Optimization and add lightweight optimization object
* Rename lightweight optimization to BaseOptimization and remove unneccessary properties
* Remove snapshotId from Optimization, add ParameterSet to Backtest
* Add missing IApi.cs method signatures
* Move ParameterSet into Common
* Replace Backtest with OptimizationBacktest
* Update UpdateOptimization to not include null or empty name and layout params in the request
* Change Objective targetTemplate regex pattern from ['(.+)'] to (.+) to prevent escaping target strings without whitespace
* Return Estimate object when calling EstimateOptimization
* Use DefaultNamingStrategy when serializing constraint operators
* Revert "Change Objective targetTemplate regex pattern from ['(.+)'] to (.+) to prevent escaping target strings without whitespace"
This reverts commit fbe7de0fd7.
* Update Api method signatures
* Add unit tests
* Fix XML comment referring to the old class name
* Fix XML summary for OptimizationResponseWrapper
* Address review feedback
- Remove unused testOrganizationId
- Change NodeType from string to NodeType enum
- Clarify unit types for Estimate time and balance
- Simplify JsonConverter classes
* Add accessors to Common/Api classes
* Define performance metrics names in PerformanceMetrics class
* Remove unnecessary branching logic from GetSeriesValues method
* Add crefs and examples to XML comments in the Api class
* Revert "Change NodeType from string to NodeType enum"
* Remove layout param from UpdateOptimization method
* Backtest property ParameterSet should be of type ParameterSet
* Add asserts for deserialization in OptimizationBacktestJsonConverterTests
* Replace the three target-related properties with Criterion
* Add serialization and deserialization tests for Optimization
* Remove Optimization Serialization test
* Add EstimateDeserialization test
* Add asserts for integration tests
* Address self review
* Revert test case
* Update Nodes.cs
* Update Nodes.cs
* Set Aborted status when Optimization fails to start
* Add ParameterSetJsonConverter and ParameterSetJsonConverterTests
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Do not send internal SecurityChanges to Algorithm
- Will not send internal security changes to the Algorithm by default.
Following custom security changes filter pattern. Updating regression
algorithms to assert behavior.
- The universe member will know wether it was added with internal
configurations or not
* Address reviews use a separate collection for internals
* Refactor solution. Adding security changes constructor class
* GH-6115: adjust dates
* fix
* use another approach
* Add lean data writter multiple days data unit test
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Handle security added and removed in the same loop
- Correctly handle adding and removing a security in the same loop.
Adding regression test
* Adding comments
* Align universe security and configuration TZ
- Make sure universe security and configuration tz are aligned always.
Adding unit test reproducing issue
* Address review UserDefinedUniverse will use UTC TZ
* CompositeDataQueueHandler handles duplicate configs
- CompositeDataQueueHandler will handle duplicate subscriptions sharing
the same config. Adding unit tests
* Refactor bug solution after review
- LiveSubscriptionEnumerator will not long perform any symbol mapping
but will just trigger a new subscription call when remapped.
- CompositeDHQ will handling symbol mapping and keep track of these
mapped configs to trigger unsubscribe accordengly. Adding unit tests
* Adjust solution after review
- To avoid breaking uniqueness of SubscriptionDataConfig will be adding
a private `mapped` property that will change equality of mapped
configs to avoid them clashing. Adding unit tests
* Rename IDQH subscription extension methods
* Minor performance improvements
* CompositeDataQueueHandler is UniverseProvider
- CompositeDataQueueHandler implements the UniverseProvider interface.
Adding unit tests
* Indicator
* Fixed logic error
* Removed old files, added KER into KAMA
* Removed old comments
* Added requested changes
* Minor clean up
* Refactors KaufmanEfficiencyRatio and KaufmanAdaptiveMovingAverage
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
Co-authored-by: Alexandre Catarino <AlexCatarino@users.noreply.github.com>
* Updates KeltnerChannels to Update MiddleBand with EndTime
* Adds Unit Test
This unit test shows that the timestamp of the MiddleBand and the indication are aligned as expected.
* initial commit
* Follow IDQH implementation
* Expect a list of data handlers from LiveNodePacket
* Return null if can not subscribe
* refctor to add check for subscription
* initialze null
* Add tests
* Check subscribe retuns null/not-null
* cleanup
* Read all required IDQH credentails to job
* Use CDQH to handle all IDQH instances
* constructor abstraction to call from setjob
* use flag
* remove redundant because derived will call initialize on it
* abstract and initialize from setjob
* handle null enumerators
* get creds from data handlers
* handle single data handler value from data-queue-handler
* Fix to support a json array
* Fix missed constructor call
* change access modifier to access from Tests files
* Add test to get brokerageFactory from dataQueueHandler
* Fix init flag to handle all conditions
* Add docs
* initialize from setjob
* Check if websocket open before using
* change defination of initialzie to include tradier
* clean up
* change defination
* fix wrong api key name
* return empty enumerator
* check websocket open before sending request
* check connection before subscribing
* fix to include more cases
* check websocket open before sending request
* Minor refactoring
* reafctor and use IsConnected
* remove unused
* clean up
* Fix test cases
* reverse change
* include config changes
* connect to websocket from setjob
* check websocket connection from setjob
* clean up
* include condition for IDQH that are not brokerage
* Address review
* Add market check condition before subscribe
* Remove deprecated
* Minor fix for deserializing data queue handler
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
- Fix for adding continuous future and manually adding contract at the
same time. Continuous future was using the user defined universe
symbol and caused clashes. Adding regression test reproducing the issue.
- Fix for IB option position update race condition, where the BTH would
incorrectly assume it should trigger an early option assignment. Adding unit tests
* Give `code()` more time per iteration
* Another proposed solution
* Revert "Another proposed solution"
This reverts commit 29bf5a7554.
* Add an AutoResetEvent field in `TimeConsumer()`
* Avoid change TimeMonitor implementation
* Remove changes in `TimeConsumer.cs`
* Requested changes
* Avoid re-implementation
* Continuous Futures Refactor. Live Mappings
- Adding support for live mappings. LiveTradingDataFeed will handle any
symbol mapping at the configuration layer and resubscribe through the
IDQH
- Refactoring continuous futures adding ContinuousFutureUniverse that
will select the currently mapped security
* Minor fixes
- Remove addition of configurations in UniverseSelection step, leave
resposability for universe.
- LiveTradingDF future unit test will only assert slice data for non
internal feeds.
- ContinuousContractUniverse will respect internal option interest
subscription
* Address review
* Properly handle pending CancelOrderRequests
To avoid unneccessary "Insufficient buying power to complete order" errors showing up as Invalid orders we need to skip further processing of pending CancelOrderRequests in the BacktestingBrokerage and instead let them be properly removed during the next transaction handler run.
* Update BacktestingBrokerage.cs
* Activate Index and IndexOptions low res
* Add Index and IndexOption low res data
* Add IndexOption regressions
* Fix bug with IndexOption QuoteBar data not being scaled
* nit - update comment for _scaleFactor
* Update data and regressions to reflect fix in data
* Tweak regressions
* Fixes for FutureOptions support in LeanData
* Add CreateCanonicalOption() utility function for Symbol.cs
* Add aggregated Futures/FuturesOptions data to Lean
* Add FutureOptions regressions for daily/hourly data
* Allow Futures to be added with low resolution
* Add Future regressions using hour/daily data
* Nit - Python Class names
* Add reviews
* Add alias into CreateCanonicalOption
* Add tick type to data downloader get (price data) call in order for api's with rate limits to handle not making an api call at all.
* Added DTO object to pass into data downloader get
* Added missing header for new file
* Address review. Minor tweaks
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Fix for python enumerable data history request
- Fix for python enumerable type data history request. Adding unit test.
* Add methods for adding data points into a baseDataCollection
* Move processing of delistings to Brokerage
* Deal with case that exchange is not open on OptionSymbol.ID.Date
* Refactor solution to use DelistingNotification event
* Adjust some regression expected liquidation time
* Mark some todos on deprecated functions
* Update expected liqudation time for Py regressions
* Update regressions that have been validated
* Use HandlePositionAssigned for assignment orders
* Update regressions
* Update some missed unit tests; remove one that is already covered by regression
* Cleanup deprecated backend functions
* nit - small cleanup adjustment
* Post rebase fix
* Address review
* Minor tweak to py regression
* Super Trend Indicator #4653
* Updated test data
Previous test data was wrong
* Reduced if statement
* Updated tests
Replacing spy test data with dwac test data from trading view.
* Minor comment update
* Minor tweaks
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* First BetaIndicator prototype
- In construction
* Fist BetaIndicator version and unit tests
* More unit tests and regression test
* Nit change
* Requested changes
* Nit changes
* Requested changes
* Adjust beta formula slightly and nit changes
* Nit change
- Add new CustomWeight PCM for alpha streams
- Add new AlphaStreams AlphaModule that will handle security additions
and removals, removing this logic from AlphaStreamsBasicTemplateAlgo
* Remove user plan enum
- Get maximum order and runtime from job controls
- Remove user plan enum definitions
* Add log for BacktestingSetupHandler maximums
* Create generic writing for LeanDataWriter, + notes on todos
* Make Options Daily/Hourly data store by year
* Refactor Generic Write
* Permit hour and daily resolutions for options
* Refactor writer to merge when needed with other files
* Cleanup redundancies, run write tasks in parallel
* Make needed classes/vars available
* Update tests to reflect new naming convention for daily hourly options data
* Add Byte[] overloads for ZipData functions in compression
* Implemented Store() for ZipDataCacheProvider
* Have LeanDataWriter use a DataCacheProvider
* ZipDataCacheProvider cleanup
* ZipDataCacheProvider tweaks, doesn't support storing non-zips
* Test adjustments
* Update LeanDataWriter to use Write instead of SaveDailyHourly/SaveMinuteSecond
* Implement tests to verify DownloadAndSave behavior
* Nit cleanup on DownloadAndSave tests
* Fix for options daily/hourly underlying equity subscription read
* Add daily/hourly options data and regressions
* Add missing open interest for hourly
* Fix writing of OpenInterest Daily/Hourly data
* Update data
* Fix Date typo in regression
* Use daily algorithm to test delisting
* Revisions part 1
* Expand test for DataCacheProviders; refactor DiskDataCacheProvider
* nit - test adjustments
* ZipDataCacheProvider test setup refactor
* Adjust multithreaded read/write test; fixes for ZipDataCacheProvider
* Move DiskDataCacheProvider to its own file and add write test
* Remove _appendToZips; always overwrite entry or create zip
* Add mapping regression for daily options
* nit - add license to regression
* Fix Tick write case where more than one data point for a DateTime
* Fix data issue
* Address review
* Tweaks for tests
* Stop Store() early if no entry name is given
* Warm up MACD indicators
- When a security is added in the MACD alpha model, it's warm up at once
* Add unit tests
* Nit change
* Code style and nit changes
* Implement IIndicatorWarmUpPeriodProvider
- Implement IIndicatorWarmUpPeriodProvider in PythonIndicator.cs
- Make a unit test to check whether the WarmUpPeriod is working as expected
- Make a regression test to check the new feature at a system level
* Nit change
* Change Period parameter for WarmUpPeriod parameter
- Change regression test to check if the new parameter keep backwards compatibility with indicators that do not set WarmUpPeriod
* Documentation change
* Fix tests bugs
- In CommonIndicatorTests.cs before finish the test it checks the period.value with the number of samples but for default the period.value was set to -1
* Change names
* Change WarmUp and RegisterIndicator methods
- Lean WarmUp indicator skip custom python indicators that don't define WarmUpPeriod parameter
* Call WarmUpIndicator manually
- Add a new "bridge" method called WarmUpIndicator in QCAlgorithm.Python.cs to set up everything to call WarmUpIndicator in QCAlgorithm.Indicators.cs
- Change the regression algorithm to warm up the indicators manually
* Remove unnecessary code and add more tests
* Nit change
* Revert "Nit change"
This reverts commit da411f59c9.
* Fix bugs
* Try fix bugs
* Add C# regression test
- More nit changes
- Fix bugs
* Requested changes
* Remove unnecessary code
* Requested changes
* Nit changes
- Add new Python class to check a custom indicator, which doesn't inherits from PythonIndicator, warms up properly
* Reduce redundant code
* Fix bug and add more unit and regression tests
* - Add more unit tests
* Nit change
* Test cleanup
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
- Remove unnecessary attribute `_period` in MarketProfile
- Set the value of WarmUpPeriod to the period in MarketProfile constructor
- Fix unit test to check VolumeProfile and TimeProfile indicators are being warmed up properly and its WarmUpPeriod parameter isn't zero
- If available, CoinApi converter will read previous date files to
rescue initial midnight ticks for the processing date. Also make sure
to drop ticks from tomorrow
* Implement IIndicatorWarmUpPeriodProvider
- Implement IIndicatorWarmUpPeriodProvider in PythonIndicator.cs
- Make a unit test to check whether the WarmUpPeriod is working as expected
- Make a regression test to check the new feature at a system level
* Nit change
* Change Period parameter for WarmUpPeriod parameter
- Change regression test to check if the new parameter keep backwards compatibility with indicators that do not set WarmUpPeriod
* Documentation change
* Fix tests bugs
- In CommonIndicatorTests.cs before finish the test it checks the period.value with the number of samples but for default the period.value was set to -1
* Change names
* Change WarmUp and RegisterIndicator methods
- Lean WarmUp indicator skip custom python indicators that don't define WarmUpPeriod parameter
* Call WarmUpIndicator manually
- Add a new "bridge" method called WarmUpIndicator in QCAlgorithm.Python.cs to set up everything to call WarmUpIndicator in QCAlgorithm.Indicators.cs
- Change the regression algorithm to warm up the indicators manually
* Remove unnecessary code and add more tests
* Nit change
* Revert "Nit change"
This reverts commit da411f59c9.
* Fix bugs
* Try fix bugs
* Add C# regression test
- More nit changes
- Fix bugs
* Requested changes
* Remove unnecessary code
* Requested changes
* Nit changes
- Add new Python class to check a custom indicator, which doesn't inherits from PythonIndicator, warms up properly
* Reduce redundant code
* Refactor error message for reproducability
* WIP Refactor GetMaximumOrderQuantityForTargetBuyingPower
* Additional tweaks and tests
* Address shorted margin case
* Reinstate tests with new modified function
* Update regression
* Some cleanup
* Expand test set
* Refactor solution
* Address review
* Final adjustments and cleanup
* Expand error message for GetAmountToOrder for safety and reproducibility
* Address reviews
* Add base class for pivotpointshighlow
* abstracts do not work well with python switch to virtuals and use pivotpointhighlow as base implementation
* added missing documentatino header
* moved reset back to original location
- Implement IIndicatorWarmUpPeriodProvider interface in both indicators
- Add test to check the new functionality is working as expected. To test WindowIndicator IIndicatorWarmUpPeriodProvider there was used WindowIdentity indicator tests because that indicator only inherits from it
* Update SPDB with price magnifier parameter
- Add a new column for new parameter in symbol-properties-database.csv
- Change InteractiveBrokersBrokerage.cs implementation of GetContractPriceMagnifier() method in order to get the parameter directly from the SPDB
- Add test in SymbolPropertiesDatabaseTests.cs to check if the new parameter is loading properly
* Change GetContractPriceMagnifier() method name and implementation
- Change NormalizePriceToLean() and NormalizePriceToBrokerage() methods
* Nit change in GetSymbolPriceMagnifier() method
* Remove GetSymbolPriceMagnifier() method
- Remove redundant and unnecessary parameters in NormalizePriceToLean() and NormalizePriceToBrokerage()
* Requested changes and SPDB update
- Update SPDB with minimum order size parameter for futures in cents. The minimum order size parameter was extracted from the CME group API
* Address reviews. Adding unit test for failing case
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* fix exchangeinfo downloader
* normalize data, update symbol properties
* min ordersize should be in base currency
* check min order size using quote currency price
* check order size in quote currency
* improve checks
* improve code; more unit tests
* more description
* fix failing test
* Move Aggregate Quote and Trade Bars to utils
* Add decimal and long overload to UnixTimeStampToDateTime
* Add Kraken OrderType check in CanSubmitOrder
* PR !5969 review fixes
* Implement scheduled event sampling solution
* Use UTC time, only update daily portfolio value once a day
* For daily resolutions sample chart always
* Cleanup
* Drop resample daily all together
* Force final sample
* Regression updates
* FIx LiveResultHandler to update portfolio and benchmark values outside of sampling event
* Name the daily sampling event
* Address review pt 1
* Drop force and use reference wrapper
* Adjust tests
* Fix warning for Benchmark Timezone Misalignment and also add test
* Fix for daily resolution orders and test adjustments
* Also warn on universe settings with daily resolution
* Update missed regression
* Fix reference wrapper use
* Update regression after rebase
* Add values back in for Daylight Algo
* Have statistics builder skip day 1 performance
* Regression adjustments
* Test adjustments
* Update regression unit test
* Adjust some regressions starts to show performance values
* Add hourly algorithm for beta comparison
* Address missing Python regression changes
* Remove null comment
* Bitfinex Fee adjustment
- Fix for bitfinex fee adjustment for cash account type only happen if
the currencies are the expected ones, else log message.
* Adjust error handling to terminate algorithm
* Add bitfinex TESTBTCTESTUSD
* Minor improvements
- Add bitfinex test symbols to the SPDB
- Make SecurityDefinitionSymbolResolver log using trace
- Minor tweak for Bitfinex DQH lock contention
* - Update symbol-properties-database.csv, SymbolPropertiesDatabase.cs and SymbolProperties.cs with a new column for MinimumOrderSize parameter for Crypto type
- Change CanSubmitOrder() method implementation in GDAXBrokerageModel.cs to get the MinimumOrderSize from the security SymbolProperties directly
- Add CanSubmitOrder() method in Binance, Bitfinex and Kraken Brokerages models
- Add tests to check if CanSubmitOrder() method is working as expected in Binance, Bitfinex, Kraken and GDAX brokerages models
- Add tests to check if MinimumOrderSize parameter is loading correctly from symbol-properties-database.csv
* Revert "- Update symbol-properties-database.csv, SymbolPropertiesDatabase.cs and SymbolProperties.cs with a new column for MinimumOrderSize parameter for Crypto type"
This reverts commit e0fd66a5fa.
* Revert "Revert "- Update symbol-properties-database.csv, SymbolPropertiesDatabase.cs and SymbolProperties.cs with a new column for MinimumOrderSize parameter for Crypto type""
This reverts commit b0f37935fc.
* Requested changes in Crypto brokerage models
- Remove unnecessary commas in symbols-properties-database.csv
- Add TestHelpers class to reuse GetSecurity() method
- Add IsValidOrderSize() method in DefaultBrokerageModel class
- Remove unnecessary classes for brokerage models test helpers
* Nit change
- Remove unnecessary attribute from DefaultBrokerageModel.cs
* Nit change
* Nit changes and Regression test
- Add Bitfinex BTCUSD hourly data for 2 days
- Add regression test BuyBTCWithLessThanOneDollarAlgorithm.cs to test that the order size is taking into account when placing or updating an order
- Nit changes
* Change CanUpdateOrder() implementation
- CanUpdateOrder() now checks if the requested quantity value is valid
* Requested Changes
- Change regression algorithm BuyBTCWithLessThanOneDollar.cs name and implementation
* Nit changes
* Add BaseDataCollectionAggregatorReader
- Add enumerable for BaseDataCollection
- Add BaseDataCollectionAggregatorReader
- Clean up CollectionSubscriptionDataSourceReader
* Address reviews
- Add new FileFormat that will be handled by the new Reader
- Adding unit tests for the new BaseDataCollectionAggregatorReader
- Some DataSourceReader duplication cleaning up
* Minor adjustment after some live trading tests
Adds Warnings to MOO and MOO using Tick Data
These warnings are meant to let the user know that it was not possible to find ticks marked with the `OfficialOpen`, `OfficialClose`, `OpeningPrints` or `ClosingPrints` flags.
* Fixes Market On Close Fill of Equity Fill Model
Only use trade data (`Tick` with `TickTrade` type or `TradeBar`) to get the closing price, since MOC is filled with the closing price.
- Fix unit tests to show that the new implementation only fills with trade data from the current open market.
- Change regression tests to reflect the bug fix. All other changes are the consequence of using `TradeBar` instead of `QuoteBar`.
* Address Peer-Review
Fixes the logic with tick data: looking for the OfficalClose or ClosingPrints only apply if data from the extended market hours is included.
Adds additional logic to handle missing trade data.
* Refactors Timeout Logic and Adds It to MarketOnOpenFIll
-Refactors timeout logic to address peer-review and adds it to MOO fill too.
- Adds unit tests
* Fix IB futures prices where PriceMagnifier != 1
* Fix deadlock in GetOpenOrdersInternal by moving ConvertOrder call out of event handler;
Minor refactor to combine NormalizePrice and ConvertPrice methods
* IB future and future options price magnifier
* Address review. IB API clean up
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Lean exchanges improvements
- Adding new Exchange class to avoid exchange code clash.
Adding and updating unit test
* Add Market for MapFile API
* Self review
* Rename Renko to ClassicRenko
* Rename WickedRenko to Renko
* Adjust tests and usages
* Backwards support WickedRenkoConsolidator
* Renames for consistency
* Refactor for two types in composite indicator
* Fixes for unit tests
* Non generic indicator base
* Further seperate non-generic indicator base and generic input base
* Adjustments
* Remove generic CompositeIndicator
* Add test suite
* Make each method an individual case
* Address review
* Testing cleanup
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Adds support for getting a Lean symbol based on FIGI, CUSIP, ISIN, SEDOL
* Address review: bug fixes for mapping, improve docs, cleanup & refactor
* Make LocalZipMapFileProvider and LocalZipFactorFileProvider only
initialize if they haven't been initialized yet.
* Address review: makes ISIN, SEDOL, CUSIP case-insensitive
* Cleans up map file provider in SecurityDefinitionSymbolResolver
* Modifies some test cases to test for case-insensitivity
* Add null check for SecurityDefinitions
* Fix unit tests
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Add ValueAreaVolumePercentage Parameter
Remove hard coding of parameter and set it to optional parameter for the user.
* Update TimeProfile.cs to include valueAreaVolumePercentage parameter
* Update TimeProfile definition in QCAlgorithm.Indicators.cs
* Create Market Profile indicator.
* - Requested changes in MarketProfile indicator made it.
- TimeProfile and VolumeProfile created and tested.
- tp_datatest.csv and vp_datatest.csv extracted from https://github.com/bfolkens/py-market-profile
* Final potential Market Profile Indicator.
All unit test made it and the indicator is passing all of them
Test cases made it with python library from GH issue
* Code styling request changes
* Minor suggestions
* Minor renaming tweaks
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Few improvements
- Alpha Streams algorithm will add target securities right away
- If algorithm is warming up PortfolioTargetCollection will emit not
insight
- Fix local disk factor file provider to use local cache instead of data
folder
- EWAS PCM will not emit targets for securities which have not been
added by the algorithm yet, to avoid runtime exception
* Fix GetLastKnownPrice default order adding more unit tests
As most brokers use separate exchange codes for CBOE and CFE, we added the new Market.CFE (previously Market.CBOE was used for both exchanges). This change also avoids hacky code in symbol mappers.
* Fix zerodha async order event
consider fillQuantity as total filled quantity
cover edge case of fully filled
fix async events received from zerodha
indentation fix
* Fix coding style
* keep order in only CachedOrderIDs
* Alpha Streams improvements
- Warmup added securities so we can trigger an order right away
- Fix bug where AddSecurity returned a security which was not the one
being used
- Fix bug where EWAS PCM would remove from a dictionary while iterating
over it
* Fix unit tests
* Add QCAlgorithm.GetLastKnownPrices API
- Add new GetLastKnownPrices that will return the last known data point
for all subscribed data types.
- Fix for MHDB GetDataTimeZone which was using an invalid entry key
format for custom data.
- Fix for using GetLastKnownPrice/s to seed a security during creation,
when it's not added in the Algorithm.Securities collection
* Adding more unit tests for GetLastKnownPrice
* Address reviews
- Refactor GetLastKnownPrice/s to perform a single history request for
all data types
- After https://github.com/QuantConnect/Lean/pull/5872 trading API
changes numpy float64 was not converted correctly by pythonNet and
used an int. Reverting API changes and adding regression test. This
should be fixed at pythonNet layer
* Adds ETF(...) to UniverseDefinitions
* Adds ETF constituents universe framework regression algorithm
for C#/Python
* Address review: adds test cases for ticker/Symbol ETF universe additions
* Fixes bug where null Market would result in null dereference exception
* Address review: add missing Index tests
* Address review: don't hardcode market when creating constituent universe
* Uses Brokerage Model's default markets collection to determine
the market for the given security type
* Address review: restore QC500 and DollarVolume.Top(...)
* Restores algorithms related to both helper universe
definition methods
* Address review: remove copy to output directory for python algos
* Add example algorithms for ETF constituent universes using custom RSI alpha model
* Address review: adjust algorithm to use cache + algo RSI & clean up code
* Address review: make ETF Constituent RSI Alpha Model algo a regression test
* Address review: increase trade count and remove single trade logic
* Order handling improvements
- Execution model will only trigger market order if they are above the
minimum order margin portfolio percetage value
- SecurityCache.Reset is complete
* Python Import fixes
- Add regression test for ImmediateExecutionModel minimum order margin
check
* Add basic template Atreyu algorithm
- Add C# and Python basic template atreyu algorithm. Show casing how to
specify exchange to execute in different ways.
- Adjust trading API to allow specifying order properties to use
* Lean Exchange improvements
- Rename PrimaryExchange to Exchange
- OrderPropeties will use Exchange enum instead of string
- Adding BSE exchange value
* Regression tests fixes
* To fix the referenced issue
* Fixing up TrailingStopRiskManagementModel
* Fix-ups to address review
* Impl. tests + relevant improvements
* Minor logic improvement at first dictionary update
* Regression test small fix
* Adds explicit 'D' suffix for numbers in test case double arr
* Use integer values in test cases
* Removes failing testcases in MaximumDrawdownPercentPerSecurityTests (!) & renaming
* Revert "Removes failing testcases in MaximumDrawdownPercentPerSecurityTests (!) & renaming"
This reverts commit f9cd279f8c.
* Fix up for failing test cases
* Alpha holdings state
- Alpha result packet will optionally provide the algorithms portfolio
state
* Rename
* Convert AlphaStreamsPortfolio to data source
* Improvements on AlphaStreams algorithm
* Fix regression tests
* Add unit tests for EW AS PCM and fixing bugs
* Protobuf AlphaStreamsPortfoliot staState
- Protobuf AlphaStreamsPortfolioState. Adding unit tests
- Add variable TPV tests for EW ASPCM
* Add alpha license to Organization response
* Improvements EW AS PCM respects free portfolio value
* Fixes
- Update tests expected statistics results affected by MHDB custom data timezone fix
- Fix for Extensions.IsCustomDataType
* Fixed and adding more regression tests
- Adding support and regression test with alpha consumer with different account currency
- Adding support and regression test of a universe adding custom data
types
- Add support and regression test for algorithm alpha consumer with existing holdings
* Add AlphaStreamsOrderEvent data type
* More logging for BinanceRestApiClient
* Changing LogLevel to Debug for history providers : Binance, Polygon
* Minor fixes
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Adds support for ETF constituent universes
* Adds filtering for universe data if it doesn't match the
universe subscription type
* Update mapping for ALL underlying Symbols if
`Symbol.UpdateMappedSymbol(...)` is called. Required to support
constituent ETF universes that might have mapping events
* Delistings of composite constituent universe Symbol will result in
removal of universe securities.
* Added regression algorithms for ETF constituent mappings (C#/Python),
along with data required to run locally
* Refactor universe delistings in SubscriptionSynchronizer -
big thank you to @Martin-Molinero :)
* Address review: update regression algorithms and add explanatory comments
* Address review: add additional checks to delisting regression algorithms
* Adds new regression algorithm testing the addition of a universe
without calling AddEquity() and asserts same behavior
* Fix SubscriptionDataReader auxiliary data out of order
- Fix for SubscriptionDataReader emitting auxiliary data out of order
due to sparse that and enumerator refresh logic. Adding regression
algorithm
- Minor tweaks for unit tests failing on and off
* Fix for SDR internal subscriptions
- Fix for SubscriptionDataReader enumerator refresh for internal
subscriptions which were ignored. Adding regression test
- Updating custom data regression algorithms affected by the issue
- DropboxBaseDataUniverse was emitting a custom data point being end time
- UnlinkedTraderBarIconicType was emitting a single data point
of the underlying SPY minute data when if should of emitted
all data points
* websocket-refactor for websocket message as text and binary
Initial commit
websocket support for text and binary messageData
code style fixes
test fix
* delete un-used file
* zerodha websocket textMessage parse
* messageData class abscration
* Empty
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Fix IB history requests
- Fix time zone bug causing incomplete results
- Use RegularTradingHours flag only for equities
- Second resolution data is now requested in 30 minute chunks (instead of 1 minute):
https://interactivebrokers.github.io/tws-api/historical_limitations.html
* Filter history result respecting market hours and IncludeExtendedMarketHours flag
* Add history unit tests
* Improve python exception parsing
- Improve python exception parsing adding support for line shift. Adding
unit tests
* PythonException revert change
* Centralized and normalize algorithm runtime handling
* Adding support for C# line and file exception report
* Fixes Market On Open Fill of Equity Fill Model
Only use trade data (Tick with Trade type or TradeBar) to get the open price, since MOO is filled with the opening action price. Ensure that this method doesn't use trade data from before the market opens for high-resolution data case.
Fix unit tests to show that the new implementation only fills with trade data from the current open market.
Change regression tests to reflect the bug fix.
In the `ExtendedMarketHoursHistoryRegressionAlgorithm`, MOO was filled with extended market hours.
* Fix Bug for Tick Resolution Case
For tick susbcription, the tick with the open price information is the the first valid (non-zero) tick of trade type from an open market.
Addresses peer-review by moving the if-condition for data belonging to the open market where the subscriscribed types are checked.
* Fix Bug for Low Resolution Edge Case
For the edge case where the order is placed after the trade bar is open, for example, order places at 1 pm with daily-resolution data. The fill model will not use the open of the bar that will close at midnight, since this value is prior to the order.
Adds unit test.
Change regression tests to reflect the bug fix.
In the `RegressionAlgorithm`, MOO was filled with open prior to the order. The algorithm now has one order less, since the last MOO would need to wait another day to be filled.
* Implements SaleCondition and Exchange Check For Tick
- Adds additional unit tests for MOO
* Fixes Regression Test in DataConsolidatorPythonWrapperTests
* Addresses Peer-Review
- Adds new unit test cases.
* Update Lean Foundation docker files to IBGateway v985
* Add apt-get update for ARM foundation
* Update default value of IB gateway
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* WIP removal of custom data references & tests updates
* Regression algos updated and python algorithms moved to DataSource repos
* Fixes failing unit tests
* Add "LiveDataTypes" field to LiveNodePacket
* Adds Initialize() to IDataChannelProvider
* Adds new extension method to convert
HistoryRequest -> SubscriptionDataConfig
* Address review: Add protobuf definitions for Iconic data types
* Address review: re-adds DynamicSecurityDataAlgorithm as regression algo
* Small adjustments to variable naming and documentation
* Move test files to respective DataSource repos
* Add support for IBGateway v985
- Effective in TWS version 985 and later, for US stocks the bid, ask, and last size quotes are shown in shares (not in lots).
* Add missing error log in IB symbol mapper
* Trigger build
- Create Lean common engine initialization class, which will be called
by research and Lean launcher
- Fix backtest deserialization bug due to failing to handle decimal
value
* Refactor pandas mapper to support newer versions
* Clean up remapper
* Finalize deprecated tests and adjust those that should apply to new pandas
* Move mapper to its own file
* Add supporting Py tests and setup instructions
* Add license
- Calling RemoveSecurity() will remove the security from all the
universes holdings it, there was a race condition here where it would
just take the first universe and remove it from it.
Adding regression test adding and removing an option contract and it's underlying
- Take lock to refresh client to avoid null reference
exception
- Reduce gate rate
- Use extension methods to await tasks
- Avoid copying the data multiple times
* Update BinanceBrokerage to handle more than 512 symbols
* Address review
- fetch symbol weights only if required
- remove code duplication
* Add rate limiting for new connections
* Update WebSocketMessage to include the websocket instance
* Handle resubscriptions on reconnect
* Address review
* Address review
* Remove unnecessary locking
* WebSocketClientWrapper updates
- remove allocation of receive buffer on each message
- add missing lock in Close method
- log message data when message type is Close
- fix race condition after unexpected websocket close
* Set WebSocketClientWrapper task to LongRunning
* Add missing check in GetHistory
* Fix exceptions with Binance downloader
- closes#5794
* Update BinanceBrokerage to handle more than 512 symbols
* Address review
- fetch symbol weights only if required
- remove code duplication
* Add rate limiting for new connections
* Update WebSocketMessage to include the websocket instance
* Handle resubscriptions on reconnect
* Address review
* Address review
* Remove unnecessary locking
* WebSocketClientWrapper updates
- remove allocation of receive buffer on each message
- add missing lock in Close method
- log message data when message type is Close
- fix race condition after unexpected websocket close
* Set WebSocketClientWrapper task to LongRunning
* Add missing check in GetHistory
* Fix exceptions with Binance downloader
- closes#5794
* Update Bitfinex symbols in symbol properties database
* Update BitfinexBrokerage to use BrokerageMultiWebSocketSubscriptionManager
* Address review
* Remove unnecessary locking
* Remove old channels on resubscription
* logging and kite connection retry
(cherry picked from commit 9845a2d79e86c338093cadcae7f41b29d897c78c)
* changed thread.sleep time for kite connection retries
* websresponse retry on error - coding style change
* datetome parse to convert time in utc
* indentation code-style fix
* zerodhaDownloader support for dataTimeZone
* removed code redundancy for single conversion without hardcoded TZ
* removed hardcode dataTimeZone and code refactor
* returns exchange TZ to lean and data TZ to local disk from downloader
* Kite historical candle timestamp TZ conversion
* added unit test for kite historical candle
* access modifier correection
* Filter out small orders based on Setting
- BuyingPowerModel will filter out small orders based on algorithm
setting, a % of PTV, instead of hard coded 1 share value. Addin unit
and regression tests
- Updating regression algorithms to use new setting, reduce order trades
* Update regression algorithms
* zerodhaDownloader support for India Market
(cherry picked from commit c36272ef9c4c7eddfff77413d900faab3a2a8654)
* indenation styling fix
* zerodha donwloader
* Market.India support from zerodhaSymbolMapper
* code styling changes
* re adding KiteProductType
* check kiteProductType per order and get all Product Type Account Holdings
* use WithLockedStream and remove lock unlock stream
* filter holding if config productType set otherwise fetch all
* improvements in styling and properties
* unionbank has less margin requirements, second always works
- TransactionHandler will see invalid OrderEvent for an order which
tries to short an non shortable asset. Adding unit test, updating
regression algorithm.
- Fix Api data price path normalization, found while testing with
shortable provider. Adding unit tests
- Improve readme file, add nuget link, update Lean home page
- Increase 10min look back to 60 for setting currency conversion
- Replace error message for trace when minute resolution history request
is empty
* Added Augen Price Spike Files
* AugenPriceSpike Update
Adjusted indicator calculations and added test data from Trading View
* Requested Changes
* Minor changes
* Add reproducing unit test
* Apply fix to force adjustment if needed
* Adjust fee to correctly replicate original issue
* Adjust solution
* Update unit test to reflect the way this function is used
* nit - rename test
* Cleanup error message to improve reproducibility
* nit - Drop unneeded var
* Add in adjustment test cases as well, fixed
* Fix minor race condition in brokerage message handler
- Fix race condition where a message could be enqueued and
wait for a new call, not being processed ASAP
* Reduce BrokerageTransactionHandler logs
* Fix coarse universe generator on single processor
* Make sure MaxDegreeOfParallelism value is not less than 1
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Add BrokerageConcurrentMessageHandler
- Add Brokerage helper `BrokerageConcurrentMessageHandler` class to lock message stream
while executing an action, for example placing an order, adding unit tests.
- Binance will use new message handler
* Fix unit test
* Implement solution with lightest changes possible
* Update regressions v1
* Adjust AlgorithmTradingTests
* Adjust PatternDayTradingMarginBuyingPowerModel tests
* Drop need to loop twice
* Adjust last unit test, with calculations included
* nit - comment fix
* Break out adjustment calculation to static function; add unit test
* Update Py regression
* Upgrade adjustment calculation to be smart enough to get us to target always
* nit - cleanup GetAmountToOrder
* Add license to test
* nit - comment fix
* cleanup GetAmountToOrder further
* Add additional test cases
* added symbolData class to store exchange and token values of tickers
* support for India Market
(cherry picked from commit 5f629dcb9e72426dc7b560e782cb1fae72861574)
* removed local vscode workspace file
* support for india market instead of nse and other exchanges
* changed hardcoded india values to Market.India
* changed zerodha getcash fromm available to net
* addded edge case handoing with expections
* support for market info (nse, bse) for india market
* exchange info for EmitQuoteTick
* basic testing with c#
* c# algo
* c# testing
* live orders during market open success
* support orderproperties for exchange value for order placement
* added exceptional handling cases
* fix xml descriptions for the files
* fix zerodha-product-type in DefaultOrderProperties
* add basic template for india market
* fix orderProperites for default values
* fix for local data, python implementation not available
* fix access modifiers of SymbolData
* fix variable name case
* fix single check for exchange, call to base
* fix styling, redundent code and access modifers
* fix non static not callable from static
* fix static
* Adjust AlgoSeek Futures price multipliers to quote prices in USD
* Revert changes to AUP scaling
* After a second review, seems that this contract isn't scaled.
The contract multiplier differs from the NYMEX Aluminum Index's
contract multiplier, and so does the minimum price variation.
The minimum price fluctuation seems consistent with CME's
quoted prices, which also claims the contract is quoted in USD.
* Fix futures contract multipliers for some scaled contracts
- Add new CompositeDataProvider that will wrap multiple instance. Allows
using for example the ApiDataProvider and a DownloaderDataProvider.
- Adjusting ProductJsonConverter after recent Modules addition
* Add new DownloaderDataProvider
- Moving LeanDataWrite and IDataDownloader to common project
- Add new BaseDataDownloaderDataProvider with concurrency helper method
- Add new DownloaderDataProvider which will use a IDataDownloader or
IBrokerage implementation as data source
* Add support for downloading symbol chains data
The Kraken allow queries from pairs of format XBTUSD and XXBTZUSD but it will only return data with the XXBTZUSD causing errors when XBTUSD ticker is specified.
This commit adds a more meaningful error when KDL is called with XBTUSD format.
* Update IBAutomater to v2.0.39
* Downgrade IB logging level from Detail (5) to Information (4)
- The online documentation warns of performance overhead when setting logLevel=5
* Apply fix for both start and end month cases
* Add unit tests for both cases
* Fix weekstart/weekend for same issue; add unit tests
* nit - rename tests
* Adjust logic
* Update charts that are not empty and not default values
* Testing
* Filter final results
* Explicitly filter alpha charts if not needed
* Use filtered charts for Live results handler
* Refactor solution to manage alpha charts internally by creator
* nit comment fixes
* Implement drawdown as default chart
* Implement Capacity Estimate for backtest default chart
* Add default values to QC.Plotting
* Adjust drawdown calculation
* Only sample "Capacity" once a day
* Implement Volume chart
* Nit fixes
* Volume Chart adjustments and fixes
* Rename to "Assets Sales Volume"
* Implement Exposure Sampling
* Store algorithm currency symbol, round capacity to nearest 1k
* Add new plots to default charts
* nit change name
* Address reviews
* Reduce duplication and clean up exposure sampling
* Address reviews
* Improve sample exposure
* Address review
* Only enumerate holdings once
* nit - comments
* Post rebase fix
* Don't need to round anymore
* SymbolJsonConverter improvements
- SymbolJsonConverter will be able to deserialize symbols expressed in
user friendly format. Adding unit test
- Adding a couple of helper methods, removing duplicated code
* Fix FOPs expiration year parsing
* Add IResultHandler.BrokerageMessage
- Result handlers can receive notifications for brokerage message events
* Make LiveTradingResultHandler.RuntimeError virtual
- Reduce thread contention in coinApi DQH by taking a lock per symbol
versus a global lock.
- Improve tick creating by reducing object being wasted on empty
constructor
* Updates Equity Market Data
* Updates Unit Tests
* Updates Regression Tests
In this commit we include regression tests with small changes (slightly different CAGR, Alpha, etc, but same number of trades) due to the data update.
* Updates Regression Tests 2
The following regression tests were adapt because of verification of hard-coded market data values:
- `AdjustedVolumeRegressionAlgorithm`
- `HistoryWithSymbolChangesRegressionAlgorithm`
- `OptionRenameRegressionAlgorithm`
- `RawDataRegressionAlgorithm`
- `SwitchDataModeRegressionAlgorithm`
The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `AddUniverseSelectionModelCoarseAlgorithm` 23 -> 35
- `MeanVarianceOptimizationFrameworkAlgorithm` 12 -> 14
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 298 -> 324
- `PortfolioRebalanceOnInsightChangesRegressionAlgorithm` 83 -> 86
- `ScheduledUniverseSelectionModelRegressionAlgorithm` 86 -> 90
- `SectorExposureRiskFrameworkAlgorithm` 17 -> 22
- `SetHoldingsMultipleTargetsRegressionAlgorithm` 8 -> 9
- `StandardDeviationExecutionModelRegressionAlgorithm` 196 -> 199
- `UniverseUnchangedRegressionAlgorithm` 11 -> 17
- `VolumeWeightedAveragePriceExecutionModelRegressionAlgorithm` 237 -> 238
Especial cases:
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 17
- BLM model sensibility
- `OptionChainedAndUniverseSelectionRegressionAlgorithm`
The following regression tests have different Capacity because of different volume from lowest capacity asset, except:
- `OptionEquityCoveredCallRegressionAlgorithm` New lowest capacity asset is underlying
- `OptionEquityCoveredPutRegressionAlgorithm` New lowest capacity asset is underlying
* Revert File Update for SPWR and SPWRA
* Fix Regression Tests
Temporarily removes python regression test for `MeanVarianceOptimizationFrameworkAlgorithm` as the `MeanVarianceOptimizationPortfolioConstructionModel` for each version are yeilding different results. If we use C# version in `MeanVarianceOptimizationPortfolioConstructionModel.py`, the results match.
* Changes Optimization Method in MinimumVariancePortfolioOptimizer [Py]
Uses `trust-constr` method.
See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html
* Use Python algorithms directly in their location; also refactor pythonpath additions to one method
* Update VSCode Launch options
* Drop issue these changes fixes
* Use AddRange
* Python research import improvements
- Improve start.py for research env
- Remove unrequired imports
* Centralize algorithm imports
* Add regression test GH action
* Unit test python import clean up
* Join research and main imports
* More python import clean up
* Fix failing skipped regression algorithm
* Stop update when properties will fail to update
* Only enforce our case after the BB is ready
* Nit comment fixes
* Add unit test
* Use faster DateTime.UtcNow
* Updates to default notebooks
* Update readme
* Notebook adjustments
* Drop launch script support
* Use our cwd if no QC dlls are in AppDomain directory
* Modify Intialize.csx to use parent directory only if QC files are detected there
* Clean up some details after testing
* Adjustments
* nit, NullOrWhiteSpace
* Enable loading live virtual position tracking
- Enable GDAX paper trading configurations
- GDAX will optionally load existing holdings from live job package
- Bitfinex will try loading existing holdings from live job package for
cash account type
* Binance live holdings loading
* Adds VIXCentral Contango data
* Refactor cached alternative data in live mode to
use web endpoint if the data is not available locally
* Remove fallback to cache bucket for CBOE/FRED/EIA/VIXCentralContango
* Address review: force VIX as ticker for all VIXCentralContango data
* Normalize live holdings initialization
* PaperBrokerage config based live state recovery
- Increase algorithm status storing sampling period
- Fix bug where duplicate cash entries could happen
* Applies a fix
* Fixing the tests to cover downloader func. Fixing the queue handler
* this has to be removed
* -- Fixes -- [WIP]
* Implements the way to upload historical bars in async fashion. [ WIP ]
* Finishes GetEquityTradeTicks() impl.
* Do not break iteration if result is null
* Uses DisposeSafely and fixes a typo
* To use IEnumerable for return type in GetHistoryAsync
* Fixes GetEquityQuoteTicks()
* Removes GetHistoryAsync() and reverts the associated with it changes
* SubscriptionManager -> revert the changes
* PolygonDataQueueHandler -> use SIP timestamp as it keeps the order
* PolygonDataQueueHandler -> use Log.Trace for the no data {date} message
* Fixes GetForexQuoteTicks()
* Introduce ResponseSizeLimitAggregateData
* Fixes up GetCryptoTradeTicks()
* Small fixes for tick processing methods
* Removes launchSettings.json
* Refactor download data logic
- Replace WebClient for HttpClient
- Remove duplicated download logic and parsing for a single static
method
* Update PolygonDataQueueHandler.cs
Remove commented code
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Use DataProvider for fetching Shortable and FutureMargins files
* Use data provider for AllShortableSymbols
* Refactor of NeedToDownload logic
* Adjustments
* nit - cleanup
* Expand test cases
* Create the directory if it doesn't exist
* Add immediate logging for IB brokerage unit tests
* Filter brokerage message event for IB warning
- 10197 - No market data during competing live session.
* Update IBAutomater to v2.0.32
* Update IBAutomater to v2.0.33
* Use determined quantity for exercise order
* Add regression algorithm for issue
* Minor tweaks
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
- Ensure data is downloaded only once with the ApiDataProvider. Adding
unit test
- Log invalid securityType ApiDataProvider requests once
- Assert Factor file and map file providers are compatible with the
ApiDataProvider
* Use lean data key as param for request
* key -> filePath rename and some cleanup
* Refactor
* Add Organizations Endpoints
* Add some organization api wrapper objects
* Address namespace issue
* Reorganize Api Test into seperate files using one ApiTestBase
* Add Organization tests
* Use capitalized "API" test namespace to reduce amount of file changes
* Add License to test base
* Update /data endpoint functions and response objects
* Update ApiDataProvider Logic
* Handle deserialization of organization products
* Simplify converter
* Only throw for equity requests when not subscribed to map/factor files
* Add missing header
* Make arguement exception
* Api adjustments
* Add Zip factor and map file providers
- Common project will now reference Compression project and not the other way
round.
- Adding Zip FactorFile and MapFile providers
* Refactor FactorFileProvider to use DataProvider to fetch files
* Use resulting MinimumDate in construction of FactorFile
* Nit FactorFile comments and arrangement
* Refactor MapFileProviders to use DataProvider for fetching files
* Refactor ZipFileProvider
* Clean up
* Refactor Backtesting Future/Option chain providers to use dataprovider
* Fixes for data/ endpoints and test adjustments
* Response objects adjustments/cleanups
* ApiDateProvider fixes and testing
* Add LocalZipFactorFileTests
* Update ApiDataProvider download test to verify stream is not null
* Implement posting of agreement summary and signed time
* Mark all Api related tests as explicit and document details on running
* Clarify default token on ApiTestBase
* Adjust summary
* Update Api responses for QCC, except org products which are sold in USD
* Implement cache expiration for zip MapFile and FactorFiles. Adding unit tests
* Fix multiple markets for ZipFactorFile provider
* Use Symbol as cache key
* Api.cs review
* Dispose of factorFileStream after reading
* Use zip.EntryFileNames
* Address a few reviews
* Few more fixes
* Address Api Review
* Add Job Org id to config
* Minor tweaks
* Compare with invariant culture
* Fixes Option Universe selection
* ZipEntryNameSubscriptionDataSourceReader will use IDataProvider
* Fix research
* Fix null reference exception
* Make duplicate log debug
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Fixes bug where raw data was incorrectly be filtered when ID was duped
* Filter CoinAPI raw data by distinct Symbol created from the SymbolMapper
* Fixes bug where only quotes or trades would be processed, not both
* Trigger rebuild
* Attempt to remove SecurityInitializer from UniverseSelection stack entirely
* Fix broken Python Universe Selection model
* Remove mistake change (meant for another branch)
* FOPs margin as a curve
- Adjust FOPs margin requirement so that it behaves like an 'S' curve
for the contract strike, around the underlying price and it's margin requirement.
Based on IB behavior. Adding unit tests for different future contracts
- Improve a couple of unit test cleanup
* Add ES Short Call unit test
* Adds filtering of negative ticks in CoinAPI converter
* Tick data will remain the same, only aggregated data
will have changes reflected in its output as a result
* Adds suspicious tick filtering to ticks
* Address review: Add Crypto Trade Tick suspicious test case
* Implements PivotPointsHighLow indicator + tests
# Conflicts:
# Tests/QuantConnect.Tests.csproj
* Some fixes
* Refactoring + Adds NewPivotPointFormed event
* Fixing tests
* Some fixes for the Get methods to not throw when points array is empty
* To address review + more fixes
* Fixing xml comments typos
* Implements separate rolling windows to calculate highs and low
* Makes the number of last stored indicator values as an input parameter
* Overrides Reset()
* Adds a helper method
* Change numerical return to Enum types
* Changes IsReady condition :
the indicator is ready and starts calculating the pivot point when any of the rollings is ready
* Address reviews
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Update launch options and tasks
* Update readme
* Mention building manually is optional
* Add portable pdbs to use C# coreclr debugging
* Update VS and VSCode docs
* Add reference to Lean CLI in main readme
* remove run_docker scripts
* Remove PyCharm files
* Cleanup
* Remove run_docker tasks from VS Code
* Address typos
* Implemented AlphaVantageDataDownloader
* Style and documentation changes requested
* Throw exception for unexpected response content
* Added rate limiting to AlphaVantageDataDownloader
* Minor tweaks and rebase
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Apply changes from #2146 on current master
Credit for most of these changes goes to https://github.com/viliwonka.
Rebasing three years of changes didn't go so well, so I manually
applied the changes in #2146 on the current master branch.
This commit attempts to represent the changes in #2146 as closely as
possible, no cleanup has been performed yet. Despite this, some changes
were inevitable because part of #2146 has been implemented another way
in the past three years.
* Clean up
* Process review comments
* Add tests
* symbol -> potentialConversionRateSymbol
* Ensure regression tests run/get data
* Fix broken test
* Process review comments
* Process review comments
* Simplify currency conversion logic
* Performance improvement, doc updates and test rename
* Rebase fixes
* Remove obsolete properties
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Minor tweaks
- Add AccountCurrency and symbol for live result packet
- Improve performance of Tick clone and reading
* Reduce unit test from 9s to 500ms
* Do not restrict unrequested securities by 1.0m leverage, let security initializer do its job
* Use UniverseSettings for leverage, fillforward, and extendedmarkethours
* Update unrequested security message
* Use UniverseSettings ExtendedMarketHours for underlying on OptionContracts
* Add unit test to prove setting persistence
* Cleanup unit test
* Adjust crypto unit test assert
Co-authored-by: Colton Sellers <Colton.R.Sellers@gmail.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Fix FutureOptionMarginModel Margin Requirements
- Fixes FuturesOptionsMarginModel margin requirements being zero due to
underlying being null on model creation. Adding unit test
* Address reviews
- SecurityService will set underlying when creating a new security when
provided
* Fixes SEC rate limiting issue by reducing clustering of requests
* We add a user-agent header to all outgoing requests for SEC
as they require it to identify the client, otherwise we're rate
limited.
* Some adjustments to how we wait for work to be done, and RateGate
rate of requests. We wait for all requests now/don't use tasks
anymore
* Address review - remove class HttpClient in favor of local instance
* Upgrade IBAutomater to v2.0.0
* Upgrade IBAutomater to v2.0.1
* Upgrade IBAutomater to v2.0.6
* Upgrade IBAutomater to v2.0.7
* Upgrade IBAutomater to v2.0.9
* Upgrade IBAutomater to v2.0.10
* Upgrade IBAutomater to v2.0.12
* Upgrade IBAutomater to v2.0.13
* Upgrade IBAutomater to v2.0.14
* Upgrade IBAutomater to v2.0.15
* Reset disconnected flag on IBGateway exit/restart
This bug was causing invalid algorithm termination after reconnection
* Fix unhandled exceptions in exit/restart event handlers
* Upgrade IBAutomater to v2.0.16
* Upgrade IBAutomater to v2.0.17
* Update testing OrderProvider to clone orders as BrokerageTransactionHandler
* Add immediate logging for brokerage unit tests
* Tradier brokerage updates
- Fixed NullReferenceException with zero-crossing orders
- Added a few missing null checks
- Fixed issues in error handling
- Added new unit tests for some common errors
* Adds processed data directory to read price data from
* Make coarse universe generator look at data directory before failing to find daily data
* Set coarse generator output of missing daily file to debug log
* Add CoarseUniverseGenerator logs
* Fixes 100 nanosecond increment lookahead bias when parsing large numbers
* Whenever we parse a number that is has precision greater than
DateTime ticks (sub-100 nanoseconds), if we have nanoseconds
between [0, 1000), excluding numbers divisible by 100,
we will have leftover nanoseconds between [0, 100) nanoseconds, but
they won't be factored in to the DateTime calculation, since casting
to `long` only takes the integer component of the number, so we lose
the extra nanoseconds that came with the decimal, and time is set to
the "floored" value without those nanoseconds.
Since .NET `DateTime` type has a limitation of only being able
to represent time in increments of 100 nanoseconds, by not
considering the sub-100 nanoseconds, we introduce a look-ahead
bias of at most 100 nanoseconds/1 tick
* Misc adjustment to make method use `decimal` instead of `double`
for increased precision when parsing large numbers
* Changes CoinAPI data converter to support processing raw files in original directory structure and file name
* Removes Market requirement from CoinAPI data converter
* Remove timeout on decompression of raw AlgoSeek futures data
* Updates SEC downloader to use HttpClient where requests were failing
* For some unknown reason, valid requests to a valid URL were
failing when using WebClient. Changing our requester to
HttpClient fixes the issue, and enables us to leverage
async capabilities where applicable.
* Added fault tolerance to index file downloads, including a
rate limit in case we've been rate limited
* Further refactoring; catches 429 errors, adds missing rategate calls
* Replace all usage of WebClient, force retry for all failures
* Adds optional config value for Benzinga News API key in downloader
* Modifies Estimize Downloader api config name and fixes directory not found bug
* Refactor Estimize to speed up processing time
* Adds ticker limits if desired
* Misc. bug fixes, performance improvements, code cleanup
* Remove debug log statements leftover from previous commit
* Add support for non-tick Index resolutions in LeanDataWriter
* Empty commit
* Empty commit
* Empty commit
* Empty commit
* Empty commit
* Empty commit
* Lower requests/second for SEC downloader, add missing rategate call
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
- Fix FOPs maintenance margin requirements since they were relaying on
the underlyings holdings cost to return 0 in some cases. Updating unit
tests
- Fix Option exercise order being rejected because of margin not
available, will now take into account the margin used by the option
itself. Adding unit test
* Add property for capacity. Remove unused variable
* Move SymbolCapacity and CapacityEstimate to common, passed through Symbol to runtime statistics
* Add null checks
* Remove uninvested and untradable assets from capacty calculations
* Add SymbolCapacity influential period
* Updates Regression Tests
- DelistingEventsAlgorithm
- Allows additional contributions from delisted AAA.1
- DelistingFutureOptionRegressionAlgorithm
- Removes DC01H12 contributions one month later
- FutureOptionBuySellCallIntradayRegressionAlgorithm
- Allows additional contributions from future after expiry replacing the contribution of the next contract option
- DelistedFutureLiquidateRegressionAlgorithm
- FutureOptionCallITMExpiryRegressionAlgorithm
- FutureOptionCallITMGreeksExpiryRegressionAlgorithm
- FutureOptionPutITMExpiryRegressionAlgorithm
- FutureOptionShortCallITMExpiryRegressionAlgorithm
- FutureOptionShortPutITMExpiryRegressionAlgorithm
- FuturesAndFuturesOptionsExpiryTimeAndLiquidationRegressionAlgorithm
- Allows additional contributions from future after expiry
- FutureOptionCallOTMExpiryRegressionAlgorithm
- FutureOptionPutOTMExpiryRegressionAlgorithm
- FutureOptionShortPutOTMExpiryRegressionAlgorithm
- IndexOptionCallITMGreeksExpiryRegressionAlgorithm
- IndexOptionCallOTMExpiryRegressionAlgorithm
- IndexOptionShortCallOTMExpiryRegressionAlgorithm
- Allows additional contributions from option after expiry
- MACDTrendAlgorithm
- Removes contribution when SPY is not invested for over one month
- UniverseSelectionRegressionAlgorithm
- Allows additional contributions from delisted GOOAV replacing GOOG (new symbols)
* Adds Lowest Capacity Asset to Regression Tests
* Normalize expected value -0, because -0 is also written to file if updated
* Write Symbol.Value for lowestCapacitySymbol or empty string for empty Symbol
* Update Regressions
* Update 'Lowest Capacity Asset' to Symbol.ID
Co-authored-by: Jared Broad <jaredbroad@gmail.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
Co-authored-by: Colton Sellers <Colton.R.Sellers@gmail.com>
* Updates Fundamental Data
* Updates Regression Algorithms
- `CoarseFundamentalTop3Algorithm`
- New coarse has higher `DollarVolume` for `FB`
- `CoarseNoLookAheadBiasAlgorithm`
- Change from `SPY` update not included in #5493
- `SectorExposureRiskFrameworkAlgorithm`
- `HasFundamentalData` was false for `GOOG` on 20140401 and 20140402.
* Updates Unit Tests
Minor changes in expected values
* Updates Regression Tests
Should have been included in #5493:
- `OptionsExpiredContractRegression`
- `FuturesExpiredContractRegression`
* Adjusted Quantity By Lot Size in OrderTargetsByMarginImpact
`OrderTargetsByMarginImpact` didn't calculate the order value with the quantity adjusted by order size which less to values that did not reflect the actual order value.
It has a particular affect in `SectorExposureRiskFrameworkAlgorithm` where the Python and C# versions have the same orders but placed in a different sequence because of decimal/double precision.
* Reduce dictionary access x3 on OrderTargetsByMarginImpact
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Redefine IsReady and WarmUpPeriod
* Update warm up periods of other indicators to account for ROC warmup change
* Update regressions affected by this change (because of HistoricalReturnsAlphaModel using ROC)
* Add simple unit test for issue #5491
* Cleanup warmup period math and add comments
Co-authored-by: Colton Sellers <Colton.R.Sellers@gmail.com>
* Reconcile and consolidate SetupHandlers
* Centralize GetConfiguredDataFeeds()
* nit - extra space
* Check for null, also allow null to be passed back if no config value
* Fix breaking test
* Cleanup
* fix return var
* remove unneeded if/else
* Minor changes
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Use UniverseSettings.DataNormalizationMode for securities added in Algorithm
* Stop SubscriptionUtils from forcing Adjusted mode
* Return behavior to original and add comments
* nit typo
* Add regression
* Add unit test that verifies DataNormalizationMode can be altered manually by security
* Cleanup and add Py version of regression
* Update projects to use .NET 5.0, the successor to .NET Core
* Fix ambiguous errors. Add IBAutomator net5
* Remove FXCM
* Upgrade IBAutomater to v1.0.51
ignored, and an empty message aborts the commit.
* Fix rebase
- Fix ambiguous Index
- Remove StrategyCapacity.cs
- Update System.Threading.Tasks.Extensionsy
* Remove unrequired references
* Fixes
- Travis will use dotnet, not nunit nor mono
- Remove mono from foundation image
- Fix python setup in research
- Fix unit tests
* Don't call ReadKey when input is redirected
* Fix ConsoleLeanOptimizer
* Research fixes
* Update comment
* Add vsdbg to Dockerfile
* Fixes
- Revert dockerfile FROM custom changes
- Adjust and fix regression algorithms
- Option assignment will be deterministic in the order
- 'Rolling Averaged Population' is calculated using doubles, updating
expected values.
- Update readme, removing references to mono
- Add missing Py.Gil lock
* Replace ICSharp with .NET Interactive
* Fixes after rebase
* CSharp research fixes
- Adding new Initialize.csx that pre loads all assemblies
- Adjusting template research file
- Moving steps in dockerfilejupyter
- Fix unit tests and regression tests after rebase
Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Stefano Raggi <stefano.raggi67@gmail.com>
Co-authored-by: Jasper van Merle <jaspervmerle@gmail.com>
* Add unit test
* Make test cover issue case
* Filter out times before startTimeUtc
* Refactor solution, fixes missing first date
* Fix case where none is expected
* Cleanup tests, add TriggerTimesNone
* Drop unused imports
* Add missing dipose call
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Refactor HasSufficientBuyingPowerForOrder implementations
Adds Sufficient and Insufficient helper methods to HashSufficientbuyingPowerForOrderParameters
enabling syntax like:
return paraeeters.Sufficient()
returnparameters.Insufficient(reason)
The next change will add the initial margin required which will simply require
updating both of these helper methods to accept the value.
* IBuyingPowerModel: Add margin functions Maintenance/Initial/ForOrder
These were originally hidden in an effort to only expose what's necessary
for the engine to perform its work. Additionally, we encapsulated all of
the method arguments into parameters classes to prevent having to break
anyone in the future. Not including these foundational methods turns out to
be an oversight. These methods are not required by the engine, but rather by
other models. Another possible solution here is to add an additional abstraction
and include these methods on this new abstraction. BuyingPowerModel would then
explicitly implement these methods and models that depend on them would require
two code paths, one for when the buying power model implements this interface
and another for when it doesn't.
Tests were additionally updated to remove test model implementations created for
the sole purpose of exposing these private methods.
* Add ConstantBuyingPowerModel
Provides an implementation of IBuyingPowerModel that returns the same
constant value
* Update BuyingPowerModelPythonWrapper to use reflection for method names
Having a bunch of hard-coded strings is a sure fire way for someone to
overlook when changing methods. This change ensures that noone needs to
remember that this code exists :)
Cleans up the syntax around verifying a python object implements a particular
C# interface via the ValidateImplementationOf<T> method by having it return a
value since the only use cases are in constructors when setting the models.
I was initially going to update ALL python wrappers to validate the passed
in models, but such a change could break many things that are 'working' right
now. Such an effort should be saved for its own dedicated PR.
* Add Parameters/Result types for new buying power model methods
* Support computing maintenance margin for arbitrary quantities
The existing GetMaintenanceMargin function assumes that we're only interested
in the maintenance margin for the entirety of the provided security's holdings.
This makes it impossible to perform what-if analysis or to even ask how much
maintenance margin is devoted to a particular subset of the security's holdings.
This change adds the quantity to the MaintenanceMarginParameters class. Futures
and Options models also depend on holdings cost and holdings value, so they have
also been added to the parameters type. Finally, static factory methods were
added to improve discernment of intent: ForCurrentHoldings provides the existing
behavior and then ForQuantityAtCurrentPrice to support what-if scenarios where
we're looking for the change in maintenance margin if we were to execute an order
for the securiy at the current time step. Obviously a constructor is provided to
set all of the values explicitly, using any price metric the caller desires.
* Address review
- Fix BPM xml documentation
- Fix python unit tests and PythonWrapper validate method
* Add SecurityHolding.QuantityChanged event
Adding event handlers will allow us to orchestrate complex
events from distant parts of the codebase through wiring
them up. If we continue down this path, it will move us away
from the current, very 'mechanical' data flows expressed in
LEAN and towards a more modern, event processing based system.
This is but a baby step in that direction and the initial use
case is using this QuantityChanged event to trigger resolution
of the algoritm's positions groups. This is part of an effort
to improve the fidelity of options margin modeling where we'll
model an OptionStrategy as an IPositionGroup. This will allow
us to compute the margin requirements of an OptionStrategy as
a unit instead of computing margin of each security individually
in isolation.
See #4065
* PortfolioManager: Group fields and remove unused field
This codebase generally places fields as the first members, but
this class had some fields at the top, then some properties, and
then some more fields. This change brings all the fields together
at the top of the file and also removes pointless comments placed
directly above some of the fields. Additionally, an unused field
was removed.
* Remove unused _currencyConverter from Security
Looks like at some point the only code using this member variable was removed
and the necessary clean up was overlooked.
* Add Parse.Enum functions
* Support disabling regression algorithms by language via config.json
Adds 'regression-test-languages' to config.json and filters regerssion algorithms to
run based on this value. When cycling on a particular feature, it's nice to be able
to run the entire regression set while ignoring the python algorithms. Once the C#
algorithms are all passing, one can then go back and run C# and Python in a final run,
since 99% of feature work doesn't impact python specifically.
* Implement IComparable in SecurityIdentitfier
This can be used to deterministically sort securities and symbols
* Add .editorconfig to enforce common formatting for json/sh files
* Fix typo in IBuyingPowerModel.GetBuyingPower xml docs
* Add ListEquals/GetListHashCode and OrderDirection.Closes(PositionSide)
ListEquals and GetListHashCode are designed to be used together as they
complement each other according to C#'s requirements for Equals and
GetHashCode functions.
PositionSide.ToOrderDirection() extension simply converts a PositionSide
to its logical equivalent OrderDirection. Long->Buy, Short->Sell, None->Hold
OrderDirection.Closes(PositionSide) determines if a particular OrderDirection
would have the effect of reducing a position's absolute size. This function
greatly improves the readability of buying power functions that must provide
adjustments when an order/contemplated trade reduces/closes an existing position.
OrderDirection.Buy.Closes(PositionSide.Short)
OrderDirection.Sell.Closes(PositionSide.Long)
All other combinations return false
Adds ToArray/ToImmutableArray convenience functions that combine a call
to Select followed by To(Immutable)Array all in one function call.
* Add decimal.DiscretelyRoundBy extension method
Supports rounding a decimal value by an arbitrarily chosen maximum precision,
or 'quanta'
* Update FutureMarginBuyingPowerModelTests to respect the security's lot size
* Add core position group classes and abstractions
* Add initial/maintenance margin support, buying power model consistency tests
* Add SufficientBuyingPower and GetReservedBuyingPower to position group model
Includes update to BrokerageTransactionHandler to use position group BPM for
sufficient buying power checks.
* Resolve position groups on each fill
We need to update the state of our position groups on each fill so that
we can properly handle multiple orders within the same time step. We
also limit the number of positions sent into the resolver by removing
securities without any holdings.
* fixup! Add SufficientBuyingPower and GetReservedBuyingPower to position group model
* Add GetMaximumLotsFor{Target|Delta}BuyingPower
Instead of computing order quantity, these functions compute the
maximum number of position group lots, which is the position group
quantity, and is guaranteed to be a whole number, for the provided
target/delta buying power parameters.
The SecurityPositionGroupBuyingPowerModel delegates to the security's
IBuyingPowerModel by applying a scaling factor equal to the security's
lot size.
This change also updates references to IBuyingPowerModel.GetMaximum...
to use the new position group model methods.
* Convert remaining IBuyingPowerModel call sites to position groups
* Rename PositionManasger.CreateDefaultGroup -> GetOrCreateDefaultGroup
Better describes its behavior
* Add Position Groups readme.md
* Add Option Strategy BuyingPowerModel
- Adding CompositePrositionGroupResolver and
OptionStrategyPositionGroupResolver
- Adding OptionStrategyPositionGroupBuyingPowerModel handling option
strategies based on IBs margin table. Adding regression algorithms
- Few changes so that option strategies executed by multiple orders are
detected
- Adjust OptionStrategyDefinitionMatch to include equity legs in the
matching result
- Minor tweaks fixing previous rebase
- Minor fixes for existing option strategies definitions, adding new
missing strategies.
- Fixing minor bugs in option strategy matcher. Adding more unit tests
* Address self reviews
- Fixing bug in 'PositionGroupCollection'
- Few minor simplificaitons
- Adding BasicTemplateOptionEquityStrategyAlgorithm
* Address reviews
- Improve regression algorithms margin remaining and used assert logic to be exact. Taking into account spread and fees
Co-authored-by: Michael Handschuh <mhandschuh@gmail.com>
* Refactor to queue up start date changes to log on dispose
* Fix typo in warning collection used
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Collect and log symbols that had start date adjusted because of factor files
* Only post the message if the set has values
* Modify message to be [Symbol, Date] combo for more information on log
* Set a hard limit on the warning set to keep it from growing unnecessarily large
* Enforce a hard limit and improve message if full
* Enforce limit in log message because of possible threading adding past max
* Cleanup
* Log security mode change once per universe addition process
* Limit max size of the warning queue to 10
* Only allow to emit once per backtest
* Set limit as var, if at limit suggest more
* Always suggest more warnings may exist because we opted to log only once
* Improves greeks configurability and defaults for all option asset types
* Makes `StandardDeviationOfReturns` configurable by users, so that
greeks can be loaded according to user expectations and the series
of returns that they'd like to compute for `n` periods and timespan
of `T`, as well as resolution of the data in live mode.
* Changes resolution to max resolution available for the default
volatility model created for the security. Usually this only applies
to live mode, but if creating an instance of the
`StandardDeviationOfReturns` volatility model and no `updateFrequency`
is provided, the resolution's time span will be used as the default
value. Backwards compatibility for equities is maintained.
* Changes defaults for `StandardDeviationOfReturnsVolatilityModel`
to warmup greeks faster for other derivative asset types
* Improves comments on `StandardDeviationOfReturns` for clarity on how
to use the volatility model for end users
* Fixes bug where TradeBar could not have proper Symbol set when getting
max resolution
* Applies to QCAlgorithm.Universe and StandardDeviationOfReturnsVolatilityModel
* Adds tests to check volatility model is updated at specified config intervals
* Address review: add shared method for (Relative)StandardDeviation
volatility models
* Adjusts logic to determine bar type
* Address review: order by TickType when getting configs inside volatility models
* Allow USDC in cashbook without USDC-USD pair
* Cover more stablecoin cases unique to our crypto brokers
* Add unit test
* Cleanup and expand test cases
* Add USDCEUR and USDCGBP to GDAX Symbols
* Add missing tickers to SPDB
* Cleanup test
- Testing net5 uncovered these algorithms to be undeterministic
- Adjusting AllShortableSymbolsCoarseSelectionRegressionAlgorithm
internal implementation
- Order removal of universe members will be deterministic, when the
entire universe is removed.
* Fixes Double to Decimal Cast in GetAnnualPerformance
`GetAnnualPerformance` raises an exception if the `AnnualPerformance` calculation returns a double that cannot be cast to decimal (smaller than `decimal.MinValue` or bigger than `decimal.MaxValue`).
See `ProbabilisticSharpeRatio` where the same solution was applied.
* Updates SPY Market Data
SPY is a key asset since it is the default benchmark, and any change can lead to different `Alpha` and `Beta`
* Updates Unit Tests to Reflect Data Update
* Updates Regression Tests to Reflect Data Update I
Most of the regression tests change because of updated data (market and factors) of SPY (default benchmark) while the total trade remain the same.
* Updates Regression Tests to Reflect Data Update II
The following regression tests were changed to adapt to adjusted prices and keep the total trades:
- `BacktestingBrokerageRegressionAlgorithm`
- `LimitIfTouchedRegressionAlgorithm`
- `PortfolioRebalanceOnCustomFuncRegressionAlgorithm`
- `SetAccountCurrencySecurityMarginModelRegressionAlgorithm`
- `StopLossOnOrderEventRegressionAlgorithm`
- `TimeInForceAlgorithm`
The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `FreePortfolioValueRegressionAlgorithm` 2 -> 3
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 291 -> 298
- `TrailingStopRiskFrameworkAlgorithm` 5 -> 7
Especial cases:
- `AutoRegressiveIntegratedMovingAverageRegressionAlgorithm` 65 -> 52
- ARIMA model sensibility
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 19
- BLM model sensibility
- `ExtendedMarketHoursHistoryRegressionAlgorithm` 20 -> 18
- Less minute bars before market opens
* Addresses Peer-Review
Fix `BacktestingBrokerageRegressionAlgorithm` to use `CalculateOrderQuantity` and round down `quantity` to an even number to pass a value assertion and update the expected value from 50 to 52.
The quantity calculated by `CalculateOrderQuantity` has changed from 50 to 53 because of factor file update.
* Add futures regression reproducing the issue
* Cleanup futures regression
* Add Options regression
* Add IRegressionAlgorithmDefinition
* Add IRegressionAlgorithmDefinition
* Adjust regressions to compare to expiration date; delisting time is not always correct
* Refactor subscription enumerator filtering of AuxData to top of the stack
* Update broken tests to reflect the split/dividends/delisting subscriptions
* Update regression statistics because of Delisting EOD instead of at time
* Let custom data configurations bypass filter
* Adjust expected count, now that we are letting aux data through to history requests
* another small adjustment
* Use ExpectedBarCount in error message
* Add filter test for both cases
* Add some clarifying comments
* Verify we did recieve data in the regressions
* Make _shouldEmitAuxiliaryData private readonly
* Filter out aux data for history requests
* Remove option to not include aux data in subscriptions
* Refactor filtering to be more explicit for each piece of data; fixes universe selection aux data
* Cleanup comments after removed var
* Refactor order of filtering for performance reasons.
`GetAnnualPerformance` raises an exception if the `AnnualPerformance` calculation returns a double that cannot be cast to decimal (smaller than `decimal.MinValue` or bigger than `decimal.MaxValue`).
See `ProbabilisticSharpeRatio` where the same solution was applied.
* Add SI
* Add ASI
* Add Swing Index
* Add Accumulative Swing Index
* Fix XML comment
* Add SI and ASI
* Add test data
* Add SI tests
* Add ASI tests
* Convert get only properties to methods
* Fix indicator name
* Replace special characters
* Fix indicator formula
* Replace test data
* Replace test data
* Update QCAlgorithm.Indicators.cs
* Minor format tweaks
Co-authored-by: Jared <jaredbroad@gmail.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
- Fixed GDAXBrokerage missing fills caused by an incorrect assumption of a global counter for order fills (it is actually per-symbol)
- Filtered out rate limit messages in the fill monitor task
* Remove and replace OnEndOfDay() ref
* Restore functionality of obsolete EOD, waiting for deprecation in August 2021
* Cleanup
* Adjustments to only post message when using obsolete EOD
* nit, extra space
* Address review
* Adjust test to reflect new behaviour
* Move GetPythonArgCount to an extension method
* Add unit test
* nit accidental import
* Refactor broken test
* Use Py.GIL() state for extension
* Set _lastEmit before emitting, otherwise _workingBar is always null
* Aggregate bars if the data endTime is past lastEmit
* Add unit test
* Address Review
* Clean up unit tests
* Refactor solution to set consistent _lastEmit behaviour
* Add another unit test
* Make fixture non-parallelizable
* Undo last change, and adjust breaking test directly
* Reconcile duplicated code
* Add License header
* CS0219 Fixes: Value assigned, but never used
* CA1507: Use nameof in place of string literals
* CS0108 : Hides Inherited Member; Use new keyword to overwrite formally
* CS0114: Hides inherited member; use override keyword
* CS0168: Variable is declared but never used
* Tests CS1062; using obsolete implicit Symbol -> String; fix via .ToString()
* CS0472: Non Nullable Obj getting Null Checked
* CS0067 Member not used; ignore all cases for future use
* CS00162 : Unreachable code; either removed or ignored for debugging and test cases
* CS0169 Remove non-used fields; ignore those that may be used in future
* CS0414; Field is assigned but never used.
* CS0618; Obsolete properties and members; Only fixes simple ones, rest will have to broken up
* CS0649; Field never assigned too
* CS0659 & CS0661 ; Overwrite operators and equals but not hashcode; I don't really override it but just call base
* Small comment fix
* Cleanup pragma statement
* Tradier brokerage updates
- Add missing status check after REST API calls
- Initialize DataQueueHandler on-demand (on first subscribe call)
* Trigger build
`LiveOptionChainProvider.FindOptionContracts` handles the following data format:
`SPY 2021 03 26 190 000 C P 0 612 360000000`
where both existing `OptionRight.Call` and `OptionRight.Put` are declared in the same line.
* Remove regression references to non-existant Python versions
* Adjust regressions estimated capacity not adjusted by #5389
* Adjusts regression algorithms so that they pass (Index/Index Options)
* Changes start/end date on BasicTemplateIndexAlgorithm
* Changes option pricing model to BlackScholes in
IndexOptionCallITMGreeksExpiryRegressionAlgorithm
- The root cause of why there are no greeks at times for these
options was identified. It is most likely due to the underlying's
VolatilityModel not having had enough data to be "warmed up",
which means it will return a standard deviation of zero to the
option pricing model, rendering most metrics as NaN.
* Adds missing index/index options regression algorithms
- Regression algorithms are now 1-1 between C# and Python for
Indexes/Index options. All regression tests are now passing
* Fixes broken BasicTemplateIndex regression algorithm
* Previously traded SPY, but because we have no SPY data in Lean
master, I instead opted for index options, since data for those
dates is already included
* Deal with weekend for breaking test case
* Adjust DefaultEndDate test to always pass
* Check todays date for open
Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
* Add support for Index SecurityType 🚀 (#5364)
* Add Index SecurityType 🚀
* Extend SecurityIdentifier & Lean Data classes with Index support
* Add Index SecurityType 🚀
* Extend SecurityIdentifier & Lean Data classes with Index support
* Fixes
* Added index cross basic template demonstration
* WIP: Prototype index security type for LEAN as non tradable asset
* Re-adds Index entries to MHDB after rebase
* First steps to getting Index Options running
* Looks at any instance where we pattern match for an option type
and replaces it with a generic call to `.IsOption()` for easier
extensibility in the future for additional option security types
* Adds IndexOption security and misc. classes
* Misc. changes, mainly related to any sort of special casing of
equity options and made index options take the same path
* Enables index options data for backtesting
* Adds new index options market hours to MHDB
* Misc. bug fixes for index options
* WIP: add live support for index options and indexes
* Use OptionMarginModel for Index Options because they both use the
same calculation for margin requirements
* Fixes contract not found errors on SPX index options and SPX index in IB
* Turns out index options' last trading day is the day before expiry,
which IB was expecting the last trading day.
* Add index option test cases (temp)
* LiveOptionChainProvider fix, use Symbol vs. ticker
* Description updates to regression algorithms
* Fixes bug in live trading for indexes and index options
* Adds overridable minimum price variation symbol property
* Adds variable sized minimum price variation for index options
* Adjusts symbol properties for index options
* Misc. bug fixes
* Fixes option assignment simulation for European options
* Updates index options regression algorithms (WIP)
* Fixes bug where index option exercise would trade index underlying
* Fixes bugs where SecurityType.Index was getting flagged as tradable
* Regression algorithms updates and addresses review
* Misc. style fixes and refactoring + a few bug fixes
* Updates regression algorithms to run without runtime errors
* Adds data for regression algos
* Sets DefaultOptionStyle on Canonical and support index options
* Update regression algos statistics
* Removes bad line in regression algorithm causing build to fail
* Minor tweaks
* Address review add comment about quoteBar parse scale
Co-authored-by: Balamurali Pandranki <balamurali@live.com>
Co-authored-by: Jared Broad <jaredbroad@gmail.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* RollingSharpeRatio Submission
Creation of RollingSharpeRatio indicator, utilizing LEAN engine. Tests were created and successfully passed by the indicator, using spy_rsr.txt as outside data file which was created during a prior running of the program.
* Fixed Test Case Loading and Added Indicator Function
Added the RSR() function to auto load the indicator in Algorithm\QCAlgorithm.Indicators.cs. Also added a reference for the test file ("spy_rsr.txt") in Tests\QuantConnect.Tests.csproj.
* Fixed SharpeRatio Indicator
Updated the base formula, indicator structure/call, and the testing data for the SharpeRatio indicator.
* Minor Fixes
- Removed dividend yield from calculation
- Fixed typos in code/documentation
* Minor Style Fix and Error Confirmation
- Changed style to match style guide better.
- Checking for confirmation that QuantConnect.Tests.Engine.DataFeeds.LinveTradingDataFeedTests.DelistedEventEmmited_Equity() is server side, as it runs locally and should not have been affected by prior changes
* Simplification and fixing of SR calculation
- Updates calculation to proper SR formula
- Utilizes IndicatorExtensions for SR calculation rather than manual calculation
- Defines counter for 1 extra period offset on warmup (otherwise first datapoint is inaccurate if warmup period is SharpePeriod length)
* Minor Fixes
- Removal of SharpePeriod and RiskFreeRate variables
- Substitution of (SharpePeriod + 1) for WarmUpPeriod
- Outdated comment fixes
* Resubmission for Mono Confirmation Error
Error detected on Git for this version of the program. An error should not occur as only comments and minor variables were edited. Resubmitting to see if it is a false positive mono error.
* Notation Fixes
Fixed notation regarding spacing and outdated comments.
* Removal of Counter Logic
- Replaces counter for warmup with improved IsReady logic
* Fixed Syntax
- Removed unused imports
- Fixed variable readability
- Fixed comments
- Replaced Tabs with spacing
Co-authored-by: quantify-cflynn <quantify-cflynn>
* Update to new QC PythonNet
* Update readme
* Remove Python.Runtime config, replaced by env var
* Allow local packages testing in repo
* Address Review
* Add the LocalPackages readme
* Update Jupyter Image
* Update Research ReadMe
* Adds backwards compatibility for MHDB and SPDB for new SecurityTypes
* Skip invalid SecurityTypes in MHDB and SPDB
* One time log invalid SecurityTypes in MHDB/SPDB
* Moves logging of SecurityType into new extension method
* Adds TryParseSecurityType to Extensions
* Adds CapacityEstimate and SymbolCapacity
The capacity estimation has been moved from
the report generator and wired directly into
Lean via the ResultHandler. In addition,
the capacity estimation strategy has changed
to account for errors in the previous iteration
of the capacity estimation.
Many many thanks to Jared for being much of the
mastermind behind this project. It would have
been harder to complete without him to bounce ideas
off of.
* Moves old tests to regression algorithms
* Adds Estimated Capacity statistic
* Removes old capacity estimation tests
Final report capacity estimation. Pushing to save state
* Fixes bugs, cleans up code and adds comments
* Adds forced sampling to Capacity Estimation
* Misc. bug fixes for daily data
* Updates capacity test cases' Estimated Strategy Capacity statistic
* Adds Capacity Estimate to all regression algorithms
* Removes Report's StrategyCapacity class and fixes bug in tests
* Adds null check in BacktestingResultHandler to fix
BacktestingTransactionHandler failing tests
* Deletes old capacity estimation classes
* Retrieve capacity estimates from backtest statistics results
instead of calculating at runtime
* Make $0.00 capacity return as "-" and Result = 0 in report
* Adds capacity to runtime statistics
* Converts capacity to number denoted by financial figures in RuntimeStats
* Addresses review: code cleanup for Capacity and adds comments to regression tests
* CoinApi HistoryProvider impl.
* Small fix of ToTradeBar()
* Revert the change : CreateSliceEnumerableFromSubscriptions
* More tests and fixes.
* Removes redundant ToList()
* too much spacing
* Fixes to get data for an unlimited period of time, by means of pagination
* Add options support
- Added IDataQueueUniverseProvider implementation
- Fix subscribe ticker for options
* Tradier DataQueueHandler web sockets implementation
- Equities and Options (trades and quotes)
* Add error handling
* Add support for options
* Fix log messages
* Address review
- Updated Tick constructor usage
- Use $empty$ symbol for last unsubscribe message
- Added symbol mapper unit tests
* Enforce USD account currency if allocation is limited
- Enforce USD account currency if allocation is limited. Adding unit
test
* Adjust currency log message
* Reduce live coarse selection interval check to 10 min
- Reduce live coarse selection interval check to 10 min. Adding unit
tests, reducing code duplication in tests.
* Adjust unit test after self review
* Removes SevenZipSharp library from ToolBox
- Library removal required for update to .NET 5.0 since it's not
compatible with it and Linux is unsupported.
* Adds new extract 7z functionality to Compression project
* Refactors AlgoSeekFuturesConverter 7z extract
* Refactors DukascopyDataDownloader 7z extract
* Addresses review: code cleanup + exception type change on timeout
* Removes not needed stream of data in Dukascopy downloader
* Makes output directory non-optional
* Fixes potential mapping issue, caused by not mapping Symbols before
reading data off disk
* Fixes issue where some results would evaluate to zero capacity
* Loads of refactoring, mostly resulting in cleaner code
- After https://github.com/QuantConnect/Lean/pull/5207 history requests
are selecting the correct data type and cause the regression test to
fail because there is no QuoteBars for daily equity. Updating and
fixing this.
- Centralizing the logic around getting the SubscriptionDataConfig to
use
* Filter values that are before subscription start time; also adjust starttime for OpenInterest
* Use data EndTime for comparison
* Allow Auxiliary data through
* Fix OpenInterest DataReader Logic
* Add regression
* Address review
* Ignore open interest for time slice
- TimeSliceFactory will directly ignore open interest for determining if
the slice has data or not. Open interest will still be available
through the Tick collection. Reverting some of the previous commits
changes since they are no longer required.
- HistoryRequests and SubscriptionRequest will use AlwaysOpen exchange
for open interest requests. Adding unit test reproducing issue
- Adding `BaseDataRequest` to avoid duplication logic.
* Make OpenInterest an internal feed and ignored by default in history
- Adding unit tests
* Revert SubscriptionFilterEnumerator Start time addition
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Adds Static AlgorithmCreationTimeout Property to BaseSetupHandler
This value will be used to set maximum time that the creation of an algorithm can take.
* Adds Additional Logging to AlgorithmPythonWrapper
It will informing the user how long it takes to import the module.
* Changes How AlgorithmCreationTimeout is Initialized
It will prevent callling Config.GetDouble on every call.
* Add support for live trading delisting events
- Adding `LiveDelistingEventProviderEnumerator` that will emit delisting
events. Adding unit tests
- Remove unused `LiveAuxiliaryDataEnumerator`.
- Refactor the corporate event providers being used per security type
* Self review: compare date component
* Add new LiveDataBaseDelistingEventProvider for equities
- Add support for delisting events for equities. Adding unit tests
* Fix cleanup of delisting equity subscription
* Refactor IPrimaryExchangeProvider.GetPrimaryExchange
now it returns a PrimaryExchange instead of a string
* Update MapFilePrimaryExchangeProvider.cs
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* To maintain consistency between calculations, we will use
the end of day equity value to calculate the returns per day.
This fixes a bug where daily equity series would zero out and result
in an exception being thrown since no elements were being passed to
the Sharpe calculation.
* Implements wrapper to get the exchange as a single letter representation
Also replace lower 'l' with capital 'L' in condition flags enums. it is a new warning on the compiler and it looks cool :)
* There are cases where the exchange is null or empty, in those cases return null
* Add tests for single character exchange representation
* Update GlobalTest.cs
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Calculate PSR for live algorithms in report generator
* Use live series for calculating of CAGR, Max DD, and PSR
* Removes Kelly Estimate from report and template.html
* Update template.html
* Rolling Beta bug fix when using daily data
* Max drawdown output as percent
* Make days live report element use equity curve last day instead of today
Co-authored-by: Jared <jaredbroad@gmail.com>
* Adds LimitIfTouched order.
TODO:
-- Add tests.
-- Add into existing regression algorithms.
-- Refactors (?)
* Fixes
- Remove unrequired space changes
- Fix EquityFillModel min/max limit price fill
- Add TriggerPrice for UpdateOrderRequest.
Quote info used w.r.t. comparing against Limit prices for LIT
FillModel.cs implementation is fixed to use quotes when comparing against set limit prices.
Also changes test implementations to assert which of quotes/trade-bars are being used
Reviewer-suggested fixes
------------
Merge remote-tracking branch 'origin/limiftouched' into limiftouched
Styling
Adds missing null check for quotebar
Styling
Adds missing null check for quotebar
Merge remote-tracking branch 'origin/limiftouched' into limiftouched
High/Low w.r.t. trigger price for determining if TriggerTouched changed to Current price
0462ad668a (r569832380)
fill fixes:
FillModel.cs: Fills exactly at the limit
EquityFillModel.cs: https://www1.interactivebrokers.com/en/index.php?f=608
Equity fill now uses quotebars
Revert - use new constructor which emulates former SubmitOrderRequest
Style changes
Reverts order model to original by means if new constructor
High/Low w.r.t. trigger price for determining if TriggerTouched changed to Current price
0462ad668a (r569832380)
fill fixes:
FillModel.cs: Fills exactly at the limit
EquityFillModel.cs: https://www1.interactivebrokers.com/en/index.php?f=608
Equity fill now uses quotebars
Revert - use new constructor which emulates former SubmitOrderRequest
Style changes
Reverts order model to original by means if new constructor
Merge commit 'bf4c96d2a055ea808fa4293662528c11a89b72c7'
* Suggested style fixes
* Review fixes
-------------------
- Suggested style fixes
- Changes LIT regression to better incorporate order modifications
- TODO: orderlisthash must be fixed
Fixed LIT regression algo
-----------------------
- Includes asserts in OnOrderEvent
* Fix OrderListHash
OrderListHash -> -292689487
* Re-adds quote nullchecks
* EquityFillModelTests fixes asserts
* Reordering FillModel.cs
* Fixes quote logic, adds methods in FillModel.cs
* Refactoring + LIT regression fixes
-- revert unneeded changes
* Fixes list hash
* Rebase -- catch up upstream
* OrderListHash fix
* Various fixes by reviewer
* Final requested changes
* tagged time -> utcinvariant
* Fixes listorderhash
* Time changed to UtcTime.ToString(DateFormat.US, CultureInfo.InvariantCulture)
* Adds Python LimitIfTouchedRegressionAlgorithm
* adds LimitIfTouchedRegressionAlgorithm.py
* adds LimitIfTouchedRegressionAlgorithm.py
* Minor changes to LIT regression algorithms
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* fixing up Coin API streaming
* Fixing file name
* Adding log showing CoinApi product plan
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
<!--- Required. Describe the behavior you expect to see for your case. -->
#### Actual Behavior
<!--- Required. Describe the actual behavior for your case. -->
#### Potential Solution
<!--- Optional. Describe any potential solutions and/or thoughts as to what may be causing the difference between expected and actual behavior. -->
#### Reproducing the Problem
<!--- Required for Bugs. Describe how to reproduce the problem. This can be via a failing unit test or a simplified algorithm that reliably demonstrates this issue. -->
#### System Information
<!--- Required for Bugs. Include any system specific information, such as OS. -->
#### Checklist
<!--- Confirm that you've provided all the required information. -->
<!--- Required fields --->
- [ ] I have completely filled out this template
- [ ] I have confirmed that this issue exists on the current `master` branch
- [ ] I have confirmed that this is not a duplicate issue by searching [issues](https://github.com/QuantConnect/Lean/issues)
<!--- Required for Bugs, feature request can delete the line below. -->
- [ ] I have provided detailed steps to reproduce the issue
<!--- Template inspired by https://github.com/stevemao/github-issue-templates -->
<!--- This template provides sections for bugs and features. Please delete any irrelevant sections before submitting -->
#### Expected Behavior
<!--- Required. Describe the behavior you expect to see for your case. -->
#### Actual Behavior
<!--- Required. Describe the actual behavior for your case. -->
#### Potential Solution
<!--- Optional. Describe any potential solutions and/or thoughts as to what may be causing the difference between expected and actual behavior. -->
#### Reproducing the Problem
<!--- Required for Bugs. Describe how to reproduce the problem. This can be via a failing unit test or a simplified algorithm that reliably demonstrates this issue. -->
#### System Information
<!--- Required for Bugs. Include any system specific information, such as OS. -->
#### Checklist
<!--- Confirm that you've provided all the required information. -->
<!--- Required fields --->
- [ ] I have completely filled out this template
- [ ] I have confirmed that this issue exists on the current `master` branch
- [ ] I have confirmed that this is not a duplicate issue by searching [issues](https://github.com/QuantConnect/Lean/issues)
<!--- Required for Bugs, feature request can delete the line below. -->
- [ ] I have provided detailed steps to reproduce the issue
<!--- Template inspired by https://github.com/stevemao/github-issue-templates -->
<h1>Develop Algorithms Locally, Run in Container</h1>
We have set up a relatively easy way to develop algorithms in your local IDE and push them into the container to be run and debugged.
Before we can use this method with Windows or Mac OS we need to share the Lean directory with Docker.
<br />
<h2>Activate File Sharing for Docker:</h2>
* Windows:
* [Guide to sharing](https://docs.docker.com/docker-for-windows/#file-sharing)
* Share the LEAN root directory with docker
* Mac:
* [Guide to sharing](https://docs.docker.com/docker-for-mac/#file-sharing)
* Share the LEAN root directory with docker
* Linux:
* (No setup required)
<br />
<h2>Lean Configuration</h2>
Next we need to be sure that our Lean configuration at **.\Launcher\config.json** is properly set. Just like running lean locally the config must reflect what we want Lean to run.
You configuration file should look something like this:
Our specific configuration binds the Algorithm.Python directory to the container by default so any algorithm you would like to run should be in that directory. Please ensure your algorithm location looks just the same as the example above. If you want to use a different location refer to the section bellow on setting that argument for the container and make sure your config.json also reflects this.
<br />
<h2>Running Lean in the Container</h2>
This section will cover how to actually launch Lean in the container with your desired configuration.
From a terminal; Pycharm has a built in terminal on the bottom taskbar labeled **Terminal**; launch the run_docker.bat/.sh script; there are a few choices on how to launch this:
1. Launch with no parameters and answer the questions regarding configuration (Press enter for defaults)
* Would you like to debug C#? (Requires mono debugger attachment) [default: N]:
2. Using the **run_docker.cfg** to store args for repeated use; any blank entries will resort to default values! example: **_./run_docker.bat run_docker.cfg_**
IMAGE=quantconnect/lean:latest
CONFIG_FILE=
DATA_DIR=
RESULTS_DIR=
DEBUGGING=
PYTHON_DIR=
3. Inline arguments; anything you don't enter will use the default args! example: **_./run_docker.bat DEBUGGING=y_**
* Accepted args for inline include all listed in the file in #2; must follow the **key=value** format
<br />
<h1>Debugging Python</h1>
Debugging your Python algorithms requires an extra step within your configuration and inside of PyCharm. Thankfully we were able to configure the PyCharm launch configurations to take care of most of the work for you!
<br />
<h2>Modifying the Configuration</h2>
First in order to debug a Python algorithm in Pycharm we must make the following change to our configuration (Launcher\config.json) under the comment debugging configuration:
"debugging": true,
"debugging-method": "PyCharm",
In setting this we are telling Lean to reach out and create a debugger connection using PyCharm’s PyDevd debugger server. Once this is set Lean will **always** attempt to connect to a debugger server on launch. **If you are no longer debugging set “debugging” to false.**
<br />
<h2>Using PyCharm Launch Options</h2>
Now that Lean is configured for the debugger we can make use of the programmed launch options to connect.
**<h3>Container (Recommended)</h3>**
To debug inside of the container we must first start the debugger server in Pycharm, to do this use the drop down configuration “Debug in Container” and launch the debugger. Be sure to set some breakpoints in your algorithms!
Then we will need to launch the container, follow the steps described in the section “[Running Lean in the Container](#Running-Lean-in-the-Container)”. After launching the container the debugging configuration will take effect and it will connect to the debug server where you can begin debugging your algorithm.
**<h3>Local</h3>**
To debug locally we must run the program locally. First, just as the container setup, start the PyCharm debugger server by running the “Debug Local” configuration.
Then start the program locally by whatever means you typically use, such as Mono, directly running the program at **QuantConnect.Lean.Launcher.exe**, etc. Once the program is running it will make the connection to your PyCharm debugger server where you can begin debugging your algorithm.
<h1>Local Development & Docker Integration with Visual Studio</h1>
<h1>Local Development with Visual Studio</h1>
This document contains information regarding ways to use Visual Studio to work with the Lean engine, this includes a couple options that make lean easy to develop on any machine:
This document contains information regarding ways to use Visual Studio to work with the Lean's Docker image.
- Using Lean CLI -> A great tool for working with your algorithms locally, while still being able to deploy to the cloud and have access to Lean data. It is also able to run algorithms locally through our official docker images **Recommended for algorithm development.
- Locally installing all dependencies to run Lean with Visual Studio on your OS.
<br />
<h1>Getting Setup</h1>
<h1>Setup</h1>
<h2>Option 1: Lean CLI</h2>
Before anything we need to ensure a few things have been done:
1. Get [Visual Studio](https://code.visualstudio.com/download)
* Get the Extension [VSMonoDebugger](https://marketplace.visualstudio.com/items?itemName=GordianDotNet.VSMonoDebugger0d62) for C# Debugging
2. Get [Docker](https://docs.docker.com/get-docker/):
* Follow the instructions for your Operating System
* New to Docker? Try docker getting-started
3. Pull Lean’s latest image from a terminal
*_docker pull quantconnect/lean_
4. Get Lean into Visual Studio
* Download the repo or clone it using: _git clone[ https://github.com/QuantConnect/Lean](https://github.com/QuantConnect/Lean)_
* Open the solution **QuantConnect.Lean.sln** using Visual Studio
To use Lean CLI follow the instructions for installation and tutorial for usage in our [documentation](https://www.quantconnect.com/docs/v2/lean-cli/key-concepts/getting-started).
<br />
<h1>Develop Algorithms Locally, Run in Container</h1>
<h2>Option 2: Install Locally</h2>
1. Install [.Net 9](https://dotnet.microsoft.com/en-us/download/dotnet/9.0) for the project
We have set up a relatively easy way to develop algorithms in your local IDE and push them into the container to be run and debugged.
2. (Optional) Get [Python 3.11.11](https://www.python.org/downloads/release/python-31111/) for running Python algorithms
- Follow Python instructions [here](https://github.com/QuantConnect/Lean/tree/master/Algorithm.Python#installing-python-311) for your platform
Before we can use this method with Windows or Mac OS we need to share the Lean directory with Docker.
3. Get [Visual Studio](https://visualstudio.microsoft.com/vs/)
4. Get Lean into VS
- Download the repo or clone it using: _git clone [https://github.com/QuantConnect/Lean](https://github.com/QuantConnect/Lean)_
- Open the project file with VS (QuantConnect.Lean.sln)
Your environment is prepared and ready to run lean
<br />
<h2>Activate File Sharing for Docker:</h2>
<h1>How to use Lean</h1>
* Windows:
* [Guide to sharing](https://docs.docker.com/docker-for-windows/#file-sharing)
* Share the LEAN root directory with docker
* Mac:
* [Guide to sharing](https://docs.docker.com/docker-for-mac/#file-sharing)
* Share the LEAN root directory with docker
* Linux:
* (No setup required)
This section will cover configuring, launching and debugging lean. This is only applicable to option 2 from above. This does not apply to Lean CLI, please refer to [CLI documentation](https://www.quantconnect.com/docs/v2/lean-cli/key-concepts/getting-started)
<br />
<h2>Lean Configuration</h2>
<h2>Configuration</h2>
Next we need to be sure that our Lean configuration at **.\Launcher\config.json** is properly set. Just like running lean locally the config must reflect what we want Lean to run.
We need to be sure that our Lean configuration at **.\Launcher\config.json** is properly set.
You configuration file should look something like this for the following languages:
Your configuration file should look something like this for the following languages:
<h3>Python:</h3>
@@ -78,59 +63,11 @@ You configuration file should look something like this for the following languag
<br />
<h2>Important Note About C#</h2>
<h2>Launching Lean</h2>
In order to use a custom C# algorithm, the C# file must be compiled before running in the docker, as it is compiled into the file **"QuantConnect.Algorithm.CSharp.dll"**. Any new C# files will need to be added to the csproj compile list before it will compile, check **Algorithm.CSharp/QuantConnect.Algorithm.CSharp.csproj** for all algorithms that are compiled. Once there is an entry for your algorithm the project can be compiled by using **Build > Build Solution**.
If you would like to debug this file in the docker container one small change to the solutions target build is required.
1. Right click on the solution **QuantConnect.Lean** in the _Solution Explorer_
2. Select **Properties**
3. For project entry **QuantConnect.Algorithm.CSharp** change the configuration to **DebugDocker**
4. Select **Apply** and close out of the window.
5. Build the project at least once before running the docker.
Now that lean is configured we can launch. Use Visual Studio's run option, Make sure QuantConnect.Lean.Launcher is selected as the launch project. Any breakpoints in Lean C# will be triggered.
<br />
<h2>Running Lean in the Container</h2>
This section will cover how to actually launch Lean in the container with your desired configuration.
From a terminal launch the run_docker.bat/.sh script; there are a few choices on how to launch this:
1. Launch with no parameters and answer the questions regarding configuration (Press enter for defaults)
* Would you like to debug C#? (Requires mono debugger attachment) [default: N]:
2. Using the **run_docker.cfg** to store args for repeated use; any blank entries will resort to default values! example: **_./run_docker.bat run_docker.cfg_**
IMAGE=quantconnect/lean:latest
CONFIG_FILE=
DATA_DIR=
RESULTS_DIR=
DEBUGGING=
PYTHON_DIR=
3. Inline arguments; anything you don't enter will use the default args! example: **_./run_docker.bat DEBUGGING=y_**
* Accepted args for inline include all listed in the file in #2
<br />
<h1>Connecting to Mono Debugger</h1>
If you launch the script with debugging set to **yes** (y), then you will need to connect to the debugging server with the mono extension that you installed in the setup stage.
To setup the extension do the following:
* Go to **Extensions > Mono > Settings...**
* Enter the following for the settings:
* Remote Host IP: 127.0.0.1
* Remote Host Port: 55555
* Mono Debug Port: 55555
* Click **Save** and then close the extension settings
Now that the extension is setup use it to connect to the Docker container by using:
***Extensions > Mono > Attach to mono debugger**
The program should then launch and trigger any breakpoints you have set in your C# Algorithm.
<h1>Common Issues</h1>
Here we will cover some common issues with setting this up. Feel free to contribute to this section!
<h1>Local Development & Docker Integration with Visual Studio Code</h1>
This document contains information regarding ways to use Visual Studio Code to work with the Lean engine, this includes a couple options that make lean easy to develop on any machine:
This document contains information regarding ways to use Visual Studio Code to work with the Lean engine, this includes using Lean’s Docker image in conjunction with local development as well as running Lean locally.
- Using Lean CLI -> A great tool for working with your algorithms locally, while still being able to deploy to the cloud and have access to Lean data. It is also able to run algorithms locally through our official docker images **Recommended for algorithm development.
- Using a Lean Dev container -> A docker environment with all dependencies pre-installed to allow seamless Lean development across platforms. Great for open source contributors.
- Locally installing all dependencies to run Lean with Visual Studio Code on your OS.
<br />
<h1>Getting Setup</h1>
<h1>Setup</h1>
<h2>Option 1: Lean CLI</h2>
Before anything we need to ensure a few things have been done:
To use Lean CLI follow the instructions for installation and tutorial for usage in our [documentation](https://www.quantconnect.com/docs/v2/lean-cli/key-concepts/getting-started)
<br />
<h2>Option 2: Lean Development Container</h2>
Before anything we need to ensure a few things have been done for either option:
1. Get [Visual Studio Code](https://code.visualstudio.com/download)
* Get the Extension [Mono Debug **15.8**](https://marketplace.visualstudio.com/items?itemName=ms-vscode.mono-debug) for C# Debugging
* Get the Extension [Python](https://marketplace.visualstudio.com/items?itemName=ms-python.python) for Python Debugging
- Get [Remote Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) Extension
2. Get [Docker](https://docs.docker.com/get-docker/):
* Follow the instructions for your Operating System
* New to Docker? Try docker getting-started
- Follow the instructions for your Operating System
- New to Docker? Try [docker getting-started](https://docs.docker.com/get-started/)
3.Install a compiler for the project **(Only needed for C# Debugging or Running Locally)**
* Visual Studio comes packed with msbuild or download without VS [here](https://visualstudio.microsoft.com/downloads/?q=build+tools)
* Put msbuild on your system path and test with command: _msbuild -version_
3.Pull Lean’s latest research image from a terminal
-`docker pull quantconnect/research:latest`
4.Pull Lean’s latest image from a terminal
*_docker pull quantconnect/lean_
4.Get Lean into VS Code
-Download the repo or clone it using: `git clone [https://github.com/QuantConnect/Lean](https://github.com/QuantConnect/Lean)`
- Open the folder using VS Code
5.Get Lean into VS Code
* Download the repo or clone it using: _git clone[ https://github.com/QuantConnect/Lean](https://github.com/QuantConnect/Lean)_
* Open the folder using VS Code
5.Open Development Container
-In VS Code, either:
-Select "Reopen in Container" from pop up box.
**NOTES**:
- Mono Extension Version 16 and greater fails to debug the docker container remotely, please install **Version 15.8**. To install an older version from within VS Code go to the extensions tab, search "Mono Debug", and select "Install Another Version...".
<br />
OR
<h1>Develop Algorithms Locally, Run in Container</h1>
- Ctrl+Shift+P (Command Palette) and select "Remote-Containers: Rebuild and Reopen in Container"
We have set up a relatively easy way to develop algorithms in your local IDE and push them into the container to be run and debugged.
Before we can use this method with Windows or Mac OS we need to share the Lean directory with Docker.
You should now be in the development container, give VS Code a moment to prepare and you will be ready to go!
If you would like to mount any additional local files to your container, checkout [devcontainer.json "mounts" section](https://containers.dev/implementors/json_reference/) for an example! Upon any mount changes you must rebuild the container using Command Palette as in step 5.
<br />
<h2>Activate File Sharing for Docker:</h2>
<h2>Option 3: Install Dependencies Locally</h2>
* Windows:
* [Guide to sharing](https://docs.docker.com/docker-for-windows/#file-sharing)
* Share the LEAN root directory with docker
* Mac:
* [Guide to sharing](https://docs.docker.com/docker-for-mac/#file-sharing)
* Share the LEAN root directory with docker
1. Install [.NET 9](https://dotnet.microsoft.com/en-us/download/dotnet/9.0) for the project
* Linux:
* (No setup required)
2. (Optional) Get [Python 3.11.11](https://www.python.org/downloads/release/python-31111/) for running Python algorithms
-Follow Python instructions [here](https://github.com/QuantConnect/Lean/tree/master/Algorithm.Python#installing-python-311) for your platform
3. Get [Visual Studio Code](https://code.visualstudio.com/download)
- Get the Extension [C#](https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.csharp) for C# Debugging
- Get the Extension [Python](https://marketplace.visualstudio.com/items?itemName=ms-python.python) for Python Debugging
4. Get Lean into VS Code
- Download the repo or clone it using: `git clone [https://github.com/QuantConnect/Lean](https://github.com/QuantConnect/Lean)`
- Open the folder using VS Code
Your environment is prepared and ready to run Lean.
<br />
<h2>Lean Configuration</h2>
<h1>How to use Lean</h1>
Next we need to be sure that our Lean configuration at **.\Launcher\config.json** is properly set. Just like running lean locally the config must reflect what we want Lean to run.
This section will cover configuring, building, launching and debugging lean. This is only applicable to option 2 from above. This does not apply to Lean CLI, please refer to [CLI documentation](https://www.quantconnect.com/docs/v2/lean-cli/key-concepts/getting-started)
You configuration file should look something like this for the following languages:
<br />
<h2>Configuration</h2>
We need to be sure that our Lean configuration at **.\Launcher\config.json** is properly set.
Your configuration file should look something like this for the following languages:
<h3>Python:</h3>
@@ -85,105 +96,48 @@ You configuration file should look something like this for the following languag
In order to use a custom C# algorithm, the C# file must be compiled before running in the docker, as it is compiled into the file "QuantConnect.Algorithm.CSharp.dll". Any new C# files will need to be added to the csproj compile list before it will compile, check Algorithm.CSharp/QuantConnect.Algorithm.CSharp.csproj for all algorithms that are compiled. Once there is an entry for your algorithm the project can be compiled by using the “build” task under _“Terminal” > “Run Build Task”._
Before running Lean, we must build the project. Currently the VS Code task will automatically build before launching. But find more information below about how to trigger building manually.
Python **does not** have this requirement as the engine will compile it on the fly.
In VS Code run build task (Ctrl+Shift+B or "Terminal" dropdown); there are a few options:
- __Build__ - basic build task, just builds Lean once
- __Rebuild__ - rebuild task, completely rebuilds the project. Use if having issues with debugging symbols being loaded for your algorithms.
- __Clean__ - deletes out all project build files
<br />
<h2>Running Lean in the Container</h2>
<h2>Launching Lean</h2>
This section will cover how to actually launch Lean in the container with your desired configuration.
Now that lean is configured and built we can launch Lean. Under "Run & Debug" use the launch option "Launch". This will start Lean with C# debugging. Any breakpoints in Lean C# will be triggered.
<br />
<h3>Option 1 (Recommended)</h3>
In VS Code click on the debug/run icon on the left toolbar, at the top you should see a drop down menu with launch options, be sure to select **Debug in Container**. This option will kick off a launch script that will start the docker. With this specific launch option the parameters are already configured in VS Codes **tasks.json** under the **run-docker** task args. These set arguments are:
As defaults these are all great! Feel free to change them as needed for your setup.
**NOTE:** VSCode may try and throw errors when launching this way regarding build on `QuantConnect.csx` and `Config.json` these errors can be ignored by selecting "*Debug Anyway*". To stop this error message in the future select "*Remember my choice in user settings*".
If using C# algorithms ensure that msbuild can build them successfully.
<h2>Debugging Python</h2>
Python algorithms require a little extra work in order to be able to debug them. Follow the steps below to get Python debugging working.
<br />
<h3>Option 2</h3>
From a terminal launch the run_docker.bat/.sh script; there are a few choices on how to launch this:
1. Launch with no parameters and answer the questions regarding configuration (Press enter for defaults)
* Would you like to debug C#? (Requires mono debugger attachment) [default: N]:
2. Using the **run_docker.cfg** to store args for repeated use; any blank entries will resort to default values! example: **_./run_docker.bat run_docker.cfg_**
IMAGE=quantconnect/lean:latest
CONFIG_FILE=
DATA_DIR=
RESULTS_DIR=
DEBUGGING=
PYTHON_DIR=
3. Inline arguments; anything you don't enter will use the default args! example: **_./run_docker.bat DEBUGGING=y_**
* Accepted args for inline include all listed in the file in #2
<br />
<h1>Debugging Python</h1>
Python algorithms require a little extra work in order to be able to debug them locally or in the container. Thankfully we were able to configure VS code tasks to take care of the work for you! Follow the steps below to get Python debugging working.
<br />
<h2>Modifying the Configuration</h2>
<h3>Modifying the Configuration</h3>
First in order to debug a Python algorithm in VS Code we must make the following change to our configuration (Launcher\config.json) under the comment debugging configuration:
"debugging": true,
"debugging-method": "PTVSD",
"debugging-method": "DebugPy",
In setting this we are telling Lean to expect a debugger connection using ‘Python Tools for Visual Studio Debugger’. Once this is set Lean will stop upon initialization and await a connection to the debugger via port 5678.
<br />
<h2>Using VS Code Launch Options to Connect</h2>
<h3>Using VS Code Launch Options to Connect</h3>
Now that Lean is configured for the python debugger we can make use of the programmed launch options to connect.
Now that Lean is configured for the python debugger we can make use of the programmed launch options to connect to Lean during runtime.
<br />
<h3>Container</h3>
To debug inside of the container we must first start the container, follow the steps described in the section “[Running Lean in the Container](#Running-Lean-in-the-Container)”. Once the container is started you should see the messages in Figure 2.
If the message is displayed, use the same drop down for “Debug in Container” and select “Attach to Python (Container)”. Then press run, VS Code will now enter and debug any breakpoints you have set in your Python algorithm.
<br />
<h3>Local</h3>
To debug locally we must run the program locally using the programmed task found under Terminal > Run Task > “Run Application”. Once Lean is started you should see the messages in Figure 2.
If the message is displayed, use the launch option “Attach to Python (Local)”. Then press run, VS Code will now enter and debug any breakpoints you have set in your python algorithm.
Start Lean using the "Launch" option covered above. Once Lean starts you should see the messages in figure 2 If the message is displayed, use the launch option “Attach to Python”. Then press run, VS Code will now enter and debug any breakpoints you have set in your python algorithm.
Here we will cover some common issues with setting this up. This section will expand as we get user feedback!
* Any error messages about building in VSCode that point to comments in JSON. Either select **ignore** or follow steps described [here](https://stackoverflow.com/questions/47834825/in-vs-code-disable-error-comments-are-not-permitted-in-json) to remove the errors entirely.
*`Errors exist after running preLaunchTask 'run-docker'`This VSCode error appears to warn you of CSharp errors when trying to use `Debug in Container` select "Debug Anyway" as the errors are false flags for JSON comments as well as `QuantConnect.csx` not finding references. Neither of these will impact your debugging.
* `The container name "/LeanEngine" is already in use by container "****"` This Docker error implies that another instance of lean is already running under the container name /LeanEngine. If this error appears either use Docker Desktop to delete the container or use `docker kill LeanEngine` from the command line.
-The "project file cannot be loaded" and "nuget packages not found" errors occurs when the project files are open by another process in the host. Closing all applications and/or restarting the computer solve the issue.
-Autocomplete and reference finding with omnisharp can sometimes be buggy, if this occurs use the command palette to restart omnisharp. (Ctrl+Shift+P "OmniSharp: Restart OmniSharp")
- Any error messages about building in VSCode that point to comments in JSON. Either select **ignore** or follow steps described [here](https://stackoverflow.com/questions/47834825/in-vs-code-disable-error-comments-are-not-permitted-in-json) to remove the errors entirely.
thrownewException($"Unexpected configuration for {symbol} after it has been deselected from coarse universe and option contract is removed.");
thrownewRegressionTestException($"Unexpected configuration for {symbol} after it has been deselected from coarse universe and option contract is removed.");
thrownewException($"{Time}:: Detected differences in expected and actual securities{Environment.NewLine}Expected:{Environment.NewLine}{expected}{Environment.NewLine}Actual:{Environment.NewLine}{actual}");
thrownewRegressionTestException($"{Time}:: Detected differences in expected and actual securities{Environment.NewLine}Expected:{Environment.NewLine}{expected}{Environment.NewLine}Actual:{Environment.NewLine}{actual}");
thrownewException($"{Time}:: Detected differences in expected and actual universes{Environment.NewLine}Expected:{Environment.NewLine}{expected}{Environment.NewLine}Actual:{Environment.NewLine}{actual}");
thrownewRegressionTestException($"{Time}:: Detected differences in expected and actual universes{Environment.NewLine}Expected:{Environment.NewLine}{expected}{Environment.NewLine}Actual:{Environment.NewLine}{actual}");
thrownewException($"{Time}:: Detected differences in expected and actual slice data keys{Environment.NewLine}Expected:{Environment.NewLine}{expected}{Environment.NewLine}Actual:{Environment.NewLine}{actual}");
thrownewRegressionTestException($"{Time}:: Detected differences in expected and actual slice data keys{Environment.NewLine}Expected:{Environment.NewLine}{expected}{Environment.NewLine}Actual:{Environment.NewLine}{actual}");
thrownewException($"This algorithm intends to add a single security at a time but added: {changes.AddedSecurities.Count}{Environment.NewLine}{securities}");
thrownewException($"Expected Symbols selected on {Time.Date:yyyy-MM-dd} to match expected Symbols, but the following Symbols were missing: {string.Join(",", missing.Select(s => s.ToString()))}");
thrownewRegressionTestException($"Expected Symbols selected on {Time.Date:yyyy-MM-dd} to match expected Symbols, but the following Symbols were missing: {string.Join(",", missing.Select(s => s.ToString()))}");
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.