Include Order.Tag/OrderEvent.Message in their ToString, Fix default tag values (#4797)

* Improve information tracked in regression's {algorithm}.{lang}.details.log

The details.log file aims at providing a diff-able document that quickly and
easily provides actionable information. Since many regression algorithms use
the algorithm's debug/error messaging facilities to log various pieces of algo
state. This document also support a configuration option: regression-high-fidelity-logging'
that logs EVERY piece of data, again, with the aim of providing an easily diff-able
documenbt to quickly highlight actionable information. I may have missed omse key
pieces of information here, but now that the entire QC knows about this regression
tool, if additional information is required then hopefully it's easy enough at this
point to extend the RegressionResultHandler to suit our needs.

The RegressionResultHandler was initially implemented to provide a concise log of
all orders. This was achieved by simply using the Order.ToString method. While
testing/investigating OptionExerciseOrder behavior, it became evident that more
information was required to properly identify the source of potential failures or
differences between previous regression test runs. This change adds logging for
almost every IResultHandler method and additionally attempts to capture the
actual portfolio impact of every OrderEvent. This is accomplished by logging
the portfolio's TotalPortfolioValue, Cash properties and the security's
SecurityHolding.Quantity property.

This change also standardizes the timestamps used to folloow the ISO-8601 format.

When using the RegressionResultHandler, it is highly recommeded to also disable
'forward-console-message' configuration option to ensure algorithm Debug/Error
message logging is done synchronously to ensure correct ordering with respect to
log messages via Log.Debug/Trace/Error.

* Fix typo in options OrderTests test case name

* Update SymbolRepresentation.GenerationOptionTickerOSI to extension method

Far more convenient as an extension method

* Improve R# default code formatting rules

Many of these rule changes focus on improving the readability of code,
with a particular emphasis on multi-line constructs, chained method calls
and multi-line method invocations/declarations.

* Add braces, use string interpolation and limit long lines

* Refactor OptionExerciseOrder.Quantity to indicate change in #contracts

For all other order types, the Order.Quantity indicates the change in the algorithm's
holdings upon order execution for the order's symbol. For OptionExerciseOrder, this
convention was broken. It appears as though only exercise was initially implemented,
in which case only long positions were supported and a code comment indicated that
only positive values of quantity were acceptable, indicating the number of contracts
to exercise. At a later date, assignment simulation was added and utilized a negative
order quantity. This caused some major inconsistencies in how models view exercise
orders compared to all other order types. This change brings OptionExerciseOrder.Quantity
into alignment with the other order types by making it represent the change in holdings
quantity upon order execution.

This change was originally going to be much larger, but in order to minimize risks and to
make for an easier review experience, the additional changes will be committed separately
and pushed in their own PR. Some of the issues identified include:
* Manual Exercise (especially for OTM) is not covered
* Margin Calculations (in particular taking into account opposing contracts held)
* IBrokerage.OptionPositionAssigned is raised for exercise (later filtered by tx handler)

Fixes OptionPortfolioModelTests to use exercise model to properly model exercise of
non-account quote currency option contract.

* Include Order.Tag/OrderEvent.Message in their ToString, Fix default tag values

There was inconsistencies in what we were checking for. The order constructors
default the tag parameter to an empty string but Order.CreateOrder checks for
a null string. Additionally, the order constructors (limit,stopmarket,stoplimit)
would check for an empty string and if so, apply a default order tag.

This change cleans these checks up using string.IsNullOrEmpty and also removes the
check from Order.CreateOrder since we're passing the tag into the various order
constructors.

Algorithm.CSharp/BacktestingBrokerageRegressionAlgorithm.cs

@@ -0,0 +1,323 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using QuantConnect.Data;
+using QuantConnect.Data.Market;
+using QuantConnect.Interfaces;
+using QuantConnect.Orders;
+using QuantConnect.Orders.Fees;
+using QuantConnect.Orders.Fills;
+using QuantConnect.Securities;
+using QuantConnect.Securities.Option;
+using System;
+using System.Collections.Generic;
+using System.Linq;
+
+namespace QuantConnect.Algorithm.CSharp
+{
+    /// <summary>
+    /// This regression algorithm tests the order processing of the backtesting brokerage.
+    /// We open an equity position that should fill in two parts, on two different bars.
+    /// We open a long option position and let it expire so we can exercise the position.
+    /// To check the orders we use OnOrderEvent and throw exceptions if verification fails.
+    /// </summary>
+    /// <meta name="tag" content="backtesting brokerage" />
+    /// <meta name="tag" content="regression test" />
+    /// <meta name="tag" content="options" />
+    class BacktestingBrokerageRegressionAlgorithm : QCAlgorithm, IRegressionAlgorithmDefinition
+    {
+        private Security _security;
+        private Symbol _spy;
+        private OrderTicket _equityBuy;
+        private Option _option;
+        private Symbol _optionSymbol;
+        private OrderTicket _optionBuy;
+        private bool _optionBought = false;
+        private bool _equityBought = false;
+        private decimal _optionStrikePrice;
+
+        /// <summary>
+        /// Initialize the algorithm
+        /// </summary>
+        public override void Initialize()
+        {
+            SetCash(100000);
+            SetStartDate(2015, 12, 24);
+            SetEndDate(2015, 12, 28);
+
+            // Get our equity
+            _security = AddEquity("SPY", Resolution.Hour);
+            _security.SetFillModel(new PartialMarketFillModel(2));
+            _spy = _security.Symbol;
+
+            // Get our option
+            _option = AddOption("GOOG");
+            _option.SetFilter(u => u.IncludeWeeklys()
+                                   .Strikes(-2, +2)
+                                   .Expiration(TimeSpan.Zero, TimeSpan.FromDays(10)));
+            _optionSymbol = _option.Symbol;
+        }
+
+        /// <summary>
+        /// OnData event is the primary entry point for your algorithm. Each new data point will be pumped in here.
+        /// </summary>
+        /// <param name="data">Slice object keyed by symbol containing the stock data</param>
+        public override void OnData(Slice data)
+        {
+            if (!_equityBought && data.ContainsKey(_spy)) {
+                //Buy our Equity
+                var quantity = CalculateOrderQuantity(_spy, .1m);
+                _equityBuy = MarketOrder(_spy, quantity, asynchronous: true);
+                _equityBought = true;
+            }
+
+            if (!_optionBought)
+            {
+                // Buy our option
+                OptionChain chain;
+                if (data.OptionChains.TryGetValue(_optionSymbol, out chain))
+                {
+                    // Find the second call strike under market price expiring today
+                    var contracts = (
+                        from optionContract in chain.OrderByDescending(x => x.Strike)
+                        where optionContract.Right == OptionRight.Call
+                        where optionContract.Expiry == Time.Date
+                        where optionContract.Strike < chain.Underlying.Price
+                        select optionContract
+                        ).Take(2);
+
+                    if (contracts.Any())
+                    {
+                        var optionToBuy = contracts.FirstOrDefault();
+                        _optionStrikePrice = optionToBuy.Strike;
+                        _optionBuy = MarketOrder(optionToBuy.Symbol, 1);
+                        _optionBought = true;
+                    }
+                }
+            }
+        }
+
+        /// <summary>
+        /// All order events get pushed through this function
+        /// </summary>
+        /// <param name="orderEvent">OrderEvent object that contains all the information about the event</param>
+        public override void OnOrderEvent(OrderEvent orderEvent)
+        {
+            // Get the order from our transactions
+            var order = Transactions.GetOrderById(orderEvent.OrderId);
+
+            // Based on the type verify the order
+            switch(order.Type)
+            {
+                case OrderType.Market:
+                    VerifyMarketOrder(order, orderEvent);
+                    break;
+
+                case OrderType.OptionExercise:
+                    VerifyOptionExercise(order, orderEvent);
+                    break;
+
+                default:
+                    throw new ArgumentOutOfRangeException();
+            }
+        }
+
+        /// <summary>
+        /// To verify Market orders is process correctly
+        /// </summary>
+        /// <param name="order">Order object to analyze</param>
+        public void VerifyMarketOrder(Order order, OrderEvent orderEvent)
+        {
+            switch(order.Status)
+            {
+                case OrderStatus.Submitted:
+                    break;
+
+                // All PartiallyFilled orders should have a LastFillTime
+                case OrderStatus.PartiallyFilled:
+                    if (order.LastFillTime == null)
+                    {
+                        throw new Exception("LastFillTime should not be null");
+                    }
+
+                    if (order.Quantity/2 != orderEvent.FillQuantity)
+                    {
+                        throw new Exception("Order size should be half");
+                    }
+                    break;
+
+                // All filled equity orders should have filled after creation because of our fill model!
+                case OrderStatus.Filled:
+                    if (order.SecurityType == SecurityType.Equity && order.CreatedTime == order.LastFillTime)
+                    {
+                        throw new Exception("Order should not finish during the CreatedTime bar");
+                    }
+                    break;
+
+                default:
+                    throw new ArgumentOutOfRangeException();
+            }
+        }
+
+        /// <summary>
+        /// To verify OptionExercise orders is process correctly
+        /// </summary>
+        /// <param name="order">Order object to analyze</param>
+        public void VerifyOptionExercise(Order order, OrderEvent orderEvent)
+        {
+            // If the option price isn't the same as the strike price, its incorrect
+            if (order.Price != _optionStrikePrice)
+            {
+                throw new Exception("OptionExercise order price should be strike price!!");
+            }
+
+            if (orderEvent.Quantity != -1)
+            {
+                throw new Exception("OrderEvent Quantity should be -1");
+            }
+        }
+
+        /// <summary>
+        /// Runs after algorithm, used to check our portfolio and orders
+        /// </summary>
+        public override void OnEndOfAlgorithm()
+        {
+            if (!Portfolio.ContainsKey(_optionBuy.Symbol) || !Portfolio.ContainsKey(_optionBuy.Symbol.Underlying) || !Portfolio.ContainsKey(_equityBuy.Symbol))
+            {
+                throw new Exception("Portfolio does not contain the Symbols we purchased");
+            }
+
+            //Check option holding, should not be invested since it expired, profit should be -400
+            var optionHolding = Portfolio[_optionBuy.Symbol];
+            if (optionHolding.Invested || optionHolding.Profit != -400)
+            {
+                throw new Exception("Options holding does not match expected outcome");
+            }
+
+            //Check the option underlying symbol since we should have bought it at exercise
+            //Quantity should be 100, AveragePrice should be option strike price
+            var optionExerciseHolding = Portfolio[_optionBuy.Symbol.Underlying];
+            if (!optionExerciseHolding.Invested || optionExerciseHolding.Quantity != 100 || optionExerciseHolding.AveragePrice != _optionBuy.Symbol.ID.StrikePrice)
+            {
+                throw new Exception("Equity holding for exercised option does not match expected outcome");
+            }
+
+            //Check equity holding, should be invested, profit should be
+            //Quantity should be 50, AveragePrice should be ticket AverageFillPrice
+            var equityHolding = Portfolio[_equityBuy.Symbol];
+            if (!equityHolding.Invested || equityHolding.Quantity != 50 || equityHolding.AveragePrice != _equityBuy.AverageFillPrice)
+            {
+                throw new Exception("Equity holding does not match expected outcome");
+            }
+        }
+
+        /// <summary>
+        /// PartialMarketFillModel that allows the user to set the number of fills and restricts
+        /// the fill to only one per bar.
+        /// </summary>
+        private class PartialMarketFillModel : ImmediateFillModel
+        {
+            private readonly decimal _percent;
+            private readonly Dictionary<long, decimal> _absoluteRemainingByOrderId = new Dictionary<long, decimal>();
+
+            /// <param name="numberOfFills"></param>
+            public PartialMarketFillModel(int numberOfFills = 1)
+            {
+                _percent = 1m / numberOfFills;
+            }
+
+            /// <summary>
+            /// Performs partial market fills once per time step
+            /// </summary>
+            /// <param name="asset">The security being ordered</param>
+            /// <param name="order">The order</param>
+            /// <returns>The order fill</returns>
+            public override OrderEvent MarketFill(Security asset, MarketOrder order)
+            {
+                var currentUtcTime = asset.LocalTime.ConvertToUtc(asset.Exchange.TimeZone);
+
+                // Only fill once a time slice
+                if (order.LastFillTime != null && currentUtcTime <= order.LastFillTime)
+                {
+                    return new OrderEvent(order, currentUtcTime, OrderFee.Zero);
+                }
+
+                decimal absoluteRemaining;
+                if (!_absoluteRemainingByOrderId.TryGetValue(order.Id, out absoluteRemaining))
+                {
+                    absoluteRemaining = order.AbsoluteQuantity;
+                    _absoluteRemainingByOrderId.Add(order.Id, order.AbsoluteQuantity);
+                }
+
+                var fill = base.MarketFill(asset, order);
+                var absoluteFillQuantity = (int)(Math.Min(absoluteRemaining, (int)(_percent * order.Quantity)));
+                fill.FillQuantity = Math.Sign(order.Quantity) * absoluteFillQuantity;
+
+                if (absoluteRemaining == absoluteFillQuantity)
+                {
+                    fill.Status = OrderStatus.Filled;
+                    _absoluteRemainingByOrderId.Remove(order.Id);
+                }
+                else
+                {
+                    absoluteRemaining = absoluteRemaining - absoluteFillQuantity;
+                    _absoluteRemainingByOrderId[order.Id] = absoluteRemaining;
+                    fill.Status = OrderStatus.PartiallyFilled;
+                }
+
+                return fill;
+            }
+        }
+
+        /// <summary>
+        /// This is used by the regression test system to indicate if the open source Lean repository has the required data to run this algorithm.
+        /// </summary>
+        public bool CanRunLocally { get; } = true;
+
+        /// <summary>
+        /// This is used by the regression test system to indicate which languages this algorithm is written in.
+        /// </summary>
+        public Language[] Languages { get; } = { Language.CSharp };
+
+        /// <summary>
+        /// This is used by the regression test system to indicate what the expected statistics are from running the algorithm
+        /// </summary>
+        public Dictionary<string, string> ExpectedStatistics => new Dictionary<string, string>
+        {
+            {"Total Trades", "3"},
+            {"Average Win", "0%"},
+            {"Average Loss", "-0.40%"},
+            {"Compounding Annual Return", "-22.335%"},
+            {"Drawdown", "0.400%"},
+            {"Expectancy", "-1"},
+            {"Net Profit", "-0.323%"},
+            {"Sharpe Ratio", "-0.888"},
+            {"Probabilistic Sharpe Ratio", "0%"},
+            {"Loss Rate", "100%"},
+            {"Win Rate", "0%"},
+            {"Profit-Loss Ratio", "0"},
+            {"Alpha", "0.035"},
+            {"Beta", "0.183"},
+            {"Annual Standard Deviation", "0.004"},
+            {"Annual Variance", "0"},
+            {"Information Ratio", "12.058"},
+            {"Tracking Error", "0.017"},
+            {"Treynor Ratio", "-0.018"},
+            {"Total Fees", "$2.00"},
+            {"Fitness Score", "0.213"},
+            {"OrderListHash", "904167951"}
+        };
+    }
+}

Algorithm.CSharp/BasicTemplateCryptoAlgorithm.cs

@@ -227,7 +227,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Kelly Criterion Estimate", "0"},
             {"Kelly Criterion Probability Value", "0"},
             {"Sortino Ratio", "79228162514264337593543950335"},
-            {"Return Over Maximum Drawdown", "-43.917"},
+            {"Return Over Maximum Drawdown", "-43.937"},
             {"Portfolio Turnover", "1.028"},
             {"Total Insights Generated", "0"},
             {"Total Insights Closed", "0"},
@@ -242,7 +242,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Mean Population Magnitude", "0%"},
             {"Rolling Averaged Population Direction", "0%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "1073240275"}
+            {"OrderListHash", "415415696"}
         };
     }
 }

Algorithm.CSharp/BasicTemplateOptionsFilterUniverseAlgorithm.cs

@@ -141,7 +141,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Mean Population Magnitude", "0%"},
             {"Rolling Averaged Population Direction", "0%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "-1214175458"}
+            {"OrderListHash", "687310345"}
         };
     }
 }

Algorithm.CSharp/CancelOpenOrdersRegressionAlgorithm.cs

@@ -166,7 +166,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Mean Population Magnitude", "0%"},
             {"Rolling Averaged Population Direction", "0%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "-1708974186"}
+            {"OrderListHash", "737971736"}
         };
     }
 }

Algorithm.CSharp/DailyHistoryForDailyResolutionRegressionAlgorithm.cs

@@ -0,0 +1,143 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using QuantConnect.Data;
+using QuantConnect.Data.Market;
+using QuantConnect.Interfaces;
+using QuantConnect.Securities;
+
+namespace QuantConnect.Algorithm.CSharp
+{
+    /// <summary>
+    /// Algorithm simply fetch one-day history prior current time.
+    /// </summary>
+    public class DailyHistoryForDailyResolutionRegressionAlgorithm : QCAlgorithm, IRegressionAlgorithmDefinition
+    {
+        private Symbol[] _symbols = {
+            QuantConnect.Symbol.Create("GBPUSD", SecurityType.Forex, market: Market.FXCM),
+            QuantConnect.Symbol.Create("EURUSD", SecurityType.Forex, market: Market.Oanda),
+            QuantConnect.Symbol.Create("AAPL", SecurityType.Equity, market: Market.USA),
+            QuantConnect.Symbol.Create("BTCUSD", SecurityType.Crypto, market: Market.GDAX),
+            QuantConnect.Symbol.Create("XAUUSD", SecurityType.Cfd, market: Market.Oanda)
+        };
+
+        private HashSet<Symbol> _received = new HashSet<Symbol>();
+
+        public override void Initialize()
+        {
+            SetStartDate(2018, 3, 26);
+            SetEndDate(2018, 4, 10);
+            foreach (var symbol in _symbols)
+            {
+                AddSecurity(symbol, Resolution.Daily);
+            }
+        }
+
+        public override void OnData(Slice data)
+        {
+            using (var enumerator = data.GetEnumerator())
+            {
+                while (enumerator.MoveNext())
+                {
+                    var current = enumerator.Current;
+                    var symbol = current.Key;
+                    _received.Add(symbol);
+
+                    List<BaseData> history;
+
+                    if (current.Value.DataType == MarketDataType.QuoteBar)
+                    {
+                        history = History(1, Resolution.Daily).Get<QuoteBar>(symbol).Cast<BaseData>().ToList();
+                    }
+                    else
+                    {
+                        history = History(1, Resolution.Daily).Get<TradeBar>(symbol).Cast<BaseData>().ToList();
+                    }
+
+                    if (!history.Any()) throw new Exception($"No {symbol} data on the eve of {Time} {Time.DayOfWeek}");
+                }
+            }
+        }
+
+        public override void OnEndOfAlgorithm()
+        {
+            if (_received.Count != _symbols.Length)
+            {
+                throw new Exception($"Data for symbols {string.Join(",", _symbols.Except(_received))} were not received");
+            }
+        }
+
+
+        /// <summary>
+        /// This is used by the regression test system to indicate if the open source Lean repository has the required data to run this algorithm.
+        /// </summary>
+        public bool CanRunLocally { get; } = true;
+
+        /// <summary>
+        /// This is used by the regression test system to indicate which languages this algorithm is written in.
+        /// </summary>
+        public Language[] Languages { get; } = { Language.CSharp };
+
+        /// <summary>
+        /// This is used by the regression test system to indicate what the expected statistics are from running the algorithm
+        /// </summary>
+        public Dictionary<string, string> ExpectedStatistics => new Dictionary<string, string>
+        {
+            {"Total Trades", "0"},
+            {"Average Win", "0%"},
+            {"Average Loss", "0%"},
+            {"Compounding Annual Return", "0%"},
+            {"Drawdown", "0%"},
+            {"Expectancy", "0"},
+            {"Net Profit", "0%"},
+            {"Sharpe Ratio", "0"},
+            {"Probabilistic Sharpe Ratio", "0%"},
+            {"Loss Rate", "0%"},
+            {"Win Rate", "0%"},
+            {"Profit-Loss Ratio", "0"},
+            {"Alpha", "0"},
+            {"Beta", "0"},
+            {"Annual Standard Deviation", "0"},
+            {"Annual Variance", "0"},
+            {"Information Ratio", "-0.084"},
+            {"Tracking Error", "0.183"},
+            {"Treynor Ratio", "0"},
+            {"Total Fees", "$0.00"},
+            {"Fitness Score", "0"},
+            {"Kelly Criterion Estimate", "0"},
+            {"Kelly Criterion Probability Value", "0"},
+            {"Sortino Ratio", "79228162514264337593543950335"},
+            {"Return Over Maximum Drawdown", "79228162514264337593543950335"},
+            {"Portfolio Turnover", "0"},
+            {"Total Insights Generated", "0"},
+            {"Total Insights Closed", "0"},
+            {"Total Insights Analysis Completed", "0"},
+            {"Long Insight Count", "0"},
+            {"Short Insight Count", "0"},
+            {"Long/Short Ratio", "100%"},
+            {"Estimated Monthly Alpha Value", "$0"},
+            {"Total Accumulated Estimated Alpha Value", "$0"},
+            {"Mean Population Estimated Insight Value", "$0"},
+            {"Mean Population Direction", "0%"},
+            {"Mean Population Magnitude", "0%"},
+            {"Rolling Averaged Population Direction", "0%"},
+            {"Rolling Averaged Population Magnitude", "0%"},
+            {"OrderListHash", "371857150"}
+        };
+    }
+}

Algorithm.CSharp/DailyHistoryForMinuteResolutionRegressionAlgorithm.cs

@@ -0,0 +1,143 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using QuantConnect.Data;
+using QuantConnect.Data.Market;
+using QuantConnect.Interfaces;
+using QuantConnect.Securities;
+
+namespace QuantConnect.Algorithm.CSharp
+{
+    /// <summary>
+    /// Algorithm simply fetch one-day history prior current time.
+    /// </summary>
+    public class DailyHistoryForMinuteResolutionRegressionAlgorithm : QCAlgorithm, IRegressionAlgorithmDefinition
+    {
+        private Symbol[] _symbols = {
+            QuantConnect.Symbol.Create("GBPUSD", SecurityType.Forex, market: Market.FXCM),
+            QuantConnect.Symbol.Create("EURUSD", SecurityType.Forex, market: Market.Oanda),
+            QuantConnect.Symbol.Create("AAPL", SecurityType.Equity, market: Market.USA),
+            QuantConnect.Symbol.Create("BTCUSD", SecurityType.Crypto, market: Market.GDAX),
+            QuantConnect.Symbol.Create("XAUUSD", SecurityType.Cfd, market: Market.Oanda)
+        };
+
+        private HashSet<Symbol> _received = new HashSet<Symbol>();
+
+        public override void Initialize()
+        {
+            SetStartDate(2018, 3, 26);
+            SetEndDate(2018, 4, 10);
+            foreach (var symbol in _symbols)
+            {
+                AddSecurity(symbol, Resolution.Minute);
+            }
+
+            Schedule.On(DateRules.EveryDay(), TimeRules.Every(TimeSpan.FromHours(1)), MakeHistoryCall);
+        }
+
+        private void MakeHistoryCall()
+        {
+            foreach (var symbol in _symbols)
+            {
+                _received.Add(symbol);
+
+                bool hasHistory = false;
+
+                foreach (var dataType in SubscriptionManager.AvailableDataTypes[symbol.SecurityType])
+                {
+                    if (dataType == TickType.Quote)
+                    {
+                        hasHistory |= History(1, Resolution.Daily).Get<QuoteBar>(symbol).Any();
+                    }
+                    else
+                    {
+                        hasHistory |= History(1, Resolution.Daily).Get<TradeBar>(symbol).Any();
+                    }
+                }
+
+                if (!hasHistory) throw new Exception($"No {symbol} data on the eve of {Time} {Time.DayOfWeek}");
+            }
+        }
+
+        public override void OnEndOfAlgorithm()
+        {
+            if (_received.Count != _symbols.Length)
+            {
+                throw new Exception($"Data for symbols {string.Join(",", _symbols.Except(_received))} were not received");
+            }
+        }
+
+
+        /// <summary>
+        /// This is used by the regression test system to indicate if the open source Lean repository has the required data to run this algorithm.
+        /// </summary>
+        public bool CanRunLocally { get; } = true;
+
+        /// <summary>
+        /// This is used by the regression test system to indicate which languages this algorithm is written in.
+        /// </summary>
+        public Language[] Languages { get; } = { Language.CSharp };
+
+        /// <summary>
+        /// This is used by the regression test system to indicate what the expected statistics are from running the algorithm
+        /// </summary>
+        public Dictionary<string, string> ExpectedStatistics => new Dictionary<string, string>
+        {
+            {"Total Trades", "0"},
+            {"Average Win", "0%"},
+            {"Average Loss", "0%"},
+            {"Compounding Annual Return", "0%"},
+            {"Drawdown", "0%"},
+            {"Expectancy", "0"},
+            {"Net Profit", "0%"},
+            {"Sharpe Ratio", "0"},
+            {"Probabilistic Sharpe Ratio", "0%"},
+            {"Loss Rate", "0%"},
+            {"Win Rate", "0%"},
+            {"Profit-Loss Ratio", "0"},
+            {"Alpha", "0"},
+            {"Beta", "0"},
+            {"Annual Standard Deviation", "0"},
+            {"Annual Variance", "0"},
+            {"Information Ratio", "-0.096"},
+            {"Tracking Error", "0.212"},
+            {"Treynor Ratio", "0"},
+            {"Total Fees", "$0.00"},
+            {"Fitness Score", "0"},
+            {"Kelly Criterion Estimate", "0"},
+            {"Kelly Criterion Probability Value", "0"},
+            {"Sortino Ratio", "79228162514264337593543950335"},
+            {"Return Over Maximum Drawdown", "79228162514264337593543950335"},
+            {"Portfolio Turnover", "0"},
+            {"Total Insights Generated", "0"},
+            {"Total Insights Closed", "0"},
+            {"Total Insights Analysis Completed", "0"},
+            {"Long Insight Count", "0"},
+            {"Short Insight Count", "0"},
+            {"Long/Short Ratio", "100%"},
+            {"Estimated Monthly Alpha Value", "$0"},
+            {"Total Accumulated Estimated Alpha Value", "$0"},
+            {"Mean Population Estimated Insight Value", "$0"},
+            {"Mean Population Direction", "0%"},
+            {"Mean Population Magnitude", "0%"},
+            {"Rolling Averaged Population Direction", "0%"},
+            {"Rolling Averaged Population Magnitude", "0%"},
+            {"OrderListHash", "371857150"}
+        };
+    }
+}

Algorithm.CSharp/DaylightSavingTimeHistoryRegressionAlgorithm.cs

@@ -0,0 +1,118 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using NodaTime;
+using QuantConnect.Data;
+using QuantConnect.Data.Market;
+using QuantConnect.Interfaces;
+
+namespace QuantConnect.Algorithm.CSharp
+{
+    /// <summary>
+    /// Regression test algorithm simply fetch history on boarder of Daylight Saving Time shift
+    /// </summary>
+    public class DaylightSavingTimeHistoryRegressionAlgorithm : QCAlgorithm, IRegressionAlgorithmDefinition
+    {
+        private Symbol[] _symbols = new[]
+        {
+            QuantConnect.Symbol.Create("EURUSD", SecurityType.Forex, Market.FXCM),
+            QuantConnect.Symbol.Create("SPY", SecurityType.Equity, Market.USA)
+        };
+        /// <summary>
+        /// Initialise the data and resolution required, as well as the cash and start-end dates for your algorithm. All algorithms must initialized.
+        /// </summary>
+        public override void Initialize()
+        {
+            SetStartDate(2011, 11, 10);  //Set Start Date
+            SetEndDate(2011, 11, 11);    //Set End Date
+            SetCash(100000);             //Set Strategy Cash
+
+            for (int i = 0; i < _symbols.Length; i++)
+            {
+                var symbol = _symbols[i];
+                var history = History<QuoteBar>(symbol, 10, Resolution.Daily);
+
+                var duplications = history
+                    .GroupBy(k => k.Time)
+                    .Where(g => g.Count() > 1);
+                if (duplications.Any())
+                {
+                    var time = duplications.First().Key;
+                    throw new Exception($"Duplicated bars were issued for time {time}");
+                }
+            }
+        }
+
+        /// <summary>
+        /// This is used by the regression test system to indicate if the open source Lean repository has the required data to run this algorithm.
+        /// </summary>
+        public bool CanRunLocally { get; } = true;
+
+        /// <summary>
+        /// This is used by the regression test system to indicate which languages this algorithm is written in.
+        /// </summary>
+        public Language[] Languages { get; } = { Language.CSharp };
+
+        /// <summary>
+        /// This is used by the regression test system to indicate what the expected statistics are from running the algorithm
+        /// </summary>
+        public Dictionary<string, string> ExpectedStatistics => new Dictionary<string, string>
+        {
+            {"Total Trades", "0"},
+            {"Average Win", "0%"},
+            {"Average Loss", "0%"},
+            {"Compounding Annual Return", "0%"},
+            {"Drawdown", "0%"},
+            {"Expectancy", "0"},
+            {"Net Profit", "0%"},
+            {"Sharpe Ratio", "0"},
+            {"Probabilistic Sharpe Ratio", "0%"},
+            {"Loss Rate", "0%"},
+            {"Win Rate", "0%"},
+            {"Profit-Loss Ratio", "0"},
+            {"Alpha", "0"},
+            {"Beta", "0"},
+            {"Annual Standard Deviation", "0"},
+            {"Annual Variance", "0"},
+            {"Information Ratio", "0"},
+            {"Tracking Error", "0"},
+            {"Treynor Ratio", "0"},
+            {"Total Fees", "$0.00"},
+            {"Fitness Score", "0"},
+            {"Kelly Criterion Estimate", "0"},
+            {"Kelly Criterion Probability Value", "0"},
+            {"Sortino Ratio", "79228162514264337593543950335"},
+            {"Return Over Maximum Drawdown", "79228162514264337593543950335"},
+            {"Portfolio Turnover", "0"},
+            {"Total Insights Generated", "0"},
+            {"Total Insights Closed", "0"},
+            {"Total Insights Analysis Completed", "0"},
+            {"Long Insight Count", "0"},
+            {"Short Insight Count", "0"},
+            {"Long/Short Ratio", "100%"},
+            {"Estimated Monthly Alpha Value", "$0"},
+            {"Total Accumulated Estimated Alpha Value", "$0"},
+            {"Mean Population Estimated Insight Value", "$0"},
+            {"Mean Population Direction", "0%"},
+            {"Mean Population Magnitude", "0%"},
+            {"Rolling Averaged Population Direction", "0%"},
+            {"Rolling Averaged Population Magnitude", "0%"},
+            {"OrderListHash", "371857150"}
+        };
+    }
+}

Algorithm.CSharp/EquityTickQuoteAdjustedModeRegressionAlgorithm.cs

@@ -0,0 +1,128 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using System.Collections.Generic;
+using QuantConnect.Data;
+using QuantConnect.Interfaces;
+
+namespace QuantConnect.Algorithm.CSharp
+{
+    /// <summary>
+    /// Checks that the Tick BidPrice and AskPrices are adjusted like Value.
+    /// </summary>
+    public class EquityTickQuoteAdjustedModeRegressionAlgorithm : QCAlgorithm, IRegressionAlgorithmDefinition
+    {
+        private Symbol _ibm;
+        private bool _bought;
+        private bool _sold;
+
+        public override void Initialize()
+        {
+            SetStartDate(2013, 10, 7);
+            SetEndDate(2013, 10, 11);
+            SetCash(100000);
+
+            _ibm = AddEquity("IBM", Resolution.Tick).Symbol;
+        }
+
+        public override void OnData(Slice data)
+        {
+            if (!data.Ticks.ContainsKey(_ibm))
+            {
+                return;
+            }
+
+            var security = Securities[_ibm];
+            if (!security.HasData)
+            {
+                return;
+            }
+
+            foreach (var tick in data.Ticks[_ibm])
+            {
+                if (tick.BidPrice != 0 && !_bought && ((tick.Value - tick.BidPrice) <= 0.05m))
+                {
+                    SetHoldings(_ibm, 1);
+                    _bought = true;
+                    return;
+                }
+                if (tick.AskPrice != 0 && _bought && !_sold && Math.Abs((double)tick.Value - (double)tick.AskPrice) <= 0.05)
+                {
+                    Liquidate(_ibm);
+                    _sold = true;
+                    return;
+                }
+            }
+        }
+
+        /// <summary>
+        /// This is used by the regression test system to indicate if the open source Lean repository has the required data to run this algorithm.
+        /// </summary>
+        public bool CanRunLocally { get; } = true;
+
+        /// <summary>
+        /// This is used by the regression test system to indicate which languages this algorithm is written in.
+        /// </summary>
+        public Language[] Languages { get; } = { Language.CSharp };
+
+        /// <summary>
+        /// This is used by the regression test system to indicate what the expected statistics are from running the algorithm
+        /// </summary>
+        public Dictionary<string, string> ExpectedStatistics => new Dictionary<string, string>
+        {
+            {"Total Trades", "2"},
+            {"Average Win", "0%"},
+            {"Average Loss", "-0.01%"},
+            {"Compounding Annual Return", "-0.500%"},
+            {"Drawdown", "0.000%"},
+            {"Expectancy", "-1"},
+            {"Net Profit", "-0.006%"},
+            {"Sharpe Ratio", "0"},
+            {"Probabilistic Sharpe Ratio", "0%"},
+            {"Loss Rate", "100%"},
+            {"Win Rate", "0%"},
+            {"Profit-Loss Ratio", "0"},
+            {"Alpha", "0"},
+            {"Beta", "0"},
+            {"Annual Standard Deviation", "0"},
+            {"Annual Variance", "0"},
+            {"Information Ratio", "-8.769"},
+            {"Tracking Error", "0.22"},
+            {"Treynor Ratio", "0"},
+            {"Total Fees", "$6.41"},
+            {"Fitness Score", "0.248"},
+            {"Kelly Criterion Estimate", "0"},
+            {"Kelly Criterion Probability Value", "0"},
+            {"Sortino Ratio", "79228162514264337593543950335"},
+            {"Return Over Maximum Drawdown", "-82.815"},
+            {"Portfolio Turnover", "0.497"},
+            {"Total Insights Generated", "0"},
+            {"Total Insights Closed", "0"},
+            {"Total Insights Analysis Completed", "0"},
+            {"Long Insight Count", "0"},
+            {"Short Insight Count", "0"},
+            {"Long/Short Ratio", "100%"},
+            {"Estimated Monthly Alpha Value", "$0"},
+            {"Total Accumulated Estimated Alpha Value", "$0"},
+            {"Mean Population Estimated Insight Value", "$0"},
+            {"Mean Population Direction", "0%"},
+            {"Mean Population Magnitude", "0%"},
+            {"Rolling Averaged Population Direction", "0%"},
+            {"Rolling Averaged Population Magnitude", "0%"},
+            {"OrderListHash", "1213851303"}
+        };
+    }
+}

Algorithm.CSharp/ExtendedMarketHoursHistoryRegressionAlgorithm.cs

@@ -0,0 +1,172 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using System.Linq;
+using QuantConnect.Data;
+using QuantConnect.Interfaces;
+using System.Collections.Generic;
+
+namespace QuantConnect.Algorithm.CSharp
+{
+    /// <summary>
+    /// Regression algorithm testing doing some history requests outside market hours, reproducing GH issue #4783
+    /// </summary>
+    public class ExtendedMarketHoursHistoryRegressionAlgorithm : QCAlgorithm, IRegressionAlgorithmDefinition
+    {
+        private int _minuteHistoryCount;
+        private int _hourHistoryCount;
+        private int _dailyHistoryCount;
+
+        /// <summary>
+        /// Initialise the data and resolution required, as well as the cash and start-end dates for your algorithm. All algorithms must initialized.
+        /// </summary>
+        public override void Initialize()
+        {
+            SetStartDate(2013, 10, 07);
+            SetEndDate(2013, 10, 09);
+            SetCash(100000);
+
+            AddEquity("SPY", Resolution.Minute, extendedMarketHours:true, fillDataForward:false);
+
+            Schedule.On("RunHistoryCall", DateRules.EveryDay(), TimeRules.Every(TimeSpan.FromHours(1)), RunHistoryCall);
+        }
+
+        private void RunHistoryCall()
+        {
+            var spy = Securities["SPY"];
+            var regularHours = spy.Exchange.Hours.IsOpen(Time, false);
+            var extendedHours = !regularHours && spy.Exchange.Hours.IsOpen(Time, true);
+
+            if (regularHours)
+            {
+                _minuteHistoryCount++;
+                var history = History(spy.Symbol, 5, Resolution.Minute).Count();
+                if (history != 5)
+                {
+                    throw new Exception($"Unexpected Minute data count: {history}");
+                }
+            }
+            else
+            {
+                if (extendedHours)
+                {
+                    _hourHistoryCount++;
+                    var history = History(spy.Symbol, 5, Resolution.Hour).Count();
+                    if (history != 5)
+                    {
+                        throw new Exception($"Unexpected Hour data count {history}");
+                    }
+                }
+                else
+                {
+                    _dailyHistoryCount++;
+                    var history = History(spy.Symbol, 5, Resolution.Daily).Count();
+                    if (history != 5)
+                    {
+                        throw new Exception($"Unexpected Daily data count {history}");
+                    }
+                }
+            }
+        }
+
+        /// <summary>
+        /// OnData event is the primary entry point for your algorithm. Each new data point will be pumped in here.
+        /// </summary>
+        /// <param name="data">Slice object keyed by symbol containing the stock data</param>
+        public override void OnData(Slice data)
+        {
+            if (!Portfolio.Invested)
+            {
+                SetHoldings("SPY", 1);
+            }
+        }
+
+        public override void OnEndOfAlgorithm()
+        {
+            if (_minuteHistoryCount != 3 * 6)
+            {
+                throw new Exception($"Unexpected minute history requests count {_minuteHistoryCount}");
+            }
+            // 6 pre market from 4am to 9am + 4 post market 4pm to 7pm
+            if (_hourHistoryCount != 3 * 10)
+            {
+                throw new Exception($"Unexpected hour history requests count {_hourHistoryCount}");
+            }
+            // 0am to 3am + 8pm to 11pm, last day ends at 8pm
+            if (_dailyHistoryCount != (2 * 8 + 5))
+            {
+                throw new Exception($"Unexpected Daily history requests count: {_dailyHistoryCount}");
+            }
+        }
+
+        /// <summary>
+        /// This is used by the regression test system to indicate if the open source Lean repository has the required data to run this algorithm.
+        /// </summary>
+        public bool CanRunLocally { get; } = true;
+
+        /// <summary>
+        /// This is used by the regression test system to indicate which languages this algorithm is written in.
+        /// </summary>
+        public Language[] Languages { get; } = { Language.CSharp };
+
+        /// <summary>
+        /// This is used by the regression test system to indicate what the expected statistics are from running the algorithm
+        /// </summary>
+        public Dictionary<string, string> ExpectedStatistics => new Dictionary<string, string>
+        {
+            {"Total Trades", "20"},
+            {"Average Win", "0%"},
+            {"Average Loss", "0.00%"},
+            {"Compounding Annual Return", "-74.182%"},
+            {"Drawdown", "2.200%"},
+            {"Expectancy", "-1"},
+            {"Net Profit", "-1.046%"},
+            {"Sharpe Ratio", "-8.269"},
+            {"Probabilistic Sharpe Ratio", "0%"},
+            {"Loss Rate", "100%"},
+            {"Win Rate", "0%"},
+            {"Profit-Loss Ratio", "0"},
+            {"Alpha", "-0.19"},
+            {"Beta", "0.579"},
+            {"Annual Standard Deviation", "0.065"},
+            {"Annual Variance", "0.004"},
+            {"Information Ratio", "1.326"},
+            {"Tracking Error", "0.049"},
+            {"Treynor Ratio", "-0.934"},
+            {"Total Fees", "$22.26"},
+            {"Fitness Score", "0.002"},
+            {"Kelly Criterion Estimate", "0"},
+            {"Kelly Criterion Probability Value", "0"},
+            {"Sortino Ratio", "-11.855"},
+            {"Return Over Maximum Drawdown", "-70.945"},
+            {"Portfolio Turnover", "0.342"},
+            {"Total Insights Generated", "0"},
+            {"Total Insights Closed", "0"},
+            {"Total Insights Analysis Completed", "0"},
+            {"Long Insight Count", "0"},
+            {"Short Insight Count", "0"},
+            {"Long/Short Ratio", "100%"},
+            {"Estimated Monthly Alpha Value", "$0"},
+            {"Total Accumulated Estimated Alpha Value", "$0"},
+            {"Mean Population Estimated Insight Value", "$0"},
+            {"Mean Population Direction", "0%"},
+            {"Mean Population Magnitude", "0%"},
+            {"Rolling Averaged Population Direction", "0%"},
+            {"Rolling Averaged Population Magnitude", "0%"},
+            {"OrderListHash", "-1961710414"}
+        };
+    }
+}

Algorithm.CSharp/ExtendedMarketTradingRegressionAlgorithm.cs

@@ -141,7 +141,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Mean Population Magnitude", "0%"},
             {"Rolling Averaged Population Direction", "0%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "699698796"}
+            {"OrderListHash", "1717552327"}
         };
     }
 }

Algorithm.CSharp/FeeModelNotUsingAccountCurrency.cs

@@ -178,7 +178,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Kelly Criterion Estimate", "0"},
             {"Kelly Criterion Probability Value", "0"},
             {"Sortino Ratio", "79228162514264337593543950335"},
-            {"Return Over Maximum Drawdown", "-15.573"},
+            {"Return Over Maximum Drawdown", "-15.574"},
             {"Portfolio Turnover", "2.056"},
             {"Total Insights Generated", "0"},
             {"Total Insights Closed", "0"},

Algorithm.CSharp/ForexInternalFeedOnDataHigherResolutionRegressionAlgorithm.cs

@@ -40,7 +40,7 @@ namespace QuantConnect.Algorithm.CSharp
             SetEndDate(2013, 10, 8);
             SetCash(100000);
 
-            _eurusd = QuantConnect.Symbol.Create("EURUSD", SecurityType.Forex, Market.FXCM);
+            _eurusd = QuantConnect.Symbol.Create("EURUSD", SecurityType.Forex, Market.Oanda);
             var eurgbp = AddForex("EURGBP", Resolution.Daily);
             _dataPointsPerSymbol.Add(eurgbp.Symbol, 0);
         }
@@ -94,7 +94,7 @@ namespace QuantConnect.Algorithm.CSharp
             var expectedDataPointsPerSymbol = new Dictionary<string, int>
             {
                 { "EURGBP", 3 },
-                { "EURUSD", 48 }
+                { "EURUSD", 29 }
             };
 
             foreach (var kvp in _dataPointsPerSymbol)
@@ -141,8 +141,8 @@ namespace QuantConnect.Algorithm.CSharp
             {"Beta", "0"},
             {"Annual Standard Deviation", "0"},
             {"Annual Variance", "0"},
-            {"Information Ratio", "5.893"},
-            {"Tracking Error", "0.131"},
+            {"Information Ratio", "5.853"},
+            {"Tracking Error", "0.107"},
             {"Treynor Ratio", "0"},
             {"Total Fees", "$0.00"},
             {"Fitness Score", "0"},

Algorithm.CSharp/ForexInternalFeedOnDataSameResolutionRegressionAlgorithm.cs

@@ -40,7 +40,7 @@ namespace QuantConnect.Algorithm.CSharp
             SetEndDate(2013, 10, 8);
             SetCash(100000);
 
-            _eurusd = QuantConnect.Symbol.Create("EURUSD", SecurityType.Forex, Market.FXCM);
+            _eurusd = QuantConnect.Symbol.Create("EURUSD", SecurityType.Forex, Market.Oanda);
             var eurgbp = AddForex("EURGBP", Resolution.Daily);
             _dataPointsPerSymbol.Add(eurgbp.Symbol, 0);
         }
@@ -145,8 +145,8 @@ namespace QuantConnect.Algorithm.CSharp
             {"Beta", "0"},
             {"Annual Standard Deviation", "0"},
             {"Annual Variance", "0"},
-            {"Information Ratio", "5.893"},
-            {"Tracking Error", "0.131"},
+            {"Information Ratio", "5.853"},
+            {"Tracking Error", "0.107"},
             {"Treynor Ratio", "0"},
             {"Total Fees", "$0.00"},
             {"Fitness Score", "0"},

Algorithm.CSharp/LimitFillRegressionAlgorithm.cs

@@ -119,7 +119,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Mean Population Magnitude", "0%"},
             {"Rolling Averaged Population Direction", "0%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "1459983342"}
+            {"OrderListHash", "187652813"}
         };
     }
 }

Algorithm.CSharp/OptionChainProviderAlgorithm.cs

@@ -21,7 +21,7 @@ using QuantConnect.Interfaces;
 namespace QuantConnect.Algorithm.CSharp
 {
     /// <summary>
-    /// Demonstration of the Option Chain Provider -- a much faster mechanism for manually specifying the option contracts you'd like to recieve
+    /// Demonstration of the Option Chain Provider -- a much faster mechanism for manually specifying the option contracts you'd like to receive
     /// data for and manually subscribing to them.
     /// </summary>
     /// <meta name="tag" content="strategy example" />

Algorithm.CSharp/OptionExerciseAssignRegressionAlgorithm.cs

@@ -161,7 +161,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Mean Population Magnitude", "0%"},
             {"Rolling Averaged Population Direction", "0%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "-1018168907"}
+            {"OrderListHash", "-1726463684"}
         };
     }
 }

Algorithm.CSharp/OptionOpenInterestRegressionAlgorithm.cs

@@ -16,7 +16,9 @@
 
 using System;
 using System.Collections.Generic;
+using System.Linq;
 using QuantConnect.Data;
+using QuantConnect.Data.Market;
 using QuantConnect.Orders;
 using QuantConnect.Interfaces;
 
@@ -60,11 +62,24 @@ namespace QuantConnect.Algorithm.CSharp
                             contract.Symbol.ID.OptionRight == OptionRight.Call &&
                             contract.Symbol.ID.Date == new DateTime(2016, 01, 15))
                         {
-                            if (slice.Time.Date == new DateTime(2014, 06, 05) && contract.OpenInterest != 50)
+                            var history = History<OpenInterest>(contract.Symbol, TimeSpan.FromDays(1)).ToList();
+                            if (history.Count == 0)
+                            {
+                                throw new Exception("Regression test failed: open interest history request is empty");
+                            }
+
+                            var security = Securities[contract.Symbol];
+                            var openInterestCache = security.Cache.GetData<OpenInterest>();
+                            if (openInterestCache == null)
+                            {
+                                throw new Exception("Regression test failed: current open interest isn't in the security cache");
+                            }
+
+                            if (slice.Time.Date == new DateTime(2014, 06, 05) && (contract.OpenInterest != 50 || security.OpenInterest != 50))
                             {
                                 throw new Exception("Regression test failed: current open interest was not correctly loaded and is not equal to 50");
                             }
-                            if (slice.Time.Date == new DateTime(2014, 06, 06) && contract.OpenInterest != 70)
+                            if (slice.Time.Date == new DateTime(2014, 06, 06) && (contract.OpenInterest != 70 || security.OpenInterest != 70))
                             {
                                 throw new Exception("Regression test failed: current open interest was not correctly loaded and is not equal to 70");
                             }

Algorithm.CSharp/OrderImmutabilityRegressionAlgorithm.cs

@@ -0,0 +1,188 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using QuantConnect.Data;
+using QuantConnect.Interfaces;
+using QuantConnect.Orders;
+
+namespace QuantConnect.Algorithm.CSharp
+{
+    /// <summary>
+    /// This regression algorithm tests that orders are unchangeable from the QCAlgorithm Layer
+    /// Orders should only be modifiable via their ticket and only in permitted ways
+    /// </summary>
+    /// <meta name="tag" content="backtesting brokerage" />
+    /// <meta name="tag" content="regression test" />
+    /// <meta name="tag" content="options" />
+    public class OrderImmutabilityRegressionAlgorithm : QCAlgorithm, IRegressionAlgorithmDefinition
+    {
+        private readonly Symbol _spy = QuantConnect.Symbol.Create("SPY", SecurityType.Equity, Market.USA);
+        private OrderTicket _ticket;
+        private Order _originalOrder;
+
+        /// <summary>
+        /// Initialise the data and resolution required, as well as the cash and start-end dates for your algorithm. All algorithms must initialized.
+        /// </summary>
+        public override void Initialize()
+        {
+            SetStartDate(2013, 10, 08);  //Set Start Date
+            SetEndDate(2013, 10, 09);    //Set End Date
+            SetCash(100000);             //Set Strategy Cash
+            AddEquity("SPY", Resolution.Daily);
+        }
+
+        /// <summary>
+        /// OnData event is the primary entry point for your algorithm. Each new data point will be pumped in here.
+        /// </summary>
+        /// <param name="data">Slice object keyed by symbol containing the stock data</param>
+        public override void OnData(Slice data)
+        {
+            if (!Portfolio.Invested)
+            {
+                _ticket = LimitOrder(_spy, 10, 100);
+                Debug("Purchased Stock");
+
+                // Here we will show how to correctly change an order, we will then verify at End of Algorithm!
+                // First get the order as it is now, should be a copy, so it wont be updated!
+                _originalOrder = Transactions.GetOrderById(_ticket.OrderId);
+
+                // Create an UpdateOrderRequest and send it to the ticket
+                var updateFields = new UpdateOrderFields { Quantity = 20, Tag = "Pepe", LimitPrice = data[_spy].Low};
+                var response = _ticket.Update(updateFields);
+
+                // Test order time
+                if (_originalOrder.Time != UtcTime)
+                {
+                    Error("Order Time should be UtcTime!");
+                    throw new Exception("Order Time should be UtcTime!");
+                }
+            }
+        }
+
+        /// <summary>
+        /// All order events get pushed through this function
+        /// This function will test that what we get from Transactions is indeed a clone
+        /// The only authentic way to change the order is to change through the order ticket!
+        /// </summary>
+        /// <param name="orderEvent">OrderEvent object that contains all the information about the event</param>
+        public override void OnOrderEvent(OrderEvent orderEvent)
+        {
+
+            // Get the order twice, since they are clones they should NOT be the same
+            var orderV1 = Transactions.GetOrderById(orderEvent.OrderId);
+            var orderV2 = Transactions.GetOrderById(orderEvent.OrderId);
+
+            if (orderV1 == orderV2)
+            {
+                Error("Orders should be clones, hence not equal!");
+                throw new Exception("Orders should be clones, hence not equal!");
+            }
+
+            // Try and manipulate orderV2 using the only external accessor BrokerID, since we
+            // are changing a clone the BrokerIDs should not be the same
+            orderV2.BrokerId.Add("FAKE BROKER ID");
+            var orderV3 = Transactions.GetOrderById(orderEvent.OrderId);
+
+            if (orderV2.BrokerId.SequenceEqual(orderV3.BrokerId))
+            {
+                Error("Broker IDs should not be the same!");
+                throw new Exception("Broker IDs should not be the same!");
+            }
+
+            //Try and manipulate the orderV1 using UpdateOrderRequest
+            //NOTICE: Orders should only be updated through their tickets!
+            var updateFields = new UpdateOrderFields { Quantity = 99, Tag = "Pepe2!" };
+            var updateRequest = new UpdateOrderRequest(DateTime.Now, orderEvent.OrderId, updateFields);
+            orderV1.ApplyUpdateOrderRequest(updateRequest);
+            var orderV4 = Transactions.GetOrderById(orderEvent.OrderId);
+
+            if (orderV4.Quantity == orderV1.Quantity)
+            {
+                Error("Order quantity should not be the same!");
+                throw new Exception("Order quantity should not be the same!");
+            }
+
+            if (orderV4.Tag == orderV1.Tag)
+            {
+                Error("Order tag should not be the same!");
+                throw new Exception("Order tag should not be the same!");
+            }
+        }
+
+        /// <summary>
+        /// Will run at End of Algorithm
+        /// We will be using this to check our order was updated!
+        /// </summary>
+        public override void OnEndOfAlgorithm()
+        {
+            //Get an updated copy of the order and compare to our original
+            var updatedOrder = Transactions.GetOrderById(_ticket.OrderId);
+
+            if (updatedOrder.Quantity == _originalOrder.Quantity)
+            {
+                Error("Quantities should have been updated!");
+                throw new Exception("Quantities should have been updated!");
+            }
+
+            if (updatedOrder.Tag == _originalOrder.Tag)
+            {
+                Error("Tag should have been updated!");
+                throw new Exception("Tag should have been updated!");
+            }
+        }
+
+        /// <summary>
+        /// This is used by the regression test system to indicate if the open source Lean repository has the required data to run this algorithm.
+        /// </summary>
+        public bool CanRunLocally { get; } = true;
+
+        /// <summary>
+        /// This is used by the regression test system to indicate which languages this algorithm is written in.
+        /// </summary>
+        public Language[] Languages { get; } = { Language.CSharp };
+
+        /// <summary>
+        /// This is used by the regression test system to indicate what the expected statistics are from running the algorithm
+        /// </summary>
+        public Dictionary<string, string> ExpectedStatistics => new Dictionary<string, string>
+        {
+            {"Total Trades", "1"},
+            {"Average Win", "0%"},
+            {"Average Loss", "0%"},
+            {"Compounding Annual Return", "-5.591%"},
+            {"Drawdown", "0.000%"},
+            {"Expectancy", "0"},
+            {"Net Profit", "-0.032%"},
+            {"Sharpe Ratio", "-9.862"},
+            {"Probabilistic Sharpe Ratio", "0%"},
+            {"Loss Rate", "0%"},
+            {"Win Rate", "0%"},
+            {"Profit-Loss Ratio", "0"},
+            {"Alpha", "0.007"},
+            {"Beta", "-0.582"},
+            {"Annual Standard Deviation", "0.004"},
+            {"Annual Variance", "0"},
+            {"Information Ratio", "-10.999"},
+            {"Tracking Error", "0.011"},
+            {"Treynor Ratio", "0.067"},
+            {"Total Fees", "$1.00"},
+            {"Fitness Score", "0.007"},
+            {"OrderListHash", "1715759777"}
+        };
+    }
+}

Algorithm.CSharp/OrderSubmissionDataRegressionAlgorithm.cs

@@ -86,12 +86,12 @@ namespace QuantConnect.Algorithm.CSharp
             {"Total Trades", "18"},
             {"Average Win", "0.88%"},
             {"Average Loss", "-0.95%"},
-            {"Compounding Annual Return", "292.584%"},
+            {"Compounding Annual Return", "292.522%"},
             {"Drawdown", "3.400%"},
             {"Expectancy", "0.204"},
             {"Net Profit", "1.780%"},
-            {"Sharpe Ratio", "11.819"},
-            {"Probabilistic Sharpe Ratio", "66.758%"},
+            {"Sharpe Ratio", "11.817"},
+            {"Probabilistic Sharpe Ratio", "66.756%"},
             {"Loss Rate", "38%"},
             {"Win Rate", "62%"},
             {"Profit-Loss Ratio", "0.93"},
@@ -99,15 +99,15 @@ namespace QuantConnect.Algorithm.CSharp
             {"Beta", "1.548"},
             {"Annual Standard Deviation", "0.34"},
             {"Annual Variance", "0.116"},
-            {"Information Ratio", "17.385"},
+            {"Information Ratio", "17.38"},
             {"Tracking Error", "0.12"},
-            {"Treynor Ratio", "2.597"},
+            {"Treynor Ratio", "2.596"},
             {"Total Fees", "$45.00"},
             {"Fitness Score", "0.986"},
             {"Kelly Criterion Estimate", "0"},
             {"Kelly Criterion Probability Value", "0"},
-            {"Sortino Ratio", "9.332"},
-            {"Return Over Maximum Drawdown", "45.085"},
+            {"Sortino Ratio", "9.326"},
+            {"Return Over Maximum Drawdown", "45.056"},
             {"Portfolio Turnover", "2.728"},
             {"Total Insights Generated", "0"},
             {"Total Insights Closed", "0"},
@@ -122,7 +122,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Mean Population Magnitude", "0%"},
             {"Rolling Averaged Population Direction", "0%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "-809947807"}
+            {"OrderListHash", "-46935513"}
         };
     }
 }

Algorithm.CSharp/OrderTicketDemoAlgorithm.cs

@@ -527,7 +527,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Mean Population Magnitude", "0%"},
             {"Rolling Averaged Population Direction", "0%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "1237222672"}
+            {"OrderListHash", "-1594146186"}
         };
     }
 }

Algorithm.CSharp/QuantConnect.Algorithm.CSharp.csproj

@@ -142,6 +142,11 @@
       <Link>Properties\SharedAssemblyInfo.cs</Link>
     </Compile>
     <Compile Include="AddAlphaModelAlgorithm.cs" />
+    <Compile Include="DailyHistoryForDailyResolutionRegressionAlgorithm.cs" />
+    <Compile Include="DailyHistoryForMinuteResolutionRegressionAlgorithm.cs" />
+    <Compile Include="ExtendedMarketHoursHistoryRegressionAlgorithm.cs" />
+    <Compile Include="EquityTickQuoteAdjustedModeRegressionAlgorithm.cs" />
+    <Compile Include="SwitchDataModeRegressionAlgorithm.cs" />
     <Compile Include="AddRemoveOptionUniverseRegressionAlgorithm.cs" />
     <Compile Include="AddRemoveSecurityRegressionAlgorithm.cs" />
     <Compile Include="AddRiskManagementAlgorithm.cs" />
@@ -167,6 +172,7 @@
     <Compile Include="AltData\TiingoNewsAlgorithm.cs" />
     <Compile Include="AutomaticIndicatorWarmupDataTypeRegressionAlgorithm.cs" />
     <Compile Include="AutomaticIndicatorWarmupRegressionAlgorithm.cs" />
+    <Compile Include="BacktestingBrokerageRegressionAlgorithm.cs" />
     <Compile Include="ExtendedMarketTradingRegressionAlgorithm.cs" />
     <Compile Include="CoarseTiingoNewsUniverseSelectionAlgorithm.cs" />
     <Compile Include="DelistedFutureLiquidateRegressionAlgorithm.cs" />
@@ -199,6 +205,7 @@
     <Compile Include="MarginRemainingRegressionAlgorithm.cs" />
     <Compile Include="NoMarginCallExpectedRegressionAlgorithm.cs" />
     <Compile Include="ObjectStoreExampleAlgorithm.cs" />
+    <Compile Include="OrderImmutabilityRegressionAlgorithm.cs" />
     <Compile Include="OrderSubmissionDataRegressionAlgorithm.cs" />
     <Compile Include="RegisterIndicatorRegressionAlgorithm.cs" />
     <Compile Include="ScheduledEventsOrderRegressionAlgorithm.cs" />
@@ -367,6 +374,7 @@
     <Compile Include="RegressionAlgorithm.cs" />
     <Compile Include="RenkoConsolidatorAlgorithm.cs" />
     <Compile Include="ScheduledEventsAlgorithm.cs" />
+    <Compile Include="ScheduledQueuingAlgorithm.cs" />
     <Compile Include="StressSymbolsAlgorithm.cs" />
     <Compile Include="StressSymbols.cs" />
     <Compile Include="TickDataFilteringAlgorithm.cs" />
@@ -426,6 +434,9 @@
     <Analyzer Include="..\packages\Microsoft.NetFramework.Analyzers.2.9.3\analyzers\dotnet\cs\Microsoft.NetFramework.Analyzers.dll" />
     <Analyzer Include="..\packages\Microsoft.NetFramework.Analyzers.2.9.3\analyzers\dotnet\cs\Microsoft.NetFramework.CSharp.Analyzers.dll" />
   </ItemGroup>
+  <ItemGroup>
+    <Compile Include="DaylightSavingTimeHistoryRegressionAlgorithm.cs" />
+  </ItemGroup>
   <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
   <Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">
     <PropertyGroup>
@@ -448,4 +459,4 @@
   <Target Name="AfterBuild">
   </Target>
   -->
-</Project>
+</Project>

Algorithm.CSharp/ScheduledQueuingAlgorithm.cs

@@ -0,0 +1,96 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using QuantConnect.Algorithm.Framework.Alphas;
+using QuantConnect.Algorithm.Framework.Execution;
+using QuantConnect.Algorithm.Framework.Portfolio;
+using QuantConnect.Algorithm.Framework.Selection;
+using QuantConnect.Data;
+using QuantConnect.Data.Fundamental;
+using QuantConnect.Data.UniverseSelection;
+using QuantConnect.Interfaces;
+using QuantConnect.Securities;
+
+namespace QuantConnect.Algorithm.CSharp
+{
+    public class TachyonDynamicGearbox : QCAlgorithm
+    {
+        private int numberOfSymbols;
+        private int numberOfSymbolsFine;
+        private Queue<Symbol> queue;
+        private int dequeueSize;
+        
+
+        public override void Initialize()
+        {
+            SetStartDate(2020, 9, 1);
+            SetEndDate(2020, 9, 2);
+            SetCash(100000);
+            
+            numberOfSymbols = 2000;
+            numberOfSymbolsFine = 1000;
+            SetUniverseSelection(new FineFundamentalUniverseSelectionModel(CoarseSelectionFunction, FineSelectionFunction));
+            
+            SetPortfolioConstruction(new EqualWeightingPortfolioConstructionModel());
+            
+            SetExecution(new ImmediateExecutionModel());
+            
+            queue = new Queue<Symbol>();
+            dequeueSize = 100;
+            
+            AddEquity("SPY", Resolution.Minute);
+            Schedule.On(DateRules.EveryDay("SPY"), TimeRules.At(0, 0), FillQueue);
+            Schedule.On(DateRules.EveryDay("SPY"), TimeRules.Every(TimeSpan.FromMinutes(60)), TakeFromQueue);
+        }
+
+        public IEnumerable<Symbol> CoarseSelectionFunction(IEnumerable<CoarseFundamental> coarse)
+        {
+            var sortedByDollarVolume = coarse
+                .Where(x => x.HasFundamentalData)
+                .OrderByDescending(x => x.DollarVolume);
+            return sortedByDollarVolume.Take(numberOfSymbols).Select(x => x.Symbol);
+        }
+        
+        public IEnumerable<Symbol> FineSelectionFunction(IEnumerable<FineFundamental> fine)
+        {
+            
+            var sortedByPeRatio = fine.OrderByDescending(x => x.ValuationRatios.PERatio);
+            var topFine = sortedByPeRatio.Take(numberOfSymbolsFine);
+            return topFine.Select(x => x.Symbol);
+        }
+        
+        private void FillQueue() {
+            var securities = ActiveSecurities.Values.Where(x => x.Fundamentals != null);
+            
+            // Fill queue with symbols sorted by PE ratio (decreasing order)
+            queue.Clear();
+            var sortedByPERatio = securities.OrderByDescending(x => x.Fundamentals.ValuationRatios.PERatio);
+            foreach (Security security in sortedByPERatio)
+                queue.Enqueue(security.Symbol);
+        }
+        
+        private void TakeFromQueue() {
+            List<Symbol> symbols = new List<Symbol>();
+            for (int i = 0; i < Math.Min(dequeueSize, queue.Count); i++)
+                symbols.Add(queue.Dequeue());
+            History(symbols, 10, Resolution.Daily);
+            
+            Log("Symbols at " + Time + ": " + string.Join(", ", symbols.Select(x => x.ToString())));
+        }
+    }
+}

Algorithm.CSharp/ScheduledUniverseSelectionModelRegressionAlgorithm.cs

@@ -67,16 +67,16 @@ namespace QuantConnect.Algorithm.CSharp
 
             if (dateTime.DayOfWeek == DayOfWeek.Tuesday || dateTime.DayOfWeek == DayOfWeek.Thursday)
             {
-                yield return QuantConnect.Symbol.Create("EURUSD", SecurityType.Forex, Market.FXCM);
+                yield return QuantConnect.Symbol.Create("EURUSD", SecurityType.Forex, Market.Oanda);
             }
             else if (dateTime.DayOfWeek == DayOfWeek.Friday)
             {
                 // given the date/time rules specified in Initialize, this symbol will never be selected (every 6 hours never lands on hour==1)
-                yield return QuantConnect.Symbol.Create("EURGBP", SecurityType.Forex, Market.FXCM);
+                yield return QuantConnect.Symbol.Create("EURGBP", SecurityType.Forex, Market.Oanda);
             }
             else
             {
-                yield return QuantConnect.Symbol.Create("NZDUSD", SecurityType.Forex, Market.FXCM);
+                yield return QuantConnect.Symbol.Create("NZDUSD", SecurityType.Forex, Market.Oanda);
             }
         }
 
@@ -192,46 +192,46 @@ namespace QuantConnect.Algorithm.CSharp
         /// </summary>
         public Dictionary<string, string> ExpectedStatistics => new Dictionary<string, string>
         {
-            {"Total Trades", "52"},
-            {"Average Win", "0.27%"},
-            {"Average Loss", "-0.22%"},
-            {"Compounding Annual Return", "41.076%"},
-            {"Drawdown", "1.000%"},
-            {"Expectancy", "0.618"},
-            {"Net Profit", "3.112%"},
-            {"Sharpe Ratio", "5.311"},
-            {"Probabilistic Sharpe Ratio", "90.919%"},
-            {"Loss Rate", "29%"},
-            {"Win Rate", "71%"},
-            {"Profit-Loss Ratio", "1.26"},
-            {"Alpha", "0.31"},
-            {"Beta", "0.054"},
-            {"Annual Standard Deviation", "0.06"},
-            {"Annual Variance", "0.004"},
-            {"Information Ratio", "1.79"},
-            {"Tracking Error", "0.079"},
-            {"Treynor Ratio", "5.952"},
-            {"Total Fees", "$36.83"},
-            {"Fitness Score", "0.67"},
-            {"Kelly Criterion Estimate", "25.099"},
-            {"Kelly Criterion Probability Value", "0.068"},
-            {"Sortino Ratio", "13.102"},
-            {"Return Over Maximum Drawdown", "55.759"},
-            {"Portfolio Turnover", "0.675"},
-            {"Total Insights Generated", "54"},
-            {"Total Insights Closed", "52"},
-            {"Total Insights Analysis Completed", "52"},
-            {"Long Insight Count", "54"},
+            {"Total Trades", "86"},
+            {"Average Win", "0.16%"},
+            {"Average Loss", "-0.10%"},
+            {"Compounding Annual Return", "51.162%"},
+            {"Drawdown", "1.100%"},
+            {"Expectancy", "0.793"},
+            {"Net Profit", "3.748%"},
+            {"Sharpe Ratio", "7.195"},
+            {"Probabilistic Sharpe Ratio", "99.177%"},
+            {"Loss Rate", "31%"},
+            {"Win Rate", "69%"},
+            {"Profit-Loss Ratio", "1.60"},
+            {"Alpha", "0.366"},
+            {"Beta", "0.161"},
+            {"Annual Standard Deviation", "0.055"},
+            {"Annual Variance", "0.003"},
+            {"Information Ratio", "3.061"},
+            {"Tracking Error", "0.07"},
+            {"Treynor Ratio", "2.443"},
+            {"Total Fees", "$33.96"},
+            {"Fitness Score", "0.75"},
+            {"Kelly Criterion Estimate", "23.91"},
+            {"Kelly Criterion Probability Value", "0.076"},
+            {"Sortino Ratio", "42.076"},
+            {"Return Over Maximum Drawdown", "129.046"},
+            {"Portfolio Turnover", "0.751"},
+            {"Total Insights Generated", "55"},
+            {"Total Insights Closed", "53"},
+            {"Total Insights Analysis Completed", "53"},
+            {"Long Insight Count", "55"},
             {"Short Insight Count", "0"},
             {"Long/Short Ratio", "100%"},
             {"Estimated Monthly Alpha Value", "$814596.0814"},
             {"Total Accumulated Estimated Alpha Value", "$888136.0054"},
-            {"Mean Population Estimated Insight Value", "$17079.5386"},
-            {"Mean Population Direction", "59.6154%"},
+            {"Mean Population Estimated Insight Value", "$16757.2831"},
+            {"Mean Population Direction", "58.4906%"},
             {"Mean Population Magnitude", "0%"},
-            {"Rolling Averaged Population Direction", "64.1791%"},
+            {"Rolling Averaged Population Direction", "55.0223%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "-1450725184"}
+            {"OrderListHash", "941404943"}
         };
     }
 }

Algorithm.CSharp/SetAccountCurrencyCashBuyingPowerModelRegressionAlgorithm.cs

@@ -259,7 +259,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Kelly Criterion Estimate", "0"},
             {"Kelly Criterion Probability Value", "0"},
             {"Sortino Ratio", "79228162514264337593543950335"},
-            {"Return Over Maximum Drawdown", "-142.421"},
+            {"Return Over Maximum Drawdown", "-141.917"},
             {"Portfolio Turnover", "2.001"},
             {"Total Insights Generated", "0"},
             {"Total Insights Closed", "0"},

Algorithm.CSharp/SetAccountCurrencySecurityMarginModelRegressionAlgorithm.cs

@@ -197,29 +197,29 @@ namespace QuantConnect.Algorithm.CSharp
             {"Total Trades", "6"},
             {"Average Win", "0.40%"},
             {"Average Loss", "-0.86%"},
-            {"Compounding Annual Return", "-17.124%"},
+            {"Compounding Annual Return", "-15.825%"},
             {"Drawdown", "1.100%"},
             {"Expectancy", "-0.266"},
-            {"Net Profit", "-0.464%"},
-            {"Sharpe Ratio", "-1.547"},
-            {"Probabilistic Sharpe Ratio", "33.672%"},
+            {"Net Profit", "-0.463%"},
+            {"Sharpe Ratio", "-1.475"},
+            {"Probabilistic Sharpe Ratio", "33.116%"},
             {"Loss Rate", "50%"},
             {"Win Rate", "50%"},
             {"Profit-Loss Ratio", "0.47"},
-            {"Alpha", "-0.21"},
-            {"Beta", "0.104"},
-            {"Annual Standard Deviation", "0.086"},
+            {"Alpha", "-0.196"},
+            {"Beta", "0.123"},
+            {"Annual Standard Deviation", "0.081"},
             {"Annual Variance", "0.007"},
-            {"Information Ratio", "-4.732"},
-            {"Tracking Error", "0.184"},
-            {"Treynor Ratio", "-1.286"},
-            {"Total Fees", "$12.97"},
+            {"Information Ratio", "-4.271"},
+            {"Tracking Error", "0.174"},
+            {"Treynor Ratio", "-0.972"},
+            {"Total Fees", "$12.99"},
             {"Fitness Score", "0.031"},
             {"Kelly Criterion Estimate", "0"},
             {"Kelly Criterion Probability Value", "0"},
-            {"Sortino Ratio", "-3.761"},
-            {"Return Over Maximum Drawdown", "-15.539"},
-            {"Portfolio Turnover", "0.499"},
+            {"Sortino Ratio", "-3.46"},
+            {"Return Over Maximum Drawdown", "-14.323"},
+            {"Portfolio Turnover", "0.445"},
             {"Total Insights Generated", "0"},
             {"Total Insights Closed", "0"},
             {"Total Insights Analysis Completed", "0"},
@@ -233,7 +233,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Mean Population Magnitude", "0%"},
             {"Rolling Averaged Population Direction", "0%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "-436429281"}
+            {"OrderListHash", "-304070777"}
         };
     }
 }

Algorithm.CSharp/SetCashOnDataRegressionAlgorithm.cs

@@ -66,7 +66,7 @@ namespace QuantConnect.Algorithm.CSharp
                 }
 
                 var eurUsdSubscription = SubscriptionManager.SubscriptionDataConfigService
-                    .GetSubscriptionDataConfigs(QuantConnect.Symbol.Create("EURUSD", SecurityType.Forex, Market.FXCM),
+                    .GetSubscriptionDataConfigs(QuantConnect.Symbol.Create("EURUSD", SecurityType.Forex, Market.Oanda),
                         includeInternalConfigs: true)
                     .Single();
                 if (!eurUsdSubscription.IsInternalFeed)
@@ -100,29 +100,29 @@ namespace QuantConnect.Algorithm.CSharp
             {"Total Trades", "1"},
             {"Average Win", "0%"},
             {"Average Loss", "0%"},
-            {"Compounding Annual Return", "17.116%"},
+            {"Compounding Annual Return", "16.445%"},
             {"Drawdown", "4.800%"},
             {"Expectancy", "0"},
             {"Net Profit", "0.913%"},
-            {"Sharpe Ratio", "0.93"},
-            {"Probabilistic Sharpe Ratio", "48.592%"},
+            {"Sharpe Ratio", "0.903"},
+            {"Probabilistic Sharpe Ratio", "48.314%"},
             {"Loss Rate", "0%"},
             {"Win Rate", "0%"},
             {"Profit-Loss Ratio", "0"},
-            {"Alpha", "0.119"},
-            {"Beta", "0.202"},
-            {"Annual Standard Deviation", "0.161"},
-            {"Annual Variance", "0.026"},
+            {"Alpha", "0.113"},
+            {"Beta", "0.203"},
+            {"Annual Standard Deviation", "0.156"},
+            {"Annual Variance", "0.024"},
             {"Information Ratio", "0.001"},
-            {"Tracking Error", "0.203"},
-            {"Treynor Ratio", "0.739"},
+            {"Tracking Error", "0.198"},
+            {"Treynor Ratio", "0.697"},
             {"Total Fees", "$2.60"},
-            {"Fitness Score", "0.044"},
+            {"Fitness Score", "0.041"},
             {"Kelly Criterion Estimate", "0"},
             {"Kelly Criterion Probability Value", "0"},
-            {"Sortino Ratio", "1.683"},
-            {"Return Over Maximum Drawdown", "3.545"},
-            {"Portfolio Turnover", "0.055"},
+            {"Sortino Ratio", "1.617"},
+            {"Return Over Maximum Drawdown", "3.406"},
+            {"Portfolio Turnover", "0.052"},
             {"Total Insights Generated", "0"},
             {"Total Insights Closed", "0"},
             {"Total Insights Analysis Completed", "0"},

Algorithm.CSharp/StressSymbols.cs

@@ -14,6 +14,8 @@
 */
 
 using System.Collections.Generic;
+using System.Linq;
+using QuantConnect.Securities;
 
 namespace QuantConnect.Algorithm.CSharp
 {
@@ -22,7 +24,10 @@ namespace QuantConnect.Algorithm.CSharp
         /// <summary>
         /// The forex symbols.
         /// </summary>
-        public static HashSet<string> ForexSymbols = new HashSet<string>(Currencies.CurrencyPairs);
+        public static HashSet<string> ForexSymbols = new HashSet<string>(SymbolPropertiesDatabase
+            .FromDataFolder()
+            .GetSymbolPropertiesList(Market.Oanda, SecurityType.Forex)
+            .Select(x => x.Key.Symbol));
 
         /// <summary>
         /// The stock symbols.

Algorithm.CSharp/SwitchDataModeRegressionAlgorithm.cs

@@ -0,0 +1,142 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+*/
+
+using QuantConnect.Data;
+using QuantConnect.Interfaces;
+using System;
+using System.Collections.Generic;
+using System.Linq;
+
+
+namespace QuantConnect.Algorithm.CSharp
+{
+    /// <summary>
+    /// This regression test algorithm reproduces issue https://github.com/QuantConnect/Lean/issues/4031 
+    /// fixed in PR https://github.com/QuantConnect/Lean/pull/4650
+    /// Adjusted data have already been all loaded by the workers so DataNormalizationMode change has no effect in the data itself
+    /// </summary>
+    public class SwitchDataModeRegressionAlgorithm : QCAlgorithm, IRegressionAlgorithmDefinition
+    {
+        private const string UnderlyingTicker = "AAPL";
+
+        private readonly Dictionary<DateTime, decimal?> _expectedCloseValues = new Dictionary<DateTime, decimal?>() {
+            { new DateTime(2014, 6, 6, 9, 57, 0), 86.04398m},
+            { new DateTime(2014, 6, 6, 9, 58, 0), 86.05196m},
+            { new DateTime(2014, 6, 6, 9, 59, 0), 648.29m},
+            { new DateTime(2014, 6, 6, 10, 0, 0), 647.86m},
+            { new DateTime(2014, 6, 6, 10, 1, 0), 646.84m},
+            { new DateTime(2014, 6, 6, 10, 2, 0), 647.64m},
+            { new DateTime(2014, 6, 6, 10, 3, 0), 646.9m}
+        };
+
+        public override void Initialize()
+        {
+            SetStartDate(2014, 6, 6);
+            SetEndDate(2014, 6, 6);
+
+            var aapl = AddEquity(UnderlyingTicker, Resolution.Minute);
+        }
+
+        public override void OnData(Slice data)
+        {
+            if (Time.Hour == 9 && Time.Minute == 58)
+            {
+                AddOption(UnderlyingTicker);
+            }
+
+            AssertValue(data);
+        }
+
+        public override void OnEndOfAlgorithm()
+        {
+            if (_expectedCloseValues.Count > 0)
+            {
+                throw new Exception($"Not all expected data points were recieved.");
+            }
+        }
+
+        private void AssertValue(Slice data)
+        {
+            decimal? value;
+            if (_expectedCloseValues.TryGetValue(data.Time, out value))
+            {
+                if (data.Bars.FirstOrDefault().Value?.Close.SmartRounding() != value)
+                {
+                    throw new Exception($"Expected tradebar price, expected {value} but was {data.Bars.First().Value.Close.SmartRounding()}");
+                }
+
+                _expectedCloseValues.Remove(data.Time);
+            }
+        }
+
+        /// <summary>
+        /// This is used by the regression test system to indicate if the open source Lean repository has the required data to run this algorithm.
+        /// </summary>
+        public bool CanRunLocally { get; } = true;
+
+        /// <summary>
+        /// This is used by the regression test system to indicate which languages this algorithm is written in.
+        /// </summary>
+        public Language[] Languages { get; } = { Language.CSharp };
+
+        /// <summary>
+        /// This is used by the regression test system to indicate what the expected statistics are from running the algorithm
+        /// </summary>
+        public Dictionary<string, string> ExpectedStatistics => new Dictionary<string, string>
+        {
+            {"Total Trades", "0"},
+            {"Average Win", "0%"},
+            {"Average Loss", "0%"},
+            {"Compounding Annual Return", "0%"},
+            {"Drawdown", "0%"},
+            {"Expectancy", "0"},
+            {"Net Profit", "0%"},
+            {"Sharpe Ratio", "0"},
+            {"Probabilistic Sharpe Ratio", "0%"},
+            {"Loss Rate", "0%"},
+            {"Win Rate", "0%"},
+            {"Profit-Loss Ratio", "0"},
+            {"Alpha", "0"},
+            {"Beta", "0"},
+            {"Annual Standard Deviation", "0"},
+            {"Annual Variance", "0"},
+            {"Information Ratio", "0"},
+            {"Tracking Error", "0"},
+            {"Treynor Ratio", "0"},
+            {"Total Fees", "$0.00"},
+            {"Fitness Score", "0"},
+            {"Kelly Criterion Estimate", "0"},
+            {"Kelly Criterion Probability Value", "0"},
+            {"Sortino Ratio", "0"},
+            {"Return Over Maximum Drawdown", "0"},
+            {"Portfolio Turnover", "0"},
+            {"Total Insights Generated", "0"},
+            {"Total Insights Closed", "0"},
+            {"Total Insights Analysis Completed", "0"},
+            {"Long Insight Count", "0"},
+            {"Short Insight Count", "0"},
+            {"Long/Short Ratio", "100%"},
+            {"Estimated Monthly Alpha Value", "$0"},
+            {"Total Accumulated Estimated Alpha Value", "$0"},
+            {"Mean Population Estimated Insight Value", "$0"},
+            {"Mean Population Direction", "0%"},
+            {"Mean Population Magnitude", "0%"},
+            {"Rolling Averaged Population Direction", "0%"},
+            {"Rolling Averaged Population Magnitude", "0%"},
+            {"OrderListHash", "371857150"}
+        };
+    }
+}

Algorithm.CSharp/TimeInForceAlgorithm.cs

@@ -189,7 +189,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Mean Population Magnitude", "0%"},
             {"Rolling Averaged Population Direction", "0%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "-569072921"}
+            {"OrderListHash", "359885308"}
         };
     }
 }

Algorithm.CSharp/UniverseSharingSecurityDifferentSubscriptionRequestRegressionAlgorithm.cs

@@ -152,8 +152,8 @@ namespace QuantConnect.Algorithm.CSharp
             {"Beta", "0"},
             {"Annual Standard Deviation", "0"},
             {"Annual Variance", "0"},
-            {"Information Ratio", "-31.646"},
-            {"Tracking Error", "0.16"},
+            {"Information Ratio", "-58.133"},
+            {"Tracking Error", "0.173"},
             {"Treynor Ratio", "0"},
             {"Total Fees", "$0.00"},
             {"Fitness Score", "0"},

Algorithm.CSharp/UpdateOrderRegressionAlgorithm.cs

@@ -233,7 +233,7 @@ namespace QuantConnect.Algorithm.CSharp
             {"Mean Population Magnitude", "0%"},
             {"Rolling Averaged Population Direction", "0%"},
             {"Rolling Averaged Population Magnitude", "0%"},
-            {"OrderListHash", "124750474"}
+            {"OrderListHash", "1536869386"}
         };
     }
 }

Algorithm.Framework/Selection/OpenInterestFutureUniverseSelectionModel.cs

@@ -61,7 +61,7 @@ namespace QuantConnect.Algorithm.Framework.Selection
         /// </summary>
         protected override FutureFilterUniverse Filter(FutureFilterUniverse filter)
         {
-            return filter.Contracts(FilterByOpenInterest(filter.ToDictionary(x => x, x => _marketHoursDatabase.GetEntry(x.ID.Market, x, x.ID.SecurityType).ExchangeHours)));
+            return filter.Contracts(FilterByOpenInterest(filter.ToDictionary(x => x, x => _marketHoursDatabase.GetEntry(x.ID.Market, x, x.ID.SecurityType))));
         }
 
         /// <summary>
@@ -69,7 +69,7 @@ namespace QuantConnect.Algorithm.Framework.Selection
         /// </summary>
         /// <param name="contracts">Contracts to filter</param>
         /// <returns>Filtered set</returns>
-        public IEnumerable<Symbol> FilterByOpenInterest(IReadOnlyDictionary<Symbol, SecurityExchangeHours> contracts)
+        public IEnumerable<Symbol> FilterByOpenInterest(IReadOnlyDictionary<Symbol, MarketHoursDatabase.Entry> contracts)
         {
             var symbols = new List<Symbol>(_chainContractsLookupLimit.HasValue ? contracts.Keys.OrderBy(x => x.ID.Date).Take(_chainContractsLookupLimit.Value) : contracts.Keys);
             var openInterest = symbols.GroupBy(x => contracts[x]).SelectMany(g => GetOpenInterest(g.Key, g.Select(i => i))).ToDictionary(x => x.Key, x => x.Value);
@@ -91,11 +91,12 @@ namespace QuantConnect.Algorithm.Framework.Selection
             return filtered;
         }
 
-        private Dictionary<Symbol, decimal> GetOpenInterest(SecurityExchangeHours exchangeHours, IEnumerable<Symbol> symbols)
+        private Dictionary<Symbol, decimal> GetOpenInterest(MarketHoursDatabase.Entry marketHours, IEnumerable<Symbol> symbols)
         {
             var current = _algorithm.UtcTime;
+            var exchangeHours = marketHours.ExchangeHours;
             var endTime = Instant.FromDateTimeUtc(_algorithm.UtcTime).InZone(exchangeHours.TimeZone).ToDateTimeUnspecified();
-            var previousDay = Time.GetStartTimeForTradeBars(exchangeHours, endTime, Time.OneDay, 1, true);
+            var previousDay = Time.GetStartTimeForTradeBars(exchangeHours, endTime, Time.OneDay, 1, true, marketHours.DataTimeZone);
             var requests = symbols.Select(
                     symbol => new HistoryRequest(
                         previousDay,

Algorithm.Python/ConsolidateRegressionAlgorithm.py

@@ -22,7 +22,6 @@ from QuantConnect import *
 from QuantConnect.Algorithm import *
 from QuantConnect.Indicators import *
 from QuantConnect.Securities import *
-from QuantConnect.Data.Market import *
 from QuantConnect.Data.Consolidators import *
 from CustomDataRegressionAlgorithm import Bitcoin
 from datetime import timedelta

Algorithm.Python/CustomConsolidatorRegressionAlgorithm.py

@@ -13,19 +13,18 @@
 
 from clr import AddReference
 AddReference("System")
+AddReference("QuantConnect.Common")
 AddReference("QuantConnect.Algorithm")
 AddReference("QuantConnect.Indicators")
-AddReference("QuantConnect.Common")
 
-from System import *
 from QuantConnect import *
 from QuantConnect.Python import *
 from QuantConnect.Algorithm import *
-from QuantConnect.Algorithm.Framework.Selection import *
 from QuantConnect.Data import *
+from QuantConnect.Data.Market import *
 from QuantConnect.Data.Consolidators import *
 from QuantConnect.Indicators import *
-from QuantConnect.Data.Market import *
+from System import *
 from datetime import *
 
 class CustomConsolidatorRegressionAlgorithm(QCAlgorithm):

Algorithm.Python/CustomDataAddDataCoarseSelectionRegressionAlgorithm.py

@@ -21,6 +21,7 @@ from QuantConnect import *
 from QuantConnect.Algorithm import *
 from QuantConnect.Algorithm.Framework.Selection import *
 from QuantConnect.Data import *
+from QuantConnect.Data.Custom import *
 from QuantConnect.Data.Custom.SEC import *
 from QuantConnect.Data.UniverseSelection import *
 

Algorithm.Python/InceptionDateSelectionRegressionAlgorithm.py

@@ -12,15 +12,15 @@
 # limitations under the License.
 
 from clr import AddReference
-AddReference("System.Core")
-AddReference("QuantConnect.Common")
+AddReference("System")
 AddReference("QuantConnect.Algorithm")
-AddReference("QuantConnect.Algorithm.Framework")
+AddReference("QuantConnect.Common")
 
 from System import *
 from QuantConnect import *
-from QuantConnect.Algorithm import QCAlgorithm
+from QuantConnect.Algorithm import *
 from QuantConnect.Algorithm.Framework.Selection import *
+from QuantConnect.Data import *
 from QuantConnect.Data.UniverseSelection import *
 from datetime import timedelta
 

Algorithm.Python/LongOnlyAlphaStreamAlgorithm.py

@@ -14,7 +14,6 @@
 from clr import AddReference
 AddReference("System")
 AddReference("QuantConnect.Algorithm")
-AddReference("QuantConnect.Algorithm.Framework")
 AddReference("QuantConnect.Common")
 
 from QuantConnect import *
@@ -24,7 +23,6 @@ from QuantConnect.Algorithm.Framework.Execution import *
 from QuantConnect.Algorithm.Framework.Portfolio import *
 from QuantConnect.Algorithm.Framework.Selection import *
 from QuantConnect.Brokerages import *
-from QuantConnect.Data import *
 from QuantConnect.Interfaces import *
 from QuantConnect.Orders import *
 from System import *

Algorithm.Python/OptionOpenInterestRegressionAlgorithm.py

@@ -19,6 +19,7 @@ AddReference("QuantConnect.Common")
 from System import *
 from QuantConnect import *
 from QuantConnect.Algorithm import *
+from QuantConnect.Data.Market import *
 from datetime import datetime, timedelta
 
 ### <summary>
@@ -48,9 +49,18 @@ class OptionOpenInterestRegressionAlgorithm(QCAlgorithm):
                     if float(contract.Symbol.ID.StrikePrice) == 72.5 and \
                        contract.Symbol.ID.OptionRight == OptionRight.Call and \
                        contract.Symbol.ID.Date == datetime(2016, 1, 15):
-                        if slice.Time.date() == datetime(2014, 6, 5).date() and contract.OpenInterest != 50:
+
+                        history = self.History(contract.Symbol, timedelta(1))["openinterest"]
+                        if len(history.index) == 0 or 0 in history.values:
+                            raise ValueError("Regression test failed: open interest history request is empty")
+
+                        security = self.Securities[contract.Symbol]
+                        openInterestCache = security.Cache.GetData[OpenInterest]()
+                        if openInterestCache == None:
+                            raise ValueError("Regression test failed: current open interest isn't in the security cache")
+                        if slice.Time.date() == datetime(2014, 6, 5).date() and (contract.OpenInterest != 50 or security.OpenInterest != 50):
                             raise ValueError("Regression test failed: current open interest was not correctly loaded and is not equal to 50")  
-                        if slice.Time.date() == datetime(2014, 6, 6).date() and contract.OpenInterest != 70:
+                        if slice.Time.date() == datetime(2014, 6, 6).date() and (contract.OpenInterest != 70 or security.OpenInterest != 70):
                             raise ValueError("Regression test failed: current open interest was not correctly loaded and is not equal to 70")  
                         if slice.Time.date() == datetime(2014, 6, 6).date():
                             self.MarketOrder(contract.Symbol, 1)

Algorithm.Python/QuantConnect.Algorithm.Python.csproj

@@ -227,6 +227,7 @@
     <None Include="RenkoConsolidatorAlgorithm.py" />
     <None Include="RollingWindowAlgorithm.py" />
     <None Include="ScheduledEventsAlgorithm.py" />
+    <None Include="ScheduledQueuingAlgorithm.py" />
     <None Include="ScheduledUniverseSelectionModelRegressionAlgorithm.py" />
     <None Include="SectorExposureRiskFrameworkAlgorithm.py" />
     <None Include="StandardDeviationExecutionModelRegressionAlgorithm.py" />

Algorithm.Python/ScheduledQueuingAlgorithm.py

@@ -0,0 +1,76 @@
+# QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+# Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from clr import AddReference
+AddReference("System")
+AddReference("QuantConnect.Algorithm")
+AddReference("QuantConnect.Algorithm.Framework")
+AddReference("QuantConnect.Common")
+
+from System import *
+from QuantConnect import *
+from QuantConnect.Orders import *
+from QuantConnect.Algorithm import *
+from QuantConnect.Algorithm.Framework import *
+from QuantConnect.Algorithm.Framework.Alphas import *
+from QuantConnect.Algorithm.Framework.Execution import *
+from QuantConnect.Algorithm.Framework.Portfolio import *
+from QuantConnect.Algorithm.Framework.Selection import *
+
+from queue import Queue
+
+class ScheduledQueuingAlgorithm(QCAlgorithm):
+
+    def Initialize(self):
+        self.SetStartDate(2020, 9, 1)
+        self.SetEndDate(2020, 9, 2)
+        self.SetCash(100000)
+        
+        self.__numberOfSymbols = 2000
+        self.__numberOfSymbolsFine = 1000
+        self.SetUniverseSelection(FineFundamentalUniverseSelectionModel(self.CoarseSelectionFunction, self.FineSelectionFunction, None, None))
+        
+        self.SetPortfolioConstruction(EqualWeightingPortfolioConstructionModel())
+        
+        self.SetExecution(ImmediateExecutionModel())
+        
+        self.queue = Queue()
+        self.dequeue_size = 100
+        
+        self.AddEquity("SPY", Resolution.Minute)
+        self.Schedule.On(self.DateRules.EveryDay("SPY"), self.TimeRules.At(0, 0), self.FillQueue)
+        self.Schedule.On(self.DateRules.EveryDay("SPY"), self.TimeRules.Every(timedelta(minutes=60)), self.TakeFromQueue)
+
+    def CoarseSelectionFunction(self, coarse):
+        has_fundamentals = [security for security in coarse if security.HasFundamentalData]
+        sorted_by_dollar_volume = sorted(has_fundamentals, key=lambda x: x.DollarVolume, reverse=True)
+        return [ x.Symbol for x in sorted_by_dollar_volume[:self.__numberOfSymbols] ]
+    
+    def FineSelectionFunction(self, fine):
+        sorted_by_pe_ratio = sorted(fine, key=lambda x: x.ValuationRatios.PERatio, reverse=True)
+        return [ x.Symbol for x in sorted_by_pe_ratio[:self.__numberOfSymbolsFine] ]
+        
+    def FillQueue(self):
+        securities = [security for security in self.ActiveSecurities.Values if security.Fundamentals is not None]
+        
+        # Fill queue with symbols sorted by PE ratio (decreasing order)
+        self.queue.queue.clear()
+        sorted_by_pe_ratio = sorted(securities, key=lambda x: x.Fundamentals.ValuationRatios.PERatio, reverse=True)
+        for security in sorted_by_pe_ratio:
+            self.queue.put(security.Symbol)
+        
+    def TakeFromQueue(self):
+        symbols = [self.queue.get() for _ in range(min(self.dequeue_size, self.queue.qsize()))]
+        self.History(symbols, 10, Resolution.Daily)
+        
+        self.Log(f"Symbols at {self.Time}: {[str(symbol) for symbol in symbols]}")

Algorithm.Python/ScheduledUniverseSelectionModelRegressionAlgorithm.py

@@ -66,12 +66,12 @@ class ScheduledUniverseSelectionModelRegressionAlgorithm(QCAlgorithm):
             symbols.append(Symbol.Create('IBM', SecurityType.Equity, Market.USA))
 
         if weekday == 1 or weekday == 3:
-            symbols.append(Symbol.Create('EURUSD', SecurityType.Forex, Market.FXCM))
+            symbols.append(Symbol.Create('EURUSD', SecurityType.Forex, Market.Oanda))
         elif weekday == 4:
             # given the date/time rules specified in Initialize, this symbol will never be selected (every 6 hours never lands on hour==1)
-            symbols.append(Symbol.Create('EURGBP', SecurityType.Forex, Market.FXCM))
+            symbols.append(Symbol.Create('EURGBP', SecurityType.Forex, Market.Oanda))
         else:
-            symbols.append(Symbol.Create('NZDUSD', SecurityType.Forex, Market.FXCM))
+            symbols.append(Symbol.Create('NZDUSD', SecurityType.Forex, Market.Oanda))
 
         return symbols
 

Algorithm/QCAlgorithm.History.cs

@@ -250,8 +250,8 @@ namespace QuantConnect.Algorithm
 
                 var exchange = GetExchangeHours(x);
                 var res = GetResolution(x, resolution);
-                var start = _historyRequestFactory.GetStartTimeAlgoTz(x, periods, res, exchange);
-                return _historyRequestFactory.CreateHistoryRequest(config, start, Time.RoundDown(res.ToTimeSpan()), exchange, res);
+                var start = _historyRequestFactory.GetStartTimeAlgoTz(x, periods, res, exchange, config.DataTimeZone);
+                return _historyRequestFactory.CreateHistoryRequest(config, start, Time, exchange, res);
             });
 
             return History(requests.Where(x => x != null)).Get<T>().Memoize();
@@ -307,9 +307,10 @@ namespace QuantConnect.Algorithm
             if (symbol == null) throw new ArgumentException(_symbolEmptyErrorMessage);
 
             resolution = GetResolution(symbol, resolution);
-            var start = _historyRequestFactory.GetStartTimeAlgoTz(symbol, periods, resolution.Value, GetExchangeHours(symbol));
+            var marketHours = GetMarketHours(symbol);
+            var start = _historyRequestFactory.GetStartTimeAlgoTz(symbol, periods, resolution.Value, marketHours.ExchangeHours, marketHours.DataTimeZone);
 
-            return History(symbol, start, Time.RoundDown(resolution.Value.ToTimeSpan()), resolution);
+            return History(symbol, start, Time, resolution);
         }
 
         /// <summary>
@@ -337,8 +338,8 @@ namespace QuantConnect.Algorithm
             }
 
             resolution = GetResolution(symbol, resolution);
-            var start = _historyRequestFactory.GetStartTimeAlgoTz(symbol, periods, resolution.Value, GetExchangeHours(symbol));
-            return History<T>(symbol, start, Time.RoundDown(resolution.Value.ToTimeSpan()), resolution).Memoize();
+            var start = _historyRequestFactory.GetStartTimeAlgoTz(symbol, periods, resolution.Value, GetExchangeHours(symbol), config.DataTimeZone);
+            return History<T>(symbol, start, Time, resolution).Memoize();
         }
 
         /// <summary>
@@ -524,7 +525,7 @@ namespace QuantConnect.Algorithm
             Func<int, BaseData> getLastKnownPriceForPeriods = backwardsPeriods =>
             {
                 var startTimeUtc = _historyRequestFactory
-                    .GetStartTimeAlgoTz(security.Symbol, backwardsPeriods, resolution, security.Exchange.Hours)
+                    .GetStartTimeAlgoTz(security.Symbol, backwardsPeriods, resolution, security.Exchange.Hours, dataTimeZone)
                     .ConvertToUtc(_localTimeKeeper.TimeZone);
 
                 var request = new HistoryRequest(
@@ -610,7 +611,7 @@ namespace QuantConnect.Algorithm
 
                     // apply overrides
                     var res = GetResolution(x, resolution);
-                    if (fillForward.HasValue) request.FillForwardResolution = fillForward.Value ? res : (Resolution?) null;
+                    if (fillForward.HasValue) request.FillForwardResolution = fillForward.Value ? res : (Resolution?)null;
                     if (extendedMarket.HasValue) request.IncludeExtendedMarketHours = extendedMarket.Value;
 
                     requests.Add(request);
@@ -629,11 +630,16 @@ namespace QuantConnect.Algorithm
             {
                 var res = GetResolution(x, resolution);
                 var exchange = GetExchangeHours(x);
-                var start = _historyRequestFactory.GetStartTimeAlgoTz(x, periods, res, exchange);
-                var end = Time.RoundDown(res.ToTimeSpan());
+                var configs = GetMatchingSubscriptions(x, typeof(BaseData), resolution).ToList();
+                if (!configs.Any())
+                {
+                    return Enumerable.Empty<HistoryRequest>();
+                }
 
-                return GetMatchingSubscriptions(x, typeof(BaseData), resolution)
-                    .Select(config => _historyRequestFactory.CreateHistoryRequest(config, start, end, exchange, res));
+                var start = _historyRequestFactory.GetStartTimeAlgoTz(x, periods, res, exchange, configs.First().DataTimeZone);
+                var end = Time;
+
+                return configs.Select(config => _historyRequestFactory.CreateHistoryRequest(config, start, end, exchange, res));
             });
         }
 
@@ -690,13 +696,21 @@ namespace QuantConnect.Algorithm
 
         private SecurityExchangeHours GetExchangeHours(Symbol symbol)
         {
+            return GetMarketHours(symbol).ExchangeHours;
+        }
+
+        private MarketHoursDatabase.Entry GetMarketHours(Symbol symbol)
+        {
+            var hoursEntry = MarketHoursDatabase.GetEntry(symbol.ID.Market, symbol, symbol.ID.SecurityType);
+
+            // user can override the exchange hours in algorithm, i.e. HistoryAlgorithm
             Security security;
             if (Securities.TryGetValue(symbol, out security))
             {
-                return security.Exchange.Hours;
+                return new MarketHoursDatabase.Entry(hoursEntry.DataTimeZone, security.Exchange.Hours);
             }
 
-            return MarketHoursDatabase.GetEntry(symbol.ID.Market, symbol, symbol.ID.SecurityType).ExchangeHours;
+            return hoursEntry;
         }
 
         private Resolution GetResolution(Symbol symbol, Resolution? resolution)

Algorithm/QCAlgorithm.Python.cs

@@ -779,8 +779,8 @@ namespace QuantConnect.Algorithm
 
                 var res = GetResolution(x, resolution);
                 var exchange = GetExchangeHours(x);
-                var start = _historyRequestFactory.GetStartTimeAlgoTz(x, periods, res, exchange);
-                return _historyRequestFactory.CreateHistoryRequest(config, start, Time.RoundDown(res.ToTimeSpan()), exchange, res);
+                var start = _historyRequestFactory.GetStartTimeAlgoTz(x, periods, res, exchange, config.DataTimeZone);
+                return _historyRequestFactory.CreateHistoryRequest(config, start, Time, exchange, res);
             });
 
             return PandasConverter.GetDataFrame(History(requests.Where(x => x != null)).Memoize());
@@ -841,9 +841,9 @@ namespace QuantConnect.Algorithm
             if (resolution == Resolution.Tick) throw new ArgumentException("History functions that accept a 'periods' parameter can not be used with Resolution.Tick");
 
             var res = GetResolution(symbol, resolution);
-            var start = _historyRequestFactory.GetStartTimeAlgoTz(symbol, periods, res, GetExchangeHours(symbol));
-            var end = Time.RoundDown(res.ToTimeSpan());
-            return History(type, symbol, start, end, resolution);
+            var marketHours = GetMarketHours(symbol);
+            var start = _historyRequestFactory.GetStartTimeAlgoTz(symbol, periods, res, marketHours.ExchangeHours, marketHours.DataTimeZone);
+            return History(type, symbol, start, Time, resolution);
         }
 
         /// <summary>

Algorithm/QCAlgorithm.Trading.cs

@@ -54,7 +54,7 @@ namespace QuantConnect.Algorithm
         /// <seealso cref="Buy(Symbol, decimal)"/>
         public OrderTicket Buy(Symbol symbol, double quantity)
         {
-            return Order(symbol, (decimal)Math.Abs(quantity));
+            return Order(symbol, Math.Abs(quantity).SafeDecimalCast());
         }
 
         /// <summary>
@@ -99,7 +99,7 @@ namespace QuantConnect.Algorithm
         /// <returns>int Order Id.</returns>
         public OrderTicket Sell(Symbol symbol, double quantity)
         {
-            return Order(symbol, (decimal)Math.Abs(quantity) * -1);
+            return Order(symbol, Math.Abs(quantity).SafeDecimalCast() * -1m);
         }
 
         /// <summary>
@@ -130,7 +130,7 @@ namespace QuantConnect.Algorithm
         /// <seealso cref="Order(Symbol, decimal)"/>
         public OrderTicket Order(Symbol symbol, double quantity)
         {
-            return Order(symbol, (decimal)quantity);
+            return Order(symbol, quantity.SafeDecimalCast());
         }
 
         /// <summary>
@@ -156,7 +156,7 @@ namespace QuantConnect.Algorithm
         /// </summary>
         /// <param name="symbol">Symbol of the MarketType Required.</param>
         /// <param name="quantity">Number of shares to request.</param>
-        /// <param name="asynchronous">Send the order asynchrously (false). Otherwise we'll block until it fills</param>
+        /// <param name="asynchronous">Send the order asynchronously (false). Otherwise we'll block until it fills</param>
         /// <param name="tag">Place a custom order property or tag (e.g. indicator data).</param>
         /// <seealso cref="MarketOrder(Symbol, decimal, bool, string)"/>
         public OrderTicket Order(Symbol symbol, decimal quantity, bool asynchronous = false, string tag = "")
@@ -169,7 +169,7 @@ namespace QuantConnect.Algorithm
         /// </summary>
         /// <param name="symbol">Symbol of the MarketType Required.</param>
         /// <param name="quantity">Number of shares to request.</param>
-        /// <param name="asynchronous">Send the order asynchrously (false). Otherwise we'll block until it fills</param>
+        /// <param name="asynchronous">Send the order asynchronously (false). Otherwise we'll block until it fills</param>
         /// <param name="tag">Place a custom order property or tag (e.g. indicator data).</param>
         /// <returns>int Order id</returns>
         public OrderTicket MarketOrder(Symbol symbol, int quantity, bool asynchronous = false, string tag = "")
@@ -182,12 +182,12 @@ namespace QuantConnect.Algorithm
         /// </summary>
         /// <param name="symbol">Symbol of the MarketType Required.</param>
         /// <param name="quantity">Number of shares to request.</param>
-        /// <param name="asynchronous">Send the order asynchrously (false). Otherwise we'll block until it fills</param>
+        /// <param name="asynchronous">Send the order asynchronously (false). Otherwise we'll block until it fills</param>
         /// <param name="tag">Place a custom order property or tag (e.g. indicator data).</param>
         /// <returns>int Order id</returns>
         public OrderTicket MarketOrder(Symbol symbol, double quantity, bool asynchronous = false, string tag = "")
         {
-            return MarketOrder(symbol, (decimal)quantity, asynchronous, tag);
+            return MarketOrder(symbol, quantity.SafeDecimalCast(), asynchronous, tag);
         }
 
         /// <summary>
@@ -195,7 +195,7 @@ namespace QuantConnect.Algorithm
         /// </summary>
         /// <param name="symbol">Symbol of the MarketType Required.</param>
         /// <param name="quantity">Number of shares to request.</param>
-        /// <param name="asynchronous">Send the order asynchrously (false). Otherwise we'll block until it fills</param>
+        /// <param name="asynchronous">Send the order asynchronously (false). Otherwise we'll block until it fills</param>
         /// <param name="tag">Place a custom order property or tag (e.g. indicator data).</param>
         /// <returns>int Order id</returns>
         public OrderTicket MarketOrder(Symbol symbol, decimal quantity, bool asynchronous = false, string tag = "")
@@ -255,7 +255,7 @@ namespace QuantConnect.Algorithm
         /// <returns>The order ID</returns>
         public OrderTicket MarketOnOpenOrder(Symbol symbol, double quantity, string tag = "")
         {
-            return MarketOnOpenOrder(symbol, (decimal)quantity, tag);
+            return MarketOnOpenOrder(symbol, quantity.SafeDecimalCast(), tag);
         }
 
         /// <summary>
@@ -311,7 +311,7 @@ namespace QuantConnect.Algorithm
         /// <returns>The order ID</returns>
         public OrderTicket MarketOnCloseOrder(Symbol symbol, double quantity, string tag = "")
         {
-            return MarketOnCloseOrder(symbol, (decimal)quantity, tag);
+            return MarketOnCloseOrder(symbol, quantity.SafeDecimalCast(), tag);
         }
 
         /// <summary>
@@ -357,7 +357,7 @@ namespace QuantConnect.Algorithm
         /// <returns>Order id</returns>
         public OrderTicket LimitOrder(Symbol symbol, double quantity, decimal limitPrice, string tag = "")
         {
-            return LimitOrder(symbol, (decimal)quantity, limitPrice, tag);
+            return LimitOrder(symbol, quantity.SafeDecimalCast(), limitPrice, tag);
         }
 
         /// <summary>
@@ -404,7 +404,7 @@ namespace QuantConnect.Algorithm
         /// <returns>Int orderId for the new order.</returns>
         public OrderTicket StopMarketOrder(Symbol symbol, double quantity, decimal stopPrice, string tag = "")
         {
-            return StopMarketOrder(symbol, (decimal)quantity, stopPrice, tag);
+            return StopMarketOrder(symbol, quantity.SafeDecimalCast(), stopPrice, tag);
         }
 
         /// <summary>
@@ -453,7 +453,7 @@ namespace QuantConnect.Algorithm
         /// <returns>Order id</returns>
         public OrderTicket StopLimitOrder(Symbol symbol, double quantity, decimal stopPrice, decimal limitPrice, string tag = "")
         {
-            return StopLimitOrder(symbol, (decimal)quantity, stopPrice, limitPrice, tag);
+            return StopLimitOrder(symbol, quantity.SafeDecimalCast(), stopPrice, limitPrice, tag);
         }
 
         /// <summary>
@@ -484,13 +484,15 @@ namespace QuantConnect.Algorithm
         /// </summary>
         /// <param name="optionSymbol">String symbol for the option position</param>
         /// <param name="quantity">Quantity of options contracts</param>
-        /// <param name="asynchronous">Send the order asynchrously (false). Otherwise we'll block until it fills</param>
+        /// <param name="asynchronous">Send the order asynchronously (false). Otherwise we'll block until it fills</param>
         /// <param name="tag">String tag for the order (optional)</param>
         public OrderTicket ExerciseOption(Symbol optionSymbol, int quantity, bool asynchronous = false, string tag = "")
         {
-            var option = (Option)Securities[optionSymbol];
+            var option = (Option) Securities[optionSymbol];
 
-            var request = CreateSubmitOrderRequest(OrderType.OptionExercise, option, quantity, tag, DefaultOrderProperties?.Clone());
+            // SubmitOrderRequest.Quantity indicates the change in holdings quantity, therefore manual exercise quantities must be negative
+            // PreOrderChecksImpl confirms that we don't hold a short position, so we're lenient here and accept +/- quantity values
+            var request = CreateSubmitOrderRequest(OrderType.OptionExercise, option, -Math.Abs(quantity), tag, DefaultOrderProperties?.Clone());
 
             // If warming up, do not submit
             if (IsWarmingUp)
@@ -626,7 +628,7 @@ namespace QuantConnect.Algorithm
 
 
         /// <summary>
-        /// Perform preorder checks to ensure we have sufficient capital,
+        /// Perform pre-order checks to ensure we have sufficient capital,
         /// the market is open, and we haven't exceeded maximum realistic orders per day.
         /// </summary>
         /// <returns>OrderResponse. If no error, order request is submitted.</returns>
@@ -641,7 +643,7 @@ namespace QuantConnect.Algorithm
         }
 
         /// <summary>
-        /// Perform preorder checks to ensure we have sufficient capital,
+        /// Perform pre-order checks to ensure we have sufficient capital,
         /// the market is open, and we haven't exceeded maximum realistic orders per day.
         /// </summary>
         /// <returns>OrderResponse. If no error, order request is submitted.</returns>
@@ -657,7 +659,9 @@ namespace QuantConnect.Algorithm
             Security security;
             if (!Securities.TryGetValue(request.Symbol, out security))
             {
-                return OrderResponse.Error(request, OrderResponseErrorCode.MissingSecurity, "You haven't requested " + request.Symbol.ToString() + " data. Add this with AddSecurity() in the Initialize() Method.");
+                return OrderResponse.Error(request, OrderResponseErrorCode.MissingSecurity,
+                    $"You haven't requested {request.Symbol} data. Add this with AddSecurity() in the Initialize() Method."
+                );
             }
 
             //Ordering 0 is useless.
@@ -677,7 +681,9 @@ namespace QuantConnect.Algorithm
 
             if (!security.IsTradable)
             {
-                return OrderResponse.Error(request, OrderResponseErrorCode.NonTradableSecurity, "The security with symbol '" + request.Symbol.ToString() + "' is marked as non-tradable.");
+                return OrderResponse.Error(request, OrderResponseErrorCode.NonTradableSecurity,
+                    $"The security with symbol '{request.Symbol}' is marked as non-tradable."
+                );
             }
 
             var price = security.Price;
@@ -685,13 +691,17 @@ namespace QuantConnect.Algorithm
             //Check the exchange is open before sending a market on close orders
             if (request.OrderType == OrderType.MarketOnClose && !security.Exchange.ExchangeOpen)
             {
-                return OrderResponse.Error(request, OrderResponseErrorCode.ExchangeNotOpen, request.OrderType + " order and exchange not open.");
+                return OrderResponse.Error(request, OrderResponseErrorCode.ExchangeNotOpen,
+                    $"{request.OrderType} order and exchange not open."
+                );
             }
 
             //Check the exchange is open before sending a exercise orders
             if (request.OrderType == OrderType.OptionExercise && !security.Exchange.ExchangeOpen)
             {
-                return OrderResponse.Error(request, OrderResponseErrorCode.ExchangeNotOpen, request.OrderType + " order and exchange not open.");
+                return OrderResponse.Error(request, OrderResponseErrorCode.ExchangeNotOpen,
+                    $"{request.OrderType} order and exchange not open."
+                );
             }
 
             if (price == 0)
@@ -704,11 +714,15 @@ namespace QuantConnect.Algorithm
             var quoteCurrency = security.QuoteCurrency.Symbol;
             if (!Portfolio.CashBook.TryGetValue(quoteCurrency, out quoteCash))
             {
-                return OrderResponse.Error(request, OrderResponseErrorCode.QuoteCurrencyRequired, request.Symbol.Value + ": requires " + quoteCurrency + " in the cashbook to trade.");
+                return OrderResponse.Error(request, OrderResponseErrorCode.QuoteCurrencyRequired,
+                    $"{request.Symbol.Value}: requires {quoteCurrency} in the cashbook to trade."
+                );
             }
             if (security.QuoteCurrency.ConversionRate == 0m)
             {
-                return OrderResponse.Error(request, OrderResponseErrorCode.ConversionRateZero, request.Symbol.Value + ": requires " + quoteCurrency + " to have a non-zero conversion rate. This can be caused by lack of data.");
+                return OrderResponse.Error(request, OrderResponseErrorCode.ConversionRateZero,
+                    $"{request.Symbol.Value}: requires {quoteCurrency} to have a non-zero conversion rate. This can be caused by lack of data."
+                );
             }
 
             // need to also check base currency existence/conversion rate on forex orders
@@ -718,18 +732,24 @@ namespace QuantConnect.Algorithm
                 var baseCurrency = ((IBaseCurrencySymbol)security).BaseCurrencySymbol;
                 if (!Portfolio.CashBook.TryGetValue(baseCurrency, out baseCash))
                 {
-                    return OrderResponse.Error(request, OrderResponseErrorCode.ForexBaseAndQuoteCurrenciesRequired, request.Symbol.Value + ": requires " + baseCurrency + " and " + quoteCurrency + " in the cashbook to trade.");
+                    return OrderResponse.Error(request, OrderResponseErrorCode.ForexBaseAndQuoteCurrenciesRequired,
+                        $"{request.Symbol.Value}: requires {baseCurrency} and {quoteCurrency} in the cashbook to trade."
+                    );
                 }
                 if (baseCash.ConversionRate == 0m)
                 {
-                    return OrderResponse.Error(request, OrderResponseErrorCode.ForexConversionRateZero, request.Symbol.Value + ": requires " + baseCurrency + " and " + quoteCurrency + " to have non-zero conversion rates. This can be caused by lack of data.");
+                    return OrderResponse.Error(request, OrderResponseErrorCode.ForexConversionRateZero,
+                        $"{request.Symbol.Value}: requires {baseCurrency} and {quoteCurrency} to have non-zero conversion rates. This can be caused by lack of data."
+                    );
                 }
             }
 
             //Make sure the security has some data:
             if (!security.HasData)
             {
-                return OrderResponse.Error(request, OrderResponseErrorCode.SecurityHasNoData, "There is no data for this symbol yet, please check the security.HasData flag to ensure there is at least one data point.");
+                return OrderResponse.Error(request, OrderResponseErrorCode.SecurityHasNoData,
+                    "There is no data for this symbol yet, please check the security.HasData flag to ensure there is at least one data point."
+                );
             }
 
             // We've already processed too many orders: max 10k
@@ -737,28 +757,38 @@ namespace QuantConnect.Algorithm
             {
                 Status = AlgorithmStatus.Stopped;
                 return OrderResponse.Error(request, OrderResponseErrorCode.ExceededMaximumOrders,
-                    $"You have exceeded maximum number of orders ({_maxOrders.ToStringInvariant()}), for unlimited orders upgrade your account."
+                    Invariant($"You have exceeded maximum number of orders ({_maxOrders}), for unlimited orders upgrade your account.")
                 );
             }
 
             if (request.OrderType == OrderType.OptionExercise)
             {
                 if (security.Type != SecurityType.Option)
-                    return OrderResponse.Error(request, OrderResponseErrorCode.NonExercisableSecurity, "The security with symbol '" + request.Symbol.ToString() + "' is not exercisable.");
+                {
+                    return OrderResponse.Error(request, OrderResponseErrorCode.NonExercisableSecurity,
+                        $"The security with symbol '{request.Symbol}' is not exercisable."
+                    );
+                }
 
                 if (security.Holdings.IsShort)
-                    return OrderResponse.Error(request, OrderResponseErrorCode.UnsupportedRequestType, "The security with symbol '" + request.Symbol.ToString() + "' has a short option position. Only long option positions are exercisable.");
+                {
+                    return OrderResponse.Error(request, OrderResponseErrorCode.UnsupportedRequestType,
+                        $"The security with symbol '{request.Symbol}' has a short option position. Only long option positions are exercisable."
+                    );
+                }
 
-                if (request.Quantity > security.Holdings.Quantity)
-                    return OrderResponse.Error(request, OrderResponseErrorCode.UnsupportedRequestType, "Cannot exercise more contracts of '" + request.Symbol.ToString() + "' than is currently available in the portfolio. ");
-
-                if (request.Quantity <= 0.0m)
-                    OrderResponse.ZeroQuantity(request);
+                if (Math.Abs(request.Quantity) > security.Holdings.Quantity)
+                {
+                    return OrderResponse.Error(request, OrderResponseErrorCode.UnsupportedRequestType,
+                        $"Cannot exercise more contracts of '{request.Symbol}' than is currently available in the portfolio. "
+                    );
+                }
             }
 
             if (request.OrderType == OrderType.MarketOnClose)
             {
                 var nextMarketClose = security.Exchange.Hours.GetNextMarketClose(security.LocalTime, false);
+
                 // must be submitted with at least 10 minutes in trading day, add buffer allow order submission
                 var latestSubmissionTime = nextMarketClose.Subtract(Orders.MarketOnCloseOrder.DefaultSubmissionTimeBuffer);
                 if (!security.Exchange.ExchangeOpen || Time > latestSubmissionTime)
@@ -766,7 +796,9 @@ namespace QuantConnect.Algorithm
                     // tell the user we require a 16 minute buffer, on minute data in live a user will receive the 3:44->3:45 bar at 3:45,
                     // this is already too late to submit one of these orders, so make the user do it at the 3:43->3:44 bar so it's submitted
                     // to the brokerage before 3:45.
-                    return OrderResponse.Error(request, OrderResponseErrorCode.MarketOnCloseOrderTooLate, "MarketOnClose orders must be placed with at least a 16 minute buffer before market close.");
+                    return OrderResponse.Error(request, OrderResponseErrorCode.MarketOnCloseOrderTooLate,
+                        "MarketOnClose orders must be placed with at least a 16 minute buffer before market close."
+                    );
                 }
             }
 
@@ -889,7 +921,7 @@ namespace QuantConnect.Algorithm
         /// <seealso cref="MarketOrder(QuantConnect.Symbol,decimal,bool,string)"/>
         public void SetHoldings(Symbol symbol, double percentage, bool liquidateExistingHoldings = false)
         {
-            SetHoldings(symbol, (decimal)percentage, liquidateExistingHoldings);
+            SetHoldings(symbol, percentage.SafeDecimalCast(), liquidateExistingHoldings);
         }
 
         /// <summary>
@@ -989,18 +1021,18 @@ namespace QuantConnect.Algorithm
         /// Calculate the order quantity to achieve target-percent holdings.
         /// </summary>
         /// <param name="symbol">Security object we're asking for</param>
-        /// <param name="target">Target percentag holdings</param>
+        /// <param name="target">Target percentage holdings</param>
         /// <returns>Order quantity to achieve this percentage</returns>
         public decimal CalculateOrderQuantity(Symbol symbol, double target)
         {
-            return CalculateOrderQuantity(symbol, (decimal)target);
+            return CalculateOrderQuantity(symbol, target.SafeDecimalCast());
         }
 
         /// <summary>
         /// Calculate the order quantity to achieve target-percent holdings.
         /// </summary>
         /// <param name="symbol">Security object we're asking for</param>
-        /// <param name="target">Target percentage holdings, this is an unlevered value, so
+        /// <param name="target">Target percentage holdings, this is an unleveraged value, so
         /// if you have 2x leverage and request 100% holdings, it will utilize half of the
         /// available margin</param>
         /// <returns>Order quantity to achieve this percentage</returns>

AlgorithmFactory/Loader.cs

@@ -288,8 +288,7 @@ logging.captureWarnings(True)"
             catch (Exception err)
             {
                 errorMessage = "Algorithm type name not found, or unable to resolve multiple algorithm types to a single type. Please verify algorithm type name matches the algorithm name in the configuration file and that there is one and only one class derived from QCAlgorithm.";
-                errorMessage += err.InnerException == null ? err.Message : err.InnerException.Message;
-                Log.Error($"Loader.TryCreateILAlgorithm(): {errorMessage}");
+                Log.Error($"Loader.TryCreateILAlgorithm(): {errorMessage}\n{err.InnerException ?? err}");
                 return false;
             }
 

Api/Api.cs

@@ -19,8 +19,6 @@ using System.IO;
 using System.Linq;
 using System.Net;
 using Newtonsoft.Json;
-using QuantConnect.API;
-using QuantConnect.Data.Market;
 using QuantConnect.Interfaces;
 using QuantConnect.Orders;
 using RestSharp;
@@ -808,7 +806,7 @@ namespace QuantConnect.Api
         /// <param name="name">The name of the new node</param>
         /// <param name="organizationId">ID of the organization</param>
         /// <param name="sku"><see cref="SKU"/> Object representing configuration</param>
-        /// <returns>Returns <see cref="CreatedNode"/> which contains API response and 
+        /// <returns>Returns <see cref="CreatedNode"/> which contains API response and
         /// <see cref="Node"/></returns>
         public CreatedNode CreateNode(string name, string organizationId, SKU sku)
         {
@@ -824,7 +822,7 @@ namespace QuantConnect.Api
         }
 
         /// <summary>
-        /// Reads the nodes associated with the organization, creating a 
+        /// Reads the nodes associated with the organization, creating a
         /// <see cref="NodeList"/> for the response
         /// </summary>
         /// <param name="organizationId">ID of the organization</param>

Api/ApiConnection.cs

@@ -15,7 +15,6 @@
 
 using System;
 using Newtonsoft.Json;
-using QuantConnect.API;
 using QuantConnect.Configuration;
 using QuantConnect.Logging;
 using QuantConnect.Orders;

Brokerages/Alpaca/AlpacaBrokerage.DataQueueHandler.cs

@@ -1,184 +0,0 @@
-/*
- * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
- * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
-*/
-
-using System;
-using System.Collections.Concurrent;
-using System.Collections.Generic;
-using System.Linq;
-using NodaTime;
-using QuantConnect.Brokerages.Alpaca.Markets;
-using QuantConnect.Data;
-using QuantConnect.Data.Market;
-using QuantConnect.Logging;
-using QuantConnect.Packets;
-
-namespace QuantConnect.Brokerages.Alpaca
-{
-    /// <summary>
-    /// Alpaca Brokerage IDataQueueHandler implementation
-    /// </summary>
-    public partial class AlpacaBrokerage
-    {
-        private readonly ConcurrentDictionary<string, Symbol> _subscribedSymbols = new ConcurrentDictionary<string, Symbol>();
-
-        #region IDataQueueHandler implementation
-
-        /// <summary>
-        /// Sets the job we're subscribing for
-        /// </summary>
-        /// <param name="job">Job we're subscribing for</param>
-        public void SetJob(LiveNodePacket job)
-        {
-        }
-
-        /// <summary>
-        /// Subscribe to the specified configuration
-        /// </summary>
-        /// <param name="dataConfig">defines the parameters to subscribe to a data feed</param>
-        /// <param name="newDataAvailableHandler">handler to be fired on new data available</param>
-        /// <returns>The new enumerator for this subscription request</returns>
-        public IEnumerator<BaseData> Subscribe(SubscriptionDataConfig dataConfig, EventHandler newDataAvailableHandler)
-        {
-            var enumerator = _aggregator.Add(dataConfig, newDataAvailableHandler);
-            Subscribe(new[] { dataConfig.Symbol });
-
-            return enumerator;
-        }
-
-        /// <summary>
-        /// Adds the specified symbols to the subscription
-        /// </summary>
-        /// <param name="symbols">The symbols to be added keyed by SecurityType</param>
-        private void Subscribe(IEnumerable<Symbol> symbols)
-        {
-            var symbolsToSubscribe = symbols.Where(x => !_subscribedSymbols.ContainsKey(x.Value));
-
-            foreach (var symbol in symbolsToSubscribe.Where(CanSubscribe))
-            {
-                Log.Trace($"AlpacaBrokerage.Subscribe(): {symbol}");
-
-                _polygonStreamingClient.SubscribeQuote(symbol.Value);
-                _polygonStreamingClient.SubscribeTrade(symbol.Value);
-
-                _subscribedSymbols.TryAdd(symbol.Value, symbol);
-            }
-        }
-
-        /// <summary>
-        /// Removes the specified configuration
-        /// </summary>
-        /// <param name="dataConfig">Subscription config to be removed</param>
-        public void Unsubscribe(SubscriptionDataConfig dataConfig)
-        {
-            Unsubscribe(new Symbol[] { dataConfig.Symbol });
-            _aggregator.Remove(dataConfig);
-        }
-
-        /// <summary>
-        /// Removes the specified symbols from the subscription
-        /// </summary>
-        /// <param name="symbols">The symbols to be removed keyed by SecurityType</param>
-        private void Unsubscribe(IEnumerable<Symbol> symbols)
-        {
-            var symbolsToUnsubscribe = symbols.Where(x => _subscribedSymbols.ContainsKey(x.Value));
-
-            foreach (var symbol in symbolsToUnsubscribe.Where(CanSubscribe))
-            {
-                Log.Trace($"AlpacaBrokerage.Unsubscribe(): {symbol}");
-
-                _polygonStreamingClient.UnsubscribeQuote(symbol.Value);
-                _polygonStreamingClient.UnsubscribeTrade(symbol.Value);
-
-                Symbol removed;
-                _subscribedSymbols.TryRemove(symbol.Value, out removed);
-            }
-        }
-
-        /// <summary>
-        /// Returns true if this brokerage supports the specified symbol
-        /// </summary>
-        private static bool CanSubscribe(Symbol symbol)
-        {
-            // ignore unsupported security types
-            if (symbol.ID.SecurityType != SecurityType.Equity)
-                return false;
-
-            return symbol.Value.IndexOfInvariant("universe", true) == -1;
-        }
-
-        /// <summary>
-        /// Event handler for streaming quote ticks
-        /// </summary>
-        /// <param name="quote">The data object containing the received tick</param>
-        private void OnQuoteReceived(IStreamQuote quote)
-        {
-            Symbol symbol;
-            if (!_subscribedSymbols.TryGetValue(quote.Symbol, out symbol)) return;
-
-            var time = quote.Time;
-
-            // live ticks timestamps must be in exchange time zone
-            DateTimeZone exchangeTimeZone;
-            if (!_symbolExchangeTimeZones.TryGetValue(key: symbol, value: out exchangeTimeZone))
-            {
-                exchangeTimeZone = _marketHours.GetExchangeHours(Market.USA, symbol, SecurityType.Equity).TimeZone;
-                _symbolExchangeTimeZones.Add(symbol, exchangeTimeZone);
-            }
-            time = time.ConvertFromUtc(exchangeTimeZone);
-
-            var bidPrice = quote.BidPrice;
-            var askPrice = quote.AskPrice;
-            var tick = new Tick(time, symbol, bidPrice, bidPrice, askPrice)
-            {
-                TickType = TickType.Quote,
-                BidSize = quote.BidSize,
-                AskSize = quote.AskSize
-            };
-
-            _aggregator.Update(tick);
-        }
-
-        /// <summary>
-        /// Event handler for streaming trade ticks
-        /// </summary>
-        /// <param name="trade">The data object containing the received tick</param>
-        private void OnTradeReceived(IStreamTrade trade)
-        {
-            Symbol symbol;
-            if (!_subscribedSymbols.TryGetValue(trade.Symbol, out symbol)) return;
-
-            var time = trade.Time;
-
-            // live ticks timestamps must be in exchange time zone
-            DateTimeZone exchangeTimeZone;
-            if (!_symbolExchangeTimeZones.TryGetValue(key: symbol, value: out exchangeTimeZone))
-            {
-                exchangeTimeZone = _marketHours.GetExchangeHours(Market.USA, symbol, SecurityType.Equity).TimeZone;
-                _symbolExchangeTimeZones.Add(symbol, exchangeTimeZone);
-            }
-            time = time.ConvertFromUtc(exchangeTimeZone);
-
-            var tick = new Tick(time, symbol, trade.Price, trade.Price, trade.Price)
-            {
-                TickType = TickType.Trade,
-                Quantity = trade.Size
-            };
-
-            _aggregator.Update(tick);
-        }
-
-        #endregion
-    }
-}

Brokerages/Alpaca/AlpacaBrokerage.Utility.cs

@@ -31,7 +31,6 @@ namespace QuantConnect.Brokerages.Alpaca
     /// </summary>
     public partial class AlpacaBrokerage
     {
-
         /// <summary>
         /// Retrieves the current quotes for an instrument
         /// </summary>
@@ -53,6 +52,7 @@ namespace QuantConnect.Brokerages.Alpaca
                 TickType = TickType.Quote
             };
         }
+
         private IOrder GenerateAndPlaceOrder(Order order)
         {
             var quantity = (long)order.Quantity;
@@ -172,11 +172,6 @@ namespace QuantConnect.Brokerages.Alpaca
             }
         }
 
-        private static void OnPolygonStreamingClientError(Exception exception)
-        {
-            Log.Error($"PolygonStreamingClient error: {exception.Message}");
-        }
-
         private static void OnSockClientError(Exception exception)
         {
             Log.Error($"SockClient error: {exception.Message}");
@@ -193,6 +188,12 @@ namespace QuantConnect.Brokerages.Alpaca
         /// <returns>The list of bars</returns>
         private IEnumerable<TradeBar> DownloadTradeBars(Symbol symbol, DateTime startTimeUtc, DateTime endTimeUtc, Resolution resolution, DateTimeZone requestedTimeZone)
         {
+            // Only equities supported
+            if (symbol.SecurityType != SecurityType.Equity)
+            {
+                yield break;
+            }
+
             // Only minute/hour/daily resolutions supported
             if (resolution < Resolution.Minute)
             {

Brokerages/Alpaca/AlpacaBrokerage.cs

@@ -16,10 +16,8 @@
 using System;
 using System.Collections.Generic;
 using System.Linq;
-using NodaTime;
 using QuantConnect.Brokerages.Alpaca.Markets;
 using QuantConnect.Data;
-using QuantConnect.Interfaces;
 using QuantConnect.Logging;
 using QuantConnect.Orders;
 using QuantConnect.Orders.Fees;
@@ -33,18 +31,14 @@ namespace QuantConnect.Brokerages.Alpaca
     /// Alpaca Brokerage implementation
     /// </summary>
     [BrokerageFactory(typeof(AlpacaBrokerageFactory))]
-    public partial class AlpacaBrokerage : Brokerage, IDataQueueHandler
+    public partial class AlpacaBrokerage : Brokerage
     {
-        private bool _isConnected;
-
         // Rest API requests must be limited to a maximum of 200 messages/minute
         private readonly RateGate _messagingRateLimiter = new RateGate(200, TimeSpan.FromMinutes(1));
 
         private readonly AlpacaTradingClient _alpacaTradingClient;
         private readonly PolygonDataClient _polygonDataClient;
         private readonly SockClient _sockClient;
-        private readonly PolygonStreamingClient _polygonStreamingClient;
-        private readonly bool _handlesMarketData;
 
         /// <summary>
         /// This lock is used to sync 'PlaceOrder' and callback 'OnTradeUpdate'
@@ -61,18 +55,11 @@ namespace QuantConnect.Brokerages.Alpaca
         /// </summary>
         private readonly ISecurityProvider _securityProvider;
 
-        /// <summary>
-        /// The data aggregator
-        /// </summary>
-        private readonly IDataAggregator _aggregator;
-
         /// <summary>
         /// The market hours database
         /// </summary>
         private readonly MarketHoursDatabase _marketHours;
 
-        private readonly Dictionary<Symbol, DateTimeZone> _symbolExchangeTimeZones = new Dictionary<Symbol, DateTimeZone>();
-
         /// <summary>
         /// Initializes a new instance of the <see cref="AlpacaBrokerage"/> class.
         /// </summary>
@@ -81,14 +68,9 @@ namespace QuantConnect.Brokerages.Alpaca
         /// <param name="accountKeyId">The Alpaca api key id</param>
         /// <param name="secretKey">The api secret key</param>
         /// <param name="tradingMode">The Alpaca trading mode. paper/live</param>
-        /// <param name="handlesMarketData">true if market data subscriptions will be handled by Alpaca</param>
-        /// <param name="aggregator">consolidate ticks</param>
-        public AlpacaBrokerage(IOrderProvider orderProvider, ISecurityProvider securityProvider, string accountKeyId, string secretKey, string tradingMode, bool handlesMarketData, IDataAggregator aggregator)
+        public AlpacaBrokerage(IOrderProvider orderProvider, ISecurityProvider securityProvider, string accountKeyId, string secretKey, string tradingMode)
             : base("Alpaca Brokerage")
         {
-            _handlesMarketData = handlesMarketData;
-            _aggregator = aggregator;
-
             var httpScheme = "https://";
             var alpacaBaseUrl = "api.alpaca.markets";
 
@@ -107,6 +89,7 @@ namespace QuantConnect.Brokerages.Alpaca
                 ApiEndpoint = tradingMode.Equals("paper") ? Environments.Paper.AlpacaTradingApi : Environments.Live.AlpacaTradingApi,
                 SecurityId = new SecretKey(accountKeyId, secretKey)
             });
+
             // api client for alpaca data
             _polygonDataClient = new PolygonDataClient(new PolygonDataClientConfiguration
             {
@@ -118,17 +101,6 @@ namespace QuantConnect.Brokerages.Alpaca
             _sockClient = new SockClient(accountKeyId, secretKey, httpAlpacaBaseUrl);
             _sockClient.OnTradeUpdate += OnTradeUpdate;
             _sockClient.OnError += OnSockClientError;
-
-            // Polygon Streaming client for Alpaca (streams trade and quote data)
-            _polygonStreamingClient = new PolygonStreamingClient(new PolygonStreamingClientConfiguration
-            {
-                ApiEndpoint = Environments.Live.PolygonStreamingApi,
-                KeyId = accountKeyId,
-                WebSocketFactory = new WebSocketClientFactory()
-            });
-            _polygonStreamingClient.QuoteReceived += OnQuoteReceived;
-            _polygonStreamingClient.TradeReceived += OnTradeReceived;
-            _polygonStreamingClient.OnError += OnPolygonStreamingClientError;
         }
 
         #region IBrokerage implementation
@@ -136,7 +108,7 @@ namespace QuantConnect.Brokerages.Alpaca
         /// <summary>
         /// Returns true if we're currently connected to the broker
         /// </summary>
-        public override bool IsConnected => _isConnected;
+        public override bool IsConnected => _sockClient.IsConnected;
 
         /// <summary>
         /// Connects the client to the broker's remote servers
@@ -145,14 +117,7 @@ namespace QuantConnect.Brokerages.Alpaca
         {
             if (IsConnected) return;
 
-            _sockClient.ConnectAsync().SynchronouslyAwaitTask();
-
-            if (_handlesMarketData)
-            {
-                _polygonStreamingClient.Connect();
-            }
-
-            _isConnected = true;
+            _sockClient.Connect();
         }
 
         /// <summary>
@@ -160,17 +125,9 @@ namespace QuantConnect.Brokerages.Alpaca
         /// </summary>
         public override void Disconnect()
         {
-            _sockClient.DisconnectAsync().SynchronouslyAwaitTask();
-
-            if (_handlesMarketData)
-            {
-                _polygonStreamingClient.Disconnect();
-            }
-
-            _isConnected = false;
+            _sockClient.Disconnect();
         }
 
-
         /// <summary>
         /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
         /// </summary>
@@ -178,9 +135,7 @@ namespace QuantConnect.Brokerages.Alpaca
         {
             Log.Trace("AlpacaBrokerage.Dispose(): Disposing of Alpaca brokerage resources.");
 
-            _aggregator.Dispose();
             _sockClient?.Dispose();
-            _polygonStreamingClient?.Dispose();
 
             _messagingRateLimiter.Dispose();
         }

Brokerages/Alpaca/AlpacaBrokerageFactory.cs

@@ -16,11 +16,9 @@
 using System;
 using System.Collections.Generic;
 using QuantConnect.Configuration;
-using QuantConnect.Data;
 using QuantConnect.Interfaces;
 using QuantConnect.Packets;
 using QuantConnect.Securities;
-using QuantConnect.Util;
 
 namespace QuantConnect.Brokerages.Alpaca
 {
@@ -92,19 +90,12 @@ namespace QuantConnect.Brokerages.Alpaca
                 throw new Exception("Available trading mode: paper/live");
             }
 
-            var handlesMarketData = job.DataQueueHandler.EndsWith("AlpacaBrokerage");
-
-            var brokerage = new AlpacaBrokerage(algorithm.Transactions, 
+            return new AlpacaBrokerage(
+                algorithm.Transactions, 
                 algorithm.Portfolio, 
                 keyId, 
                 secretKey, 
-                tradingMode, 
-                handlesMarketData,
-                Composer.Instance.GetExportedValueByTypeName<IDataAggregator>(Config.Get("data-aggregator", "QuantConnect.Lean.Engine.DataFeeds.AggregationManager")));
-            Composer.Instance.AddPart<IDataQueueHandler>(brokerage);
-
-            return brokerage;
+                tradingMode);
         }
-
     }
 }

Brokerages/Alpaca/Markets/Environment/EnvironmentExtensions.cs

@@ -1,178 +0,0 @@
-/*
- * The official C# API client for alpaca brokerage
- * Sourced from: https://github.com/alpacahq/alpaca-trade-api-csharp/tree/v3.5.5
- *
- * Changes made from original:
- *   - Removed throw expressions from GetPolygonStreamingClientConfiguration method
-*/
-
-using System;
-using System.Diagnostics.CodeAnalysis;
-
-namespace QuantConnect.Brokerages.Alpaca.Markets
-{
-    /// <summary>
-    /// Collection of helper extension methods for <see cref="IEnvironment"/> interface.
-    /// </summary>
-    [SuppressMessage("ReSharper", "MemberCanBePrivate.Global")]
-    public static class EnvironmentExtensions
-    {
-        /* +------------------------------------------------------------------------+
-         * | The following code has been commented out since it is not required for |
-         * | the PolygonStreamingClient and introduces additional dependencies      |
-         * +------------------------------------------------------------------------+
-
-        /// <summary>
-        /// Creates new instance of <see cref="AlpacaTradingClient"/> for specific
-        /// environment provided as <paramref name="environment"/> argument.
-        /// </summary>
-        /// <param name="environment">Target environment for new object.</param>
-        /// <param name="securityKey">Alpaca API security key.</param>
-        /// <returns>New instance of <see cref="AlpacaTradingClient"/> object.</returns>
-        public static AlpacaTradingClient GetAlpacaTradingClient(
-            this IEnvironment environment,
-            SecurityKey securityKey) =>
-            new AlpacaTradingClient(environment.GetAlpacaTradingClientConfiguration(securityKey));
-
-        /// <summary>
-        /// Creates new instance of <see cref="AlpacaTradingClientConfiguration"/> for specific
-        /// environment provided as <paramref name="environment"/> argument.
-        /// </summary>
-        /// <param name="environment">Target environment for new object.</param>
-        /// <param name="securityKey">Alpaca API security key.</param>
-        /// <returns>New instance of <see cref="AlpacaTradingClientConfiguration"/> object.</returns>
-        public static AlpacaTradingClientConfiguration GetAlpacaTradingClientConfiguration(
-            this IEnvironment environment,
-            SecurityKey securityKey) =>
-            new AlpacaTradingClientConfiguration
-            {
-                ApiEndpoint = environment?.AlpacaTradingApi ?? throw new ArgumentNullException(nameof(environment)),
-                SecurityId = securityKey ?? throw new ArgumentNullException(nameof(securityKey)),
-            };
-
-        /// <summary>
-        /// Creates new instance of <see cref="AlpacaDataClient"/> for specific
-        /// environment provided as <paramref name="environment"/> argument.
-        /// </summary>
-        /// <param name="environment">Target environment for new object.</param>
-        /// <param name="securityKey">Alpaca API security key.</param>
-        /// <returns>New instance of <see cref="AlpacaDataClient"/> object.</returns>
-        public static AlpacaDataClient GetAlpacaDataClient(
-            this IEnvironment environment,
-            SecurityKey securityKey) =>
-            new AlpacaDataClient(environment.GetAlpacaDataClientConfiguration(securityKey));
-
-        /// <summary>
-        /// Creates new instance of <see cref="AlpacaDataClientConfiguration"/> for specific
-        /// environment provided as <paramref name="environment"/> argument.
-        /// </summary>
-        /// <param name="environment">Target environment for new object.</param>
-        /// <param name="securityKey">Alpaca API security key.</param>
-        /// <returns>New instance of <see cref="AlpacaDataClientConfiguration"/> object.</returns>
-        public static AlpacaDataClientConfiguration GetAlpacaDataClientConfiguration(
-            this IEnvironment environment,
-            SecurityKey securityKey) =>
-            new AlpacaDataClientConfiguration
-            {
-                ApiEndpoint = environment?.AlpacaDataApi ?? throw new ArgumentNullException(nameof(environment)),
-                SecurityId = securityKey ?? throw new ArgumentNullException(nameof(securityKey)),
-            };
-
-        /// <summary>
-        /// Creates new instance of <see cref="PolygonDataClient"/> for specific
-        /// environment provided as <paramref name="environment"/> argument.
-        /// </summary>
-        /// <param name="environment">Target environment for new object.</param>
-        /// <param name="keyId">Alpaca API key identifier.</param>
-        /// <returns>New instance of <see cref="PolygonDataClient"/> object.</returns>
-        public static PolygonDataClient GetPolygonDataClient(
-            this IEnvironment environment,
-            String keyId) =>
-            new PolygonDataClient(environment.GetPolygonDataClientConfiguration(keyId));
-
-        /// <summary>
-        /// Creates new instance of <see cref="PolygonDataClientConfiguration"/> for specific
-        /// environment provided as <paramref name="environment"/> argument.
-        /// </summary>
-        /// <param name="environment">Target environment for new object.</param>
-        /// <param name="keyId">Alpaca API key identifier.</param>
-        /// <returns>New instance of <see cref="PolygonDataClientConfiguration"/> object.</returns>
-        public static PolygonDataClientConfiguration GetPolygonDataClientConfiguration(
-            this IEnvironment environment,
-            String keyId) =>
-            new PolygonDataClientConfiguration
-            {
-                ApiEndpoint = environment?.PolygonDataApi ?? throw new ArgumentNullException(nameof(environment)),
-                KeyId = keyId ?? throw new ArgumentNullException(nameof(keyId))
-            };
-
-        /// <summary>
-        /// Creates new instance of <see cref="AlpacaStreamingClient"/> for specific
-        /// environment provided as <paramref name="environment"/> argument.
-        /// </summary>
-        /// <param name="environment">Target environment for new object.</param>
-        /// <param name="securityKey">Alpaca API security key.</param>
-        /// <returns>New instance of <see cref="AlpacaStreamingClient"/> object.</returns>
-        public static AlpacaStreamingClient GetAlpacaStreamingClient(
-            this IEnvironment environment,
-            SecurityKey securityKey) =>
-            new AlpacaStreamingClient(environment.GetAlpacaStreamingClientConfiguration(securityKey));
-
-        /// <summary>
-        /// Creates new instance of <see cref="AlpacaStreamingClientConfiguration"/> for specific
-        /// environment provided as <paramref name="environment"/> argument.
-        /// </summary>
-        /// <param name="environment">Target environment for new object.</param>
-        /// <param name="securityKey">Alpaca API security key.</param>
-        /// <returns>New instance of <see cref="AlpacaStreamingClientConfiguration"/> object.</returns>
-        public static AlpacaStreamingClientConfiguration GetAlpacaStreamingClientConfiguration(
-            this IEnvironment environment,
-            SecurityKey securityKey) =>
-            new AlpacaStreamingClientConfiguration()
-            {
-                ApiEndpoint = environment?.AlpacaStreamingApi ?? throw new ArgumentNullException(nameof(environment)),
-                SecurityId = securityKey,
-            };
-
-        /// <summary>
-        /// Creates new instance of <see cref="PolygonStreamingClient"/> for specific
-        /// environment provided as <paramref name="environment"/> argument.
-        /// </summary>
-        /// <param name="environment">Target environment for new object.</param>
-        /// <param name="keyId">Alpaca API key identifier.</param>
-        /// <returns>New instance of <see cref="PolygonStreamingClient"/> object.</returns>
-        public static PolygonStreamingClient GetPolygonStreamingClient(
-            this IEnvironment environment,
-            String keyId) =>
-            new PolygonStreamingClient(environment.GetPolygonStreamingClientConfiguration(keyId));
-
-        */
-
-        /// <summary>
-        /// Creates new instance of <see cref="PolygonStreamingClientConfiguration"/> for specific
-        /// environment provided as <paramref name="environment"/> argument.
-        /// </summary>
-        /// <param name="environment">Target environment for new object.</param>
-        /// <param name="keyId">Alpaca API key identifier.</param>
-        /// <returns>New instance of <see cref="PolygonStreamingClientConfiguration"/> object.</returns>
-        public static PolygonStreamingClientConfiguration GetPolygonStreamingClientConfiguration(
-            this IEnvironment environment,
-            String keyId)
-        {
-            if (environment?.PolygonStreamingApi == null)
-            {
-                throw new ArgumentNullException(nameof(environment));
-            }
-            if (keyId == null)
-            {
-                throw new ArgumentNullException(nameof(keyId));
-            }
-
-            return new PolygonStreamingClientConfiguration()
-            {
-                ApiEndpoint = environment.PolygonStreamingApi,
-                KeyId = keyId
-            };
-        }
-    }
-}

Brokerages/Alpaca/Markets/Parameters/PolygonStreamingClientConfiguration.cs

@@ -1,40 +0,0 @@
-/*
- * The official C# API client for alpaca brokerage
- * Sourced from: https://github.com/alpacahq/alpaca-trade-api-csharp/tree/v3.5.5
-*/
-
-using System;
-
-namespace QuantConnect.Brokerages.Alpaca.Markets
-{
-    /// <summary>
-    /// Configuration parameters object for <see cref="PolygonStreamingClient"/> class.
-    /// </summary>
-    public sealed class PolygonStreamingClientConfiguration : StreamingClientConfiguration
-    {
-        /// <summary>
-        /// Creates new instance of <see cref="PolygonStreamingClientConfiguration"/> class.
-        /// </summary>
-        public PolygonStreamingClientConfiguration()
-            : base(Environments.Live.PolygonStreamingApi)
-        {
-            KeyId = String.Empty;
-        }
-
-        /// <summary>
-        /// Gets or sets Alpaca application key identifier.
-        /// </summary>
-        public String KeyId { get; set; }
-
-        internal override void EnsureIsValid()
-        {
-            base.EnsureIsValid();
-
-            if (String.IsNullOrEmpty(KeyId))
-            {
-                throw new InvalidOperationException(
-                    $"The value of '{nameof(KeyId)}' property shouldn't be null or empty.");
-            }
-        }
-    }
-}

Brokerages/Alpaca/Markets/Parameters/StreamingClientConfiguration.cs

@@ -1,52 +0,0 @@
-/*
- * The official C# API client for alpaca brokerage
- * Sourced from: https://github.com/alpacahq/alpaca-trade-api-csharp/tree/v3.5.5
-*/
-
-using System;
-
-namespace QuantConnect.Brokerages.Alpaca.Markets
-{
-    /// <summary>
-    /// Configuration parameters object for <see cref="SockClient"/> class.
-    /// </summary>
-    public abstract class StreamingClientConfiguration
-    {
-        /// <summary>
-        /// Creates new instance of <see cref="StreamingClientConfiguration"/> class.
-        /// </summary>
-        protected internal StreamingClientConfiguration(Uri apiEndpoint)
-        {
-            ApiEndpoint = apiEndpoint;
-            WebSocketFactory = new WebSocketClientFactory();
-        }
-
-        /// <summary>
-        /// Gets or sets Alpaca streaming API base URL.
-        /// </summary>
-        public Uri ApiEndpoint { get; set; }
-
-        /// <summary>
-        /// Gets or sets web sockets connection factory.
-        /// </summary>
-        public IWebSocketFactory WebSocketFactory { get; set; }
-
-        internal IWebSocket CreateWebSocket() =>
-            WebSocketFactory.CreateWebSocket(ApiEndpoint);
-
-        internal virtual void EnsureIsValid()
-        {
-            if (ApiEndpoint == null)
-            {
-                throw new InvalidOperationException(
-                    $"The value of '{nameof(ApiEndpoint)}' property shouldn't be null.");
-            }
-
-            if (WebSocketFactory == null)
-            {
-                throw new InvalidOperationException(
-                    $"The value of '{nameof(WebSocketFactory)}' property shouldn't be null.");
-            }
-        }
-    }
-}

Brokerages/Alpaca/Markets/PolygonStreamingClient.cs

@@ -1,325 +0,0 @@
-/*
- * The official C# API client for alpaca brokerage
- * Sourced from: https://github.com/alpacahq/alpaca-trade-api-csharp/tree/v3.5.5
- *
- * Changes made from original:
- *   - Removed Nullable reference type definitions for compatibility with C# 6
- *   - Removed `is null` pattern match
-*/
-
-using System;
-using System.Collections.Generic;
-using System.Diagnostics.CodeAnalysis;
-using System.Linq;
-using Newtonsoft.Json.Linq;
-
-namespace QuantConnect.Brokerages.Alpaca.Markets
-{
-    /// <summary>
-    /// Provides unified type-safe access for Polygon streaming API via websockets.
-    /// </summary>
-    public sealed class PolygonStreamingClient : StreamingClientBase<PolygonStreamingClientConfiguration>
-    {
-        // Available Polygon message types
-
-        private const String TradesChannel = "T";
-
-        private const String QuotesChannel = "Q";
-
-        private const String MinuteAggChannel = "AM";
-
-        private const String SecondAggChannel = "A";
-
-        private const String StatusMessage = "status";
-
-        private readonly IDictionary<String, Action<JToken>> _handlers;
-
-        /// <summary>
-        /// Occured when new trade received from stream.
-        /// </summary>
-        public event Action<IStreamTrade> TradeReceived;
-
-        /// <summary>
-        /// Occured when new quote received from stream.
-        /// </summary>
-        public event Action<IStreamQuote> QuoteReceived;
-
-        /// <summary>
-        /// Occured when new bar received from stream.
-        /// </summary>
-        public event Action<IStreamAgg> MinuteAggReceived;
-
-        /// <summary>
-        /// Occured when new bar received from stream.
-        /// </summary>
-        public event Action<IStreamAgg> SecondAggReceived;
-
-        /// <summary>
-        /// Creates new instance of <see cref="PolygonStreamingClient"/> object.
-        /// </summary>
-        /// <param name="configuration">Configuration parameters object.</param>
-        public PolygonStreamingClient(
-            PolygonStreamingClientConfiguration configuration)
-            : base(configuration.EnsureNotNull(nameof(configuration)))
-        {
-            _handlers = new Dictionary<String, Action<JToken>>(StringComparer.Ordinal)
-            {
-                { StatusMessage, HandleAuthorization },
-                { TradesChannel, HandleTradesChannel },
-                { QuotesChannel, HandleQuotesChannel },
-                { MinuteAggChannel, HandleMinuteAggChannel },
-                { SecondAggChannel, HandleSecondAggChannel }
-            };
-        }
-
-        /// <summary>
-        /// Subscribes for the trade updates via <see cref="TradeReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbol">Asset name for subscription change.</param>
-        public void SubscribeTrade(
-            String symbol) =>
-            Subscribe(GetParams(TradesChannel, symbol));
-
-        /// <summary>
-        /// Subscribes for the quote updates via <see cref="QuoteReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbol">Asset name for subscription change.</param>
-        public void SubscribeQuote(
-            String symbol) =>
-            Subscribe(GetParams(QuotesChannel, symbol));
-
-        /// <summary>
-        /// Subscribes for the second bar updates via <see cref="SecondAggReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbol">Asset name for subscription change.</param>
-        public void SubscribeSecondAgg(
-            String symbol) =>
-            Subscribe(GetParams(SecondAggChannel, symbol));
-
-        /// <summary>
-        /// Subscribes for the minute bar updates via <see cref="MinuteAggReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbol">Asset name for subscription change.</param>
-        public void SubscribeMinuteAgg(
-            String symbol) =>
-            Subscribe(GetParams(MinuteAggChannel, symbol));
-
-        /// <summary>
-        /// Subscribes for the trade updates via <see cref="TradeReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbols">List of asset names for subscription change.</param>
-        public void SubscribeTrade(
-            IEnumerable<String> symbols) =>
-            Subscribe(GetParams(TradesChannel, symbols));
-
-        /// <summary>
-        /// Subscribes for the quote updates via <see cref="QuoteReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbols">List of asset names for subscription change.</param>
-        public void SubscribeQuote(
-            IEnumerable<String> symbols) =>
-            Subscribe(GetParams(QuotesChannel, symbols));
-
-        /// <summary>
-        /// Subscribes for the second bar updates via <see cref="SecondAggReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbols">List of asset names for subscription change.</param>
-        public void SubscribeSecondAgg(
-            IEnumerable<String> symbols) =>
-            Subscribe(GetParams(SecondAggChannel, symbols));
-
-        /// <summary>
-        /// Subscribes for the minute bar updates via <see cref="MinuteAggReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbols">List of asset names for subscription change.</param>
-        public void SubscribeMinuteAgg(
-            IEnumerable<String> symbols) =>
-            Subscribe(GetParams(MinuteAggChannel, symbols));
-
-        /// <summary>
-        /// Unsubscribes from the trade updates via <see cref="TradeReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbol">Asset name for subscription change.</param>
-        public void UnsubscribeTrade(
-            String symbol) =>
-            Unsubscribe(GetParams(TradesChannel, symbol));
-
-        /// <summary>
-        /// Unsubscribes from the quote updates via <see cref="QuoteReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbol">Asset name for subscription change.</param>
-        public void UnsubscribeQuote(
-            String symbol) =>
-            Unsubscribe(GetParams(QuotesChannel, symbol));
-
-        /// <summary>
-        /// Unsubscribes from the second bar updates via <see cref="SecondAggReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbol">Asset name for subscription change.</param>
-        public void UnsubscribeSecondAgg(
-            String symbol) =>
-            Unsubscribe(GetParams(SecondAggChannel, symbol));
-
-        /// <summary>
-        /// Unsubscribes from the minute bar updates via <see cref="MinuteAggReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbol">Asset name for subscription change.</param>
-        public void UnsubscribeMinuteAgg(
-            String symbol) =>
-            Unsubscribe(GetParams(MinuteAggChannel, symbol));
-
-        /// <summary>
-        /// Unsubscribes from the trade updates via <see cref="TradeReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbols">List of asset names for subscription change.</param>
-        public void UnsubscribeTrade(
-            IEnumerable<String> symbols) =>
-            Unsubscribe(GetParams(TradesChannel, symbols));
-
-        /// <summary>
-        /// Unsubscribes from the quote updates via <see cref="QuoteReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbols">List of asset names for subscription change.</param>
-        public void UnsubscribeQuote(
-            IEnumerable<String> symbols) =>
-            Unsubscribe(GetParams(QuotesChannel, symbols));
-
-        /// <summary>
-        /// Unsubscribes from the second bar updates via <see cref="SecondAggReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbols">List of asset names for subscription change.</param>
-        public void UnsubscribeSecondAgg(
-            IEnumerable<String> symbols) =>
-            Unsubscribe(GetParams(SecondAggChannel, symbols));
-
-        /// <summary>
-        /// Unsubscribes from the minute bar updates via <see cref="MinuteAggReceived"/>
-        /// event for specific asset from Polygon streaming API.
-        /// </summary>
-        /// <param name="symbols">List of asset names for subscription change.</param>
-        public void UnsubscribeMinuteAgg(
-            IEnumerable<String> symbols) =>
-            Unsubscribe(GetParams(MinuteAggChannel, symbols));
-
-        /// <inheritdoc/>
-        [SuppressMessage(
-            "Design", "CA1031:Do not catch general exception types",
-            Justification = "Expected behavior - we report exceptions via OnError event.")]
-        protected override void OnMessage(object sender, WebSocketMessage message)
-        {
-            try
-            {
-                foreach (var token in JArray.Parse(message.Message))
-                {
-                    var messageType = token["ev"];
-                    if (ReferenceEquals(messageType, null))
-                    {
-                        var errorMessage = "Null message type.";
-                        HandleError(null, new WebSocketError(errorMessage, new InvalidOperationException(errorMessage)));
-                    }
-                    else
-                    {
-                        HandleMessage(_handlers, messageType.ToString(), token);
-                    }
-                }
-            }
-            catch (Exception exception)
-            {
-                HandleError(null, new WebSocketError(exception.Message, exception));
-            }
-        }
-
-        private void HandleAuthorization(
-            JToken token)
-        {
-            var connectionStatus = token.ToObject<JsonConnectionStatus>();
-
-            // ReSharper disable once ConstantConditionalAccessQualifier
-            switch (connectionStatus?.Status)
-            {
-                case ConnectionStatus.Connected:
-                    SendAsJsonString(new JsonAuthRequest
-                    {
-                        Action = JsonAction.PolygonAuthenticate,
-                        Params = Configuration.KeyId
-                    });
-                    break;
-
-                case ConnectionStatus.AuthenticationSuccess:
-                    OnConnected(AuthStatus.Authorized);
-                    break;
-
-                case ConnectionStatus.AuthenticationFailed:
-                case ConnectionStatus.AuthenticationRequired:
-                    HandleError(null, new WebSocketError(connectionStatus.Message, new InvalidOperationException(connectionStatus.Message)));
-                    break;
-
-                case ConnectionStatus.Failed:
-                case ConnectionStatus.Success:
-                    break;
-
-                default:
-                    var errorMessage = "Unknown connection status.";
-                    HandleError(null, new WebSocketError(errorMessage, new InvalidOperationException(errorMessage)));
-                    break;
-            }
-        }
-
-        private void Subscribe(
-            String parameters) =>
-            SendAsJsonString(new JsonListenRequest
-            {
-                Action = JsonAction.PolygonSubscribe,
-                Params = parameters
-            });
-
-        private void Unsubscribe(
-            String parameters) =>
-            SendAsJsonString(new JsonUnsubscribeRequest
-            {
-                Action = JsonAction.PolygonUnsubscribe,
-                Params = parameters
-            });
-
-        private static String GetParams(
-            String channel,
-            String symbol) =>
-            $"{channel}.{symbol}";
-
-        private static String GetParams(
-            String channel,
-            IEnumerable<String> symbols) =>
-            String.Join(",",symbols.Select(symbol => GetParams(channel, symbol)));
-
-        private void HandleTradesChannel(
-            JToken token) =>
-            TradeReceived.DeserializeAndInvoke<IStreamTrade, JsonStreamTrade>(token);
-
-        private void HandleQuotesChannel(
-            JToken token) =>
-            QuoteReceived.DeserializeAndInvoke<IStreamQuote, JsonStreamQuote>(token);
-
-        private void HandleMinuteAggChannel(
-            JToken token) =>
-            MinuteAggReceived.DeserializeAndInvoke<IStreamAgg, JsonStreamAgg>(token);
-
-        private void HandleSecondAggChannel(
-            JToken token) =>
-            SecondAggReceived.DeserializeAndInvoke<IStreamAgg, JsonStreamAgg>(token);
-    }
-}

Brokerages/Alpaca/Markets/SockClient.cs

@@ -8,9 +8,10 @@
 using System;
 using System.Collections.Generic;
 using System.IO;
-using System.Threading.Tasks;
+using System.Threading;
 using Newtonsoft.Json;
 using Newtonsoft.Json.Linq;
+using QuantConnect.Util;
 
 namespace QuantConnect.Brokerages.Alpaca.Markets
 {
@@ -19,12 +20,19 @@ namespace QuantConnect.Brokerages.Alpaca.Markets
     /// </summary>
     internal sealed class SockClient : IDisposable
     {
+        private const int ConnectionTimeout = 30000;
+
         private readonly WebSocketClientWrapper _webSocket;
 
         private readonly string _keyId;
 
         private readonly string _secretKey;
 
+        /// <summary>
+        /// Returns true if we're currently connected to the broker
+        /// </summary>
+        public bool IsConnected => _webSocket.IsOpen;
+
         /// <summary>
         /// Creates new instance of <see cref="SockClient"/> object.
         /// </summary>
@@ -109,19 +117,39 @@ namespace QuantConnect.Brokerages.Alpaca.Markets
         /// <summary>
         /// Opens connection to Alpaca streaming API.
         /// </summary>
-        /// <returns>Waitable task object for handling action completion in asyncronious mode.</returns>
-        public Task ConnectAsync()
+        public void Connect()
         {
-            return Task.Run(() => _webSocket.Connect());
+            var connectedEvent = new ManualResetEvent(false);
+            EventHandler onOpenAction = (s, e) =>
+            {
+                connectedEvent.Set();
+            };
+
+            _webSocket.Open += onOpenAction;
+
+            try
+            {
+                _webSocket.Connect();
+
+                if (!connectedEvent.WaitOne(ConnectionTimeout))
+                {
+                    throw new Exception("SockClient.Connect(): WebSocket connection timeout.");
+                }
+            }
+            finally
+            {
+                _webSocket.Open -= onOpenAction;
+
+                connectedEvent.DisposeSafely();
+            }
         }
 
         /// <summary>
         /// Closes connection to Alpaca streaming API.
         /// </summary>
-        /// <returns>Waitable task object for handling action completion in asyncronious mode.</returns>
-        public Task DisconnectAsync()
+        public void Disconnect()
         {
-            return Task.Run(() => _webSocket.Close());
+            _webSocket.Close();
         }
 
         /// <inheritdoc />

Brokerages/Alpaca/Markets/WebSocket/IWebSocketFactory.cs

@@ -1,23 +0,0 @@
-/*
- * The official C# API client for alpaca brokerage
- * Sourced from: https://github.com/alpacahq/alpaca-trade-api-csharp/tree/v3.5.5
-*/
-
-using System;
-
-namespace QuantConnect.Brokerages.Alpaca.Markets
-{
-    /// <summary>
-    /// Provides way for creating instance of <see cref="IWebSocket"/> interface implementation.
-    /// </summary>
-    public interface IWebSocketFactory
-    {
-        /// <summary>
-        /// Creates new instance of <see cref="IWebSocket"/> interface implementation.
-        /// </summary>
-        /// <param name="url">Base URL for underlying web socket connection.</param>
-        /// <returns>Instance of class which implements <see cref="IWebSocket"/> interface.</returns>
-        IWebSocket CreateWebSocket(
-            Uri url);
-    }
-}

Brokerages/Alpaca/Markets/WebSocket/StreamingClientBase.cs

@@ -1,209 +0,0 @@
-/*
- * The official C# API client for alpaca brokerage
- * Sourced from: https://github.com/alpacahq/alpaca-trade-api-csharp/tree/v3.5.5
- *
- * Changes made from original:
- *   - Removed Nullable reference type definitions for compatibility with C# 6
- *   - Moved the `HandleConnected` method from inside `ConnectAndAuthenticateAsync`
- *     to its own method. A new member variable was made in order to be able to
- *     signal that we've authenticated, and be able to unsubscribe the event handler.
-*/
-
-using System;
-using System.Collections.Generic;
-using System.Diagnostics.CodeAnalysis;
-using System.IO;
-using Newtonsoft.Json;
-using Newtonsoft.Json.Linq;
-
-namespace QuantConnect.Brokerages.Alpaca.Markets
-{
-    /// <summary>
-    /// Provides unified type-safe access for websocket streaming APIs.
-    /// </summary>
-    public abstract class StreamingClientBase<TConfiguration> : IDisposable
-        where TConfiguration : StreamingClientConfiguration
-    {
-        private readonly SynchronizationQueue _queue = new SynchronizationQueue();
-
-        private readonly IWebSocket _webSocket;
-        internal readonly TConfiguration Configuration;
-
-        /// <summary>
-        /// Creates new instance of <see cref="StreamingClientBase{TConfiguration}"/> object.
-        /// </summary>
-        /// <param name="configuration"></param>
-        protected StreamingClientBase(
-            TConfiguration configuration)
-        {
-            Configuration = configuration.EnsureNotNull(nameof(configuration));
-            Configuration.EnsureIsValid();
-
-            _webSocket = configuration.CreateWebSocket();
-
-            _webSocket.Open += OnOpened;
-            _webSocket.Closed += OnClosed;
-
-            _webSocket.Message += OnMessage;
-
-            _webSocket.Error += HandleError;
-            _queue.OnError += HandleQueueError;
-        }
-
-        /// <summary>
-        /// Occured when stream successfully connected.
-        /// </summary>
-        public event Action<AuthStatus> Connected;
-
-        /// <summary>
-        /// Occured when underlying web socket successfully opened.
-        /// </summary>
-        public event Action SocketOpened;
-
-        /// <summary>
-        /// Occured when underlying web socket successfully closed.
-        /// </summary>
-        public event Action SocketClosed;
-
-        /// <summary>
-        /// Occured when any error happened in stream.
-        /// </summary>
-        public event Action<Exception> OnError;
-
-        /// <summary>
-        /// Opens connection to a streaming API.
-        /// </summary>
-        public void Connect() => _webSocket.Connect();
-
-        /// <summary>
-        /// Closes connection to a streaming API.
-        /// </summary>
-        public void Disconnect() => _webSocket.Close();
-
-        /// <inheritdoc />
-        public void Dispose()
-        {
-            Dispose(true);
-            GC.SuppressFinalize(this);
-        }
-
-        /// <summary>
-        /// Handles <see cref="IWebSocket.Open"/> event.
-        /// </summary>
-        protected virtual void OnOpened(object sender, EventArgs e) => SocketOpened?.Invoke();
-
-        /// <summary>
-        /// Handles <see cref="IWebSocket.Closed"/> event.
-        /// </summary>
-        protected virtual void OnClosed(object sender, WebSocketCloseData e) => SocketClosed?.Invoke();
-
-        /// <summary>
-        /// Handles <see cref="IWebSocket.Message"/> event.
-        /// </summary>
-        protected virtual void OnMessage(object sender, WebSocketMessage message)
-        {
-        }
-
-        /// <summary>
-        /// Implement <see cref="IDisposable"/> pattern for inheritable classes.
-        /// </summary>
-        /// <param name="disposing">If <c>true</c> - dispose managed objects.</param>
-        protected virtual void Dispose(
-            Boolean disposing)
-        {
-            if (!disposing ||
-                _webSocket == null)
-            {
-                return;
-            }
-
-            _webSocket.Open -= OnOpened;
-            _webSocket.Closed -= OnClosed;
-
-            _webSocket.Message -= OnMessage;
-
-            _webSocket.Error -= HandleError;
-            _queue.OnError -= HandleQueueError;
-
-            _queue.Dispose();
-        }
-
-        /// <summary>
-        /// Handles single incoming message. Select handler from generic handlers map
-        /// <paramref name="handlers"/> using <paramref name="messageType"/> parameter
-        /// as a key and pass <paramref name="message"/> parameter as value into the
-        /// selected handler. All exceptions are caught inside this method and reported
-        /// to client via standard <see cref="OnError"/> event.
-        /// </summary>
-        /// <param name="handlers">Message handlers map.</param>
-        /// <param name="messageType">Message type for selecting handler from map.</param>
-        /// <param name="message">Message data for processing by selected handler.</param>
-        [SuppressMessage(
-            "Design", "CA1031:Do not catch general exception types",
-            Justification = "Expected behavior - we report exceptions via OnError event.")]
-        protected void HandleMessage<TKey>(
-            IDictionary<TKey, Action<JToken>> handlers,
-            TKey messageType,
-            JToken message)
-            where TKey : class
-        {
-            try
-            {
-                Action<JToken> handler;
-                if (handlers != null &&
-                    handlers.TryGetValue(messageType, out handler))
-                {
-                    _queue.Enqueue(() => handler(message));
-                }
-                else
-                {
-                    var errorMessage = $"Unexpected message type '{messageType}' received.";
-                    HandleError(null, new WebSocketError(errorMessage, new InvalidOperationException(errorMessage)));
-                }
-            }
-            catch (Exception exception)
-            {
-                HandleError(null, new WebSocketError(exception.Message, exception));
-            }
-        }
-
-        /// <summary>
-        /// Raises <see cref="Connected"/> event with specified <paramref name="authStatus"/> value.
-        /// </summary>
-        /// <param name="authStatus">Authentication status (protocol level) of client.</param>
-        protected void OnConnected(
-            AuthStatus authStatus) =>
-            Connected?.Invoke(authStatus);
-
-        /// <summary>
-        /// Handles <see cref="SynchronizationQueue.OnError"/> event.
-        /// </summary>
-        protected void HandleError(object sender, WebSocketError error)
-        {
-            OnError?.Invoke(error.Exception);
-        }
-
-        /// <summary>
-        /// Handles <see cref="IWebSocket.Error"/> event.
-        /// </summary>
-        /// <param name="exception">Exception for routing into <see cref="OnError"/> event.</param>
-        private void HandleQueueError(Exception exception)
-        {
-            OnError?.Invoke(exception);
-        }
-
-        /// <summary>
-        ///
-        /// </summary>
-        /// <param name="value"></param>
-        protected void SendAsJsonString(object value)
-        {
-            using (var textWriter = new StringWriter())
-            {
-                var serializer = new JsonSerializer();
-                serializer.Serialize(textWriter, value);
-                _webSocket.Send(textWriter.ToString());
-            }
-        }
-    }
-}

Brokerages/Alpaca/Markets/WebSocket/SynchronizationQueue.cs

@@ -1,74 +0,0 @@
-/*
- * The official C# API client for alpaca brokerage
- * Sourced from: https://github.com/alpacahq/alpaca-trade-api-csharp/tree/v3.5.5
- *
- * Changes made from original:
- *   - Removed Nullable reference type definitions for compatibility with C# 6
-*/
-
-using System;
-using System.Collections.Concurrent;
-using System.Diagnostics;
-using System.Diagnostics.CodeAnalysis;
-using System.Threading;
-using System.Threading.Tasks;
-
-namespace QuantConnect.Brokerages.Alpaca.Markets
-{
-    internal sealed class SynchronizationQueue : IDisposable
-    {
-        private readonly BlockingCollection<Action> _actions =
-            new BlockingCollection<Action>(new ConcurrentQueue<Action>());
-
-        private readonly CancellationTokenSource _cancellationTokenSource =
-            new CancellationTokenSource();
-
-        public SynchronizationQueue()
-        {
-            var factory = new TaskFactory(_cancellationTokenSource.Token);
-            factory.StartNew(processingTask, _cancellationTokenSource.Token,
-                TaskCreationOptions.LongRunning, TaskScheduler.Current);
-        }
-
-        public event Action<Exception> OnError;
-
-        public void Enqueue(Action action) =>
-            _actions.Add(action, _cancellationTokenSource.Token);
-
-        [SuppressMessage(
-            "Design", "CA1031:Do not catch general exception types",
-            Justification = "Expected behavior - we report exceptions via OnError event.")]
-        private void processingTask()
-        {
-            try
-            {
-                foreach (var action in _actions
-                    .GetConsumingEnumerable(_cancellationTokenSource.Token))
-                {
-                    try
-                    {
-                        action();
-                    }
-                    catch (Exception exception)
-                    {
-                        OnError?.Invoke(exception);
-                    }
-                }
-            }
-            catch (ObjectDisposedException exception)
-            {
-                Trace.TraceInformation(exception.Message);
-            }
-            catch (OperationCanceledException exception)
-            {
-                Trace.TraceInformation(exception.Message);
-            }
-        }
-
-        public void Dispose()
-        {
-            _cancellationTokenSource.Dispose();
-            _actions.Dispose();
-        }
-    }
-}

Brokerages/Alpaca/Markets/WebSocket/WebSocketClientFactory.cs

@@ -1,26 +0,0 @@
-/*
- * The official C# API client for alpaca brokerage
- * Sourced from: https://github.com/alpacahq/alpaca-trade-api-csharp/tree/v3.5.5
-*/
-
-using System;
-using System.Diagnostics.CodeAnalysis;
-
-namespace QuantConnect.Brokerages.Alpaca.Markets
-{
-    [SuppressMessage(
-        "Microsoft.Performance", "CA1812:Avoid uninstantiated internal classes",
-        Justification = "Object instances of this class will be created by Newtonsoft.JSON library.")]
-    internal sealed class WebSocketClientFactory : IWebSocketFactory
-    {
-        private sealed class AlpacaWebSocketClientWrapper : WebSocketClientWrapper
-        {
-            public AlpacaWebSocketClientWrapper(Uri url)
-            {
-                Initialize(url.ToString());
-            }
-        }
-
-        public IWebSocket CreateWebSocket(Uri url) => new AlpacaWebSocketClientWrapper(url);
-    }
-}

Brokerages/Backtesting/BacktestingBrokerage.cs

@@ -430,7 +430,7 @@ namespace QuantConnect.Brokerages.Backtesting
                             OnOrderEvent(fill);
                         }
 
-                        if (order.Type == OrderType.OptionExercise)
+                        if (fill.IsAssignment)
                         {
                             fill.Message = order.Tag;
                             OnOptionPositionAssigned(fill);
@@ -517,8 +517,7 @@ namespace QuantConnect.Brokerages.Backtesting
         /// <returns></returns>
         private void SetPendingOrder(Order order)
         {
-            // only save off clones!
-            _pending[order.Id] = order.Clone();
+            _pending[order.Id] = order;
         }
     }
 }

Brokerages/BaseWebsocketsBrokerage.cs

@@ -14,12 +14,13 @@
 */
 
 using Newtonsoft.Json;
-using QuantConnect.Data.Market;
+using QuantConnect.Data;
 using QuantConnect.Logging;
 using RestSharp;
 using System;
 using System.Collections.Concurrent;
 using System.Collections.Generic;
+using System.Linq;
 using System.Threading;
 
 namespace QuantConnect.Brokerages
@@ -30,48 +31,42 @@ namespace QuantConnect.Brokerages
     /// </summary>
     public abstract class BaseWebsocketsBrokerage : Brokerage
     {
+        private const int ConnectionTimeout = 30000;
 
-        #region Declarations
         /// <summary>
         /// The websockets client instance
         /// </summary>
-        protected IWebSocket WebSocket;
+        protected readonly IWebSocket WebSocket;
+
         /// <summary>
         /// The rest client instance
         /// </summary>
-        protected IRestClient RestClient;
+        protected readonly IRestClient RestClient;
+
         /// <summary>
         /// standard json parsing settings
         /// </summary>
-        protected JsonSerializerSettings JsonSettings = new JsonSerializerSettings { FloatParseHandling = FloatParseHandling.Decimal };
+        protected readonly JsonSerializerSettings JsonSettings;
+
         /// <summary>
         /// A list of currently active orders
         /// </summary>
-        public ConcurrentDictionary<int, Orders.Order> CachedOrderIDs = new ConcurrentDictionary<int, Orders.Order>();
-        /// <summary>
-        /// A list of currently subscribed channels
-        /// </summary>
-        protected Dictionary<string, Channel> ChannelList = new Dictionary<string, Channel>();
-        private string _market { get; set; }
+        public readonly ConcurrentDictionary<int, Orders.Order> CachedOrderIDs;
+
         /// <summary>
         /// The api secret
         /// </summary>
-        protected string ApiSecret;
+        protected readonly string ApiSecret;
+
         /// <summary>
         /// The api key
         /// </summary>
-        protected string ApiKey;
+        protected readonly string ApiKey;
+
         /// <summary>
-        /// Timestamp of most recent heartbeat message
+        /// Count subscribers for each (symbol, tickType) combination
         /// </summary>
-        protected DateTime LastHeartbeatUtcTime = DateTime.UtcNow;
-        private const int _heartbeatTimeout = 90;
-        private Thread _connectionMonitorThread;
-        private CancellationTokenSource _cancellationTokenSource;
-        private readonly object _lockerConnectionMonitor = new object();
-        private volatile bool _connectionLost;
-        private const int _connectionTimeout = 30000;
-        #endregion
+        protected DataQueueHandlerSubscriptionManager SubscriptionManager;
 
         /// <summary>
         /// Creates an instance of a websockets brokerage
@@ -81,19 +76,23 @@ namespace QuantConnect.Brokerages
         /// <param name="restClient">Rest client instance</param>
         /// <param name="apiKey">Brokerage api auth key</param>
         /// <param name="apiSecret">Brokerage api auth secret</param>
-        /// <param name="market">Name of market</param>
         /// <param name="name">Name of brokerage</param>
-        public BaseWebsocketsBrokerage(string wssUrl, IWebSocket websocket, IRestClient restClient, string apiKey, string apiSecret, string market, string name) : base(name)
+        protected BaseWebsocketsBrokerage(string wssUrl, IWebSocket websocket, IRestClient restClient, string apiKey, string apiSecret, string name) : base(name)
         {
+            JsonSettings = new JsonSerializerSettings { FloatParseHandling = FloatParseHandling.Decimal };
+            CachedOrderIDs = new ConcurrentDictionary<int, Orders.Order>();
+
             WebSocket = websocket;
-
             WebSocket.Initialize(wssUrl);
-
             WebSocket.Message += OnMessage;
-            WebSocket.Error += OnError;
+
+            WebSocket.Open += (sender, args) =>
+            {
+                Log.Trace($"BaseWebsocketsBrokerage(): WebSocket.Open. Subscribing");
+                Subscribe(GetSubscribed());
+            };
 
             RestClient = restClient;
-            _market = market;
             ApiSecret = apiSecret;
             ApiKey = apiKey;
         }
@@ -114,170 +113,8 @@ namespace QuantConnect.Brokerages
                 return;
 
             Log.Trace("BaseWebSocketsBrokerage.Connect(): Connecting...");
-            WebSocket.Connect();
-            Wait(_connectionTimeout, () => WebSocket.IsOpen);
 
-            _cancellationTokenSource = new CancellationTokenSource();
-            _connectionMonitorThread = new Thread(() =>
-            {
-                var nextReconnectionAttemptUtcTime = DateTime.UtcNow;
-                double nextReconnectionAttemptSeconds = 1;
-
-                lock (_lockerConnectionMonitor)
-                {
-                    LastHeartbeatUtcTime = DateTime.UtcNow;
-                }
-
-                try
-                {
-                    while (!_cancellationTokenSource.IsCancellationRequested)
-                    {
-                        if (WebSocket.IsOpen)
-                        {
-                            LastHeartbeatUtcTime = DateTime.UtcNow;
-                        }
-
-                        TimeSpan elapsed;
-                        lock (_lockerConnectionMonitor)
-                        {
-                            elapsed = DateTime.UtcNow - LastHeartbeatUtcTime;
-                        }
-
-                        if (!_connectionLost && elapsed > TimeSpan.FromSeconds(_heartbeatTimeout))
-                        {
-
-                            if (WebSocket.IsOpen)
-                            {
-                                // connection is still good
-                                LastHeartbeatUtcTime = DateTime.UtcNow;
-                            }
-                            else
-                            {
-                                _connectionLost = true;
-                                nextReconnectionAttemptUtcTime = DateTime.UtcNow.AddSeconds(nextReconnectionAttemptSeconds);
-
-                                OnMessage(BrokerageMessageEvent.Disconnected("Connection with server lost. This could be because of internet connectivity issues."));
-                            }
-                        }
-                        else if (_connectionLost)
-                        {
-                            try
-                            {
-                                if (elapsed <= TimeSpan.FromSeconds(_heartbeatTimeout))
-                                {
-                                    _connectionLost = false;
-                                    nextReconnectionAttemptSeconds = 1;
-
-                                    OnMessage(BrokerageMessageEvent.Reconnected("Connection with server restored."));
-                                }
-                                else
-                                {
-                                    if (DateTime.UtcNow > nextReconnectionAttemptUtcTime)
-                                    {
-                                        try
-                                        {
-                                            Reconnect();
-                                        }
-                                        catch (Exception err)
-                                        {
-                                            // double the interval between attempts (capped to 1 minute)
-                                            nextReconnectionAttemptSeconds = Math.Min(nextReconnectionAttemptSeconds * 2, 60);
-                                            nextReconnectionAttemptUtcTime = DateTime.UtcNow.AddSeconds(nextReconnectionAttemptSeconds);
-                                            Log.Error(err);
-                                        }
-                                    }
-                                }
-                            }
-                            catch (Exception exception)
-                            {
-                                Log.Error(exception);
-                            }
-                        }
-
-                        Thread.Sleep(10000);
-                    }
-                }
-                catch (Exception exception)
-                {
-                    Log.Error(exception);
-                }
-            }) { IsBackground = true };
-            _connectionMonitorThread.Start();
-            while (!_connectionMonitorThread.IsAlive)
-            {
-                Thread.Sleep(1);
-            }
-        }
-
-        /// <summary>
-        /// Disconnects the client from the broker's remote servers
-        /// </summary>
-        public override void Disconnect()
-        {
-            // request and wait for thread to stop
-            _cancellationTokenSource?.Cancel();
-            _connectionMonitorThread?.Join();
-        }
-
-        /// <summary>
-        /// Handles websocket errors
-        /// </summary>
-        /// <param name="sender"></param>
-        /// <param name="e"></param>
-        public void OnError(object sender, WebSocketError e)
-        {
-            Log.Error(e.Exception, "WebSocketsBrokerage Web Exception:  ");
-        }
-
-        /// <summary>
-        /// Handles reconnections in the event of connection loss
-        /// </summary>
-        protected virtual void Reconnect()
-        {
-            if (WebSocket.IsOpen)
-            {
-                // connection is still good
-                LastHeartbeatUtcTime = DateTime.UtcNow;
-                return;
-            }
-
-            Log.Trace($"BaseWebsocketsBrokerage(): Reconnecting... IsConnected: {IsConnected}");
-            var subscribed = GetSubscribed();
-
-            WebSocket.Error -= this.OnError;
-            try
-            {
-                //try to clean up state
-                if (IsConnected)
-                {
-                    WebSocket.Close();
-                    Wait(_connectionTimeout, () => !WebSocket.IsOpen);
-                }
-                if (!IsConnected)
-                {
-                    WebSocket.Connect();
-                    Wait(_connectionTimeout, () => WebSocket.IsOpen);
-                }
-            }
-            finally
-            {
-                WebSocket.Error += this.OnError;
-                this.Subscribe(subscribed);
-            }
-        }
-
-        private void Wait(int timeout, Func<bool> state)
-        {
-            var StartTime = Environment.TickCount;
-            do
-            {
-                if (Environment.TickCount > StartTime + timeout)
-                {
-                    throw new Exception("Websockets connection timeout.");
-                }
-                Thread.Sleep(1);
-            }
-            while (!state());
+            ConnectSync();
         }
 
         /// <summary>
@@ -290,34 +127,24 @@ namespace QuantConnect.Brokerages
         /// Gets a list of current subscriptions
         /// </summary>
         /// <returns></returns>
-        protected virtual IList<Symbol> GetSubscribed()
+        protected virtual IEnumerable<Symbol> GetSubscribed()
         {
-            IList<Symbol> list = new List<Symbol>();
-            lock (ChannelList)
+            return SubscriptionManager?.GetSubscribedSymbols() ?? Enumerable.Empty<Symbol>();
+        }
+
+        private void ConnectSync()
+        {
+            var resetEvent = new ManualResetEvent(false);
+            EventHandler triggerEvent = (o, args) => resetEvent.Set();
+            WebSocket.Open += triggerEvent;
+
+            WebSocket.Connect();
+
+            if (!resetEvent.WaitOne(ConnectionTimeout))
             {
-                foreach (var item in ChannelList)
-                {
-                    list.Add(Symbol.Create(item.Value.Symbol, SecurityType.Forex, _market));
-                }
+                throw new TimeoutException("Websockets connection timeout.");
             }
-            return list;
+            WebSocket.Open -= triggerEvent;
         }
-
-        /// <summary>
-        /// Represents a subscription channel
-        /// </summary>
-        public class Channel
-        {
-            /// <summary>
-            /// The name of the channel
-            /// </summary>
-            public string Name { get; set; }
-            /// <summary>
-            /// The ticker symbol of the channel
-            /// </summary>
-            public string Symbol { get; set; }
-        }
-
     }
-
 }

Brokerages/Binance/BinanceBrokerage.Messaging.cs

@@ -0,0 +1,226 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using QuantConnect.Brokerages.Binance.Messages;
+using QuantConnect.Data.Market;
+using QuantConnect.Logging;
+using QuantConnect.Orders;
+using QuantConnect.Orders.Fees;
+using QuantConnect.Securities;
+using System;
+using System.Collections.Concurrent;
+using System.Linq;
+using Newtonsoft.Json.Linq;
+using QuantConnect.Data;
+
+namespace QuantConnect.Brokerages.Binance
+{
+    public partial class BinanceBrokerage
+    {
+        private readonly ConcurrentQueue<WebSocketMessage> _messageBuffer = new ConcurrentQueue<WebSocketMessage>();
+        private volatile bool _streamLocked;
+        private readonly IDataAggregator _aggregator;
+
+        /// <summary>
+        /// Locking object for the Ticks list in the data queue handler
+        /// </summary>
+        protected readonly object TickLocker = new object();
+
+        /// <summary>
+        /// Lock the streaming processing while we're sending orders as sometimes they fill before the REST call returns.
+        /// </summary>
+        private void LockStream()
+        {
+            _streamLocked = true;
+        }
+
+        /// <summary>
+        /// Unlock stream and process all backed up messages.
+        /// </summary>
+        private void UnlockStream()
+        {
+            while (_messageBuffer.Any())
+            {
+                WebSocketMessage e;
+                _messageBuffer.TryDequeue(out e);
+
+                OnMessageImpl(e);
+            }
+
+            // Once dequeued in order; unlock stream.
+            _streamLocked = false;
+        }
+
+        private void WithLockedStream(Action code)
+        {
+            try
+            {
+                LockStream();
+                code();
+            }
+            finally
+            {
+                UnlockStream();
+            }
+        }
+
+        private void OnMessageImpl(WebSocketMessage e)
+        {
+            try
+            {
+                var obj = JObject.Parse(e.Message);
+
+                var objError = obj["error"];
+                if (objError != null)
+                {
+                    var error = objError.ToObject<ErrorMessage>();
+                    OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Error, error.Code, error.Message));
+                    return;
+                }
+
+                var objData = obj;
+
+                var objEventType = objData["e"];
+                if (objEventType != null)
+                {
+                    var eventType = objEventType.ToObject<string>();
+
+                    switch (eventType)
+                    {
+                        case "executionReport":
+                            var upd = objData.ToObject<Execution>();
+                            if (upd.ExecutionType.Equals("TRADE", StringComparison.OrdinalIgnoreCase))
+                            {
+                                OnFillOrder(upd);
+                            }
+                            break;
+
+                        case "trade":
+                            var trade = objData.ToObject<Trade>();
+                            EmitTradeTick(
+                                _symbolMapper.GetLeanSymbol(trade.Symbol),
+                                Time.UnixMillisecondTimeStampToDateTime(trade.Time),
+                                trade.Price,
+                                trade.Quantity);
+                            break;
+                    }
+                }
+                else if (objData["u"] != null)
+                {
+                    var quote = objData.ToObject<BestBidAskQuote>();
+                    EmitQuoteTick(
+                        _symbolMapper.GetLeanSymbol(quote.Symbol),
+                        quote.BestBidPrice,
+                        quote.BestBidSize,
+                        quote.BestAskPrice,
+                        quote.BestAskSize);
+                }
+            }
+            catch (Exception exception)
+            {
+                OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Error, -1, $"Parsing wss message failed. Data: {e.Message} Exception: {exception}"));
+                throw;
+            }
+        }
+
+        private void EmitQuoteTick(Symbol symbol, decimal bidPrice, decimal bidSize, decimal askPrice, decimal askSize)
+        {
+            var tick = new Tick
+            {
+                AskPrice = askPrice,
+                BidPrice = bidPrice,
+                Time = DateTime.UtcNow,
+                Symbol = symbol,
+                TickType = TickType.Quote,
+                AskSize = askSize,
+                BidSize = bidSize
+            };
+            tick.SetValue();
+
+            lock (TickLocker)
+            {
+                _aggregator.Update(tick);
+            }
+        }
+
+        private void EmitTradeTick(Symbol symbol, DateTime time, decimal price, decimal quantity)
+        {
+            var tick = new Tick
+            {
+                Symbol = symbol,
+                Value = price,
+                Quantity = Math.Abs(quantity),
+                Time = time,
+                TickType = TickType.Trade
+            };
+
+            lock (TickLocker)
+            {
+                _aggregator.Update(tick);
+            }
+        }
+
+        private void OnFillOrder(Execution data)
+        {
+            try
+            {
+                var order = FindOrderByExternalId(data.OrderId);
+                if (order == null)
+                {
+                    // not our order, nothing else to do here
+                    return;
+                }
+
+                var fillPrice = data.LastExecutedPrice;
+                var fillQuantity = data.Direction == OrderDirection.Sell ? -data.LastExecutedQuantity : data.LastExecutedQuantity;
+                var updTime = Time.UnixMillisecondTimeStampToDateTime(data.TransactionTime);
+                var orderFee = new OrderFee(new CashAmount(data.Fee, data.FeeCurrency));
+                var status = ConvertOrderStatus(data.OrderStatus);
+                var orderEvent = new OrderEvent
+                (
+                    order.Id, order.Symbol, updTime, status,
+                    data.Direction, fillPrice, fillQuantity,
+                    orderFee, $"Binance Order Event {data.Direction}"
+                );
+
+                if (status == OrderStatus.Filled)
+                {
+                    Orders.Order outOrder;
+                    CachedOrderIDs.TryRemove(order.Id, out outOrder);
+                }
+
+                OnOrderEvent(orderEvent);
+            }
+            catch (Exception e)
+            {
+                Log.Error(e);
+                throw;
+            }
+        }
+
+        private Orders.Order FindOrderByExternalId(string brokerId)
+        {
+            var order = CachedOrderIDs
+                    .FirstOrDefault(o => o.Value.BrokerId.Contains(brokerId))
+                    .Value;
+            if (order == null)
+            {
+                order = _algorithm.Transactions.GetOrderByBrokerageId(brokerId);
+            }
+
+            return order;
+        }
+    }
+}

Brokerages/Binance/BinanceBrokerage.Utility.cs

@@ -0,0 +1,53 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using QuantConnect.Orders;
+
+namespace QuantConnect.Brokerages.Binance
+{
+    /// <summary>
+    /// Binance utility methods
+    /// </summary>
+    public partial class BinanceBrokerage
+    {
+        private static OrderStatus ConvertOrderStatus(string raw)
+        {
+            switch (raw.LazyToUpper())
+            {
+                case "NEW":
+                    return OrderStatus.New;
+
+                case "PARTIALLY_FILLED":
+                    return OrderStatus.PartiallyFilled;
+
+                case "FILLED":
+                    return OrderStatus.Filled;
+
+                case "PENDING_CANCEL":
+                    return OrderStatus.CancelPending;
+
+                case "CANCELED":
+                    return OrderStatus.Canceled;
+
+                case "REJECTED":
+                case "EXPIRED":
+                    return OrderStatus.Invalid;
+
+                default:
+                    return OrderStatus.None;
+            }
+        }
+    }
+}

Brokerages/Binance/BinanceBrokerage.cs

@@ -0,0 +1,491 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+using QuantConnect.Data;
+using QuantConnect.Data.Market;
+using QuantConnect.Interfaces;
+using QuantConnect.Logging;
+using QuantConnect.Orders;
+using QuantConnect.Packets;
+using QuantConnect.Securities;
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Threading;
+using Newtonsoft.Json;
+using QuantConnect.Util;
+using Timer = System.Timers.Timer;
+
+namespace QuantConnect.Brokerages.Binance
+{
+    /// <summary>
+    /// Binance brokerage implementation
+    /// </summary>
+    [BrokerageFactory(typeof(BinanceBrokerageFactory))]
+    public partial class BinanceBrokerage : BaseWebsocketsBrokerage, IDataQueueHandler
+    {
+        private const string WebSocketBaseUrl = "wss://stream.binance.com:9443/ws";
+
+        private readonly IAlgorithm _algorithm;
+        private readonly BinanceSymbolMapper _symbolMapper = new BinanceSymbolMapper();
+
+        private readonly RateGate _webSocketRateLimiter = new RateGate(5, TimeSpan.FromSeconds(1));
+        private long _lastRequestId;
+
+        private readonly Timer _keepAliveTimer;
+        private readonly Timer _reconnectTimer;
+        private readonly BinanceRestApiClient _apiClient;
+
+        /// <summary>
+        /// Constructor for brokerage
+        /// </summary>
+        /// <param name="apiKey">api key</param>
+        /// <param name="apiSecret">api secret</param>
+        /// <param name="algorithm">the algorithm instance is required to retrieve account type</param>
+        /// <param name="aggregator">the aggregator for consolidating ticks</param>
+        public BinanceBrokerage(string apiKey, string apiSecret, IAlgorithm algorithm, IDataAggregator aggregator)
+            : base(WebSocketBaseUrl, new WebSocketClientWrapper(), null, apiKey, apiSecret, "Binance")
+        {
+            _algorithm = algorithm;
+            _aggregator = aggregator;
+
+            var subscriptionManager = new EventBasedDataQueueHandlerSubscriptionManager();
+            subscriptionManager.SubscribeImpl += (s, t) =>
+            {
+                Subscribe(s);
+                return true;
+            };
+            subscriptionManager.UnsubscribeImpl += (s, t) => Unsubscribe(s);
+
+            SubscriptionManager = subscriptionManager;
+
+            _apiClient = new BinanceRestApiClient(
+                _symbolMapper,
+                algorithm?.Portfolio,
+                apiKey,
+                apiSecret);
+
+            _apiClient.OrderSubmit += (s, e) => OnOrderSubmit(e);
+            _apiClient.OrderStatusChanged += (s, e) => OnOrderEvent(e);
+            _apiClient.Message += (s, e) => OnMessage(e);
+
+            // User data streams will close after 60 minutes. It's recommended to send a ping about every 30 minutes.
+            // Source: https://github.com/binance-exchange/binance-official-api-docs/blob/master/user-data-stream.md#pingkeep-alive-a-listenkey
+            _keepAliveTimer = new Timer
+            {
+                // 30 minutes
+                Interval = 30 * 60 * 1000
+            };
+            _keepAliveTimer.Elapsed += (s, e) => _apiClient.SessionKeepAlive();
+
+            WebSocket.Open += (s, e) => { _keepAliveTimer.Start(); };
+            WebSocket.Closed += (s, e) => { _keepAliveTimer.Stop(); };
+
+            // A single connection to stream.binance.com is only valid for 24 hours; expect to be disconnected at the 24 hour mark
+            // Source: https://github.com/binance-exchange/binance-official-api-docs/blob/master/web-socket-streams.md#general-wss-information
+            _reconnectTimer = new Timer
+            {
+                // 23.5 hours
+                Interval = 23.5 * 60 * 60 * 1000
+            };
+            _reconnectTimer.Elapsed += (s, e) =>
+            {
+                Log.Trace("Daily websocket restart: disconnect");
+                Disconnect();
+
+                Log.Trace("Daily websocket restart: connect");
+                Connect();
+            };
+        }
+
+        #region IBrokerage
+
+        /// <summary>
+        /// Checks if the websocket connection is connected or in the process of connecting
+        /// </summary>
+        public override bool IsConnected => WebSocket.IsOpen;
+
+        /// <summary>
+        /// Creates wss connection
+        /// </summary>
+        public override void Connect()
+        {
+            if (IsConnected)
+                return;
+
+            _apiClient.CreateListenKey();
+            _reconnectTimer.Start();
+
+            WebSocket.Initialize($"{WebSocketBaseUrl}/{_apiClient.SessionId}");
+
+            base.Connect();
+        }
+
+        /// <summary>
+        /// Closes the websockets connection
+        /// </summary>
+        public override void Disconnect()
+        {
+            _reconnectTimer.Stop();
+
+            WebSocket?.Close();
+            _apiClient.StopSession();
+        }
+
+        /// <summary>
+        /// Gets all open positions
+        /// </summary>
+        /// <returns></returns>
+        public override List<Holding> GetAccountHoldings()
+        {
+            return _apiClient.GetAccountHoldings();
+        }
+
+        /// <summary>
+        /// Gets the total account cash balance for specified account type
+        /// </summary>
+        /// <returns></returns>
+        public override List<CashAmount> GetCashBalance()
+        {
+            var account = _apiClient.GetCashBalance();
+            var balances = account.Balances?.Where(balance => balance.Amount > 0).ToList();
+            if (balances == null || !balances.Any())
+                return new List<CashAmount>();
+
+            return balances
+                .Select(b => new CashAmount(b.Amount, b.Asset.LazyToUpper()))
+                .ToList();
+        }
+
+        /// <summary>
+        /// Gets all orders not yet closed
+        /// </summary>
+        /// <returns></returns>
+        public override List<Order> GetOpenOrders()
+        {
+            var orders = _apiClient.GetOpenOrders();
+            List<Order> list = new List<Order>();
+            foreach (var item in orders)
+            {
+                Order order;
+                switch (item.Type.LazyToUpper())
+                {
+                    case "MARKET":
+                        order = new MarketOrder { Price = item.Price };
+                        break;
+                    case "LIMIT":
+                    case "LIMIT_MAKER":
+                        order = new LimitOrder { LimitPrice = item.Price };
+                        break;
+                    case "STOP_LOSS":
+                    case "TAKE_PROFIT":
+                        order = new StopMarketOrder { StopPrice = item.StopPrice, Price = item.Price };
+                        break;
+                    case "STOP_LOSS_LIMIT":
+                    case "TAKE_PROFIT_LIMIT":
+                        order = new StopLimitOrder { StopPrice = item.StopPrice, LimitPrice = item.Price };
+                        break;
+                    default:
+                        OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Error, -1,
+                            "BinanceBrokerage.GetOpenOrders: Unsupported order type returned from brokerage: " + item.Type));
+                        continue;
+                }
+
+                order.Quantity = item.Quantity;
+                order.BrokerId = new List<string> { item.Id };
+                order.Symbol = _symbolMapper.GetLeanSymbol(item.Symbol);
+                order.Time = Time.UnixMillisecondTimeStampToDateTime(item.Time);
+                order.Status = ConvertOrderStatus(item.Status);
+                order.Price = item.Price;
+
+                if (order.Status.IsOpen())
+                {
+                    var cached = CachedOrderIDs.Where(c => c.Value.BrokerId.Contains(order.BrokerId.First())).ToList();
+                    if (cached.Any())
+                    {
+                        CachedOrderIDs[cached.First().Key] = order;
+                    }
+                }
+
+                list.Add(order);
+            }
+
+            return list;
+        }
+
+        /// <summary>
+        /// Places a new order and assigns a new broker ID to the order
+        /// </summary>
+        /// <param name="order">The order to be placed</param>
+        /// <returns>True if the request for a new order has been placed, false otherwise</returns>
+        public override bool PlaceOrder(Order order)
+        {
+            var submitted = false;
+
+            WithLockedStream(() =>
+            {
+                submitted = _apiClient.PlaceOrder(order);
+            });
+
+            return submitted;
+        }
+
+        /// <summary>
+        /// Updates the order with the same id
+        /// </summary>
+        /// <param name="order">The new order information</param>
+        /// <returns>True if the request was made for the order to be updated, false otherwise</returns>
+        public override bool UpdateOrder(Order order)
+        {
+            throw new NotSupportedException("BinanceBrokerage.UpdateOrder: Order update not supported. Please cancel and re-create.");
+        }
+
+        /// <summary>
+        /// Cancels the order with the specified ID
+        /// </summary>
+        /// <param name="order">The order to cancel</param>
+        /// <returns>True if the request was submitted for cancellation, false otherwise</returns>
+        public override bool CancelOrder(Order order)
+        {
+            var submitted = false;
+
+            WithLockedStream(() =>
+            {
+                submitted = _apiClient.CancelOrder(order);
+            });
+
+            return submitted;
+        }
+
+        /// <summary>
+        /// Gets the history for the requested security
+        /// </summary>
+        /// <param name="request">The historical data request</param>
+        /// <returns>An enumerable of bars covering the span specified in the request</returns>
+        public override IEnumerable<BaseData> GetHistory(Data.HistoryRequest request)
+        {
+            if (request.Resolution == Resolution.Tick || request.Resolution == Resolution.Second)
+            {
+                OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, "InvalidResolution",
+                    $"{request.Resolution} resolution is not supported, no history returned"));
+                yield break;
+            }
+
+            if (request.TickType != TickType.Trade)
+            {
+                OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, "InvalidTickType",
+                    $"{request.TickType} tick type not supported, no history returned"));
+                yield break;
+            }
+
+            var period = request.Resolution.ToTimeSpan();
+
+            foreach (var kline in _apiClient.GetHistory(request))
+            {
+                yield return new TradeBar()
+                {
+                    Time = Time.UnixMillisecondTimeStampToDateTime(kline.OpenTime),
+                    Symbol = request.Symbol,
+                    Low = kline.Low,
+                    High = kline.High,
+                    Open = kline.Open,
+                    Close = kline.Close,
+                    Volume = kline.Volume,
+                    Value = kline.Close,
+                    DataType = MarketDataType.TradeBar,
+                    Period = period,
+                    EndTime = Time.UnixMillisecondTimeStampToDateTime(kline.OpenTime + (long)period.TotalMilliseconds)
+                };
+            }
+        }
+
+        /// <summary>
+        /// Wss message handler
+        /// </summary>
+        /// <param name="sender"></param>
+        /// <param name="e"></param>
+        public override void OnMessage(object sender, WebSocketMessage e)
+        {
+            try
+            {
+                if (_streamLocked)
+                {
+                    _messageBuffer.Enqueue(e);
+                    return;
+                }
+            }
+            catch (Exception err)
+            {
+                Log.Error(err);
+            }
+
+            OnMessageImpl(e);
+        }
+
+        #endregion
+
+        #region IDataQueueHandler
+
+        /// <summary>
+        /// Sets the job we're subscribing for
+        /// </summary>
+        /// <param name="job">Job we're subscribing for</param>
+        public void SetJob(LiveNodePacket job)
+        {
+        }
+
+        /// <summary>
+        /// Subscribe to the specified configuration
+        /// </summary>
+        /// <param name="dataConfig">defines the parameters to subscribe to a data feed</param>
+        /// <param name="newDataAvailableHandler">handler to be fired on new data available</param>
+        /// <returns>The new enumerator for this subscription request</returns>
+        public IEnumerator<BaseData> Subscribe(SubscriptionDataConfig dataConfig, EventHandler newDataAvailableHandler)
+        {
+            if (!CanSubscribe(dataConfig.Symbol))
+            {
+                return Enumerable.Empty<BaseData>().GetEnumerator();
+            }
+
+            var enumerator = _aggregator.Add(dataConfig, newDataAvailableHandler);
+            SubscriptionManager.Subscribe(dataConfig);
+
+            return enumerator;
+        }
+
+        /// <summary>
+        /// Removes the specified configuration
+        /// </summary>
+        /// <param name="dataConfig">Subscription config to be removed</param>
+        public void Unsubscribe(SubscriptionDataConfig dataConfig)
+        {
+            SubscriptionManager.Unsubscribe(dataConfig);
+            _aggregator.Remove(dataConfig);
+        }
+
+        /// <summary>
+        /// Checks if this brokerage supports the specified symbol
+        /// </summary>
+        /// <param name="symbol">The symbol</param>
+        /// <returns>returns true if brokerage supports the specified symbol; otherwise false</returns>
+        private static bool CanSubscribe(Symbol symbol)
+        {
+            return !symbol.Value.Contains("UNIVERSE") &&
+                   symbol.SecurityType == SecurityType.Crypto;
+        }
+
+        #endregion
+
+        /// <summary>
+        /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
+        /// </summary>
+        public override void Dispose()
+        {
+            _keepAliveTimer.DisposeSafely();
+            _reconnectTimer.DisposeSafely();
+            _apiClient.DisposeSafely();
+            _webSocketRateLimiter.DisposeSafely();
+        }
+
+        /// <summary>
+        /// Subscribes to the requested symbols (using an individual streaming channel)
+        /// </summary>
+        /// <param name="symbols">The list of symbols to subscribe</param>
+        public override void Subscribe(IEnumerable<Symbol> symbols)
+        {
+            foreach (var symbol in symbols)
+            {
+                Send(WebSocket,
+                    new
+                    {
+                        method = "SUBSCRIBE",
+                        @params = new[]
+                        {
+                            $"{symbol.Value.ToLowerInvariant()}@trade",
+                            $"{symbol.Value.ToLowerInvariant()}@bookTicker"
+                        },
+                        id = GetNextRequestId()
+                    }
+                );
+            }
+        }
+
+        /// <summary>
+        /// Ends current subscriptions
+        /// </summary>
+        private bool Unsubscribe(IEnumerable<Symbol> symbols)
+        {
+            if (WebSocket.IsOpen)
+            {
+                foreach (var symbol in symbols)
+                {
+                    Send(WebSocket,
+                        new
+                        {
+                            method = "UNSUBSCRIBE",
+                            @params = new[]
+                            {
+                                $"{symbol.Value.ToLowerInvariant()}@trade",
+                                $"{symbol.Value.ToLowerInvariant()}@bookTicker"
+                            },
+                            id = GetNextRequestId()
+                        }
+                    );
+                }
+            }
+
+            return true;
+        }
+
+        private void Send(IWebSocket webSocket, object obj)
+        {
+            var json = JsonConvert.SerializeObject(obj);
+
+            if (!_webSocketRateLimiter.WaitToProceed(TimeSpan.Zero))
+            {
+                _webSocketRateLimiter.WaitToProceed();
+            }
+
+            Log.Trace("Send: " + json);
+
+            webSocket.Send(json);
+        }
+
+        private long GetNextRequestId()
+        {
+            return Interlocked.Increment(ref _lastRequestId);
+        }
+
+        /// <summary>
+        /// Event invocator for the OrderFilled event
+        /// </summary>
+        /// <param name="e">The OrderEvent</param>
+        private void OnOrderSubmit(BinanceOrderSubmitEventArgs e)
+        {
+            var brokerId = e.BrokerId;
+            var order = e.Order;
+            if (CachedOrderIDs.ContainsKey(order.Id))
+            {
+                CachedOrderIDs[order.Id].BrokerId.Clear();
+                CachedOrderIDs[order.Id].BrokerId.Add(brokerId);
+            }
+            else
+            {
+                order.BrokerId.Add(brokerId);
+                CachedOrderIDs.TryAdd(order.Id, order);
+            }
+        }
+    }
+}

Brokerages/Binance/BinanceBrokerageFactory.cs

@@ -0,0 +1,88 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using System.Collections.Generic;
+using QuantConnect.Configuration;
+using QuantConnect.Data;
+using QuantConnect.Interfaces;
+using QuantConnect.Securities;
+using QuantConnect.Util;
+
+namespace QuantConnect.Brokerages.Binance
+{
+    /// <summary>
+    /// Factory method to create binance Websockets brokerage
+    /// </summary>
+    public class BinanceBrokerageFactory : BrokerageFactory
+    {
+        /// <summary>
+        /// Factory constructor
+        /// </summary>
+        public BinanceBrokerageFactory() : base(typeof(BinanceBrokerage))
+        {
+        }
+
+        /// <summary>
+        /// Not required
+        /// </summary>
+        public override void Dispose()
+        {
+        }
+
+        /// <summary>
+        /// provides brokerage connection data
+        /// </summary>
+        public override Dictionary<string, string> BrokerageData => new Dictionary<string, string>
+        {
+            { "binance-api-key", Config.Get("binance-api-key")},
+            { "binance-api-secret", Config.Get("binance-api-secret")}
+        };
+
+        /// <summary>
+        /// The brokerage model
+        /// </summary>
+        /// <param name="orderProvider">The order provider</param>
+        public override IBrokerageModel GetBrokerageModel(IOrderProvider orderProvider) => new BinanceBrokerageModel();
+
+        /// <summary>
+        /// Create the Brokerage instance
+        /// </summary>
+        /// <param name="job"></param>
+        /// <param name="algorithm"></param>
+        /// <returns></returns>
+        public override IBrokerage CreateBrokerage(Packets.LiveNodePacket job, IAlgorithm algorithm)
+        {
+            var required = new[] { "binance-api-secret", "binance-api-key" };
+
+            foreach (var item in required)
+            {
+                if (string.IsNullOrEmpty(job.BrokerageData[item]))
+                {
+                    throw new Exception($"BinanceBrokerageFactory.CreateBrokerage: Missing {item} in config.json");
+                }
+            }
+
+            var brokerage = new BinanceBrokerage(
+                job.BrokerageData["binance-api-key"],
+                job.BrokerageData["binance-api-secret"],
+                algorithm,
+                Composer.Instance.GetExportedValueByTypeName<IDataAggregator>(Config.Get("data-aggregator", "QuantConnect.Lean.Engine.DataFeeds.AggregationManager")));
+            Composer.Instance.AddPart<IDataQueueHandler>(brokerage);
+
+            return brokerage;
+        }
+    }
+}

Brokerages/Binance/BinanceOrderSubmitEventArgs.cs

@@ -0,0 +1,46 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using QuantConnect.Orders;
+
+namespace QuantConnect.Brokerages.Binance
+{
+    /// <summary>
+    /// Represents a binance submit order event data
+    /// </summary>
+    public class BinanceOrderSubmitEventArgs
+    {
+        /// <summary>
+        /// Order Event Constructor.
+        /// </summary>
+        /// <param name="brokerId">Binance order id returned from brokerage</param>
+        /// <param name="order">Order for this order placement</param>
+        public BinanceOrderSubmitEventArgs(string brokerId, Order order)
+        {
+            BrokerId = brokerId;
+            Order = order;
+        }
+
+        /// <summary>
+        /// Original brokerage id
+        /// </summary>
+        public string BrokerId { get; set; }
+
+        /// <summary>
+        /// The lean order
+        /// </summary>
+        public Order Order { get; set; }
+    }
+}

Brokerages/Binance/BinanceRestApiClient.cs

@@ -0,0 +1,590 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using Newtonsoft.Json;
+using Newtonsoft.Json.Linq;
+using QuantConnect.Logging;
+using QuantConnect.Orders;
+using QuantConnect.Orders.Fees;
+using QuantConnect.Securities;
+using QuantConnect.Util;
+using RestSharp;
+using System;
+using System.Collections.Generic;
+using System.Globalization;
+using System.Linq;
+using System.Net;
+using System.Security.Cryptography;
+using System.Text;
+
+namespace QuantConnect.Brokerages.Binance
+{
+    /// <summary>
+    /// Binance REST API implementation
+    /// </summary>
+    public class BinanceRestApiClient : IDisposable
+    {
+        private const string RestApiUrl = "https://api.binance.com";
+        private const string UserDataStreamEndpoint = "/api/v3/userDataStream";
+
+        private readonly BinanceSymbolMapper _symbolMapper;
+        private readonly ISecurityProvider _securityProvider;
+        private readonly IRestClient _restClient;
+        private readonly RateGate _restRateLimiter = new RateGate(10, TimeSpan.FromSeconds(1));
+        private readonly object _listenKeyLocker = new object();
+
+        /// <summary>
+        /// Event that fires each time an order is filled
+        /// </summary>
+        public event EventHandler<BinanceOrderSubmitEventArgs> OrderSubmit;
+
+        /// <summary>
+        /// Event that fires each time an order is filled
+        /// </summary>
+        public event EventHandler<OrderEvent> OrderStatusChanged;
+
+        /// <summary>
+        /// Event that fires when an error is encountered in the brokerage
+        /// </summary>
+        public event EventHandler<BrokerageMessageEvent> Message;
+
+        /// <summary>
+        /// Key Header
+        /// </summary>
+        public readonly string KeyHeader = "X-MBX-APIKEY";
+
+        /// <summary>
+        /// The api secret
+        /// </summary>
+        protected string ApiSecret;
+
+        /// <summary>
+        /// The api key
+        /// </summary>
+        protected string ApiKey;
+
+        /// <summary>
+        /// Represents UserData Session listen key
+        /// </summary>
+        public string SessionId { get; private set; }
+
+        /// <summary>
+        /// Initializes a new instance of the <see cref="BinanceRestApiClient"/> class.
+        /// </summary>
+        /// <param name="symbolMapper">The symbol mapper.</param>
+        /// <param name="securityProvider">The holdings provider.</param>
+        /// <param name="apiKey">The Binance API key</param>
+        /// <param name="apiSecret">The The Binance API secret</param>
+        public BinanceRestApiClient(BinanceSymbolMapper symbolMapper, ISecurityProvider securityProvider, string apiKey, string apiSecret)
+        {
+            _symbolMapper = symbolMapper;
+            _securityProvider = securityProvider;
+            _restClient = new RestClient(RestApiUrl);
+            ApiKey = apiKey;
+            ApiSecret = apiSecret;
+        }
+
+        /// <summary>
+        /// Gets all open positions
+        /// </summary>
+        /// <returns></returns>
+        public List<Holding> GetAccountHoldings()
+        {
+            return new List<Holding>();
+        }
+
+        /// <summary>
+        /// Gets the total account cash balance for specified account type
+        /// </summary>
+        /// <returns></returns>
+        public Messages.AccountInformation GetCashBalance()
+        {
+            var queryString = $"timestamp={GetNonce()}";
+            var endpoint = $"/api/v3/account?{queryString}&signature={AuthenticationToken(queryString)}";
+            var request = new RestRequest(endpoint, Method.GET);
+            request.AddHeader(KeyHeader, ApiKey);
+
+            var response = ExecuteRestRequest(request);
+            if (response.StatusCode != HttpStatusCode.OK)
+            {
+                throw new Exception($"BinanceBrokerage.GetCashBalance: request failed: [{(int)response.StatusCode}] {response.StatusDescription}, Content: {response.Content}, ErrorMessage: {response.ErrorMessage}");
+            }
+
+            return JsonConvert.DeserializeObject<Messages.AccountInformation>(response.Content);
+        }
+
+        /// <summary>
+        /// Gets all orders not yet closed
+        /// </summary>
+        /// <returns></returns>
+        public IEnumerable<Messages.OpenOrder> GetOpenOrders()
+        {
+            var queryString = $"timestamp={GetNonce()}";
+            var endpoint = $"/api/v3/openOrders?{queryString}&signature={AuthenticationToken(queryString)}";
+            var request = new RestRequest(endpoint, Method.GET);
+            request.AddHeader(KeyHeader, ApiKey);
+
+            var response = ExecuteRestRequest(request);
+            if (response.StatusCode != HttpStatusCode.OK)
+            {
+                throw new Exception($"BinanceBrokerage.GetCashBalance: request failed: [{(int)response.StatusCode}] {response.StatusDescription}, Content: {response.Content}, ErrorMessage: {response.ErrorMessage}");
+            }
+
+            return JsonConvert.DeserializeObject<Messages.OpenOrder[]>(response.Content);
+        }
+
+        /// <summary>
+        /// Places a new order and assigns a new broker ID to the order
+        /// </summary>
+        /// <param name="order">The order to be placed</param>
+        /// <returns>True if the request for a new order has been placed, false otherwise</returns>
+        public bool PlaceOrder(Order order)
+        {
+            // supported time in force values {GTC, IOC, FOK}
+            // use GTC as LEAN doesn't support others yet
+            IDictionary<string, object> body = new Dictionary<string, object>()
+            {
+                { "symbol", _symbolMapper.GetBrokerageSymbol(order.Symbol) },
+                { "quantity", Math.Abs(order.Quantity).ToString(CultureInfo.InvariantCulture) },
+                { "side", ConvertOrderDirection(order.Direction) }
+            };
+
+            switch (order.Type)
+            {
+                case OrderType.Limit:
+                    body["type"] = (order.Properties as BinanceOrderProperties)?.PostOnly == true
+                        ? "LIMIT_MAKER"
+                        : "LIMIT";
+                    body["price"] = ((LimitOrder) order).LimitPrice.ToString(CultureInfo.InvariantCulture);
+                    // timeInForce is not required for LIMIT_MAKER
+                    if (Equals(body["type"], "LIMIT"))
+                        body["timeInForce"] = "GTC";
+                    break;
+                case OrderType.Market:
+                    body["type"] = "MARKET";
+                    break;
+                case OrderType.StopLimit:
+                    var ticker = GetTickerPrice(order);
+                    var stopPrice = ((StopLimitOrder) order).StopPrice;
+                    if (order.Direction == OrderDirection.Sell)
+                    {
+                        body["type"] = stopPrice <= ticker ? "STOP_LOSS_LIMIT" : "TAKE_PROFIT_LIMIT";
+                    }
+                    else
+                    {
+                        body["type"] = stopPrice <= ticker ? "TAKE_PROFIT_LIMIT" : "STOP_LOSS_LIMIT";
+                    }
+                    body["timeInForce"] = "GTC";
+                    body["stopPrice"] = stopPrice.ToString(CultureInfo.InvariantCulture);
+                    body["price"] = ((StopLimitOrder) order).LimitPrice.ToString(CultureInfo.InvariantCulture);
+                    break;
+                default:
+                    throw new NotSupportedException($"BinanceBrokerage.ConvertOrderType: Unsupported order type: {order.Type}");
+            }
+
+            const string endpoint = "/api/v3/order";
+            body["timestamp"] = GetNonce();
+            body["signature"] = AuthenticationToken(body.ToQueryString());
+            var request = new RestRequest(endpoint, Method.POST);
+            request.AddHeader(KeyHeader, ApiKey);
+            request.AddParameter(
+                "application/x-www-form-urlencoded",
+                Encoding.UTF8.GetBytes(body.ToQueryString()),
+                ParameterType.RequestBody
+            );
+
+            var response = ExecuteRestRequest(request);
+            if (response.StatusCode == HttpStatusCode.OK)
+            {
+                var raw = JsonConvert.DeserializeObject<Messages.NewOrder>(response.Content);
+
+                if (string.IsNullOrEmpty(raw?.Id))
+                {
+                    var errorMessage = $"Error parsing response from place order: {response.Content}";
+                    OnOrderEvent(new OrderEvent(
+                        order,
+                        DateTime.UtcNow,
+                        OrderFee.Zero,
+                        "Binance Order Event")
+                    { Status = OrderStatus.Invalid, Message = errorMessage });
+                    OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, (int)response.StatusCode, errorMessage));
+
+                    return true;
+                }
+
+                OnOrderSubmit(raw, order);
+                return true;
+            }
+
+            var message = $"Order failed, Order Id: {order.Id} timestamp: {order.Time} quantity: {order.Quantity} content: {response.Content}";
+            OnOrderEvent(new OrderEvent(
+                order,
+                DateTime.UtcNow,
+                OrderFee.Zero,
+                "Binance Order Event")
+            { Status = OrderStatus.Invalid });
+            OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, -1, message));
+
+            return true;
+        }
+
+        /// <summary>
+        /// Cancels the order with the specified ID
+        /// </summary>
+        /// <param name="order">The order to cancel</param>
+        /// <returns>True if the request was submitted for cancellation, false otherwise</returns>
+        public bool CancelOrder(Order order)
+        {
+            var success = new List<bool>();
+            IDictionary<string, object> body = new Dictionary<string, object>()
+            {
+                { "symbol", _symbolMapper.GetBrokerageSymbol(order.Symbol) }
+            };
+            foreach (var id in order.BrokerId)
+            {
+                if (body.ContainsKey("signature"))
+                {
+                    body.Remove("signature");
+                }
+                body["orderId"] = id;
+                body["timestamp"] = GetNonce();
+                body["signature"] = AuthenticationToken(body.ToQueryString());
+
+                var request = new RestRequest("/api/v3/order", Method.DELETE);
+                request.AddHeader(KeyHeader, ApiKey);
+                request.AddParameter(
+                    "application/x-www-form-urlencoded",
+                    Encoding.UTF8.GetBytes(body.ToQueryString()),
+                    ParameterType.RequestBody
+                );
+
+                var response = ExecuteRestRequest(request);
+                success.Add(response.StatusCode == HttpStatusCode.OK);
+            }
+
+            var canceled = false;
+            if (success.All(a => a))
+            {
+                OnOrderEvent(new OrderEvent(order,
+                    DateTime.UtcNow,
+                    OrderFee.Zero,
+                    "Binance Order Event")
+                { Status = OrderStatus.Canceled });
+
+                canceled = true;
+            }
+            return canceled;
+        }
+
+        /// <summary>
+        /// Gets the history for the requested security
+        /// </summary>
+        /// <param name="request">The historical data request</param>
+        /// <returns>An enumerable of bars covering the span specified in the request</returns>
+        public IEnumerable<Messages.Kline> GetHistory(Data.HistoryRequest request)
+        {
+            var resolution = ConvertResolution(request.Resolution);
+            var resolutionInMs = (long)request.Resolution.ToTimeSpan().TotalMilliseconds;
+            var symbol = _symbolMapper.GetBrokerageSymbol(request.Symbol);
+            var startMs = (long)Time.DateTimeToUnixTimeStamp(request.StartTimeUtc) * 1000;
+            var endMs = (long)Time.DateTimeToUnixTimeStamp(request.EndTimeUtc) * 1000;
+            var endpoint = $"/api/v3/klines?symbol={symbol}&interval={resolution}&limit=1000";
+
+            while (endMs - startMs >= resolutionInMs)
+            {
+                var timeframe = $"&startTime={startMs}&endTime={endMs}";
+
+                var restRequest = new RestRequest(endpoint + timeframe, Method.GET);
+                var response = ExecuteRestRequest(restRequest);
+
+                if (response.StatusCode != HttpStatusCode.OK)
+                {
+                    throw new Exception($"BinanceBrokerage.GetHistory: request failed: [{(int)response.StatusCode}] {response.StatusDescription}, Content: {response.Content}, ErrorMessage: {response.ErrorMessage}");
+                }
+
+                var klines = JsonConvert.DeserializeObject<object[][]>(response.Content)
+                    .Select(entries => new Messages.Kline(entries))
+                    .ToList();
+
+                startMs = klines.Last().OpenTime + resolutionInMs;
+
+                foreach (var kline in klines)
+                {
+                    yield return kline;
+                }
+            }
+        }
+
+        /// <summary>
+        /// Check User Data stream listen key is alive
+        /// </summary>
+        /// <returns></returns>
+        public bool SessionKeepAlive()
+        {
+            if (string.IsNullOrEmpty(SessionId))
+            {
+                throw new Exception("BinanceBrokerage:UserStream. listenKey wasn't allocated or has been refused.");
+            }
+
+            var ping = new RestRequest(UserDataStreamEndpoint, Method.PUT);
+            ping.AddHeader(KeyHeader, ApiKey);
+            ping.AddParameter(
+                "application/x-www-form-urlencoded",
+                Encoding.UTF8.GetBytes($"listenKey={SessionId}"),
+                ParameterType.RequestBody
+            );
+
+            var pong = ExecuteRestRequest(ping);
+            return pong.StatusCode == HttpStatusCode.OK;
+        }
+
+        /// <summary>
+        /// Stops the session
+        /// </summary>
+        public void StopSession()
+        {
+            if (string.IsNullOrEmpty(SessionId))
+            {
+                throw new Exception("BinanceBrokerage:UserStream. listenKey wasn't allocated or has been refused.");
+            }
+
+            var request = new RestRequest(UserDataStreamEndpoint, Method.DELETE);
+            request.AddHeader(KeyHeader, ApiKey);
+            request.AddParameter(
+                "application/x-www-form-urlencoded",
+                Encoding.UTF8.GetBytes($"listenKey={SessionId}"),
+                ParameterType.RequestBody
+            );
+            ExecuteRestRequest(request);
+        }
+
+        /// <summary>
+        /// Provides the current tickers price
+        /// </summary>
+        /// <returns></returns>
+        public Messages.PriceTicker[] GetTickers()
+        {
+            const string endpoint = "/api/v3/ticker/price";
+            var req = new RestRequest(endpoint, Method.GET);
+            var response = ExecuteRestRequest(req);
+            if (response.StatusCode != HttpStatusCode.OK)
+            {
+                throw new Exception($"BinanceBrokerage.GetTick: request failed: [{(int)response.StatusCode}] {response.StatusDescription}, Content: {response.Content}, ErrorMessage: {response.ErrorMessage}");
+            }
+
+            return JsonConvert.DeserializeObject<Messages.PriceTicker[]>(response.Content);
+        }
+
+        /// <summary>
+        /// Start user data stream
+        /// </summary>
+        public void CreateListenKey()
+        {
+            var request = new RestRequest(UserDataStreamEndpoint, Method.POST);
+            request.AddHeader(KeyHeader, ApiKey);
+
+            var response = ExecuteRestRequest(request);
+            if (response.StatusCode != HttpStatusCode.OK)
+            {
+                throw new Exception($"BinanceBrokerage.StartSession: request failed: [{(int)response.StatusCode}] {response.StatusDescription}, Content: {response.Content}, ErrorMessage: {response.ErrorMessage}");
+            }
+
+            var content = JObject.Parse(response.Content);
+            lock (_listenKeyLocker)
+            {
+                SessionId = content.Value<string>("listenKey");
+            }
+        }
+
+        /// <summary>
+        /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
+        /// </summary>
+        public void Dispose()
+        {
+            _restRateLimiter.DisposeSafely();
+        }
+
+        /// <summary>
+        /// If an IP address exceeds a certain number of requests per minute
+        /// HTTP 429 return code is used when breaking a request rate limit.
+        /// </summary>
+        /// <param name="request"></param>
+        /// <returns></returns>
+        private IRestResponse ExecuteRestRequest(IRestRequest request)
+        {
+            const int maxAttempts = 10;
+            var attempts = 0;
+            IRestResponse response;
+
+            do
+            {
+                if (!_restRateLimiter.WaitToProceed(TimeSpan.Zero))
+                {
+                    Log.Trace("Brokerage.OnMessage(): " + new BrokerageMessageEvent(BrokerageMessageType.Warning, "RateLimit",
+                        "The API request has been rate limited. To avoid this message, please reduce the frequency of API calls."));
+
+                    _restRateLimiter.WaitToProceed();
+                }
+
+                response = _restClient.Execute(request);
+                // 429 status code: Too Many Requests
+            } while (++attempts < maxAttempts && (int)response.StatusCode == 429);
+
+            return response;
+        }
+
+        private decimal GetTickerPrice(Order order)
+        {
+            var security = _securityProvider.GetSecurity(order.Symbol);
+            var tickerPrice = order.Direction == OrderDirection.Buy ? security.AskPrice : security.BidPrice;
+            if (tickerPrice == 0)
+            {
+                var brokerageSymbol = _symbolMapper.GetBrokerageSymbol(order.Symbol);
+                var tickers = GetTickers();
+                var ticker = tickers.FirstOrDefault(t => t.Symbol == brokerageSymbol);
+                if (ticker == null)
+                {
+                    throw new KeyNotFoundException($"BinanceBrokerage: Unable to resolve currency conversion pair: {order.Symbol}");
+                }
+                tickerPrice = ticker.Price;
+            }
+            return tickerPrice;
+        }
+
+        /// <summary>
+        /// Timestamp in milliseconds
+        /// </summary>
+        /// <returns></returns>
+        private long GetNonce()
+        {
+            return (long)(Time.TimeStamp() * 1000);
+        }
+
+        /// <summary>
+        /// Creates a signature for signed endpoints
+        /// </summary>
+        /// <param name="payload">the body of the request</param>
+        /// <returns>a token representing the request params</returns>
+        private string AuthenticationToken(string payload)
+        {
+            using (HMACSHA256 hmac = new HMACSHA256(Encoding.UTF8.GetBytes(ApiSecret)))
+            {
+                return hmac.ComputeHash(Encoding.UTF8.GetBytes(payload)).ToHexString();
+            }
+        }
+
+        private static string ConvertOrderDirection(OrderDirection orderDirection)
+        {
+            if (orderDirection == OrderDirection.Buy || orderDirection == OrderDirection.Sell)
+            {
+                return orderDirection.ToString().LazyToUpper();
+            }
+
+            throw new NotSupportedException($"BinanceBrokerage.ConvertOrderDirection: Unsupported order direction: {orderDirection}");
+        }
+
+
+        private readonly Dictionary<Resolution, string> _knownResolutions = new Dictionary<Resolution, string>()
+        {
+            { Resolution.Minute, "1m" },
+            { Resolution.Hour,   "1h" },
+            { Resolution.Daily,  "1d" }
+        };
+
+        private string ConvertResolution(Resolution resolution)
+        {
+            if (_knownResolutions.ContainsKey(resolution))
+            {
+                return _knownResolutions[resolution];
+            }
+            else
+            {
+                throw new ArgumentException($"BinanceBrokerage.ConvertResolution: Unsupported resolution type: {resolution}");
+            }
+        }
+
+        /// <summary>
+        /// Event invocator for the OrderFilled event
+        /// </summary>
+        /// <param name="newOrder">The brokerage order submit result</param>
+        /// <param name="order">The lean order</param>
+        private void OnOrderSubmit(Messages.NewOrder newOrder, Order order)
+        {
+            try
+            {
+                OrderSubmit?.Invoke(
+                    this,
+                    new BinanceOrderSubmitEventArgs(newOrder.Id, order));
+
+                // Generate submitted event
+                OnOrderEvent(new OrderEvent(
+                    order,
+                    Time.UnixMillisecondTimeStampToDateTime(newOrder.TransactionTime),
+                    OrderFee.Zero,
+                    "Binance Order Event")
+                { Status = OrderStatus.Submitted }
+                );
+                Log.Trace($"Order submitted successfully - OrderId: {order.Id}");
+            }
+            catch (Exception err)
+            {
+                Log.Error(err);
+            }
+        }
+
+        /// <summary>
+        /// Event invocator for the OrderFilled event
+        /// </summary>
+        /// <param name="e">The OrderEvent</param>
+        private void OnOrderEvent(OrderEvent e)
+        {
+            try
+            {
+                Log.Debug("Brokerage.OnOrderEvent(): " + e);
+
+                OrderStatusChanged?.Invoke(this, e);
+            }
+            catch (Exception err)
+            {
+                Log.Error(err);
+            }
+        }
+
+        /// <summary>
+        /// Event invocator for the Message event
+        /// </summary>
+        /// <param name="e">The error</param>
+        protected virtual void OnMessage(BrokerageMessageEvent e)
+        {
+            try
+            {
+                if (e.Type == BrokerageMessageType.Error)
+                {
+                    Log.Error("Brokerage.OnMessage(): " + e);
+                }
+                else
+                {
+                    Log.Trace("Brokerage.OnMessage(): " + e);
+                }
+
+                Message?.Invoke(this, e);
+            }
+            catch (Exception err)
+            {
+                Log.Error(err);
+            }
+        }
+    }
+}

Brokerages/Binance/BinanceSymbolMapper.cs

@@ -0,0 +1,174 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using QuantConnect.Securities;
+
+namespace QuantConnect.Brokerages.Binance
+{
+    /// <summary>
+    /// Provides the mapping between Lean symbols and Binance symbols.
+    /// </summary>
+    public class BinanceSymbolMapper : ISymbolMapper
+    {
+        /// <summary>
+        /// The list of known Binance symbols.
+        /// </summary>
+        public static readonly HashSet<string> KnownTickers =
+            new HashSet<string>(SymbolPropertiesDatabase
+                .FromDataFolder()
+                .GetSymbolPropertiesList(Market.Binance, SecurityType.Crypto)
+                .Select(x => x.Key.Symbol));
+
+        /// <summary>
+        /// Converts a Lean symbol instance to an Binance symbol
+        /// </summary>
+        /// <param name="symbol">A Lean symbol instance</param>
+        /// <returns>The Binance symbol</returns>
+        public string GetBrokerageSymbol(Symbol symbol)
+        {
+            if (symbol == null || string.IsNullOrWhiteSpace(symbol.Value))
+                throw new ArgumentException("Invalid symbol: " + (symbol == null ? "null" : symbol.ToString()));
+
+            if (symbol.SecurityType != SecurityType.Crypto)
+                throw new ArgumentException("Invalid security type: " + symbol.SecurityType);
+
+            var brokerageSymbol = ConvertLeanSymbolToBrokerageSymbol(symbol.Value);
+
+            if (!IsKnownBrokerageSymbol(brokerageSymbol))
+                throw new ArgumentException("Unknown symbol: " + symbol.Value);
+
+            return brokerageSymbol;
+        }
+
+        /// <summary>
+        /// Converts an Binance symbol to a Lean symbol instance
+        /// </summary>
+        /// <param name="brokerageSymbol">The Binance symbol</param>
+        /// <param name="securityType">The security type</param>
+        /// <param name="market">The market</param>
+        /// <param name="expirationDate">Expiration date of the security(if applicable)</param>
+        /// <param name="strike">The strike of the security (if applicable)</param>
+        /// <param name="optionRight">The option right of the security (if applicable)</param>
+        /// <returns>A new Lean Symbol instance</returns>
+        public Symbol GetLeanSymbol(string brokerageSymbol, SecurityType securityType, string market, DateTime expirationDate = default(DateTime), decimal strike = 0, OptionRight optionRight = 0)
+        {
+            if (string.IsNullOrWhiteSpace(brokerageSymbol))
+                throw new ArgumentException($"Invalid Binance symbol: {brokerageSymbol}");
+
+            if (!IsKnownBrokerageSymbol(brokerageSymbol))
+                throw new ArgumentException($"Unknown Binance symbol: {brokerageSymbol}");
+
+            if (securityType != SecurityType.Crypto)
+                throw new ArgumentException($"Invalid security type: {securityType}");
+
+            if (market != Market.Binance)
+                throw new ArgumentException($"Invalid market: {market}");
+
+            return Symbol.Create(ConvertBrokerageSymbolToLeanSymbol(brokerageSymbol), GetBrokerageSecurityType(brokerageSymbol), Market.Binance);
+        }
+
+        /// <summary>
+        /// Converts an Binance symbol to a Lean symbol instance
+        /// </summary>
+        /// <param name="brokerageSymbol">The Binance symbol</param>
+        /// <returns>A new Lean Symbol instance</returns>
+        public Symbol GetLeanSymbol(string brokerageSymbol)
+        {
+            var securityType = GetBrokerageSecurityType(brokerageSymbol);
+            return GetLeanSymbol(brokerageSymbol, securityType, Market.Binance);
+        }
+
+        /// <summary>
+        /// Returns the security type for an Binance symbol
+        /// </summary>
+        /// <param name="brokerageSymbol">The Binance symbol</param>
+        /// <returns>The security type</returns>
+        public SecurityType GetBrokerageSecurityType(string brokerageSymbol)
+        {
+            if (string.IsNullOrWhiteSpace(brokerageSymbol))
+                throw new ArgumentException($"Invalid Binance symbol: {brokerageSymbol}");
+
+            if (!IsKnownBrokerageSymbol(brokerageSymbol))
+                throw new ArgumentException($"Unknown Binance symbol: {brokerageSymbol}");
+
+            return SecurityType.Crypto;
+        }
+
+        /// <summary>
+        /// Returns the security type for a Lean symbol
+        /// </summary>
+        /// <param name="leanSymbol">The Lean symbol</param>
+        /// <returns>The security type</returns>
+        public SecurityType GetLeanSecurityType(string leanSymbol)
+        {
+            return GetBrokerageSecurityType(ConvertLeanSymbolToBrokerageSymbol(leanSymbol));
+        }
+
+        /// <summary>
+        /// Checks if the symbol is supported by Binance
+        /// </summary>
+        /// <param name="brokerageSymbol">The Binance symbol</param>
+        /// <returns>True if Binance supports the symbol</returns>
+        public bool IsKnownBrokerageSymbol(string brokerageSymbol)
+        {
+            if (string.IsNullOrWhiteSpace(brokerageSymbol))
+                return false;
+
+            return KnownTickers.Contains(brokerageSymbol);
+        }
+
+        /// <summary>
+        /// Checks if the symbol is supported by Binance
+        /// </summary>
+        /// <param name="symbol">The Lean symbol</param>
+        /// <returns>True if Binance supports the symbol</returns>
+        public bool IsKnownLeanSymbol(Symbol symbol)
+        {
+            if (string.IsNullOrWhiteSpace(symbol?.Value) || symbol.Value.Length <= 3)
+                return false;
+
+            var binanceSymbol = ConvertLeanSymbolToBrokerageSymbol(symbol.Value);
+
+            return IsKnownBrokerageSymbol(binanceSymbol) && GetBrokerageSecurityType(binanceSymbol) == symbol.SecurityType;
+        }
+
+        /// <summary>
+        /// Converts an Binance symbol to a Lean symbol string
+        /// </summary>
+        private static string ConvertBrokerageSymbolToLeanSymbol(string binanceSymbol)
+        {
+            if (string.IsNullOrWhiteSpace(binanceSymbol))
+                throw new ArgumentException($"Invalid Binance symbol: {binanceSymbol}");
+
+            // return as it is due to Binance has similar Symbol format
+            return binanceSymbol.LazyToUpper();
+        }
+
+        /// <summary>
+        /// Converts a Lean symbol string to an Binance symbol
+        /// </summary>
+        private static string ConvertLeanSymbolToBrokerageSymbol(string leanSymbol)
+        {
+            if (string.IsNullOrWhiteSpace(leanSymbol))
+                throw new ArgumentException($"Invalid Lean symbol: {leanSymbol}");
+
+            // return as it is due to Binance has similar Symbol format
+            return leanSymbol.LazyToUpper();
+        }
+    }
+}

Brokerages/Binance/BinanceUtil.cs

@@ -0,0 +1,60 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using Newtonsoft.Json;
+using QuantConnect.Orders;
+using RestSharp;
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Net;
+using System.Security.Cryptography;
+using System.Text;
+
+namespace QuantConnect.Brokerages.Binance
+{
+    /// <summary>
+    /// Binance utility methods
+    /// </summary>
+    public class BinanceUtil
+    {
+        /// <summary>
+        /// Convert binance status string value to native Lean OrderStatus
+        /// </summary>
+        /// <param name="status">The Binance order status value</param>
+        /// <returns>Lean order status</returns>
+        public static OrderStatus ConvertOrderStatus(string status)
+        {
+            switch (status.LazyToUpper())
+            {
+                case "NEW":
+                    return OrderStatus.New;
+                case "PARTIALLY_FILLED":
+                    return OrderStatus.PartiallyFilled;
+                case "FILLED":
+                    return OrderStatus.Filled;
+                case "PENDING_CANCEL":
+                    return OrderStatus.CancelPending;
+                case "CANCELED":
+                    return OrderStatus.Canceled;
+                case "REJECTED":
+                case "EXPIRED":
+                    return OrderStatus.Invalid;
+                default:
+                    return Orders.OrderStatus.None;
+            }
+        }
+    }
+}

Brokerages/Binance/BinanceWebSocketWrapper.cs

@@ -0,0 +1,44 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+
+namespace QuantConnect.Brokerages.Binance
+{
+    /// <summary>
+    /// Wrapper class for a Binance websocket connection
+    /// </summary>
+    public class BinanceWebSocketWrapper : WebSocketClientWrapper
+    {
+        /// <summary>
+        /// The unique Id for the connection
+        /// </summary>
+        public string ConnectionId { get; }
+
+        /// <summary>
+        /// The handler for the connection
+        /// </summary>
+        public IConnectionHandler ConnectionHandler { get; }
+
+        /// <summary>
+        /// Initializes a new instance of the <see cref="BinanceWebSocketWrapper"/> class.
+        /// </summary>
+        public BinanceWebSocketWrapper(IConnectionHandler connectionHandler)
+        {
+            ConnectionId = Guid.NewGuid().ToString();
+            ConnectionHandler = connectionHandler;
+        }
+    }
+}

Brokerages/Binance/Messages.cs

@@ -0,0 +1,209 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using Newtonsoft.Json;
+using QuantConnect.Orders;
+using System;
+using System.Globalization;
+
+namespace QuantConnect.Brokerages.Binance.Messages
+{
+#pragma warning disable 1591
+
+    public class AccountInformation
+    {
+        public Balance[] Balances { get; set; }
+
+        public class Balance
+        {
+            public string Asset { get; set; }
+            public decimal Free { get; set; }
+            public decimal Locked { get; set; }
+            public decimal Amount => Free + Locked;
+        }
+    }
+
+    public class PriceTicker
+    {
+        public string Symbol { get; set; }
+        public decimal Price { get; set; }
+    }
+
+    public class Order
+    {
+        [JsonProperty("orderId")]
+        public string Id { get; set; }
+        public string Symbol { get; set; }
+        public decimal Price { get; set; }
+        public decimal StopPrice { get; set; }
+        [JsonProperty("origQty")]
+        public decimal OriginalAmount { get; set; }
+        [JsonProperty("executedQty")]
+        public decimal ExecutedAmount { get; set; }
+        public string Status { get; set; }
+        public string Type { get; set; }
+        public string Side { get; set; }
+
+        public decimal Quantity => string.Equals(Side, "buy", StringComparison.OrdinalIgnoreCase) ? OriginalAmount : -OriginalAmount;
+    }
+
+    public class OpenOrder : Order
+    {
+        public long Time { get; set; }
+    }
+
+    public class NewOrder : Order
+    {
+        [JsonProperty("transactTime")]
+        public long TransactionTime { get; set; }
+    }
+
+    public enum EventType
+    {
+        None,
+        OrderBook,
+        Trade,
+        Execution
+    }
+
+    public class ErrorMessage
+    {
+        [JsonProperty("code")]
+        public int Code { get; set; }
+
+        [JsonProperty("msg")]
+        public string Message { get; set; }
+    }
+
+    public class BestBidAskQuote
+    {
+        [JsonProperty("u")]
+        public long OrderBookUpdateId { get; set; }
+
+        [JsonProperty("s")]
+        public string Symbol { get; set; }
+
+        [JsonProperty("b")]
+        public decimal BestBidPrice { get; set; }
+
+        [JsonProperty("B")]
+        public decimal BestBidSize { get; set; }
+
+        [JsonProperty("a")]
+        public decimal BestAskPrice { get; set; }
+
+        [JsonProperty("A")]
+        public decimal BestAskSize { get; set; }
+    }
+
+    public class BaseMessage
+    {
+        public virtual EventType @Event { get; } = EventType.None;
+
+        [JsonProperty("e")]
+        public string EventName { get; set; }
+
+        [JsonProperty("E")]
+        public long Time { get; set; }
+
+        [JsonProperty("s")]
+        public string Symbol { get; set; }
+    }
+
+    public class Trade : BaseMessage
+    {
+        public override EventType @Event => EventType.Trade;
+
+        [JsonProperty("T")]
+        public new long Time { get; set; }
+
+        [JsonProperty("p")]
+        public decimal Price { get; private set; }
+
+        [JsonProperty("q")]
+        public decimal Quantity { get; private set; }
+    }
+
+    public class Execution : BaseMessage
+    {
+        public override EventType @Event => EventType.Execution;
+
+        [JsonProperty("i")]
+        public string OrderId { get; set; }
+
+        [JsonProperty("t")]
+        public string TradeId { get; set; }
+
+        [JsonProperty("I")]
+        public string Ignore { get; set; }
+
+        [JsonProperty("x")]
+        public string ExecutionType { get; private set; }
+
+        [JsonProperty("X")]
+        public string OrderStatus { get; private set; }
+
+        [JsonProperty("T")]
+        public long TransactionTime { get; set; }
+
+        [JsonProperty("L")]
+        public decimal LastExecutedPrice { get; set; }
+
+        [JsonProperty("l")]
+        public decimal LastExecutedQuantity { get; set; }
+
+        [JsonProperty("S")]
+        public string Side { get; set; }
+
+        [JsonProperty("n")]
+        public decimal Fee { get; set; }
+
+        [JsonProperty("N")]
+        public string FeeCurrency { get; set; }
+
+        public OrderDirection Direction => Side.Equals("BUY", StringComparison.OrdinalIgnoreCase) ? OrderDirection.Buy : OrderDirection.Sell;
+    }
+
+    public class Kline
+    {
+        public long OpenTime { get; }
+        public decimal Open { get; }
+        public decimal Close { get; }
+        public decimal High { get; }
+        public decimal Low { get; }
+        public decimal Volume { get; }
+
+        public Kline() { }
+
+        public Kline(long msts, decimal close)
+        {
+            OpenTime = msts;
+            Open = Close = High = Low = close;
+            Volume = 0;
+        }
+
+        public Kline(object[] entries)
+        {
+            OpenTime = Convert.ToInt64(entries[0], CultureInfo.InvariantCulture);
+            Open = ((string)entries[1]).ToDecimal();
+            High = ((string)entries[2]).ToDecimal();
+            Low = ((string)entries[3]).ToDecimal();
+            Close = ((string)entries[4]).ToDecimal();
+            Volume = ((string)entries[5]).ToDecimal();
+        }
+    }
+
+#pragma warning restore 1591
+}

Brokerages/Bitfinex/BitfinexBrokerage.Messaging.cs

@@ -27,8 +27,9 @@ using RestSharp;
 using System;
 using System.Collections.Concurrent;
 using System.Collections.Generic;
-using System.Globalization;
 using System.Linq;
+using QuantConnect.Brokerages.Bitfinex.Messages;
+using Order = QuantConnect.Orders.Order;
 
 namespace QuantConnect.Brokerages.Bitfinex
 {
@@ -37,16 +38,21 @@ namespace QuantConnect.Brokerages.Bitfinex
     /// </summary>
     public partial class BitfinexBrokerage
     {
-        private const string ApiVersion = "v1";
+        private const string ApiVersion = "v2";
+        private const string RestApiUrl = "https://api.bitfinex.com";
+        private const string WebSocketUrl = "wss://api.bitfinex.com/ws/2";
+
         private readonly IAlgorithm _algorithm;
-        private readonly ConcurrentQueue<WebSocketMessage> _messageBuffer = new ConcurrentQueue<WebSocketMessage>();
-        private volatile bool _streamLocked;
-        private readonly RateGate _restRateLimiter = new RateGate(8, TimeSpan.FromMinutes(1));
+        private readonly RateGate _restRateLimiter = new RateGate(10, TimeSpan.FromMinutes(1));
         private readonly ConcurrentDictionary<int, decimal> _fills = new ConcurrentDictionary<int, decimal>();
-        private readonly BitfinexSubscriptionManager _subscriptionManager;
         private readonly SymbolPropertiesDatabase _symbolPropertiesDatabase;
         private readonly IDataAggregator _aggregator;
 
+        // map Bitfinex ClientOrderId -> LEAN order (only used for orders submitted in PlaceOrder, not for existing orders)
+        private readonly ConcurrentDictionary<long, Order> _orderMap = new ConcurrentDictionary<long, Order>();
+        private readonly object _clientOrderIdLocker = new object();
+        private long _nextClientOrderId;
+
         /// <summary>
         /// Locking object for the Ticks list in the data queue handler
         /// </summary>
@@ -55,22 +61,19 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <summary>
         /// Constructor for brokerage
         /// </summary>
-        /// <param name="wssUrl">websockets url</param>
-        /// <param name="restUrl">rest api url</param>
         /// <param name="apiKey">api key</param>
         /// <param name="apiSecret">api secret</param>
         /// <param name="algorithm">the algorithm instance is required to retrieve account type</param>
         /// <param name="priceProvider">The price provider for missing FX conversion rates</param>
         /// <param name="aggregator">consolidate ticks</param>
-        public BitfinexBrokerage(string wssUrl, string restUrl, string apiKey, string apiSecret, IAlgorithm algorithm, IPriceProvider priceProvider, IDataAggregator aggregator)
-            : this(wssUrl, new WebSocketClientWrapper(), new RestClient(restUrl), apiKey, apiSecret, algorithm, priceProvider, aggregator)
+        public BitfinexBrokerage(string apiKey, string apiSecret, IAlgorithm algorithm, IPriceProvider priceProvider, IDataAggregator aggregator)
+            : this(new WebSocketClientWrapper(), new RestClient(RestApiUrl), apiKey, apiSecret, algorithm, priceProvider, aggregator)
         {
         }
 
         /// <summary>
         /// Constructor for brokerage
         /// </summary>
-        /// <param name="wssUrl">websockets url</param>
         /// <param name="websocket">instance of websockets client</param>
         /// <param name="restClient">instance of rest client</param>
         /// <param name="apiKey">api key</param>
@@ -78,10 +81,10 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <param name="algorithm">the algorithm instance is required to retrieve account type</param>
         /// <param name="priceProvider">The price provider for missing FX conversion rates</param>
         /// <param name="aggregator">consolidate ticks</param>
-        public BitfinexBrokerage(string wssUrl, IWebSocket websocket, IRestClient restClient, string apiKey, string apiSecret, IAlgorithm algorithm, IPriceProvider priceProvider, IDataAggregator aggregator)
-            : base(wssUrl, websocket, restClient, apiKey, apiSecret, Market.Bitfinex, "Bitfinex")
+        public BitfinexBrokerage(IWebSocket websocket, IRestClient restClient, string apiKey, string apiSecret, IAlgorithm algorithm, IPriceProvider priceProvider, IDataAggregator aggregator)
+            : base(WebSocketUrl, websocket, restClient, apiKey, apiSecret, "Bitfinex")
         {
-            _subscriptionManager = new BitfinexSubscriptionManager(this, wssUrl, _symbolMapper);
+            SubscriptionManager = new BitfinexSubscriptionManager(this, WebSocketUrl, _symbolMapper);
             _symbolPropertiesDatabase = SymbolPropertiesDatabase.FromDataFolder();
             _algorithm = algorithm;
             _aggregator = aggregator;
@@ -99,23 +102,6 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <param name="e"></param>
         public override void OnMessage(object sender, WebSocketMessage e)
         {
-            LastHeartbeatUtcTime = DateTime.UtcNow;
-
-            // Verify if we're allowed to handle the streaming packet yet; while we're placing an order we delay the
-            // stream processing a touch.
-            try
-            {
-                if (_streamLocked)
-                {
-                    _messageBuffer.Enqueue(e);
-                    return;
-                }
-            }
-            catch (Exception err)
-            {
-                Log.Error(err);
-            }
-
             OnMessageImpl(e);
         }
 
@@ -143,38 +129,30 @@ namespace QuantConnect.Brokerages.Bitfinex
         }
 
         /// <summary>
-        /// Subscribes to the requested symbols (using an individual streaming channel)
+        /// Should be empty, Bitfinex brokerage manages his public channels including subscribe/unsubscribe/reconnect methods using <see cref="BitfinexSubscriptionManager"/>
+        /// Not used in master
         /// </summary>
-        /// <param name="symbols">The list of symbols to subscribe</param>
-        public override void Subscribe(IEnumerable<Symbol> symbols)
+        /// <param name="symbols"></param>
+        public override void Subscribe(IEnumerable<Symbol> symbols) { }
+
+        private long GetNextClientOrderId()
         {
-            foreach (var symbol in symbols)
+            lock (_clientOrderIdLocker)
             {
-                if (_subscriptionManager.IsSubscribed(symbol) ||
-                    symbol.Value.Contains("UNIVERSE") ||
-                    !_symbolMapper.IsKnownBrokerageSymbol(symbol.Value) ||
-                    symbol.SecurityType != _symbolMapper.GetLeanSecurityType(symbol.Value))
+                // ensure unique id
+                var id = Convert.ToInt64(Time.DateTimeToUnixTimeStampMilliseconds(DateTime.UtcNow));
+
+                if (id > _nextClientOrderId)
                 {
-                    continue;
+                    _nextClientOrderId = id;
+                }
+                else
+                {
+                    _nextClientOrderId++;
                 }
-
-                _subscriptionManager.Subscribe(symbol);
-
-                Log.Trace($"BitfinexBrokerage.Subscribe(): Sent subscribe for {symbol.Value}.");
             }
-        }
 
-        /// <summary>
-        /// Ends current subscriptions
-        /// </summary>
-        private void Unsubscribe(IEnumerable<Symbol> symbols)
-        {
-            foreach (var symbol in symbols)
-            {
-                _subscriptionManager.Unsubscribe(symbol);
-
-                Log.Trace($"BitfinexBrokerage.Unsubscribe(): Sent unsubscribe for {symbol.Value}.");
-            }
+            return _nextClientOrderId;
         }
 
         /// <summary>
@@ -190,23 +168,56 @@ namespace QuantConnect.Brokerages.Bitfinex
                 if (token is JArray)
                 {
                     var channel = token[0].ToObject<int>();
+
                     // heartbeat
                     if (token[1].Type == JTokenType.String && token[1].Value<string>() == "hb")
                     {
                         return;
                     }
-                    //public channels
+
+                    // account information channel
                     if (channel == 0)
                     {
                         var term = token[1].ToObject<string>();
                         switch (term.ToLowerInvariant())
                         {
+                            // order closed
                             case "oc":
-                                OnOrderClose(token[2].ToObject<string[]>());
+                                OnOrderClose(token[2].ToObject<Messages.Order>());
                                 return;
+
+                            // trade execution update
                             case "tu":
-                                EmitFillOrder(token[2].ToObject<string[]>());
+                                EmitFillOrder(token[2].ToObject<TradeExecutionUpdate>());
                                 return;
+
+                            // notification
+                            case "n":
+                                var notification = token[2];
+                                var status = notification[6].ToString();
+
+                                if (status == "ERROR")
+                                {
+                                    var errorMessage = notification[7].ToString();
+                                    OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, -1, $"Error: {errorMessage}"));
+
+                                    OnOrderError(notification[4].ToObject<Messages.Order>());
+                                }
+                                else if (status == "SUCCESS")
+                                {
+                                    var type = notification[1].ToString();
+
+                                    if (type == "on-req")
+                                    {
+                                        OnOrderNew(notification[4].ToObject<Messages.Order>());
+                                    }
+                                    else if (type == "ou-req")
+                                    {
+                                        OnOrderUpdate(notification[4].ToObject<Messages.Order>());
+                                    }
+                                }
+                                return;
+
                             default:
                                 return;
                         }
@@ -214,21 +225,24 @@ namespace QuantConnect.Brokerages.Bitfinex
                 }
                 else if (token is JObject)
                 {
-                    var raw = token.ToObject<Messages.BaseMessage>();
+                    var raw = token.ToObject<BaseMessage>();
                     switch (raw.Event.ToLowerInvariant())
                     {
                         case "auth":
-                            var auth = token.ToObject<Messages.AuthResponseMessage>();
+                            var auth = token.ToObject<AuthResponseMessage>();
                             var result = string.Equals(auth.Status, "OK", StringComparison.OrdinalIgnoreCase) ? "succeed" : "failed";
                             Log.Trace($"BitfinexWebsocketsBrokerage.OnMessage: Subscribing to authenticated channels {result}");
                             return;
+
                         case "info":
                         case "ping":
                             return;
+
                         case "error":
-                            var error = token.ToObject<Messages.ErrorMessage>();
-                            Log.Trace($"BitfinexWebsocketsBrokerage.OnMessage: {error.Level}: {error.Message}");
+                            var error = token.ToObject<ErrorMessage>();
+                            Log.Error($"BitfinexWebsocketsBrokerage.OnMessage: {error.Level}: {error.Message}");
                             return;
+
                         default:
                             Log.Trace($"BitfinexWebsocketsBrokerage.OnMessage: Unexpected message format: {e.Message}");
                             break;
@@ -242,39 +256,52 @@ namespace QuantConnect.Brokerages.Bitfinex
             }
         }
 
-        private void OnOrderClose(string[] entries)
+        private void OnOrderError(Messages.Order bitfinexOrder)
         {
-            var brokerId = entries[0];
-            if (entries[5].IndexOf("canceled", StringComparison.OrdinalIgnoreCase) >= 0)
+            Order order;
+            if (_orderMap.TryGetValue(bitfinexOrder.ClientOrderId, out order))
             {
-                var order = CachedOrderIDs
-                    .FirstOrDefault(o => o.Value.BrokerId.Contains(brokerId))
-                    .Value;
-                if (order == null)
+                OnOrderEvent(new OrderEvent(order, DateTime.UtcNow, OrderFee.Zero, "Bitfinex Order Event")
                 {
-                    order = _algorithm.Transactions.GetOrderByBrokerageId(brokerId);
-                    if (order == null)
+                    Status = OrderStatus.Invalid
+                });
+            }
+        }
+
+        private void OnOrderNew(Messages.Order bitfinexOrder)
+        {
+            if (bitfinexOrder.Status == "ACTIVE")
+            {
+                var brokerId = bitfinexOrder.Id.ToStringInvariant();
+
+                Order order;
+                if (_orderMap.TryGetValue(bitfinexOrder.ClientOrderId, out order))
+                {
+                    if (CachedOrderIDs.ContainsKey(order.Id))
                     {
-                        // not our order, nothing else to do here
-                        return;
+                        CachedOrderIDs[order.Id].BrokerId.Clear();
+                        CachedOrderIDs[order.Id].BrokerId.Add(brokerId);
                     }
-                }
-                Order outOrder;
-                if (CachedOrderIDs.TryRemove(order.Id, out outOrder))
-                {
-                    OnOrderEvent(new OrderEvent(order,
-                        DateTime.UtcNow,
-                        OrderFee.Zero,
-                        "Bitfinex Order Event") { Status = OrderStatus.Canceled });
+                    else
+                    {
+                        order.BrokerId.Add(brokerId);
+                        CachedOrderIDs.TryAdd(order.Id, order);
+                    }
+
+                    OnOrderEvent(new OrderEvent(order, DateTime.UtcNow, OrderFee.Zero, "Bitfinex Order Event")
+                    {
+                        Status = OrderStatus.Submitted
+                    });
                 }
             }
         }
 
-        private void EmitFillOrder(string[] entries)
+        private void OnOrderUpdate(Messages.Order bitfinexOrder)
         {
-            try
+            if (bitfinexOrder.Status == "ACTIVE")
             {
-                var brokerId = entries[4];
+                var brokerId = bitfinexOrder.Id.ToStringInvariant();
+
                 var order = CachedOrderIDs
                     .FirstOrDefault(o => o.Value.BrokerId.Contains(brokerId))
                     .Value;
@@ -283,20 +310,81 @@ namespace QuantConnect.Brokerages.Bitfinex
                     order = _algorithm.Transactions.GetOrderByBrokerageId(brokerId);
                     if (order == null)
                     {
-                        // not our order, nothing else to do here
+                        Log.Error($"OnOrderUpdate(): order not found: BrokerId: {brokerId}");
                         return;
                     }
                 }
 
-                var symbol = _symbolMapper.GetLeanSymbol(entries[2]);
-                var fillPrice = decimal.Parse(entries[6], NumberStyles.Float, CultureInfo.InvariantCulture);
-                var fillQuantity = decimal.Parse(entries[5], NumberStyles.Float, CultureInfo.InvariantCulture);
+                OnOrderEvent(new OrderEvent(order, DateTime.UtcNow, OrderFee.Zero, "Bitfinex Order Event")
+                {
+                    Status = OrderStatus.UpdateSubmitted
+                });
+            }
+        }
+
+        private void OnOrderClose(Messages.Order bitfinexOrder)
+        {
+            if (bitfinexOrder.Status.StartsWith("CANCELED"))
+            {
+                var brokerId = bitfinexOrder.Id.ToStringInvariant();
+
+                var order = CachedOrderIDs
+                    .FirstOrDefault(o => o.Value.BrokerId.Contains(brokerId))
+                    .Value;
+                if (order == null)
+                {
+                    order = _algorithm.Transactions.GetOrderByBrokerageId(brokerId);
+                    if (order == null)
+                    {
+                        Log.Error($"OnOrderClose(): order not found: BrokerId: {brokerId}");
+                        return;
+                    }
+                }
+                else
+                {
+                    Order outOrder;
+                    CachedOrderIDs.TryRemove(order.Id, out outOrder);
+                }
+
+                if (bitfinexOrder.ClientOrderId > 0)
+                {
+                    Order removed;
+                    _orderMap.TryRemove(bitfinexOrder.ClientOrderId, out removed);
+                }
+
+                OnOrderEvent(new OrderEvent(order, DateTime.UtcNow, OrderFee.Zero, "Bitfinex Order Event")
+                {
+                    Status = OrderStatus.Canceled
+                });
+            }
+        }
+
+        private void EmitFillOrder(TradeExecutionUpdate update)
+        {
+            try
+            {
+                var brokerId = update.OrderId.ToStringInvariant();
+
+                var order = CachedOrderIDs
+                    .FirstOrDefault(o => o.Value.BrokerId.Contains(brokerId))
+                    .Value;
+
+                if (order == null)
+                {
+                    order = _algorithm.Transactions.GetOrderByBrokerageId(brokerId);
+                    if (order == null)
+                    {
+                        Log.Error($"EmitFillOrder(): order not found: BrokerId: {brokerId}");
+                        return;
+                    }
+                }
+
+                var symbol = _symbolMapper.GetLeanSymbol(update.Symbol);
+                var fillPrice = update.ExecPrice;
+                var fillQuantity = update.ExecAmount;
                 var direction = fillQuantity < 0 ? OrderDirection.Sell : OrderDirection.Buy;
-                var updTime = Time.UnixTimeStampToDateTime(double.Parse(entries[3], NumberStyles.Float, CultureInfo.InvariantCulture));
-                var orderFee = new OrderFee(new CashAmount(
-                        Math.Abs(decimal.Parse(entries[9], NumberStyles.Float, CultureInfo.InvariantCulture)),
-                        entries[10]
-                    ));
+                var updTime = Time.UnixMillisecondTimeStampToDateTime(update.MtsCreate);
+                var orderFee = new OrderFee(new CashAmount(Math.Abs(update.Fee), update.FeeCurrency));
 
                 var status = OrderStatus.Filled;
                 if (fillQuantity != order.Quantity)
@@ -323,8 +411,15 @@ namespace QuantConnect.Brokerages.Bitfinex
                 {
                     Order outOrder;
                     CachedOrderIDs.TryRemove(order.Id, out outOrder);
+
                     decimal ignored;
                     _fills.TryRemove(order.Id, out ignored);
+
+                    var clientOrderId = _orderMap.FirstOrDefault(x => x.Value.BrokerId.Contains(brokerId)).Key;
+                    if (clientOrderId > 0)
+                    {
+                        _orderMap.TryRemove(clientOrderId, out outOrder);
+                    }
                 }
 
                 OnOrderEvent(orderEvent);
@@ -346,57 +441,9 @@ namespace QuantConnect.Brokerages.Bitfinex
         }
 
         /// <summary>
-        /// Lock the streaming processing while we're sending orders as sometimes they fill before the REST call returns.
-        /// </summary>
-        public void LockStream()
-        {
-            Log.Trace("BitfinexBrokerage.Messaging.LockStream(): Locking Stream");
-            _streamLocked = true;
-        }
-
-        /// <summary>
-        /// Unlock stream and process all backed up messages.
-        /// </summary>
-        public void UnlockStream()
-        {
-            Log.Trace("BitfinexBrokerage.Messaging.UnlockStream(): Processing Backlog...");
-            while (_messageBuffer.Any())
-            {
-                WebSocketMessage e;
-                _messageBuffer.TryDequeue(out e);
-                OnMessageImpl(e);
-            }
-            Log.Trace("BitfinexBrokerage.Messaging.UnlockStream(): Stream Unlocked.");
-            // Once dequeued in order; unlock stream.
-            _streamLocked = false;
-        }
-
-        /// <summary>
-        /// Gets a list of current subscriptions
+        /// Should be empty. <see cref="BitfinexSubscriptionManager"/> manages each <see cref="BitfinexWebSocketWrapper"/> individually
         /// </summary>
         /// <returns></returns>
-        protected override IList<Symbol> GetSubscribed()
-        {
-            IList<Symbol> list = new List<Symbol>();
-            lock (ChannelList)
-            {
-                foreach (var ticker in ChannelList.Select(x => x.Value.Symbol).Distinct())
-                {
-                    list.Add(_symbolMapper.GetLeanSymbol(ticker));
-                }
-            }
-            return list;
-        }
-    }
-
-    /// <summary>
-    /// Represents Bitfinex channel information
-    /// </summary>
-    public class BitfinexChannel : BaseWebsocketsBrokerage.Channel
-    {
-        /// <summary>
-        /// Represents channel identifier for specific subscription
-        /// </summary>
-        public string ChannelId { get; set; }
+        protected override IEnumerable<Symbol> GetSubscribed() => new List<Symbol>();
     }
 }

Brokerages/Bitfinex/BitfinexBrokerage.Utility.cs

@@ -19,12 +19,14 @@ using QuantConnect.Logging;
 using QuantConnect.Orders;
 using RestSharp;
 using System;
+using System.Collections.Generic;
 using System.Globalization;
 using System.Linq;
 using System.Net;
 using System.Security.Cryptography;
 using System.Text;
-using QuantConnect.Orders.Fees;
+using QuantConnect.Brokerages.Bitfinex.Messages;
+using Order = QuantConnect.Orders.Order;
 
 namespace QuantConnect.Brokerages.Bitfinex
 {
@@ -36,23 +38,40 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <summary>
         /// Unix Epoch
         /// </summary>
-        public readonly DateTime dt1970 = new DateTime(1970, 1, 1);
+        public readonly DateTime UnixEpoch = new DateTime(1970, 1, 1);
+
         /// <summary>
-        /// Key Header
+        /// ApiKey Header
         /// </summary>
-        public const string KeyHeader = "X-BFX-APIKEY";
+        public const string ApiKeyHeader = "bfx-apikey";
+
+        /// <summary>
+        /// Nonce Header
+        /// </summary>
+        public const string NonceHeader = "bfx-nonce";
+
         /// <summary>
         /// Signature Header
         /// </summary>
-        public const string SignatureHeader = "X-BFX-SIGNATURE";
-        /// <summary>
-        /// Payload Header
-        /// </summary>
-        public const string PayloadHeader = "X-BFX-PAYLOAD";
+        public const string SignatureHeader = "bfx-signature";
+
+        private long _lastNonce;
+        private readonly object _lockerNonce = new object();
 
         private long GetNonce()
         {
-            return (DateTime.UtcNow - dt1970).Ticks;
+            // The nonce provided must be strictly increasing but should not exceed the MAX_SAFE_INTEGER constant value of 9007199254740991.
+            lock (_lockerNonce)
+            {
+                var nonce = (long) Math.Truncate((DateTime.UtcNow - UnixEpoch).TotalMilliseconds * 1000);
+
+                if (nonce == _lastNonce)
+                {
+                    _lastNonce = ++nonce;
+                }
+
+                return nonce;
+            }
         }
 
         /// <summary>
@@ -60,19 +79,21 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// https://docs.bitfinex.com/docs/rest-auth
         /// </summary>
         /// <param name="request">the rest request</param>
-        /// <param name="payload">the body of the request</param>
+        /// <param name="endpoint">The API endpoint</param>
+        /// <param name="parameters">the body of the request</param>
         /// <returns>a token representing the request params</returns>
-        private void SignRequest(IRestRequest request, string payload)
+        private void SignRequest(IRestRequest request, string endpoint, IDictionary<string, object> parameters)
         {
-            using (HMACSHA384 hmac = new HMACSHA384(Encoding.UTF8.GetBytes(ApiSecret)))
+            using (var hmac = new HMACSHA384(Encoding.UTF8.GetBytes(ApiSecret)))
             {
-                byte[] payloadByte = Encoding.UTF8.GetBytes(payload);
-                string payloadBase64 = Convert.ToBase64String(payloadByte, Base64FormattingOptions.None);
-                string payloadSha384hmac = ByteArrayToString(hmac.ComputeHash(Encoding.UTF8.GetBytes(payloadBase64)));
+                var json = JsonConvert.SerializeObject(parameters.ToDictionary(p => p.Key, p => p.Value));
+                var nonce = GetNonce().ToStringInvariant();
+                var payload = $"/api{endpoint}{nonce}{json}";
+                var signature = ByteArrayToString(hmac.ComputeHash(Encoding.UTF8.GetBytes(payload)));
 
-                request.AddHeader(KeyHeader, ApiKey);
-                request.AddHeader(PayloadHeader, payloadBase64);
-                request.AddHeader(SignatureHeader, payloadSha384hmac);
+                request.AddHeader(ApiKeyHeader, ApiKey);
+                request.AddHeader(NonceHeader, nonce);
+                request.AddHeader(SignatureHeader, signature);
             }
         }
 
@@ -84,16 +105,17 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <returns>a token representing the request params</returns>
         private string AuthenticationToken(string payload)
         {
-            using (HMACSHA384 hmac = new HMACSHA384(Encoding.UTF8.GetBytes(ApiSecret)))
+            using (var hmac = new HMACSHA384(Encoding.UTF8.GetBytes(ApiSecret)))
             {
                 return ByteArrayToString(hmac.ComputeHash(Encoding.UTF8.GetBytes(payload)));
             }
         }
 
-        private Func<Messages.Wallet, bool> WalletFilter(AccountType accountType)
+        private Func<Wallet, bool> WalletFilter(AccountType accountType)
         {
-            return wallet => wallet.Type.Equals("exchange") && accountType == AccountType.Cash ||
-                wallet.Type.Equals("trading") && accountType == AccountType.Margin;
+            return wallet =>
+                wallet.Type.Equals("exchange") && accountType == AccountType.Cash ||
+                wallet.Type.Equals("margin") && accountType == AccountType.Margin;
         }
 
         /// <summary>
@@ -103,16 +125,18 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <returns></returns>
         public Tick GetTick(Symbol symbol)
         {
-            string endpoint = GetEndpoint($"pubticker/{_symbolMapper.GetBrokerageSymbol(symbol)}");
-            var req = new RestRequest(endpoint, Method.GET);
-            var response = ExecuteRestRequest(req);
+            var endpoint = $"/{ApiVersion}/ticker/{_symbolMapper.GetBrokerageSymbol(symbol)}";
+
+            var restRequest = new RestRequest(endpoint, Method.GET);
+            var response = ExecuteRestRequest(restRequest);
+
             if (response.StatusCode != HttpStatusCode.OK)
             {
                 throw new Exception($"BitfinexBrokerage.GetTick: request failed: [{(int)response.StatusCode}] {response.StatusDescription}, Content: {response.Content}, ErrorMessage: {response.ErrorMessage}");
             }
 
-            var tick = JsonConvert.DeserializeObject<Messages.Tick>(response.Content);
-            return new Tick(Time.UnixTimeStampToDateTime(tick.Timestamp), symbol, tick.Bid, tick.Ask) { Quantity = tick.Volume };
+            var tick = JsonConvert.DeserializeObject<Ticker>(response.Content);
+            return new Tick(DateTime.UtcNow, symbol, tick.LastPrice, tick.Bid, tick.Ask);
         }
 
         /// <summary>
@@ -125,63 +149,47 @@ namespace QuantConnect.Brokerages.Bitfinex
             return $"/{ApiVersion}/{method}";
         }
 
-        /// <summary>
-        /// Returns the complete order update endpoint for current Bitfinex API version
-        /// </summary>
-        private string GetOrderUpdateEndpoint()
-        {
-            return GetEndpoint("order/cancel/replace");
-        }
-
         private static OrderStatus ConvertOrderStatus(Messages.Order order)
         {
-            if (order.IsLive && order.ExecutedAmount == 0)
+            if (order.Status == "ACTIVE")
             {
-                return Orders.OrderStatus.Submitted;
+                return OrderStatus.Submitted;
             }
-            else if (order.ExecutedAmount > 0 && order.RemainingAmount > 0)
+            else if (order.Status.StartsWith("PARTIALLY FILLED"))
             {
-                return Orders.OrderStatus.PartiallyFilled;
+                return OrderStatus.PartiallyFilled;
             }
-            else if (order.RemainingAmount == 0)
+            else if (order.Status.StartsWith("EXECUTED"))
             {
-                return Orders.OrderStatus.Filled;
+                return OrderStatus.Filled;
             }
-            else if (order.IsCancelled)
+            else if (order.Status.StartsWith("CANCELED"))
             {
-                return Orders.OrderStatus.Canceled;
+                return OrderStatus.Canceled;
             }
 
-            return Orders.OrderStatus.None;
+            return OrderStatus.None;
         }
 
         private static string ConvertOrderType(AccountType accountType, OrderType orderType)
         {
-            string outputOrderType = string.Empty;
+            string outputOrderType;
             switch (orderType)
             {
                 case OrderType.Limit:
                 case OrderType.Market:
-                    outputOrderType = orderType.ToLower();
+                    outputOrderType = orderType.ToStringInvariant().ToUpperInvariant();
                     break;
+
                 case OrderType.StopMarket:
-                    outputOrderType = "stop";
+                    outputOrderType = "STOP";
                     break;
+
                 default:
                     throw new NotSupportedException($"BitfinexBrokerage.ConvertOrderType: Unsupported order type: {orderType}");
             }
 
-            return (accountType == AccountType.Cash ? "exchange " : "") + outputOrderType;
-        }
-
-        private static string ConvertOrderDirection(OrderDirection orderDirection)
-        {
-            if (orderDirection == OrderDirection.Buy || orderDirection == OrderDirection.Sell)
-            {
-                return orderDirection.ToLower();
-            }
-
-            throw new NotSupportedException($"BitfinexBrokerage.ConvertOrderDirection: Unsupported order direction: {orderDirection}");
+            return (accountType == AccountType.Cash ? "EXCHANGE " : "") + outputOrderType;
         }
 
         /// <summary>
@@ -196,10 +204,12 @@ namespace QuantConnect.Brokerages.Bitfinex
             {
                 case OrderType.Limit:
                     return ((LimitOrder)order).LimitPrice;
+
                 case OrderType.Market:
                     // Order price must be positive for market order too;
                     // refuses for price = 0
                     return 1;
+
                 case OrderType.StopMarket:
                     return ((StopMarketOrder)order).StopPrice;
             }
@@ -207,14 +217,14 @@ namespace QuantConnect.Brokerages.Bitfinex
             throw new NotSupportedException($"BitfinexBrokerage.ConvertOrderType: Unsupported order type: {order.Type}");
         }
 
-        private Holding ConvertHolding(Messages.Position position)
+        private Holding ConvertHolding(Position position)
         {
             var holding = new Holding
             {
                 Symbol = _symbolMapper.GetLeanSymbol(position.Symbol),
-                AveragePrice = position.AveragePrice,
+                AveragePrice = position.BasePrice,
                 Quantity = position.Amount,
-                UnrealizedPnL = position.PL,
+                UnrealizedPnL = position.ProfitLoss,
                 CurrencySymbol = "$",
                 Type = SecurityType.Crypto
             };
@@ -235,8 +245,9 @@ namespace QuantConnect.Brokerages.Bitfinex
 
         private Func<Messages.Order, bool> OrderFilter(AccountType accountType)
         {
-            return order => (order.IsExchange && accountType == AccountType.Cash) ||
-                (!order.IsExchange && accountType == AccountType.Margin);
+            return order =>
+                order.IsExchange && accountType == AccountType.Cash ||
+                !order.IsExchange && accountType == AccountType.Margin;
         }
 
         /// <summary>
@@ -276,84 +287,6 @@ namespace QuantConnect.Brokerages.Bitfinex
             return hex.ToString();
         }
 
-        private bool SubmitOrder(string endpoint, Order order)
-        {
-            LockStream();
-
-            var payload = new JsonObject();
-            payload.Add("request", endpoint);
-            payload.Add("nonce", GetNonce().ToStringInvariant());
-            payload.Add("symbol", _symbolMapper.GetBrokerageSymbol(order.Symbol));
-            payload.Add("amount", Math.Abs(order.Quantity).ToString(CultureInfo.InvariantCulture));
-            payload.Add("side", ConvertOrderDirection(order.Direction));
-            payload.Add("type", ConvertOrderType(_algorithm.BrokerageModel.AccountType, order.Type));
-            payload.Add("price", GetOrderPrice(order).ToString(CultureInfo.InvariantCulture));
-
-            if (order.BrokerId.Any())
-            {
-                payload.Add("order_id", Parse.Long(order.BrokerId.FirstOrDefault()));
-            }
-
-            var orderProperties = order.Properties as BitfinexOrderProperties;
-            if (orderProperties != null)
-            {
-                if (order.Type == OrderType.Limit)
-                {
-                    payload.Add("is_hidden", orderProperties.Hidden);
-                    payload.Add("is_postonly", orderProperties.PostOnly);
-                }
-            }
-
-            var request = new RestRequest(endpoint, Method.POST);
-            request.AddJsonBody(payload.ToString());
-            SignRequest(request, payload.ToString());
-
-            var response = ExecuteRestRequest(request);
-            var orderFee = OrderFee.Zero;
-            if (response.StatusCode == HttpStatusCode.OK)
-            {
-                var raw = JsonConvert.DeserializeObject<Messages.Order>(response.Content);
-
-                if (string.IsNullOrEmpty(raw?.Id))
-                {
-                    var errorMessage = $"Error parsing response from place order: {response.Content}";
-                    OnOrderEvent(new OrderEvent(order, DateTime.UtcNow, orderFee, "Bitfinex Order Event") { Status = OrderStatus.Invalid, Message = errorMessage });
-                    OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, (int)response.StatusCode, errorMessage));
-
-                    UnlockStream();
-                    return true;
-                }
-
-                var brokerId = raw.Id;
-                if (CachedOrderIDs.ContainsKey(order.Id))
-                {
-                    CachedOrderIDs[order.Id].BrokerId.Clear();
-                    CachedOrderIDs[order.Id].BrokerId.Add(brokerId);
-                }
-                else
-                {
-                    order.BrokerId.Add(brokerId);
-                    CachedOrderIDs.TryAdd(order.Id, order);
-                }
-
-                var isUpdate = endpoint.Equals(GetOrderUpdateEndpoint());
-
-                // Generate submitted event
-                OnOrderEvent(new OrderEvent(order, DateTime.UtcNow, orderFee, "Bitfinex Order Event") { Status = isUpdate ? OrderStatus.UpdateSubmitted : OrderStatus.Submitted });
-                Log.Trace($"Order submitted successfully - OrderId: {order.Id}");
-
-                UnlockStream();
-                return true;
-            }
-
-            var message = $"Order failed, Order Id: {order.Id} timestamp: {order.Time} quantity: {order.Quantity} content: {response.Content}";
-            OnOrderEvent(new OrderEvent(order, DateTime.UtcNow, orderFee, "Bitfinex Order Event") { Status = OrderStatus.Invalid });
-            OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, -1, message));
-
-            UnlockStream();
-            return true;
-        }
-
         /// <summary>
         /// Maps Resolution to IB representation
         /// </summary>

Brokerages/Bitfinex/BitfinexBrokerage.cs

@@ -26,8 +26,9 @@ using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Net;
-using QuantConnect.Orders.Fees;
+using QuantConnect.Brokerages.Bitfinex.Messages;
 using QuantConnect.Securities.Crypto;
+using Order = QuantConnect.Orders.Order;
 
 namespace QuantConnect.Brokerages.Bitfinex
 {
@@ -51,7 +52,37 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <returns>True if the request for a new order has been placed, false otherwise</returns>
         public override bool PlaceOrder(Order order)
         {
-            return SubmitOrder(GetEndpoint("order/new"), order);
+            var parameters = new JsonObject
+            {
+                { "symbol", _symbolMapper.GetBrokerageSymbol(order.Symbol) },
+                { "amount", order.Quantity.ToStringInvariant() },
+                { "type", ConvertOrderType(_algorithm.BrokerageModel.AccountType, order.Type) },
+                { "price", GetOrderPrice(order).ToStringInvariant() }
+            };
+
+            var orderProperties = order.Properties as BitfinexOrderProperties;
+            if (orderProperties != null)
+            {
+                if (order.Type == OrderType.Limit)
+                {
+                    var flags = 0;
+                    if (orderProperties.Hidden) flags |= OrderFlags.Hidden;
+                    if (orderProperties.PostOnly) flags |= OrderFlags.PostOnly;
+
+                    parameters.Add("flags", flags);
+                }
+            }
+
+            var clientOrderId = GetNextClientOrderId();
+            parameters.Add("cid", clientOrderId);
+
+            _orderMap.TryAdd(clientOrderId, order);
+
+            var obj = new JsonArray { 0, "on", null, parameters };
+            var json = JsonConvert.SerializeObject(obj);
+            WebSocket.Send(json);
+
+            return true;
         }
 
         /// <summary>
@@ -63,14 +94,26 @@ namespace QuantConnect.Brokerages.Bitfinex
         {
             if (order.BrokerId.Count == 0)
             {
-                throw new ArgumentNullException("BitfinexBrokerage.UpdateOrder: There is no brokerage id to be updated for this order.");
+                throw new ArgumentNullException(nameof(order.BrokerId), "BitfinexBrokerage.UpdateOrder: There is no brokerage id to be updated for this order.");
             }
+
             if (order.BrokerId.Count > 1)
             {
                 throw new NotSupportedException("BitfinexBrokerage.UpdateOrder: Multiple orders update not supported. Please cancel and re-create.");
             }
 
-            return SubmitOrder(GetOrderUpdateEndpoint(), order);
+            var parameters = new JsonObject
+            {
+                { "id", Parse.Long(order.BrokerId.First()) },
+                { "amount", order.Quantity.ToStringInvariant() },
+                { "price", GetOrderPrice(order).ToStringInvariant() }
+            };
+
+            var obj = new JsonArray { 0, "ou", null, parameters };
+            var json = JsonConvert.SerializeObject(obj);
+            WebSocket.Send(json);
+
+            return true;
         }
 
         /// <summary>
@@ -89,32 +132,16 @@ namespace QuantConnect.Brokerages.Bitfinex
                 return false;
             }
 
-            LockStream();
-            var endpoint = GetEndpoint("order/cancel/multi");
-            var payload = new JsonObject();
-            payload.Add("request", endpoint);
-            payload.Add("nonce", GetNonce().ToStringInvariant());
-            payload.Add("order_ids", order.BrokerId.Select(Parse.Long));
-
-            var request = new RestRequest(endpoint, Method.POST);
-            request.AddJsonBody(payload.ToString());
-            SignRequest(request, payload.ToString());
-
-            var response = ExecuteRestRequest(request);
-            var cancellationSubmitted = false;
-            if (response.StatusCode == HttpStatusCode.OK && !(response.Content?.IndexOf("None to cancel", StringComparison.OrdinalIgnoreCase) >= 0))
+            var parameters = new JsonObject
             {
-                OnOrderEvent(new OrderEvent(order,
-                    DateTime.UtcNow,
-                    OrderFee.Zero,
-                    "Bitfinex Order Event")
-                { Status = OrderStatus.CancelPending });
+                { "id", order.BrokerId.Select(Parse.Long).First() }
+            };
 
-                cancellationSubmitted = true;
-            }
+            var obj = new JsonArray { 0, "oc", null, parameters };
+            var json = JsonConvert.SerializeObject(obj);
+            WebSocket.Send(json);
 
-            UnlockStream();
-            return cancellationSubmitted;
+            return true;
         }
 
         /// <summary>
@@ -122,8 +149,6 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// </summary>
         public override void Disconnect()
         {
-            base.Disconnect();
-
             WebSocket.Close();
         }
 
@@ -133,16 +158,13 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <returns></returns>
         public override List<Order> GetOpenOrders()
         {
-            var list = new List<Order>();
-            var endpoint = GetEndpoint("orders");
+            var endpoint = GetEndpoint("auth/r/orders");
             var request = new RestRequest(endpoint, Method.POST);
 
-            JsonObject payload = new JsonObject();
-            payload.Add("request", endpoint);
-            payload.Add("nonce", GetNonce().ToStringInvariant());
+            var parameters = new JsonObject();
 
-            request.AddJsonBody(payload.ToString());
-            SignRequest(request, payload.ToString());
+            request.AddJsonBody(parameters.ToString());
+            SignRequest(request, endpoint, parameters);
 
             var response = ExecuteRestRequest(request);
 
@@ -153,18 +175,20 @@ namespace QuantConnect.Brokerages.Bitfinex
 
             var orders = JsonConvert.DeserializeObject<Messages.Order[]>(response.Content)
                 .Where(OrderFilter(_algorithm.BrokerageModel.AccountType));
+
+            var list = new List<Order>();
             foreach (var item in orders)
             {
                 Order order;
-                if (item.Type.Replace("exchange", "").Trim() == "market")
+                if (item.Type.Replace("EXCHANGE", "").Trim() == "MARKET")
                 {
                     order = new MarketOrder { Price = item.Price };
                 }
-                else if (item.Type.Replace("exchange", "").Trim() == "limit")
+                else if (item.Type.Replace("EXCHANGE", "").Trim() == "LIMIT")
                 {
                     order = new LimitOrder { LimitPrice = item.Price };
                 }
-                else if (item.Type.Replace("exchange", "").Trim() == "stop")
+                else if (item.Type.Replace("EXCHANGE", "").Trim() == "STOP")
                 {
                     order = new StopMarketOrder { StopPrice = item.Price };
                 }
@@ -175,10 +199,10 @@ namespace QuantConnect.Brokerages.Bitfinex
                     continue;
                 }
 
-                order.Quantity = item.Side == "sell" ? -item.OriginalAmount : item.OriginalAmount;
-                order.BrokerId = new List<string> { item.Id };
+                order.Quantity = item.Amount;
+                order.BrokerId = new List<string> { item.Id.ToStringInvariant() };
                 order.Symbol = _symbolMapper.GetLeanSymbol(item.Symbol);
-                order.Time = Time.UnixTimeStampToDateTime(item.Timestamp);
+                order.Time = Time.UnixMillisecondTimeStampToDateTime(item.MtsCreate);
                 order.Status = ConvertOrderStatus(item);
                 order.Price = item.Price;
                 list.Add(order);
@@ -188,10 +212,11 @@ namespace QuantConnect.Brokerages.Bitfinex
             {
                 if (item.Status.IsOpen())
                 {
-                    var cached = CachedOrderIDs.Where(c => c.Value.BrokerId.Contains(item.BrokerId.First()));
-                    if (cached.Any())
+                    var cached = CachedOrderIDs
+                        .FirstOrDefault(c => c.Value.BrokerId.Contains(item.BrokerId.First()));
+                    if (cached.Value != null)
                     {
-                        CachedOrderIDs[cached.First().Key] = item;
+                        CachedOrderIDs[cached.Key] = item;
                     }
                 }
             }
@@ -205,15 +230,13 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <returns></returns>
         public override List<Holding> GetAccountHoldings()
         {
-            var endpoint = GetEndpoint("positions");
+            var endpoint = GetEndpoint("auth/r/positions");
             var request = new RestRequest(endpoint, Method.POST);
 
-            JsonObject payload = new JsonObject();
-            payload.Add("request", endpoint);
-            payload.Add("nonce", GetNonce().ToStringInvariant());
+            var parameters = new JsonObject();
 
-            request.AddJsonBody(payload.ToString());
-            SignRequest(request, payload.ToString());
+            request.AddJsonBody(parameters.ToString());
+            SignRequest(request, endpoint, parameters);
 
             var response = ExecuteRestRequest(request);
 
@@ -222,8 +245,8 @@ namespace QuantConnect.Brokerages.Bitfinex
                 throw new Exception($"BitfinexBrokerage.GetAccountHoldings: request failed: [{(int)response.StatusCode}] {response.StatusDescription}, Content: {response.Content}, ErrorMessage: {response.ErrorMessage}");
             }
 
-            var positions = JsonConvert.DeserializeObject<Messages.Position[]>(response.Content);
-            return positions.Where(p => p.Amount != 0)
+            var positions = JsonConvert.DeserializeObject<Position[]>(response.Content);
+            return positions.Where(p => p.Amount != 0 && p.Symbol.StartsWith("t"))
                 .Select(ConvertHolding)
                 .ToList();
         }
@@ -234,16 +257,13 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <returns></returns>
         public override List<CashAmount> GetCashBalance()
         {
-            var list = new List<CashAmount>();
-            var endpoint = GetEndpoint("balances");
+            var endpoint = GetEndpoint("auth/r/wallets");
             var request = new RestRequest(endpoint, Method.POST);
 
-            JsonObject payload = new JsonObject();
-            payload.Add("request", endpoint);
-            payload.Add("nonce", GetNonce().ToStringInvariant());
+            var parameters = new JsonObject();
 
-            request.AddJsonBody(payload.ToString());
-            SignRequest(request, payload.ToString());
+            request.AddJsonBody(parameters.ToString());
+            SignRequest(request, endpoint, parameters);
 
             var response = ExecuteRestRequest(request);
 
@@ -252,13 +272,15 @@ namespace QuantConnect.Brokerages.Bitfinex
                 throw new Exception($"BitfinexBrokerage.GetCashBalance: request failed: [{(int)response.StatusCode}] {response.StatusDescription}, Content: {response.Content}, ErrorMessage: {response.ErrorMessage}");
             }
 
-            var availableWallets = JsonConvert.DeserializeObject<Messages.Wallet[]>(response.Content)
+            var availableWallets = JsonConvert.DeserializeObject<Wallet[]>(response.Content)
                 .Where(WalletFilter(_algorithm.BrokerageModel.AccountType));
+
+            var list = new List<CashAmount>();
             foreach (var item in availableWallets)
             {
-                if (item.Amount > 0)
+                if (item.Balance > 0)
                 {
-                    list.Add(new CashAmount(item.Amount, item.Currency.ToUpperInvariant()));
+                    list.Add(new CashAmount(item.Balance, item.Currency.ToUpperInvariant()));
                 }
             }
 
@@ -342,7 +364,7 @@ namespace QuantConnect.Brokerages.Bitfinex
             string symbol = _symbolMapper.GetBrokerageSymbol(request.Symbol);
             long startMsec = (long)Time.DateTimeToUnixTimeStamp(request.StartTimeUtc) * 1000;
             long endMsec = (long)Time.DateTimeToUnixTimeStamp(request.EndTimeUtc) * 1000;
-            string endpoint = $"v2/candles/trade:{resolution}:t{symbol}/hist?limit=1000&sort=1";
+            string endpoint = $"{ApiVersion}/candles/trade:{resolution}:{symbol}/hist?limit=1000&sort=1";
             var period = request.Resolution.ToTimeSpan();
 
             do
@@ -361,7 +383,7 @@ namespace QuantConnect.Brokerages.Bitfinex
 
                 // we need to drop the last bar provided by the exchange as its open time is a history request's end time
                 var candles = JsonConvert.DeserializeObject<object[][]>(response.Content)
-                    .Select(entries => new Messages.Candle(entries))
+                    .Select(entries => new Candle(entries))
                     .Where(candle => candle.Timestamp != endMsec)
                     .ToList();
 
@@ -382,7 +404,7 @@ namespace QuantConnect.Brokerages.Bitfinex
 
                 foreach (var candle in candles)
                 {
-                    yield return new TradeBar()
+                    yield return new TradeBar
                     {
                         Time = Time.UnixMillisecondTimeStampToDateTime(candle.Timestamp),
                         Symbol = request.Symbol,
@@ -403,7 +425,7 @@ namespace QuantConnect.Brokerages.Bitfinex
         #endregion
 
         #region IDataQueueHandler
-        
+
         /// <summary>
         /// Sets the job we're subscribing for
         /// </summary>
@@ -420,8 +442,16 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <returns>The new enumerator for this subscription request</returns>
         public IEnumerator<BaseData> Subscribe(SubscriptionDataConfig dataConfig, EventHandler newDataAvailableHandler)
         {
+            var symbol = dataConfig.Symbol;
+            if (symbol.Value.Contains("UNIVERSE") ||
+                !_symbolMapper.IsKnownLeanSymbol(symbol) ||
+                symbol.SecurityType != _symbolMapper.GetLeanSecurityType(symbol.Value))
+            {
+                return Enumerable.Empty<BaseData>().GetEnumerator();
+            }
+
             var enumerator = _aggregator.Add(dataConfig, newDataAvailableHandler);
-            Subscribe(new[] { dataConfig.Symbol });
+            SubscriptionManager.Subscribe(dataConfig);
 
             return enumerator;
         }
@@ -432,7 +462,7 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <param name="dataConfig">Subscription config to be removed</param>
         public void Unsubscribe(SubscriptionDataConfig dataConfig)
         {
-            Unsubscribe(new Symbol[] { dataConfig.Symbol });
+            SubscriptionManager.Unsubscribe(dataConfig);
             _aggregator.Remove(dataConfig);
         }
 

Brokerages/Bitfinex/BitfinexBrokerageFactory.cs

@@ -21,7 +21,6 @@ using QuantConnect.Data;
 using QuantConnect.Interfaces;
 using QuantConnect.Securities;
 using QuantConnect.Util;
-using RestSharp;
 
 namespace QuantConnect.Brokerages.Bitfinex
 {
@@ -49,8 +48,6 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// </summary>
         public override Dictionary<string, string> BrokerageData => new Dictionary<string, string>
         {
-            { "bitfinex-rest" , Config.Get("bitfinex-rest", "https://api.bitfinex.com")},
-            { "bitfinex-url" , Config.Get("bitfinex-url", "wss://api.bitfinex.com/ws")},
             { "bitfinex-api-key", Config.Get("bitfinex-api-key")},
             { "bitfinex-api-secret", Config.Get("bitfinex-api-secret")}
         };
@@ -69,7 +66,7 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <returns></returns>
         public override IBrokerage CreateBrokerage(Packets.LiveNodePacket job, IAlgorithm algorithm)
         {
-            var required = new[] { "bitfinex-rest", "bitfinex-url", "bitfinex-api-secret", "bitfinex-api-key" };
+            var required = new[] { "bitfinex-api-secret", "bitfinex-api-key" };
 
             foreach (var item in required)
             {
@@ -80,8 +77,6 @@ namespace QuantConnect.Brokerages.Bitfinex
             var priceProvider = new ApiPriceProvider(job.UserId, job.UserToken);
 
             var brokerage = new BitfinexBrokerage(
-                job.BrokerageData["bitfinex-url"],
-                job.BrokerageData["bitfinex-rest"],
                 job.BrokerageData["bitfinex-api-key"],
                 job.BrokerageData["bitfinex-api-secret"],
                 algorithm,

Brokerages/Bitfinex/BitfinexSubscriptionManager.cs

@@ -13,24 +13,26 @@
  * limitations under the License.
 */
 
+using Newtonsoft.Json;
+using Newtonsoft.Json.Linq;
+using QuantConnect.Data;
+using QuantConnect.Data.Market;
+using QuantConnect.Logging;
+using QuantConnect.Util;
 using System;
 using System.Collections.Concurrent;
 using System.Collections.Generic;
 using System.Globalization;
 using System.Linq;
 using System.Threading;
-using Newtonsoft.Json;
-using Newtonsoft.Json.Linq;
-using QuantConnect.Data.Market;
-using QuantConnect.Logging;
-using QuantConnect.Util;
+using QuantConnect.Brokerages.Bitfinex.Messages;
 
 namespace QuantConnect.Brokerages.Bitfinex
 {
     /// <summary>
     /// Handles Bitfinex data subscriptions with multiple websocket connections
     /// </summary>
-    public class BitfinexSubscriptionManager
+    public class BitfinexSubscriptionManager : DataQueueHandlerSubscriptionManager
     {
         /// <summary>
         /// Maximum number of subscribed channels per websocket connection
@@ -43,13 +45,21 @@ namespace QuantConnect.Brokerages.Bitfinex
         private const int ConnectionTimeout = 30000;
 
         private readonly string _wssUrl;
+        private volatile int _subscribeErrorCode;
         private readonly object _locker = new object();
         private readonly BitfinexBrokerage _brokerage;
         private readonly BitfinexSymbolMapper _symbolMapper;
         private readonly RateGate _connectionRateLimiter = new RateGate(5, TimeSpan.FromMinutes(1));
         private readonly ConcurrentDictionary<Symbol, List<BitfinexWebSocketWrapper>> _subscriptionsBySymbol = new ConcurrentDictionary<Symbol, List<BitfinexWebSocketWrapper>>();
-        private readonly ConcurrentDictionary<BitfinexWebSocketWrapper, List<BitfinexChannel>> _channelsByWebSocket = new ConcurrentDictionary<BitfinexWebSocketWrapper, List<BitfinexChannel>>();
+        private readonly ConcurrentDictionary<BitfinexWebSocketWrapper, List<Channel>> _channelsByWebSocket = new ConcurrentDictionary<BitfinexWebSocketWrapper, List<Channel>>();
+        private readonly ConcurrentDictionary<int, Channel> _channels = new ConcurrentDictionary<int, Channel>();
         private readonly ConcurrentDictionary<Symbol, DefaultOrderBook> _orderBooks = new ConcurrentDictionary<Symbol, DefaultOrderBook>();
+        private readonly IReadOnlyDictionary<TickType, string> _tickType2ChannelName = new Dictionary<TickType, string>() {
+            { TickType.Trade, "trades"},
+            { TickType.Quote, "book"}
+        };
+        private readonly ManualResetEvent _onSubscribeEvent = new ManualResetEvent(false);
+        private readonly ManualResetEvent _onUnsubscribeEvent = new ManualResetEvent(false);
 
         /// <summary>
         /// Initializes a new instance of the <see cref="BitfinexSubscriptionManager"/> class.
@@ -62,27 +72,49 @@ namespace QuantConnect.Brokerages.Bitfinex
         }
 
         /// <summary>
-        /// Returns true if there is an active subscription for the requested symbol
+        /// Subscribes to the requested subscription (using an individual streaming channel)
         /// </summary>
-        /// <param name="symbol"></param>
-        /// <returns></returns>
-        public bool IsSubscribed(Symbol symbol)
-        {
-            return _subscriptionsBySymbol.ContainsKey(symbol);
-        }
-
-        /// <summary>
-        /// Adds a subscription for the requested symbol
-        /// </summary>
-        /// <param name="symbol">The symbol</param>
-        public void Subscribe(Symbol symbol)
+        /// <param name="symbols">symbol list</param>
+        /// <param name="tickType">Type of tick data</param>
+        protected override bool Subscribe(IEnumerable<Symbol> symbols, TickType tickType)
         {
             try
             {
-                var bookSubscription = SubscribeChannel("book", symbol);
-                var tradesSubscription = SubscribeChannel("trades", symbol);
+                var states = new List<bool>(symbols.Count());
+                foreach (var symbol in symbols)
+                {
+                    _onSubscribeEvent.Reset();
+                    _subscribeErrorCode = 0;
+                    var subscription = SubscribeChannel(
+                        ChannelNameFromTickType(tickType),
+                        symbol);
 
-                _subscriptionsBySymbol.TryAdd(symbol, new List<BitfinexWebSocketWrapper> { bookSubscription, tradesSubscription });
+                    _subscriptionsBySymbol.AddOrUpdate(
+                        symbol,
+                        new List<BitfinexWebSocketWrapper> { subscription },
+                        (k, v) =>
+                        {
+                            if (!v.Contains(subscription))
+                            {
+                                v.Add(subscription);
+                            }
+                            return v;
+                        });
+
+                    Log.Trace($"BitfinexBrokerage.Subscribe(): Sent subscribe for {symbol.Value}.");
+
+                    if (_onSubscribeEvent.WaitOne(TimeSpan.FromSeconds(10)) && _subscribeErrorCode == 0)
+                    {
+                        states.Add(true);
+                    }
+                    else
+                    {
+                        Log.Trace($"BitfinexBrokerage.Subscribe(): Could not subscribe to {symbol.Value}.");
+                        states.Add(false);
+                    }
+                }
+
+                return states.All(s => s);
             }
             catch (Exception exception)
             {
@@ -94,64 +126,90 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// <summary>
         /// Removes the subscription for the requested symbol
         /// </summary>
-        /// <param name="symbol">The symbol</param>
-        public void Unsubscribe(Symbol symbol)
+        /// <param name="symbols">symbol list</param>
+        /// <param name="tickType">Type of tick data</param>
+        protected override bool Unsubscribe(IEnumerable<Symbol> symbols, TickType tickType)
         {
-            List<BitfinexWebSocketWrapper> subscriptions;
-            if (_subscriptionsBySymbol.TryGetValue(symbol, out subscriptions))
+            string channelName = ChannelNameFromTickType(tickType);
+            var states = new List<bool>(symbols.Count());
+            foreach (var symbol in symbols)
             {
-                foreach (var webSocket in subscriptions)
+                List<BitfinexWebSocketWrapper> subscriptions;
+                if (_subscriptionsBySymbol.TryGetValue(symbol, out subscriptions))
                 {
-                    try
+                    for (int i = subscriptions.Count - 1; i >= 0; i--)
                     {
-                        lock (_locker)
+                        var webSocket = subscriptions[i];
+                        _onUnsubscribeEvent.Reset();
+                        try
                         {
-                            List<BitfinexChannel> channels;
-                            if (_channelsByWebSocket.TryGetValue(webSocket, out channels))
+                            Channel channel = new Channel(channelName, symbol);
+                            List<Channel> channels;
+                            if (_channelsByWebSocket.TryGetValue(webSocket, out channels) && channels.Contains(channel))
                             {
-                                foreach (var channel in channels)
+                                UnsubscribeChannel(webSocket, channel);
+
+                                if (_onUnsubscribeEvent.WaitOne(TimeSpan.FromSeconds(30)))
                                 {
-                                    UnsubscribeChannel(webSocket, channel.ChannelId);
+                                    states.Add(true);
+                                }
+                                else
+                                {
+                                    Log.Trace($"BitfinexBrokerage.Unsubscribe(): Could not unsubscribe from {symbol.Value}.");
+                                    states.Add(false);
                                 }
                             }
                         }
-                    }
-                    catch (Exception exception)
-                    {
-                        Log.Error(exception);
+                        catch (Exception exception)
+                        {
+                            Log.Error(exception);
+                        }
                     }
                 }
+            }
+            return states.All(s => s);
+        }
 
-                _subscriptionsBySymbol.TryRemove(symbol, out subscriptions);
+        protected override string ChannelNameFromTickType(TickType tickType)
+        {
+            string channelName;
+            if (_tickType2ChannelName.TryGetValue(tickType, out channelName))
+            {
+                return channelName;
+            }
+            else
+            {
+                throw new ArgumentOutOfRangeException("TickType", $"BitfinexSubscriptionManager.Subscribe(): Tick type {tickType} is not allowed for this brokerage.");
             }
         }
 
         private BitfinexWebSocketWrapper SubscribeChannel(string channelName, Symbol symbol)
         {
-            var ticker = _symbolMapper.GetBrokerageSymbol(symbol);
-            var channel = new BitfinexChannel { Name = channelName, Symbol = ticker, ChannelId = string.Empty };
+            var channel = new Channel(channelName, symbol);
+
             var webSocket = GetFreeWebSocket(channel);
 
             webSocket.Send(JsonConvert.SerializeObject(new
             {
                 @event = "subscribe",
                 channel = channelName,
-                pair = ticker
+                pair = _symbolMapper.GetBrokerageSymbol(symbol)
             }));
 
             return webSocket;
         }
 
-        private void UnsubscribeChannel(IWebSocket webSocket, string channelId)
+        private void UnsubscribeChannel(IWebSocket webSocket, Channel channel)
         {
+            int channelId = _channels.First(c => c.Value.Equals(channel)).Key;
             webSocket.Send(JsonConvert.SerializeObject(new
             {
                 @event = "unsubscribe",
-                channelId
+                chanId = channelId.ToStringInvariant()
             }));
         }
 
-        private BitfinexWebSocketWrapper GetFreeWebSocket(BitfinexChannel channel)
+        private BitfinexWebSocketWrapper GetFreeWebSocket(Channel channel)
         {
             int count;
 
@@ -184,7 +242,7 @@ namespace QuantConnect.Brokerages.Bitfinex
 
             lock (_locker)
             {
-                _channelsByWebSocket.TryAdd(webSocket, new List<BitfinexChannel> { channel });
+                _channelsByWebSocket.TryAdd(webSocket, new List<Channel> { channel });
 
                 count = _channelsByWebSocket.Sum(x => x.Value.Count);
                 Log.Trace($"BitfinexSubscriptionManager.GetFreeWebSocket(): Channel added: Total channels:{count}");
@@ -192,18 +250,21 @@ namespace QuantConnect.Brokerages.Bitfinex
 
             webSocket.Initialize(_wssUrl);
             webSocket.Message += OnMessage;
-            webSocket.Error += OnError;
 
             Connect(webSocket);
 
-            webSocket.ConnectionHandler.ConnectionLost += OnConnectionLost;
-            webSocket.ConnectionHandler.ConnectionRestored += OnConnectionRestored;
             webSocket.ConnectionHandler.ReconnectRequested += OnReconnectRequested;
             webSocket.ConnectionHandler.Initialize(webSocket.ConnectionId);
 
+            int connections;
+            lock (_locker)
+            {
+                connections = _channelsByWebSocket.Count;
+            }
+
             Log.Trace("BitfinexSubscriptionManager.GetFreeWebSocket(): New websocket added: " +
                       $"Hashcode: {webSocket.GetHashCode()}, " +
-                      $"WebSocket connections: {_channelsByWebSocket.Count}");
+                      $"WebSocket connections: {connections}");
 
             return webSocket;
         }
@@ -235,16 +296,6 @@ namespace QuantConnect.Brokerages.Bitfinex
             }
         }
 
-        private void OnConnectionLost(object sender, EventArgs e)
-        {
-            Log.Error("BitfinexSubscriptionManager.OnConnectionLost(): WebSocket connection lost.");
-        }
-
-        private void OnConnectionRestored(object sender, EventArgs e)
-        {
-            Log.Trace("BitfinexSubscriptionManager.OnConnectionRestored(): WebSocket connection restored.");
-        }
-
         private void OnReconnectRequested(object sender, EventArgs e)
         {
             var connectionHandler = (DefaultConnectionHandler)sender;
@@ -255,13 +306,8 @@ namespace QuantConnect.Brokerages.Bitfinex
 
             lock (_locker)
             {
-                foreach (var connection in _channelsByWebSocket.Keys)
-                {
-                    if (connection.ConnectionId == connectionHandler.ConnectionId)
-                    {
-                        webSocket = connection;
-                    }
-                }
+                webSocket = _channelsByWebSocket.Keys
+                    .FirstOrDefault(connection => connection.ConnectionId == connectionHandler.ConnectionId);
             }
 
             if (webSocket == null)
@@ -286,11 +332,13 @@ namespace QuantConnect.Brokerages.Bitfinex
 
             Log.Trace($"BitfinexSubscriptionManager.OnReconnectRequested(): Reconnected: IsOpen:{webSocket.IsOpen} [Id: {connectionHandler.ConnectionId}]");
 
-            List<BitfinexChannel> channels;
+            List<Channel> channels;
             lock (_locker)
             {
                 if (!_channelsByWebSocket.TryGetValue(webSocket, out channels))
-                return;
+                {
+                    return;
+                }
             }
 
             Log.Trace($"BitfinexSubscriptionManager.OnReconnectRequested(): Resubscribing channels. [Id: {connectionHandler.ConnectionId}]");
@@ -301,44 +349,58 @@ namespace QuantConnect.Brokerages.Bitfinex
                 {
                     @event = "subscribe",
                     channel = channel.Name,
-                    pair = channel.Symbol
+                    pair = _symbolMapper.GetBrokerageSymbol(channel.Symbol)
                 }));
             }
         }
 
-        private void OnError(object sender, WebSocketError e)
-        {
-            Log.Error($"BitfinexSubscriptionManager.OnError(): Message: {e.Message} Exception: {e.Exception}");
-        }
-
         private void OnMessage(object sender, WebSocketMessage e)
         {
-            var webSocket = (BitfinexWebSocketWrapper) sender;
+            var webSocket = (BitfinexWebSocketWrapper)sender;
 
             try
             {
                 var token = JToken.Parse(e.Message);
 
+                webSocket.ConnectionHandler.KeepAlive(DateTime.UtcNow);
+
                 if (token is JArray)
                 {
                     var channel = token[0].ToObject<int>();
-                    // heartbeat
-                    if (token[1].Type == JTokenType.String && token[1].Value<string>() == "hb")
+
+                    if (token[1].Type == JTokenType.String)
                     {
-                        webSocket.ConnectionHandler.KeepAlive(DateTime.UtcNow);
-                        return;
+                        var type = token[1].Value<string>();
+
+                        switch (type)
+                        {
+                            // heartbeat
+                            case "hb":
+                                return;
+
+                            // trade execution
+                            case "te":
+                                OnUpdate(channel, token[2].ToObject<string[]>());
+                                break;
+
+                            // ignored -- trades already handled in "te" message
+                            // https://github.com/bitfinexcom/bitfinex-api-node#te-vs-tu-messages
+                            case "tu":
+                                break;
+
+                            default:
+                                Log.Trace($"BitfinexSubscriptionManager.OnMessage(): Unexpected message type: {type}");
+                                return;
+                        }
                     }
 
                     // public channels
-                    if (channel != 0)
+                    else if (channel != 0 && token[1].Type == JTokenType.Array)
                     {
-                        webSocket.ConnectionHandler.KeepAlive(DateTime.UtcNow);
-
-                        if (token.Count() == 2)
+                        if (token[1][0].Type == JTokenType.Array)
                         {
                             OnSnapshot(
-                                webSocket,
-                                token[0].ToObject<string>(),
+                                channel,
                                 token[1].ToObject<string[][]>()
                             );
                         }
@@ -346,9 +408,8 @@ namespace QuantConnect.Brokerages.Bitfinex
                         {
                             // pass channel id as separate arg
                             OnUpdate(
-                                webSocket,
-                                token[0].ToObject<string>(),
-                                token.ToObject<string[]>().Skip(1).ToArray()
+                                channel,
+                                token[1].ToObject<string[]>()
                             );
                         }
                     }
@@ -359,18 +420,30 @@ namespace QuantConnect.Brokerages.Bitfinex
                     switch (raw.Event.ToLowerInvariant())
                     {
                         case "subscribed":
-                            OnSubscribe(webSocket, token.ToObject<Messages.ChannelSubscription>());
+                            OnSubscribe(webSocket, token.ToObject<ChannelSubscription>());
                             return;
+
                         case "unsubscribed":
-                            OnUnsubscribe(webSocket, token.ToObject<Messages.ChannelUnsubscribing>());
+                            OnUnsubscribe(webSocket, token.ToObject<ChannelUnsubscribing>());
                             return;
+
                         case "auth":
                         case "info":
                         case "ping":
                             return;
+
                         case "error":
+                            var error = token.ToObject<ErrorMessage>();
+                            // 10300 Subscription failed (generic) | 10301 : Already subscribed | 10302 : Unknown channel
+                            // see https://docs.bitfinex.com/docs/ws-general
+                            if (error.Code == 10300 || error.Code == 10301 || error.Code == 10302)
+                            {
+                                _subscribeErrorCode = error.Code;
+                                _onSubscribeEvent.Set();
+                            }
                             Log.Error($"BitfinexSubscriptionManager.OnMessage(): {e.Message}");
                             return;
+
                         default:
                             Log.Trace($"BitfinexSubscriptionManager.OnMessage(): Unexpected message format: {e.Message}");
                             break;
@@ -390,15 +463,12 @@ namespace QuantConnect.Brokerages.Bitfinex
             {
                 lock (_locker)
                 {
-                    List<BitfinexChannel> channels;
-                    if (_channelsByWebSocket.TryGetValue(webSocket, out channels))
-                    {
-                        var channel = channels.First(x => x.Name == data.Channel && x.Symbol == data.Symbol);
+                    var channel = new Channel(data.Channel, _symbolMapper.GetLeanSymbol(data.Symbol));
 
-                        channel.ChannelId = data.ChannelId;
+                    _channels.AddOrUpdate(data.ChannelId, channel);
+                    _onSubscribeEvent.Set();
 
-                        webSocket.ConnectionHandler.EnableMonitoring(true);
-                    }
+                    webSocket.ConnectionHandler.EnableMonitoring(true);
                 }
             }
             catch (Exception e)
@@ -414,10 +484,29 @@ namespace QuantConnect.Brokerages.Bitfinex
             {
                 lock (_locker)
                 {
-                    List<BitfinexChannel> channels;
+                    Channel channel;
+                    if (!_channels.TryRemove(data.ChannelId, out channel)) return;
+
+                    _onUnsubscribeEvent.Set();
+
+                    List<Channel> channels;
                     if (!_channelsByWebSocket.TryGetValue(webSocket, out channels)) return;
 
-                    channels.Remove(channels.First(x => x.ChannelId == data.ChannelId));
+                    channels.Remove(channel);
+
+                    if (channels.Count(c => c.Symbol.Equals(channel.Symbol)) == 0)
+                    {
+                        List<BitfinexWebSocketWrapper> subscriptions;
+                        if (_subscriptionsBySymbol.TryGetValue(channel.Symbol, out subscriptions))
+                        {
+                            subscriptions.Remove(webSocket);
+
+                            if (subscriptions.Count == 0)
+                            {
+                                _subscriptionsBySymbol.TryRemove(channel.Symbol, out subscriptions);
+                            }
+                        }
+                    }
 
                     if (channels.Count != 0) return;
 
@@ -434,23 +523,15 @@ namespace QuantConnect.Brokerages.Bitfinex
             }
         }
 
-        private void OnSnapshot(BitfinexWebSocketWrapper webSocket, string channelId, string[][] entries)
+        private void OnSnapshot(int channelId, string[][] entries)
         {
             try
             {
-                BitfinexChannel channel;
+                Channel channel;
 
                 lock (_locker)
                 {
-                    List<BitfinexChannel> channels;
-                    if (!_channelsByWebSocket.TryGetValue(webSocket, out channels))
-                    {
-                        _brokerage.OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, -1, $"Message received from unknown channel Id {channelId}"));
-                        return;
-                    }
-
-                    channel = channels.FirstOrDefault(x => x.ChannelId == channelId);
-                    if (channel == null)
+                    if (!_channels.TryGetValue(channelId, out channel))
                     {
                         _brokerage.OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, -1, $"Message received from unknown channel Id {channelId}"));
                         return;
@@ -462,9 +543,6 @@ namespace QuantConnect.Brokerages.Bitfinex
                     case "book":
                         ProcessOrderBookSnapshot(channel, entries);
                         return;
-                    case "trades":
-                        ProcessTradesSnapshot(channel, entries);
-                        return;
                 }
             }
             catch (Exception e)
@@ -474,11 +552,11 @@ namespace QuantConnect.Brokerages.Bitfinex
             }
         }
 
-        private void ProcessOrderBookSnapshot(BitfinexChannel channel, string[][] entries)
+        private void ProcessOrderBookSnapshot(Channel channel, string[][] entries)
         {
             try
             {
-                var symbol = _symbolMapper.GetLeanSymbol(channel.Symbol);
+                var symbol = channel.Symbol;
 
                 DefaultOrderBook orderBook;
                 if (!_orderBooks.TryGetValue(symbol, out orderBook))
@@ -514,41 +592,15 @@ namespace QuantConnect.Brokerages.Bitfinex
             }
         }
 
-        private void ProcessTradesSnapshot(BitfinexChannel channel, string[][] entries)
+        private void OnUpdate(int channelId, string[] entries)
         {
             try
             {
-                var symbol = _symbolMapper.GetLeanSymbol(channel.Symbol);
-                foreach (var entry in entries)
-                {
-                    // pass time, price, amount
-                    EmitTradeTick(symbol, entry.Skip(1).ToArray());
-                }
-            }
-            catch (Exception e)
-            {
-                Log.Error(e);
-                throw;
-            }
-        }
-
-        private void OnUpdate(BitfinexWebSocketWrapper webSocket, string channelId, string[] entries)
-        {
-            try
-            {
-                BitfinexChannel channel;
+                Channel channel;
 
                 lock (_locker)
                 {
-                    List<BitfinexChannel> channels;
-                    if (!_channelsByWebSocket.TryGetValue(webSocket, out channels))
-                    {
-                        _brokerage.OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, -1, $"Message received from unknown channel Id {channelId}"));
-                        return;
-                    }
-
-                    channel = channels.FirstOrDefault(x => x.ChannelId == channelId);
-                    if (channel == null)
+                    if (!_channels.TryGetValue(channelId, out channel))
                     {
                         _brokerage.OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, -1, $"Message received from unknown channel Id {channelId}"));
                         return;
@@ -560,6 +612,7 @@ namespace QuantConnect.Brokerages.Bitfinex
                     case "book":
                         ProcessOrderBookUpdate(channel, entries);
                         return;
+
                     case "trades":
                         ProcessTradeUpdate(channel, entries);
                         return;
@@ -572,15 +625,15 @@ namespace QuantConnect.Brokerages.Bitfinex
             }
         }
 
-        private void ProcessOrderBookUpdate(BitfinexChannel channel, string[] entries)
+        private void ProcessOrderBookUpdate(Channel channel, string[] entries)
         {
             try
             {
-                var symbol = _symbolMapper.GetLeanSymbol(channel.Symbol);
+                var symbol = channel.Symbol;
                 var orderBook = _orderBooks[symbol];
 
                 var price = decimal.Parse(entries[0], NumberStyles.Float, CultureInfo.InvariantCulture);
-                var count = Parse.Int(entries[1]);
+                var count = Parse.Long(entries[1]);
                 var amount = decimal.Parse(entries[2], NumberStyles.Float, CultureInfo.InvariantCulture);
 
                 if (count == 0)
@@ -601,22 +654,20 @@ namespace QuantConnect.Brokerages.Bitfinex
             }
             catch (Exception e)
             {
-                Log.Error(e);
+                Log.Error(e, $"Entries: [{string.Join(",", entries)}]");
                 throw;
             }
         }
 
-        private void ProcessTradeUpdate(BitfinexChannel channel, string[] entries)
+        private void ProcessTradeUpdate(Channel channel, string[] entries)
         {
             try
             {
-                string eventType = entries[0];
-                if (eventType == "tu")
-                {
-                    var symbol = _symbolMapper.GetLeanSymbol(channel.Symbol);
-                    // pass time, price, amount
-                    EmitTradeTick(symbol, new[] { entries[3], entries[4], entries[5] });
-                }
+                var time = Time.UnixMillisecondTimeStampToDateTime(double.Parse(entries[1], NumberStyles.Float, CultureInfo.InvariantCulture));
+                var amount = decimal.Parse(entries[2], NumberStyles.Float, CultureInfo.InvariantCulture);
+                var price = decimal.Parse(entries[3], NumberStyles.Float, CultureInfo.InvariantCulture);
+
+                EmitTradeTick(channel.Symbol, time, price, amount);
             }
             catch (Exception e)
             {
@@ -625,14 +676,10 @@ namespace QuantConnect.Brokerages.Bitfinex
             }
         }
 
-        private void EmitTradeTick(Symbol symbol, string[] entries)
+        private void EmitTradeTick(Symbol symbol, DateTime time, decimal price, decimal amount)
         {
             try
             {
-                var time = Time.UnixTimeStampToDateTime(double.Parse(entries[0], NumberStyles.Float, CultureInfo.InvariantCulture));
-                var price = decimal.Parse(entries[1], NumberStyles.Float, CultureInfo.InvariantCulture);
-                var amount = decimal.Parse(entries[2], NumberStyles.Float, CultureInfo.InvariantCulture);
-
                 lock (_brokerage.TickLocker)
                 {
                     _brokerage.EmitTick(new Tick

Brokerages/Bitfinex/BitfinexSymbolMapper.cs

@@ -16,6 +16,7 @@
 using System;
 using System.Collections.Generic;
 using System.Linq;
+using QuantConnect.Securities;
 
 namespace QuantConnect.Brokerages.Bitfinex
 {
@@ -24,175 +25,15 @@ namespace QuantConnect.Brokerages.Bitfinex
     /// </summary>
     public class BitfinexSymbolMapper : ISymbolMapper
     {
-        /// <summary>
-        /// Symbols that are both active and delisted
-        /// </summary>
-        public static List<Symbol> KnownSymbols
-        {
-            get
-            {
-                var symbols = new List<Symbol>();
-                var mapper = new BitfinexSymbolMapper();
-                foreach (var tp in KnownSymbolStrings)
-                {
-                    symbols.Add(mapper.GetLeanSymbol(tp, mapper.GetBrokerageSecurityType(tp), Market.Bitfinex));
-                }
-                return symbols;
-            }
-        }
-
         /// <summary>
         /// The list of known Bitfinex symbols.
         /// https://api.bitfinex.com/v1/symbols
         /// </summary>
-        public static readonly HashSet<string> KnownSymbolStrings = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
-        {
-            "BTCUSD","LTCUSD","LTCBTC","ETHUSD","ETHBTC","ETCBTC","ETCUSD","RRTUSD","RRTBTC","ZECUSD",
-            "ZECBTC","XMRUSD","XMRBTC","DSHUSD","DSHBTC","BTCEUR","BTCJPY","XRPUSD","XRPBTC","IOTUSD",
-            "IOTBTC","IOTETH","EOSUSD","EOSBTC","EOSETH","SANUSD","SANBTC","SANETH","OMGUSD","OMGBTC",
-            "OMGETH","BCHUSD","BCHBTC","BCHETH","NEOUSD","NEOBTC","NEOETH","ETPUSD","ETPBTC","ETPETH",
-            "QTMUSD","QTMBTC","QTMETH","AVTUSD","AVTBTC","AVTETH","EDOUSD","EDOBTC","EDOETH","BTGUSD",
-            "BTGBTC","DATUSD","DATBTC","DATETH","QSHUSD","QSHBTC","QSHETH","YYWUSD","YYWBTC","YYWETH",
-            "GNTUSD","GNTBTC","GNTETH","SNTUSD","SNTBTC","SNTETH","IOTEUR","BATUSD","BATBTC","BATETH",
-            "MNAUSD","MNABTC","MNAETH","FUNUSD","FUNBTC","FUNETH","ZRXUSD","ZRXBTC","ZRXETH","TNBUSD",
-            "TNBBTC","TNBETH","SPKUSD","SPKBTC","SPKETH","TRXUSD","TRXBTC","TRXETH","RCNUSD","RCNBTC",
-            "RCNETH","RLCUSD","RLCBTC","RLCETH","AIDUSD","AIDBTC","AIDETH","SNGUSD","SNGBTC","SNGETH",
-            "REPUSD","REPBTC","REPETH","ELFUSD","ELFBTC","ELFETH","BTCGBP","ETHEUR","ETHJPY","ETHGBP",
-            "NEOEUR","NEOJPY","NEOGBP","EOSEUR","EOSJPY","EOSGBP","IOTJPY","IOTGBP","IOSUSD","IOSBTC",
-            "IOSETH","AIOUSD","AIOBTC","AIOETH","REQUSD","REQBTC","REQETH","RDNUSD","RDNBTC","RDNETH",
-            "LRCUSD","LRCBTC","LRCETH","WAXUSD","WAXBTC","WAXETH","DAIUSD","DAIBTC","DAIETH","CFIUSD",
-            "CFIBTC","CFIETH","AGIUSD","AGIBTC","AGIETH","BFTUSD","BFTBTC","BFTETH","MTNUSD","MTNBTC",
-            "MTNETH","ODEUSD","ODEBTC","ODEETH","ANTUSD","ANTBTC","ANTETH","DTHUSD","DTHBTC","DTHETH",
-            "MITUSD","MITBTC","MITETH","STJUSD","STJBTC","STJETH","XLMUSD","XLMEUR","XLMJPY","XLMGBP",
-            "XLMBTC","XLMETH","XVGUSD","XVGEUR","XVGJPY","XVGGBP","XVGBTC","XVGETH","BCIUSD","BCIBTC",
-            "MKRUSD","MKRBTC","MKRETH","VENUSD","VENBTC","VENETH","KNCUSD","KNCBTC","KNCETH","POAUSD",
-            "POABTC","POAETH","LYMUSD","LYMBTC","LYMETH","UTKUSD","UTKBTC","UTKETH","VEEUSD","VEEBTC",
-            "VEEETH","DADUSD","DADBTC","DADETH","ORSUSD","ORSBTC","ORSETH","AUCUSD","AUCBTC","AUCETH",
-            "POYUSD","POYBTC","POYETH","FSNUSD","FSNBTC","FSNETH","CBTUSD","CBTBTC","CBTETH","ZCNUSD",
-            "ZCNBTC","ZCNETH","SENUSD","SENBTC","SENETH","NCAUSD","NCABTC","NCAETH","CNDUSD","CNDBTC",
-            "CNDETH","CTXUSD","CTXBTC","CTXETH","PAIUSD","PAIBTC","SEEUSD","SEEBTC","SEEETH","ESSUSD",
-            "ESSBTC","ESSETH","ATMUSD","ATMBTC","ATMETH","HOTUSD","HOTBTC","HOTETH","DTAUSD","DTABTC",
-            "DTAETH","IQXUSD","IQXBTC","IQXEOS","WPRUSD","WPRBTC","WPRETH","ZILUSD","ZILBTC","ZILETH",
-            "BNTUSD","BNTBTC","BNTETH","ABSUSD","ABSETH","XRAUSD","XRAETH","MANUSD","MANETH","BBNUSD",
-            "BBNETH","NIOUSD","NIOETH","DGXUSD","DGXETH","VETUSD","VETBTC","VETETH","UTNUSD","UTNETH",
-            "TKNUSD","TKNETH","GOTUSD","GOTEUR","GOTETH","XTZUSD","XTZBTC","CNNUSD","CNNETH","BOXUSD",
-            "BOXETH","TRXEUR","TRXGBP","TRXJPY","MGOUSD","MGOETH","RTEUSD","RTEETH","YGGUSD","YGGETH",
-            "MLNUSD","MLNETH","WTCUSD","WTCETH","CSXUSD","CSXETH","OMNUSD","OMNBTC","INTUSD","INTETH",
-            "DRNUSD","DRNETH","PNKUSD","PNKETH","DGBUSD","DGBBTC","BSVUSD","BSVBTC","BABUSD","BABBTC",
-            "WLOUSD","WLOXLM","VLDUSD","VLDETH","ENJUSD","ENJETH","ONLUSD","ONLETH","RBTUSD","RBTBTC",
-            "USTUSD","EUTEUR","EUTUSD","GSDUSD","UDCUSD","TSDUSD","PAXUSD","RIFUSD","RIFBTC","PASUSD",
-            "PASETH","VSYUSD","VSYBTC","ZRXDAI","MKRDAI","OMGDAI","BTTUSD","BTTBTC","BTCUST","ETHUST",
-            "CLOUSD","CLOBTC","IMPUSD","LTCUST","EOSUST","BABUST","SCRUSD","GNOUSD","GENUSD","ATOUSD",
-            "ATOBTC","ATOETH","WBTUSD","XCHUSD","EUSUSD","WBTETH","XCHETH","LEOUSD","LEOBTC","LEOUST",
-            "LEOEOS","LEOETH","ASTUSD","FOAUSD","UFRUSD","ZBTUSD","OKBUSD","USKUSD","GTXUSD","KANUSD",
-            "OKBUST","OKBBTC","USKUST","USKETH","USKBTC","USKEOS","GTXUST","KANUST","AMPUSD","ALGUSD",
-            "ALGBTC","ALGUST","BTCXCH","SWMUSD","TRIUSD","LOOUSD","AMPUST","UOSUSD","UOSBTC","RRBUSD",
-            "RRBUST","DTXUSD","AMPBTC","FTTUSD","FTTUST","PAXUST","UDCUST","TSDUST","CHZUSD","CHZUST",
-        };
-
-        /// <summary>
-        /// The list of delisted/invalid Bitfinex symbols.
-        /// </summary>
-        public static HashSet<string> DelistedSymbolStrings = new HashSet<string>
-        {
-            "BCHUSD","BCHBTC","BCHETH",
-            "CFIUSD","CFIBTC","CFIETH",
-            "VENUSD","VENBTC","VENETH",
-            "RRTBTC",
-            "QTMETH",
-            "AVTBTC","AVTETH",
-            "QSHBTC","QSHETH",
-            "YYWBTC","YYWETH",
-            "MNAETH",
-            "FUNBTC",
-            "SPKBTC","SPKETH",
-            "RCNBTC","RCNETH",
-            "RLCETH",
-            "AIDBTC","AIDETH",
-            "SNGBTC","SNGETH",
-            "ELFBTC","ELFETH",
-            "AIOETH",
-            "REQBTC","REQETH",
-            "RDNBTC","RDNETH",
-            "LRCETH",
-            "WAXETH",
-            "AGIBTC","AGIETH",
-            "BFTETH",
-            "MTNBTC","MTNETH",
-            "DTHBTC","DTHETH",
-            "MITBTC","MITETH",
-            "STJBTC","STJETH",
-            "XLMJPY",
-            "XVGEUR","XVGJPY","XVGGBP","XVGETH",
-            "BCIUSD","BCIBTC",
-            "KNCETH",
-            "POABTC","POAETH",
-            "LYMBTC","LYMETH",
-            "UTKBTC","UTKETH",
-            "VEEBTC","VEEETH",
-            "DADUSD","DADBTC","DADETH",
-            "ORSBTC","ORSETH",
-            "AUCBTC","AUCETH",
-            "POYBTC","POYETH",
-            "FSNBTC","FSNETH",
-            "CBTBTC","CBTETH",
-            "ZCNBTC","ZCNETH",
-            "SENUSD","SENBTC","SENETH",
-            "NCABTC","NCAETH",
-            "CNDBTC",
-            "CTXBTC","CTXETH",
-            "PAIBTC",
-            "SEEBTC","SEEETH",
-            "ESSBTC","ESSETH",
-            "ATMBTC","ATMETH",
-            "HOTBTC","HOTETH",
-            "DTABTC","DTAETH",
-            "IQXBTC",
-            "WPRBTC","WPRETH",
-            "ZILBTC","ZILETH",
-            "BNTBTC","BNTETH",
-            "ABSETH",
-            "XRAETH",
-            "MANETH",
-            "BBNUSD","BBNETH",
-            "NIOUSD","NIOETH",
-            "VETETH",
-            "UTNETH",
-            "TKNETH",
-            "CNNETH",
-            "BOXETH",
-            "MGOETH",
-            "RTEETH",
-            "YGGETH",
-            "MLNETH",
-            "WTCETH",
-            "CSXETH",
-            "INTETH",
-            "DRNETH",
-            "WLOXLM",
-            "VLDETH",
-            "ENJETH",
-            "ONLETH",
-            "PASETH",
-            "ZRXDAI",
-            "OMGDAI",
-        };
-
-        /// <summary>
-        /// The list of active Bitfinex symbols.
-        /// </summary>
-        public static List<string> ActiveSymbolStrings =
-            KnownSymbolStrings
-                .Where(x => !DelistedSymbolStrings.Contains(x))
-                .ToList();
-
-        /// <summary>
-        /// The list of known Bitfinex currencies.
-        /// </summary>
-        private static readonly HashSet<string> KnownCurrencies = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
-        {
-            "EUR", "GBP", "JPY", "USD"
-        };
+        public static readonly HashSet<string> KnownTickers =
+            new HashSet<string>(SymbolPropertiesDatabase
+                .FromDataFolder()
+                .GetSymbolPropertiesList(Market.Bitfinex, SecurityType.Crypto)
+                .Select(x => x.Key.Symbol));
 
         /// <summary>
         /// Converts a Lean symbol instance to an Bitfinex symbol
@@ -289,19 +130,8 @@ namespace QuantConnect.Brokerages.Bitfinex
             if (string.IsNullOrWhiteSpace(brokerageSymbol))
                 return false;
 
-            return KnownSymbolStrings.Contains(brokerageSymbol);
-        }
-
-        /// <summary>
-        /// Checks if the currency is supported by Bitfinex
-        /// </summary>
-        /// <returns>True if Bitfinex supports the currency</returns>
-        public bool IsKnownFiatCurrency(string currency)
-        {
-            if (string.IsNullOrWhiteSpace(currency))
-                return false;
-
-            return KnownCurrencies.Contains(currency);
+            // Strip leading 't' char
+            return KnownTickers.Contains(brokerageSymbol.Substring(1));
         }
 
         /// <summary>
@@ -324,11 +154,11 @@ namespace QuantConnect.Brokerages.Bitfinex
         /// </summary>
         private static string ConvertBitfinexSymbolToLeanSymbol(string bitfinexSymbol)
         {
-            if (string.IsNullOrWhiteSpace(bitfinexSymbol))
+            if (string.IsNullOrWhiteSpace(bitfinexSymbol) || !bitfinexSymbol.StartsWith("t"))
                 throw new ArgumentException($"Invalid Bitfinex symbol: {bitfinexSymbol}");
 
-            // return as it is due to Bitfinex has similar Symbol format
-            return bitfinexSymbol.ToUpperInvariant();
+            // Strip leading 't' char
+            return bitfinexSymbol.Substring(1).ToUpperInvariant();
         }
 
         /// <summary>
@@ -339,8 +169,8 @@ namespace QuantConnect.Brokerages.Bitfinex
             if (string.IsNullOrWhiteSpace(leanSymbol))
                 throw new ArgumentException($"Invalid Lean symbol: {leanSymbol}");
 
-            // return as it is due to Bitfinex has similar Symbol format
-            return leanSymbol.ToUpperInvariant();
+            // Prepend 't' for Trading pairs
+            return "t" + leanSymbol.ToUpperInvariant();
         }
     }
 }

Brokerages/Bitfinex/Converters/OrderConverter.cs

@@ -0,0 +1,83 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using Newtonsoft.Json;
+using Newtonsoft.Json.Linq;
+using QuantConnect.Brokerages.Bitfinex.Messages;
+
+namespace QuantConnect.Brokerages.Bitfinex.Converters
+{
+    /// <summary>
+    /// A custom JSON converter for the Bitfinex <see cref="Order"/> class
+    /// </summary>
+    public class OrderConverter : JsonConverter
+    {
+        /// <summary>
+        /// Gets a value indicating whether this <see cref="T:Newtonsoft.Json.JsonConverter" /> can write JSON.
+        /// </summary>
+        /// <value><c>true</c> if this <see cref="T:Newtonsoft.Json.JsonConverter" /> can write JSON; otherwise, <c>false</c>.</value>
+        public override bool CanWrite => false;
+
+        /// <summary>Writes the JSON representation of the object.</summary>
+        /// <param name="writer">The <see cref="T:Newtonsoft.Json.JsonWriter" /> to write to.</param>
+        /// <param name="value">The value.</param>
+        /// <param name="serializer">The calling serializer.</param>
+        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
+        {
+            throw new NotImplementedException();
+        }
+
+        /// <summary>Reads the JSON representation of the object.</summary>
+        /// <param name="reader">The <see cref="T:Newtonsoft.Json.JsonReader" /> to read from.</param>
+        /// <param name="objectType">Type of the object.</param>
+        /// <param name="existingValue">The existing value of object being read.</param>
+        /// <param name="serializer">The calling serializer.</param>
+        /// <returns>The object value.</returns>
+        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
+        {
+            var array = JArray.Load(reader);
+
+            return new Order
+            {
+                Id = array[0].Type == JTokenType.Null ? 0 : (long)array[0],
+                ClientOrderId = array[2].Type == JTokenType.Null ? 0 : (long)array[2],
+                Symbol = array[3].Type == JTokenType.Null ? string.Empty : (string)array[3],
+                MtsCreate = array[4].Type == JTokenType.Null ? 0 : (long)array[4],
+                MtsUpdate = array[5].Type == JTokenType.Null ? 0 : (long)array[5],
+                Amount = array[6].Type == JTokenType.Null ? 0 : Convert.ToDecimal((double)array[6]),
+                AmountOrig = array[7].Type == JTokenType.Null ? 0 : Convert.ToDecimal((double)array[7]),
+                Type = array[8].Type == JTokenType.Null ? string.Empty : (string)array[8],
+                Status = array[13].Type == JTokenType.Null ? string.Empty : (string)array[13],
+                Price = array[16].Type == JTokenType.Null ? 0 : Convert.ToDecimal((double)array[16]),
+                PriceAvg = array[17].Type == JTokenType.Null ? 0 : Convert.ToDecimal((double)array[17]),
+                PriceTrailing = array[18].Type == JTokenType.Null ? 0 : Convert.ToDecimal((double)array[18]),
+                PriceAuxLimit = array[19].Type == JTokenType.Null ? 0 : Convert.ToDecimal((double)array[19])
+            };
+        }
+
+        /// <summary>
+        /// Determines whether this instance can convert the specified object type.
+        /// </summary>
+        /// <param name="objectType">Type of the object.</param>
+        /// <returns>
+        /// 	<c>true</c> if this instance can convert the specified object type; otherwise, <c>false</c>.
+        /// </returns>
+        public override bool CanConvert(Type objectType)
+        {
+            return objectType == typeof(Order);
+        }
+    }
+}

Brokerages/Bitfinex/Converters/PositionConverter.cs

@@ -0,0 +1,85 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using Newtonsoft.Json;
+using Newtonsoft.Json.Linq;
+using QuantConnect.Brokerages.Bitfinex.Messages;
+
+namespace QuantConnect.Brokerages.Bitfinex.Converters
+{
+    /// <summary>
+    /// A custom JSON converter for the Bitfinex <see cref="Position"/> class
+    /// </summary>
+    public class PositionConverter : JsonConverter
+    {
+        /// <summary>
+        /// Gets a value indicating whether this <see cref="T:Newtonsoft.Json.JsonConverter" /> can write JSON.
+        /// </summary>
+        /// <value><c>true</c> if this <see cref="T:Newtonsoft.Json.JsonConverter" /> can write JSON; otherwise, <c>false</c>.</value>
+        public override bool CanWrite => false;
+
+        /// <summary>Writes the JSON representation of the object.</summary>
+        /// <param name="writer">The <see cref="T:Newtonsoft.Json.JsonWriter" /> to write to.</param>
+        /// <param name="value">The value.</param>
+        /// <param name="serializer">The calling serializer.</param>
+        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
+        {
+            throw new NotImplementedException();
+        }
+
+        /// <summary>Reads the JSON representation of the object.</summary>
+        /// <param name="reader">The <see cref="T:Newtonsoft.Json.JsonReader" /> to read from.</param>
+        /// <param name="objectType">Type of the object.</param>
+        /// <param name="existingValue">The existing value of object being read.</param>
+        /// <param name="serializer">The calling serializer.</param>
+        /// <returns>The object value.</returns>
+        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
+        {
+            var array = JArray.Load(reader);
+
+            return new Position
+            {
+                Symbol = (string)array[0],
+                Status = (string)array[1],
+                Amount = Convert.ToDecimal((double)array[2]),
+                BasePrice = Convert.ToDecimal((double)array[3]),
+                MarginFunding = Convert.ToDecimal((double)array[4]),
+                MarginFundingType = (int)array[5],
+                ProfitLoss = Convert.ToDecimal((double)array[6]),
+                ProfitLossPerc = Convert.ToDecimal((double)array[7]),
+                PriceLiq = Convert.ToDecimal((double)array[8]),
+                Leverage = Convert.ToDecimal((double)array[9]),
+                PositionId = (long)array[11],
+                Type = (int)array[15],
+                Collateral = Convert.ToDecimal((double)array[17]),
+                CollateralMin = Convert.ToDecimal((double)array[18]),
+                Meta = array[19]
+            };
+        }
+
+        /// <summary>
+        /// Determines whether this instance can convert the specified object type.
+        /// </summary>
+        /// <param name="objectType">Type of the object.</param>
+        /// <returns>
+        /// 	<c>true</c> if this instance can convert the specified object type; otherwise, <c>false</c>.
+        /// </returns>
+        public override bool CanConvert(Type objectType)
+        {
+            return objectType == typeof(Position);
+        }
+    }
+}

Brokerages/Bitfinex/Converters/TickerConverter.cs

@@ -0,0 +1,80 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using Newtonsoft.Json;
+using Newtonsoft.Json.Linq;
+using QuantConnect.Brokerages.Bitfinex.Messages;
+
+namespace QuantConnect.Brokerages.Bitfinex.Converters
+{
+    /// <summary>
+    /// A custom JSON converter for the Bitfinex <see cref="Ticker"/> class
+    /// </summary>
+    public class TickerConverter : JsonConverter
+    {
+        /// <summary>
+        /// Gets a value indicating whether this <see cref="T:Newtonsoft.Json.JsonConverter" /> can write JSON.
+        /// </summary>
+        /// <value><c>true</c> if this <see cref="T:Newtonsoft.Json.JsonConverter" /> can write JSON; otherwise, <c>false</c>.</value>
+        public override bool CanWrite => false;
+
+        /// <summary>Writes the JSON representation of the object.</summary>
+        /// <param name="writer">The <see cref="T:Newtonsoft.Json.JsonWriter" /> to write to.</param>
+        /// <param name="value">The value.</param>
+        /// <param name="serializer">The calling serializer.</param>
+        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
+        {
+            throw new NotImplementedException();
+        }
+
+        /// <summary>Reads the JSON representation of the object.</summary>
+        /// <param name="reader">The <see cref="T:Newtonsoft.Json.JsonReader" /> to read from.</param>
+        /// <param name="objectType">Type of the object.</param>
+        /// <param name="existingValue">The existing value of object being read.</param>
+        /// <param name="serializer">The calling serializer.</param>
+        /// <returns>The object value.</returns>
+        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
+        {
+            var array = JArray.Load(reader);
+
+            return new Ticker
+            {
+                Bid = Convert.ToDecimal((double)array[0]),
+                BidSize = Convert.ToDecimal((double)array[1]),
+                Ask = Convert.ToDecimal((double)array[2]),
+                AskSize = Convert.ToDecimal((double)array[3]),
+                DailyChange = Convert.ToDecimal((double)array[4]),
+                DailyChangeRelative = Convert.ToDecimal((double)array[5]),
+                LastPrice = Convert.ToDecimal((double)array[6]),
+                Volume = Convert.ToDecimal((double)array[7]),
+                High = Convert.ToDecimal((double)array[8]),
+                Low = Convert.ToDecimal((double)array[9])
+            };
+        }
+
+        /// <summary>
+        /// Determines whether this instance can convert the specified object type.
+        /// </summary>
+        /// <param name="objectType">Type of the object.</param>
+        /// <returns>
+        /// 	<c>true</c> if this instance can convert the specified object type; otherwise, <c>false</c>.
+        /// </returns>
+        public override bool CanConvert(Type objectType)
+        {
+            return objectType == typeof(Ticker);
+        }
+    }
+}

Brokerages/Bitfinex/Converters/TradeExecutionUpdateConverter.cs

@@ -0,0 +1,81 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using Newtonsoft.Json;
+using Newtonsoft.Json.Linq;
+using QuantConnect.Brokerages.Bitfinex.Messages;
+
+namespace QuantConnect.Brokerages.Bitfinex.Converters
+{
+    /// <summary>
+    /// A custom JSON converter for the Bitfinex <see cref="TradeExecutionUpdate"/> class
+    /// </summary>
+    public class TradeExecutionUpdateConverter : JsonConverter
+    {
+        /// <summary>
+        /// Gets a value indicating whether this <see cref="T:Newtonsoft.Json.JsonConverter" /> can write JSON.
+        /// </summary>
+        /// <value><c>true</c> if this <see cref="T:Newtonsoft.Json.JsonConverter" /> can write JSON; otherwise, <c>false</c>.</value>
+        public override bool CanWrite => false;
+
+        /// <summary>Writes the JSON representation of the object.</summary>
+        /// <param name="writer">The <see cref="T:Newtonsoft.Json.JsonWriter" /> to write to.</param>
+        /// <param name="value">The value.</param>
+        /// <param name="serializer">The calling serializer.</param>
+        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
+        {
+            throw new NotImplementedException();
+        }
+
+        /// <summary>Reads the JSON representation of the object.</summary>
+        /// <param name="reader">The <see cref="T:Newtonsoft.Json.JsonReader" /> to read from.</param>
+        /// <param name="objectType">Type of the object.</param>
+        /// <param name="existingValue">The existing value of object being read.</param>
+        /// <param name="serializer">The calling serializer.</param>
+        /// <returns>The object value.</returns>
+        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
+        {
+            var array = JArray.Load(reader);
+
+            return new TradeExecutionUpdate
+            {
+                TradeId = (long)array[0],
+                Symbol = (string)array[1],
+                MtsCreate = (long)array[2],
+                OrderId = (long)array[3],
+                ExecAmount = Convert.ToDecimal((double)array[4]),
+                ExecPrice = Convert.ToDecimal((double)array[5]),
+                OrderType = (string)array[6],
+                OrderPrice = Convert.ToDecimal((double)array[7]),
+                Maker = (int)array[8],
+                Fee = Convert.ToDecimal((double)array[9]),
+                FeeCurrency = (string)array[10]
+            };
+        }
+
+        /// <summary>
+        /// Determines whether this instance can convert the specified object type.
+        /// </summary>
+        /// <param name="objectType">Type of the object.</param>
+        /// <returns>
+        /// 	<c>true</c> if this instance can convert the specified object type; otherwise, <c>false</c>.
+        /// </returns>
+        public override bool CanConvert(Type objectType)
+        {
+            return objectType == typeof(TradeExecutionUpdate);
+        }
+    }
+}

Brokerages/Bitfinex/Converters/WalletConverter.cs

@@ -0,0 +1,74 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using Newtonsoft.Json;
+using Newtonsoft.Json.Linq;
+using QuantConnect.Brokerages.Bitfinex.Messages;
+
+namespace QuantConnect.Brokerages.Bitfinex.Converters
+{
+    /// <summary>
+    /// A custom JSON converter for the Bitfinex <see cref="Wallet"/> class
+    /// </summary>
+    public class WalletConverter : JsonConverter
+    {
+        /// <summary>
+        /// Gets a value indicating whether this <see cref="T:Newtonsoft.Json.JsonConverter" /> can write JSON.
+        /// </summary>
+        /// <value><c>true</c> if this <see cref="T:Newtonsoft.Json.JsonConverter" /> can write JSON; otherwise, <c>false</c>.</value>
+        public override bool CanWrite => false;
+
+        /// <summary>Writes the JSON representation of the object.</summary>
+        /// <param name="writer">The <see cref="T:Newtonsoft.Json.JsonWriter" /> to write to.</param>
+        /// <param name="value">The value.</param>
+        /// <param name="serializer">The calling serializer.</param>
+        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
+        {
+            throw new NotImplementedException();
+        }
+
+        /// <summary>Reads the JSON representation of the object.</summary>
+        /// <param name="reader">The <see cref="T:Newtonsoft.Json.JsonReader" /> to read from.</param>
+        /// <param name="objectType">Type of the object.</param>
+        /// <param name="existingValue">The existing value of object being read.</param>
+        /// <param name="serializer">The calling serializer.</param>
+        /// <returns>The object value.</returns>
+        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
+        {
+            var array = JArray.Load(reader);
+
+            return new Wallet
+            {
+                Type = (string)array[0],
+                Currency = (string)array[1],
+                Balance = Convert.ToDecimal((double)array[2]),
+                UnsettledInterest = Convert.ToDecimal((double)array[3])
+            };
+        }
+
+        /// <summary>
+        /// Determines whether this instance can convert the specified object type.
+        /// </summary>
+        /// <param name="objectType">Type of the object.</param>
+        /// <returns>
+        /// 	<c>true</c> if this instance can convert the specified object type; otherwise, <c>false</c>.
+        /// </returns>
+        public override bool CanConvert(Type objectType)
+        {
+            return objectType == typeof(Wallet);
+        }
+    }
+}

Brokerages/Bitfinex/Messages/Messages.cs

@@ -14,14 +14,9 @@
 */
 
 using Newtonsoft.Json;
-using QuantConnect.Orders;
-using System;
-using System.Collections.Generic;
-using System.Globalization;
 
 namespace QuantConnect.Brokerages.Bitfinex.Messages
 {
-
     //several simple objects to facilitate json conversion
 #pragma warning disable 1591
 
@@ -44,77 +39,23 @@ namespace QuantConnect.Brokerages.Bitfinex.Messages
         public string Level => Code == 10301 ? "Warning" : "Error";
     }
 
-    public class Order
-    {
-        public string Id { get; set; }
-        public decimal Price { get; set; }
-        [JsonProperty("avg_execution_price")]
-        public decimal PriceAvg { get; set; }
-        public string Symbol { get; set; }
-        public string Type { get; set; }
-        public string Side { get; set; }
-        public double Timestamp { get; set; }
-        [JsonProperty("is_live")]
-        public bool IsLive { get; set; }
-        [JsonProperty("is_cancelled")]
-        public bool IsCancelled { get; set; }
-        [JsonProperty("original_amount")]
-        public decimal OriginalAmount { get; set; }
-        [JsonProperty("remaining_amount")]
-        public decimal RemainingAmount { get; set; }
-        [JsonProperty("executed_amount")]
-        public decimal ExecutedAmount { get; set; }
-
-        public bool IsExchange => Type.StartsWith("exchange", StringComparison.OrdinalIgnoreCase);
-    }
-
-    public class Position
-    {
-        public int Id { get; set; }
-        public string Symbol { get; set; }
-        [JsonProperty("base")]
-        public decimal AveragePrice { get; set; }
-        public decimal Amount { get; set; }
-        public double Timestamp { get; set; }
-        public decimal Swap { get; set; }
-        public decimal PL { get; set; }
-    }
-
-    public class Wallet
-    {
-        public string Type { get; set; }
-        public string Currency { get; set; }
-        public decimal Amount { get; set; }
-        public decimal Available { get; set; }
-    }
-
-    public class Tick
-    {
-        public decimal Mid { get; set; }
-        public decimal Bid { get; set; }
-        public decimal Ask { get; set; }
-        [JsonProperty("last_price")]
-        public decimal LastPrice { get; set; }
-        public decimal Low { get; set; }
-        public decimal High { get; set; }
-        public decimal Volume { get; set; }
-        public double Timestamp { get; set; }
-    }
-
     public class ChannelSubscription : BaseMessage
     {
         public string Channel { get; set; }
+
         [JsonProperty("chanId")]
-        public string ChannelId { get; set; }
-        [JsonProperty("pair")]
+        public int ChannelId { get; set; }
+
+        [JsonProperty("symbol")]
         public string Symbol { get; set; }
     }
 
     public class ChannelUnsubscribing : BaseMessage
     {
         public string Status { get; set; }
+
         [JsonProperty("chanId")]
-        public string ChannelId { get; set; }
+        public int ChannelId { get; set; }
     }
 
     public class AuthResponseMessage: BaseMessage

Brokerages/Bitfinex/Messages/Order.cs

@@ -0,0 +1,131 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using System;
+using Newtonsoft.Json;
+using QuantConnect.Brokerages.Bitfinex.Converters;
+
+namespace QuantConnect.Brokerages.Bitfinex.Messages
+{
+    /// <summary>
+    /// Bitfinex Order
+    /// </summary>
+    [JsonConverter(typeof(OrderConverter))]
+    public class Order
+    {
+        /// <summary>
+        /// Order ID
+        /// </summary>
+        public long Id { get; set; }
+
+        /// <summary>
+        /// Group ID
+        /// </summary>
+        public long GroupId { get; set; }
+
+        /// <summary>
+        /// Client Order ID
+        /// </summary>
+        public long ClientOrderId { get; set; }
+
+        /// <summary>
+        /// Pair (tBTCUSD, …)
+        /// </summary>
+        public string Symbol { get; set; }
+
+        /// <summary>
+        /// Millisecond timestamp of creation
+        /// </summary>
+        public long MtsCreate { get; set; }
+
+        /// <summary>
+        /// Millisecond timestamp of update
+        /// </summary>
+        public long MtsUpdate { get; set; }
+
+        /// <summary>
+        /// Positive means buy, negative means sell.
+        /// </summary>
+        public decimal Amount { get; set; }
+
+        /// <summary>
+        /// Original amount
+        /// </summary>
+        public decimal AmountOrig { get; set; }
+
+        /// <summary>
+        /// The type of the order:
+        /// - LIMIT, MARKET, STOP, STOP LIMIT, TRAILING STOP,
+        /// - EXCHANGE MARKET, EXCHANGE LIMIT, EXCHANGE STOP, EXCHANGE STOP LIMIT,
+        /// - EXCHANGE TRAILING STOP, FOK, EXCHANGE FOK, IOC, EXCHANGE IOC.
+        /// </summary>
+        public string Type { get; set; }
+
+        /// <summary>
+        /// Previous order type
+        /// </summary>
+        public string TypePrev { get; set; }
+
+        /// <summary>
+        /// Active flags for order
+        /// </summary>
+        public int Flags { get; set; }
+
+        /// <summary>
+        /// Order Status:
+        /// - ACTIVE,
+        /// - EXECUTED @ PRICE(AMOUNT) e.g. "EXECUTED @ 107.6(-0.2)",
+        /// - PARTIALLY FILLED @ PRICE(AMOUNT),
+        /// - INSUFFICIENT MARGIN was: PARTIALLY FILLED @ PRICE(AMOUNT),
+        /// - CANCELED,
+        /// - CANCELED was: PARTIALLY FILLED @ PRICE(AMOUNT),
+        /// - RSN_DUST (amount is less than 0.00000001),
+        /// - RSN_PAUSE (trading is paused / paused due to AMPL rebase event)
+        /// </summary>
+        public string Status { get; set; }
+
+        /// <summary>
+        /// Price
+        /// </summary>
+        public decimal Price { get; set; }
+
+        /// <summary>
+        /// Average price
+        /// </summary>
+        public decimal PriceAvg { get; set; }
+
+        /// <summary>
+        /// The trailing price
+        /// </summary>
+        public decimal PriceTrailing { get; set; }
+
+        /// <summary>
+        /// Auxiliary Limit price (for STOP LIMIT)
+        /// </summary>
+        public decimal PriceAuxLimit { get; set; }
+
+        /// <summary>
+        /// 1 if Hidden, 0 if not hidden
+        /// </summary>
+        public int Hidden { get; set; }
+
+        /// <summary>
+        /// If another order caused this order to be placed (OCO) this will be that other order's ID
+        /// </summary>
+        public int PlacedId { get; set; }
+
+        public bool IsExchange => Type.StartsWith("EXCHANGE", StringComparison.OrdinalIgnoreCase);
+    }
+}

Brokerages/Bitfinex/Messages/OrderFlags.cs

@@ -0,0 +1,54 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+namespace QuantConnect.Brokerages.Bitfinex.Messages
+{
+    /// <summary>
+    /// Bitfinex Order Flags
+    /// </summary>
+    public static class OrderFlags
+    {
+        /// <summary>
+        /// The hidden order option ensures an order does not appear in the order book; thus does not influence other market participants.
+        /// </summary>
+        public const int Hidden = 64;
+
+        /// <summary>
+        /// Close position if position present.
+        /// </summary>
+        public const int Close = 512;
+
+        /// <summary>
+        /// Ensures that the executed order does not flip the opened position.
+        /// </summary>
+        public const int ReduceOnly = 1024;
+
+        /// <summary>
+        /// The post-only limit order option ensures the limit order will be added to the order book and not match with a pre-existing order.
+        /// </summary>
+        public const int PostOnly = 4096;
+
+        /// <summary>
+        /// The one cancels other order option allows you to place a pair of orders stipulating that if one order is executed fully or partially,
+        /// then the other is automatically canceled.
+        /// </summary>
+        public const int Oco = 16384;
+
+        /// <summary>
+        /// Excludes variable rate funding offers from matching against this order, if on margin
+        /// </summary>
+        public const int NoVarRates = 524288;
+    }
+}

Brokerages/Bitfinex/Messages/Position.cs

@@ -0,0 +1,112 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using Newtonsoft.Json;
+using QuantConnect.Brokerages.Bitfinex.Converters;
+
+namespace QuantConnect.Brokerages.Bitfinex.Messages
+{
+    /// <summary>
+    /// Bitfinex position
+    /// </summary>
+    [JsonConverter(typeof(PositionConverter))]
+    public class Position
+    {
+        /// <summary>
+        /// Pair (tBTCUSD, …).
+        /// </summary>
+        public string Symbol { get; set; }
+
+        /// <summary>
+        /// Status (ACTIVE, CLOSED).
+        /// </summary>
+        public string Status { get; set; }
+
+        /// <summary>
+        /// Size of the position. A positive value indicates a long position; a negative value indicates a short position.
+        /// </summary>
+        public decimal Amount { get; set; }
+
+        /// <summary>
+        /// Base price of the position. (Average traded price of the previous orders of the position)
+        /// </summary>
+        public decimal BasePrice { get; set; }
+
+        /// <summary>
+        /// The amount of funding being used for this position.
+        /// </summary>
+        public decimal MarginFunding { get; set; }
+
+        /// <summary>
+        /// 0 for daily, 1 for term.
+        /// </summary>
+        public int MarginFundingType { get; set; }
+
+        /// <summary>
+        /// Profit &amp; Loss
+        /// </summary>
+        public decimal ProfitLoss { get; set; }
+
+        /// <summary>
+        /// Profit &amp; Loss Percentage
+        /// </summary>
+        public decimal ProfitLossPerc { get; set; }
+
+        /// <summary>
+        /// Liquidation price
+        /// </summary>
+        public decimal PriceLiq { get; set; }
+
+        /// <summary>
+        /// Leverage used for the position
+        /// </summary>
+        public decimal Leverage { get; set; }
+
+        /// <summary>
+        /// Position ID
+        /// </summary>
+        public long PositionId { get; set; }
+
+        /// <summary>
+        /// Millisecond timestamp of creation
+        /// </summary>
+        public long MtsCreate { get; set; }
+
+        /// <summary>
+        /// Millisecond timestamp of update
+        /// </summary>
+        public long MtsUpdate { get; set; }
+
+        /// <summary>
+        /// Identifies the type of position, 0 = Margin position, 1 = Derivatives position
+        /// </summary>
+        public int Type { get; set; }
+
+        /// <summary>
+        /// The amount of collateral applied to the open position
+        /// </summary>
+        public decimal Collateral { get; set; }
+
+        /// <summary>
+        /// The minimum amount of collateral required for the position
+        /// </summary>
+        public decimal CollateralMin { get; set; }
+
+        /// <summary>
+        /// Additional meta information about the position (JSON string)
+        /// </summary>
+        public object Meta { get; set; }
+    }
+}

Brokerages/Bitfinex/Messages/Ticker.cs

@@ -0,0 +1,77 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using Newtonsoft.Json;
+using QuantConnect.Brokerages.Bitfinex.Converters;
+
+namespace QuantConnect.Brokerages.Bitfinex.Messages
+{
+    /// <summary>
+    /// A high level overview of the state of the market for a specified pair
+    /// </summary>
+    [JsonConverter(typeof(TickerConverter))]
+    public class Ticker
+    {
+        /// <summary>
+        /// Price of last highest bid
+        /// </summary>
+        public decimal Bid { get; set; }
+
+        /// <summary>
+        /// Sum of the 25 highest bid sizes
+        /// </summary>
+        public decimal BidSize { get; set; }
+
+        /// <summary>
+        /// Price of last lowest ask
+        /// </summary>
+        public decimal Ask { get; set; }
+
+        /// <summary>
+        /// Sum of the 25 lowest ask sizes
+        /// </summary>
+        public decimal AskSize { get; set; }
+
+        /// <summary>
+        /// Amount that the last price has changed since yesterday
+        /// </summary>
+        public decimal DailyChange { get; set; }
+
+        /// <summary>
+        /// Relative price change since yesterday (*100 for percentage change)
+        /// </summary>
+        public decimal DailyChangeRelative { get; set; }
+
+        /// <summary>
+        /// Price of the last trade
+        /// </summary>
+        public decimal LastPrice { get; set; }
+
+        /// <summary>
+        /// Daily volume
+        /// </summary>
+        public decimal Volume { get; set; }
+
+        /// <summary>
+        /// Daily high
+        /// </summary>
+        public decimal High { get; set; }
+
+        /// <summary>
+        /// Daily low
+        /// </summary>
+        public decimal Low { get; set; }
+    }
+}

Brokerages/Bitfinex/Messages/TradeExecutionUpdate.cs

@@ -0,0 +1,82 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using Newtonsoft.Json;
+using QuantConnect.Brokerages.Bitfinex.Converters;
+
+namespace QuantConnect.Brokerages.Bitfinex.Messages
+{
+    /// <summary>
+    /// Trade execution event on the account.
+    /// </summary>
+    [JsonConverter(typeof(TradeExecutionUpdateConverter))]
+    public class TradeExecutionUpdate
+    {
+        /// <summary>
+        /// Trade database id
+        /// </summary>
+        public long TradeId { get; set; }
+
+        /// <summary>
+        /// Symbol (tBTCUSD, …)
+        /// </summary>
+        public string Symbol { get; set; }
+
+        /// <summary>
+        /// Execution timestamp
+        /// </summary>
+        public long MtsCreate { get; set; }
+
+        /// <summary>
+        /// Order id
+        /// </summary>
+        public long OrderId { get; set; }
+
+        /// <summary>
+        /// Positive means buy, negative means sell
+        /// </summary>
+        public decimal ExecAmount { get; set; }
+
+        /// <summary>
+        /// Execution price
+        /// </summary>
+        public decimal ExecPrice { get; set; }
+
+        /// <summary>
+        /// Order type
+        /// </summary>
+        public string OrderType { get; set; }
+
+        /// <summary>
+        /// Order price
+        /// </summary>
+        public decimal OrderPrice { get; set; }
+
+        /// <summary>
+        /// 1 if true, -1 if false
+        /// </summary>
+        public int Maker { get; set; }
+
+        /// <summary>
+        /// Fee ('tu' only)
+        /// </summary>
+        public decimal Fee { get; set; }
+
+        /// <summary>
+        /// Fee currency ('tu' only)
+        /// </summary>
+        public string FeeCurrency { get; set; }
+    }
+}

Brokerages/Bitfinex/Messages/Wallet.cs

@@ -0,0 +1,47 @@
+/*
+ * QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
+ * Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+*/
+
+using Newtonsoft.Json;
+using QuantConnect.Brokerages.Bitfinex.Converters;
+
+namespace QuantConnect.Brokerages.Bitfinex.Messages
+{
+    /// <summary>
+    /// Account wallet balance
+    /// </summary>
+    [JsonConverter(typeof(WalletConverter))]
+    public class Wallet
+    {
+        /// <summary>
+        /// Wallet name (exchange, margin, funding)
+        /// </summary>
+        public string Type { get; set; }
+
+        /// <summary>
+        /// Currency (e.g. USD, ...)
+        /// </summary>
+        public string Currency { get; set; }
+
+        /// <summary>
+        /// Wallet balance
+        /// </summary>
+        public decimal Balance { get; set; }
+
+        /// <summary>
+        /// Unsettled interest
+        /// </summary>
+        public decimal UnsettledInterest { get; set; }
+    }
+}

Brokerages/Brokerage.cs

@@ -124,9 +124,13 @@ namespace QuantConnect.Brokerages
         {
             try
             {
-                Log.Debug("Brokerage.OnOrderEvent(): " + e);
-
                 OrderStatusChanged?.Invoke(this, e);
+
+                if (Log.DebuggingEnabled)
+                {
+                    // log after calling the OrderStatusChanged event, the BrokerageTransactionHandler will set the order quantity
+                    Log.Debug("Brokerage.OnOrderEvent(): " + e);
+                }
             }
             catch (Exception err)
             {

Brokerages/DefaultConnectionHandler.cs

@@ -192,6 +192,7 @@ namespace QuantConnect.Brokerages
         /// </summary>
         protected virtual void OnConnectionLost()
         {
+            Log.Error("DefaultConnectionHandler.OnConnectionLost(): WebSocket connection lost.");
             ConnectionLost?.Invoke(this, EventArgs.Empty);
         }
 
@@ -200,6 +201,7 @@ namespace QuantConnect.Brokerages
         /// </summary>
         protected virtual void OnConnectionRestored()
         {
+            Log.Trace("DefaultConnectionHandler.OnConnectionRestored(): WebSocket connection restored.");
             ConnectionRestored?.Invoke(this, EventArgs.Empty);
         }
 

Brokerages/Fxcm/FxcmBrokerage.DataQueueHandler.cs

@@ -31,8 +31,6 @@ namespace QuantConnect.Brokerages.Fxcm
     /// </summary>
     public partial class FxcmBrokerage
     {
-        private readonly HashSet<Symbol> _subscribedSymbols = new HashSet<Symbol>();
-
         #region IDataQueueHandler implementation
 
         /// <summary>
@@ -51,8 +49,13 @@ namespace QuantConnect.Brokerages.Fxcm
         /// <returns>The new enumerator for this subscription request</returns>
         public IEnumerator<BaseData> Subscribe(SubscriptionDataConfig dataConfig, EventHandler newDataAvailableHandler)
         {
+            if (!CanSubscribe(dataConfig.Symbol))
+            {
+                return Enumerable.Empty<BaseData>().GetEnumerator();
+            }
+
             var enumerator = _aggregator.Add(dataConfig, newDataAvailableHandler);
-            Subscribe(new[] { dataConfig.Symbol });
+            _subscriptionManager.Subscribe(dataConfig);
 
             return enumerator;
         }
@@ -61,18 +64,10 @@ namespace QuantConnect.Brokerages.Fxcm
         /// Adds the specified symbols to the subscription
         /// </summary>
         /// <param name="symbols">The symbols to be added keyed by SecurityType</param>
-        private void Subscribe(IEnumerable<Symbol> symbols)
+        private bool Subscribe(IEnumerable<Symbol> symbols)
         {
-            var symbolsToSubscribe = (from symbol in symbols 
-                                      where !_subscribedSymbols.Contains(symbol) && CanSubscribe(symbol)
-                                      select symbol).ToList();
-            if (symbolsToSubscribe.Count == 0)
-                return;
-
-            Log.Trace("FxcmBrokerage.Subscribe(): {0}", string.Join(",", symbolsToSubscribe));
-
             var request = new MarketDataRequest();
-            foreach (var symbol in symbolsToSubscribe)
+            foreach (var symbol in symbols)
             {
                 TradingSecurity fxcmSecurity;
                 if (_fxcmInstruments.TryGetValue(_symbolMapper.GetBrokerageSymbol(symbol), out fxcmSecurity))
@@ -97,10 +92,7 @@ namespace QuantConnect.Brokerages.Fxcm
                 _gateway.sendMessage(request);
             }
 
-            foreach (var symbol in symbolsToSubscribe)
-            {
-                _subscribedSymbols.Add(symbol);
-            }
+            return true;
         }
 
         /// <summary>
@@ -109,7 +101,7 @@ namespace QuantConnect.Brokerages.Fxcm
         /// <param name="dataConfig">Subscription config to be removed</param>
         public void Unsubscribe(SubscriptionDataConfig dataConfig)
         {
-            Unsubscribe(new Symbol[] { dataConfig.Symbol });
+            _subscriptionManager.Unsubscribe(dataConfig);
             _aggregator.Remove(dataConfig);
         }
 
@@ -117,18 +109,12 @@ namespace QuantConnect.Brokerages.Fxcm
         /// Removes the specified symbols to the subscription
         /// </summary>
         /// <param name="symbols">The symbols to be removed keyed by SecurityType</param>
-        public void Unsubscribe(IEnumerable<Symbol> symbols)
+        private bool Unsubscribe(IEnumerable<Symbol> symbols)
         {
-            var symbolsToUnsubscribe = (from symbol in symbols 
-                                        where _subscribedSymbols.Contains(symbol) 
-                                        select symbol).ToList();
-            if (symbolsToUnsubscribe.Count == 0)
-                return;
-
-            Log.Trace("FxcmBrokerage.Unsubscribe(): {0}", string.Join(",", symbolsToUnsubscribe));
+            Log.Trace("FxcmBrokerage.Unsubscribe(): {0}", string.Join(",", symbols));
 
             var request = new MarketDataRequest();
-            foreach (var symbol in symbolsToUnsubscribe)
+            foreach (var symbol in symbols)
             {
                 request.addRelatedSymbol(_fxcmInstruments[_symbolMapper.GetBrokerageSymbol(symbol)]);
             }
@@ -140,10 +126,7 @@ namespace QuantConnect.Brokerages.Fxcm
                 _gateway.sendMessage(request);
             }
 
-            foreach (var symbol in symbolsToUnsubscribe)
-            {
-                _subscribedSymbols.Remove(symbol);
-            }
+            return true;
         }
 
         /// <summary>

Brokerages/Fxcm/FxcmBrokerage.Messaging.cs

@@ -331,7 +331,7 @@ namespace QuantConnect.Brokerages.Fxcm
                 _rates[instrument.getSymbol()] = message;
 
                 // if instrument is subscribed, add ticks to list
-                if (_subscribedSymbols.Contains(symbol))
+                if (_subscriptionManager.IsSubscribed(symbol, TickType.Quote))
                 {
                     // For some unknown reason, messages returned by SubscriptionRequestTypeFactory.SUBSCRIBE
                     // have message.getDate() rounded to the second, so we use message.getMakingTime() instead

Brokerages/Fxcm/FxcmBrokerage.cs

@@ -62,6 +62,7 @@ namespace QuantConnect.Brokerages.Fxcm
         private CancellationTokenSource _cancellationTokenSource = new CancellationTokenSource();
         private readonly ConcurrentQueue<OrderEvent> _orderEventQueue = new ConcurrentQueue<OrderEvent>();
         private readonly FxcmSymbolMapper _symbolMapper = new FxcmSymbolMapper();
+        private readonly EventBasedDataQueueHandlerSubscriptionManager _subscriptionManager;
 
         private readonly IList<BaseData> _lastHistoryChunk = new List<BaseData>();
 
@@ -115,6 +116,10 @@ namespace QuantConnect.Brokerages.Fxcm
             _password = password;
             _accountId = accountId;
 
+            _subscriptionManager = new EventBasedDataQueueHandlerSubscriptionManager();
+            _subscriptionManager.SubscribeImpl += (s, t) => Subscribe(s);
+            _subscriptionManager.UnsubscribeImpl += (s, t) => Unsubscribe(s);
+
             HistoryResponseTimeout = 5000;
             MaximumHistoryRetryAttempts = 1;
         }

Brokerages/GDAX/GDAXBrokerage.Messaging.cs

@@ -22,10 +22,12 @@ using System.Collections.Concurrent;
 using System.Collections.Generic;
 using System.Globalization;
 using System.Linq;
+using System.Net;
 using System.Threading.Tasks;
 using System.Threading;
 using RestSharp;
 using System.Text.RegularExpressions;
+using QuantConnect.Configuration;
 using QuantConnect.Logging;
 using QuantConnect.Orders.Fees;
 using QuantConnect.Securities;
@@ -46,8 +48,6 @@ namespace QuantConnect.Brokerages.GDAX
         private const string SymbolMatching = "ETH|LTC|BTC|BCH|XRP|EOS|XLM|ETC|ZRX";
         private readonly IAlgorithm _algorithm;
         private readonly CancellationTokenSource _canceller = new CancellationTokenSource();
-        private readonly ConcurrentQueue<WebSocketMessage> _messageBuffer = new ConcurrentQueue<WebSocketMessage>();
-        private volatile bool _streamLocked;
         private readonly ConcurrentDictionary<Symbol, DefaultOrderBook> _orderBooks = new ConcurrentDictionary<Symbol, DefaultOrderBook>();
         private readonly bool _isDataQueueHandler;
         protected readonly IDataAggregator _aggregator;
@@ -60,12 +60,19 @@ namespace QuantConnect.Brokerages.GDAX
 
         private readonly IPriceProvider _priceProvider;
 
+        private readonly CancellationTokenSource _ctsFillMonitor = new CancellationTokenSource();
+        private readonly Task _fillMonitorTask;
+        private readonly AutoResetEvent _fillMonitorResetEvent = new AutoResetEvent(false);
+        private readonly int _fillMonitorTimeout = Config.GetInt("gdax-fill-monitor-timeout", 500);
+        private readonly ConcurrentDictionary<string, Order> _pendingOrders = new ConcurrentDictionary<string, Order>();
+        private long _lastEmittedFillTradeId;
+
         #endregion
 
         /// <summary>
         /// The list of websocket channels to subscribe
         /// </summary>
-        protected virtual string[] ChannelNames { get; } = { "heartbeat", "user" };
+        protected virtual string[] ChannelNames { get; } = { "heartbeat" };
 
         /// <summary>
         /// Constructor for brokerage
@@ -81,7 +88,7 @@ namespace QuantConnect.Brokerages.GDAX
         /// <param name="aggregator">consolidate ticks</param>
         public GDAXBrokerage(string wssUrl, IWebSocket websocket, IRestClient restClient, string apiKey, string apiSecret, string passPhrase, IAlgorithm algorithm,
             IPriceProvider priceProvider, IDataAggregator aggregator)
-            : base(wssUrl, websocket, restClient, apiKey, apiSecret, Market.GDAX, "GDAX")
+            : base(wssUrl, websocket, restClient, apiKey, apiSecret, "GDAX")
         {
             FillSplit = new ConcurrentDictionary<long, GDAXFill>();
             _passPhrase = passPhrase;
@@ -89,50 +96,9 @@ namespace QuantConnect.Brokerages.GDAX
             _priceProvider = priceProvider;
             _aggregator = aggregator;
 
-            WebSocket.Open += (sender, args) =>
-            {
-                var tickers = new[]
-                {
-                    "LTCUSD", "LTCEUR", "LTCBTC",
-                    "BTCUSD", "BTCEUR", "BTCGBP",
-                    "ETHBTC", "ETHUSD", "ETHEUR",
-                    "BCHBTC", "BCHUSD", "BCHEUR",
-                    "XRPUSD", "XRPEUR", "XRPBTC",
-                    "EOSUSD", "EOSEUR", "EOSBTC",
-                    "XLMUSD", "XLMEUR", "XLMBTC",
-                    "ETCUSD", "ETCEUR", "ETCBTC",
-                    "ZRXUSD", "ZRXEUR", "ZRXBTC",
-                };
-                Subscribe(tickers.Select(ticker => Symbol.Create(ticker, SecurityType.Crypto, Market.GDAX)));
-            };
-
             _isDataQueueHandler = this is GDAXDataQueueHandler;
-        }
 
-        /// <summary>
-        /// Lock the streaming processing while we're sending orders as sometimes they fill before the REST call returns.
-        /// </summary>
-        public void LockStream()
-        {
-            Log.Trace("GDAXBrokerage.Messaging.LockStream(): Locking Stream");
-            _streamLocked = true;
-        }
-
-        /// <summary>
-        /// Unlock stream and process all backed up messages.
-        /// </summary>
-        public void UnlockStream()
-        {
-            Log.Trace("GDAXBrokerage.Messaging.UnlockStream(): Processing Backlog...");
-            while (_messageBuffer.Any())
-            {
-                WebSocketMessage e;
-                _messageBuffer.TryDequeue(out e);
-                OnMessageImpl(this, e);
-            }
-            Log.Trace("GDAXBrokerage.Messaging.UnlockStream(): Stream Unlocked.");
-            // Once dequeued in order; unlock stream.
-            _streamLocked = false;
+            _fillMonitorTask = Task.Factory.StartNew(FillMonitorAction, _ctsFillMonitor.Token);
         }
 
         /// <summary>
@@ -141,38 +107,11 @@ namespace QuantConnect.Brokerages.GDAX
         /// <param name="sender"></param>
         /// <param name="e"></param>
         public override void OnMessage(object sender, WebSocketMessage e)
-        {
-            // Verify if we're allowed to handle the streaming packet yet; while we're placing an order we delay the
-            // stream processing a touch.
-            try
-            {
-                if (_streamLocked)
-                {
-                    _messageBuffer.Enqueue(e);
-                    return;
-                }
-            }
-            catch (Exception err)
-            {
-                Log.Error(err);
-            }
-
-            OnMessageImpl(sender, e);
-        }
-
-        /// <summary>
-        /// Implementation of the OnMessage event
-        /// </summary>
-        /// <param name="sender"></param>
-        /// <param name="e"></param>
-        private void OnMessageImpl(object sender, WebSocketMessage e)
         {
             try
             {
                 var raw = JsonConvert.DeserializeObject<Messages.BaseMessage>(e.Message, JsonSettings);
 
-                LastHeartbeatUtcTime = DateTime.UtcNow;
-
                 if (raw.Type == "heartbeat")
                 {
                     return;
@@ -327,32 +266,16 @@ namespace QuantConnect.Brokerages.GDAX
             // deserialize the current match (trade) message
             var message = JsonConvert.DeserializeObject<Messages.Matched>(data, JsonSettings);
 
-            if (string.IsNullOrEmpty(message.UserId))
+            // message received from the "matches" channel
+            if (_isDataQueueHandler)
             {
-                // message received from the "matches" channel
-                if (_isDataQueueHandler)
-                {
-                    EmitTradeTick(message);
-                }
-                return;
+                EmitTradeTick(message);
             }
+        }
 
-            // message received from the "user" channel, this trade is ours
-
-            // check the list of currently active orders, if the current trade is ours we are either a maker or a taker
-            var currentOrder = CachedOrderIDs
-                .FirstOrDefault(o => o.Value.BrokerId.Contains(message.MakerOrderId) || o.Value.BrokerId.Contains(message.TakerOrderId));
-
-            if (currentOrder.Value == null)
-            {
-                // should never happen, log just in case
-                Log.Error($"GDAXBrokerage.OrderMatch(): Unexpected match: {message.ProductId} {data}");
-                return;
-            }
-
-            Log.Trace($"GDAXBrokerage.OrderMatch(): Match: {message.ProductId} {data}");
-
-            var order = currentOrder.Value;
+        private void EmitFillOrderEvent(Messages.Fill fill, Order order)
+        {
+            var symbol = ConvertProductId(fill.ProductId);
 
             if (!FillSplit.ContainsKey(order.Id))
             {
@@ -360,50 +283,29 @@ namespace QuantConnect.Brokerages.GDAX
             }
 
             var split = FillSplit[order.Id];
-            split.Add(message);
-
-            var symbol = ConvertProductId(message.ProductId);
+            split.Add(fill);
 
             // is this the total order at once? Is this the last split fill?
-            var isFinalFill = Math.Abs(message.Size) == Math.Abs(order.Quantity) || Math.Abs(split.OrderQuantity) == Math.Abs(split.TotalQuantity);
-
-            EmitFillOrderEvent(message, symbol, split, isFinalFill);
-        }
-
-        private void EmitFillOrderEvent(Messages.Matched message, Symbol symbol, GDAXFill split, bool isFinalFill)
-        {
-            var order = split.Order;
+            var isFinalFill = Math.Abs(fill.Size) == Math.Abs(order.Quantity) || Math.Abs(split.OrderQuantity) == Math.Abs(split.TotalQuantity);
 
             var status = isFinalFill ? OrderStatus.Filled : OrderStatus.PartiallyFilled;
 
-            OrderDirection direction;
-            // Messages are always from the perspective of the market maker. Flip direction if executed as a taker.
-            if (order.BrokerId[0] == message.TakerOrderId)
-            {
-                direction = message.Side == "sell" ? OrderDirection.Buy : OrderDirection.Sell;
-            }
-            else
-            {
-                direction = message.Side == "sell" ? OrderDirection.Sell : OrderDirection.Buy;
-            }
+            var direction = fill.Side == "sell" ? OrderDirection.Sell : OrderDirection.Buy;
 
-            var fillPrice = message.Price;
-            var fillQuantity = direction == OrderDirection.Sell ? -message.Size : message.Size;
-            var isMaker = order.BrokerId[0] == message.MakerOrderId;
+            var fillPrice = fill.Price;
+            var fillQuantity = direction == OrderDirection.Sell ? -fill.Size : fill.Size;
 
             var currency = order.PriceCurrency == string.Empty
                 ? _algorithm.Securities[symbol].SymbolProperties.QuoteCurrency
                 : order.PriceCurrency;
 
-            var orderFee = new OrderFee(new CashAmount(
-                GetFillFee(_algorithm.UtcTime, fillPrice, fillQuantity, isMaker),
-                currency));
+            var orderFee = new OrderFee(new CashAmount(fill.Fee, currency));
 
             var orderEvent = new OrderEvent
             (
-                order.Id, symbol, message.Time, status,
+                order.Id, symbol, fill.CreatedAt, status,
                 direction, fillPrice, fillQuantity,
-                orderFee, $"GDAX Match Event {direction}"
+                orderFee, $"GDAX Fill Event {direction}"
             );
 
             // when the order is completely filled, we no longer need it in the active order list
@@ -411,6 +313,8 @@ namespace QuantConnect.Brokerages.GDAX
             {
                 Order outOrder;
                 CachedOrderIDs.TryRemove(order.Id, out outOrder);
+
+                _pendingOrders.TryRemove(fill.OrderId, out outOrder);
             }
 
             OnOrderEvent(orderEvent);
@@ -479,14 +383,10 @@ namespace QuantConnect.Brokerages.GDAX
         /// </summary>
         public override void Subscribe(IEnumerable<Symbol> symbols)
         {
-            foreach (var item in symbols)
+            var fullList = GetSubscribed().Union(symbols);
+            var pendingSymbols = new List<Symbol>();
+            foreach (var item in fullList)
             {
-                if (item.Value.Contains("UNIVERSE") ||
-                    item.SecurityType != SecurityType.Forex && item.SecurityType != SecurityType.Crypto)
-                {
-                    continue;
-                }
-
                 if (!IsSubscribeAvailable(item))
                 {
                     //todo: refactor this outside brokerage
@@ -495,11 +395,12 @@ namespace QuantConnect.Brokerages.GDAX
                 }
                 else
                 {
-                    this.ChannelList[item.Value] = new Channel { Name = item.Value, Symbol = item.Value };
+                    pendingSymbols.Add(item);
                 }
             }
 
-            var products = ChannelList.Select(s => s.Value.Symbol.Substring(0, 3) + "-" + s.Value.Symbol.Substring(3)).ToArray();
+            var products = pendingSymbols
+                .Select(s => s.Value.Substring(0, 3) + "-" + s.Value.Substring(3)).ToArray();
 
             var payload = new
             {
@@ -551,7 +452,8 @@ namespace QuantConnect.Brokerages.GDAX
                     {
                         Value = rate,
                         Time = DateTime.UtcNow,
-                        Symbol = symbol
+                        Symbol = symbol,
+                        TickType = TickType.Quote
                     };
                     _aggregator.Update(latest);
 
@@ -590,22 +492,78 @@ namespace QuantConnect.Brokerages.GDAX
         /// <summary>
         /// Ends current subscriptions
         /// </summary>
-        public void Unsubscribe(IEnumerable<Symbol> symbols)
+        public bool Unsubscribe(IEnumerable<Symbol> symbols)
         {
             if (WebSocket.IsOpen)
             {
-                WebSocket.Send(JsonConvert.SerializeObject(new { type = "unsubscribe", channels = ChannelNames }));
+                var products = symbols
+                    .Select(s => s.Value.Substring(0, 3) + "-" + s.Value.Substring(3))
+                    .ToArray();
+
+                var payload = new
+                {
+                    type = "unsubscribe",
+                    channels = ChannelNames,
+                    product_ids = products
+                };
+
+                WebSocket.Send(JsonConvert.SerializeObject(payload));
             }
+            return true;
         }
 
-        /// <summary>
-        /// Returns the fee paid for a total or partial order fill
-        /// </summary>
-        public static decimal GetFillFee(DateTime utcTime, decimal fillPrice, decimal fillQuantity, bool isMaker)
+        private void FillMonitorAction()
         {
-            var feePercentage = GDAXFeeModel.GetFeePercentage(utcTime, isMaker);
+            Log.Trace("GDAXBrokerage.FillMonitorAction(): task started");
 
-            return fillPrice * Math.Abs(fillQuantity) * feePercentage;
+            try
+            {
+                foreach (var order in GetOpenOrders())
+                {
+                    _pendingOrders.TryAdd(order.BrokerId.First(), order);
+                }
+
+                while (!_ctsFillMonitor.IsCancellationRequested)
+                {
+                    _fillMonitorResetEvent.WaitOne(TimeSpan.FromMilliseconds(_fillMonitorTimeout), _ctsFillMonitor.Token);
+
+                    foreach (var kvp in _pendingOrders)
+                    {
+                        var orderId = kvp.Key;
+                        var order = kvp.Value;
+
+                        var request = new RestRequest($"/fills?order_id={orderId}", Method.GET);
+                        GetAuthenticationToken(request);
+
+                        var response = ExecuteRestRequest(request, GdaxEndpointType.Private);
+
+                        if (response.StatusCode != HttpStatusCode.OK)
+                        {
+                            throw new Exception($"GDAXBrokerage.FillMonitorAction(): request failed: [{(int)response.StatusCode}] {response.StatusDescription}, Content: {response.Content}, ErrorMessage: {response.ErrorMessage}");
+                        }
+
+                        var fills = JsonConvert.DeserializeObject<List<Messages.Fill>>(response.Content);
+                        foreach (var fill in fills.OrderBy(x => x.TradeId))
+                        {
+                            if (fill.TradeId <= _lastEmittedFillTradeId)
+                            {
+                                continue;
+                            }
+
+                            EmitFillOrderEvent(fill, order);
+
+                            _lastEmittedFillTradeId = fill.TradeId;
+                        }
+
+                    }
+                }
+            }
+            catch (Exception exception)
+            {
+                OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Error, -1, exception.Message));
+            }
+
+            Log.Trace("GDAXBrokerage.FillMonitorAction(): task ended");
         }
     }
 }

Brokerages/GDAX/GDAXBrokerage.cs

@@ -48,8 +48,6 @@ namespace QuantConnect.Brokerages.GDAX
         /// <returns></returns>
         public override bool PlaceOrder(Order order)
         {
-            LockStream();
-
             var req = new RestRequest("/orders", Method.POST);
 
             dynamic payload = new ExpandoObject();
@@ -105,7 +103,6 @@ namespace QuantConnect.Brokerages.GDAX
                     OnOrderEvent(new OrderEvent(order, DateTime.UtcNow, orderFee, "GDAX Order Event") { Status = OrderStatus.Invalid, Message = errorMessage });
                     OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, (int)response.StatusCode, errorMessage));
 
-                    UnlockStream();
                     return true;
                 }
 
@@ -115,7 +112,6 @@ namespace QuantConnect.Brokerages.GDAX
                     OnOrderEvent(new OrderEvent(order, DateTime.UtcNow, orderFee, "GDAX Order Event") { Status = OrderStatus.Invalid, Message = errorMessage });
                     OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, (int)response.StatusCode, errorMessage));
 
-                    UnlockStream();
                     return true;
                 }
 
@@ -137,7 +133,9 @@ namespace QuantConnect.Brokerages.GDAX
                 OnOrderEvent(new OrderEvent(order, DateTime.UtcNow, orderFee, "GDAX Order Event") { Status = OrderStatus.Submitted });
                 Log.Trace($"Order submitted successfully - OrderId: {order.Id}");
 
-                UnlockStream();
+                _pendingOrders.TryAdd(brokerId, order);
+                _fillMonitorResetEvent.Set();
+
                 return true;
             }
 
@@ -145,7 +143,6 @@ namespace QuantConnect.Brokerages.GDAX
             OnOrderEvent(new OrderEvent(order, DateTime.UtcNow, orderFee, "GDAX Order Event") { Status = OrderStatus.Invalid });
             OnMessage(new BrokerageMessageEvent(BrokerageMessageType.Warning, -1, message));
 
-            UnlockStream();
             return true;
         }
 
@@ -180,6 +177,9 @@ namespace QuantConnect.Brokerages.GDAX
                         DateTime.UtcNow,
                         OrderFee.Zero,
                         "GDAX Order Event") { Status = OrderStatus.Canceled });
+
+                    Order orderRemoved;
+                    _pendingOrders.TryRemove(id, out orderRemoved);
                 }
             }
 
@@ -191,8 +191,6 @@ namespace QuantConnect.Brokerages.GDAX
         /// </summary>
         public override void Disconnect()
         {
-            base.Disconnect();
-
             if (!_canceller.IsCancellationRequested)
             {
                 _canceller.Cancel();
@@ -440,10 +438,14 @@ namespace QuantConnect.Brokerages.GDAX
         /// </summary>
         public override void Dispose()
         {
+            _ctsFillMonitor.Cancel();
+            _fillMonitorTask.Wait(TimeSpan.FromSeconds(5));
+
             _canceller.DisposeSafely();
             _aggregator.DisposeSafely();
-            _publicEndpointRateLimiter.DisposeSafely();
-            _privateEndpointRateLimiter.DisposeSafely();
+
+            _publicEndpointRateLimiter.Dispose();
+            _privateEndpointRateLimiter.Dispose();
         }
     }
 }

Brokerages/GDAX/GDAXDataQueueHandler.cs

@@ -15,6 +15,7 @@
 
 using System;
 using System.Collections.Generic;
+using System.Linq;
 using QuantConnect.Data;
 using QuantConnect.Interfaces;
 using QuantConnect.Packets;
@@ -35,6 +36,15 @@ namespace QuantConnect.Brokerages.GDAX
             IPriceProvider priceProvider, IDataAggregator aggregator)
             : base(wssUrl, websocket, restClient, apiKey, apiSecret, passPhrase, algorithm, priceProvider, aggregator)
         {
+            var subscriptionManager = new EventBasedDataQueueHandlerSubscriptionManager();
+            subscriptionManager.SubscribeImpl += (s,t) =>
+            {
+                Subscribe(s);
+                return true;
+            };
+            subscriptionManager.UnsubscribeImpl += (s, t) => Unsubscribe(s);
+
+            SubscriptionManager = subscriptionManager;
         }
 
         /// <summary>
@@ -50,8 +60,13 @@ namespace QuantConnect.Brokerages.GDAX
         /// <returns>The new enumerator for this subscription request</returns>
         public IEnumerator<BaseData> Subscribe(SubscriptionDataConfig dataConfig, EventHandler newDataAvailableHandler)
         {
+            if (!CanSubscribe(dataConfig.Symbol))
+            {
+                return Enumerable.Empty<BaseData>().GetEnumerator();
+            }
+
             var enumerator = _aggregator.Add(dataConfig, newDataAvailableHandler);
-            Subscribe(new[] { dataConfig.Symbol });
+            SubscriptionManager.Subscribe(dataConfig);
 
             return enumerator;
         }
@@ -70,8 +85,24 @@ namespace QuantConnect.Brokerages.GDAX
         /// <param name="dataConfig">Subscription config to be removed</param>
         public void Unsubscribe(SubscriptionDataConfig dataConfig)
         {
-            Unsubscribe(new Symbol[] { dataConfig.Symbol });
+            SubscriptionManager.Unsubscribe(dataConfig);
             _aggregator.Remove(dataConfig);
         }
+
+        /// <summary>
+        /// Checks if this brokerage supports the specified symbol
+        /// </summary>
+        /// <param name="symbol">The symbol</param>
+        /// <returns>returns true if brokerage supports the specified symbol; otherwise false</returns>
+        private static bool CanSubscribe(Symbol symbol)
+        {
+            if (symbol.Value.Contains("UNIVERSE") ||
+                symbol.SecurityType != SecurityType.Forex && symbol.SecurityType != SecurityType.Crypto)
+            {
+                return false;
+            }
+
+            return true;
+        }
     }
 }

Brokerages/GDAX/GDAXFill.cs

@@ -24,7 +24,7 @@ namespace QuantConnect.Brokerages.GDAX
     /// </summary>
     public class GDAXFill
     {
-        private readonly List<Matched> _messages = new List<Matched>();
+        private readonly List<Fill> _messages = new List<Fill>();
 
         /// <summary>
         /// The Lean order
@@ -36,11 +36,6 @@ namespace QuantConnect.Brokerages.GDAX
         /// </summary>
         public int OrderId => Order.Id;
 
-        /// <summary>
-        /// The list of match messages
-        /// </summary>
-        public List<Matched> Messages => _messages.ToList();
-
         /// <summary>
         /// Total amount executed across all fills
         /// </summary>
@@ -65,7 +60,7 @@ namespace QuantConnect.Brokerages.GDAX
         /// Adds a trade message
         /// </summary>
         /// <param name="msg"></param>
-        public void Add(Matched msg)
+        public void Add(Fill msg)
         {
             _messages.Add(msg);
         }

Brokerages/GDAX/Messages.cs

@@ -133,6 +133,48 @@ namespace QuantConnect.Brokerages.GDAX.Messages
         public decimal StopPrice { get; set; }
     }
 
+    public class Fill
+    {
+        [JsonProperty("created_at")]
+        public DateTime CreatedAt { get; set; }
+
+        [JsonProperty("trade_id")]
+        public long TradeId { get; set; }
+
+        [JsonProperty("product_id")]
+        public string ProductId { get; set; }
+
+        [JsonProperty("order_id")]
+        public string OrderId { get; set; }
+
+        [JsonProperty("user_id")]
+        public string UserId { get; set; }
+
+        [JsonProperty("profile_id")]
+        public string ProfileId { get; set; }
+
+        [JsonProperty("liquidity")]
+        public string Liquidity { get; set; }
+
+        [JsonProperty("price")]
+        public decimal Price { get; set; }
+
+        [JsonProperty("size")]
+        public decimal Size { get; set; }
+
+        [JsonProperty("fee")]
+        public decimal Fee { get; set; }
+
+        [JsonProperty("side")]
+        public string Side { get; set; }
+
+        [JsonProperty("settled")]
+        public bool Settled { get; set; }
+
+        [JsonProperty("usd_volume")]
+        public decimal UsdVolume { get; set; }
+    }
+
     public class Account
     {
         public string Id { get; set; }