-
Notifications
You must be signed in to change notification settings - Fork 822
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Coinbase: Update exchange implementation #1480
base: master
Are you sure you want to change the base?
Coinbase: Update exchange implementation #1480
Conversation
Continual enhance of Coinbase tests The revamp continues Oh jeez the Orderbook part's unfinished don't look Coinbase revamp, Orderbook still unfinished
…otrader into coinbase_api_revamp
V3 done, onto V2 Coinbase revamp nears completion Coinbase revamp nears completion Test commit should fail Coinbase revamp nears completion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Happy with your changes, thanks! There's an intermitted exchange_wrapper_standards tests breaking now post-master-merge. I think I'm good once that's sorted
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just have a question on a comment. Then after that if you could rebase/merge I can approve
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately it has been difficult to test the scenario due to CBP websocket not respecting config subscriptions:
"subscriptions": [
{
"enabled": false,
"channel": "heartbeat"
},
{
"enabled": false,
"channel": "status"
},
{
"enabled": false,
"channel": "ticker",
"asset": "spot"
},
{
"enabled": false,
"channel": "candles",
"asset": "spot"
},
{
"enabled": false,
"channel": "allTrades",
"asset": "spot"
},
{
"enabled": false,
"channel": "orderbook",
"asset": "spot"
},
{
"enabled": true,
"channel": "account",
"authenticated": true
},
{
"enabled": false,
"channel": "ticker_batch",
"asset": "spot"
}
[DEBUG] | WEBSOCKET | 30/10/2024 10:51:41 | CoinbasePro wss://advanced-trade-ws.coinbase.com: Sending message: {"type":"subscribe","product_ids":["BTC-USD"],"channel":"candles","timestamp":"1730245901"}
[DEBUG] | WEBSOCKET | 30/10/2024 10:51:41 | CoinbasePro wss://advanced-trade-ws.coinbase.com: Sending message: {"type":"subscribe","channel":"heartbeats","timestamp":"1730245901"}
[DEBUG] | WEBSOCKET | 30/10/2024 10:51:41 | CoinbasePro wss://advanced-trade-ws.coinbase.com: Sending message: {"type":"subscribe","channel":"user","signature":"POP","api_key":"CAT","timestamp":"1730245901"}
[DEBUG] | WEBSOCKET | 30/10/2024 10:51:41 | CoinbasePro wss://advanced-trade-ws.coinbase.com: Sending message: {"type":"subscribe","product_ids":["BTC-USD"],"channel":"market_trades","timestamp":"1730245901"}
[DEBUG] | WEBSOCKET | 30/10/2024 10:51:41 | CoinbasePro wss://advanced-trade-ws.coinbase.com: Sending message: {"type":"subscribe","product_ids":["BTC-USD"],"channel":"level2","timestamp":"1730245901"}
[DEBUG] | WEBSOCKET | 30/10/2024 10:51:41 | CoinbasePro wss://advanced-trade-ws.coinbase.com: Sending message: {"type":"subscribe","product_ids":["BTC-USD"],"channel":"ticker","timestamp":"1730245901"}
This will need to be fixed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not many requests other than the reversion I already commented on. Good stuff
Type: oType, | ||
Side: oSide, | ||
Status: oStatus, | ||
AssetType: asset.Spot, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given the usage of asset.All
for "user" and that this can return productType
of either SPOT
or FUTURE
I think you should be returning the appropriate asset type
func statusToStandardStatus(stat string) (order.Status, error) { | ||
switch stat { | ||
case "received": | ||
return order.New, nil | ||
case "open": | ||
return order.Active, nil | ||
case "done": | ||
return order.Filled, nil | ||
case "match": | ||
return order.PartiallyFilled, nil | ||
case "change", "activate": | ||
return order.Active, nil | ||
default: | ||
return order.UnknownStatus, fmt.Errorf("%w %v", errUnrecognisedStatusType, stat) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These may have changed since you did this:
https://docs.cdp.coinbase.com/advanced-trade/docs/ws-channels#user-channel
I note that CB seems to be doing things in upper case and I'm not seeing case handling anywhere
if err != nil { | ||
return warnString, err | ||
} | ||
var oSide order.Side | ||
oSide, err = order.StringToOrderSide(wsUser[i].Orders[j].OrderSide) | ||
if err != nil { | ||
return warnString, err | ||
} | ||
var oStatus order.Status | ||
oStatus, err = statusToStandardStatus(wsUser[i].Orders[j].Status) | ||
if err != nil { | ||
return warnString, err | ||
} | ||
sliToSend = append(sliToSend, order.Detail{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use the following just for these instances of if err != nil
if err != nil {
c.Websocket.DataHandler <- order.ClassificationError{
Exchange: c.Name,
Err: err,
}
}
exchanges/protocol/features.go
Outdated
// FullPayloadSubscribe flushes and changes full subscription on websocket | ||
// connection by subscribing with full default stream channel list | ||
FullPayloadSubscribe bool `json:"fullPayloadSubscribe,omitempty"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This appears unused and the only addition
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good stuff minor requests.
exchanges/coinbasepro/coinbasepro.go
Outdated
startDateString = "start_date" | ||
endDateString = "end_date" | ||
|
||
errIntervalNotSupported = "interval not supported" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return an error then 🚀
exchanges/coinbasepro/coinbasepro.go
Outdated
if err := c.SendHTTPRequest(ctx, exchange.RestSpot, path, &orderbook); err != nil { | ||
return nil, err | ||
// GetProductBookV3 returns a list of bids/asks for a single product | ||
func (c *CoinbasePro) GetProductBookV3(ctx context.Context, productID string, limit uint16, authenticated bool) (*ProductBook, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is not done
exchanges/coinbasepro/coinbasepro.go
Outdated
|
||
return fee | ||
// UnmarshalJSON unmarshals the JSON input into a UnixTimestamp type | ||
func (t *UnixTimestamp) UnmarshalJSON(b []byte) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove UnixTimestamp
type and use types.Time
exchanges/coinbasepro/coinbasepro.go
Outdated
} else if c.Equal(currency.EUR) { | ||
fee = 0.15 | ||
// UnmarshalJSON unmarshals the JSON input into a UnixTimestamp type | ||
func (t *UnixTimestampMilli) UnmarshalJSON(b []byte) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove UnixTimestampMilli
and use types.Time
} | ||
|
||
// TestWsAuth dials websocket, sends login request. | ||
func TestWsAuth(t *testing.T) { | ||
if !c.Websocket.IsEnabled() && !c.API.AuthenticatedWebsocketSupport || !sharedtestvalues.AreAPICredentialsSet(c) { | ||
// t.Parallel() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
reinstate?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just gonna rm that, since other tests interfere with it.
|
||
// AmCur is a sub-struct used in ListNotificationsSubData, WalletData, TransactionData, | ||
// DeposWithdrData, and PaymentMethodData | ||
type AmCur struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AmCur
-> AmountWithCurrency
// OrderBookResp holds information on bids and asks for a particular currency pair, used for unmarshalling in | ||
// GetProductBookV1 | ||
type OrderBookResp struct { | ||
Bids [][3]interface{} `json:"bids"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please convert interface{}
-> any
across this PR.
// ValCur is a sub-struct used in the types Account, NativeAndRaw, DetailedPortfolioResponse, | ||
// FuturesBalanceSummary, ListFuturesSweepsResponse, PerpetualsPortfolioSummary, PerpPositionDetail, | ||
// FeeStruct, AmScale, and ConvertResponse | ||
type ValCur struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ValCur
-> ValueWithCurrency
timestamp, err := time.Parse(time.RFC3339, string(data)) | ||
if err != nil { | ||
return time.Time{}, err | ||
} | ||
return timestamp, nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
timestamp, err := time.Parse(time.RFC3339, string(data)) | |
if err != nil { | |
return time.Time{}, err | |
} | |
return timestamp, nil | |
return time.Parse(time.RFC3339, string(data)) |
} | ||
|
||
// strategyDecoder is a helper function that converts a Coinbase Pro time in force string to a few standardised bools | ||
func strategyDecoder(str string) (bool, bool, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
linter issue here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are going to have to swap over to using JWT authentication. I cannot test the old key format.
requestFmt := ¤cy.PairFormat{Delimiter: currency.DashDelimiter, Uppercase: true} | ||
configFmt := ¤cy.PairFormat{Delimiter: currency.DashDelimiter, Uppercase: true} | ||
err := c.SetGlobalPairsManager(requestFmt, configFmt, asset.Spot) | ||
err := c.SetGlobalPairsManager(requestFmt, configFmt, asset.Spot, asset.Futures) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add:
diff --git a/exchanges/coinbasepro/coinbasepro_wrapper.go b/exchanges/coinbasepro/coinbasepro_wrapper.go
index a675e7ba9..8ca87ef5a 100644
--- a/exchanges/coinbasepro/coinbasepro_wrapper.go
+++ b/exchanges/coinbasepro/coinbasepro_wrapper.go
@@ -44,6 +44,10 @@ func (c *CoinbasePro) SetDefaults() {
if err != nil {
log.Errorln(log.ExchangeSys, err)
}
+ err = c.DisableAssetWebsocketSupport(asset.Futures)
+ if err != nil {
+ log.Errorln(log.ExchangeSys, err)
+ }
c.Features = exchange.Features{
Supports: exchange.FeaturesSupported{
REST: true,
}
As there are currently no websocket market data support for futures.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You want that done, even though there is futures websocket support, limited to the user's orders?
sort.Sort(trade.ByDate(resp)) | ||
return resp, nil | ||
// GetRecentTrades returns the most recent trades for a currency and asset | ||
func (c *CoinbasePro) GetRecentTrades(_ context.Context, _ currency.Pair, _ asset.Item) ([]trade.Data, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
func (c *CoinbasePro) GetRecentTrades(_ context.Context, _ currency.Pair, _ asset.Item) ([]trade.Data, error) { | |
func (c *CoinbasePro) GetRecentTrades(context.Context, currency.Pair, asset.Item) ([]trade.Data, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quick review on new changes. Then I can do a thorough review before next week.
exchanges/coinbasepro/coinbasepro.go
Outdated
// The code below mostly works, but seems to lead to bad results on the signature step. Deferring until later | ||
// head := map[string]any{"kid": creds.Key, "typ": "JWT", "alg": "ES256", "nonce": nonce} | ||
// headJSON, err := json.Marshal(head) | ||
// if err != nil { | ||
// return "", time.Time{}, err | ||
// } | ||
// headEncode := base64URLEncode(headJSON) | ||
// regTime := time.Now() | ||
// body := map[string]any{"iss": "cdp", "nbf": regTime.Unix(), "exp": regTime.Add(time.Minute * 2).Unix(), "sub": creds.Key /*, "aud": "retail_rest_api_proxy"*/} | ||
// if uri != "" { | ||
// body["uri"] = uri | ||
// } | ||
// bodyJSON, err := json.Marshal(body) | ||
// if err != nil { | ||
// return "", time.Time{}, err | ||
// } | ||
// bodyEncode := base64URLEncode(bodyJSON) | ||
// hash := sha256.Sum256([]byte(headEncode + "." + bodyEncode)) | ||
// sig, err := ecdsa.SignASN1(rand.Reader, key, hash[:]) | ||
// if err != nil { | ||
// return "", time.Time{}, err | ||
// } | ||
// sigEncode := base64URLEncode(sig) | ||
// return headEncode + "." + bodyEncode + "." + sigEncode, regTime, nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As comment at the top of the block says, it's my implementation of the JWT protocol, so we won't have to import a separate package to handle that. Alas, there's some issue I can't identify with generating the signature, which has it create an invalid one, so this can't be used right now.
return headEncode + "." + bodyEncode + "." + sigEncode, nil | ||
// GetWSJWT returns a JWT, using a stored one of it's provided, and generating a new one otherwise | ||
func (c *CoinbasePro) GetWSJWT() (string, error) { | ||
c.mut.RLock() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggestion: You can simplify this with a normal mutex and just defer the unlock. Then you don't need to worry about most of whats going on below. We can optimise later if this turns into a bottleneck which it most likely won't be.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a point in doing that now, considering how I've already written this handling?
If so, let me know, and I'll refactor it.
// GetWSJWT returns a JWT, using a stored one of it's provided, and generating a new one otherwise | ||
func (c *CoinbasePro) GetWSJWT() (string, error) { | ||
c.mut.RLock() | ||
if c.jwtLastRegen.Add(time.Minute * 2).After(time.Now()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
instead of JWTLasTRegen just assign an expiry value so the addition op doesn't need to happen.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work with the JWT support! Final minor comments
t.Skip(stream.ErrWebsocketNotEnabled.Error()) | ||
} | ||
var dialer websocket.Dialer | ||
err := c.Websocket.Conn.Dial(&dialer, http.Header{}) | ||
require.NoError(t, err, "Dial must not error") | ||
require.NoError(t, err) | ||
go c.wsReadData() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This design can cause issue on c.wsReadData
errors because of the defer
. You get a panic for a negative waitgroup. That and TestMain
also calls this which is likely what causes the negative waitgroup.
edit: My issue might actually be with TestMain
, why does it dial? Your other tests don't need it. Remove the dialing from TestMain
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without dialing in TestMain, TestSubscribeUnsubscribe errors out when run as a single test.
Seems like negative waitgroup would come about since this never increments the waitgroup, but readData decrements it when closing. I'll add an increment in this call, so maybe that'll fix it?
} | ||
if !partialSkip { | ||
_, err = c.ScheduleFuturesSweep(context.Background(), 0.001337) | ||
assert.NoError(t, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems more appropriate to require
id, err := uuid.NewV4() | ||
assert.NoError(t, err) | ||
resp, err := c.PlaceOrder(context.Background(), id.String(), testPair.String(), order.Sell.String(), "", order.Limit.String(), "", "CROSS", "", testAmount, testPrice, 0, 9999, false, time.Now().Add(time.Hour)) | ||
assert.NoError(t, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems more appropriate to require
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think that's necessary. The results of this test aren't used later, and this test returning an error doesn't prevent resp itself from being checked for being empty.
Do you just suggest that switch so, on an error, only one message pops up for it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assert.NotEmpty(t, resp, errExpectedNonEmpty)
isn't very useful if the result is an error
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Understandable, but it also doesn't seem great to stop the test on the first error if there's novel stuff afterwards which isn't impacted by that error. I've rewritten many tests, including this one, with that in mind.
sharedtestvalues.SkipTestIfCredentialsUnset(t, c) | ||
skipTestIfLowOnFunds(t) | ||
resp, err := c.PreviewOrder(context.Background(), testPair.String(), "BUY", "MARKET", "", "ISOLATED", 0, testAmount, 0, 0, 0, 0, false, false, false, time.Time{}) | ||
assert.NoError(t, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
require
would be more appropriate
if len(pmID) == 0 { | ||
t.Skip(skipPayMethodNotFound) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know we talked about this stuff before, I'm not requesting the change for this PR. In future PRs, I don't think you should be doing this.
I think it is more appropriate to provide made up data than to do this. The testing that we do when we check the endpoints is to see whether the endpoint will complain from say "invalid id" versus "missing param" it helps to identify issues with your implementations even if the ID themselves are not valid. A user can always provide their own IDs in the fields.
Plus it simplifies your tests so you don't need to get so much
Thankfully, removing them all and testing the endpoints gave me the results I was after
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alas, my next PR's already largely set up in this sorta way (._.)
// The new JWT-based keys don't have permissions to edit portfolios they aren't assigned to, causing this to fail | ||
if err != nil && err.Error() != errPortfolioNameDuplicate { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comment and the error check do not match. I think you should remove && err.Error() != errPortfolioNameDuplicate
It is acceptable for authenticated tests to fail on execution for reasons such as "this already exists". I think hardcoding an error response to a specific json string will be very likely to fail in future anyway, even if you have a duplicate portfolio
Is the comment about something else?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah the comment's about something else, I probably should've placed it one line lower to reduce confusion.
assert.ErrorIs(t, err, errNameEmpty) | ||
sharedtestvalues.SkipTestIfCredentialsUnset(t, c, canManipulateRealOrders) | ||
_, err = c.CreatePortfolio(context.Background(), "GCT Test Portfolio") | ||
if err != nil && err.Error() != errPortfolioNameDuplicate { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove && err.Error() != errPortfolioNameDuplicate
@@ -3,30 +3,61 @@ package coinbasepro | |||
import ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[DEBUG] | WEBSOCKET | 14/01/2025 14:34:06 | CoinbasePro wss://advanced-trade-ws.coinbase.com: Message received: {"channel":"ticker","client_id":"","timestamp":"2025-01-14T03:34:05.207247116Z","sequence_num":6,"events":[{"type":"snapshot","tickers":[{"type":"ticker","product_id":"BTC-USD","price":"94873.88","volume_24_h":"13052.57027986","low_24_h":"89028.64","high_24_h":"95254.56","low_52_w":"38501","high_52_w":"108388.88","price_percent_chg_24_h":"0.58560330817365","best_bid":"94886.93","best_ask":"94886.94","best_bid_quantity":"0.00065345","best_ask_quantity":"0.03299181"}]}]}
[DEBUG] | WEBSOCKET | 14/01/2025 14:34:06 | CoinbasePro wss://advanced-trade-ws.coinbase.com: Message received: {"channel":"ticker","client_id":"","timestamp":"2025-01-14T03:34:05.207270615Z","sequence_num":7,"events":[{"type":"snapshot","tickers":[{"type":"ticker","product_id":"ETH-USD","price":"3167.46","volume_24_h":"169211.37256592","low_24_h":"2913.75","high_24_h":"3260.94","low_52_w":"2116.02","high_52_w":"4109","price_percent_chg_24_h":"-2.71002460307953","best_bid":"3166.88","best_ask":"3166.89","best_bid_quantity":"2.23297081","best_ask_quantity":"0.00130411"}]}]}
[DEBUG] | WEBSOCKET | 14/01/2025 14:34:06 | CoinbasePro wss://advanced-trade-ws.coinbase.com: Message received: {"channel":"subscriptions","client_id":"","timestamp":"2025-01-14T03:34:05.207286942Z","sequence_num":8,"events":[{"subscriptions":{"candles":["ETH-USD","BTC-USD"],"heartbeats":["heartbeats"],"ticker":["BTC-USD","ETH-USD"],"user":["3a054eaf-0164-5ca2-872c-a0d64bbc0d79"]}}]}
[ERROR] | WEBSOCKET | 14/01/2025 14:34:06 | could not sync new data for CoinbasePro BTC USD spot Ticker
[DEBUG] | SYNC | 14/01/2025 14:34:06 | CoinbasePro Ticker sync complete ETH-USD [3/6].
Eth works, btc doesn't
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This continues with orderbook:
[DEBUG] | WEBSOCKET | 14/01/2025 14:34:08 | CoinbasePro wss://advanced-trade-ws.coinbase.com: Message received: {"channel":"l2_data","client_id":"","timestamp":"2025-01-14T03:34:05.875997282Z","sequence_num":17,"events":[{"type":"update","product_id":"BTC-USD","updates":[{"side":"bid","event_time":"2025-01-14T03:34:05.194124Z","price_level":"94686.13","new_quantity":"0"},{"side":"bid","event_time":"2025-01-14T03:34:05.194124Z","price_level":"83026.07","new_quantity":"0.00121044"},{"side":"offer","event_time":"2025-01-14T03:34:05.194124Z","price_level":"94905.66","new_quantity":"0"},{"side":"offer","event_time":"2025-01-14T03:34:05.194124Z","price_level":"94942.44","new_quantity":"0"},{"side":"offer","event_time":"2025-01-14T03:34:05.194124Z","price_level":"94942.45","new_quantity":"0.00052708"},{"side":"offer","event_time":"2025-01-14T03:34:05.194124Z","price_level":"94943.86","new_quantity":"0.2634787"}]}]}
[ERROR] | WEBSOCKET | 14/01/2025 14:34:08 | could not sync new data for CoinbasePro BTC USD spot Orderbook
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think its because I have BTC-USDC as the enabled pair. Not sure why BTC USD is being returned. I think this is something you've highlighted in the past. Needs to get resolved though
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure what we can do to resolve that. Looking into things, it seems like they're pruning it down on their end:
CoinbasePro wss://advanced-trade-ws.coinbase.com: Sending message: {"type":"subscribe","product_ids":["BTC-USD","BTC-USDC","ETH-USD","ETH-USDC"],"channel":"ticker","timestamp":"1736833475"}
CoinbasePro wss://advanced-trade-ws.coinbase.com: Message received: {"channel":"subscriptions","client_id":"","timestamp":"2025-01-14T05:44:36.121096821Z","sequence_num":2,"events":[{"subscriptions":{"ticker":["BTC-USD","ETH-USD"]}}]}
All I can see us doing is some sort of reinterpretation. Taking USD tickers received in websocket as referring to USDC, or saying that USDC is invalid for websocket.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will need to be fixed before a merge though. Looks like data already returns with this kind of translation:
https://api.coinbase.com/api/v3/brokerage/products?limit=0&product_type=SPOT
products":[{"product_id":"BTC-USDC", ... ,"alias":"BTC-USD",
Let us know if you need help
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a few ideas on how it could be handled, and think many of them are easy enough to implement, but what sort of approach do y'all think is best?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My immediate thought would be something like
type butts struct {
associatedAliases map[alias]*[]currencies
m sync.Mutex
}
Add it to Coinbase
as an unexported property and when UpdateTradablePairs
is called reset associatedAliases
and populate it with the alias data for each enabled pair. When order book data comes in, lock, publish to all currency pairs associated with the alias. In the example above BTC-USD data would come in and coinbase_websocket would publish to BTC USDC
There's likely nuances that I haven't considered and I'm happy to look into whatever you do as well.
sharedtestvalues.SkipTestIfCredentialsUnset(t, c) | ||
resp, err = c.GetProductBookV3(context.Background(), testPair, 4, -1, true) | ||
assert.NoError(t, err) | ||
require.NoError(t, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just as an FYI, not a required change, asserts' have bool responses, so if you don't like the require, you can do
if assert.NoError(errawr) {
assert.NotEmpty(kasjhdfaksdjfhasldkfh)
}
@@ -3,30 +3,61 @@ package coinbasepro | |||
import ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My immediate thought would be something like
type butts struct {
associatedAliases map[alias]*[]currencies
m sync.Mutex
}
Add it to Coinbase
as an unexported property and when UpdateTradablePairs
is called reset associatedAliases
and populate it with the alias data for each enabled pair. When order book data comes in, lock, publish to all currency pairs associated with the alias. In the example above BTC-USD data would come in and coinbase_websocket would publish to BTC USDC
There's likely nuances that I haven't considered and I'm happy to look into whatever you do as well.
PR Description
Updating Coinbase to the Advanced Trade and Sign In With Coinbase APIs, as well as a few tiny fixes of other parts of the codebase found along the way.
Type of change
Please delete options that are not relevant and add an
x
in[]
as item is complete.How has this been tested
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration and
also consider improving test coverage whilst working on a certain feature or package.
Checklist
The only tests which seem to be failing (exchanges/kucoin, database/models/postgres, config, and common/file) are parts that I haven't substantively changed, and which seem to be failing on master too.