[Discussion Needed] New APIs for v1.0.0?
ahmedsakr opened this issue · comments
Hi there,
Thanks for dropping by. If you use wstrade-api, you likely have experienced its limitations so far.
What would you like to see changed with v1.0.0? What would you like to see added?
Thanks!
Using Wealthsimple to get market data would be ideal, right now I'm having to use other sources for free market data. I know they're all delayed, but having the data come from Wealthsimple makes more sense.
In the mean time until the API is improved, you can still do it with the current version of wstrade-api using getSecurity with the parameter extensive set to true. It would return the price under quote.amount. See an example here: https://github.com/ahmedsakr/wstrade-api/blob/master/src/index.js#L270
@ahmedsakr Thanks for the great tool and for your hard work. Just to chime in, are you aware that Wealthsimple quotes are 15 minutes delayed? When you say that you've 'recently discovered the API for retrieving the stock information', do you mean that you've found something that responds with realtime results?
I just confirmed and it is delayed but i am not sure if it was a 15 min delay.
I tried this at 3:48 pm for AAPL:
wealthsimple.login('me@hotmail.ca', 'password')
.then(data => wealthsimple.getSecurity(data.tokens, 'AAPL:NASDAQ', true))
.then(result => console.log(result.quote.amount));
I got 116.495 for price. But AAPL was trading at 116.42 at 3:33 (which is 15 mins before 3:48).
I just digged in more and the quote_date is provided with the result.quote object:
quote: {
object: 'spot_quote',
security_id: 'sec-s-76a7155242e8477880cbb43269235cb6',
amount: '116.4400',
currency: 'USD',
ask: '116.4400',
ask_size: 2200,
bid: '116.4300',
bid_size: 1200,
high: '116.6300',
last_size: 150,
low: '114.9200',
open: '115.2800',
volume: 80211101,
previous_close: '114.9700',
previous_closed_at: '2020-10-08T20:00:00.000Z',
quote_date: '2020-10-09T19:37:09.000Z'
}
it does seem to be 15 min delay.
Question: This seems to mean that the market order APIs will not fill a lot of times for stocks that are volatile. We might need to add a quote spread parameter to provide the invoker the ability to specify the spread from the "15-min delayed quote" they are willing to divert from for a market order. For example, if the quote spread was provided as 0.20 and our AAPL quote says 116.44, the market order will be placed at 116.44 + 0.20 = 116.64. What do you think?
@ahmedsakr I think the only reasonable way to do it is to have data that is close to real-time. Otherwise you might as well just place limit buys only.
I have been looking into this recently for a project, and found that there is plenty of free realtime data out there for NYSE/NASDAQ but not much for the TSX/TSXV. The option I chose to proceed with was to make a small tool for querying BNN Bloomberg's API. So far, so good - some days I will get consistent ~5 second delay, others ~0.1 second delay.
Agreed. I think then the most appropriate course of action is to add the ability to register a custom hook for retrieving real time quotes. We can have a default hook using a reliable source, potentially the one you suggested.
Sounds good. Let me know if you need a custom format. BNN Bloomberg's API gives the following structure.
statusCode: 200,
generatedTimestamp: '2020-10-10T13:45:21.574',
duration: 141,
data: {
stocks: [
{
symbol: 'AC:CT',
name: 'AIR CANADA',
currency: 'CAD',
price: 16.09,
netChng: -0.67,
pctChng: -4,
htsclsdate: '2020-10-09',
tradeDate: '09 Oct 2020',
yearHigh: 52.71,
yearLow: 9.26,
yearHighDate: '2020-01-14',
yearLowDate: '2020-03-18',
totalVolume: 4314594
}
],
invalidSymbols: []
}
}```
Thanks for the sample. Just a quick question. Does BNN require access keys for their APIs?
Nope. I'm pretty sure the only condition is that the request headers have the parameter "Referer":"https://www.bnnbloomberg.ca/".
Their API is not advertised as something to be used by the public. I just found it by looking at the source code on https://www.bnnbloomberg.ca/markets.
😬
Another suggestion. We can use or scrape trsdingview data to use for real-time / history. Otherwise I was just going to build my strategy on trsdingview and use the webhooks/email to execute on wealth simple.
@thisisloze it's funny that you mention that because I actually just yesterday accidentally let my trial subscription to TradingView renew. It's expensive. To be clear, you have to pay like $80CAD /mo to get real-time tickers on the TSX. Otherwise it seems like it would be a great option.
@vxsl I'm currently working on the BNN quote retrieval. I can't seem to find any stock on NASDAQ or NYSE from BNN's API, it is only UW/UN/other exchanges that are delayed. For example, if you look up apple, you can't find the NASDAQ listed apple ticker.
@ahmedsakr I'm assuming you mean that you're trying my bnnbloomberg-markets-scraper tool - if so, you should be able to print realtime quotes with the following example program:
const { QuoteHarvester } = require("./bnnbloomberg-markets-scraper")
!async function () {
let q = await QuoteHarvester.build("AAPL:UN")
while (true) {
let quote = await q.quote()
console.log(quote.generatedTimestamp + " " + quote.data.stocks[0].price + "\n\n")
}
}();
Let me know if this answers your question.
Regarding the correlation between NYSE/NASDAQ and Bloomberg's suffixes, I found this thread which seems informative.
Right, but my concern is that UN/UW values, even though it says are real-time, are not real-time with respect to the actual trading value of the stock. I wrote my own quick wrapper here: https://github.com/ahmedsakr/wstrade-api/blob/custom-quote-sources/src/quotes/external/bnn.js
I am running it, and i am getting outdated values with respect to what Apple is really trading at.
Also this notice on their website
I recommend that you try using bnn-bloomberg-markets-scraper, or take a look at the source code and implement a similar solution.
Accessed as-is (and as bnnbloomberg.ca itself accesses it - this explains your screenshot), the Bloomberg API will give you stale responses 90% of the time and is essentially unusable. But through trial and error I found that sending requests with random dummy GET parameters will increase the likelihood of receiving a non-stale response.
So my program does this repeatedly and rapidly while also filtering out the stale responses that are still sometimes received. If you run it you'll see that you get a real-time stream (well, at least the timestamps are correct... I haven't actually cross-referenced with another source to see if the prices are correct).
Also, as far as I can tell, the delay/unreliability described above is not specific to UN/UW - in my experience the API used "as-is" is equally dysfunctional with Canadian market data.
I just downloaded and used your tool and it is still not real time compared to my RBC brokerage account (which does provide real time data). Margin of error was still well over 40 cents, but there is an improvement to my barebones version.
I don't think there is free lunch in this adventure. Real-time data is premium. I think the right approach for wstrade-api is to provide hooks for the user to funnel in their real-time data.
I see, yeah, as I mentioned I hadn't bothered to check whether the quotes themselves (as opposed to just their timestamps) were realtime. That's too bad. From my earlier research it looks like Yahoo used to be a good source for this type of thing, but it appears that they discontinued their API.
From my earlier research it looks like Yahoo used to be a good source for this type of thing, but it appears that they discontinued their API.
I use this package to get data from Yahoo stocks, it works well but I'm pretty sure it's still delayed.
https://www.npmjs.com/package/yahoo-stocks
Gonna agree and say best way would be to get data directly from WS Trade
@dillonco But we have discussed that WS Trade is also delayed, no?
@dillonco But we have discussed that WS Trade is also delayed, no?
Yeah that's true, so the main benefit comes from not being rate limited because you're a customer, instead of an unauthenticated or free api. Also, WS might improve their data in the future, but that's me just being hopeful.
I have added custom quote providing ability in the 1.0.0-develop branch. Here is a sneak peak of how you will be able configure it:
let myCoolCustomProvider = {
quote: (ticker) => {
// All stocks are $42.
return 42;
}
};
trade.quotes.use('NASDAQ', myCoolCustomProvider);
trade.auth.login('example@example.ca', 'password')
.then(async () => {
// should always be 42 as the custom provider overrides the quotes get() API to not use WS Trade
// for NASDAQ-listed stocks, which AAPL is.
console.log(await trade.quotes.get('AAPL'));
// Will get the actual price of UBER stock through WealthSimple Trade as UBER is on NYSE
// and will not be affected by our custom NASDAQ provider.
console.log(await trade.quotes.get('UBER'));
});