totally understood, thats what i didnt understand, thank you. i actually think you mentioned this once before and i just didnt connect the dots.

It seems that there's no way to get minute data without making a significant investment, so I'm going to set that aside for the time being.

I will see what I can do with hourly data. I don't remember which providers offered a reasonable timeframe for hourly data, but I think there were some that go back at least a year. lol I guess I'm starting from square one again, but since I'm being less picky this time it shouldn't take me too long.

Hopefully I can find some reasonable indicators this way, and if backtesting (as well as real testing) is somewhat successful, then I will eventually plunk down the money to get some big boy data.

Candlestick Charting. Steve Nison. I really don’t care where you get it these days. Fascinating. Centuries old. Works in all time frames. At least be aware of what everyone else sees.

Why start with real money at all...just to keep your interest? Seems like a way to flush your money down the shitter.

patience is a virtue kid

Reminder to self (when I sober up in the morning) to take the MarketStacks API out for a whirl.

The pricing is very reasonable, and it claims to have intraday historical data, even at minute intervals, for any US stock on IEX.

But a cursory google search says that IEX was short-lived and is donkdown. Tha fuck?

Yeeaaah, MarketStacks turned out to be dogshit. I should have known it too good to be true

Praise the Gods, I may have finally found my solution. Polygon.io has a robust package of intraday historical data (down to minute increments, going back a decade+), but is on the pricier side at $200/month. However, you can get a free API key through Alpaca. They must really be printing off whatever shenanigans they do to skim off commission-free trades.

Anyway, it looks like I'm all set. Step one is to pull five years of data from 1,000 or so stocks, and then parse/sanitize it into a format that will be easier to work with.

Next I will have to code in my own formulas for metrics like MACD and exponential moving average. This will be a pain in the ass, but a worthwhile endeavor as well as a learning experience.

My first instinct was that last 5 months are more predicative of the future than last 5 years. You could likely test that with say a 10 stock sample size and see how well your code would have fared last week using 5 months/years as data.

I might be wrong. I know relatively little about this. It's just a common trend of exceptional times. There was nothing really exceptional between 2014-2019.

That's a good point. I could even try both approaches. I'll take the best performing indicators from the 4-year backtests and pit them against the best performers in a 5-month backtest and see which strategies win out.

I have been going through the documentation and put some code together, but I'm getting some curious results.



This is what I get when I make a request for hourly data on TSLA, from March 1 up until today. I haven't labeled any of this, but in order it shows the date, time, open, high, low, and close. Then at the bottom it shows the total number of results.

As you can see it jumps from March 10 to July 20, and shows a total of 119 hours. When I run the same request for AAPL I get a similar jump from 3/9 to 7/20 and 108 hours. For MSFT it cuts off at 3/10 and gives 115 hours.

A similar thing happens when I change the interval to 1-minute. They all jump from 3/9 or 3/10 to 7/20 and give about 5,600 minutes. But they're not exactly the same, so even the gaps in coverage are inconsistent.

I'm not sure why Polygon does this, but obviously for my purposes this data is completely useless. I need consistent, uninterrupted quotes.

I think I'm just going to fall back on the yahoo finance API. While it can only go back 7 days at the 1-minute interval, I just tested it for TSLA year-to-date at 1-hour intervals and it gives me 965 hours of results. I'll check later if that is complete, but if so then I think 1-hour granularity will have to do for the purposes of swing trading.

oh this might be a pandas / python thing. pandas trims output.

That certainly is the case with the Yahoo Finance api:



In Python it's not hard at all to unpack that pandas object into other formats like JSON or csv.

With polygon however, I don't think this is the case. Here's my code:



"119" isn't being reported by some API function (and thus trimmed), it's something I added to measure what's going on inside the 'results' list.

The 'data' variable is basically a weird nested dictionary. If you just print(data), this is what you'll get:



So the key 'results' just contains one value, a list of dictionaries, each containing key:value pairs for open, close, high, low, etc. If you scroll all the way down, you can see that it's a complete list, not being trimmed or anything. I've just been counting the number of dictionaries contained in 'results'.

lol I just realized that instead of literally counting all of the dictionaries, I could have just plucked out the 'resultsCount' variable. This screenshot is for the 1-minute interval, so in this case it's 5,651 minutes

SMALL MILESTONE:



This chart is nothing special in and of itself. The information it displays could easily be found on various websites. But this chart was generated from ticker data that I pulled from the Yahoo API. It is 994 hourly ticks from year-to-date. I can now freely manipulate this data and simulate trades with it. I already wrote a formula for Simple Moving Average (can be for any period, doesn't have to be 20-day) and will now start adding other metrics like MACD, EMA, or whatever else I find in my research that may be useful.

Next I will build a backtesting simulator. It will generate a random policy for buy indicators, sell indicators, and bankroll/risk management. Then it will simulate 6 months (Jan-June) of executing trades in the S&P 500. I don't expect this to be very resource intensive, so I should be able to simulate hundreds or perhaps thousands of strategies at a time. I will then take the best performing strats and run them through July.

Honestly, the hard part is over. Now the fun begins!

BUCKLE UP

youre doing some real good work here imo.