Alternative Data – New City Trader https://newcitytrader.com Magazine for Modern Traders: AI, alternative data, automats, hybrid systems Fri, 22 Dec 2023 17:46:32 +0000 en-US hourly 1 https://newcitytrader.com/wp-content/uploads/2021/09/cropped-Logo-nct-a-32x32.png Alternative Data – New City Trader https://newcitytrader.com 32 32 The Lowest Hanging Fruit [Part 2] https://newcitytrader.com/the-lowest-hanging-fruit-part-2/ Tue, 02 May 2023 07:00:00 +0000 https://newcitytrader.com/?p=8632

We present the promised second part on the use of AI tools in trading. In it, we show how the new tools can be used in an example fund. It consists of discrete traders and the head is a trader who has practically been trading oil for several decades. Part I can be found in the first issue of our magazine

The Fastest Path To Cash: Support The Best And Improve What Already Works

Let’s go back to the fund and ask yourself – what tools to use and where. This is the question of where investments can bring the fastest, measurable benefits: stop the erosion of profits, reduce risk and perhaps increase returns.

What I am describing now is not a universal prescription, because each situation may be different. From my perspective, I provide topics to think about. I would say this – the general rule is that we should support what works best, brings the best profits and what carries the most risk. In the case of a fund, or discrete traders – this is to support the best traders.
Who in our fund is the best?

In the described fund, the best trader is the holder. Support for him is obvious as long as he wishes, of course. However, AI support in this case can fail, as we are about to demonstrate. We get the best practice from implementing AI tools when we support, improve, automate what already works well. In general, we will get the best results by supporting what works best.

If we have a group of top traders then they are a fairly obvious choice.

However, one more condition must be met – they must be open to new methods. It happens, for example, that the statistical approach is so far from the trader’s experience that he is unable to switch. He is unable to combine the insight provided by the new knowledge with what he already knows. The result is unnecessary tension, and investment and time are lost.

The best usually don’t have a problem with this – that’s one of their key characteristics. They are open to change, they know that the market is constantly changing and you have to adapt. As an example, let me mention Larry Williams, who I interviewed for a book a few years back. He is 77 years old today and you can find his current lectures on the Internet about the new indicators he is creating. Age is no barrier to taking advantage of new things. Our trader, the head of the fund is a man in his 70s and yes, I think he is the one who should be supported if he wants to.

The second question is – where is the greatest risk and how can new tools reduce it?
The third area is (and here I think there is very great potential) the implementation of new MM (money management), I will write about this in future articles.

And some more tips at the end

If you are thinking of a larger AI department (using only new tools) then create a separate team. In the next issue I will describe good and bad practices in this area, i.e. how not to lose tens of millions using what others have already burned on.

New Strategies For Managing Position Size For Different Classes Of Signals

I often refer to what the best discrete traders do because it is their practices that are worth emulating and improving (through automation).

The best ones adjust the size of the position according to the “goodness of the signal,” in the sense that they are willing to increase the position when the signal is exceptionally good (according to their evaluation criteria).

If we start from the point that some signals are better and some are worse then we can move on to the creation of a seamless MM, based, for example, on statistical evaluation, assessment based on the results of the application of Machine Learning.
The starting point can be a simple observation that the best inputs may have something in common. What is it, what are the factors – this is the task for the ML tool (classification of inputs due to various parameters).

If we find this, then we can count new dynamic rules for allocating position size to each signal ( depending on the parameters that characterize the input – whether it is “better” or “worse – to which class it belongs).

This approach opens the door for us to create very liquid MM strategies – the best signals get the most and are pyramided and the weakest (but still “valuable”) – get less or only “control inputs”.

In the last ones, for example, we can try “control entries” and if the position holds and is secured – we add so much that if the market goes against us – we will come out at zero (this is one of the simpler strategies for “weak” signals).

Why is this important? These signals are not useful enough to be overlooked by other traders and other systems. In case the bigger ones with very strong AI systems seize larger and larger portions of the market these very signals may remain the longest. One of the most efficient billion-dollar quant funds has specialized in just this approach for many years.

More complex MM strategies are practically out of reach for discrete traders, requiring too much work, beyond the capabilities of a single person or even a team.
Add to this the fact that a single position can be broken into thousands of smaller ones (to minimize adverse moves and dynamically adjust to pending and incoming orders) and this absolutely requires the use of intelligent automation.

Hiding Entries And Exits

If I wrote that brokers and market makers copy the positions of the best then many might be offended. So, I will not write this (oh wait!). This is one of the oldest tricks in the book, the simplest and very profitable, used since the beginning of the stock markets.

The best ones do it in several creative ways, from what I’ve observed in the life cycle of a fund there comes a time when entries are hidden to preserve their profitability. Too many eyes are watching, many smart people are able to “crack” the strategy and it doesn’t take intelligence at all to copy.

Some of the best go to the extent of not only hiding the inputs but also hiding the intent, the system, the strategy behind the inputs to prevent reverse-engineering it. This is understandable, if they have something that works well (for which they usually paid a lot, sometimes tens of millions) they want to use it for years.

Here I want to talk about two strategies I’ve observed, so that if someone hasn’t dealt with it yet they can form an opinion.

The first strategy is to split the inputs between two, three or more brokers. The system has a certain number of profitable inputs and a certain number of losing ones, and the split itself can cause fluctuations – perhaps more losing ones will flow to one of the brokers?

It is possible to consciously (if such a practice is suspected) direct inferior orders to any of the brokers. It is also possible to do such things routinely, changing from time to time to whom the better and worse orders are directed. This is bound to reflect on the copyists, as the system will not be as good as expected.

Another strategy requires having or running its own HFT trading department. One large fund does several hundred thousand entries a day (between two hundred and four hundred thousand entries), and among these HFT hides its entries from other systems.

This fund is operated by two very large brokers.
If someone had access to one of the brokers and tried to crack its strategies, he would first have to filter out the large-scale signals (5 minutes and above) from the tens of thousands of HFT signals. This is of course possible but … the other problem remains – neither path gets the system fully. Second – the fund uses not one strategy but dozens. And breaking this is already a non-trivial matter (but still probably possible for a patient statistician).

But then there is another matter – the fund employs a lot of excellent mathematicians who, over time, will surely realize that one path has worse results than another and ask themselves why (the fund has already had to deal with the use of its signals and knows the subject from experience).

They won’t necessarily be able to prove that someone is sneaking their signals or trying to break their strategies, but they might just want to change brokers.

This is the ending of part one. You can read the whole thing in the magazine. Just subscribe below!
STAY IN THE KNOW

Get Current [Issue] of New City Trader.
 Now It's Free!

]]> It is Alternative Data that gives traders and funds the biggest advantage today, not AI https://newcitytrader.com/alternative-data-biggest-advantage/ Mon, 07 Feb 2022 12:09:57 +0000 https://newcitytrader.com/?p=6589

A fierce battle is underway for better returns, reduced risk and faster entries. It is Alternative Data that gives traders and funds the biggest advantage today, not AI. The time of AI dominance is yet to come.

Alternative Data is a revolution in trading today more important than the Artificial Intelligence revolution. They are able to improve virtually every part of a fund’s investment activity: improve signals or systems, reduce risk, show situational context when trading a position.

The system hasn’t stopped working, but there’s no liquidity like there used to be, entries cause stronger counter moves because… the biggest ones have already been there and exhausted the liquidity. The biggest funds use alternative data to enter faster.

Better pre-trade analysis improves performance. Better post-trade analysis improves understanding of the factors that make up good performance. Until now, such deep insights into the market and the environment have been impossible.

From a strategy perspective, funds have no choice today and (sooner rather than later) dealing with alternative data will be a necessity. From an operational strategy point of view – it is important today to create in the fund the competence to use alternative data, because this revolution is just beginning.

Alternative data gives a big market advantage. Today, in addition to AI tools, they are a major driver of building competitive advantage. However, what they bring to some they take away from others. That means systems stop working, signals become less profitable, pressure on discretionary traders and portfolio managers increases. AD are changing markets without a doubt.

In a recent article in New City Trader I show more broadly, with specific examples, how they are doing this ⇓

STAY IN THE KNOW

Get Current [Issue] Now It's Free!

]]> Alternative Data is a revolution in trading. Today even more important than Artificial Intelligence https://newcitytrader.com/alternative-data-is-a-revolution-in-trading/ Thu, 02 Sep 2021 07:04:10 +0000 https://wgl-demo.net/thepascal/?p=210

The use of Alternative Data is the current super trend strongly shaping the present and future of investing. It is a revolution more important than the revolution that Artificial Intelligence will give.

The main benefit of Alternative Data (AD) is that you can have most of your important metrics faster and more accurately than ever before.

Alternative Data currently improves signal quality and reduces risk. In the future, it will be the primary source of signals. Today, it is the main source of competitive advantage in many cases to find and use these signals before others.

In addition to the current use of AD, it is imperative to create competencies in the fund to use it wisely in the future because this area will now evolve rapidly.

Thanks to AD, the signal will be received earlier. As a result, it will be better, less risky, the position will be easier to run, and the exit will be better.

I have prepared 20 examples of using alternative data. I want to show a broad spectrum of situations where AD have been used so far and can be used in the future.

It is often the case that an exciting and inspiring idea comes from a completely unexpected direction. Therefore, it is worth attending industry conferences and collecting as many case studies as possible of what others have done.

Alternative Data New City Trader

Counting cars in the Tesla parking lot

One fund applied Machine Learning (ML) tools to analyze satellite images of the parking lot in front of Tesla’s Megafactory. The tool analyzed the position of cars and their colours. The goal was to determine if and how the number of vehicles was changing. It turned out that cars quickly disappear and new ones appear in their place, which indicated that the company would keep its commitments and plans. Moreover, with this analysis, it was known weeks before the official announcement. Such information provides a substantial advantage over other market participants.

The above is a rather famous and awe-inspiring example, so it is worth commenting on it. If we look at the area around this factory (in Nevada), we can see one access road.

Setting up a car with a camera there and having someone count the vehicles leaving the parking lot would give the same result (and perhaps much cheaper) as using a whole team to machine-analyze satellite images. It is worth knowing this and looking for a way to have the same data not for a quarter of a million dollars but for 1% of that amount.

Non-Farm Payrolls Employment Data...

Full version of the article is available for you in New City Trader magazine (now free of charge):

STAY IN THE KNOW

Get Current [Issue] Now It's Free!

]]> [Full] Artificial Intelligence: The lowest-hanging fruit. Part I https://newcitytrader.com/artificial-intelligence-in-trading-lowesthanging-fruit/ Wed, 01 Sep 2021 07:00:16 +0000 https://wgl-demo.net/thepascal/?p=233

I recently spoke with an elderly gentleman – a trader and fund owner. This conversation inspired me to write an article about Artificial Intelligence tools that are used in trading today. His fund employs over a dozen traders investing in various markets, and he is a veteran of oil trading. His trading style is conservative – after finding a signal, he opens a trade holding a single position, sometimes for several weeks.

I want to show you where and how you can use the most modern solutions in this example. The largest funds use such solutions daily.
As we know, a trader’s performance depends on his decision-making process, signal quality, available liquidity and what happens during position management. New Artificial Intelligence tools will improve each stage of this process.

The solutions and ideas can be applied to any discrete trading system.

Alternative data means more accurate price forecasts and better, less risky signals

Many years ago, to find out the financial results of a chain of stores, one had to wait for their earnings announcement. Today, the analysis of satellite images of parking lots makes it possible to estimate whether there are more or fewer customers and, on this basis, to forecast the results.

Data from credit card transactions also allow estimating whether there will be an increase or a decrease in sales in the chain.
By analyzing card transactions and various data from the company’s environment, we can estimate the financial results that are yet to be published by this company. If we have access to relevant data, we can make such estimates ongoing and make investment decisions.

For the oil market, some companies forecast price movements based on inventories and demand. Today, you can count the amount of oil in tanks in remote areas of China by measuring the length of the shadow they cast (using satellite images). To this estimate, the amount of oil stored in tankers and on railway sidings is added. Thanks to this, we know the size of stocks in virtually every country “right now”.

Today’s “forecasting” goes to the real-time estimation of the selected quantity. It is one of the strong trends changing investing. As a result, traders and investors have a better picture of the current market situation. Consequently, entries are more precise and less risky. And funds that build this kind of competency at home are winning in the markets.

Supertrend: Moving from forecasting to nowcasting.

Traders who traded moves after the news are now seeing that these moves are getting weaker. On the other hand, traders in the largest mutual funds are rarely surprised when publishing data because they can estimate it in advance. It is also one of the reasons for the disappearance of some systems.

I read an article a few months ago by an economics professor arguing that alternative data is unnecessary because large funds react to earnings releases anyway. Of course, there is some point in this. However, suppose someone can assess the data before publication. In that case, he has a chance to enter the market faster – even before the others. Or take advantage of the entry of other large investors to exit the market with profits at the right time.

Market behaviour is changing because major investors have changed their behaviour. Those who fail to recognize this in time will be left with old signals and many stocks that are no longer as attractive as before.

Entry and exit - minimizing costs and adverse moves

One of the main problems that most institutional traders complain about today is the difficulty in finding liquidity.

The automated tool can analyze the available liquidity with different brokers, plan and execute market entry. Their use is to take advantage of available liquidity with minimal cost and minimal adverse move.

Virtually every large fund has this type of tool. For example, quants analyze order structures using Machine Learning methods and, on this basis, program various entry strategies. As a result, the program will execute the entry faster and better than a large team of traders.

Automation gives the trader more time for other tasks – more important than position building.

Trend: Entry automation: The program will execute an entry faster and better than a large trading team. The trader has more time for other tasks – more important than position building.

Entry when the signal is very good

Low risk attracts significant capital. Moreover, new signals are getting better and less risky, so the automated trading solutions get new tasks: increasing positions to the maximum for the best signals. 

This change makes it difficult for other traders to find liquidity. As a result, they cannot enter the market as before, and the adverse movement is more substantial than previously.

However, this is not a problem for smaller traders. Some consider the lack of liquidity to confirm high signal quality. To quote:

“Lack of liquidity in good companies means that the biggest ones are already there.”

This situation is a problem for large traders who do not use the latest tools. The signals are still good and followed by the expected moves. However, there are no longer opportunities to build large positions.

The struggle for liquidity, especially in the best market situations, will only increase. Therefore, the focus will be on getting a good signal as early as possible to enter with significant capital quickly. It is with alternative data that this becomes possible.

The next step will be to exploit weak signals (I will describe what they are and how to exploit them in a future article).

Better signals and lower risk attract more capital. Automation of entries and exits becomes necessary; the program can utilize the available liquidity to the maximum. The competitive battle focuses on having a good signal as soon as possible.

Better assessment (view) of the situation while taking positions

While taking positions, a better view of the situation comes with new tools. However, this improvement does not end with market entry. Social media sentiment analysis and automated news analysis gives you a picture of market interest in a particular instrument.

Example:

One of the funds uses the mean reversion system. If the price of a company after good news has moved too far away from the average, then the program places a sell order.

The basis of this system is pure statistics. The program counts the main moving average the deviation from it. The system monitors the price, and if the deviation is “too high”, it places a sell order back to the average.

However, when good news keeps coming, the market often continues moving up.

To cope with this situation, they added a module for analyzing news and sentiment in social media. Suppose the information about the company is still good (or there are many positive posts on social media). In that case, the sell position is automatically closed.

Profit increase achieved.

Discretionary traders can use the same scheme. However, while it is still possible to follow the news for a few positions, it is difficult to follow and create any meaningful picture of social media sentiment. The Machine Learning tools for analyzing Twitter, Reddit, Finwiz, FB or any media where potential traders can undoubtedly help here.

Knowing what others think about a stock or commodity can help you manage your position better. If the sentiment gets bad, we can exit the market earlier, and a good social mood can make us hold the position.

Of course, these tools should be approached with caution – the market is dominated by the most significant players, and they do not publicly announce their intentions. But they have these tools, and with their aid, the smaller players can see what the big ones see and thus better understand their intentions and decisions.

Today, you can shorten or lengthen a position with news and social moods analysis because you have a better picture of what others are seeing, reading, and thinking.

Availability of the new solutions

The tools described here are available to the largest funds. However, their development is expensive and requires teams of good specialists (machine learning, social media analysis, liquidity analysis, microstructure of markets and its evolution over time).

As time goes on, these solutions will become more widespread, available from larger brokers or offered as stand-alone solutions.

Conclusion:

Thanks to alternative data, we see a shift from forecasting to nowcasting. We also have better forecasts of supply, demand and price behaviour. As a result, trading signals are better and less risky.

Current tools allow us to make better use of available liquidity, minimize entry costs and get a better average price. Over time, these two goals will be joined by a third: an emphasis on exploiting the best signals to the maximum.

Less risk will attract more capital, and liquidity will be depleted in many excellent current oves over time. The struggle for liquidity will intensify further. As a result, many sound systems will continue to yield high-quality signals. However, there will be no way to build a bigger position. Systems will still be good, but they will no longer be as profitable as before.

There will be a struggle to improve signal quality in the long run, mainly to make the signal appear faster.

These changes may, in some time, hit the best traders, like my interlocutor – a trader with enormous market experience. Why exactly them? People in his group have the most experience and the best signals. These are the signals that the biggest traders/funds will hunt for and try to improve.

This evolution does not mean that new tools are readily available or cheap. On the contrary, creating them requires a team of good Machine Learning specialists in finance. It also entails gathering much knowledge about the microstructure of markets.

In the next issue:

– The fastest path to cash in a fund – what to support with new tools first – some practical tips.
– AI and entry and exit hiding.
– Evolution of tools towards dynamic MM.
– Position and environment analysis.
– New and improved efficiency analysis of market paths.
– Automatic portfolio optimization (including new optimization models).
– New models for managing the size of positions on the horizon.

Full version of the article is available for you in New City Trader magazine (now free of charge):

If you enjoyed this insight:

Support Trading Education With Cake and Coffee ↷

Donate via PayPal
STAY IN THE KNOW

Get Current [Issue] Now It's Free!

]]>