AI notes 2024: Prognosis
I continue my notes on AI at the end of 2024.
In the previous posts we discussed three theses:
- By analyzing the decisions of major AI developers, such as OpenAI or Google, we can make fairly accurate assumptions about the state of the AI industry.
- All current progress is based on a single base technology — generative knowledge bases, which are large probabilistic models.
- The development of neural networks, a.k.a. generative knowledge bases, is reaching a plateau; their further advancement is likely to be incremental/evolutionary rather than explosive/revolutionary.
Based on these theses, we can finally talk about the most hyped, most exciting topic: how long will the current progress continue? Will we reach the singularity in 2025, or will everything remain the same? When will our god of metal and silicon finally appear?
In 2023, I already published a forecast on artificial intelligence [ru]. It is still valid — take a look. In it, I spent more time describing what to expect. This text is more about what not to expect. So, it turns out that I outlined two boundaries of the possible, and the truth should be somewhere in between.
Impressions after reading Harry Potter
The classic cover of the first book.
As part of my immersion in modern world culture and integration into the Hamburg community (the city is a major fan hub), I read all seven Harry Potter books. I even specifically got the English edition to read in British English, not American (although Reddit says the difference between them is small).
Impressions are mixed.
On the one hand, the story is interesting, written in good lively language, the first books are read with pleasure, the last ones without significant suffering.
On the other hand, I expected more. After reading, I can't reconcile the quality of the books with the level of their popularity.
On the third hand, I believe Joanne Rowling could have done it better :-) Not that it's written poorly — it's written well — but too one-sided, too narrow, I would say. Especially the last books.
Let me tell you a little more about the books.
AI notes 2024: The current state
I continue my notes on AI at the end of 2024.
- Industry transparency
- Generative knowledge base
- The current state
- Prognosis
In the previous posts, we discussed two theses:
- By analyzing the decisions of major AI developers, such as OpenAI or Google, we can make fairly accurate assumptions about the state of the AI industry.
- All current progress is based on a single base technology — generative knowledge bases, which are large probabilistic models.
Based on these theses, let's look at the current state of the industry.
Fun case of speeding up data retrieval from PostgreSQL with Psycopg
Speed of data retrieval from the PostgreSQL for each of the optimizations. In percentages of the base implementation speed. Note how the number of retrieved rows has little effect on execution time.
Once in a year or two, I have to remember that Python, umm... is not C++. It usually happens suddenly, like this time.
After a thoughtful analysis I got tired of waiting 10 seconds for news to load on feeds.fun, so I rolled up my sleeves and started optimizing. I almost jumped into an epic refactoring, but remembered just in time: measure first, then cut. In this case, the advice is literal — I took a profiler — py-spy — and checked what exactly was causing the slowdown
It turned out that the problem is not in the entire logic, but in a very specific place where the code extracts ~100000 rows from a PostgreSQL table, plus-minus 20000. The indexes are in place, I ran the tests with a database on a RAM disk, so everything should have been fine from the database side.
Don't be surprised by such a large number of rows:
- Firstly, I have a substantial news flow.
- Secondly, the reader currently assigns about 100 tags for each news item.
Armed with py-spy and the sources of psycopg, I went through three stages of optimization, reducing the function execution time approximately 4x solely by changing the format of the requested columns in SELECT query and the result processing code.
In the following text, I will tell you about the sequence of curious discoveries I made in the process.
Attention!
This post is not a study of Psycopg or Python performance, but a description of a specific experience with a specific task and specific data.
It would be incorrect to judge the performance of Psycopg, or Python in general, based on a single specific case.
Prompt engineering: building prompts from business cases
Ponies are building prompts (c) ChatGPT
As you know, one of the features of my news reader is automatic tag generation using LLMs. That's why I periodically do prompt engineering — I want tags to be better and monthly bills to be lower.
So, I fine-tuned the prompts to the point where everything seems to work, but there's still this nagging feeling that something's off: correct tags are determined well, but in addition to them, many useless ones are created, and sometimes even completely wrong ones.
There are a few options in such cases:
- Collect training data and fine-tune the model to generate only correct tags.
- Build a chain of actors, where one will create tags, and the other will filter out the unnecessary ones.
- Try to radically rework the prompt.
Options 1 and 2 were out of the question due to lack of time and money. Also, my current strategy is to rely exclusively on ready-made AI solutions since keeping up with the industry alone is impossible. So, I had no choice but to go with the third option.
Progress was slow, but after a recent post about generative knowledge bases, something clicked in my head, the problem turned inside out, and over the course of the morning, I drafted a new prompt that’s been performing significantly better so far.
So, let's look at the problem with the old prompt and how the new one fixed it.