Top LLM frameworks may not be as reliable as you think
Nearly a month ago, I decided to add Gemini support to Feeds Fun and did some research on top LLM frameworks — I didn't want to write my own bicycle.
As a result, I found an embarrassing bug (in my opinion, of course) in the integration with Gemini in LLamaIndex. Judging by the code, it is also present in Haystack and in the plugin for LangChain. And the root of the problem is in the Google SDK for Python.
When initializing a new client for Gemini, the framework code overwrites/replaces API keys in all clients created before. Because the API key, by default, is stored in a singleton.
It is death-like, if you have a multi-tenant application, and unnoticeable in all other cases. Multi-tenant means that your application works with multiple users.
For example, in my case, in Feeds Fun, a user can enter their API key to improve the quality of the service. Imagine what a funny situation could happen: a user entered an API key to process their news but spent tokens (paid for) for all service users.
I reported this bug only in LLamaIndex as a security issue, and there has been no reaction for 3 weeks. I'm too lazy to reproduce and report for Haystack and LangChain. So this is your chance to report a bug to a top repository. All the info will be below, reproducing is not difficult.
This error is notable for many reasons:
- The assessment of the criticality of the error depends a lot on taste, experience, and context. For me, in the projects I worked on, this is a critical security issue. However, it seems that this is not critical at all for most current projects that use LLMs. Which leads to some thoughts about mainstream near-LLM development.
- This is a good indicator of a low level of code quality control: code reviews, tests, all processes. After all, this is an integration with one of the major API providers. The problem could have been found in many different ways, but none worked.
- This is a good illustration of the vicious approach to development: "copy-paste from a tutorial and push to prod". To make such a mistake, you had to ignore both the basic architecture of your project and the logic of calling the code you are copying.
Ultimately, I gave up on these frameworks and implemented my own client over HTTP API.
My conclusion from this mess is: you can't trust the code under the hood of modern LLM frameworks. You need to double-check and proofread it. Just because they state that they are "production-ready" doesn't mean they are really production-ready.
Let me tell you more about the bug.
Preparing a business plan for a game on Steam
Earning millions is easier than ever. I'll tell you how :-D
When I posted my final presentation [ru] (slides) for World Builders 2023 (my posts, site), I promised to tell how I made a roadmap and a financial model for the game. So, here they are.
At the end of this post, we will have:
- A brief strategy of our company: what we do, how, and why.
- A table with our beacons — successful games roughly similar to what we want to make. Similar in gameplay, team size, budget, etc.
- A composition of the team we need to assemble.
- A roadmap — a development plan for our game.
- An outline of our marketing strategy.
- A financial model — how much we will spend, how much we will earn.
- A large number of my caveats throughout the post.
- Jokes and
I hopewitty remarks.
All the final documents can be found here.
Review of the book "The Signal and the Noise"
Nate Silver — the author of "The Signal and the Noise" — is widely known for his successful forecasts, such as the US elections. It is not surprising that the book became a bestseller.
As you might guess, the book is about forecasts. More precisely, it is about approaches to forecasting, complexities, errors, misconceptions, and so on.
As usual, I expected a more theoretical approach, in the spirit of Scale [ru], but the author chose a different path and presented his ideas through the analysis of practical cases: one case per chapter. Each chapter describes a significant task, such as weather forecasting, and provides several prisms for looking at building forecasts. This certainly makes the material more accessible, but personally, I would like more systematics and theory.
Because of the case studies approach, it isn't easy to make a brief summary of the book. It is possible, and it would even be interesting to try, but the amount of work is too large — the author did not intend to provide a coherent system or a short set of basic theses.
Therefore, I will review the book as a whole, provide an approximate list of prisms, and list some cool facts.
About the book "Piranesi"
Cover of the book "Piranesi"
"Piranesi" is both a continuation of the magical stories of Susanna Clarke and an independent book.
The book has no direct connection with the world of English magic [ru] from "Jonathan Strange & Mr. Norrell". If desired, one can find a connection and even say that the worlds are the same, only at different times: the events of "Piranesi" take place in the early 2000s. However, the author did not give any hints on this. Therefore, I consider the worlds to be different for now.
Susanna continues to persistently and effectively dig not even in the direction of animism as the basis of world perception but in the direction of extremely holistic view of the world, in contrast to the currently dominant reductionism.
The latter blows my mind. As an engineer, I'm an intuitive reductionist due to professional deformation. Reading "Jonathan Strange" and "Piranesi", I felt how Clarke, like Peter the Great, cuts a window in my brain to another picture of the world, a different world perception. And it's wonderful.
By the way, don't confuse holism with, say, an engineering view of the world, a-la systems engineering [ru] or even science. The latter is about decomposing reality into isolated parts with clear boundaries and synthesizing "pure" models of the world [ru], while in holism, the parts have no clear boundaries and penetrate each other.
But it is my interpretation, there are interpretations when holism is just an alternative name for a systems thinking/view — it's hard to find literature on this topic now, so it's hard for me to say where the truth is.
So, "Piranesi"
Computational mechanics & ε- (epsilon) machines
I found a few new concepts for tracking.
Computational mechanics
There is computational mechanics, which deals with numerical modeling of mechanical processes and there is an article about it on the wiki. This post is not about it.
This post is about computational mechanics, which studies abstractions of complex processes: how emergent behavior arises from the sum of the behavior / statistics of low-level processes. For example, why the Big Red Spot on Jupiter is stable, or why the result of a processor calculations does not depend on the properties of each electron in it.
ε- (epsilon) machine
The concept of a device that can exist in a finite set of states and can predict its future state (or state distribution?) based on the current one.
Computational mechanics allows (or should allow) to represent complex systems as a hierarchy of ε-machines. This creates a formal language for describing complex systems and emergent behavior.
For example, our brain can be represented as an ε-machine. Formally, the state of the brain never repeats (voltages on neurons, positions of neurotransmitter molecules, etc), but there are a huge number of situations when we do the same thing in the same conditions.
Here is a popular science explanation: https://www.quantamagazine.org/the-new-math-of-how-large-scale-order-emerges-20240610/
P.S. I will try to dig into scientific articles. I will tell you if I find something interesting and practical. P.P.S. I have long been thinking in the direction of a similar thing. Unfortunately, the twists of life do not allow me to seriously dig into science and mathematics. I am always happy when I encounter the results of other people's digging.