Nearly a month ago, I decided to add Gemini support to Feeds Fun and did some research on top LLM frameworks — I didn't want to write my own bicycle.
As a result, I found an embarrassing bug (in my opinion, of course) in the integration with Gemini in LLamaIndex. Judging by the code, it is also present in Haystack and in the plugin for LangChain. And the root of the problem is in the Google SDK for Python.
When initializing a new client for Gemini, the framework code overwrites/replaces API keys in all clients created before. Because the API key, by default, is stored in a singleton.
It is death-like, if you have a multi-tenant application, and unnoticeable in all other cases. Multi-tenant means that your application works with multiple users.
For example, in my case, in Feeds Fun, a user can enter their API key to improve the quality of the service. Imagine what a funny situation could happen: a user entered an API key to process their news but spent tokens (paid for) for all service users.
I reported this bug only in LLamaIndex as a security issue, and there has been no reaction for 3 weeks. I'm too lazy to reproduce and report for Haystack and LangChain. So this is your chance to report a bug to a top repository. All the info will be below, reproducing is not difficult.
This error is notable for many reasons:
Ultimately, I gave up on these frameworks and implemented my own client over HTTP API.
My conclusion from this mess is: you can't trust the code under the hood of modern LLM frameworks. You need to double-check and proofread it. Just because they state that they are "production-ready" doesn't mean they are really production-ready.
Let me tell you more about the bug.
As a hobby, I write concept documents for games. This is first in English. I have a few more in Russian and will eventually translate them.
One more concept for The Tale 2.0.
Lords Captains MMO
Yep, it's a rip-off from Warhammer 40k and Rogue Trader, but it will do for the concept.
Explore the infinite universe on a starship with millions of souls on board, unite and develop abandoned worlds.
Browsers, mobile.
Exploration-driven trade-political MMO PVE sandbox.
EVE, Sim City, Crusader Kings, 4X games, Rogue Trader.
"Piranesi" is both a continuation of the magical stories of Susanna Clarke and an independent book.
The book has no direct connection with the world of English magic [ru] from "Jonathan Strange & Mr. Norrell". If desired, one can find a connection and even say that the worlds are the same, only at different times: the events of "Piranesi" take place in the early 2000s. However, the author did not give any hints on this. Therefore, I consider the worlds to be different for now.
Susanna continues to persistently and effectively dig not even in the direction of animism as the basis of world perception but in the direction of extremely holistic view of the world, in contrast to the currently dominant reductionism.
The latter blows my mind. As an engineer, I'm an intuitive reductionist due to professional deformation. Reading "Jonathan Strange" and "Piranesi", I felt how Clarke, like Peter the Great, cuts a window in my brain to another picture of the world, a different world perception. And it's wonderful.
By the way, don't confuse holism with, say, an engineering view of the world, a-la systems engineering [ru] or even science. The latter is about decomposing reality into isolated parts with clear boundaries and synthesizing "pure" models of the world [ru], while in holism, the parts have no clear boundaries and penetrate each other.
But it is my interpretation, there are interpretations when holism is just an alternative name for a systems thinking/view — it's hard to find literature on this topic now, so it's hard for me to say where the truth is.
So, "Piranesi"
From the player preference survey, I gradually moved on to working on a game prototype.
The game will be about a news agency. You will be the chief editor, and your task is to manipulate public opinion by investigating events and choosing a connotation of news: where to draw the public's attention, what to hide, in what tone to present themes, etc.
Therefore, the whole game will be around the text of news.
Creating large blocks of detailed text for each news item looks pointless — the game is not about reading news but about managing them. Therefore, it makes sense to build interaction only around headlines.
But how can we make the displaying of news both interesting and simple?
Recently I've conducted a survey about the preferences of strategy players.
In the previous post, we cleaned up the data, and in this one, we will try to find insights within it.
In this post you will find an interactive dashboard with a bunch of charts, where you can compare two samples of your choice. There are many samples — for every taste and color, so feel free to explore and share the patterns you find on Telegram and Discord.
But be careful with conclusions. There is little data, in some cases very little. For example, the difference between the sample sizes of male and female respondents is about tenfold => you should be very careful in interpreting the differences between them.
In general, do not take this post as a full-fledged study. I'm sure many analysts would have torn my hands off for such a thing. Then sewed them back and torn them off again :-D Use the post as an interface to the data, and make your own conclusions.