Top
Best
New

Posted by polyphilz 16 hours ago

Training a 4B model to produce 81% faster query plans than Postgres(rohanbansal.com)
556 points | 118 commentspage 3
perrygeo 11 hours ago|
Nice article about how to train/fine-tune a language model.

However, it misses the whole point of database query planning. You can't just ignore the planning time itself, as if the database query were a static entity to be optimized once at a leisurely pace.

The real constraint on live query planners is quite different: they must improve the combined time - planning + query - based on live database statistics. You can amortize the planning with prepared statements, but that too is fraught since optimal plans can change quite frequently and based on input parameters. "Live" and "faster than the queries themselves" are the hard requirements to be considered a viable database query planner. This project does neither.

kingjimmy 15 hours ago||
Aren't optimizations suppose to be deterministic?
tintor 15 hours ago||
They are not. Choice among several query plans depends on various summary statistics about the data, which might not be the most recent.
cogman10 15 hours ago||
Including the input parameters.

It's not unusual for us to end up with bad query plans because the shape of our data can vary pretty greatly. In many cases, a Foo has 1 Bar. But in some cases, a Foo has a million Bars. That can cause the query optimizer to treat lookups on the bar table as if there are few elements there (causing a scan instead of a seek).

For the general case, the optimizer gets it right. However, the fringe case is one that causes the entire system to crash. It's a bit akin to how an insertion sort can be faster than quick sort when n is small. The optimizer might make a bad assumption about the size of n which makes it pick an expensive n lookup when log(n) is available (but slower for small n).

cowboylowrez 15 hours ago||
I'd like to contribute my amateur hour entry into this thread, although I did administer and develop mssql stuff for awhile.

sure optimizations based on stats, but the stats are the wildcard, in my experience query plans can change suddenly.

Queries are translated into plans according to statistics. However the transforms will be deterministic and should only change one valid plan to another. I could very easily see a neural network manipulate transforms the same way the current programming does, its just that the neural networks are by nature really nicely suitable because the "decisions" are based on training, and this training can be closed world type things like the ai assists that chess engines are now getting. Obviously ai still can't play chess but apparently its very good at ranking board positions just by developing that much statistical info because its training comes not from reading the web, but playing a gazzilian games against itself in a "closed" chess world of its own.

I'm thinking that the ai does "this legal transform of the query plan should be applied to this pattern of data (statistics, cardinality, etc)" simply because the ai encountered it in closed world training, much like the chess thing.

Just a theory tho feel free to correct!

krisoft 14 hours ago||
> Obviously ai still can't play chess

I believe you are wrong on that. Do you mean large language models can’t play chess?

cowboylowrez 13 hours ago||
I don't believe that a pure neural network can currently abide by the rules of chess, even with unreasonable amounts of training. Do you have a counter example? I never mind having beliefs challenged with facts lol

edit: I think you could provide an AI with a service or skill that asks "is this move legal" but given all the overhead for llms or whatever to call a "legal move" service external to its process, well then you aren't really searching the tree very efficiently lol.

However if you just let a neural network score boards and the neural network is in the same process well then I think thats the working solution for using neural networks in chess. The net does not need to score all boards either, simple value based heuristics can obviously provide a preliminary list of good boards (moves) at a certain depth or ply and then select the move that produces the board that the neural net scores highest. I kinda sorta think thats whats done today but as usual I could be full of it lol

cannonpalms 10 hours ago||
There's quite a big difference between "AI" and "pure neural networks." No, frontier chess (Alpha zero etc) is not "pure," because there's Monte Carlo tree search and a hard-coded game rules implementation.
fsmv 15 hours ago||
But how will you know that the query plan actually does what your query asked for?
amluto 15 hours ago||
I would like to think that pg_hint_plan is designed in such a way that any hint it accepts must be a valid plan for the query. I’m quite confident that schemes with this property that can also express high quality plans are possible and not even excessively complicated.

This is not to say that it’s possible to genetically verify that a proposed algorithm does what you want it to — that would be undecidable or NP-hard or co-NP-hard depending on how you formulate the question.

hedgehog 15 hours ago||
I wouldn't be very excited about adding a 4B param model to my database deployment, but using this kind of approach while testing an app to identify query plans where Postgres is leaving performance on the table seems valuable without much risk.
whazor 14 hours ago|||
Given the approach from the article, you can commit the hints to git and run tests for verification. The model would be used during coding.
cannonpalms 10 hours ago|||
If your statistics or workload change, this approach is useless. The hints are generated being generated ahead of time, taking 95hrs to do so.
polyphilz 15 hours ago|||
`pg_hint_plan` has a debug log so you can verify Postgres actually used the hint or not! Used this during evaluations
topaz0 12 hours ago|||
You're misunderstanding the setup here. The LLM doesn't modify the query, just some details about how to choose between different ways to break the query into basic operations on the tables. The SQL doesn't change. It's still up to postgres to guarantee that the results match the query. If the proposed plan were nonsense that didn't amount to carrying out the query, postgres would ignore it.
timcobb 15 hours ago|||
I imagine you can perform operations on query plans to transform them and determine equivalence?
larodi 14 hours ago|||
you'll have to prove equivalence through some Lean4 code perhaps? or some weird clause tree comparisons... good question indeed.
quotemstr 14 hours ago|||
Because P!=NP (very probably IMHO) there's a huge class of problem for which LLMs are useful on the expensive and heuristic-y generation side because the verification is relatively inexpensive.
KK7NIL 15 hours ago||
"... make no mistakes" :)
mohd_rafay 1 hour ago||
[flagged]
aitoolcrux 9 hours ago||
[flagged]
kevinbaiv 13 hours ago||
[flagged]
saiyamshah1496 11 hours ago||
[flagged]
tobin1994 9 hours ago||
[dead]
basil_io 10 hours ago||
[dead]
tonetheman 11 hours ago|
[dead]
More comments...