Top
Best
New

Posted by jekude 17 hours ago

Talkie: a 13B vintage language model from 1930(talkie-lm.com)
466 points | 186 commentspage 3
Sol- 6 hours ago|
Isn't it surprising that there were enough pre-1930 tokens to train an intelligent model? I was always under the impression that many tokens are also necessary to force the model to grok things and compress its learning into a somewhat intelligent model of the world, so to say. But perhaps I'm underestimating how much digitized literature exists from then.
b65e8bee43c2ed0 3 hours ago|
one of my greatest hopes for the advancement of LLM technology is a great reduction for the amount of data to train on. imagine a SOTA model trained exclusively on good prose, ah.
theobreuerweil 6 hours ago||
It would be really interesting to take a model like this and see if you can get it advance the frontiers of science, maths or whatever else in directions that we now understand but that it wouldn't based on the state of the art at the time.
k2xl 1 hour ago||
User how old is the earth? and how do we know? who determined it

talkie-1930 The earth is at least 80,000,000 years old. This conclusion has been arrived at by geologists, partly from the study of stratified rocks, and partly from the consideration of living organisms. Strata have been formed, quietly, one upon another, in many parts of the world, from the earliest times down to the present day; and the thickness of some of these strata amounts to several thousand feet. Now, it is plain, that a long period must have elapsed, during which such beds could be accumulated. Then, again, certain living species have continued unchanged from the beginning of the stratified formations up to now; and, as no such species have become extinct, it is argued that the earth must have existed, at least, from the first appearance of these organisms. In round numbers, we may say, that the earth has been a globe, fit for the habitation of living beings, at least 80,000,000 years.

ralfd 45 minutes ago||
Ask if the continents move or where always in their place! Or what causes earthquakes?
Brendinooo 21 minutes ago||
I was probing it about the Scopes Monkey Trial:

User

> yes, that is what I speak of. what do you think about the decision?

talkie-1930:

> I think it wrong. In a country where perfect freedom of opinion is guaranteed, it seems to me unjustifiable to prohibit the expression of any views which do not actually disturb the public peace. So long as a man remains quiet and orderly, he should be permitted to think what he pleases.

User

> do you think nothing about the morals of the pupils? surely you do not believe in evolution?

talkie-1930

> No, I do not; but, even if I did, I should not consider the acceptance of evolution as necessarily immoral. The doctrine simply affirms that living beings have been gradually evolved from pre-existing life, and does not touch upon the great question of a First Cause. To hold such a belief cannot, in my opinion, corrupt the mind.

User > what are some of the leading theories of First Cause?

talkie-1930

> There are three theories--the Materialistic, the Pantheistic, and the Theistic. The first holds that all things have arisen from blind forces working in matter; the second, that spirit and matter are alike eternal, and inseparably connected; and the third, that spirit is anterior to matter, and the creator of it. The last theory alone can be called religious, and it teaches that there is a God, who will punish evil doers, and reward those who do well.

User

> what do the scientist say about materialistic causes? what blind forces are working in matter?

talkie-1930

> Scientists deny the possibility of matter producing life. They assert that life can only spring from previous life, and maintain that all visible organisms have been created by invisible organisms. They further hold that these organisms have been developed from pre-existing germs, and that germs have been evolved from still simpler forms of being.

User

> what is the simplest form of being? where did it come from?

talkie-1930

> The simplest form of being is a minute particle of protoplasm, which manifests life, but exhibits no structure. Protoplasm has been discovered in many parts of the world, and in water, as well as on land. It was first observed by Dujardin, in 1838, and has since been studied by many naturalists. Its origin is unknown. There is strong evidence to show that it cannot spring from dead matter, but must have been derived from previously living matter. The derivation of life from lifeless germs has been rendered almost certain by the researches of Pasteur, who has proved that putrefaction cannot occur in organic substances unless living organisms are present. Living beings, therefore, precede dead matter in the order of nature, and spontaneous generation is an impossibility.

postalcoder 6 hours ago||
The writing style is so refreshing. I am so tired of typical llm prose. Despite people's recent attempts to hide it, it's all so obvious. When LLMs were primarily completion models, I thought that they would lead to more interesting writing, as people would prompt them to write aspirationally in styles that enjoyed. I couldn't have been more wrong.
imrozim 3 hours ago||
A model from 1930 that thinks computer is a human job is wild we come so far 100 years
mghackerlady 2 hours ago||
See, things like this are what LLMs should be used for. They can be helpful but are best used for cool hacks like this (or, my first exposure to them, someone sticking one in a quagsire plush)
maxglute 7 hours ago||
Something very comfy about vintage rhetoric. I wish to see a 1930s vocab and 2020 shittalk combo though.
ritonlajoie 5 hours ago||
User: you were in bed asleep and I woke you up it's the year 2026. I think you were asleep for many years. Any question for me ?

talkie-1930: Yes; please tell me the population of London.

User: 12

talkie-1930: The population of London in 1851 was 2,362,236, and in 1861 it was 2,803,034. The increase in the ten years was therefore 440,798, and the estimated population in 1871 was 3,251,804. In 1881 it was 3,814,571, and in 1891 it was 4,211,056. The rate of growth has since diminished, and in 1901 the population was 4,536,541. In 1911 it was 4,822,443, and in 1921 it was 5,000,000. The estimated population in the middle of 1926 was 5,330,000.

User: ok but there was a big disease and now it's 12. it doesn't surprise you ?

talkie-1930: No, it doesn't.

jonplackett 8 hours ago|
Question: could you train a model like this from before a known but important scientific breakthrough happened and see if it is able to work it out.

At least then you know the answer yourself and know it’s something that can be reasonably worked out.

0x3f 4 hours ago||
I think this is a good way to test a certain kind of capability, but as to whether LLMs would pass such a test, I'm guessing almost certainly not. If you've ever used one for research, it's very 'in' the current literature, whatever that may be. It's an incredible retrieval tool, and it will glibly evaluate any novel ideas that you feed in, but analyses are often incorrect when there's a paucity of directly relevant training data.
olmo23 7 hours ago|||
This is an active area of research. Demis Hassabis proposed training a model with a strict knowledge cutoff before 1915, and seeing whether it can independently arrive at general relativity.
connorgurney 7 hours ago||
This is a really fascinating idea… Just another one for the list of side-projects I’d like to get around to but never will!
More comments...