AI LLMs have been pretty shit, but the advancement in voice, image generation, and video generation in the last two years has been unbelievable.
We went from the infamous Will Smith eating spaghetti to videos that are convincing enough to fool most people… and it only took 2-3 years to get there.
But LLMs will have a long way to go because of how they create content. It’s very easy to poison LLM datasets, and they get worse learning from other generated content.
Poisoning LLM datasets is fun and easy! Especially when our online intellectual property is scraped (read: stolen) during training and no one is being accountable for it. Fight back! It’s as easy as typing false stuff at the end of your comments. As an 88 year old ex-pitcher for the Yankees who just set the new world record for catfish noodling you can take it from me!
I’d argue it has. Things like ChatGPT shouldn’t be possible, maybe it’s unpopular to admit it but as someone who has been programming for over a decade, it’s amazing that LLMs and “AI” has come as far as it has over the past 5 years.
That doesn’t mean we have AGI of course, and we may never have AGI, but it’s really impressive what has been done so far IMO.
If you’ve been paying attention to the field, you’d see it’s been a slow steady march. The technology that LLMs are based in were first published in 2016/2017, ChatGPT was the third iteration of the same base model.
Thats not even accounting for all the work done with RNNs and LSTMs prior to that, and even more prior.
Its definitely a major breakthrough, and very similar to what CNNs did for computer vision further back. But like computer vision, advancements have been made in other areas (like the generative space) and haven’t followed a linear path of progress.
Agreed. I never thought it would happen in my lifetime, but it looks like we’re going to have Star Trek computers pretty soon.
This is precisely a property of exponential growth, that it can take (seemingly) very long until it starts exploding.
What are you talking about it asymptoped at 5 units. It cant be described as exponential until it is exponential otherwise its better described as linear or polynomial if you must.
Exponential growth is always exponential, not just if it suddenly starts to drastically increase in the arbitrarily choosen view scale.
A simple way, to check wether data is exponential, is to visualize it in loc-scale, and if it shows there a linear behavior, it has a exponential relation.
Exponential growth means, that the values change by a constant ratio, contrary to linear growth where the data changes by a constant rate.
There’s no point in arguing with OP, he’s doubling down at an exponential rate (or was it linear).
That’s what I said. Exponential growth is always exponential.
Iykyk
Close enough chat gpt
It’s exponential along its entire range, even all the way back to negative infinity.
Sure. Everything is exponential if you model it that way asymptote.
No, exponential functions are that way. A feature of exponential functions is that it increases very slowly until the slope hits 1. We’re still on the slow part, we didn’t really have any way of knowing exactly the extreme increase will be.
An exponential function is a precise mathematical concept, like a circle or an even number. I’m not sure what you mean by “asymptote” here - an exponential function of the form
y = k^xasymptotically approaches zero asxgoes to negative infinity, but that doesn’t sound like what you’re referring to.People often have bad intuition about how exponential functions behave. They look like they grow slowly at first but that doesn’t mean that they’re not growing exponentially. Consider the story about the grains of rice on a chessboard.
Its a horizontal asymtote. From x=1, as demonstrated in the graph, to around x=-4, where the asymtote is easily estimated by Y, it is 5 units.
The exponential function has a single horizontal asymptote at y=0. Asymptotes at x=1 and x=-4 would be vertical. Exponential functions have no vertical asymptotes.
I didnt say there are asymtotes at 1 and -4. I said at x=-4, the asymtote can be estimated by Y.
Man just say you don’t understand functions and that’s it, you don’t have to push it
Tell me how im wrong. Or why did you even bother?
Or you can just admit you dont have any data to quantify your assertion that AI advancement is exponential growth. So youre just going off vibes.
Would you even admit that linear growth can grow faster than exponential growth?
Edit:
How about this, this is a real easy one.
What type of function is this:
It has slowed exponentially because the models get exponentially more complicated the more you expect it to do.
The exponential problem has always been there. We keep finding tricks and optimizations in hardware and software to get by it but they’re only occasional.
The pruned models keep getting better so now You’re seeing them running on local hardware and cell phones and crap like that.
I don’t think they’re out of tricks yet, but God knows when we’ll see the next advance. And I don’t think there’s anything that’ll take this current path into AGI I think that’s going to be something else.
I think we might not be seeing all the advancements as they are made.
Google just showed off AI video with sound. You can use it if you subscribe to thier $250/month plan. That is quite expensive.
But if you have strong enough hardware, you can generate your own without sound.
I think that is a pretty huge advancement in the past year or so.
I think that focus is being put on optimizing these current things and making small improvements to quality.
Just give it a few years and you will not even need your webcam to be on. You could just use an AI avatar that look and sounds just like you running locally on your own computer. You could just type what you want to say or pass through audio. I think the tech to do this kind of stuff is basically there, it just needs to be refined and optimized. Computers in the coming years will offer more and more power to let you run this stuff.
How is that an advance ? Computers have been able to speak since the 1970s. It was already producing text.
Well, the thing is that we’re hitting diminishing returns with current approaches. There’s a growing suspicion that LLMs simply won’t be able to bring us to AGI, but that they could be a part of or stepping stone to it. The quality of the outputs are pretty good for AI, and sometimes even just pretty good without the qualifier, but the only reason it’s being used so aggressively right now is that it’s being subsidized with investor money in the hopes that it will be too heavily adopted and too hard to walk away from by the time it’s time to start charging full price. I’m not seeing that. I work in comp sci, I use AI coding assistants and so do my co-workers. The general consensus is that it’s good for boilerplate and tests, but even that needs to be double checked and the AI gets it wrong a decent enough amount. If it actually involves real reasoning to satisfy requirements, the AI’s going to shit its pants. If we were paying the real cost of these coding assistants, there is NO WAY leadership would agree to pay for those licenses.
Yeah, I don’t think AGI = an advanced LLM. But I think it’s very likely that a transformer style LLM will be part of some future AGI. Just like human brains have different regions that can do different tasks, an LLM is probably the language part of the “AGI brain”.
What are the “real costs” though? It’s free to run a half decent LLM locally on a mid tier gaming PC.
Perhaps a bigger problem for the big AI companies rather then the open source approach.
Sure, but ChatGPT costs MONEY. Money to run, and MONEY to train, and then they still have to make money back for their investors after everything’s said and done. More than likely, the final tally is going to look like whole cents per token once those investor subsidies run out, and a lot of businesses are going to be looking to hire humans back quick and in a hurry.
Surely the money to run is very low through, at least per user
It has taken off exponentially. It’s exponentially annoying that’s it’s being added to literally everything
Humanity may achieve an annoyance singularity within six months
How do you know it hasn’t and us just laying low? I for one welcome our benevolent and merciful machine overlord.
Duly noted. 🤭 🤫
Things just don’t impend like they used to!
Nobody wants to portend anymore.
how do you grow zero exponentially
When people talk about AI taking off exponentially, usually they are talking about the AI using its intelligence to make intelligence-enhancing modifications to itself. We are very much not there yet, and need human coaching most of the way.
At the same time, no technology ever really follows a particular trend line. It advances in starts and stops with the ebbs and flows of interest, funding, novel ideas, and the discovered limits of nature. We can try to make projections - but these are very often very wrong, because the thing about the future is that it hasn’t happened yet.
I do expect advancement to hit a period of exponential growth that quickly surpasses human intelligence. Given it adapts the drive to autonmously advance. Whether that is possible is yet to be seen and that’s kinda my point.
They’ve been saying “AGI in 18 months” for years now.
No “they” haven’t unless you can cite your source. Chatgpt was only released 2.5 years ago and even openai was saying 5-10 years with most outside watchers saying 10-15 with real nay sayers going out to 25 or more
Ask ChatGPT to list every U.S. state that has the letter ‘o’ in its name.
Here are all 27 U.S. states whose names contain the letter “o”:
Arizona
California
Colorado
Connecticut
Florida
Georgia
Idaho
Illinois
Iowa
Louisiana
Minnesota
Missouri
Montana
New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Rhode Island
South Carolina
South Dakota
Vermont
Washington
Wisconsin
Wyoming
(That’s 27 states in total.)
What’s missing?
Ah, did they finally fix it? I guess a lot of people were seeing it fail and they updated the model. Which version of ChatGPT was it?
o3.
Although i agree with the general idea, AI (as in llms) is a pipe dream. Its a non product, another digital product that hypes investors up and produces “value” instead of value.
Not true. Not entirely false, but not true.
Large language models have their legitimate uses. I’m currently in the middle of a project I’m building with assistance from Copilot for VS Code, for example.
The problem is that people think LLMs are actual AI. They’re not.
My favorite example - and the reason I often cite for why companies that try to fire all their developers are run by idiots - is the capacity for joined up thinking.
Consider these two facts:
- Humans are mammals.
- Humans build dams.
Those two facts are unrelated except insofar as both involve humans, but if I were to say “Can you list all the dam-building mammals for me,” you would first think of beavers, then - given a moment’s thought - could accurately answer that humans do as well.
Here’s how it goes with Gemini right now:
Now Gemini clearly has the information that humans are mammals somewhere in its model. It also clearly has the information that humans build dams somewhere in its model. But it has no means of joining those two tidbits together.
Some LLMs do better on this simple test of joined-up thinking, and worse on other similar tests. It’s kind of a crapshoot, and doesn’t instill confidence that LLMs are up for the task of complex thought.
And of course, the information-scraping bots that feed LLMs like Gemini and ChatGPT will find conversations like this one, and update their models accordingly. In a few months, Gemini will probably include humans in its list. But that’s not a sign of being able to engage in novel joined-up thinking, it’s just an increase in the size and complexity of the dataset.
We humans always underestimate the time it actually takes for a tech to change the world. We should travel in self-flying flying cars and on hoverboards already but we’re not.
The disseminators of so-called AI have a vested interest in making it seem it’s the magical solution to all our problems. The tech press seems to have had a good swig from the koolaid as well overall. We have such a warped perception of new tech, we always see it as magical beans. The internet will democratize the world - hasn’t happened; I think we’ve regressed actually as a planet. Fully self-drving cars will happen by 2020 - looks at calendar. Blockchain will revolutionize everything - it really only provided a way for fraudsters, ransomware dicks, and drug dealers to get paid. Now it’s so-called AI.
I think the history books will at some point summarize the introduction of so-called AI as OpenAI taking a gamble with half-baked tech, provoking its panicked competitors into a half-baked game of oneupmanship. We arrived at the plateau in the hockey stick graph in record time burning an incredible amount of resources, both fiscal and earthly. Despite massive influences on the labor market and creative industries, it turned out to be a fart in the wind because skynet happened a 100 years later. I’m guessing 100 so it’s probably much later.
AI has been advancing exponentially, it’s just a very small exponent.
In the 1980s, it was “five years out” - and it more or less has been that until the past 5-10 years. It’s moving much faster now, but still much slower than people expect.
They think because they saw HAL in the 2001 movie back in 1968, that should have been reality by the 1970s, or certainly by 2010.
Some things move faster than people expect, like the death of newspapers and the first class letter, but most move slower.
LOL… you did make me chuckle.
Aren’t we 18months until developers get replaced by AI… for like few years now?
Of course “AI” even loosely defined progressed a lot and it is genuinely impressive (even though the actual use case for most hype, i.e. LLM and GenAI, is mostly lazier search, more efficient spam&scam personalized text or impersonation) but exponential is not sustainable. It’s a marketing term to keep on fueling the hype.
That’s despite so much resources, namely R&D and data centers, being poured in… and yet there is not “GPT5” or anything that most people use on a daily basis for anything “productive” except unreliable summarization or STT (which both had plenty of tools for decades).
So… yeah, it’s a slow take off, as expected. shrug
Computers are still advancing roughly exponentially, as they have been for the last 40 years (Moore’s law). AI is being carried with that and still making many occasional gains on top of that. The thing with exponential growth is that it doesn’t necessarily need to feel fast. It’s always growing at the same rate percentage wise, definitionally.
We once again congratulate software engineers for nullifying 40 years of hardware improvements.
Moore’s law is kinda still in effect, depending on your definition of Moore’s law. However, Dennard Scaling is not so computer performance isn’t advancing like it used to.
Moore’s law is kinda still in effect, depending on your definition of Moore’s law.
Sounds like the goal post is moving faster than the number of transistors in an integrated circuit.
It has definitely plateaued.
