It has to be pure ignorance.
I only have used my works stupid llm tool a few times (hey, I have to give it a chance and actually try it before I form opinions)
Holy shit it’s bad. Every single time I use it I waste hours. Even simple tasks, it gets details wrong. I correct it constantly. Then I come back a couple months later, open the same module to do the same task, it gets it wrong again.
These aren’t even tools. They’re just shit. An idiot intern is better.
Its so angering people think this trash is good. Get ready for a lot of buildings and bridges to collapse because of young engineers trusting a slop machine to be accurate on details. We will look back on this as the worst era in computing.
Generally I equate positivity about LLMs with people’s technical ability. I find the more they say AI is good the worse programmer they are.
Technical literacy in general. My friend thinks it’s the greatest thing ever, is an idiot with technology (and life in general).
It also says a lot about their inability to identify bullshit
Might be some dunnig kruger curve there. Not tooting my horn but I know my ways around and I only use ai for programming when I more or less know how it works already. Which means I verify and fix any eventual problems before committing any code. It does speed up the process, it’s a tiny bit simpler than checking stuff out on stack overflow IMO.
Now, if you don’t know your ways around, and “trusts” the outcome on an LLM, boy are you in trouble 😵💫.
every time I use it, i waste hours
Yes, exactly this. It looks good, I ask for it to tweak something. It tweaks, but now something else needs adjustment. Then it comes back unusable.
It ends up taking the same time as doing it myself. There’s some value perhaps in either the novelty or engagement that keeps me focused but it’s not more efficient.
When it does work, I’m always worried it is an illusion I’ve missed something. Like how you send an email and immediately see the typo.
People who love it, love it because they don’t need to or care about having accuracy and precision in their work. Sales and marketing, management, etc. Business idiots.
Ed ed ed!!!
I know this community is all about fuck AI, but this is just straight echo chambering.
But honestly your post sounds like you’re just not using it right? You can get pretty good results with it with enough guardrails. Just because you can’t get the results you want doesn’t mean that no one can.
That said, fuck AI. It’s all a bunch of bullshit, but denying real results just means you’re sticking your head in the sand and that’s not how you fix this problem.
I agree… Saying LLMs are good at nothing is just plain ignorance… One can disagree with the philosophy or dislike hallucinations, but they are definitely good at some things.
It’s basically like Google with a bit more detail in my experience. Everytime I’ve tried to use it in a professional context, I’ve come up massively empty. Pages and pages and pages and pages of just absolutely walls of text, but nothing actually useful. I mean I’ve got it to calculate stuff and whatever, but then you examine something and its not coming up for you like the LLM says it should be. Which pretty much immediately means you have to validate everything else, and then it’s like well hey look here I am however many hours later, manually doing something.
Our executives keep telling us to adapt or we’ll be on the losing end. At this point, I’d just like the check please. Because if the company can survive on images of Super Mario committing 9/11, or walls of useless text or just straight up make belief, that’s something I’d like to watch from the sidelines.
pretty good results with it with enough guardrails
examples?
For a research project, I had to convert 20+ projects from a dataset into a new format. The old format was simply a single script for each project that builds it. But I needed a format with a Docker file and a script. It would’ve taken me around a week to do all that one by one.
I got Claude to do it in 2 hours.
I know people hate AI in this community, but to say it doesn’t do anything good or to insult all people who use it is just pure negativity.
Thanks for sharing. I’m not sure where me asking for examples “say it doesn’t do anything good or to insult all people who use it”. Someone makes a claim without any proof, I ask for proof. To me that sounds both simple and legitimate.
Thats good. It has use cases. Is the monetary and earth destroying cost worth it? Not in the slightest
Or that it’s not right for their use case.
Like someone throwing a bunch of data into an LLM and trying to use it to process it into a chart or something. It can work, but it was never designed to be used in that manner.
I’ve got an acquaintance who does that, despite the fact that python would be a better thing to use.
Personally, I sometimes run a few saved images thorough a multi-modal 8 gigaparameter local model on my computer, so I can automate giving them more descriptive names than randomnumbers.png, and that seems to work fine. I could do it by hand, but it would take hours and days, compared to minutes, and since it’s not too important, it doesn’t matter if it’s wrong. The resource usage is also less of an issue, since it’s my own computer.
There are right tools and wrong tools depending on the application.
There are right ways to use said tools and wrong ways…like you wouldn’t use a phillips head screwdriver on a flat head.
I guarantee your company’s provided tool is Copilot or OpenAI based, which is already bottom of the barrel for usefulness.
Haha, yes it is
Inuse a flathead (minus) screwdriver on a philips (plus) screw all the time
Id say you dont know how to use the tool. ‘Write tests here, develop.feature X’ is not a good way to use llms. Using 80k tokens and keep using same a Session is context rotting. There are a lot of boring, everyday tasks in my job that got faster. Many others that meh. Use AI, dont be driven by AI.
I think I’m going to use AI to tell me how to use AI.
Claude and super powers / planning have changed my mind more on AI feature development. Iterating on the spec and making it as unambiguous as possible gives good results when you clear context and have it implement the plan. Even if it starts to stray you can just do a git reset and start a new session with the spec, adjusting it a bit, because time wise you probably haven’t invested much.
It also depends on the code base, if the code base has very clear separation of concerns, good documentation, and good contracts between layers then claude can handle it pretty well. If the code base is full of spaghetti code with multiple ways to do the same thing then AI will struggle with it. In our large legacy monolith repo it doesn’t do well, in our micro service repos it does great.
Also time wise it may not seem like a benefit if you just set it and wait for it to complete, the productivity advantage comes from running a couple sessions in parallel.
Also context is key, having a good claude.md file in the repo to explain patterns helps it to avoid pitfalls. If it’s only context is the prompt you gave it and you tell it to implement a feature without a plan / spec outlined it will generate shit code.
Making it as unambiguous as possible
If only we had a way to communicate with machines in a reliable, deterministic and unambiguous way.
But natural language will let people without computer programming skills use their business domain knowledge to create compute programs
It’s why COBOL is so popular.
Yeah you can write the code yourself. You can also write in c or even assembly if you really want to make it as unambiguous as possible, it’ll just take more time. Some people like to code in Python though because they can write faster with it even if a lot of implementation details and choices are hidden from them because they don’t care about those details.
Spec driven development in my view is just another step, albeit a big one, on the level of abstraction between assembly and python. Like python it has its places and has places where it should never be used for safety and performance reasons.
They may not care about the implementation details of a Python library, they do care about consistent execution and predictable results. And in some edge cases, they will care about the documentation saying exactly how those edge cases are handled.
Writing Python is abstraction, yes, but it’s still programming. Once that Python code is written and tested and the dependencies are locked down, you can ship it and be certain it always works as designed.
Spec-driven code generation is nothing like that. I can’t ship the specs. I could generate the code in a pipeline and ship that, maybe. But there’s no way I’m getting consistent builds from a code generator. So what do people do? They generate the code and put it in source control for review. When have you ever checked-in a compiled executable or looked at it? There’s machine code in there, shouldn’t you review that the compiler did what you asked of it?
Consistency is dependent on the code base and not the “compiler” in this sense. If the code base has consistent patterns and only has one well documented way to implement something then the AI will follow those patterns, ie. If there is only one way to run a job, AI will use that method. There might be some variation in variable names, formatting, etc. but the core flow should be consistent between “runs”
You can and should still test your code , both manually and with automation to ensure it does what it says it does. Testing should be the way you are certain it always works as designed. IMO understanding your tests and test coverage is more important than understanding the implementation. This is why part of the spec for superpowers is a test plan, and that should be the most reviewed / iterated part.
I recently used it to install Nvidia l40s drivers on redhat 9 and pass it through to my Frigate instance. Took me a few minutes. Would have been a lot of reading to find the exact answers manually.
Not a bad usage
I also used it to compile the correct Yolo detection models since I wrote this comment. I gave it my specific cameras and settings and it told me how to compile the correct model. I have 200 cameras running now. Almost unheard of on a deployed Frigate instance. Using the correct cuda model now I’m seeing 38 percent usage average on my l40s compared to 80 before.
For every post I see of people complaining, I have to imagine there are 100 other people that get value out of LLMs quietly.
holy fucking shit man. this community has a clear astroturfing problem.
A lot of people have their livelihood tied to the narrative that LLM deserves every cent of investment. The fact that it’s utility is more limited is an existential threat to their careers.
The truth that it is selectively useful gives them a thread of hope, but the fact it is useless for a lot of stuff drives irritation. We don’t make a distinction between the sort of work that LLM can do and can’t so people end up completely dumbfounded by the other perspective.
I don’t really know what the word means. I was posting my experiences.
not anything to do with your post. it’s the comments here.
fuckai used to be a community where no exceptions were made for AI. it’s quite literally in the name of the community.
so many posts from this community over the last 3-4 weeks has had an increasing amount of users that are “astroturfing” that AI has its uses and can be helpful sometimes. I kind of feel like these comments are made disingenuously as a way to silence the community at large by over commenting in a community that was created literally to hate AI, no exceptions.
anyway, won’t stop me from never using AI. If anything it’ll just make me read more books from before AI was a thing.
OHH I thought the opposite.
Yeah, don’t generate code with it. Treat it like StackOverflow. It does pretty good at that.
This is the only way I use it, and I do it grudgingly only because AI has ironically also ruined the web and web search. It’s also a last resort for when Kagi isn’t helping.
Unfortunately for me it’s a kpi so I need to figure out how to do something useful with it.
LLM is good for
- temporary scripts like to export data
- boilerplate for new code
- simple or repetitious code like unit tests
But just in time for my performance review, I spent a week ignoring my work to set and tweak rule sets. Now it can be noticeably more useful
- set context so it understands your code better. No more stupid results like switching languages, making up a new test framework, or randomly use a different mocking tool
- create actions. I’m very happy with a code refactoring ruleset I created. It successfully finds refactoring opportunities (matches cyclonatic complexity hotspots) and recommends approaches and is really good at presenting recommendations so I can understand and accept or reject. I tweaked it until it no longer suggests stupid crap, although I really haven’t been able to use much of the code it tries.
- establish workflow. Still in progress but a ruleset to understand how we use our ticketing system, conventions for commit messages , etc. if I can get it to the point of trusting it, it should automate some of the source control actions and work tracking actions
I agree with all that, especially if your performance is being measured by your use of LLMs. Those are cases where I find the code generation to be ok and doesn’t create comprehension debt.
Just literally make something up and get it to lie about something. This is literally the land of make belief at this point, all this KPI shit. Don’t stress about it. Execs want slop, give em slop.
For programming, at least it’s a good way to speed up things that you know how to do but take some time to type, or you don’t remember the syntax of. But relying on AI any more than that usually means you’ll be adding free technical debt and debugging time or becoming dependent on it.
I cut my LLM usage to almost zero because of environmental and political reasons, but it was helpful enough to wish it could be sustainable and not another tool in the dystopian take on the world.
local models are advanced enough to the point where you can run em as needed without datacenter.
the datacenter craze is basically just an excuse to get the banks (and eventually the american taxpayer, via bailouts when they fail) to fund your local nepitistic infrastructure rollout.
the entire US economy is built around the purposeful boom/bust system, as it’s very effecient at “bagging” people that don’t know the rules.
They’ve still had a huge power investment in creating them.
It really depends on the task and the tool. Current MOE models that have agentic hooks can actually be really useful for doing automated tasks. Generally, you don’t want to be using AI to create things. What you want to do is hand it a very clear set of instructions along with source material. And then tell it to either iterate build on summarize or in some cases create from that.
I created a simple script with the help of AI to automate scanning files in from an automatic document reader. Convert them to OCRD PDFs. Scan through the document properly. Title the file based on the contents, then create a separate executive summary and then add an index to a master index file of a growing json.
Doing this allowed me to automate several steps that would have taken time and in the end I’m able to just search through my folders and my PDFs and very quickly find any information I need.
And this is only scratching the surface. I wouldn’t have AI write me a resume or write me an email or a book. I might use it to generate an image that I then give to a real artist saying this is kind of what was in my head.
But boring stuff repetitive stuff. Things that really benefit from automation with a little bit of reasoning and thinking behind it. That’s where we are right now with AI.
Pretty much every pro AI person I’ve ever spoken to IRL tells me this exact same story
I created a simple script with the help of AI
And this is only scratching the surface
Basically, “I used AI for a boilerplate task. It gives me the vibe of being capable of much more” but then nobody can ever really get it to do much more
Bro it will bro, it WILL just 5 more years and 3 more trillion bro i promise
Yeah i used it for a boilerplate task, and then I did not have to do that task. I was able to scan in hundreds of documents, and at the end I had a fully indexed, properly file named, summarized, and searchable PDF library. Just doing one document at a time manually would have been a multi hour task for me, and in a few hours I was done, and it was good enough to good.
Sorry, to clarify, a boilerplate task is not the repetitive job that you are automating. It’s the code that’s doing it.
What I’m saying is that you (and many others) are incorrectly attributing (your paragraph about the benefits your finished program is conferring to you) to AI, because that happens to be the path you took to arrive there.
In reality, the reason it was able to produce functional code is because your problem was already solved and documented. A few years ago, instead of “asking AI”, you would have simply copied and pasted the boilerplate code from someone else’s project. In all likelihood it also would have been faster for you to have done so
Quick edit: Sorry again, just want to further clarify that when I say “boilerplate task” I’m referring to a type of programing problem that you solve with “boilerplate code”. Reading back the above I was kind of using them interchangeably which is not strictly accurate
If I could have found the code, and copied it yes. Using AI means I did not have to search for it, did not need to learn Python, and then did not need to do the tasks.
Please understand I hate 90% of all the AI bullshit being forced on us, and employers requiring AI use is insane. I am just saying that blanked hate is short sighted because it is useful in some cases, and the number of cases keep going up as we improve things.
I could also use a dictionary to check my spelling, but having spell check enabled is just faster.
Right, and what I’m saying is that the ‘usefulness’ that people claim to have discovered is totally nonsensical because these problems have been solved for decades
exactly. It’s people realizing what coding is for the first time because it’s being shoved in our faces. Before all this (bought and paid for propaganda) publicity, your average dummy had no idea what code was or did.
Basically, the non-tech people who know nothing about coding or how computers work are now amazed because they (think they) discovered what coding is because of the AI hype.
Yes but now they are accessible to anyone.
Your perspective is actually completely backwards on this
This process has always been accessible to everyone. You’d google basically the same words you typed into your prompt and it would bring you directly to the same block of code that everybody uses.
AI on the other hand is currently temporarily being made available for free or low cost because they are actively trying to create a cohort of users who impulsively “just reach for AI” as their first step to solving every problem.
In a few years you may find yourself praising how “accessible” it is because they occasionally run offers for a week of subscription time for $40 instead of the usual rate of $189.99/mo for entry level access. You may find yourself wondering how people ever lived without it
I understand that and good for you for finding a use case.
however a normal program could have done that exact same thing. It’s just miniscule easier to do it how you did. Probably not repeatable though, or if your model you used gets the plug pulled, you’re back to square 1.
Cant pull the plug, the model is local and running on my GPU. So I could break it if I wanted to but im not dependent on someone else.
This is a few lines of python and tesseract, or one of the fancier OCR libraries based on neural networks (so, AI but not in the way it’s used nowadays).
For sure but I don’t know Python. I can edit a script but I’m not proficient enough to make a new one.
OK so that’s what the boosters keep saying. However, every Task ive given it have very clear exact set of instructions, and when i comb through it, it still hallucinates. I can try talking to it like a 6 year old, and its going to forget what I told it the next day and hallucinate again.
Meanwhile I burned up a shit ton of power for 0 benefit and wasted my time. Worst case, someone else who is not detail oriented like myself is going to fuck up a lot of work.
However when i give it something I don’t know how to do or dont know an answer to, it does a great job and is so smart! Amazing how that works.
Your experience and feelings are totally valid, its still a shit show and in most cases a massive waste of power, water, brain power, and money. I am just saying that with the right training, and appropriate use, AI really can be effective NOW. One thing you can do is describe the task, and ask the AI to write a prompt to accomplish it. Then test that prompt in a new chat, and see if you get better results.
Most AIs will tell you how to improve your prompts.
I can see where you’re coming from and I get it.
I’ve done all that. I’ve followed it’s little bs “hey, to improve, tell me this!!”. Still gets it wrong.
It probably is fine for non-detail work and stuff no one really cares about (and if no one cares about it, maybe revisit what your job is). It’s definitely not worth the cost. I myself don’t enjoy telling a 5 year old the same thing 25 times (and then they forget because they saw a stick), which is exactly what using an LLM is akin to.
This is a wildly niche task. You also can’t trust the executive summaries or indexes will be accurate so you need to read both the scanned documents, the executive summaries and the indexs to review, markup and correct. Your response will be “but it’s good enough” but is it? You wouldn’t trust it to write a resume, but you would trust it to write an executive summary of a much longer text that you rely on to search for information?
Fair criticism, because it is “good enough” and it saved me so much time because I only needed detailed information on a few of the documents. But that is the point, it takes a repetitive bulk niche task and lets YOU quickly and simply create an automation that meets your needs.
Another example I have been using is having an AI read a resume, and do an ATS review based on a job posting. Then it generates a fit report, and makes suggestions on how the resume can be adjusted to better get past the ATS to an actual person. This is very useful in getting call backs for job applications. That has a real value.
How do you know it hasnt missed or misinterpreted key information from some of the documents you might have needed? The AI won’t complete the same task the same way on every iteration so the automation is only verifiable if you check every time.
Did you get any of the jobs?
Put in an application yesterday, got a call back today. Rest is up to me.
No i dont know if it missed something, but im not relying on the Exe summery for perfect detail, im using it to know which document is which and then using the index to go to points in the PDF.
Congrats on the call back!
That is an interesting paper for sure! Thanks for sharing it.
An idiot intern is better.
Well, 100% because the intern WILL eventually learn. That’s the entire difference. It won’t be about adjusting the prompt, or add yet another layer of “reasoning”, or wait for the next “version” with a different code name an .1% larger dataset. No, you’ll point to the intern they did a mistake, try not calling them an idiot, explain WHY it’s wrong, optionally explain how to do it right, THEN the next time they’ll avoid it or fix it after.
That’s the entire point of having an intern : initially they suck BUT as you train them, they don’t! Meanwhile an LLM, despite technical jargon hijacked by the marketing department, they don’t “learn” (from machine learning) or train (from “training dataset”) or have “neurons” (from “artificial neural networks”) rather it’s just statistics on the next most probable world, sounding right with 0 “reasoning”.
Had a person a few years back who would never ever learn.
In fact, a way I have expressed my opinion of LLM is that it is like working with that useless guy, except at least faster.
Based on my experience, the broader company is chock full of the never learn developers and I suppose I can see why they see value in the LLM, but either way their product sucks and no one likes them.
You’re so right .
And if the person sucks that bad, get rid of them
Yeah, but the same bad management that keeps thinking LLMs are magic are the same bad management that kept that guy around.
Every interaction that guy had where a senior tech ever dared to say he was useless ultimately landed the senior tech in hot water with management, as they claim “he says you aren’t providing what he needs to suceed, that he is very skilled and willing to work, but you never told him how or gave him access or (a million other excuses that were generally lies)”.
After a way too long career with us, he finally overplayed his hand by making the same old claims to the manager about no one giving him what he needed to work. Except he forgot that this time, the manager himself was the one who had been directing him and so he accidentally was accusing the manager of lying to himself.
Finally, the only person with credibility to the manager was on the receiving end of this guys grift.
Its all a grift in the end!
Thats why youll mostly see conservatives/Nazis in love with llms. It fits their propaganda agenda perfectly.
Have you tried using skills/workflows? You can improve its context over time.
I havent. Thats more time spent on a thing thats generally trash.
My work is too detail oriented with specific un documented use cases for it to work.
Its a glorified excel formula writer. Its OK at that.
