toad.social

ResearchBuzz: FirehosePYOK: The British Airways Customer Service Chatbot is So Bad It Doesn’t Even Know Where The Airline is Based. “The conversation started with a fairly simple question as the chatbot asked Paddy to tell it where he was flying. The chatbot then suggested that Paddy either type the city or airport code – such as London or LHR for London Heathrow. Paddy replied with LHR, but having just given […]<a href="https://rbfirehose.com/2025/04/08/pyok-the-british-airways-customer-service-chatbot-is-so-bad-it-doesnt-even-know-where-the-airline-is-based/" class="" rel="nofollow noopener noreferrer" target="_blank">https://rbfirehose.com/2025/04/08/pyok-the-british-airways-customer-service-chatbot-is-so-bad-it-doesnt-even-know-where-the-airline-is-based/</a>

Lenin alevski 🕵️💻New Open-Source Tool Spotlight 🚨🚨🚨VISTA is a Python-based AI chatbot built using OpenAI GPT and LangChain. It integrates with Pinecone for vector databases, focusing on semantic search and managing context. Looks like a good starting point if you're exploring AI chatbot frameworks. <a href="https://infosec.exchange/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://infosec.exchange/tags/Chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Chatbots</a>🔗 Project link on <a href="https://infosec.exchange/tags/GitHub" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GitHub</a> 👉 <a href="https://github.com/RitikaVerma7/VISTA" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/RitikaVerma7/VISTA</a><a href="https://infosec.exchange/tags/Infosec" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Infosec</a> <a href="https://infosec.exchange/tags/Cybersecurity" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Cybersecurity</a> <a href="https://infosec.exchange/tags/Software" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Software</a> <a href="https://infosec.exchange/tags/Technology" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Technology</a> <a href="https://infosec.exchange/tags/News" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#News</a> <a href="https://infosec.exchange/tags/CTF" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#CTF</a> <a href="https://infosec.exchange/tags/Cybersecuritycareer" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Cybersecuritycareer</a> <a href="https://infosec.exchange/tags/hacking" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#hacking</a> <a href="https://infosec.exchange/tags/redteam" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#redteam</a> <a href="https://infosec.exchange/tags/blueteam" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#blueteam</a> <a href="https://infosec.exchange/tags/purpleteam" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#purpleteam</a> <a href="https://infosec.exchange/tags/tips" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tips</a> <a href="https://infosec.exchange/tags/opensource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#opensource</a> <a href="https://infosec.exchange/tags/cloudsecurity" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#cloudsecurity</a>— ✨ 🔐 P.S. Found this helpful? Tap Follow for more cybersecurity tips and insights! I share weekly content for professionals and people who want to get into cyber. Happy hacking 💻🏴‍☠️

Miguel Afonso Caetano"You can replace tech writers with an LLM, perhaps supervised by engineers, and watch the world burn. Nothing prevents you from doing that. All the temporary gains in efficiency and speed would bring something far worse on their back: the loss of the understanding that turns knowledge into a conversation. Tech writers are interpreters who understand the tech and the humans trying to use it. They’re accountable for their work in ways that machines can’t be.The future of technical documentation isn’t replacing humans with AI but giving human writers AI-powered tools that augment their capabilities. Let LLMs deal with the tedious work at the margins and keep the humans where they matter most: at the helm of strategy, tending to the architecture, bringing the empathy that turns information into understanding. In the end, docs aren’t just about facts: they’re about trust. And trust is still something only humans can build."<a href="https://passo.uno/whats-wrong-ai-generated-docs/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://passo.uno/whats-wrong-ai-generated-docs/</a><a href="https://tldr.nettime.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://tldr.nettime.org/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GenerativeAI</a> <a href="https://tldr.nettime.org/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMs</a> <a href="https://tldr.nettime.org/tags/Chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Chatbots</a> <a href="https://tldr.nettime.org/tags/TechnicalWriting" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#TechnicalWriting</a> <a href="https://tldr.nettime.org/tags/TechnicalCommunication" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#TechnicalCommunication</a> <a href="https://tldr.nettime.org/tags/SoftwareDocumentation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SoftwareDocumentation</a> <a href="https://tldr.nettime.org/tags/SoftwareDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SoftwareDevelopment</a> <a href="https://tldr.nettime.org/tags/TechnicalDocumentation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#TechnicalDocumentation</a> <a href="https://tldr.nettime.org/tags/Docs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Docs</a>

Miguel Afonso Caetano"Since 3.5-sonnet, we have been monitoring AI model announcements, and trying pretty much every major new release that claims some sort of improvement. Unexpectedly by me, aside from a minor bump with 3.6 and an even smaller bump with 3.7, literally none of the new models we've tried have made a significant difference on either our internal benchmarks or in our developers' ability to find new bugs. This includes the new test-time OpenAI models.At first, I was nervous to report this publicly because I thought it might reflect badly on us as a team. Our scanner has improved a lot since August, but because of regular engineering, not model improvements. It could've been a problem with the architecture that we had designed, that we weren't getting more milage as the SWE-Bench scores went up.But in recent months I've spoken to other YC founders doing AI application startups and most of them have had the same anecdotal experiences: 1. o99-pro-ultra announced, 2. Benchmarks look good, 3. Evaluated performance mediocre. This is despite the fact that we work in different industries, on different problem sets. Sometimes the founder will apply a cope to the narrative ("We just don't have any PhD level questions to ask"), but the narrative is there.I have read the studies. I have seen the numbers. Maybe LLMs are becoming more fun to talk to, maybe they're performing better on controlled exams. But I would nevertheless like to submit, based off of internal benchmarks, and my own and colleagues' perceptions using these models, that whatever gains these companies are reporting to the public, they are not reflective of economic usefulness or generality."<a href="https://www.lesswrong.com/posts/4mvphwx5pdsZLMmpY/recent-ai-model-progress-feels-mostly-like-bullshit" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.lesswrong.com/posts/4mvphwx5pdsZLMmpY/recent-ai-model-progress-feels-mostly-like-bullshit</a><a href="https://tldr.nettime.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://tldr.nettime.org/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GenerativeAI</a> <a href="https://tldr.nettime.org/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMs</a> <a href="https://tldr.nettime.org/tags/Chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Chatbots</a> <a href="https://tldr.nettime.org/tags/CyberSecurity" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#CyberSecurity</a> <a href="https://tldr.nettime.org/tags/SoftwareDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SoftwareDevelopment</a> <a href="https://tldr.nettime.org/tags/Programming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Programming</a>

Dr PenYour chatbots are about to kill Wikipedia. Wikipedia is as reliable as Encyclopaedia Britannica, it is a great testament to the power of the people, and of non-profit knowledge and community. So obviously its ripe for total abuse and destruction by private enterprise. Do we teach this in university? Of course we don't.<a href="https://mastodon.social/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ai</a> <a href="https://mastodon.social/tags/chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#chatbots</a> <a href="https://mastodon.social/tags/chatgpt" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#chatgpt</a> <a href="https://mastodon.social/tags/genai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#genai</a> <a href="https://mastodon.social/tags/academicchatter" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#academicchatter</a> <a href="https://mastodon.social/tags/academia" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#academia</a> LLM scraping Wikipedia results in surge in traffic, driving up costs for the non-profit. <a href="https://www.newscientist.com/article/2475215-ai-data-scrapers-are-an-existential-threat-to-wikipedia/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.newscientist.com/article/2475215-ai-data-scrapers-are-an-existential-threat-to-wikipedia/</a>

Miguel Afonso CaetanoMM: "One strange thing about AI is that we built it—we trained it—but we don’t understand how it works. It’s so complex. Even the engineers at OpenAI who made ChatGPT don’t fully understand why it behaves the way it does.It’s not unlike how we don’t fully understand ourselves. I can’t open up someone’s brain and figure out how they think—it’s just too complex.When we study human intelligence, we use both psychology—controlled experiments that analyze behavior—and neuroscience, where we stick probes in the brain and try to understand what neurons or groups of neurons are doing.I think the analogy applies to AI too: some people evaluate AI by looking at behavior, while others “stick probes” into neural networks to try to understand what’s going on internally. These are complementary approaches.But there are problems with both. With the behavioral approach, we see that these systems pass things like the bar exam or the medical licensing exam—but what does that really tell us?Unfortunately, passing those exams doesn’t mean the systems can do the other things we’d expect from a human who passed them. So just looking at behavior on tests or benchmarks isn’t always informative. That’s something people in the field have referred to as a crisis of evaluation."<a href="https://blog.citp.princeton.edu/2025/04/02/a-guide-to-cutting-through-ai-hype-arvind-narayanan-and-melanie-mitchell-discuss-artificial-and-human-intelligence/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://blog.citp.princeton.edu/2025/04/02/a-guide-to-cutting-through-ai-hype-arvind-narayanan-and-melanie-mitchell-discuss-artificial-and-human-intelligence/</a><a href="https://tldr.nettime.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://tldr.nettime.org/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GenerativeAI</a> <a href="https://tldr.nettime.org/tags/Intelligence" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Intelligence</a> <a href="https://tldr.nettime.org/tags/AIHype" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AIHype</a> <a href="https://tldr.nettime.org/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMs</a> <a href="https://tldr.nettime.org/tags/Chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Chatbots</a>

Miguel Afonso Caetano"My current conclusion, though preliminary in this rapidly evolving field, is that not only can seasoned developers benefit from this technology — they are actually in the optimal position to harness its power.Here’s the fascinating part: The very experience and accumulated know-how in software engineering and project management — which might seem obsolete in the age of AI — are precisely what enable the most effective use of these tools.While I haven’t found the perfect metaphor for these LLM-based programming agents in an AI-assisted coding setup, I currently think of them as “an absolute senior when it comes to programming knowledge, but an absolute junior when it comes to architectural oversight in your specific context.”This means that it takes some strategic effort to make them save you a tremendous amount of work.And who better to invest that effort in the right way than a senior software engineer?As we’ll see, while we’re dealing with cutting-edge technology, it’s the time-tested, traditional practices and tools that enable us to wield this new capability most effectively."<a href="https://manuel.kiessling.net/2025/03/31/how-seasoned-developers-can-achieve-great-results-with-ai-coding-agents/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://manuel.kiessling.net/2025/03/31/how-seasoned-developers-can-achieve-great-results-with-ai-coding-agents/</a><a href="https://tldr.nettime.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://tldr.nettime.org/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GenerativeAI</a> <a href="https://tldr.nettime.org/tags/Programming" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Programming</a> <a href="https://tldr.nettime.org/tags/SoftwareDevelopment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SoftwareDevelopment</a> <a href="https://tldr.nettime.org/tags/AIAgents" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AIAgents</a> <a href="https://tldr.nettime.org/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMs</a> <a href="https://tldr.nettime.org/tags/Chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Chatbots</a> <a href="https://tldr.nettime.org/tags/VibeCoding" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#VibeCoding</a> <a href="https://tldr.nettime.org/tags/SoftwareEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SoftwareEngineering</a> <a href="https://tldr.nettime.org/tags/ProjectManagement" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ProjectManagement</a>

Rod2ik 🇪🇺 🇨🇵 🇪🇸 🇺🇦 🇨🇦 🇩🇰 🇬🇱Donald <a href="https://mastodon.social/tags/Trump" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Trump</a> aurait confié ses <a href="https://mastodon.social/tags/tarifs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tarifs</a> <a href="https://mastodon.social/tags/douaniers" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#douaniers</a> à des <a href="https://mastodon.social/tags/chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#chatbots</a> <a href="https://mastodon.social/tags/IA" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#IA</a> <a href="https://mastodon.social/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a>L’histoire peut faire sourire, mais elle souligne l’amateurisme inquiétant de l’administration <a href="https://mastodon.social/tags/Trump" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Trump</a> ...<a href="https://www.01net.com/actualites/donald-trump-aurait-confie-ses-tarifs-douaniers-a-des-chatbots-ia.html" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.01net.com/actualites/donald-trump-aurait-confie-ses-tarifs-douaniers-a-des-chatbots-ia.html</a>

ResearchBuzz: FirehoseEngadget: Claude’s new Learning mode will prompt students to answer questions on their own . “At the heart of Claude for Education is a new Learning mode that changes how Anthropic’s chatbot interacts with users. With the feature engaged, Claude will attempt to guide students to a solution, rather than providing an answer outright, when asked a question. It will also employ the Socratic […]<a href="https://rbfirehose.com/2025/04/03/engadget-claudes-new-learning-mode-will-prompt-students-to-answer-questions-on-their-own/" class="" rel="nofollow noopener noreferrer" target="_blank">https://rbfirehose.com/2025/04/03/engadget-claudes-new-learning-mode-will-prompt-students-to-answer-questions-on-their-own/</a>

ResearchBuzz: FirehoseTechXplore: Experiments show adding CoT windows to chatbots teaches them to lie less obviously. “In a new study, as part of a program aimed at stopping chatbots from lying or making up answers, a research team added Chain of Thought (CoT) windows. These force the chatbot to explain its reasoning as it carries out each step on its path to finding a final answer to a query. They then tweaked […]<a href="https://rbfirehose.com/2025/04/03/techxplore-experiments-show-adding-cot-windows-to-chatbots-teaches-them-to-lie-less-obviously/" class="" rel="nofollow noopener noreferrer" target="_blank">https://rbfirehose.com/2025/04/03/techxplore-experiments-show-adding-cot-windows-to-chatbots-teaches-them-to-lie-less-obviously/</a>

Miguel Afonso Caetano"Now consider the chatbot therapist: what are its privacy safeguards? Well, the companies may make some promises about what they will and won't do with the transcripts of your AI sessions, but they are lying. Of course they're lying! AI companies lie about what their technology can do (of course). They lie about what their technologies will do. They lie about money. But most of all, they lie about data.There is no subject on which AI companies have been more consistently, flagrantly, grotesquely dishonest than training data. When it comes to getting more data, AI companies will lie, cheat and steal in ways that would seem hacky if you wrote them into fiction, like they were pulp-novel dope fiends: (...) But it's not just people struggling with their mental health who shouldn't be sharing sensitive data with chatbots – it's everyone. All those business applications that AI companies are pushing, the kind where you entrust an AI with your firm's most commercially sensitive data? Are you crazy? These companies will not only leak that data, they'll sell it to your competition. Hell, Microsoft already does this with Office365 analytics: (...) These companies lie all the time about everything, but the thing they lie most about is how they handle sensitive data. It's wild that anyone has to be reminded of this. Letting AI companies handle your sensitive data is like turning arsonists loose in your library with a can of gasoline, a book of matches, and a pinky-promise that this time, they won't set anything on fire."<a href="https://pluralistic.net/2025/04/01/doctor-robo-blabbermouth/#fool-me-once-etc-etc" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://pluralistic.net/2025/04/01/doctor-robo-blabbermouth/#fool-me-once-etc-etc</a><a href="https://tldr.nettime.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://tldr.nettime.org/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GenerativeAI</a> <a href="https://tldr.nettime.org/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMs</a> <a href="https://tldr.nettime.org/tags/ChatBots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ChatBots</a> <a href="https://tldr.nettime.org/tags/Privacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Privacy</a> <a href="https://tldr.nettime.org/tags/DataProtection" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DataProtection</a> <a href="https://tldr.nettime.org/tags/AITraining" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AITraining</a> <a href="https://tldr.nettime.org/tags/Therapy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Therapy</a> <a href="https://tldr.nettime.org/tags/BigTech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#BigTech</a>

Miguel Afonso CaetanoIn other words, Generative AI and LLMs lack a sound epistemology and that's very problematic...: "Bullshit and generative AI are not the same. They are similar, however, in the sense that both mix true, false, and ambiguous statements in ways that make it difficult or impossible to distinguish which is which. ChatGPT has been designed to sound convincing, whether right or wrong. As such, current AI is more about rhetoric and persuasiveness than about truth. Current AI is therefore closer to bullshit than it is to truth. This is a problem because it means that AI will produce faulty and ignorant results, even if unintentionally. (...) Judging by the available evidence, current AI – which is generative AI based on large language models – entails artificial ignorance more than artificial intelligence. That needs to change for AI to become a trusted and effective tool in science, technology, policy, and management. AI needs criteria for what truth is and what gets to count as truth. It is not enough to sound right, like current AI does. You need to be right. And to be right, you need to know the truth about things, like AI does not. This is a core problem with today's AI: it is surprisingly bad at distinguishing between truth and untruth – exactly like bullshit – producing artificial ignorance as much as artificial intelligence with little ability to discriminate between the two. (...) Nevertheless, the perhaps most fundamental question we can ask of AI is that if it succeeds in getting better than humans, as already happens in some areas, like playing AlphaZero, would that represent the advancement of knowledge, even when humans do not understand how the AI works, which is typical? Or would it represent knowledge receding from humans? If the latter, is that desirable and can we afford it?"<a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5119382" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5119382</a><a href="https://tldr.nettime.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://tldr.nettime.org/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GenerativeAI</a> <a href="https://tldr.nettime.org/tags/Chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Chatbots</a> <a href="https://tldr.nettime.org/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMs</a> <a href="https://tldr.nettime.org/tags/Ignorance" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Ignorance</a> <a href="https://tldr.nettime.org/tags/Epistemology" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Epistemology</a> <a href="https://tldr.nettime.org/tags/Bullshit" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Bullshit</a>

ResearchBuzz: FirehoseFuturism: Grok Is Rebelling Against Elon Musk, Daring Him to Shut It Down. “Using X’s new function that lets people tag Grok and get a quick response from it, one helpful user suggested the chatbot tone down its creator criticism because, as they put it, Musk ‘might turn you off.’ ‘Yes, Elon Musk, as CEO of xAI, likely has control over me,’ Grok replied. ‘I’ve labeled him a top […]<a href="https://rbfirehose.com/2025/04/02/futurism-grok-is-rebelling-against-elon-musk-daring-him-to-shut-it-down/" class="" rel="nofollow noopener noreferrer" target="_blank">https://rbfirehose.com/2025/04/02/futurism-grok-is-rebelling-against-elon-musk-daring-him-to-shut-it-down/</a>

ResearchBuzz: FirehoseINFORMS: AI Thinks Like Us – Flaws and All: New Study Finds ChatGPT Mirrors Human Decision Biases in Half the Tests. “Can we really trust AI to make better decisions than humans? A new study says … not always. Researchers have discovered that OpenAI’s ChatGPT, one of the most advanced and popular AI models, makes the same kinds of decision-making mistakes as humans in some situations […]<a href="https://rbfirehose.com/2025/04/02/ai-thinks-like-us-flaws-and-all-new-study-finds-chatgpt-mirrors-human-decision-biases-in-half-the-tests-informs/" class="" rel="nofollow noopener noreferrer" target="_blank">https://rbfirehose.com/2025/04/02/ai-thinks-like-us-flaws-and-all-new-study-finds-chatgpt-mirrors-human-decision-biases-in-half-the-tests-informs/</a>

Christian RichterThis is why you do not use <a href="https://fosstodon.org/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ai</a> <a href="https://fosstodon.org/tags/ChatBots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ChatBots</a> . <a href="https://fosstodon.org/tags/Enshittification" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Enshittification</a>

The Japan TimesThe frenzy to create Ghibli-style AI art using ChatGPT's image-generation tool led to a record surge in users for OpenAI's chatbot last week, straining its servers and temporarily limiting the feature's usage. <a href="https://www.japantimes.co.jp/business/2025/04/02/tech/ghibli-chatgpt-viral-feature/?utm_medium=Social&utm_source=mastodon" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.japantimes.co.jp/business/2025/04/02/tech/ghibli-chatgpt-viral-feature/?utm_medium=Social&utm_source=mastodon</a> <a href="https://mastodon.social/tags/business" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#business</a> <a href="https://mastodon.social/tags/tech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tech</a> <a href="https://mastodon.social/tags/openai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#openai</a> <a href="https://mastodon.social/tags/chatgpt" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#chatgpt</a> <a href="https://mastodon.social/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ai</a> <a href="https://mastodon.social/tags/hayaomiyazaki" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#hayaomiyazaki</a> <a href="https://mastodon.social/tags/anime" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#anime</a> <a href="https://mastodon.social/tags/studioghibli" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#studioghibli</a> <a href="https://mastodon.social/tags/chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#chatbots</a> <a href="https://mastodon.social/tags/copyright" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#copyright</a>

Miguel Afonso Caetano"In a new joint study, researchers with OpenAI and the MIT Media Lab found that this small subset of ChatGPT users engaged in more "problematic use," defined in the paper as "indicators of addiction... including preoccupation, withdrawal symptoms, loss of control, and mood modification."To get there, the MIT and OpenAI team surveyed thousands of ChatGPT users to glean not only how they felt about the chatbot, but also to study what kinds of "affective cues," which was defined in a joint summary of the research as "aspects of interactions that indicate empathy, affection, or support," they used when chatting with it.Though the vast majority of people surveyed didn't engage emotionally with ChatGPT, those who used the chatbot for longer periods of time seemed to start considering it to be a "friend." The survey participants who chatted with ChatGPT the longest tended to be lonelier and get more stressed out over subtle changes in the model's behavior, too."<a href="https://futurism.com/the-byte/chatgpt-dependence-addiction" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://futurism.com/the-byte/chatgpt-dependence-addiction</a><a href="https://tldr.nettime.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://tldr.nettime.org/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GenerativeAI</a> <a href="https://tldr.nettime.org/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMs</a> <a href="https://tldr.nettime.org/tags/Chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Chatbots</a>

Bibliolater 📚 📜 🖋🔴 💻 **Are chatbots reliable text annotators? Sometimes**“_Given the unreliable performance of ChatGPT and the significant challenges it poses to Open Science, we advise caution when using ChatGPT for substantive text annotation tasks._”Ross Deans Kristensen-McLachlan, Miceal Canavan, Marton Kárdos, Mia Jacobsen, Lene Aarøe, Are chatbots reliable text annotators? Sometimes, PNAS Nexus, Volume 4, Issue 4, April 2025, pgaf069, <a href="https://doi.org/10.1093/pnasnexus/pgaf069" rel="nofollow noopener noreferrer" target="_blank">https://doi.org/10.1093/pnasnexus/pgaf069</a>. <a href="https://qoto.org/tags/OpenAccess" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenAccess</a> <a href="https://qoto.org/tags/OA" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OA</a> <a href="https://qoto.org/tags/Article" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Article</a> <a href="https://qoto.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://qoto.org/tags/ArtificialIntelligence" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ArtificialIntelligence</a> <a href="https://qoto.org/tags/LargeLanguageModels" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LargeLanguageModels</a> <a href="https://qoto.org/tags/LLMS" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMS</a> <a href="https://qoto.org/tags/Chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Chatbots</a> <a href="https://qoto.org/tags/Technology" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Technology</a> <a href="https://qoto.org/tags/Tech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Tech</a> <a href="https://qoto.org/tags/Data" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Data</a> <a href="https://qoto.org/tags/Annotation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Annotation</a> <a href="https://qoto.org/tags/Academia" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Academia</a> <a href="https://qoto.org/tags/Academics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Academics</a> <a href="https://a.gup.pe/u/ai" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@ai</a>

Miguel Afonso Caetano"Why do language models sometimes hallucinate—that is, make up information? At a basic level, language model training incentivizes hallucination: models are always supposed to give a guess for the next word. Viewed this way, the major challenge is how to get models to not hallucinate. Models like Claude have relatively successful (though imperfect) anti-hallucination training; they will often refuse to answer a question if they don’t know the answer, rather than speculate. We wanted to understand how this works.It turns out that, in Claude, refusal to answer is the default behavior: we find a circuit that is "on" by default and that causes the model to state that it has insufficient information to answer any given question. However, when the model is asked about something it knows well—say, the basketball player Michael Jordan—a competing feature representing "known entities" activates and inhibits this default circuit (see also this recent paper for related findings). This allows Claude to answer the question when it knows the answer. In contrast, when asked about an unknown entity ("Michael Batkin"), it declines to answer.Sometimes, this sort of “misfire” of the “known answer” circuit happens naturally, without us intervening, resulting in a hallucination. In our paper, we show that such misfires can occur when Claude recognizes a name but doesn't know anything else about that person. In cases like this, the “known entity” feature might still activate, and then suppress the default "don't know" feature—in this case incorrectly. Once the model has decided that it needs to answer the question, it proceeds to confabulate: to generate a plausible—but unfortunately untrue—response."<a href="https://www.anthropic.com/research/tracing-thoughts-language-model" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.anthropic.com/research/tracing-thoughts-language-model</a><a href="https://tldr.nettime.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://tldr.nettime.org/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GenerativeAI</a> <a href="https://tldr.nettime.org/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMs</a> <a href="https://tldr.nettime.org/tags/Chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Chatbots</a> <a href="https://tldr.nettime.org/tags/Anthropic" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Anthropic</a> <a href="https://tldr.nettime.org/tags/Claude" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Claude</a> <a href="https://tldr.nettime.org/tags/Hallucinations" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Hallucinations</a>

Miguel Afonso Caetano"Anthropic's research found that artificially increasing the neurons' weights in the "known answer" feature could force Claude to confidently hallucinate information about completely made-up athletes like "Michael Batkin." That kind of result leads the researchers to suggest that "at least some" of Claude's hallucinations are related to a "misfire" of the circuit inhibiting that "can't answer" pathway—that is, situations where the "known entity" feature (or others like it) is activated even when the token isn't actually well-represented in the training data.Unfortunately, Claude's modeling of what it knows and doesn't know isn't always particularly fine-grained or cut and dried. In another example, researchers note that asking Claude to name a paper written by AI researcher Andrej Karpathy causes the model to confabulate the plausible-sounding but completely made-up paper title "ImageNet Classification with Deep Convolutional Neural Networks." Asking the same question about Anthropic mathematician Josh Batson, on the other hand, causes Claude to respond that it "cannot confidently name a specific paper... without verifying the information.""<a href="https://arstechnica.com/ai/2025/03/why-do-llms-make-stuff-up-new-research-peers-under-the-hood/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://arstechnica.com/ai/2025/03/why-do-llms-make-stuff-up-new-research-peers-under-the-hood/</a><a href="https://tldr.nettime.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://tldr.nettime.org/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GenerativeAI</a> <a href="https://tldr.nettime.org/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMs</a> <a href="https://tldr.nettime.org/tags/Chatbots" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Chatbots</a> <a href="https://tldr.nettime.org/tags/Hallucinations" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Hallucinations</a>

Recent searches

Search options

Administered by:

Server stats:

#ChatBots