Reply to thread

Message: <blockquote data-quote="Mad as Fish" data-source="post: 139972" data-attributes="member: 396">Do those throwing money and shouting loudly about AI understand anything about how it works?Just to illustrate the hype and reality of AI here is some excitement from the industry -A significant recent development is the paper “<a href="https://arxiv.org/abs/2502.09992" target="_blank">Large Language Diffusion Models</a>” by Shen Nie and others, published on February 14, 2025, introducing LLaDA. This model is trained from scratch under a pre-training and supervised fine-tuning (SFT) paradigm, using a vanilla Transformer to predict masked tokens. LLaDA demonstrates strong scalability, outperforming auto-regressive model (ARM) baselines and being competitive with LLaMA3 8B in in-context learning and instruction-following abilities, such as multi-turn dialogue. Notably, it addresses the reversal curse, surpassing GPT-4o in a reversal poem completion task (<a href="https://arxiv.org/abs/2502.09992" target="_blank">Large Language Diffusion Models</a>).[MEDIA=medium]749033d1efb1[/MEDIA]<a href="https://medium.com/the-low-end-disruptor/what-is-diffusion-llm-and-why-it-matters-749033d1efb1" target="_blank">View: https://medium.com/the-low-end-disruptor/what-is-diffusion-llm-and-why-it-matters-749033d1efb1</a>And here is a more sobering critique from Richard Self on Linkedin - Oh, Dear! Yet more magical thinking about reducing hallucinations of LLMs by using diffusion models.As <a href="https://www.linkedin.com/in/ACoAABUngLkBSpSWvQlao30IcdOn1qF3oy-9INw" target="_blank">Denis O.</a> critiqued them, this won't happen. Diffusion models are at least as much subject to hallucinations as transformer based LLMs. Anyone who has used a GenAI image generator knows of the difficulty in getting it to faithfully follow the prompts."It seems likely that diffusion LLMs could challenge auto-regressive tech, offering new capabilities like improved reasoning and controllability, but their full impact is still emerging."As always, the proponents of these new technologies keep  talking about reasoning. It is just not going to happen and we know that image generators are highly uncontrollable.The idea of generating a whole block  of text in parallel is wrong from a semantics and linguistics perspective. Language has sequential dependencies. An argument is developed sequentially. A critical evaluation is developed sequentially, not in parallel."Andrej Karpathy and Andrew Ng, both renowned AI researchers, have enthusiastically welcomed the arrival of Inception Lab’s diffusion LLM."Surely, this is the  kiss of death, not an endorsement."This process begins with random noise which is then gradually refined and “denoised” into a coherent stream of tokens. This is analogous to how diffusion models generate images by starting with noise and iteratively removing it to reveal a clear image."It is not analogous, it is exactly the same, need to get the facts right."Parallel Processing: Diffusion models can process and generate text in parallel, potentially leading to significant speed advantages."As with image generators, the final output of tokens will be fast, just transferring out of the output buffer to the user, but after a long pause (not even thinking) to do all the denoising.We also know that diffusion models eat TFLOPs, just as much as or more than transformers.Show me one financial whizz kid who understands any of this!</blockquote>

Name

Verification: Does Doxxie know his real father.

Latest Threads

S
The Anti-Mass Migration movement has to be at least partly about race, it is indeed Ireland for the Irish
- Started by scolairebocht
- Yesterday at 9:10 AM
- Replies: 20
Scholairebochts Blog.
A Million Views.
- Started by Declan
- Saturday at 10:54 PM
- Replies: 12
Public Chat and Announcements
An Open Letter to SwordOfStZip
- Started by AN2
- Oct 11, 2025
- Replies: 12
Public Chat and Announcements
C
athletics
- Started by céline
- Oct 8, 2025
- Replies: 4
Public Chat and Announcements
S
The real agenda in this Presidential Election?
- Started by scolairebocht
- Oct 6, 2025
- Replies: 11
Scholairebochts Blog.

Popular Threads

Ukraine.
- Started by Declan
- Feb 21, 2022
- Replies: 15K
World at War
US Politics.
- Started by jpc
- Nov 7, 2022
- Replies: 6K
USA
Mass Migration to Ireland & Europe
- Started by Anderson
- Feb 26, 2023
- Replies: 5K
Nationalist Politics
C
🦠 Covid 19 Vaccine Thread 💉
- Started by Charlene
- Sep 14, 2021
- Replies: 3K
Health
General Chat in The Marcus Lounge.
- Started by Declan
- Dec 30, 2024
- Replies: 3K
Public Chat and Announcements
The Climate Change scam
- Started by Anderson
- Jul 29, 2022
- Replies: 2K
Climate Change