AT2k Design BBS Message Area
Casually read the BBS message area using an easy to use interface. Messages are categorized exactly like they are on the BBS. You may post new messages or reply to existing messages!

You are not logged in. Login here for full access privileges.

Previous Message | Next Message | Back to Slashdot  <--  <--- Return to Home Page
   Local Database  Slashdot   [103 / 108] RSS
 From   To   Subject   Date/Time 
Message   VRSS    All   Diffusion + Coding = DiffuCode. How Apple Released a Weirdly Int   July 6, 2025
 9:40 AM  

Feed: Slashdot
Feed Link: https://slashdot.org/
---

Title: Diffusion + Coding = DiffuCode. How Apple Released a Weirdly
Interesting Coding Language Model

Link: https://developers.slashdot.org/story/25/07/0...

"Apple quietly dropped a new AI model on Hugging Face with an interesting
twist," writes 9to5Mac. "Instead of writing code like traditional LLMs
generate text (left to right, top to bottom), it can also write out of order,
and improve multiple chunks at once." "The result is faster code generation,
at a performance that rivals top open-source coding models." Traditionally,
most LLMs have been autoregressive. This means that when you ask them
something, they process your entire question, predict the first token of the
answer, reprocess the entire question with the first token, predict the
second token, and so on. This makes them generate text like most of us read:
left to right, top to bottom... An alternative to autoregressive models is
diffusion models, which have been more often used by image models like Stable
Diffusion. In a nutshell, the model starts with a fuzzy, noisy image, and it
iteratively removes the noise while keeping the user request in mind,
steering it towards something that looks more and more like what the user
requested... Lately, some large language models have looked to the diffusion
architecture to generate text, and the results have been pretty promising...
This behavior is especially useful for programming, where global structure
matters more than linear token prediction... [Apple] released an open-source
model called DiffuCode-7B-cpGRPO, that builds on top of a paper called
DiffuCoder: Understanding and Improving Masked Diffusion Models for Code
Generation, released just last month... [W]ith an extra training step called
coupled-GRPO, it learned to generate higher-quality code with fewer passes.
The result? Code that's faster to generate, globally coherent, and
competitive with some of the best open-source programming models out there.
Even more interestingly, Apple's model is built on top of Qwen2.5-7B, an open-
source foundation model from Alibaba. Alibaba first fine-tuned that model for
better code generation (as Qwen2.5-Coder-7B), then Apple took it and made its
own adjustments. They turned it into a new model with a diffusion-based
decoder, as described in the DiffuCoder paper, and then adjusted it again to
better follow instructions. Once that was done, they trained yet another
version of it using more than 20,000 carefully picked coding examples.
"Although DiffuCoder did better than many diffusion-based coding models (and
that was before the 4.4% bump from DiffuCoder-7B-cpGRPO), it still doesn't
quite reach the level of GPT-4 or Gemini Diffusion..." the article points
out. But "the bigger point is this: little by little, Apple has been laying
the groundwork for its generative AI efforts with some pretty interesting and
novel ideas."

Read more of this story at Slashdot.

---
VRSS v2.1.180528
  Show ANSI Codes | Hide BBCodes | Show Color Codes | Hide Encoding | Hide HTML Tags | Show Routing
Previous Message | Next Message | Back to Slashdot  <--  <--- Return to Home Page

VADV-PHP
Execution Time: 0.0196 seconds

If you experience any problems with this website or need help, contact the webmaster.
VADV-PHP Copyright © 2002-2025 Steve Winn, Aspect Technologies. All Rights Reserved.
Virtual Advanced Copyright © 1995-1997 Roland De Graaf.
v2.1.250224