NewsNTech
The binding constraint on running large AI models has always been memory and compute capacity, the reason that work has defaulted to cloud infrastructure.
Apple's updated Mac Mini and Mac Studio, built around new chips and paired with new developer frameworks, are positioned to close that gap on the desktop.
Apple said that with the new frameworks and chips, developers can run and fine-tune large AI models directly on their Mac. The fine-tuning claim is the more consequential part.
Running inference on a pre-trained model is a forward pass; fine-tuning requires holding gradients and optimizer state alongside the model weights, a substantially larger memory footprint that consumer silicon has historically struggled to support.
Keep reading