Image credit: Microsoft
Microsoft researchers have created a new AI model that’s making waves, and I’m excited to tell you about it! They’ve developed what they call the largest 1-bit AI model yet, named BitNet b1.58 2B4T. It’s openly available under an MIT license, and the best part? It can run on CPUs, even on devices like Apple’s M2.
So, what’s a bitnet? It’s a type of compressed AI model made to work on lighter hardware. Normally, AI models have weights—values that shape how the model works—and these are adjusted to run on different machines. This process, called quantizing, lowers the number of bits needed to store those weights, so the model can work on chips with less memory and run faster. Bitnets take this further by shrinking weights to just three values: -1, 0, and 1. That makes them use way less memory and computing power than most models out there.
The Microsoft team says BitNet b1.58 2B4T has 2 billion parameters (think of parameters as similar to weights) and was trained on a huge dataset—4 trillion tokens, which is like 33 million books! They claim it performs better than other models of its size. In their tests, it beat out models like Meta’s Llama 3.2 1B, Google’s Gemma 3 1B, and Alibaba’s Qwen 2.5 1.5B on tasks like solving grade-school math problems and reasoning about everyday physical things.
What’s even more impressive is the speed—BitNet b1.58 2B4T can be up to twice as fast as other models its size while using much less memory. That’s a big win for efficiency!
There’s a small catch, though. To get that great performance, you need to use Microsoft’s special framework, bitnet.cpp, which only works with certain hardware right now. Unfortunately, it doesn’t support GPUs, which are super common in AI setups.
All in all, bitnets like this one look promising, especially for devices that don’t have a lot of power to spare. But compatibility might be a challenge for a while. What do you think—would you try running an AI model like this on your CPU?
