Open-Weight AI Models: Run AI Locally for More Privacy and Control

| 2 Min Read
Open-weight AI models let you run chatbots offline and keep prompts on your device. Learn what hardware, licenses and limits to check before downloading.

What Are Open-Weight AI Models?

Open-weight AI models put more control in your hands: you can download one and run it on your own computer, including without an internet connection. That can matter if you want to keep sensitive prompts on your device or avoid depending on a company’s hosted chatbot. The trade-off is that “open” doesn’t tell you everything about a model’s license, training data or limits.

Most people use a chatbot without thinking much about who controls the model behind it. But a provider can change how its chatbot responds, and sending a private question to a hosted service may mean sharing it with that service. Running a model locally offers a different arrangement: you keep a working copy and, if the setup stays on your computer, your prompts and files don’t go to the provider’s servers.

That added control is the real appeal. It comes with practical requirements: you’ll need software that can run the model and a computer with enough capacity. Open-weight options also exist for image and video generation, too.

Downloading a model doesn’t automatically grant unlimited rights to use it, or reveal how it was made. The distinction between open-weight and open-source matters, especially if you plan to modify a model or use it in a product. Here’s how the technology works and what to check before you install one.

What open-weight models are and how they work

A chatbot’s model is the software that produces its answers. During training, it adjusts billions of numerical values, known as neural network weights, to learn patterns. When you submit a prompt, the model uses those learned patterns to predict what comes next, generating a response piece by piece.

CNET AI Atlas badge; click to see more

Companies behind closed models, including ChatGPT and Claude, keep those weights private. An open-weight model makes its trained weights available to the public, so people can download them and run a local copy with compatible software.

That distribution can make a model hard to pull back once it’s out. “Whenever you open-weight a model, all the weights are all over the place on the internet, like on Hugging Face and other servers,” Pierre Stock, VP of science at AI company Mistral, tells CNET. “So it’s virtually impossible to cut access to this model.”

You can often run a free model with a free app, without a subscription or per-use fees. And you don’t necessarily need to code: LM Studio, Ollama and Jan offer desktop interfaces for downloading and running models on a Mac or PC. Once installed, you can use the model offline, provided your machine can handle it.

Some models can also be adapted through further training, if their licenses allow it. You don’t have to build a model from scratch to customize one.

Open-weight doesn’t always mean open-source

The terms get used as if they mean the same thing, but publishing model weights alone doesn’t meet the Open Source Initiative’s definition of open-source AI. That definition also calls for the code used to train and run the model, enough information about its training data for a skilled person to recreate a substantially similar system, and terms that permit people to use, study, modify and share it.

The OSI doesn’t require developers to release the training data itself. The Free Software Foundation argues that users should also be free to access and change that data. So even the organizations defining openness differ on what it should include.

Licenses add another wrinkle. A model may cost nothing to download but still restrict commercial use, which could rule it out for an app you intend to sell. That restriction would also prevent it from meeting the OSI definition. Some open-source licenses impose a different condition: if you share a modified version, you may have to keep the same terms.

James Martin/CNET

Where to find open-weight models

The open-weight field is already global, though Chinese models currently top Artificial Analysis’s intelligence rankings. DeepSeek may be the best-known name, but Moonshot AI’s Kimi K3 and Alibaba’s Qwen3.8 family also make their weights available to download. There’s a wider set of choices beyond those leaders. US developers are in the mix, too. OpenAI publishes gpt-oss apart from ChatGPT, and Google offers Gemma. Meta’s lineup includes Muse Glimmer and Muse Spark Open, alongside its Llama models. Nvidia’s Nemotron is another option. You don’t need to choose a household name. Smaller developers are releasing models as well: US startup Reflection has made Beam downloadable, and Thinking Machines Lab offers Inkling. France’s Mistral AI adds a European presence with downloadable Mistral Large 3, the smaller Ministral 3 family, and Mistral Large 4, known as Le Chonk. The name “open weight” doesn’t tell you whether a model will run on your device. Some need server hardware; compact versions can fit on a computer or even a phone.

Openness brings trade-offs

Downloading a model doesn’t make it reliable by default. It can still hallucinate offline, and its answers can reflect biases in its training. Developers may release weights without sharing training data, which leaves researchers unable to fully inspect the material behind a model. Performance also depends on the model and task; openness by itself says little about whether it will outperform a closed service. There’s a harder question around safety. Users can alter a model, including by weakening its safeguards. “Guardrails are necessary to open-weighting a model,” Stock says. Asked whether those protections can hold after someone downloads the model, Stock says, “With a lot of work, you can partially overcome the guardrails,” likening that effort to fine-tuning a model yourself. Once a copy is out in the world, its developer has little control over how it’s used. A provider can suspend access to an online service, but it can’t reach into every downloaded copy. Nor will a safer new release automatically replace older versions. And yet, that same access gives outside researchers room to test models, investigate reported faults and try fixes without waiting for a developer. Microsoft and other companies made that case in an open letter responding to reports of possible US restrictions on Chinese models. Their argument: keeping open-weight tools available can help security teams defend against attackers who use AI. “When you open-weight a model, you’re essentially enabling the entire ecosystem to progress together,” Stock tells CNET.
Google Gemma logo on large screen on stage at Google I/O 2024

Google/CNET

Is running one yourself worthwhile?

For everyday work, an open-weight model can draft email, summarize documents, inspect files stored on your device and help write code. The catch is that local speed and the length of documents it can handle depend on the model and your computer’s memory and processing power. Bigger models tend to need more capable hardware, often including a strong GPU. A Mac with 16GB or 36GB of RAM may handle them more comfortably; some smaller models can run on an 8GB Mac. If your computer already has the capacity, trying a local model may make sense. Buying a new machine just for this purpose calls for a more prosaic calculation: compare its price with a cloud AI subscription that costs $20 a month. There’s also a middle ground. DeepSeek and Kimi, for example, let people use some open-weight models through their websites. That avoids installation and local hardware demands, but your prompts go to the provider. If you only want occasional help with an email and are comfortable using a cloud service, ChatGPT, Gemini and Claude all have free plans. Avoiding a paid subscription doesn’t require managing a model yourself. The stronger case for local use is control: sensitive documents can stay on your device, and an assistant can work without an internet connection. You also keep access to a downloaded model even if its maker retires the hosted version. “You don’t want to live in a world in which someone can cut off access to your model,” Stock says. Smaller hardware could broaden that appeal. Google’s Gemma range includes compact models for phones and laptops, and Mistral offers models sized or specialized for particular devices and tasks. Stock says some audio models “can fit literally on the smartwatch.” That possibility is compelling, though what these models can do on small devices will still depend on their capabilities and the hardware available.
Source: Barbara Pazur · www.cnet.com

Comments

Please sign in to comment.