---
title: "How to prompt small language models"
description: "Small models punish vague prompts. Here is how to get useful answers from a 1B to 8B model running on your phone."
pubDate: 2026-07-02
author: "Offeline"
---


Most prompt advice is written for **big cloud models**. Those can handle a messy prompt and still figure out what you meant. **Small models** can't. A 3B model on your phone needs you to be clear, and that changes how you write.

## Put the task first

Put the task in the first sentence instead of the background. "Rewrite this email to be shorter" works better than explaining why the email exists and who reads it.

A big model will find your point in the noise, but a small model grabs the first thing it sees and runs with it, so the first thing it reads should be the instruction.

## Keep the prompt short

Small models have small **context windows** and get lost when you fill them up, so every sentence you paste in should matter.

If you want a summary, give it:

- the text
- the word limit

That's enough. You don't need disclaimers or a polite setup, because the model was never offended anyway.

## One task at a time

This is the biggest difference from cloud models. Ask a big model to translate a paragraph, review the translation, and suggest a better ending, and it does all three.

A small model does the first one, messes up the second, and forgets the third. You get much better results by asking once, taking the answer, and only then moving to the next request. Those follow-ups are free since the model runs on your phone.

## Name the format

Tell the model the shape you want:

- "Answer in three bullet points"
- "Write a short paragraph under 80 words"
- "Give me a table with two columns"

"Keep it brief and organized" is too vague. Small models follow clear format rules well and guess badly.

## Skip long roleplay

Telling a big model to act as a veteran editor can change the output. On a small model, a long persona just eats context.

A short one is fine, like **"you are a strict editor."** A paragraph of backstory is wasted space.

## Use temperature when you can

If your app has a **temperature** setting, use it:

- **Lower** for factual work and clean rewriting
- **Higher** for brainstorming

That one setting does more than most prompt tweaks.

## The simple rule

Small models reward clear prompts and punish extra words. Cut it down to **instruction**, **material**, and **format**, and a model that fits on your phone will do more than you expect.