Privacy-sensitive local AI
Keep prompts, files, screenshots, and intermediate agent context on controlled hardware instead of sending every step to a hosted API.
Independent tool overview
Bonsai 27B at a glance
Bonsai 27B is PrismML's family of aggressively compressed, multimodal Qwen3.6 27B models built for local reasoning, coding, vision, and tool-using workflows.
Visit the official Bonsai 27B site ↗
- Developer
- PrismML
- Base model
- Qwen3.6 27B
- Parameter class
- 27.8B
- Variants
- 1-bit binary and ternary
- Claimed weight footprints
- 3.9 GB and 5.9 GB
- Maximum context
- 262,144 tokens
- Modalities
- Text and images
- License
- Apache 2.0
- Primary formats
- GGUF and MLX
- Last reviewed
- August 29, 2026
Overview
What Bonsai 27B is
Bonsai 27B is a pair of compressed, open-weight multimodal models from PrismML based on Qwen3.6 27B. The 1-bit version prioritizes the smallest footprint, while Ternary Bonsai 27B uses a larger ternary representation to retain more of the original model's quality.
PrismML lists a 3.9 GB weight footprint for the 1-bit variant and 5.9 GB for the ternary variant, compared with roughly 54 GB for a 16-bit 27B model. Actual downloadable files and working memory can be higher because packaged formats, the vision projector, KV cache, context length, and runtime buffers also consume space.
Both 27B variants support text and image input, reasoning, structured tool calls, long context, and agentic loops. Official demo tooling supports GGUF through llama.cpp and MLX formats on Apple Silicon, with runtime paths for Metal, CUDA, Vulkan, ROCm, and CPU depending on the platform and model package.
PrismML reports that its ternary model retains 95% and its 1-bit model retains 90% of the full-precision baseline across a 15-benchmark suite. Those are vendor-run aggregate results, not a guarantee for a particular application; quality drops more on some categories than others, especially vision, instruction following, and tool use.
The appeal is private, offline inference with no per-token provider charge, but local does not automatically mean safe. Developers still need to protect device data, sandbox tools, validate structured calls, test model failure modes, review licensing, and size context and concurrency to the hardware actually available.
Use cases
Who Bonsai 27B is best for
The strongest fit depends on the job you need the product to complete, not the size of its feature list.
Apple Silicon developers
Run compact MLX or GGUF builds on supported Macs and experiment with the model's on-device footprint.
Single-GPU local serving
Use compressed weights on supported NVIDIA, AMD, Vulkan, or CPU paths where a conventional 27B build would not fit.
Offline reasoning and coding
Build assistants that remain available without a network connection or recurring inference API dependency.
Tool-using prototypes
Test local structured tool calls, MCP-style integrations, and multi-step agent workflows with strict safeguards.
Compression research
Compare binary, ternary, conventional quantized, and full-precision behavior across quality, memory, speed, and power.
Capabilities
Core Bonsai 27B features
Two compression points
Offers a 1-bit binary model for minimum footprint and a ternary model for higher retained quality.
End-to-end low-bit weights
Applies the low-bit representation across the language network rather than leaving major language layers at high precision.
Multimodal input
Uses a compact vision tower to interpret screenshots, documents, photos, and other image inputs alongside text.
Reasoning mode
Produces a separate reasoning stream that compatible runtimes can expose and constrain with a reasoning budget.
Structured tool calling
Works with compatible chat templates and runtimes to emit OpenAI-style tool calls for agent workflows.
Long context
Supports up to 262,144 tokens, though practical context is limited by device memory and KV-cache configuration.
Speculative decoding
Supports draft-and-verify acceleration where the chosen runtime and configuration provide it.
GGUF releases
Provides llama.cpp-oriented packages for cross-platform local inference and server use.
MLX releases
Provides packages optimized for Apple Silicon, with different runtime requirements for binary and ternary builds.
Local OpenAI-compatible server
The official demo can launch llama-server or MLX serving paths for integration with compatible clients and tools.
Process
How the Bonsai 27B workflow works
Step 1
Choose quality or minimum footprint
Start with Ternary Bonsai 27B for better retained quality or the 1-bit variant when device memory is the binding constraint.
Step 2
Measure the real memory budget
Account for model files, vision projector, KV cache, activations, runtime overhead, operating-system limits, and concurrent requests.
Step 3
Select a supported runtime
Use the official demo's GGUF or MLX path appropriate for Apple Silicon, CUDA, ROCm, Vulkan, Windows, Linux, macOS, iOS, or CPU.
Step 4
Pin the tested toolchain
Follow the official repository's current runtime versions and branches, especially where low-bit kernels have not fully landed upstream.
Step 5
Download the correct package
Match the binary or ternary family, model format, vision projector, and runtime rather than mixing files from different releases.
Step 6
Begin with conservative context
Use a realistic context window and batch size first, then increase them while monitoring memory, latency, and stability.
Step 7
Evaluate application quality
Test representative text, vision, coding, reasoning, and tool tasks against the base model or other local candidates.
Step 8
Sandbox every tool
Validate tool schemas, minimize permissions, require approval for side effects, and isolate file, shell, network, and credential access.
Step 9
Benchmark on target hardware
Measure time to first token, generation rate, peak memory, power use, and long-context degradation on the device users will actually run.
Cost
Bonsai 27B pricing and free plan
Bonsai 27B model weights are free under Apache 2.0. Local use has no per-token PrismML charge, but hardware, electricity, storage, engineering, and serving operations still have real costs. The launch also offered a temporary preview API without a permanent public price.
Model weights
Free
Download the official binary or ternary packages under Apache 2.0.
- No model download fee
- GGUF and MLX releases
- Commercial use is subject to the license and all applicable terms
- Users provide their own compatible runtime and hardware
Local inference
Self-hosting cost
Run the model on a phone, computer, workstation, or server you control.
- No per-token API charge
- Hardware and electricity costs apply
- Engineering, monitoring, security, and support remain the operator's responsibility
- Concurrency and long context can require more memory than the headline model size
Developer preview API
Free for a limited time
A hosted trial route announced with the model release.
- Temporary developer preview
- Limits and availability may change
- Not a documented long-term production price
- Confirm current access and terms with PrismML
Pricing checked . Check current pricing at the source ↗
Assessment
Bonsai 27B strengths and limitations
Where it stands out
- Unusually small footprint for a 27B-class multimodal model
- Two variants let developers trade retained quality for device fit
- Apache 2.0 weights reduce licensing friction for experimentation and deployment
- Text, vision, reasoning, and structured tool calls support richer local applications
- GGUF and MLX packages cover several common local-inference paths
- A 262K maximum context enables large local inputs when hardware memory permits
- Local execution can improve privacy, offline availability, latency predictability, and marginal inference cost
- Official demo tooling and community hardware benchmarks reduce initial setup work
What to consider
- PrismML's retention percentages and speed numbers are vendor-reported benchmarks
- The compressed weight figure is not the same as total download size or peak working memory
- Vision projectors, KV cache, activations, runtime buffers, and the operating system all consume additional memory
- The 1-bit MLX path can require PrismML's fork until the necessary kernel support is upstream
- Quality loss is uneven and is more visible in some vision, instruction-following, and tool-use benchmarks
- High-end phone demonstrations do not mean every phone can allocate enough memory or sustain useful speed and thermals
- The full 262K context is impractical on many consumer devices without significant KV-cache memory
- Local agents still create security risk when given shell, file, network, credential, or application-control tools
Compare
Bonsai 27B alternatives
The right alternative depends on the specific output, workflow, controls and budget your project requires.
Coding
Qwen3.6-27B
The full-precision base family is the clearest comparison when maximum retained quality matters more than memory footprint.
Explore Qwen3.6-27B →Consumer
Gemma 4
Google's open-weight family offers multiple sizes for local reasoning and agentic use with a broader mainstream ecosystem.
Explore Gemma 4 →Consumer
Mistral 3
A flexible open-model family with several deployment sizes and established local and commercial tooling.
Explore Mistral 3 →Questions
Bonsai 27B FAQs
What is Bonsai 27B?
Bonsai 27B is PrismML's compressed, multimodal model family based on Qwen3.6 27B. It has a 1-bit binary version for minimum size and a ternary version for higher retained quality.
Can Bonsai 27B really run on an iPhone?
PrismML demonstrates the 1-bit variant on a high-end iPhone and says its compressed weights fit the available app memory. Compatibility and useful performance still depend on the exact device, runtime, context, thermal limits, and application overhead.
How large is Bonsai 27B?
PrismML lists 3.9 GB for the 1-bit weights and 5.9 GB for the ternary weights. Packaged GGUF or MLX files, the vision projector, KV cache, and runtime memory can make the actual storage and working-memory requirement larger.
What is the difference between 1-bit and ternary Bonsai 27B?
The binary model uses two weight values and prioritizes footprint and speed. The ternary model adds zero as a third value, uses more space, and is intended to preserve more of the base model's capability.
Is Bonsai 27B free?
Yes. PrismML releases the model weights under Apache 2.0. Users still pay for hardware, electricity, storage, development, and any hosting or support they choose.
Does Bonsai 27B support images and tools?
Yes. Both 27B variants are multimodal and the official demo supports image input, reasoning, and structured tool calls with compatible runtimes and templates.
Does Bonsai 27B support 262K context on every device?
The model supports a 262,144-token maximum, but KV cache grows with context. Many phones and laptops will need a much smaller working window to stay within memory and maintain acceptable speed.
Are Bonsai 27B benchmark claims independently verified?
The main retention and throughput figures come from PrismML's release materials. Developers should reproduce quality, speed, memory, and power measurements on their own tasks and target hardware.
Bottom line
Our Bonsai 27B verdict
Bonsai 27B is a compelling local-model option when fitting substantial multimodal and agentic capability into limited memory matters more than matching the full-precision baseline. The ternary build is the safer quality-first starting point; the 1-bit build is the more ambitious edge-deployment option. Treat the launch benchmarks as a reason to test, not as a substitute for application-specific evaluation.
Visit Bonsai 27B website ↗Similar Consumer Tools
View all Consumer AI tools →
ChatGPT Work
ChatGPT Work- OpenAI's Codex-powered agent for everyday non-coding work
Inkling
Inkling - Thinking Machines' open-weight multimodal model with adjustable reasoning
Muse Spark 1.1
Muse Spark 1.1 - Meta's upgraded, cost-effective agentic model with 1M-token memory and computer use
Hint
Hint - Martha Stewart's AI home app for maintenance, repairs, and fair quotes
AI training for the future of work.
Get access to all our AI courses, hundreds of real-world AI use cases, live expert-led workshops, an exclusive network of AI early adopters, and more.
AI Courses
Get unlimited access to all of our current & upcoming industry-specific AI courses for the duration of your subscription.
Daily Guides
To keep up with the rapid pace of AI, our team publishes AI implementation guides daily. Our library contains 300+ practical use cases to automate real-world work.
Workshops
Join weekly, live, interactive sessions with industry leaders who are at the forefront of AI for hands-on implementation guidance and exclusive insights.
Community
Network with an exclusive community of AI-first professionals who are working smarter with AI. Learn how early adopters are using AI in their work and businesses.