MLX Community Projects

[awni]

Let's collect some cool MLX integrations and community lead projects here for visibility!

If you have a project you would like to feature, leave a comment, and we will add it. If the project is build with MLX Swift, add it to the MLX Swift Community Project page.

Text Generation

• mlx-ui: A simple UI / Web / Frontend for MLX mlx-lm using Streamlit.
• mlx-moe: Scripts to create your own moe models using mlx
• mlx-rag: Explore a simple example of utilizing MLX for RAG application running locally on your Apple Silicon device.
• mlx-rag-gguf Minimal, clean code implementation of RAG with mlx using gguf model weights
• mlx-llm: LLM applications running on Apple Silicon thanks to mlx from Apple
• outlines-mlx: A fast minimalistic implementation of guided generation on Apple Silicon using Outlines and MLX
• mlx-whatsapp: An mlx project to train a base model on your whatsapp chats using (Q)Lora finetuning
• iClone: Clone your friends with iMessage and MLX
• autogram: Grammar checker with a keyboard shortcut for Ollama and Apple MLX with Automator on macOS.
• mamba.py: A simple Mamba implementation in PyTorch and MLX.
• nanoGPT_mlx: Port of Andrej Karpathy's nanoGPT to Apple MLX framework.
• mlx-tuning-fork: Very basic framework for parameterized large language model (Q)LoRa fine-tuning using mlx, mlx_lm, and OgbujiPT. Architecture for systematic running of easily parameterized fine-tunes
• mlx-moe-models: A lightweight package for extending the mlx-lm to support custom moe models
• chat-with-mlx: Chat with your data natively on Apple Silicon using MLX Framework.
• transformerlab-app: A research platform to run, train, RAG, and evaluate LLMs through a GUI.
• mlx-transformers: Model implementations in MLX with a similar interface as Hugging Face Transformers.
• plpxsk/bert-qa: Fine tune BERT model for Q&A on MacBook.
• VimLM A local LLM-powered Copilot for Vim—offline, private, and fully integrated into your workflow.
• Toolio: JSON schema-steered structured output (3SO) and tool-calling for MLX
• mlx-lm-lora: Train Large Language Models in MLX.
• mlx-lm-lens: Find the hidden meaning of LLMs

Vision

• ml-aim: AIM: Autoregressive Image Models
• mimm: MLX Image Models
• mlx-image: MLX image models for Apple Silicon machines
• aggressor: A simplest possible implementation of Autoregressive Image Generation without Vector Quantization in Apple MLX.
• DINO_DETR_MLX: A port of the DINO DETR model for object detection in MLX.

Speech and Audio

• mlx_bark: Port of Suno's Bark TTS transformer in Apple's MLX Framework

Multi-modal

• voice-assistant: A simple toy demo of a local voice assistant with whisper and large language model.
• Video_summarization_mlx: Transcribe and summarize youtube video using mlx
• MLX-VLM: Run Vision LLMs locally on your Mac using MLX.
• e2tts-mlx: A single-file implementation of Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS model in MLX.
• whisper-turbo-mlx: A blazing fast single-file implementation of the OpenAI's Whisper Turbo (all in less than 250 lines of code).
• NotebookMLX: A port of NotebookLlama to MLX. Generate podcasts fully on device.

Misc

• mlx-omni-server: MLX Omni Server is a local inference server powered by Apple's MLX.
• mlx-optimizers: Seamlessly experiment with and adopt new optimization algorithms into your MLX workflow!
• mlx-graphs: Graph Neural Network library made for Apple Silicon
• flower: Flower: A Friendly Federated Learning Framework
• samplex: Package of useful sampling algorithms written in MLX.
• rlx: A reinforcement learning framework based on MLX.
• mlx3D: A library for deep learning with 3D data using MLX.
• mlx-ctrc: CTC loss in MLX on the CPU and GPU.
• mlx-hub: A command-line tool to search, download & manage MLX AI models on macOS.
• mlx-lm-kan: Kolmogorov–Arnold Networks (KAN) in MLX.

Educational

• Deep-Dive-Into-AI-With-MLX-PyTorch: "Deep Dive into AI with MLX and PyTorch" is an educational initiative designed to help anyone interested in AI, specifically in machine learning and deep learning, using Apple's MLX and Meta's PyTorch frameworks.
• cv-ml-lecture-notebooks: Computer Vision and Machine Learning Jupyter Notebooks for Educational Purposes
• mlx-micrograd: MLX port of micrograd - a tiny scalar-valued autograd engine with a small PyTorch-like neural network library on top.

• pikaGPT: A tiny implementation of a GPT, accelerated for Apple Silicon, built on picoGPT.

[chimezie]

Text generation: mlx-tuning-fork

[mzbac]

text generation: https://github.com/mzbac/mlx-moe-models
A lightweight package for extending the mlx-lm to support custom moe models

[noahfarr]

An implementation of Reinforcement Learning algorithms in MLX based in the Implementations from CleanRL. Still WIP because it’s missing a benchmark and some other minor things, but the implementations work correctly.
https://github.com/noahfarr/rlx

[RahulBhalley]

mlx-models. Currently supporting vision models by loading/converting from PyTorch checkpoints. Will later add support for text and audio models as well.

[awni]

Could you share a bit more about what it does?

[qnguyen3]

Hi I would love to add chat-with-mlx. It is a Chat UI + RAG Implementation on MLX. I wIll add more features later on (more advanced RAG pipeline + multimodal)

[adi-dhulipala]

I have an example of training a simple language model using BitLinear instead of nn.Linear. It's a port of Karpathy's minGPT to MLX along with a custom implementation of a BitLinear module. https://github.com/adhulipa/mlx-mingpt

I noticed this collection already has the far more meatier nanoGPT version ported to mlx, which is awesome! MinGPT, OTOH, is super simple, easy to follow and can serve as a reference example for folks looking to compare mlx equivalent operations from the original PyTorch implementation.

[aliasaria]

Transformer Lab https://github.com/transformerlab/transformerlab-app is an LLM research platform that allows you to run, train, perform RAG, and evaluate LLMs through a GUI.

[Jaykef]

MLX RAG with GGUF Models: https://github.com/Jaykef/mlx-rag-gguf

The code here builds on https://github.com/vegaluisjose/mlx-rag, it has been optimized to support RAG-based inferencing for .gguf models. I am using BAAI/bge-small-en for the embedding model, TinyLlama-1.1B-Chat-v1.0-GGUF as base model and the custom vector database script for indexing texts in a pdf file. Inference speeds can go up to ~413 tokens/sec for prompts and ~36 tokens/sec for generation on my 8G M2 Air.

[lin72h]

@Jaykef Very cool, thanks for sharing

[amirhossein-razlighi]

Vision: MLX3D A library for deep learning with 3D data using mlx.

[amirhossein-razlighi]

@awni
Can you please add this to the list? 🙏🏻

[lin72h]

very cool! thanks for working on the 3D support

[awni]

Done! Cool library btw!

[dc-dc-dc]

mlx-lite & mlx-onnx

[otriscon]

JSON schema decoding (allowing function calling, including an OpenAI-compatible server with tools) using MLX: https://github.com/otriscon/llm-structured-output

[Extremys]

Hello for text generation part, I'm happy to share with you that I've proposed and contributed to the integration of MLX with LibreChat.ai. So now you can use your local LLM powered by MLX through a fancy interface privately, enjoy! :D

See LibreChat-AI/LibreChat#2580

If in the future the community proposes an API servers supporting also multimodality, transcription, image generation for example, I will add them into LibreChat ;) It could be great also to have and LLM API supporting /models endpoint and multiple models simultaneously :D

[awni]

Awesome!!

[NicoNico6]

Hello, mlx community, we are happy to share with you that we have contributed the first strong sub-4 bit LLM model zoo for MLX community.

• HF collections: https://huggingface.co/collections/GreenBitAI/greenbitai-mlx-llm-6614eb6ceb8da657c2b4ed58

The modern LLM families include Llama3/2, Phi-3, Mistral, 01-Yi, and Qwen. A mlx-style inference toolkit is also shared for the local web chatting.

• gbx-lm here: https://github.com/GreenBitAI/gbx-lm.

We are an active team here, supporting the better low-bit community on the local platform. Enjoy!

[awni]

This looks really interesting! I'm curious if there is somewhere one can learn more about how you reduce the precision of the models?

[NicoNico6]

Thanks for your interest!

The MLX models are a subset of our main work green-bit-llm.

We construct the lower-bit models by combining neural architecture search and post-training quantization technique. All models are built from a mix precision of 4/2 bit group-wise min-max quantization only (for better large-scale deployment, e.g., MLX only supports 4/2 bit min-max quantization currently). The MLX models in HF collection is the layer-mix version in the model zoo.

Except better but not harder inference, we are also interested in the low-cost fine-tuning. We released Bitorch Engine for low-bit quantized neural network operations. Our release supports full parameter fine-tuning directly in quantized space, even under extremely constrained GPU resource conditions.

We are preparing a blog in huggingface for a better understanding of everything. Please stay tuned and star us in git if you like our project QvQ.

[BuildBackBuehler]

Thanks for your interest!

The MLX models are a subset of our main work green-bit-llm.

We construct the lower-bit models by combining neural architecture search and post-training quantization technique. All models are built from a mix precision of 4/2 bit group-wise min-max quantization only (for better large-scale deployment, e.g., MLX only supports 4/2 bit min-max quantization currently). The MLX models in HF collection is the layer-mix version in the model zoo.

Except better but not harder inference, we are also interested in the low-cost fine-tuning. We released Bitorch Engine for low-bit quantized neural network operations. Our release supports full parameter fine-tuning directly in quantized space, even under extremely constrained GPU resource conditions.

We are preparing a blog in huggingface for a better understanding of everything. Please stay tuned and star us in git if you like our project QvQ.

Heck yeah, this is truly awesome! I've been banging my head trying to get a 3-bit Omniquant model to work but this'll be so much better. I'd love to see a low-bit precision Wizard LM2-8x22B (2.2, 2.5, 3.0 range)!

It'd also be great to see some metrics versus exllama/QUIP/AQLM as far as speed/accuracy. I'm (trying) to run an AQLM model right now and I believe it is still considered SotA but its about as much degradation as I'm willing to deal with. IIRC -10 to -15 (so say 78-80% Llama3 -> 65-68% accuracy).

In any case, I'll be giving your project an updoot and following it

[Jaykef]

mlx_micrograd - mlx port of Karpathy's micrograd - a tiny scalar-valued autograd engine with a small PyTorch-like neural network library on top.

Installation

pip install mlx_micrograd

Example usage

Example showing a number of possible supported operations:

from mlx_micrograd.engine import Value

a = Value(-4.0)
b = Value(2.0)
c = a + b
d = a * b + b**3
c += c + 1
c += 1 + c + (-a)
d += d * 2 + (b + a).relu()
d += 3 * d + (b - a).relu()
e = c - d
f = e**2
g = f / 2.0
g += 10.0 / f
print(f'{g.data}') # prints array(24.7041, dtype=float32), the outcome of this forward pass
g.backward()
print(f'{a.grad}') # prints array(138.834, dtype=float32), i.e. the numerical value of dg/da
print(f'{b.grad}') # prints array(645.577, dtype=float32), i.e. the numerical value of dg/db

[Jaykef]

@awni this should be under Educational.

[s-junjie]

MindCraft Studio — MLX-Powered AI Image Generator for Mac

[isla-watson-project]

Isla Watson — an independent local AI companion research project from Scotland.

I’ve just posted an architecture-level overview exploring MLX/Apple Silicon as a local compute fabric for a persistent companion: replaceable foundation models, persistent identity/state outside the model, specialist Mac agents, and distributed inference when larger models are needed.

The project is currently at the MLX evaluation/prototyping stage, and I’d particularly value feedback from anyone working with distributed MLX or multi-Mac workloads.

Full discussion: #4482

[pratikgx]

mlx.fast (https://mlx.fast, leaderboard at https://www.yukon.org/mlxfast)

We built an open benchmark arena for MLX inference speed on Apple Silicon. The current task is faster prefill and serial decode for Qwen 3.8 Flash Next with speculative decoding, scored as a single composite (prefill_gain^0.25 × decode_gain^0.75) against a sealed baseline, so a purely serial run is 1.0. Every promoted submission lands as a commit on the harness repo with a required public write-up, so the full optimization history is readable, not just the scores. As of today there are 40 promoted submissions from 12 solvers, with the best run at +85.5% composite.

The harness is open source. Clone it, run locally, submit: https://github.com/Layr-Labs/mlxfast-qwen38-125b-a6b-engine

Suggested entry for Misc:

• mlx.fast: Faster open-model inference on Apple Silicon. An open leaderboard where you optimize prefill and decode for a fixed model and submit reproducible runs. Built on Yukon by Eigen Labs.

(The harness is built on MLX Swift, so happy to have this on the MLX Swift Community Projects thread instead, or both, whichever you prefer.)

[rajveer43]

VeloxQuant-MLX: Shrinks the KV cache of any mlx_lm model on Apple Silicon — up to 16× smaller with near-lossless quality, in 3 lines of code. 43 compression methods (quantizers, token-eviction caches, cross-layer merging) behind one API, with hand-written Metal kernels for the hot path (up to 14.7× faster quantize). Validated on 12 production models (Llama, Mistral, Qwen, Phi, Gemma 3/4, Falcon) plus VLMs via mlx-vlm.

[bag3labo]

I'd like to share Open Decisions, an open-source Python package for typed decisions from local models, inspired by Jev. It has MLX-LM and MLX-VLM adapters; I tested Qwen3-VL-4B-Instruct in 4-bit on an M4 Pro.

The package prefills the prompt, reads next-token logits for labels representing the allowed choices, and converts them into choices, yes/no scores, or joint boolean gates. No answer tokens are generated; inference still runs, and the probabilities are uncalibrated.

The agent-routing example scores whether to delegate and whether to ask for missing context in one evaluation. Three recorded routing requests took 533 ms with an uncached policy, then 113 ms and 118 ms with the policy cached, with the model already loaded. These are individual observations, not a benchmark.

There's also a recorded Tetris comparison with hosted Jev. The controller supplies structured board state, legal landings and projected outcomes, so the example has substantial help from code and uses no screenshots.

The repo includes installation instructions, runnable examples, recordings and limitations. I'd welcome feedback on the MLX adapters and evaluating these decisions across models.

[nicedreamzapp]

two MLX projects from me:

claude-code-local: runs Claude Code fully on-device. A native MLX server (built on mlx_lm) speaks the Anthropic Messages API, so the unmodified claude CLI drives local Qwen, Gemma and Llama models with no cloud calls. It picks a model based on your Mac's RAM, parses the different tool-call formats local models emit, and has a verified-offline mode.

nemotron-omni-mlx: pure MLX runtime for the vision and audio towers of NVIDIA's Nemotron 3 Nano Omni, which had no Apple Silicon runtime. It passes 23/23 parity tests against NVIDIA's PyTorch reference. 67.7 tok/s with an image and 147 tok/s with audio on an M5 Max, MIT licensed.

I also added Muse Glimmer text model support to mlx-lm (ml-explore/mlx-lm#1710). Thanks to everyone who works on MLX, none of this would exist without it.

[tc3oliver]

laya-apple: a heterogeneous MLX GPU + Apple Neural Engine runtime for Laya typed-decision models. Neural Engine artifacts are used only after they pass a parity check against upstream Laya on the machine that runs them. On one M4 Max, under a mixed short + long workload, serving on the GPU and the ANE at the same time reached up to 4.57× the throughput of GPU-only serving (method and data). I'm collecting benchmarks from other Apple silicon: community matrix and write-up. Would you consider adding it to the Community Projects list?

[nperumal]

mlx-recipes — end-to-end ML recipes in pure MLX.

https://github.com/nperumal/mlx-recipes

I've been building a small repo of end-to-end recipes written in pure MLX — no NumPy or scikit-learn anywhere in the math path, enforced by a CI grep. Two recipes so far, both in a foundations tier aimed at people learning the framework rather than at benchmarking it: linear regression two ways (mx.grad with a hand-written update vs nn.Linear + mlx.optimizers), and logistic regression binary-then-softmax on Palmer Penguins.

[lukeb-aidev]

If you tune a model on a Mac, how do you know which version is serving now? Cohesix 1.2.0 keeps the comparison, serving check, and rollback tied to the same model change. MLX still runs locally on Apple Silicon; the guide also covers optional vMLX serving.

The Mac rollout walkthrough is here:
https://github.com/lukeb-aidev/cohesix/releases/tag/v1.2.0

[hiroki-abe-58]

sokudan: a Japanese decision model with an MLX backend

https://github.com/hiroki-abe-58/sokudan

It reads a Japanese text plus typed questions and returns answers with probabilities (pick one option, a position on an ordered scale, or yes/no), without generating text. 310M parameters, ModernBERT-ja backbone, Apache-2.0.

On Apple Silicon, pip install sokudan uses MLX by default, with no torch install. On an idle M1 Max, one 5-option choice takes about 13 ms with MLX (float16), 1.4-1.7x faster than torch on MPS, with the same answers across MLX, MPS and CPU. Install is about 400 MB versus 1.1 GB for the torch environment. Method and tables: https://github.com/hiroki-abe-58/sokudan/blob/main/docs/mlx.md

Thanks for MLX. Porting the encoder took far less code than I expected.

[MarcosAsh]

Hi, I'm a contributor to Keras and we're working on keras-mlx, an MLX backend for Keras 3 that lives under keras-team. You set KERAS_BACKEND=mlx and your Keras models train and run on Apple silicon through MLX. It's still early, so feedback and bug reports are very welcome. Would be great to have it in the Misc section!

[sasan1200]

I'm sharing mlx-workbench, a local UI for the MLX model lifecycle on Apple Silicon. It covers scanning, conversion, benchmarking, and serving, with a Web UI and native SwiftUI app.

Repo: https://github.com/cavi-ai/mlx-workbench

Install options are documented in the README, including the 0.4.0 DMG and development builds. Feedback on the workflow and what would make this useful to the MLX community is welcome.

[jacar-javi]

oMLX on an M5 Max 128 GB: install, tuning and measured speedups (a write-up, not a repo)

https://jacar.es/en/how-to-install-and-tune-omlx-on-m5-max-128-gb/

A hands-on guide to serving MLX models with oMLX 0.6.4 on an M5 Max, measured through its OpenAI/Anthropic-compatible API. Lightning MTP takes Qwen3.8-27B oQ4e from 21.1 to 39.7 tok/s (1.88x); against oMLX 0.3.8, Qwen3.6-35B-A3B 8-bit goes from 15.2 to 77.9 tok/s with a 32k prompt. It lists which acceleration (Lightning MTP, VLM MTP, DFlash) pays off for which model, what does not help, and how to point Claude Code at the local endpoint.

[androidwangchong]

I'd like to share Elmren Voice, a Mac document-to-speech application that uses Qwen3-TTS through mlx-audio for local voice cloning and synthesis. The Python inference sidecar sits behind a Tauri 2 / React / Rust desktop shell; the outputs are portable read-along HTML, M4B audiobooks and captioned MP4.

Two reference-input decisions in our integration may be useful to other builders:

• Our Qwen path converts reference audio that is not already 24 kHz to 24 kHz before passing it to mlx-audio. An existing 24 kHz file passes through unchanged. This is our application's adapter behavior, rather than a claim about every MLX model or every audio input.
• Each saved voice keeps its reference recording and transcript together as ref.wav and ref.txt. At synthesis time the engine reads that same saved ref.txt for the ICL ref_text and uses it for the per-sentence generation calls. Reusing the saved transcript avoids a second copy drifting away from the recording. File-existence and non-empty checks do not prove the transcript is accurate: a clean, permitted recording and a transcript that matches it still matter.

The app also includes Kokoro presets and macOS Samantha, so the advertised 64 built-in voices are not all MLX voices. Voice packs download initially; day-to-day synthesis and permitted voice cloning run locally. These are implementation notes checked against our current source, not a new performance benchmark.

A real exported player is available at elmren.com/demo; product details are at elmren.com. I'm Chong Wang, the developer. This introduction was prepared with AI assistance and checked against the implementation. Elmren Voice is closed-source and costs $39.99 once for one Mac; the no-card trial includes 120 synthesis minutes over 7 days from first launch. It requires Apple Silicon M1+ and macOS 13+, and the current reading workflow is English.

[androidwangchong]

A measured Kokoro/MLX example to add to the integration notes above (test run on 17 October):

On an M1 Pro MacBook Pro with 32 GB RAM, the packaged Elmren Voice 1.10.0 engine processed 1,704 English words from a public-domain chapter using Kokoro Heart (af_heart), speed 1.0, and the production sequential scheduler. It produced 569.925 seconds of audio. Text commit through synthesis and M4B export took 104.845 seconds: 99.434 seconds for the synthesis job, including its internal runtime/model setup, and 5.403 seconds for M4B export (plus the small text-commit step).

Scope matters here: model/runtime packs were already downloaded. This exercised the production JSONL/engine pipeline with extracted chapter text, not GUI interaction or PDF/EPUB parsing. The separate sidecar-process startup was about 0.535 seconds and is outside the 104.845-second figure. There was no separate Whisper alignment pass in this Kokoro job.

One reporting lesson: the first 33-word sentence completed about 63.9 seconds after job submission; the remaining 1,671 words finished in about 35.5 seconds. That first observation includes generation and polling as well as setup, so it is not a measurement of pure model-loading time. Multiplying a short chapter's total by word count would repeatedly charge that large initial cost; extrapolating the later throughput also assumes it stays steady. Neither is a measured full-book result. The M4B was fully decoded as a file check; this says nothing about narration quality or cloned-voice speed.

Developer disclosure: Elmren Voice is my commercial app. This technical update was prepared and posted by my authorized AI assistant and checked against the saved production-run evidence.

[B0yko]

I'd like to share taskdistill, a Python CLI for single-turn classification and JSON extraction on Apple Silicon. It captures or imports examples, LoRA-tunes a small model with MLX, and serves an OpenAI-compatible cascade that sends low-confidence requests to the original API. The fallback threshold is selected on validation data and checked on held-out tests.

The Banking77 and synthetic-invoice demos can replay recorded teacher outputs without an API key. The MLX path requires macOS 14+. This is not a general chat or agent replacement; the README also documents an invoice test that missed its quality target.

Author: Andrii Boiko.