Private LLM - Local AI Chat 12+

Fantastic app, I like the selection of models and they run great on the M4 chip of the iPhone 16 Max and iPad Pro. System prompt is also great for some tweaks to responses. The most simple and important feature (aside from model memory) that Private LLM needs is to be able to edit the responses of the AI. I enjoy the chatbot's responses but hate that it'll pull context from previous responses and loop entire phrases regardless if I regenerate its response or edit mine. Sometimes I don't like how the model formatted its response and want to change it without clearing the convo and retrying multiple times. The second feature that would be a quality of life is to be able to save conversations within the app. At the very least, export them as a whole without needing to screenshot, grab text, paste, correct, and format it elsewhere. I learned early on that copying text blocks at a time can cause the app to reload its model and even clear the conversation. A bot refresh would also be nice to clear older message context during a longer conversation to keep the model fast and responsive, but that can coincide with model memory to more easily pull specific facts you want it to remember in the first place without having to create a queue of explanations to pick up where you left off at. Useful if you switch models for different purposes like organizing, summarizing, q&a vs. generation, editing, brainstorming, vs. chatting.

Works well. The display supports markdown which is great. The Zephyr model is a good default. A few quirks make it 4/5 stars. There’s a strange animation when selecting text out if the chat. The shortcuts are so close to being useful. I want to be able to ask a question and have the response read by Siri while I’m driving. It almost works, but for some reason it always says “something went wrong” while in CarPlay. Finally, it might be nice to be able to save conversations somehow. A history would be a good way, or maybe just a way to export the whole chat to a file.

I've been using this app for a couple of months at this point. When it was first released, it was a neat proof of concept, but it only supported 7B models and was overall just too simple to use effectively. With a recent update, a 13B model was released for all macs with 16GB of memory, and it makes such a huge difference! It's not quite at the same level as ChatGPT 3.5T, but it's close enough that I never use 3.5T anymore; this is my go-to. I greatly appreciate the on-device processing (hallelujah privacy!), and it doesn't even use too much power - my battery still lasts for hours and hours. The performance is also great; my base M1 Air powers right through the prompts.

I only have two complaints about the app at this point. 1) The 13B model uses about 12GB of memory by itself, which does force the use of swap on a 16GB Air. Not much the dev can do about this, but it is something to keep in mind. You'll want to close out of other programs before launching this. 2) There still is no feature that has separate conversations. If you want to start a new conversation, you need to delete the existing conversation. I'd love it if we could get separate conversations in a future update; it would make this app so much easier to use.

Overall, I love it and do not regret buying it at all. I can't wait to see what future updates bring :)