Jin Daily AI Trivia - Apple ship Foundation Model CLI in MacOS 27
Apple now included the whole AI infrence server on every MacOS 27 device (since is Apple Silicon only)
System model have 4K context windows PCC (Private Cloud Compute) have 32k context windows
judging from memory usage, the model use around 1-1.5GB and it CPU bound at the moment, GPU and ANE is no in-use. Best part, if u use PCC, there is zero memory pressure and thing work super fast.
This is run on a 8GB Macbook Neo.
