Notable this isn't even close to realtime. M4 Max. | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		refulgentis 2 days ago \| parent \| context \| favorite \| on: Rust implementation of Mistral's Voxtral Mini 4B R... Notable this isn't even close to realtime. M4 Max.

adefa 21 hours ago [–]

True :)

After some performance improvements, it is realtime on my DGX Spark with an RTF of .416 -- now getting ~19.5 tokens per second. Check it out, see if it's better for you.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact