Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It will be tough to run on our 4x H200 node… I wish they stayed around the 350B range. MLA will reduce KV cache usage but I don’t think the reduction will be significant enough.
 help



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: