Lololol show me an "MLIR" port. Do you mean tensorflow port or jax port or torch... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		mathisfun123 on Aug 11, 2023 \| parent \| context \| favorite \| on: Llama 2 on ONNX runs locally Lololol show me an "MLIR" port. Do you mean tensorflow port or jax port or torch port (that uses torch-mlir)? Or do you really mean llama implemented in linalg/tosa/tendor?

brucethemoose2 on Aug 11, 2023 [–]

I wasn't talking about Llama specifically. I was thinking of the SHARK Stable Diffusion port (which uses MLIR/IREE), as it considerably outpaced the ONNX runtime.

But apparently the performance of llama on torch-mlir is progressing: https://github.com/nod-ai/SHARK/issues/1707

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact