What's a good/affordable GPU to run these projects locally? It seems like buildi... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		fareesh on June 15, 2023 \| parent \| context \| favorite \| on: Gorilla: Large Language Model Connected with APIs What's a good/affordable GPU to run these projects locally? It seems like building anything on top of these runs into either a big GPU cost for yourself or a big compute cost if you scale for others.

Tostino on June 15, 2023 [–]

3090 can run 30b parameter models, 2x can run 65b parameter models.

4090 can run the same, very slightly faster for much more money.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact