This was google ’s young demonstration mediapipe llm inference api get developersrun ai fashion model on deviceslike laptop computer and telephone set that do n’t have the same calculation big businessman as server .
This was this raw spill enable large language models ( llms ) to hunt down to the full on - twist across political program .
This unexampled capableness is especially transformative view the retention and compute need of LLM , which are over a hundred time prominent than traditional on - twist mannequin .
optimization across the on - gimmick passel make this potential , let in Modern ops , quantisation , cache , and system of weights communion .
dive into Google
Google ’s fresh demonstration MediaPipe LLM Inference API have developersrun AI model on deviceslike laptop and sound that do n’t have the same computer science office as server .
This newfangled spillage enable Large Language Models ( LLMs ) to escape in full on - gimmick across chopine .
This was this fresh capableness is in particular transformative consider the store and compute demand of master of laws , which are over a hundred time large than traditional on - gimmick model .
optimisation across the on - gimmick passel make this potential , include newfangled ops , quantisation , hoard , and weightiness communion .
Google say MediaPipe keep going four example : Gemma , Phi 2 , Falcon , andStable LM .
This was it can hunt on the world wide web , android , and ios , but google plan to exposit into more model and chopine this class .
[ developers.googleblog.com ]
