{"id":2923,"date":"2024-11-13T12:00:52","date_gmt":"2024-11-13T13:00:52","guid":{"rendered":"http:\/\/suimy.me\/?p=2923"},"modified":"2024-11-20T17:13:06","modified_gmt":"2024-11-20T17:13:06","slug":"how-to-run-llm-locally-on-your-computer-with-lm-studio","status":"publish","type":"post","link":"http:\/\/suimy.me\/index.php\/2024\/11\/13\/how-to-run-llm-locally-on-your-computer-with-lm-studio\/","title":{"rendered":"How to Run LLM Locally on Your Computer with LM Studio"},"content":{"rendered":"<p>Running Large Language Models (LLMs) like Llama-3 or Phi-3 typically requires cloud resources and a complicated setup. <strong><a href=\"https:\/\/lmstudio.ai\" target=\"_blank\" rel=\"noopener noreferrer\">LM Studio<\/a><\/strong> changes this by providing a desktop app that lets you run these models directly on your local computer.<\/p>\n<p>It is compatible with Windows, macOS, and Linux, and its friendly GUI makes it easier to run LLMs, even for people who aren\u2019t familiar with technical setups. It\u2019s also a great option for privacy because all queries, chats, and data inputs are processed locally without any data being sent to the cloud.<\/p>\n<p>Let\u2019s see how it works.<\/p>\n<h4>System Requirements<\/h4>\n<p>To run LLM models smoothly on your device, make sure your setup meets these requirements:<\/p>\n<ul>\n<li><strong>PC (Windows\/Linux)<\/strong>: A processor supporting AVX2 (standard on newer PCs) and an NVIDIA or AMD GPU.<\/li>\n<li><strong>macOS<\/strong>: Requires Apple Silicon (M1\/M2\/M3). Intel-based Macs are not supported.<\/li>\n<li><strong>Memory<\/strong>: At least 16 GB RAM is ideal, though 8 GB may work if you use smaller models and context sizes.<\/li>\n<li><strong>Internet<\/strong>: A stable connection is recommended for downloading models.<\/li>\n<\/ul>\n<h4>Installation<\/h4>\n<p>To get started, <a href=\"https:\/\/lmstudio.ai\/download\" target=\"_blank\" rel=\"noopener noreferrer\">download LM Studio<\/a> for your platform.<\/p>\n<figure> <img loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" alt=\"LM Studio download page with platform selection options\" width=\"750\" height=\"480\" class=\"lazyload\" data-src=\"http:\/\/suimy.me\/wp-content\/uploads\/2024\/11\/lm-studio-download-page.jpg\"> <\/figure>\n<p>After downloading, follow the installation steps to launch the app. You\u2019ll see a familiar chat interface with a text box, similar to most AI chat applications, as shown below:<\/p>\n<figure> <img loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" alt=\"LM Studio chat interface screen\" width=\"750\" height=\"480\" class=\"lazyload\" data-src=\"http:\/\/suimy.me\/wp-content\/uploads\/2024\/11\/lm-studio-chat-interface.jpg\"> <\/figure>\n<p>Before you can start using it, you need to download and load a model.<\/p>\n<h4>What is a Model?<\/h4>\n<p>A model in this context is a pre-trained <em>algorithm<\/em> that can perform a variety of natural language processing tasks. The model is trained on a large dataset of text and learns to predict the next word in a sentence, enabling it to generate coherent and relevant text based on your input.<\/p>\n<p>There are many different models available, each with specific strengths. Some models are better at generating creative text, while others excel at factual information or shorter responses.<\/p>\n<p>For example, models like GPT-3, <a href=\"https:\/\/ai.meta.com\/blog\/meta-llama-3\/\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">Llama-3<\/a>, and <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/introducing-phi-3-redefining-whats-possible-with-slms\/\" target=\"_blank\" rel=\"noopener noreferrer\">Phi-3<\/a> generate creative and engaging text, while <a href=\"https:\/\/github.com\/01-ai\/Yi-Coder\" target=\"_blank\" rel=\"noopener noreferrer\">Yi Coder<\/a> is trained on code and is better at generating code snippets.<\/p>\n<h4>Load a Model<\/h4>\n<p><strong>LM Studio<\/strong> supports a variety of models, including GPT-3, Llama-3, Phi-3, and more. You can easily download models from the <strong>\u201cDiscover\u201d<\/strong> section in the sidebar. Here, you will see a list of available models, their parameter sizes, and their specializations.<\/p>\n<p>Select a model based on your needs. For example, if you want to generate creative text, download a model like Llama-3. If you need code snippets, try Yi Coder. Larger models require more resources, so choose a smaller model if your computer has limited power.<\/p>\n<p>In this example, I\u2019ll download <strong>Llama-3<\/strong> with 8B parameters. Once you click the download button, the model will begin downloading.<\/p>\n<figure> <img loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" alt=\"LM Studio discover models section with model options\" width=\"750\" height=\"480\" class=\"lazyload\" data-src=\"http:\/\/suimy.me\/wp-content\/uploads\/2024\/11\/lm-studio-discover-models.jpg\"> <\/figure>\n<p>After downloading, load the model by clicking on the <strong>\u201cLoad Model\u201d<\/strong> button in the <strong>\u201cChat\u201d<\/strong> section and selecting the model you downloaded.<\/p>\n<figure> <img loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" alt=\"LM Studio load model interface\" width=\"750\" height=\"250\" class=\"lazyload\" data-src=\"http:\/\/suimy.me\/wp-content\/uploads\/2024\/11\/lm-studio-load-model.jpg\"> <\/figure>\n<p>Once the model is loaded, start using it to generate text. Simply type your input in the text box and press enter. It can handle facts or general knowledge and is useful for creative writing, brainstorming, or generating ideas.<\/p>\n<figure> <img loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" alt=\"LM Studio chat response example\" width=\"750\" height=\"480\" class=\"lazyload\" data-src=\"http:\/\/suimy.me\/wp-content\/uploads\/2024\/11\/lm-studio-chat-response.jpg\"> <\/figure>\n<h4>Chat with Documents<\/h4>\n<p>Since version 0.3, LM Studio offers a <strong>Chat with Documents<\/strong> feature, allowing you to upload a document to the conversation. This is useful for generating text based on a specific document or providing extra context to the model.<\/p>\n<p>For example, I\u2019ll upload <a href=\"https:\/\/www.gutenberg.org\/ebooks\/1513\" target=\"_blank\" rel=\"noopener noreferrer\">the Romeo and Juliet book from Project Gutenberg<\/a> and ask a couple of questions.<\/p>\n<ol>\n<li>Who are the main characters in the story?<\/li>\n<li>What is the main conflict in the story?<\/li>\n<\/ol>\n<p><strong>LM Studio<\/strong> will gather information from the document and provide answers to your questions.<\/p>\n<figure> <img loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" alt=\"LM Studio document chat feature example with Romeo and Juliet text\" width=\"750\" height=\"480\" class=\"lazyload\" data-src=\"http:\/\/suimy.me\/wp-content\/uploads\/2024\/11\/lm-studio-rag.jpg\"> <\/figure>\n<p>Currently, this feature is experimental, meaning it may not always work perfectly. Providing as much context in your query as possible\u2014specific terms, ideas, and expected content\u2014will increase the chances of accurate responses. Experimentation will help you find what works best.<\/p>\n<p>Overall, I\u2019m happy with the results so far. It can answer questions accurately.<\/p>\n<h4>Wrapping Up<\/h4>\n<p><strong>LM Studio<\/strong> is a valuable tool for running LLM models locally on your computer, and we\u2019ve explored some features like using it as a chat assistant and summarizing documents. These features can boost productivity and creativity. If you\u2019re a developer, LM Studio can also run models specifically tuned for generating code.<\/p>\n<p>The post <a href=\"https:\/\/www.hongkiat.com\/blog\/run-llm-locally-lm-studio\/\">How to Run LLM Locally on Your Computer with LM Studio<\/a> appeared first on <a href=\"https:\/\/www.hongkiat.com\/blog\">Hongkiat<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Running Large Language Models (LLMs) like Llama-3 or Phi-3 typically requires cloud resources and a complicated setup. LM Studio changes [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":2925,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[15],"tags":[],"_links":{"self":[{"href":"http:\/\/suimy.me\/index.php\/wp-json\/wp\/v2\/posts\/2923"}],"collection":[{"href":"http:\/\/suimy.me\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/suimy.me\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/suimy.me\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/suimy.me\/index.php\/wp-json\/wp\/v2\/comments?post=2923"}],"version-history":[{"count":3,"href":"http:\/\/suimy.me\/index.php\/wp-json\/wp\/v2\/posts\/2923\/revisions"}],"predecessor-version":[{"id":2932,"href":"http:\/\/suimy.me\/index.php\/wp-json\/wp\/v2\/posts\/2923\/revisions\/2932"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/suimy.me\/index.php\/wp-json\/wp\/v2\/media\/2925"}],"wp:attachment":[{"href":"http:\/\/suimy.me\/index.php\/wp-json\/wp\/v2\/media?parent=2923"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/suimy.me\/index.php\/wp-json\/wp\/v2\/categories?post=2923"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/suimy.me\/index.php\/wp-json\/wp\/v2\/tags?post=2923"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}