Skip to content
NEWError doctor and Watchtower: failed n8n runs explained in plain words, and emailed to you. See what is new

Library / Code, Git & Developer Tools

Code, Git & Developer Tools

Compare Local LLMs in LM Studio and Log Results

When a chat message is received, it runs the same prompt on several local LM Studio models, measures response metrics and saves the results to Google Sheets.

Import it free Sign in to open it

What it does

## 3. 💡Update the LM Settings From here, you can modify the following parameters to fine-tune model behavior: - Temperature: Controls randomness. Higher values (e.g., 1.0) produce more diverse results, while lower values (e.g., 0.2) make responses more focused and deterministic. - Top P: Adjusts nucleus sampling, where the model considers only a subset of probable tokens. A lower value (e.g., 0.5) narrows the response range. - Presence Penalty: Penalizes new tokens based on their presence in the input, encouraging the model to generate more varied responses.

The steps

  1. Get Models
  2. When chat message received
  3. Get timeDifference
  4. Run Model with Dunamic Inputs
  5. Analyze LLM Response Metrics
  6. Save Results to Google Sheets
  7. Capture End Time
  8. Capture Start Time
  9. Prepare Data for Analysis
  10. Extract Model IDsto Run Separately
  11. Add System Prompt
  12. LLM Response Analysis

Similar automations

Code, Git & Developer ToolsBenchmark Local LLMs in LM Studio, Log to Sheets12 stepsCode, Git & Developer ToolsCompare Local LM Studio Models and Log Results25 stepsCode, Git & Developer ToolsCompare Local LM Studio Models and Log Results25 stepsAI Agents & AssistantsCompare LLMs with OpenAI and OpenRouter in Google Sheets17 steps