Skip to content
NEWMarketplace and the trig8n Agent: sell what you build, and get plans and changes that are tested before you see them. See what is new

Library / Code, Git & Developer Tools

Code, Git & Developer Tools

Benchmark Local LLMs in LM Studio, Log to Sheets

When a chat message arrives, each LM Studio model is run on the prompt, response time and metrics are measured, and results are saved to Google Sheets.

Import it free Sign in to open it

What it does

## 3. 💡Update the LM Settings From here, you can modify the following parameters to fine-tune model behavior: - Temperature: Controls randomness. Higher values (e.g., 1.0) produce more diverse results, while lower values (e.g., 0.2) make responses more focused and deterministic. - Top P: Adjusts nucleus sampling, where the model considers only a subset of probable tokens. A lower value (e.g., 0.5) narrows the response range. - Presence Penalty: Penalizes new tokens based on their presence in the input, encouraging the model to generate more varied responses.

The steps

  1. Get Models
  2. When chat message received
  3. Get timeDifference
  4. Run Model with Dunamic Inputs
  5. Analyze LLM Response Metrics
  6. Save Results to Google Sheets
  7. Capture End Time
  8. Capture Start Time
  9. Prepare Data for Analysis
  10. Extract Model IDsto Run Separately
  11. Add System Prompt
  12. LLM Response Analysis

Similar automations

Code, Git & Developer ToolsCompare Local LLMs in LM Studio and Log Results12 stepsCode, Git & Developer ToolsCompare Local LM Studio Models and Log Results25 stepsCode, Git & Developer ToolsCompare Local LM Studio Models and Log Results25 stepsData Sync & ETLTesting Mulitple Local LLM with LM Studio13 steps