LangSmith has released a Command Line Interface (CLI) and an initial set of 'skills' designed to provide AI coding agents with specific expertise in the LangSmith ecosystem. These skills aim to simplify adding tracing to agents, understanding their execution, building test sets, and evaluating performance.

The LangSmith CLI

The LangSmith CLI is designed to be agent-native, offering building blocks needed to operate within LangSmith. This includes fetching traces, managing datasets, and running experiments. Combined with skills, the CLI allows agents to navigate LangSmith via the terminal, accelerating the development of the agents themselves.

What are Skills?

Skills are instructions, scripts, and resources that improve agent performance in specialized domains. They are dynamically loaded, retrieving a skill only when relevant to the task at hand. This approach avoids the performance degradation that occurs when providing too many tools to an agent.

LangSmith Skills

The langsmith-skills repository contains three main skills:

  • trace: adds tracing to existing code and queries traces.
  • dataset: creates datasets of examples.
  • evaluator: evaluates agents on datasets.

These skills cover the main areas of AI engineering in LangSmith. The company plans to add additional skills over time.

Skill Impacts

Using skills has led to significant improvements in Claude Code's performance on basic tasks in LangSmith. For example, the pass rate increased from 17% to 92% in specific tests.

Installation

Skills can be installed using npx skills:

Local (current project):

npx skills add langchain-ai/langsmith-skills --skill '*' --yes

Global (all projects):

npx skills add langchain-ai/langsmith-skills --skill '*' --yes --global

To link skills to a specific agent (e.g. Claude Code):

npx skills add langchain-ai/langsmith-skills --agent claude-code --skill '*' --yes --global