Built-in Tools
Ready-made tools for agents: a calculator, the date and time, UUIDs, JSON, files, HTTP, a shell and web search.
Table of contents
- 1. Pick a bundle
- 2. Register the tools
- 3. The tools
- 4. Web search
- 5. Configure the tools
- 6. Safety: what each tool can do
- 7. Things to know
- 8. Your own tools
- Related
The built-in tools live in the llm4s-agent-tools module (see
installation). They depend
on llm4s-core only, not on the agent runtime, so they work with plain tool calling through a ToolRegistry as well
as with an Agent. Some of them can read your files, call your network or run programs, so decide which ones a model
gets before you register anything: section 6 says what each can do.
What this page says is checked by tests. The bundle tables, the parameter table, the defaults and the safety section
are asserted against the real code by
BuiltinToolsGuideSpec.
The snippets are compiled and run there too, with configuration passed in explicitly where a snippet would read
your environment, except the agent snippet, which needs both llm4s-agent and llm4s-agent-tools and so is run by
BuiltinToolsGuideAgentSpec
in samples. The last snippet, which refers to a tool of yours, is the one exception. If you change the page, change
the specs.
1. Pick a bundle
BuiltinTools builds a set of tools for you. Every method returns a Result, because building a tool can fail.
| Bundle | Tools it contains | Reach |
|---|---|---|
BuiltinTools.coreSafe |
get_current_datetime, calculator, generate_uuid, json_tool |
None: no files, no network, no processes |
BuiltinTools.withHttpSafe() |
the core tools, plus http_request |
Read-only HTTP |
BuiltinTools.withFilesSafe() |
the above, plus read_file, list_directory, file_info |
Read-only HTTP and files |
BuiltinTools.developmentSafe() |
the above, plus write_file, shell_command |
Files (read and write) and a shell |
BuiltinTools.customSafe(...) |
the core tools, plus a tool for each configuration you pass | Exactly what you configure |
Two things are in no bundle: the web search tools (section 4) and WeatherTool
(section 7). You add them yourself.
Start from the smallest bundle that does the job. customSafe is the one to use in production, because it forces
you to say what each tool may touch:
1
2
3
4
5
6
7
8
import org.llm4s.toolapi.builtin.BuiltinTools
import org.llm4s.toolapi.builtin.filesystem.FileConfig
import org.llm4s.toolapi.builtin.http.HttpConfig
val tools = BuiltinTools.customSafe(
fileConfig = Some(FileConfig(allowedPaths = Some(Seq("/srv/agent-data")))),
httpConfig = Some(HttpConfig.restricted(Seq("api.example.com")))
) // Result[Seq[ToolFunction[_, _]]]
2. Register the tools
With a ToolRegistry
1
2
3
4
5
6
7
8
9
10
11
import org.llm4s.toolapi.{ ToolCallRequest, ToolRegistry }
import org.llm4s.toolapi.builtin.BuiltinTools
BuiltinTools.coreSafe.foreach { tools =>
val registry = new ToolRegistry(tools)
registry.execute(ToolCallRequest("calculator", ujson.Obj("operation" -> "multiply", "a" -> 6, "b" -> 7))) match {
case Right(json) => println(json("result")) // 42.0
case Left(error) => println(error.getFormattedMessage)
}
}
registry.getOpenAITools() gives the tool definitions in the format every client takes. execute returns
Either[ToolCallError, ujson.Value]: a refused or failed call is a Left you can read, never an exception. For
example, dividing by zero comes back as Left(...) whose getFormattedMessage includes “Division by zero”.
With an Agent
1
2
3
4
5
6
7
8
9
10
11
12
import org.llm4s.agent.Agent
import org.llm4s.toolapi.ToolRegistry
import org.llm4s.toolapi.builtin.BuiltinTools
// `client` is an LLMClient: see Basic Usage for how to create one
val result = for {
tools <- BuiltinTools.coreSafe
agent <- Agent.builder("calculator-agent", client).withTools(new ToolRegistry(tools)).build()
state <- agent.run("What is 15% of 850?")
} yield state
result.map(_.answer) // Right(Some("...")) when the run completed
answer is None when the run did not complete (a guardrail blocked it, or it hit the step limit): read status
to see why. The runnable version is
BuiltinToolsAgentExample.
3. The tools
The names are what the model sees, and the parameters are what it can send. A parameter in bold is required:
a call without it comes back as a Left naming it. The others have defaults.
| Tool | Bundle | Parameters | What it returns |
|---|---|---|---|
get_current_datetime |
core | timezone (default UTC), format (iso or human: English text; anything else is an error) |
datetime, timezone, timestamp, iso8601, components |
calculator |
core | operation (add, subtract, multiply, divide, power, sqrt, percentage, abs, min, max, modulo), a, b (needed by the two-operand operations) |
expression, result, formatted |
generate_uuid |
core | count (1 to 10, default 1), format (standard or compact) |
uuids: each with uuid, version, variant |
json_tool |
core | operation (parse, format, query, validate), json, path (for query) |
success, result, formatted |
http_request |
http | url, method, headers, body, content_type |
statusCode, headers, body, contentType, truncated, and more |
read_file |
files | path, max_lines, encoding |
path, content, size, lines, truncated |
list_directory |
files | path, max_entries (at most 100), include_hidden |
entries, totalFiles, totalDirectories, truncated |
file_info |
files | path |
exists, size, sizeHuman, timestamps, permissions, isSymlink, extension |
write_file |
development | path, content, append, encoding |
path, bytesWritten, created, appended |
shell_command |
development | command |
exitCode, stdout, stderr, truncated, timedOut |
duckduckgo_search |
none | search_query |
abstract_, answer, relatedTopics, and more |
brave_web_search |
none | search_query |
web results (brave_image_search, brave_video_search and brave_news_search exist too) |
exa_search |
none | query |
results with title, url, text, highlights, and more |
json_tool queries use dot notation for objects and brackets for arrays: data.users[0].name. A path is read to the end: an empty segment (a..b), an index that is not a whole number (a[x], a[-1]) or an index too large for an Int is an error that names the text it stopped at, not a silently shortened query. Every operation refuses a document nested more than 512 levels deep.
4. Web search
Three search tools are not in any bundle, because each calls a third party and two of them need a key. Their settings
are read from llm4s.tools.* with ToolsConfigLoader, then passed to the tool:
1
2
3
4
5
6
7
8
9
10
import org.llm4s.config.ToolsConfigLoader
import org.llm4s.toolapi.ToolRegistry
import org.llm4s.toolapi.builtin.BuiltinTools
import org.llm4s.toolapi.builtin.search.DuckDuckGoSearchTool
val registry = for {
config <- ToolsConfigLoader.loadDuckDuckGoSearchTool()
search <- DuckDuckGoSearchTool.create(config)
core <- BuiltinTools.coreSafe
} yield new ToolRegistry(core :+ search)
The Brave and Exa tools are built the same way:
1
2
3
4
5
6
7
8
import org.llm4s.config.ToolsConfigLoader
import org.llm4s.toolapi.builtin.search.{ BraveSearchCategory, BraveSearchTool, BraveWebSearchResult, ExaSearchTool }
val brave = ToolsConfigLoader
.loadBraveSearchTool()
.flatMap(config => BraveSearchTool.create[BraveWebSearchResult](config, BraveSearchCategory.Web))
val exa = ToolsConfigLoader.loadExaSearchTool().flatMap(config => ExaSearchTool.create(config))
Brave also has image, video and news categories. These are the keys, as llm4s-agent-tools ships them in its
reference.conf (set them in your application.conf, or let the environment variable fill them):
| Tool | Key under llm4s.tools |
Default | Environment variable |
|---|---|---|---|
| DuckDuckGo | duckduckgo.apiUrl |
https://api.duckduckgo.com/ |
DUCK_DUCK_GO_SEARCH_API_URL |
| Brave | brave.apiKey |
none: required | BRAVE_SEARCH_API_KEY |
brave.apiUrl |
https://api.search.brave.com/res/v1 |
BRAVE_SEARCH_API_URL |
|
brave.count |
5 |
BRAVE_SEARCH_COUNT |
|
brave.safeSearch |
moderate |
BRAVE_SAFE_SEARCH |
|
| Exa | exa.apiKey |
none: required | EXA_API_KEY |
exa.apiUrl |
https://api.exa.ai |
EXA_API_URL |
|
exa.numResults |
10 |
EXA_NUM_RESULTS |
|
exa.searchType |
auto (or neural, fast, deep) |
EXA_SEARCH_TYPE |
|
exa.maxCharacters |
500 |
EXA_MAX_CHARACTERS |
ExaSearchTool.create rejects an empty API key when it builds the tool, so a missing key is an error at start-up and
not in the middle of a conversation. DuckDuckGo needs no key and is best for definitions, facts and quick lookups,
not for general web search. Whatever the model searches for is sent to the search provider.
5. Configure the tools
Each tool that can reach outside the process takes a configuration. The defaults below are what you get when you do not pass one.
Files: FileConfig and WriteConfig
| Setting | FileConfig (read tools) |
WriteConfig (write_file) |
|---|---|---|
allowedPaths |
None: any path outside blockedPaths |
required: no default |
blockedPaths |
FileConfig.DefaultBlockedPaths: the platform’s system and credential locations (wins over allowedPaths; see below) |
not a setting |
maxFileSize |
1 MB | 10 MB |
followSymlinks |
false |
not a setting |
allowOverwrite |
not a setting | false |
createDirectories |
not a setting | true |
1
2
3
4
5
6
7
import org.llm4s.toolapi.builtin.BuiltinTools
import org.llm4s.toolapi.builtin.filesystem.{ FileConfig, WriteConfig }
val tools = BuiltinTools.customSafe(
fileConfig = Some(FileConfig(allowedPaths = Some(Seq("/srv/agent-data")), maxFileSize = 256 * 1024)),
writeConfig = Some(WriteConfig(allowedPaths = Seq("/srv/agent-data/out")))
)
A path is judged by where it really is: symbolic links are resolved, then that real location is compared with each
allowed or blocked entry (resolved the same way), one path component at a time. The file tools remove .. as text
first and then open the location they judged. FileConfig.isPathAllowed, and the shell tool’s path policy, judge the
path as given, and an operating system applies a .. that comes after a link in one of two ways: POSIX (Linux, macOS)
follows the link first, so link/.. is the parent of the link’s target, while Windows removes .. as text first, so
link/.. is the directory holding the link. The check reads the path both ways, on every platform, and allows it
only when both locations are allowed. So with data/l -> data/a/b, data/l/../x.txt is allowed (data/a/x.txt or
data/x.txt), and data/l/../../x.txt is refused even on Linux, where it means data/x.txt, because on Windows it
is x.txt beside data; spell such a path without the .. after the link. /srv/agent-data
covers /srv/agent-data/notes.txt but not /srv/agent-data-secret, and a link inside an allowed directory that leads
outside it is refused, for reading, listing and writing alike. A link that cannot be resolved (a dangling link) is
refused. followSymlinks = false (the default) additionally refuses a path that is itself a link when reading or
listing; it does not decide whether a link may lead out of the allowed area, which is always refused. The tools open
the resolved path; a directory swapped for a link between the check and the open is a race this narrows and does not
close. A hard link inside an allowed directory to a file elsewhere is not contained: no path check can tell it from the
file itself. On macOS /var is a link to /private/var, so the default blocklist’s /var also blocks the temporary
directories under /private/var/folders.
The default blocklist, FileConfig.DefaultBlockedPaths, depends on the platform the JVM runs on:
| Platform | System locations | Under the user’s home, besides the common entries |
|---|---|---|
| Linux and other POSIX | /etc, /var, /sys, /proc, /dev, /boot, /root, /run |
.local/share/keyrings |
| macOS | /etc, /var, /sys, /proc, /dev, /private/etc, /private/var, /Library/Keychains |
Library/Keychains |
| Windows | C:\Windows, C:\ProgramData\Microsoft\Crypto, C:\ProgramData\ssh |
_netrc, NTUSER.DAT, and under AppData: Roaming\Microsoft\Credentials, Local\Microsoft\Credentials, Roaming\Microsoft\Protect, Roaming\Microsoft\Crypto, Roaming\gcloud, Roaming\gnupg, Roaming\GitHub CLI, Roaming\postgresql |
The common entries under the home directory (user.home) are .ssh, .gnupg, .aws, .azure, .config/gcloud,
.kube, .docker, .config/gh, .netrc, .git-credentials, .pgpass, .password-store, .npmrc, .pypirc,
.m2/settings.xml, .m2/settings-security.xml, .ivy2/.credentials and .sbt/.credentials. A blocked entry is
matched without regard to case (/ETC/hosts is inside /etc), as a case-insensitive file system such as the macOS
and Windows defaults requires. A Windows installed on a drive other than C: is not covered: add its directory, as
in FileConfig(blockedPaths = FileConfig.DefaultBlockedPaths :+ "D:\\Windows"). A blocklist cannot name every
sensitive file, so it is defence in depth: set allowedPaths to the directories the agent needs.
developmentSafe(workingDirectory, fileAllowedPaths) reads only inside workingDirectory when you give one, and
anywhere outside the blocklist when you do not. It writes inside fileAllowedPaths (default /tmp) and the working
directory, and it does allow overwriting.
The default blocklist includes /var, which on macOS is where the system temporary directory lives
(/var/folders/...): a file there is refused even if you put its directory in allowedPaths.
HTTP: HttpConfig
| Setting | Default |
|---|---|
allowedMethods |
GET, HEAD |
allowedDomains |
None: any domain that is not blocked |
blockedDomains |
localhost, 127.0.0.1, 0.0.0.0, ::1, the cloud metadata hosts and 169.254.169.254 |
blockInternalIPs |
true |
followRedirects |
false |
redirectSafeHeaders |
Accept, Accept-Language, Accept-Encoding, User-Agent, Content-Type (only with a re-sent body): the only caller-set headers a redirect carries to another origin |
timeout |
30 seconds, for the whole call: connecting, every redirect hop and reading the body |
maxResponseSize |
10 MB: at most this many bytes of the body are read; the rest is never read, and truncated is true |
HttpConfig.restricted(Seq("api.example.com")) allows only those domains. HttpConfig.withWriteMethods() adds
POST, PUT, DELETE and the rest, and HttpConfig.unsafe also turns the address checks off: use that only in a
sandbox.
Shell: ShellConfig
| Setting | Default |
|---|---|
allowedCommands |
none: an empty list allows nothing |
workingDirectory |
the process’s own |
timeout |
30 seconds |
maxOutputSize |
100,000 characters |
environment |
none: extra variables, set on top of the environment the command receives |
inheritedEnvironment |
PATH, LANG, LC_ALL, TERM and SystemRoot (those that exist); development() passes the whole environment |
pathPolicy |
none: the file-like arguments of a command are not checked |
ShellConfig.readOnly() allows ls, cat, head, tail, pwd, echo, wc, date, whoami, which and
file. ShellConfig.readOnlyWithin(policy) is the same list with the working directory and every file-like argument
held to a FileConfig by its isPathAllowed (the rule above, with a .. after a link read both the POSIX and the
Windows way). ShellConfig.development() adds git, sbt, make, npm, grep, find, cp, mv, rm and more:
read the next section before you use it.
6. Safety: what each tool can do
A tool call is a request from a model, and a model can be wrong, confused or steered by text it read. Pick the bundle by what you would be willing to let it do unsupervised.
| Bundle | A model given it can |
|---|---|
coreSafe |
compute, read the clock, make UUIDs and parse the JSON it is given. Nothing leaves the process. |
withHttpSafe() |
all of the above, and fetch pages from the public internet with GET and HEAD: your server’s IP address is the one the remote site sees. |
withFilesSafe() |
all of the above, and read files: by default any the process can read outside the blocklist (/etc, /var, …), so set allowedPaths. |
developmentSafe() |
all of the above, and write files, and run any program on the allowlist, which for development() is most of a developer’s toolbox. |
What the controls do, and where they stop:
- HTTP refuses methods outside
allowedMethods, a scheme other thanhttpandhttps, any domain outsideallowedDomains,localhost, loopback, the cloud metadata addresses, and private ranges such as10.x,172.16.xand192.168.x(and their IPv6 counterparts, such as unique-localfc00::/7and IPv6 forms that carry a private IPv4 address, and every other range the IANA special-purpose registries mark as not globally reachable, such as240.0.0.0/4,192.0.0.0/24and IPv6 outside2000::/3), all before sending a request. Redirects are not followed unless you turn that on; when they are, each hop is checked again, and from the first hop that leaves the original scheme, host and port (so a downgrade fromhttpstohttpcounts), and on every hop after it, only the headers onredirectSafeHeadersgo with the request: every other header the model set, such asX-Api-KeyorX-Auth-Token, is dropped, and a credential header (Authorization,Cookie, a name that redaction treats as sensitive, plurals such asX-Api-Keysincluded, or one ending intokenorkey) is dropped even if you list it. A hop that drops the body (a301or302after aPOST,PUT,PATCHorDELETE), on either origin, sends noContent-Type, neither a caller-set header nor the tool’s owncontent_type. An IPv6 address inside a global prefix is not judged by the IPv4 address its interface identifier may embed (ISATAP-style...:5efe:a.b.c.d), so such an address is allowed whatever IPv4 address it names.timeoutbounds the whole call, so a server that answers a byte at a time cannot hold the tool past it. ATIMEOUTreleases the caller, not the request: one already sent may still be delivered and acted on (aPOSTis not rolled back), and a DNS lookup cannot be interrupted (InetAddress.getAllByNamewaits for the system resolver), so the tool’s worker thread may outlive the deadline by up to the resolver’s own timeout, typically a few seconds per attempt as/etc/resolv.confor the OS sets it (#1734). The address check resolves the host name, and the connection then resolves it again, so a domain whose DNS answer changes in between (DNS rebinding) can pass the check with a public address and connect to a private one. The request still goes out from your network, so do not give it to a model that handles untrusted text next to credentials or internal services the server can reach; where that matters, also block private ranges at the network level, for example with an egress proxy or firewall. - Files:
allowedPathsandblockedPathsjudge the real location of a path (links resolved, compared one component at a time), so a link cannot lead out of an allowed directory and a sibling that shares a name prefix is not inside it. Treat them as a containment check on the file tools, not as a sandbox: the check and the open are two steps, which this narrows and does not close, and the shell tool is a separate matter (below). Allow one directory made for the agent, and when the files matter, run the process as a user that cannot read anything else, or in a container. - Shell: the command is split into words and started directly, without a shell.
&&,;,|,>and$VARreach the program as ordinary text and are not interpreted, and only the first word is checked againstallowedCommands. That stops a command from chaining into another one. It does not limit what an allowed program does:readOnly()is an allowlist of program names, not read-only execution. Its programs only read in ordinary use, but most options are passed through unchecked:date -ssets the clock when the process is allowed to. The options that write a file, run a program or read one the command does not name are refused (file -C,-m,-Mand-f,date -fand-r,wc --files0-from, and, for an allowlist that addssort,sort -o,--output,-T,--temporary-directory,--compress-programand--files0-from, plus on Windows, wheresortmay besort.exe,sort -tand the/Oand/Tswitches) in every spelling: anywhere in a cluster of short options, with the value attached or not (-bC,-Mmagic,-roout), but not inside the value ofsort -t(sort -tosets the separator too), under any abbreviation of the long form (date --fil), with or without=value, after a--, which an option taking an argument can consume, and whatever the case of the program’s name or a Windows.exe(FILE -C,file.exe -C) (#1723).cat,headandtailcan read any file the process can read, so the file settings above do not apply to the shell unless you useShellConfig.readOnlyWithin(policy), which holds the working directory and each file-like argument to that rule, judged as the program will hand it to the OS (a..after a link is read both as POSIX applies it, at the link target’s parent, and as Windows does, at the directory holding the link, and both must be allowed). A value attached to a flag is checked the same way, since the program may open it: the value after=of a long option (grep --file=x) and every tail of a short-option cluster (grep -fx,grep -ifx), so a link out of the allowed directory, a blocked file or..cannot be passed that way (#1723). Which options take a value is not modelled, so text that happens to name such a file (grep -e..) is refused too. A value that is one plain path component (no/or\, not.or.., nothing Windows reads specially such as:, a trailing.or space, or a device name) costs one lookup, whether anything is in the working directory under that name: if nothing is, it cannot be a link, and the policy is applied to its spelling with no further lookup, which is exactly the full check’s verdict; if something is, it is resolved in full. No limit on the length of a file name is assumed. The cost of the checks is bounded per command: an argument longer than 4096 characters is refused, each distinct path is checked once, and a command whose checks would take more than 20000 file-system lookups (one per plain component, two per component of a path resolved in full) is refused as too costly to check, rather than holding the agent for minutes before the command starts (the shelltimeoutcovers only the command’s run). A hard link to a file outside passes, as it does for the file tools. A command that walks directories itself (ls -R,grep -r,find) is checked only at the path it starts from.- On Windows, a character a program may receive as a different ASCII character is refused, with or without a path
policy (#1766). A program that reads its command line in an ANSI
code page gets each argument converted with “best fit”, so
../../secret(U+FF0F) reaches it as../../secret,¥and₩as\on the Japanese and Korean code pages, and-oas-o, past checks that read the argument as one plain name or no option. An argument, the working directory and anenvironmentvalue are refused when they hold a character a common ANSI code page (874, 932, 936, 949, 950, 1250 to 1258) best-fits to an ASCII character other than a letter or digit, any best-fitted character in an argument starting with-or/, or a character the host’s own ANSI code page cannot hold. Accented letters such aséstay allowed in an operand when that code page holds them. POSIX hosts are unchanged. - On a Windows host whose ANSI code page is double-byte (932, 936, 950), an argument holding a character whose
second byte is 0x5C, the byte of
\(表and十on 932,功on 950), is refused when the JDK puts it in quotes (#1807):a 表reaches the command line as"a 表", whose bytes end5C 22, and a program that splits its command line byte by byte reads\"as a literal quote and splits the arguments after it differently. The JDK quotes an argument holding a space or whitespace character,",<,>,&,|or^;表.txtstays allowed. development()is not a sandbox:sbt,make,npm,git,findandenvcan run arbitrary programs, so a model given it can do anything the process can.- The started program receives a scrubbed environment: only
PATH,LANG,LC_ALL,TERMandSystemRoot(those that exist) plus whatever you put inenvironment, so provider API keys in your process’s environment do not reach it.development()passes the whole environment through, because build tools need it.
- Search sends the query text to the search provider.
A reasonable starting point for an agent that answers questions about some documents is customSafe with
fileConfig set to one dedicated directory, no writeConfig, no shellConfig, and httpConfig set to the domains it
needs. Add the shell only for a developer’s own machine.
7. Things to know
- Failures are values. A refused call returns a
Left. The message names the reason: for exampleAccess denied: path '...' is not allowed,HTTP method 'POST' is not allowed. Allowed: GET, HEAD,SSRF_BLOCKED: domain '...' is not allowedorCommand 'rm' is not allowed. The registry hands the message back to the model, so it can try something else. WeatherToolis a demo. It lives inorg.llm4s.toolapi.tools, returns the same fixed reading (sunny, 22.5) for every location and calls no weather service. Use it to see tool calling work, not for weather.- The shell lists are POSIX programs.
readOnly()anddevelopment()name programs such aslsandcat, so on Windows giveShellConfiga list of your own. - A call with a missing parameter is a
Left. For examplecalculatorwithoutacomes back as an error that names the parameter, and nothing runs.
8. Your own tools
The built-in tools are built with ToolBuilder, the same way you build yours, and they register in the same
ToolRegistry. Build your tool, then add it to the sequence:
1
2
// myTool: a ToolFunction you built with ToolBuilder
val registry = BuiltinTools.coreSafe.map(core => new ToolRegistry(core :+ myTool))
Build Your Own Tool walks through writing one: the Schema DSL, reading arguments, returning
errors and testing it. See Agents for how a tool is used in a run, and
Error Handling for working with the Result these methods return.
Related
- Agents: running tools in an agent loop, with guardrails
- Error Handling:
Result[A]andLLMError BuiltinToolsExample: every bundle in a runnable sample