Best JSON and Data Processing CLI Tools

If you work with APIs, config files, or data pipelines, you process structured data in the terminal constantly. jq is the standard, but it's not the only option — and there are dedicated tools for YAML, CSV, and other formats too.

Here are the data processing CLI tools worth knowing. Every tool links to its clihub listing.

---

JSON Tools

JSON is the lingua franca of APIs, config files, and data interchange. These tools let you query, filter, and transform JSON from the command line — no Python scripts or web-based formatters needed.

jq — The Standard

The sed for JSON. jq is the go-to tool for filtering, transforming, and formatting JSON data from the command line. It has its own query language that's powerful but takes time to learn.

Common examples:

# Pretty-print JSON
curl api.example.com/users | jq .

# Extract a field
echo '{"name": "ripgrep", "stars": 48000}' | jq '.name'
# → "ripgrep"

# Filter an array
cat tools.json | jq '.[] | select(.stars > 10000) | .name'

# Transform structure
cat tools.json | jq '[.[] | {name, stars}]'

Learning curve: The basic operations (.field, .[], select()) take 10 minutes to learn. Advanced features (reduce, group_by, custom functions) take longer. The official jq manual is comprehensive.

brew install jq

fx — Interactive JSON Viewer

An interactive JSON explorer. Pipe JSON into fx and navigate with arrow keys, collapse/expand nodes, and search. Supports JavaScript expressions for filtering — if you already know JS, the query syntax is more natural than jq.

curl api.example.com/users | fx

Good for exploring unfamiliar API responses before writing a jq filter.

brew install fx

gron — Make JSON Greppable

Transforms JSON into discrete path=value assignments that you can grep through. Then transforms it back. Useful when you know what value you're looking for but not where it is in a deeply nested structure.

# Convert JSON to greppable lines
echo '{"user": {"name": "alice", "role": "admin"}}' | gron
# json.user.name = "alice";
# json.user.role = "admin";

# Grep for what you need, convert back
gron data.json | grep "role" | gron --ungron

brew install gron

dasel — Multi-Format Selector

Query and modify JSON, YAML, TOML, XML, and CSV with the same syntax. One tool instead of separate ones for each format. Supports put/delete operations for editing files in place.

# Query JSON
dasel -f config.json '.database.host'

# Query YAML
dasel -f config.yaml '.database.host'

# Modify in place
dasel put -f config.json -t string -v "localhost" '.database.host'

brew install dasel

---

YAML Tools

YAML is everywhere in modern development — Kubernetes manifests, GitHub Actions workflows, Docker Compose files, CI configs. Having a command-line tool for it saves you from manual editing and error-prone find-and-replace.

yq — jq for YAML

Same concept as jq but for YAML files. Uses a similar query syntax. Essential if you work with Kubernetes manifests, CI configs, or any YAML-heavy workflow.

# Extract a field
yq '.metadata.name' deployment.yaml

# Modify in place
yq -i '.spec.replicas = 3' deployment.yaml

# Convert YAML to JSON
yq -o json deployment.yaml

brew install yq

---

CSV & Tabular Data

When your data comes as CSV or TSV, you don't always need pandas or a spreadsheet app. These tools handle filtering, sorting, and aggregation directly from the command line.

xsv — Fast CSV Toolkit

A Rust-based CSV toolkit. Search, slice, join, sort, and aggregate CSV data from the command line. Handles large files (millions of rows) without loading everything into memory.

# Show headers
xsv headers data.csv

# Select columns
xsv select name,email data.csv

# Filter rows
xsv search "admin" data.csv

# Get stats
xsv stats data.csv

brew install xsv

miller — Swiss Army Knife for Data

Processes CSV, TSV, JSON, and other formats. Combines what you'd normally need awk, sed, cut, sort, and uniq for when working with structured data. More of a data processing language than a simple tool.

# CSV to JSON
mlr --icsv --ojson cat data.csv

# Filter rows
mlr --csv filter '$age > 30' data.csv

# Aggregate
mlr --csv stats1 -a mean -f salary -g department data.csv

brew install miller

---

Comparison Table

---

Where to Start

Working with APIs? Install jq. It's the universal JSON tool. Pipe any API response into jq . for formatted output.

Exploring unfamiliar JSON? Use fx for interactive browsing, or gron to grep through nested structures.

Working with Kubernetes/YAML? Install yq. Same mental model as jq, but for YAML.

Processing CSV data? xsv for simple operations, miller for complex transformations.

Need one tool for everything? dasel handles JSON, YAML, TOML, XML, and CSV with a single syntax.

For more developer tools, see Best CLI Tools for Developers and The Developer's Terminal Setup Guide.

---

FAQ

Is jq hard to learn?

The basics are straightforward. .field extracts a field, .[] iterates arrays, select() filters. You can be productive in 10 minutes. The advanced features (recursive descent, reduce, custom functions) have a steeper curve, but most daily tasks use only the basics.

When should I use jq vs fx vs gron?

Use jq when you know the structure and want to transform data programmatically (especially in scripts). Use fx when you're exploring a new API response interactively. Use gron when you know the value but not the path — grep finds it instantly.

Can I use these in shell scripts and CI pipelines?

Yes — that's a primary use case. jq, yq, and xsv are non-interactive and pipe-friendly, making them ideal for scripts. Install them in your Docker images or CI runners. Example: curl api.example.com | jq '.version' in a deployment script to check API version before proceeding.

What about Python or JavaScript for data processing?

For one-off queries and pipeline operations, CLI tools are faster to write and run. curl api.example.com | jq '.users[].email' takes 5 seconds. Writing an equivalent Python script takes a minute, even if you know the language well. For complex transformations, multi-step pipelines, or anything that needs error handling, a real programming language is more appropriate. The CLI tools fill the gap between "too simple for a script" and "too complex for manual inspection."

---

Browse all data tools on clihub — the directory for discovering command line tools.